Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
When users register with our app, we are able to infer their zip code when we validate them against a national database. What would be the best way to determine a good potential guess of their time zone from this zip code?
We are trying to minimize the amount of data we explicitly have to ask them for. They will be able to manually set the time zone later if our best guess is wrong. I realize zip codes won't help with figuring out the time zone outside the US, but in that case we'd have to manually ask anyway, and we deal predominantly with the US regardless.
I've found a lot of zip code databases, and so far only a few contain time zone information, but those that do are not free, such as this one. If it's absolutely necessary to pay a subscription to a service in order to do this, then it will not be worth it and we will just have to ask users explicitly.
Although language isn't particularly relevant as I can probably convert things however needed, we're using PHP and MySQL.
I just found a free zip database that includes time offset and participation in DST. I do like Erik J's answer, as it would help me choose the actual time zone as opposed to just the offset (because you never can be completely sure on the rules), but I think I might start with this, and have it try to find the best time zone match based on offset/dst configuration. I think I may try to set up a simple version of Development 4.0's answer to check against what I get from the zip info as a sanity test. It's definitely not as simple as I'd hope, but a combination should get me at least 90% sure of a user's time zone.
Most states are in exactly one time zone (though there are a few exceptions). Most zip codes do not cross state boundaries (though there are a few exceptions).
You could quickly come up with your own list of time zones per zip by combining those facts.
Here's a list of zip code ranges per state, and a list of states per time zone.
You can see the boundaries of zip codes and compare to the timezone map using this link, or Google Earth, to map zips to time zones for the states that are split by a time zone line.
The majority of non-US countries you are dealing with are probably in exactly one time zone (again, there are exceptions). Depending on your needs, you may want to look at where your top-N non-US visitors come from and just lookup their time zone.
I created an Open-Source (MIT licensed) MySQL database table that cross-references zip codes and timezones and uploaded it to sourceforge.net:
http://sourceforge.net/projects/zip2timezone/files/
It is sourced from four locations (primary Yahoo PlaceFinder API - thanks #Chris N)
See the README file for more information and instructions.
If you want, you can also get a feel for the timezone by asking the browser
Josh Fraser has a nice write up on it here
var rightNow = new Date();
var jan1 = new Date(rightNow.getFullYear(), 0, 1, 0, 0, 0, 0);
var temp = jan1.toGMTString();
var jan2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var std_time_offset = (jan1 - jan2) / (1000 * 60 * 60);
The second thing that you need to know is whether the location observes daylight savings time (DST) or not. Since DST is always observed during the summer, we can compare the time offset between two dates in January, to the time offset between two dates in June. If the offsets are different, then we know that the location observes DST. If the offsets are the same, then we know that the location DOES NOT observe DST.
var june1 = new Date(rightNow.getFullYear(), 6, 1, 0, 0, 0, 0);
temp = june1.toGMTString();
var june2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var daylight_time_offset = (june1 - june2) / (1000 * 60 * 60);
var dst;
if (std_time_offset == daylight_time_offset) {
dst = "0"; // daylight savings time is NOT observed
} else {
dst = "1"; // daylight savings time is observed
}
All credit for this goes to Josh Fraser.
This might help you with customers outside the US, and it might complement
your zip approach.
Here is a SO questions that touch on getting the timezone from javascript
Google has a specific API for this:
https://developers.google.com/maps/documentation/timezone/
eg:
https://maps.googleapis.com/maps/api/timezone/json?location=40.704822,-74.0137431×tamp=0
{
dstOffset: 0,
rawOffset: -18000,
status: "OK",
timeZoneId: "America/New_York",
timeZoneName: "Eastern Standard Time"
}
They require a unix timestamp on the querystring. From the response returned it appears that timeZoneName takes into account daylight savings, based on the timestamp, while timeZoneId is a DST-independent name for the timezone.
For my use, in Python, I am just passing timestamp=0 and using the timeZoneId value to get a tz_info object from pytz, I can then use that to localize any particular datetime in my own code.
I believe for PHP similarly you can find "America/New_York" in http://pecl.php.net/package/timezonedb
Ruby gem to convert zip code to timezone: https://github.com/Katlean/TZip (forked from https://github.com/farski/TZip).
> ActiveSupport::TimeZone.find_by_zipcode('90029')
=> "Pacific Time (US & Canada)"
It's fast, small, and has no external dependencies, but keep in mind that zip codes just don't map perfectly to timezones.
This will download a yaml file that will map all timezones to an array of their zipcodes:
curl https://gist.githubusercontent.com/anonymous/01bf19b21da3424f6418/raw/0d69a384f55c6f68244ddaa07e0c2272b44cb1de/timezones_to_zipcodes.yml > timezones_to_zipcodes.yml
e.g.
"Eastern Time (US & Canada)" => ["00100", "00101", "00102", "00103", "00104", ...]
"Central Time (US & Canada)" => ["35000", "35001", "35002", "35003", "35004", ...]
etc...
If you prefer the shortened timezones, you can run this one:
curl https://gist.githubusercontent.com/anonymous/4e04970131ca82945080/raw/e85876daf39a823e54d17a79258b170d0a33dac0/timezones_to_zipcodes_short.yml > timezones_to_zipcodes.yml
e.g.
"EDT" => ["00100", "00101", "00102", "00103", "00104", ...]
"CDT" => ["35000", "35001", "35002", "35003", "35004", ...]
etc...
I've been working on this problem for my own site and feel like I've come up with a pretty good solution
1) Assign time zones to all states with only one timezone (most states)
and then either
2a) use the js solution (Javascript/PHP and timezones) for the remaining states
or
2b) use a database like the one linked above by #Doug.
This way, you can find tz cheaply (and highly accurately!) for the majority of your users and then use one of the other, more expensive methods to get it for the rest of the states.
Not super elegant, but seemed better to me than using js or a database for each and every user.
In addition to Doug Kavendek answer. One could use the following approach to get closer to tz_database.
Download [Free Zip Code Latitude and Longitude Database]
Download [A shapefile of the TZ timezones of the world]
Use any free library for shapefile querying (e.g. .NET Easy GIS .NET, LGPL).
var shapeFile = new ShapeFile(shapeFilePath);
var shapeIndex = shapeFile.GetShapeIndexContainingPoint(new PointD(long, lat), 0D);
var attrValues = shapeFile.GetAttributeFieldValues(shapeIndex);
var timeZoneId = attrValues[0];
P.S. Can't insert all the links :( So please use search.
See these links for the Olson database:
http://en.wikipedia.org/wiki/Tz_database
ftp://ftp.iana.org/tz/tz-link.html (yes, ftp) http://www.twinsun.com/tz/tz-link.htm
ftp://ftp.iana.org/tz/releases ftp://elsie.nci.nih.gov/pub/
https://iana.org/time-zones
Also note, that if you happen to be using Yahoo geocoding service you can have timezone information returned to you by setting the correct flag.
http://developer.yahoo.com/geo/placefinder/guide/requests.html#flags-parameter
There's actually a great Google API for this. It takes in a location and returns the timezone for that location. Should be simple enough to create a bash or python script to get the results for each address in a CSV file or database then save the timezone information.
https://developers.google.com/maps/documentation/timezone/start
Request Endpoint:
https://maps.googleapis.com/maps/api/timezone/json?location=38.908133,-77.047119×tamp=1458000000&key=YOUR_API_KEY
Response:
{
"dstOffset" : 3600,
"rawOffset" : -18000,
"status" : "OK",
"timeZoneId" : "America/New_York",
"timeZoneName" : "Eastern Daylight Time"
}
Related
This question already has answers here:
Getting the client's time zone (and offset) in JavaScript
(33 answers)
Get timezone abbreviation using offset value
(2 answers)
Closed 6 years ago.
Using Javascript, is there a way to get a user's timezone name (PDT, EST, etc.) based on the user's device?
Code I tried:
const timezone = jstz.determine()
const userTimezone = timezone.name()
But I would like to get back the user's timezone name in PDT, EST, etc. instead of America/New_York.
Using moment.js with the moment-timezone add-on, you can do the following. The results will be consistent regardless of browser or operating system, because the abbreviations are in the data provided by the library.
The example results are from a computer set to the US Pacific time zone:
var zone = moment.tz.guess(); // "America/Los_Angeles"
var abbr = moment.tz(zone).format("z"); // either "PST" or "PDT", depending on when run
DISCLAIMER
Time zone abbreviations are not reliable or consistent. There are many different lists of them, and there are many ambiguities, such as "CST" being for Central Standard Time, Cuba Standard Time and China Standard Time. In general, you should avoid using them, and you should never parse them.
Also, the abbreviations in moment-timezone come from the IANA TZ Database. In recent editions of this data, many "invented" abbreviations have been replaced with numerical offsets instead. For example, if your user's time zone is Asia/Vladivostok, instead of getting "VLAT" like you may have in previous versions, you will instead get "+10". This is because the abbreviation "VLAT" was originally invented by the tz database to fill the data, and is not in actual use by persons in the area. Keep in mind that abbreviations are always in English also - even in areas where other language abbreviations might be in actual use.
We're currently storing local datetime of Pacific/Hawaii in the database. Assuming that I cannot change these dates to UTC, what information do I need to add to support timezone?
My thoughts are:
First, add a timezone field to indicate which timezone the user is viewing from. (The user will select this from a dropdown)
Second, add timezone field to indicate the timezone (Pacific/Hawaii) of the current datetimes values in the database.
Third, add offset to cover DST hours
So say a user from America/Los_Angeles views the site, it would pull the datetime from the database, append the offset and apply the timezone of Hawaii before converting it to Los Angeles time. For any calculation or comparison I would convert the Hawaii time to UTC first, then convert the UTC result to Los Angeles. Am I missing anything?
Your question is very broad, and without knowing more about your application, the platform, how you use collect dates and times, what they represent, etc., I can only speak in generalities.
Storing in UTC is recommended, but that is just by convention. The main necessities are that the time zone you are storing data in does not have DST (which Hawaii hasn't since 1947), and that you do not rely on your computer's operating system or environment settings to determine what time zone to use. You can use the Hawaiian time zone if you must. Be sure you document it somewhere though! It will surely be a surprise to anyone else that comes along in the lifecycle of the application.
While it would wolk, there is absolutely no advantage to doing this. You could just as easily convert your data to UTC when you roll out these changes and use UTC going forward. (That would be the preferred approach.)
The IANA time zone ID for Hawaii is "Pacific/Honolulu". If you're on Windows/.Net, the TimeZoneInfo ID is "Hawaiian Standard Time". Either way, they must be spelled, cased, and punctuated in exactly that manner.
Make sure you understand that a Time Zone Offset and a Time Zone are two different concepts. While Hawaii may use a fixed offset of -10:00, that's not guaranteed for most time zones. Please read the timezone tag wiki for further details.
You should probably not attempt to implement your own time zone logic. There are libraries for this in almost every language. Look to see what is appropriate for your platform. (If you provide details, I can offer suggestions.)
It would be a more robust solution to store the times as UTC time as time zone is local to the specific PC that is displaying the data. In your case if you store time plus offset how can you decide which offset to store? Not a workable solution if multiple time zones are involved.
Looking at the docs there is a descrepancy between the data in a checkin object for the query API s the push API with regards to timeZone.
According to https://developer.foursquare.com/overview/realtime a sample push will contain the name of the tz eg America/New_York
However according to https://developer.foursquare.com/docs/responses/checkin (and the API explorer) a checkin object will contain the timeZone offset eg 60 for GMT+1
I havn't managed to confirm whats in the Push API myself yet as I hav to setup the SSL certs, can anyone confirm of the docs are correct and we do indeed have 2 type of tz format. I would have thought that including the timeZone rather than the offset would be better as this doesn't vary with Daylight Savings unlike the figure. Europe/London will always be a constant where as the offset will switch between 0 and 60 mins
I'm not directly familiar with FourSquare's API, so I can't confirm or deny this for you. But I can tell you there are often cases where you would use both.
It is ok to only present an offset, if the data represents a particular time. Since the checkin response provides a createdAt date/time as an integer seconds since epoch (aka a "Unix Timestamp"), then it is appropriate to provide a separate offset. (Although I find it interesting that they provide the offset as a string and not as an integer number of minutes.) The other way you might do this would be with a single DateTimeOffset values, usually presented in ISO8601 format, as in 2013-06-02T01:23:45-07:00. Either are acceptable.
But as you may be aware, an offset does not uniquely identify a time zone. In the case of a single event, it doesn't need to. But if it were a recurring event, or if there was a possibility that you might want to modify the time value, then an offset alone would not suffice. That's when you need the full zone identifier.
If you have a zone identifier such as America/New_York, then you can always find out what the correct offset for any date/time would be. But not everyone has a TZDB implementation readily available. For example, in .Net on Windows, you get Microsoft's clumsy time zone database by default, and you have to find a library (like NodaTime) if you want to use the TZDB zones.
It does seem strange that the push and pull for the same type of action (a check-in) would have different values just because they were going through different APIs. My advice (to Foursquare) would be threefold:
Be consistent about the data for the same activity, regardless of push vs pull.
Provide both the TZDB identifier, and the the UTC offset associated with the event.
Provide the event's timestamp and offset in a single value, as an ISO8601 formatted string, rather than a unix time integer.
The Foursquare documentation is correct but a bit incomplete (as of time of posting). The check-in response contains a timeZoneOffset field. The real-time push response has a timeZone field and a timeZoneOffset field—the timeZone field is still there for legacy purposes.
Thanks for pointing this out; we'll update the docs to reflect that timeZoneOffset is the preferred method at this point. As Matt mentioned, the offset method is a better way to identify a particular time from createdAt.
I live in a country where they change the time twice a year. That is: there is a period in the year when the offset from UTC is -3 hours (-180 mins) and other period where the offset is -4 hours (-240 mins)
Grafically:
|------- (offset = -3) -------|------- (offset is -4) -------|
start of year mid end of year
My question is:
the "timezone" is just the number representing the offset? that is: my country has two timezones? or the timezone includes this information?
This is important because I save every date in UTC timezone (offset = 0) in my database.
Should I, instead, be saving the dates with local timezone and saving their offset (at the moment of saving) too?
Here is an example of a problem I see by saving the dates with timezone UTC:
Lets say I have a system where people send messages.
I want to have a statistics section where I plot "messages sent v/s hour" (ie: "Messages sent by hour in a regular day")
Lets say there are just two messages in the whole database:
Message 1, sent in march 1, at UTC time 5 pm (local time 2 pm)
Message 2, sent in august 1, at UTC time 5 pm (local time 1 pm)
Then, if I create the plot on august 2, converting those UTC dates to local would give me: "2 messages where sent at 1 pm", which is erratic information!
From the timezone tag wiki here on StackOverflow:
TimeZone != Offset
A time zone can not be represented solely by an offset from UTC. Many
time zones have more than one offset due to "daylight savings time" or
"summer time" rules. The dates that offsets change are also part of
the rules for the time zone, as are any historical offset changes.
Many software programs, libraries, and web services disregard this
important detail, and erroneously call the standard or current offset
the "zone". This can lead to confusion, and misuse of the data. Please
use the correct terminology whenever possible.
There are two commonly used database, the Microsoft Windows time zone db, and the IANA/Olson time zone db. See the wiki for more detail.
Your specific questions:
the "timezone" is just the number representing the offset? that is: my country has two timezones? or the timezone includes this information?
You have one "time zone". It includes two "offsets".
Should I, instead, be saving the dates with local timezone and saving their offset (at the moment of saving) too?
If you are recording the precise moment an event occurred or will occur, then you should store the offset of that particular time with it. In .Net and SQL Server, this is represented using a DateTimeOffset. There are similar datatypes in other platforms. It only contains the offset information - not the time zone that the offset originated from. Commonly, it is serialized in ISO8601 format, such as:
2013-05-09T13:29:00-04:00
If you might need to edit that time, then you cannot just store the offset. Somewhere in your system, you also need to have the time zone identifier. Otherwise, you have no way to determine what the new offset should be after the edit is made. If you desire, you can store this with the value itself. Some platforms have objects for exactly this purpose - such as ZonedDateTime in NodaTime. Example:
2013-05-09T13:29:00-04:00 America/New_York
Even when storing the zone id, you still need to record the offset. This is to resolve ambiguity during a "fall-back" transition from a daylight offset to a standard offset.
Alternatively, you could store the time at UTC with the time zone name:
2013-05-09T17:29:00Z America/New_York
This would work just as well, but you'd have to apply the time zone before displaying the value to anyone. TIMESTAMP WITH TIME ZONE in Oracle and PostgreSQL work this way.
You can read more about this in this post, while .Net focused - the idea is applicable to other platforms as well. The example problem you gave is what I call "maintaining the perspective of the observer" - which is discussed in the same article.
that is: my country has two timezones? or the timezone includes this information?
The term "timezone" usually includes that information. For example, in Java, "TimeZone represents a time zone offset, and also figures out daylight savings" (link), and on Unix-like systems, the tz database contains DST information.
However, for a single timestamp, I think it's more common to give just a UTC offset than a complete time-zone identifier.
[…] in my database.
Naturally, you should consult your database's documentation, or at least indicate what database you're using, and what tools (e.g., what drivers, what languages) you're using to access it.
Here's an example of a very popular format for describing timezones (though not what Windows uses).
You can see that it's more than a simple offset. More along the lines of offsets and the set of rules (changing over time) for when to use which offset.
Task: Given every zipcode of locations in US, I need to get the exact utf_timeoffset and exact dst(daytime savings offset), but I can ignore zip_code belonging to AP/FPO/DPO.
I need take into account of the timezone differences inside each state, and also need care about the exception of Hawaii, Arizona and several zipcode in PR, VI when it comes to daylight savings between Mar 11 2013, 2:00 and Nov 3 2013, 2:00.
I wonder if there is a standard module existing in Pytz(the famous timezone s/w package), because I can't located any proper function with zip_code parameter. Besides, I am perfectly fine with other package than pytz, and also any other programming languages other than Python.
At the same time, with some handmade code, I have managed to get "time_zone name" and "utf_time offset" for each zip_code. Based on provision of some websites, I also managed to get a copy of dataset containing zip_code and their daylight savings flag. A.K.A the current dst policy for each zip_code in US)
So far It works fine, but I dont really trust my own code, didn't get a way to validate its correctness.
No, pytz do not have such a functionality. See also: Mapping US zip code to time zone
Do you have the option to use the raw ZIP code data? Because if you do, it might be very helpful to use a zip code database such as the kind you would find here. They have the ZIP code associated with specific time zones as well as daylight savings info for each one. Looks like the annual price is currently $139 for 12 monthly updates.
I don't work there but I know a lot about ZIP codes and addresses since I work at SmartyStreets.