I'm trying to run some statistics over the Stack Overflow data dump, and for that I would like to know the time zone for each user. However, all I have to go on is the completely free-form "location" string.
I'll stress that I'm only looking for an approximation of the time zone; of course, in general this is an unsolvable problem. However, many people fill out their country, state and/or city, which should give a pretty good indication. It's okay if it fails for other cases. It doesn't have to be reliable, it doesn't have to be accurate, it doesn't have to cover all bases.
I don't want to waste too much time on this, so I'm wondering if there is some code out there that can make a reasonable guess. Any language, platform, API or library goes. Any ideas?
Check this discussion for information on how to get the lat/lon from an arbitrary location string.
Once you have the lat/lon, you can use the web services at GeoNames to retrieve the time zone.
Related
There are lots of existing questions relating to this issue, but I have looked at as many of them as I could find and did not get an answer.
I'm trying to perform an offline reverse geocoding lookup on iOS based on a latitude and longitude. I'd like to be able to provide a latitude and longitude, and be provided with the country in which that point lies. I can do this with Geonames (such as: http://api.geonames.org/countryCode?lat=45.03&lng=8.2&username=demo), but I need a similar ability offline, without Internet functionality on the device.
CLLocation does not provide offline services that work reliably enough for what I'm doing, it relies on caches made while you were previously online, etc. Messy.
I've tried this: https://github.com/drodriguez/reversegeocoding but haven't had any luck, it requires some slightly complex / confusing Terminal installations using something called Thor which I've never heard of, and was throwing up a variety of errors, so I bailed on it.
I've found a few downloadable maps, but these seem to be even more complicated, and worryingly, hundreds of megabytes or even gigabytes in size – much beyond the scope of an iOS app. I only need countries, nothing smaller than that (cities, streets, locations, etc.) so I think I should be able to get a much smaller file.
So my key question is: is there some pre-existing database or tool, preferably with iOS support, that I can feed a latitude/longitude, and get a country? And, if not, what steps should I take to get such functionality working on my own?
Thanks in advance.
ReverseGeocodeCountry is a simple lightweight offline country reverse geocoder for iOS, it has a static JSON file with country polygon data that is used to reverse geocode any lat/lng:
https://github.com/krisrak/ios-offline-reverse-geocode-country
The "Countries of the World" is a .csv text file with countries, coordinates, localised country names, capitals and other information. It seems to be free to use. You just have to import it into an SQLite database.
Edit Just noticed you want reverse geocoding. The database would only be good for forward geocoding.
You can download shapefiles for all countries at http://www.gadm.org/download. If you download a .kmz, you can unpack it to a list of coordinates for the borders. You could probably take every 5th or 10th coordinate to get smaller size (with less accuracy).
Just in case I can suggest another good written offline geocoding library.
https://github.com/Alterplay/APOfflineReverseGeocoding
I've searched and found a couple of online web services that do this, but I was wondering it there's a database in the public domain which gives a list of latitudes and longitudes for the standard time zone lines? I'd like to be able to calculate what time zone a coordinate is without using a web service. I figured that time zone boundaries are mostly static and were probably decided by some committee, so there should be a CSV or GPX or KML somewhere.
I'd like to be able to check what time zone a coordinate lies in, for example:
48.856667 2.350833 is GMT+1 or CET
This is a series of files -- timezone polygons (shapefile), which you can transform into boundaries. With work on your part.
http://efele.net/maps/tz/world/
You should note that simply knowing the TZ does not guarantee that you know anything about the current correct time or date there in terms of the local calendar. Time Ex: The Isreali Knesset (parliament) decides when Daylight time starts and ends - sometimes they have chosen a date in the past. Date Ex: The Hijri Calendar changes months based on the ability of people in Mecca to actually observe the new moon from there. Cloudy days alter the date.
Askgeo.com has a Java library under commercial license. (Scroll down their page to find information about it). They charge 2 grand up front for it though, considering the amount of work they must have put in compiling loads of vector maps, etc, I can understand they want the money.
I use their free API, but you wanted something downloadable, and I think this is one of your few bets.
I tried openstreetmap and they have the data, but it would be buried in a map of Earth, not completely useful itself. Here is one on Wikipedia though: http://en.wikipedia.org/wiki/List_of_tz_database_time_zones
I recently found this data here:
https://github.com/straup/whereonearth-timezone
I want to be able to run queries locally comparing latitude and longitude of locations so I can run queries for certain addresses I've captured based on distance.
I found a free database that has this information for zip codes but I want this information for more specific addresses. I've looked at google's geolocation service and it appears it's against the TOS to store these values in my database or to use them for anything other than doing stuff with google maps. (If somebody's looked deeper into this and I'm incorrect let me know)
Am I likely to find any (free or pay) service that will let me store these lat/lon values locally? The number of addresses I need is currently pretty small but if my site becomes popular it could expand quite a bit over time to a large number. I just need to get the coordinates of each address entered once though.
This question hasn't received enough attention...
You're correct -- it can't be done with Google's service and still conform to the TOS. Cheers to you for honestly seeking to comply with the TOS.
I work at a company called SmartyStreets where we process addresses and verify addresses -- and geocode them, too. Google's terms don't allow you to store the data returned from the API, and there's pretty strict usage limits before they throttle or cut off your access.
Screen scraping presents many challenges and problems which are both technical and ethical, and I don't suppose I'll get into them here. The Microsoft library linked to by Giorgio is for .NET only.
If you're still serious about doing this, we have a service called LiveAddress which is accessible from any platform or language. It's a RESTful API which can be called using GET or POST for example, and the output is JSON which is easy to parse in pretty much every common language/platform.
Our terms allow you to store the data you collect as long as you don't re-manufacture our product or build your own database in an attempt to duplicate ours (or something of the like). For what you've described, though, it shouldn't be a problem.
Let me know if you have further questions about address geocoding; I'll be happy to help.
By the way, there's some sample code at our GitHub repo: https://github.com/smartystreets/LiveAddressSamples
http://www.zip-info.com/cgi-local/zipsrch.exe?ll=ll&zip=13206&Go=Go could use a screen scraper if you just need to get them once.
Also Microsoft provides this service. Check if this can help you http://msdn.microsoft.com/en-us/library/cc966913.aspx
I'm working on a project that returns information based on the user's location. I also want to display the user's town in text (no map) so they can change it if it's not accurate.
If things go well I hope this will be more than a small experiment, so can anyone recommend a good reverse geocoding service with the least restrictions? I notice that Google/Yahoo have a limit to the number of daily queries along with other usage terms. I basically need to take latitude and longitude and convert them to a city/town (which I presume cannot be done using the HTML5 Geolocation API).
Geocoda just launched a geocoding and spatial database service and offers up to 1K queries a month free, with paid plans starting at $49 for 25,000 queries/month. SimpleGeo just closed their Context API so you may want to look at Geocoda or other alternatives.
You're correct, the browser geolocation API only provides coordinates.
I use SimpleGeo a lot and recommend them. They offer 10K queries a day free then 0.25USD per 1K calls after that. Their Context API is what you're going to want, it pretty much does what is says on the tin. Works server-side and client-side (without requiring you to draw a map, like Google.)
GeoNames can also do this and allows up to 30K "credits" a day, different queries expend different credit amounts. The free service has highly variable performance, the paid service is more consistent. I've used them in the past, but don't much anymore because of the difficulty of automatically dealing with their data, which is more "pure" but less meaningful to most people.
I am working on integrating geolocation services into a website and the best source of data I've found so far is MaxMind's GeoIP API with GeoLite City data. Even this data seems to often be questionable though. For example, I am located in downtown Palo Alto, but it locates my IP as being in Portola Valley, which is about 7 miles away. Palo Alto has a population of 60k+, whereas Portola Valley has a population of less than 5k. I would think if you see an IP originating somewhere around there it would make more sense to assume it was coming from the highly populated city, not the tiny one. I've also had it locate Palo Alto IPs completely across the country in Kentucky, etc.
Does anyone know of any better sources of data, or any tools/technologies/efforts to improve the accuracy of geolocation efforts? Commercial solutions are fine.
Where an IP comes up at the wrong end of the country, you probably won't find a better match elsewhere because it's probably an ISP that uses one group of IPs for customers in a wide area. My favourite example is trains here in the UK where the on-board wifi is identified as being in Sweden because they use a satellite connection to an ISP in Sweden.
A commercial supplier may be able to afford to spend more time tracking down the hard cases, but in many cases there just won't be a good answer to give you. They may, however, give you a confidence factor to tell you when they're guessing. I've heard good things about Quova, though I've never used them.
Assuming that you've got the best latitude and longitude that you can get (or can afford), then you're left dealing with cases where they pick the closest city rather than a more likely larger city nearby. Unfortunately I don't have the code to hand, but I had some success using the data from geonames to pick a "sensible" city near a point. They list lat/long and population, so you can do something like
ORDER BY ( Distance / LOG( Population ) )
You'd need to experiment with that to get something with the right level of bias towards larger cities, but I had it working quite nicely taking the centre of a Google Maps view and displaying a heading like "Showing results near London..." that changed as you moved the map.
I am not sure if this will help, but here is a site that has done a pretty good job of IP mapping. Maybe you could ask them for help :) seomoz.org
A couple of sites I saw referenced recently for free GeoIP services are
WIPmania
hostip.info