I have seen countless questions asking about converting latitude / longitude coordinates into country, state, town, etc., but I found no question which includes a time component. I have tried many searches, but only seem to be able to get isolated data sources by year or real-time APIs, not any open source data with a time component.
As an example using United States recognition, the coordinates (42.635832, 20.946381) would show Kosovo today, but in 2005 would show Serbia, and would at some point further in the past show Yugoslavia.
The question is, what data source can be used to retrieve country from latitude, longitude, and date? The ideal granularity would be day, ideal source of authority would be US recognition. The ideal data format would be some sort of structured shape data with start and end dates for validity.
Related
I am currently using Firestore for my iOS app and I need to implement a scalable solution for my posts feed. I need to get posts within say 20 miles, order them by date, and limit the amount of posts fetched for pagination. Any and all database solutions would very much appreciated! Thank you!
As a low budget/time alternative to libraries, we have implemented storing the first few digits of lat/long coordinates as a document or collection name and then accessed data that way. The first decimal place gives resolution to around 10 miles or so (exact values for longitude change depending on what latitude you are at). So in your database you could have a collection or document named something like +33.6-112.0. This would mark a reference in Firestore to put all data within (33.8 N, 112.0 W). Be careful with how you round the exact location data before placing it in the respective document or collection.
Then you can retrieve all data at any location you want. This may not give you exactly 20 miles, but some client side sorting can handle that. Note you could make the reference go to any decimal place necessary to achieve the level of precision you are looking for to minimize data base calls (to save you money) and minimize impact on the user's cell data plan.
This is a rather simple solution with limitations, maybe for an MVP, and if not careful could pull way more data than anticipated.
Below is a chart showing the approximate physical distance between each decimal place at the equator. So for example, the distance between (33.3 N, 0 W) and (33.5 N, 0 W) would be about 14 miles.
Neither of those databases have native geospatial querying capabilities. You would have to use some sort of add-on library to help with that. Geofire and Geofirestore are popular for this.
I have a .csv file with Twitter profiles including information such as username, name, description etc. One column is geolocation. In this text the user may have a country (i.e., UK), a city or town (i.e., Cambridge), an actual address (5 Tyrian Place, WR5 TY1), a state (i.e, California, CA) or something silly (i.e., West of Hell).
Is there an API/library/automatic way of taking this information and deriving the country? For example, if the location is Cambridge the output should be UK, if the address is in the UK, the output should be UK, etc.
Google has a reverse geocoding service which you can access through their Maps API:
https://developers.google.com/maps/documentation/geocoding/start
They let you make 2500 free requests per day. One nice feature is it will give you correct latitude, longitude, state, country, etc for things like "Golden Gate Bridge" and "The Big Apple." Twitter users enter all sorts of (sarcastic) phrases for their location -- like "West of Hell," "Mars," etc -- and Google will reverse geocode that as well. Though, that may not be very useful.
As another level of checking, you can compare the user's timezone ("utc_offset"), if it is present, to the place that Google returns. It's a bit involved and requires that you compare the timezone's latitude boundaries to the latitude and longitude in Google's response.
Is there a dynamic hierarchical data source out there that I can use to identify a lat lng point into a neighborhood?
For instance, if I was in Manhattan, it would recognize that I'm in Chinatown, Manhattan, New York City in that order. And if I was in a less densely populated area it would just put me into a neighborhood that would span a larger area. It can be a bit fuzzy in this concept.
Ultimately I want to group people into their nearest neighobrhood given evenly sized neighborhood population.
I know that zip codes can roll up into a metro area, but I wonder if there is something that's more granular or more dynamic.
Google's geocoding API can give a variety of levels of detail about a location. It varies by region, country, and even at state/local levels but you should be able to get close to what you're looking for.
I'm currently using a very large geo-ip database that i've built as a mixture from many freeware sites.
The problem is - the mapping of all those database is : map: (ip) -> (latitude,long)
I'm looking for a way that will deduce the location of those latitude and long points by resolution of a city and if possible - offline.
thanks
You may want to try Google Geocoding http://code.google.com/intl/en/apis/maps/documentation/geocoding/
to do it offline, you'll need a database of long/lat coordinates, such as this: http://www.maxmind.com/app/worldcities
then to match the long/lat to the cities, you'll have to build an algorithm which narrows it down to within a margin of error.
a brute-force method might be to measure the distance by using pythagoras' theorem, but that would rapidly kill your CPU. a better way may be to start by excluding results that are 1 or more above or below your target lat/long, then do your measurements on the remaining results.
you can get city and region lat/lon information from citycsv.com if you really need your info offline. It would be easy to query the data for lat/lon and get a city or region back. However as stated google would be able to take a lot of overhead off your hands with their online geocoding tools.
you could run google's geocode in burst-mode (2.500 max per day) through a cron job and fill up your offline database over the course of ....
Looking for a way to get a list of telephone area codes for a given latitude and longitude (and if necessary a given intl. code.) Note, I'm not talking about international dialing prefixes but the area codes within them.
For example, Denver Colorado is covered by the area codes 303 and 720. It's at 39.739 -104.985 and is in NANP 1. So given 39.739,-104.985,1 I'd like to get back [303,720].
Libraries, web services, DB's, or raw data that needs to be parsed into a DB, e.g., a web page of shape points, are all fine and the more global coverage the better, but just NANP 1 would be a great help.
Note I already use MaxMind and could turn the lat-lng into a fake IP and use that as the lookup key, but MaxMind claims only U.S. area codes (whether they truly mean U.S. or actually NANP I haven't tested) and seemingly only 1 per location (e.g. just 303 for Denver.) So it's a possibility, just not a great one.
UPDATE: I found some more relevant information, but no definitive solutions so I'm listing it here rather than in an answer:
I was able to find two U.S. databases http://www.area-codes.com/area-code-database.asp and http://www.nationalnanpa.com/area_codes/index.html (50% down the page, MS Access file.) The former includes lat/lng for $450 and the latter would require nearest-neighbor matching as KeithS talks about (it's probably the same DB underlying the NANPA City Query he found.)
Additionally I found information that implies Teleatlas has area code boundary maps and that ESRI includes area code shape files with copies of ArcGIS. Maponics seems to have data available: there's a Google Maps implementation of Maponics' data at http://www.usnaviguide.com/areacode.htm.
Wow. You'll definitely need some sort of pre-existing database of points. My first thought was ZIPList5 Geocode. It includes lat-long data for each active U.S. ZIP code, so you can throw this data in a DB table, index the hell out of it, and search by just about any geographic info you'd have access to. You can buy one copy for $40, with enterprise-level use for $100. Only problem is that this DB has only the "primary" area code for each ZIP code, so metro areas that have more than one (Dallas, Chicago, NYC) aren't going to show all of them.
You could try a two-pronged approach with some free data I found: for a given latitude and longitude, do a nearest-neighbors search of the data in the USGS Geographic Names Information System; it includes information on every human habitation center, and every named landmark feature, with lat/long coordinates of their centers. You now have your lat/long point mapped to the nearest town/city, ZIP code, county, and state. Now, you can compare that against this list of U.S. Area Codes, to find area codes matching any or all of the identifying information from the USGS. This is all free, and will eventually get you what you need, but you'll probably have to do some work to "massage" the two sets of data into something you can efficiently cross-reference, and/or you'll need to implement a good "search engine" that will accurately find nearest-neighbor named points, and then find area codes for locations matching the names.
One more thing to look at is NANPA, which administers area code assignment to begin with. I'm sure they have a more comprehensive downloadable DB, but the only free public access I could find was this search page, which will find area codes for any city with >20k people. You could turn your lat/long data into a city and state, and then hit this search page: NANPA City Query
Here is an option:
http://geocoder.ca/39.739,-104.985?geoit=xml
<TimeZone>America/Denver</TimeZone>
<AreaCode>720,303</AreaCode