I am thinking about how to structure the data within my app and one of the most important lookups will be closest location from a collection of location (10,000+ locations) and I'm looking for the fastest and most efficient way to do this.
Ideas I have:
Use Core Data and store cllocations or doubles, query with a predicate to order by closest to reference location
Store them in an sqlite database and use the distance formula in sql query
Load all locations into memory into some sort of a data structure (array, linked list, hashmap, etc.) and compute the distance a different way
Of these solustions, which would be the fastest/most efficient? Or is there another solution you would recommend?
As others have mentioned, you can't sort by a calculated value with Core Data, so a query for the closest location is unfortunately out. I've used the following "boxing" approach to approximate that, which might or might not meet your needs:
Calculate a box around the target location. The offset in degrees is something you'll need to work out, but the Wikipedia article on decimal degrees can be a good starting point. Offset your target by +/- some number of degrees to get a rough rectangle.
Fetch every location within that rectangle.
Sort the results in memory to find the closest result.
If you want to make one request for the closest location, you'll probably have to work with SQLite directly. I wouldn't load all of the points into memory without a careful examination of the total memory impact of doing so, and an understanding of how much memory your app is using for other reasons at the same time.
Related
I am currently using Firestore for my iOS app and I need to implement a scalable solution for my posts feed. I need to get posts within say 20 miles, order them by date, and limit the amount of posts fetched for pagination. Any and all database solutions would very much appreciated! Thank you!
As a low budget/time alternative to libraries, we have implemented storing the first few digits of lat/long coordinates as a document or collection name and then accessed data that way. The first decimal place gives resolution to around 10 miles or so (exact values for longitude change depending on what latitude you are at). So in your database you could have a collection or document named something like +33.6-112.0. This would mark a reference in Firestore to put all data within (33.8 N, 112.0 W). Be careful with how you round the exact location data before placing it in the respective document or collection.
Then you can retrieve all data at any location you want. This may not give you exactly 20 miles, but some client side sorting can handle that. Note you could make the reference go to any decimal place necessary to achieve the level of precision you are looking for to minimize data base calls (to save you money) and minimize impact on the user's cell data plan.
This is a rather simple solution with limitations, maybe for an MVP, and if not careful could pull way more data than anticipated.
Below is a chart showing the approximate physical distance between each decimal place at the equator. So for example, the distance between (33.3 N, 0 W) and (33.5 N, 0 W) would be about 14 miles.
Neither of those databases have native geospatial querying capabilities. You would have to use some sort of add-on library to help with that. Geofire and Geofirestore are popular for this.
I'd like to visualize a number of points on a map. Unfortunately, there is no consistent address associated with each one. I've used Google Fusion tables to get a rough read on where the points are, and am relatively satisfied with the approximate locations of most points (sometimes Google figures out where they're located based on a landmark, sometimes based on an intersection provided, sometimes by street address, etc.).
My goal, then, is to create a choropleth map of a city (NYC, in this case), showing the number of points located in each neighbourhood. Is it possible to do this by somehow counting the number of points that fall within each neighbourhood?
I suspect that if fusion tables give me a passable visual, I may be able to use google's geocoding service in the same manner to figure out the number of points in each area, and use this to then build a choropleth (not a heat map — I'm after some level of interaction, like tooltips over each neighbourhood).
Is there any way to do this, or am I way, way off?
For my iPhone app, I have gained user's location in a variable userLocation and I am updating it continuously using CLLocationManager didUpdateLocations method.
In my parse database, I have thousands of stored locations.
My question is what would be the most efficient way to continuously compare/tell if any of those parse db locations are within 100 meters of user's current location?
My current approach is to go through the array of database locations each time user's location changes; calculate distance from current location and tell if distance is less than 100 meters BUT as the location changes continuously, this will be heavy on processor and battery. Any suggestions?
you can combine Anna's idea (see in the comment), by keep changing the X or N value, according to user's movement speed.
if user are walking, than you just have to perform the next query for a longer time. but if user are using bike, for example, the next query should be more faster.
I would comment, but don't have rep.. Anyway, You could breakdown the locations into a tree type structure to remove half of the locations at each iteration - depending on where your locations are you could use ordnance grid squares or if not, something a little smaller. That way you only search a subset of locations which are reasonably close..? E.g Quad Trees
Occasionally FedEx or UPS may be unavailable to my app servers, or I need to process 100s of packages for a single transaction.
In these cases an estimate is better than nothing.
Currently I cache results for the exact parameters and "rounded" parameters, eg from_zip[:2], round(weight, 10)
What techniques should I look at to do better than this?
I think that a better approach would be to use some kind or interpolation to perform a proximity search of the target price. It can be as simple as finding 2 bounding "points" and interpolating the price of the target point, probably also using a "distance" threshold to not generate too "wild" guesses.
Either way, it's very important to inform the users that the prices are estimates, and subject to change.
Rather than using distance, just use a zone lookup table. You should be able to download a zone chart for your account plus associated rates to create a simpler lookup.
I'm currently using a very large geo-ip database that i've built as a mixture from many freeware sites.
The problem is - the mapping of all those database is : map: (ip) -> (latitude,long)
I'm looking for a way that will deduce the location of those latitude and long points by resolution of a city and if possible - offline.
thanks
You may want to try Google Geocoding http://code.google.com/intl/en/apis/maps/documentation/geocoding/
to do it offline, you'll need a database of long/lat coordinates, such as this: http://www.maxmind.com/app/worldcities
then to match the long/lat to the cities, you'll have to build an algorithm which narrows it down to within a margin of error.
a brute-force method might be to measure the distance by using pythagoras' theorem, but that would rapidly kill your CPU. a better way may be to start by excluding results that are 1 or more above or below your target lat/long, then do your measurements on the remaining results.
you can get city and region lat/lon information from citycsv.com if you really need your info offline. It would be easy to query the data for lat/lon and get a city or region back. However as stated google would be able to take a lot of overhead off your hands with their online geocoding tools.
you could run google's geocode in burst-mode (2.500 max per day) through a cron job and fill up your offline database over the course of ....