Most people are limited to about 5 or 6 locations on a daily basis (work, home, school, store, etc). I want to speed up address display by caching a few of these most visited locations. I've been able to get the address info using both google maps GPS and JSON and Locator.reverseGeocode. What would be the best way to cache this information and to check proximity quickly? I found this GPS distance calculation example and have it working. Is there a faster way to check for proximity?
Please see similar question first: Optimization of a distance calculation function
There are several things we can change in distance calculations to improve performance:
Measure device speed and decrease or increase period of proximity test accordingly
Trigonometric calculations takes most of performence, but it may done much faster. First make bold distance calculations using lookup table method, then if distance is less than proximity limit + uncertainty limit, use CORDIC method for more precise calculation.
Use constants for Math.PI/180.0 and 180.0/Math.PI
several links that may be helpful:
Very useful explanations of CORDIC, especially doc from Parallax for dummies
Fast transcendent / trigonometric functions for Java
Cordic.java at Trac by Thomas B. Preusser
Cordic.java at seng440 proj
Sin/Cos look-up table source at processing.org by toxi
Related
In games like StarCraft you can have up to 200 units (for player) in a map.
There are small but also big maps.
When you for example grab 50 units and tell them to go to the other side of the map some algorithm kicks in and they find path through the obsticles (river, hills, rocks and other).
My question is do you know how the game doesnt slow down because you have 50 paths to calculate. In the meantime other things happens like drones collecting minerals buildinds are made and so on. And if the map is big it should be harder and slower.
So even if the algorithm is good it will take some time for 100 units.
Do you know how this works maybe the algorithm is similar to other games.
As i said when you tell units to move you did not see any delay for calculating the path - they start to run to the destination immediately.
The question is how they make the units go through the shortest path but fast.
There is no delay in most of the games (StarCraft, WarCraft and so on)
Thank you.
I guess it just needs to subdivide the problem and memoize the results. Example: 2 units. Unit1 goes from A to C but the shortest path goes through B. Unit2 goes from B to C.
B to C only needs to be calculated once and can be reused by both.
See https://en.m.wikipedia.org/wiki/Dynamic_programming
In this wikipedia page it specifically mentions dijkstra's algorithm for path finding that works by subdividing the problem and store results to be reused.
There is also a pretty good looking alternative here http://www.gamasutra.com/blogs/TylerGlaiel/20121007/178966/Some_experiments_in_pathfinding__AI.php where it takes into account dynamic stuff like obstacles and still performs very well (video demo: https://www.youtube.com/watch?v=z4W1zSOLr_g).
Another interesting technique, does a completely different approach:
Calculate the shortest path from the goal position to every point on the map: see the full explanation here: https://www.youtube.com/watch?v=Bspb9g9nTto - although this one is inefficient for large maps
First of all 100 units is not such a large number, pathfinding is fast enough on modern computers that it is not a big resource sink. Even on older games, optimizations are made to make it even faster, and you can see that unit will sometimes get lost or stuck, which shouldn't really happen with a general algorithm like A*.
If the map does not change map, you can preprocess it to build a set of nodes representing regions of the map. For example, if the map is two islands connected by a narrow bridge, there would be three "regions" - island 1, island 2, bridge. In reality you would probably do this with some graph algorithm, not manually. For instance:
Score every tile with distance to nearest impassable tile.
Put all adjacent tiles with score above the threshold in the same region.
When done, gradually expand outwards from all regions to encompass low-score tiles as well.
Make a new graph where each region-region intersection is a node, and calculate shortest paths between them.
Then your pathfinding algorithm becomes two stage:
Find which region the unit is in.
Find which region the target is in.
If different regions, calculate shortest path to target region first using the region graph from above.
Once in the same region, calculate path normally on the tile grid.
When moving between distant locations, this should be much faster because you are now searching through a handful of nodes (on the region graph) plus a relatively small number of tiles, instead of the hundreds of tiles that comprise those regions. For example, if we have 3 islands A, B, C with bridges 1 and 2 connecting A-B and B-C respectively, then units moving from A to C don't really need to search all of B every time, they only care about shortest way from bridge 1 to bridge 2. If you have a lot of islands this can really speed things up.
Of course the problem is that regions may change due to, for instance, buildings blocking a path or units temporarily obstructing a passageway. The solution to this is up to your imagination. You could try to carefully update the region graph every time the map is altered, if the map is rarely altered in your game. Or you could just let units naively trust the region graph until they bump into an obstacle. With some games you can see particularly bad cases of the latter because a unit will continue running towards a valley even after it's been walled off, and only after hitting the wall it will turn back and go around. I think the original Starcraft had this issue when units block a narrow path. They would try to take a really long detour instead of waiting for the crowd to free up a bridge.
There's also algorithms that accomplish analogous optimizations without explicitly building the region graph, for instance JPS works roughly this way.
I have location data from a large number of users (hundreds of thousands). I store the current position and a few historical data points (minute data going back one hour).
How would I go about detecting crowds that gather around natural events like birthday parties etc.? Even smaller crowds (let's say starting from 5 people) should be detected.
The algorithm needs to work in almost real time (or at least once a minute) to detect crowds as they happen.
I have looked into many cluster analysis algorithms, but most of them seem like a bad choice. They either take too long (I have seen O(n^3) and O(2^n)) or need to know how many clusters there are beforehand.
Can someone help me? Thank you!
Let each user be it's own cluster. When she gets within distance R to another user form a new cluster and separate again when the person leaves. You have your event when:
Number of people is greater than N
They are in the same place for the timer greater than T
The party is not moving (might indicate a public transport)
It's not located in public service buildings (hospital, school etc.)
(good number of other conditions)
One minute is plenty of time to get it done even on hundreds of thousands of people. In naive implementation it would be O(n^2), but mind there is no point in comparing location of each individual, only those in close neighbourhood. In first approximation you can divide the "world" into sectors, which also makes it easy to make the task parallel - and in turn easily scale. More users? Just add a few more nodes and downscale.
One idea would be to think in terms of 'mass' and centre of gravity. First of all, do not mark something as event until the mass is not greater than e.g. 15 units. Sure, location is imprecise, but in case of events it should average around centre of the event. If your cluster grows in any direction without adding substantial mass, then most likely it isn't right. Look at methods like DBSCAN (density-based clustering), good inspiration can be also taken from physical systems, even Ising model (here you think in terms of temperature and "flipping" someone to join the crowd)ale at time of limited activity.
How to avoid "single-linkage problem" mentioned by author in comments? One idea would be to think in terms of 'mass' and centre of gravity. First of all, do not mark something as event until the mass is not greater than e.g. 15 units. Sure, location is imprecise, but in case of events it should average around centre of the event. If your cluster grows in any direction without adding substantial mass, then most likely it isn't right. Look at methods like DBSCAN (density-based clustering), good inspiration can be also taken from physical systems, even Ising model (here you think in terms of temperature and "flipping" someone to join the crowd). It is not a novel problem and I am sure there are papers that cover it (partially), e.g. Is There a Crowd? Experiences in Using Density-Based Clustering and Outlier Detection.
There is little use in doing a full clustering.
Just uses good database index.
Keep a database of the current positions.
Whenever you get a new coordinate, query the database with the desired radius, say 50 meters. A good index will do this in O(log n) for a small radius. If you get enough results, this may be an event, or someone joining an ongoing event.
I am developing a location-based application in which I need to get nearby location name of any geopoint selected by user. I'm using Google Places API which is working fine for me.
Only problem is the service returns null for geopoints in water. Is there any way that I can retrieve nearby locations for a geopoint in water or ocean?
AFAIK the API has no way to do that.
So, you've got two options, in order of the effort it takes:
When user taps water just throw a dialog saying "Please select a
point on land". Next to no effort and will slightly annoy the user.
Try to find the closest land geopoint yourself and use it to run the API request on
(instead of the original point). Below are some ideas on that.
A good approach can be based on this answer: basically you can get a KML file with land polygons. For performance reasons, you can simplify the polygons to the extent that makes sense for your zoom levels. Now if your point is in one of those polygons -- it's sea. And you can simply iterate over all polygon edges and pick the one that's closest to your point, then pick a point on it - again closest to your point - and do one little epsilon-sized step towards the outside of the polygon to get a land point you can do a geocode request on. Also, the original author suggests you can use Haversine formula to determine neares land point -- I'm not really familiar with the appliance of that one.
The downside is, you have to deal with KML, iterate over a lot of polygons and optimize them (and lose precision doing that, in addition to possible differences between marineregions.org data and Google Places data)
Another cool trick you could try is using Sobel Filter [edge detection] on the visible map fragment to determine where coastline is (although you will get some false positives there), then trace it (as in raster->vector) to get some points and edges to calculate the closest land position with, in a manner similar to the former approach. Here's a clumsy drawing of the idea
For Sobel edge detection, consider GPUImage lib -- they have the filter implemented and it's probably going to work crazy fast since the lib does all the calculations on GPU.
UPD Turns out there's also a service called Koordinates that has coastline data available, check the answer here
I have set up a system that calculates the distance between two locations using http://www.rubygeocoder.com/, however, to be able to calculate the (tax-deductible) kilometer-costs (e.g travel compensation) I would need to get distance by road, for example by using Google.
How is this best done?
You can use https://developers.google.com/maps/documentation/directions/.
For example a request to:
http://maps.googleapis.com/maps/api/directions/json?origin=Toronto&destination=Montreal&sensor=false
will get you the directions from Toronto to Montreal and along with it the total distance.
https://github.com/apneadiving/Google-Maps-for-Rails wraps the directions API so you might find that it makes your life easier. You might also find that it is overkill in this case since we're only talking about a single request.
I am trying to find the best way to solve the problem below:
Problem
I have (up to) 100,000 Lat/Lng points in Set A
I have (up to) 2000 Lat/Lng points in Set B
I need to find the nearest neighbour of points in set B to points in Set A.
Once they have been paired - I then need to calculate their distance which will be:
2000 Set A points to 2000 Set B Points.
These points are "in memory" they do not come from a database - they are the result of other calculations done the in the system.
Current Solution
Using a KDTree implementation in Ruby I can create a KDTree lookup that will match the points I have. I then use a haversine method in Ruby to calculate the distance of the points when they are paired.
KDtree code: Ruby KDTree Code
haversine Code: Haversine Code
Platform
I am running jruby - with rails as the web framework.
Issue
Its slow! Like 30 to 40 seconds slow... I think the main bottle neck is in the KDtree, but the point look up takes a long time too (i think). At smaller numbers in Set B its quick but the higher the number of points in Set B it gets a lot quicker.
The Question
Would anyone think of doing this differently? Is there something I am missing. I think a Java library might be a lot quicker, but how would I implement this, and which one would I use (Not strong in Java - I use Jruby for multithreading ruby code in the JVM)
Is it possible to persist the information to a database? Because then you can use GeoKit, which leverages a geo-aware database (MySQL, Postgres > 8.1, etc) so that you can do this:
Location.find(:all, :origin =>[37.792,-122.393], :within=>10, :order=>"distance asc")
Also, you can find the distance between two points, etc. The response time will be more on par with a DB query, and much faster than what you're seeing.
Just an idea in my mind. If you round your lat/long's to two decimal places then all the points with-in 1.11 km's will be the same. See this for more details. I'm not 100% sure about it but may be it works for you. Off-course for areas near the pols, this will not work as longitude shrinks there.
To speed up the distance calculation between two lat/long's, you can calculate euclidean distance by using simple distance formula rather than geographical distance. This distance will not be accurate off-course but will speed up your process.