I am building an app where I visualise a rather large dataset (~5 million polygons) evenly distributed over a geographic area.
Roughly 2000 polygons are displayed at once at the appropriate zoom level. When zoomed out, the data is simply hidden.
To speed up drawing of the polygons I've implemented an R*-tree that returns the polygons that overlap the area in question.
-(void)drawMapRect:(MKMapRect)mapRect zoomScale:(MKZoomScale)zoomScale inContext:(CGContextRef)context
MKCoordinateRegion region = MKCoordinateRegionForMapRect(mapRect);
NSArray *polygons = [[Polygons sharedPolygons] polygonsInRegion:region];
for(Polygon *p in polygons) {
// Draw polygon
}
}
The actual sorting once the polygons are loaded into memory seems solvable by fetching and storing only the polygons that the user sees into the R-tree. The user is only interested in features close by or in specific regions.
I have tried SQLite but it does not seem like the right choice in this case, considering the dataset in question quickly becomes fairly large (>1gb) and maybe SQLite isn't optimal for doing queries of features within specific regions?
What are some clever ways I can store this dataset in the bundle?
Are there any specific technologies you suggest I try out for this?
You will not be able to load the entire 1 GB dataset into memory.
You should store the data in an R-tree in the database so that you can make region queries directly when you load the data.
Related
I am working with the Uber H3 library. Using the poly fill function, I have populated an area with H3 indexes for a specific resolution. But I don’t need all the indexes. I want to identify and remove those indexes which are getting plotted on isolated areas like jungles, lakes, ponds, etc.
Any thoughts on how that can be achieved?
I thought that if I can map all the buildings in a city in their respective indexes, I can easily identify those indexes in which no buildings are mapped.
I’d maintain a Hashmap of h3 index as the key and a list of coordinates which lie in that index as the value.
In order to address this, you'll need some other dataset(s). Where to find this data depends largely on the city you're looking at, but a simple Google search for footprint data should provide some options.
Once you have footprint data, there are several options depending on the resolution of the grid that you're using and your performance requirements.
You could polyfill each footprint and keep the resulting hexagons
For coarser data, just using geoToH3 to get the hexagon for each vertex in each building polygon would be faster
If the footprints are significantly smaller than your hex size, you could probably just take a single coordinate from each building.
Once you have the hexagons for each building, you can simply do a set intersection with your polygon hexes and your building hexes to get the "good" set. But it may be easier in many cases to remove bad hexagons rather than including good ones - in this case you'd need a dataset of non-building features, e.g. water and park features, and do the reverse: polyfill the undesired features, and subtract these hexagons from your set.
I want to know which polygons intersect (contains, contained or overlapped) with current MKMapView screen from millions of polygon data in a geojson file. It can be handled by using QuadTree when I worked for millions of points data. But it seems this doesn't work for Polygons.
I'm wondering what the best strategy is? What is the suitable algorithm and data struct I can use?
I wonder if there any downsides of using satellite mode in MKMapView?
If it performing as good as the standard map type? Maybe it devours more RAM or downloads more data?
I'm asking because this would be a much better solution in my app to use only satelite view, but I'd like to know if there are any consequences in advance.
As I check it right now, I cannot see any performance decrease comparing to standard mapView type. However, I believe that my use case is pretty basic at the moment and probably some issues I cannot detect this way.
So my questions is about known issues with performance using satelite view.
EDIT
I played(zoomed, jump all over the world etc) with both satelite and standard map and it turns out that satelite consumes less memory than standard one. How come?
Based on doing map tile (256 X 256) captures for offline use, satellite and hybrid map tiles average around 90K Bytes each in rural areas while standard map tiles average about 10K bytes each in those same areas, so there is a major impact on the volume of data downloaded and therefore on the time required. Note that there is fairly wide variance in the sizes from tile to tile depending on content, though the ratio stays pretty close.
The application we are developing needs to show historic movement data consisting of hundreds of thousands of points. Performance degrades as the number of points per polyline increases, or the number of polylines increases. What is the best way of improving performance in this instance?
Perhaps somehow baking the polylines into maptiles and including these as additional layers? We are currently targeting iOS9+, using ios-v3.2.0.
I need to match a given point (lat, lon) against several polygons to decide if there is a match.
The easiest way would be to iterate over each polygon and apply the point-in-polygon check algorithm, but that is prohibitively expensive.
The next optimization that I did was to define a bounding rectangle for each polygon (upper bound, lower bound) and iteratively check the point against each bounding box (fewer comparisons as against checking all the points in the polygon).
Is there any other optimizations possible? Would a spatial index on the bound rectangle points or a geohash help?
Further optimizations:
The bounding box idea is good. Checking if a point is in a bounding box is extremely fast.
If you still need more speed, you can do more pre-calculation like this:
For each polgon create a bounding box.
Define equally sized "tiles" that cover your map.
For each tile, create a list of polygons that overlap. You can do that by first checking if the bounding box overlaps with the tile. If they do, you check if the polygon overlaps with the tile.
When searching, do this:
Determine the tile that you're in. That's a fast operation.
Now you have the list of potential polygons.
For each polygon, check if the point is in the bounding box.
if it is, check if the point is in the polygon using the more expensive algorithm that you've mentioned.
I've used this algorithm several times and it's very fast. By changing the tile size you can choose the right balance between memory footprint and performance:
Think of the extreme cases:
One huge tile that covers the entire map:
You'll get one list of all elements in your map, you'll have to check all of the bounding boxes.
Very tiny tiles (1x1 m for a map that only has a polygon per country):
You'll get a huge amount of tiles. All polygons will be split over many tiles, and each tile will only have one polygon. But, once you've figured out in which tile the point is (fast), it's almost 100% sure that there's just one polygon that needs to be checked.
You need to be somewhere in between. If you only need this once and a while, you might want to choose a low memory footprint over performance. The optimal tilesize can also depends on the homogeneity of the polygon sizes. So, there is no automatic way to calculate an optimal tile-size, and you'll just have to tweak a bit until you get it right.