Data
I have a dataframe which contains 35,000 lat/lon locations. The locations have been plotted onto an interactive leaflet map.
The Situation
I would like to publish the map online via a markdown document.
The Problem
When I export the map as an html page or in markdown the map is:
Laggy
Hard to navigate
Webpage Loads slowly
Questions
What is the maximum number of points you plot on a leaflet map without compromising the ability to navigate the map?
Would publishing the map as a shiny application help solve the loading speed, the maps lagginess and other performance issues?
If not, what other mapping programs integrate with R that yall would recommend?
Thank you for your suggestions!
There are a couple options I can think that might help. The best would probably be to make clusters (see Marker Clusters):
addMarkers(..., clusterOptions = markerClusterOptions())
This prevents all 35 000 points from rendering at once which speeds up the loading time.
addCircles() and addCircleMarkers() seem to load quicker than addMarkers() as well if they're okay for your purposes although they're still slow with 35 000 points.
You could then do:
addCircleMarkers(..., clusterOptions = markerClusterOptions())
which should load even faster.
Update
Use leaflet.glify (devtools::install_github("tim-salabim/leaflet.glify"))
[Now leafgl (devtools::install_github("r-spatial/leafgl")) - see comment below.]
See leaflet.glify [Now leafgl - see comment below.]
An approach I did use recently to plot more than 100k points and worked very well:
leaflet(options = leafletOptions(preferCanvas = TRUE))
This forces leaflet to render the map as canvas. More info here.
The map appearance keep the same, but is much faster in terms of navigation. I didn't try leaflet.glify yet, but it seems to be a good solution to plot thousand points using leaflet package.
Related
Is there a way to reorder points in geojson so that my line "sticks" to the road. Right now I tried sorting based on longitude, but "S" shaped curves puts some points out of gps sequence, but in sort order (hence, the zig-zag)
How would I go about reordering my points correctly? Currently I'm using turf for other stuff, but another library would also be fine.
Where did these points come from? If they were ordered either chronologically or antichronologically, then perhaps that order was fine to begin with. Perhaps there is additional metadata that can help order your points with ease.
If not, the only thing I can think of is to employ some sort of nearest neighbor sorting: https://en.wikipedia.org/wiki/Nearest-neighbor_chain_algorithm
This page: https://github.com/pastelsky/nnc seems to be the source of the animation seen on wikipedia and relies on javascript code, so perhaps you can make use of the underyling library used?
I am developing a location-based application in which I need to get nearby location name of any geopoint selected by user. I'm using Google Places API which is working fine for me.
Only problem is the service returns null for geopoints in water. Is there any way that I can retrieve nearby locations for a geopoint in water or ocean?
AFAIK the API has no way to do that.
So, you've got two options, in order of the effort it takes:
When user taps water just throw a dialog saying "Please select a
point on land". Next to no effort and will slightly annoy the user.
Try to find the closest land geopoint yourself and use it to run the API request on
(instead of the original point). Below are some ideas on that.
A good approach can be based on this answer: basically you can get a KML file with land polygons. For performance reasons, you can simplify the polygons to the extent that makes sense for your zoom levels. Now if your point is in one of those polygons -- it's sea. And you can simply iterate over all polygon edges and pick the one that's closest to your point, then pick a point on it - again closest to your point - and do one little epsilon-sized step towards the outside of the polygon to get a land point you can do a geocode request on. Also, the original author suggests you can use Haversine formula to determine neares land point -- I'm not really familiar with the appliance of that one.
The downside is, you have to deal with KML, iterate over a lot of polygons and optimize them (and lose precision doing that, in addition to possible differences between marineregions.org data and Google Places data)
Another cool trick you could try is using Sobel Filter [edge detection] on the visible map fragment to determine where coastline is (although you will get some false positives there), then trace it (as in raster->vector) to get some points and edges to calculate the closest land position with, in a manner similar to the former approach. Here's a clumsy drawing of the idea
For Sobel edge detection, consider GPUImage lib -- they have the filter implemented and it's probably going to work crazy fast since the lib does all the calculations on GPU.
UPD Turns out there's also a service called Koordinates that has coastline data available, check the answer here
my Application is given a list of Geocorrdinates and now I have to determine which of those Coordinates are inside a defined Area. For example the Search would definiton would be: Show me all Areas where 100 Coordinates are in an Area of 1km^2. So I have to find out which of these coordinates are together in Areas of 1km^2 and more than 100.
But that seems to be a hard Job for my understanding of geocoordinates and I hope someone can help me with that.
The Latitude coordinates are consistent and Distance between two degrees is 111km. For example the Distance between N50,985° and N50,995° is 1,11km. The Distance between 2 Longtitude Coordinates is not so easy and depends on the Latitude coordinate.
But to be honest, I really don't know how to start.
Does someone have an Idea and can help me?
Thank you
twickl
what you need is a geospatial database, I'd recommend PostgreSQL with PostGIS. It provides the function you need to calculate this kind of stuff. Also search for good tutorials about it. An example is a radius search like "give me all McDonalds in a radius of 10km where I live"
If the problem with having a database is simply that you don't want to host it (or pay for someone to host it) then I recommend Fusion Tables.
I don't know if it supports exactly the functionality you are looking for, however I suppose you could select a random point and do a count of everything within range of that point.
I think that what you are talking could be quite a cpu intensive task (depending on how accurate you want it to be). Not something I would personally try and unload onto a portable device.
I am working on an image manipulation problem. I have an overhead projector that projects onto a screen, and I have a camera that takes pictures of that. I can establish a 1:1 correspondence between a subset of projector coordinates and a subset of camera pixels by projecting dots on the screen and finding the centers of mass of the resulting regions on the camera. I thus have a map
proj_x, proj_y <--> cam_x, cam_y for scattered point pairs
My original plan was to regularize this map using the Mathscript function griddata. This would work fine in MATLAB, as follows
[pgridx, pgridy] = meshgrid(allprojxpts, allprojypts)
fitcx = griddata (proj_x, proj_y, cam_x, pgridx, pgridy);
fitcy = griddata (proj_x, proj_y, cam_y, pgridx, pgridy);
and the reverse for the camera to projector mapping
Unfortunately, this code causes Labview to run out of memory on the meshgrid step (the camera is 5 megapixels, which apparently is too much for labview to handle)
I then started looking through openCV, and found the cvRemap function. Unfortunately, this function takes as its starting point a regularized pixel-pixel map like the one I was trying to generate above. However, it made me hope that functions for creating such a map might be available in openCV. I couldn't find it in the openCV 1.0 API (I am stuck with 1.0 for legacy reasons), but I was hoping it's there or that someone has an easy trick.
So my question is one of the following
1) How can I interpolate from scattered points to a grid in openCV; (i.e., given z = f(x,y) for scattered values of x and y, how to fill an image with f(im_x, im_y) ?
2) How can I perform an image transform that maps image 1 to image 2, given that I know a scattered mapping of points in coordinate system 1 to coordinate system 2. This could be implemented either in Labview or OpenCV.
Note: I am tagging this post delaunay, because that's one method of doing a scattered interpolation, but the better tag would be "scattered interpolation"
So this ends up being a specific fix for bugs in Labview 8.5. Nevertheless, since they're poorly documented, and I've spent a day of pain on them, I figure I'll post them so someone else googling this problem will come across it.
1) Meshgrid bombs. Don't know when this was fixed, definitely a bug in 8.5. Solution: use the meshgrid-like function on the interpolation&extrapolation pallet instead. Or upgrade to LV2009 which apparently works (thanks Underflow)
2) Griddata is defective in 8.5. This is badly documented. The 8.6 upgrade notes say that a problem with griddata and the "cubic" setting, but it is fact also a problem with the DEFAULT LINEAR setting. Solutions in descending order of kludginess: 1) pass 'v4' flag, which does some kind of spline interpolation, but does not have bugs. 2) upgrade to at least version 8.6. 3) Beat the ni engineers with reeds until they document bugs properly.
3) I was able to use the openCV remap function to do the actual transformation from one image to another. I tried just using the matlab built in interp2 vi, but it choked on large arrays and gave me out of memory errors. On the other hand, it is fairly straightforward to map an IMAQ image to an IPL image, so this isn't that bad, except for the addition of the outside library.
I'm working on a project that contains Thomas Brothers Map page and grid numbers. Is there a way to programatically convert from this map page to a latitude & longitude?
An Example would be for the intersection of the US101 & I405 freeways.
ThomasBrothers: 561-3G (page-grid)
Not that I know of, but I don't have a lot of experience with Thomas bros maps. Are you talking about printed version of the maps or is there a link somewhere to an online map?
If you just need a few lat/longs, then you can look up the locations that correspond to the grid and get the lats and longs manually at many websites, including http://itouchmap.com/latlong.html
If you provide a link to a Thomas bros map that you are using, I might be able to help further.
By looking at the link above, you can determine that US 101 and I-405 has a latitude of 34.16073390017978 and a longitude of -118.46952438354492.
Your best source would be the map publisher. If they choose to help, someone there can tell you exactly what you need to know. If they won't help you, it's unlikely that they've released the information to anyone else.
If that's the case, you could do some work by hand to correlate one point from the map grid to your target coordinate system. Effectively, you could reverse engineer a mapping "datum" for each page. You'd also have to know what map projection was used to render the maps, so that you can calculate the transform from the map coordinates to the geographic coordinates as you move away from your "origin". Finally, you'll need to establish the orientation of the map, since different notions of "north" exist.
It sounds like the Thomas maps use a new grid for every page, rather than bleeding the grid continuously from page to page. If that's the case, you'll have to correlate one point on each map. For example, find a spot where a map grid intersection coincides with a notable road intersection. Then you can find the coordinates of the road intersection using a map with latitude and longitude (a topographic map, TerraServer, etc.). Doing this with two points on the same vertical grid line should help you establish the north used on the map as well.
The short answer is that each of the nine regions has a grid derived from a Lambert conformal conic projection with custom parameters, so you cannot write a conversion program without the parameters.
I've also got ThomasBros. pages that I would like to convert to lat/long for lookup against Google Maps API. They also provided something called TBXY ... not sure what this is -- perhaps some notation for GPS/lat/long?
<Area>"El Cajon"</Area>
<ThomasBrothers>"1297 5E"</ThomasBrothers>
<TBXY>"6481390:1827008"</TBXY>
Thomas Brothers Maps invested a lot when developing their GIS system to create their digital mapping system. Though the first "digitally produced" map was Sacramento County-1990, the development began back in 1986. I expect that their map projection equations are a well guarded trade-secret, which Rand McNally now owns. I'd don't know those equations, but would also like to know them.
There are 9 projections covering the 48 states. If you know the equations for Los Angeles, it is valid across California & Nevada. Oregon & Washington have their own projection. Arizona, New Mexico, Colorado, and Utah share another projection.
I do know this...
As many know, the page grid is an exact 1/2 mile square, or 2640 feet by 2640 feet. The coordinate measurement unit is 1 foot.
To determine the Thomas Brothers XY Coordinate, get one or more of the Thomas Guide CD- ROM maps, which were recently discontinued. The last ones produced for certain California counties were the 2008 edition. Last editions for Seattle, Portland, Las Vegas, and Phoenix/Tucson were the 2007 edition. Each is still available on the Rand McNally website for $20.
When you geo-code a group of addresses, you'll see an output file with the TGXY coordinates and Lat/Lon for the addresses you specified, and the page # and grid that point is in. Once that file is open, you can click on the map to add additional geo-coded points, which will also provide both the coordinates. The output file is saved in an Access database ".mdb" file.
If you know a lot about map projections or solid geometry, the set of corresponding TGXY and Lat/Lon coordiantes will provide you some good data for testing.
As you mentioned San Diego Page 1297, I'll provide its bordering coordinates.
West x=3062760
East x=3086520
North y=0985040
South-y=0966560
This is not in range of the "TBXY" you found on Google. Maybe it's the same projection, with a relocated origin.