I'm developing an application on Rails and MongoDB which imports CSVs with hundreds of thousands of records and draw graphs and charts on that data. I have a MongoDB collection of 800K documents, which I wanted to render fully to the front-end and then process the data on the front-end through JS. But the browser hanged while receiving the response.
I'm not sure whether this is the right approach or not. Should the data be streamed to the browser or it's not a good idea to send 800K documents to the browser? If not, I can filter the data on the backend and send the filtered data to the browser.
Looking forward to expert suggestions.
You need to use datatables (https://datatables.net/) this will filter and call 10 - 100 records to process on the browser and that is will solve your issue :)
also for the charts you need to filter the selected data to show on your view :)
Related
I have a JSON database with an API that I want to be able to search from a mobile app.
How should I go about implementing the search, especially if I want to do an incremental search that updates the results as the user types.
Should I load the data beforehand and just filter the results or should I get the data each time the search term changes?
How is it usually done?
It depends on the amount of data being returned. If the dataset is small you can do client side search but if its large you would have to do it server side and page it. Can you tell us more about what is being searched.
Customers use web queries to grab data in the tables directly from our website and place them into excel where they can automatically work on it. while trying to grab data from our website, we noticed that table markers were not shown. excel is unable to recognize the tables on the web page.
Website was developed using RoR.
can someone help us with this issue?
You may want to consider using Power Query instead of regular web queries. With the "from Web" option you can get the DOM into Power Query and access any DOM element.
I've got a REST webservice created using sinatra. I'd like to get a count of number of images processed by my app, so I can use it for analytics purposes. How would one be able to do it?
What would be a way to do this i.e like a field on the DB? or Analytics like GA? or some other way?
Any opinion is apreciated
so basically I'm trying to display the latest tweets from 5-10 different sources/twitter accounts in a new application I'm trying to make.
Similarly the way tweetro/metrotwit retrieve and display tweets in the application. But on a much smaller scale.
Any ideas on how this could be done?
Would be much appreciated.
Duncan
Simply use the Twitter API to send a Web Service Request accessing those specific tweets in a data friendly format like XML or text. Then simply bind that data to a ListView or whatever pleases you.
I'm currently working on a drilldown filter in MVC but I don't really know how to make this the fastest and most flexible as possible.
click here
Now my question is, how do you think they are doing this?
I've really no idea how to make this kind of drilldown but it seems they use some kind of hash they save for quick querying.
Maybe (pseudo)code anyone?
If you're willing to give up a little browser compatibility (it won't work on ancient and some console only browsers but then again neither will anything else), jQuery DataTables is a great way to make drilldowns.
Here is the main site, and Here is a good example of using a dropdown select to filter.
Basically all you have to do is throw all the data into a large <table> and use javascript on the client side to filter. The big benefit is there is no latency when you make a selection, unlike the site you linked.
I think it is not a good idea to put all data on the client side.
It is more reasonable to trust the data filtering to the database server (of couse it depends on your data size).
To speed up receiving the filtered data you can save it in your cache server with hash or select query as tag. Query to cache is faster than to database.
The answer after carefull looking at how they do it:
They send a normal http POST to the server with a querystring of all
choices.
The server sends back a http GET which returns an URL with the
hash.
The server caches the hash with the query so the next time the query is called it is faster.
Thanks everyone for your "usefull" responses.