In my app I have 800 000 data in server which I have to display to user. User can also search from those data. I really got confused what to do here now. How to achieve this functionality.? I am trying to load first 50 data to table and then at top part there is search bar from that user can search data but user can search by writing approximate word also (i.e if user wrote "bcd" then it will return all data having "bcd" combination). Can anyone suggest me something that will help me to get out of this situation.
You have to do pagination here without it you can't get that much data if you do that then your application will be crash. Fetch some data from the server like 30 or 40 and when you reach at 30 request for next 30 data. Then you can meet the application need.
You need to use pagination in your application .without pagination if you got 8 lakhs data in one shot then your application might be crash.
every time send request to server like"abc"
server get first 10 data from result and return those data.
now for second request server will return 11 to 20 records from resultant data
I am developer with SIMpalm. i would like to suggest you below answer.
why can't you take two array on for displaying in table view other contain all results ,when you search then search result in the Array which contains all results.and add them to the array which shown in the table.
You will have to use pagination, I don't see any other way you can do this without eating up lot of memory or the elegant way and worst case sporadic crash.
You can do the pagination in the browse and the search both. To avoid delay's for user you can preload data. e.g. for pages of 200 records, when user reaches to 150 you start fetching data for next page.
Also if your local/web server is taking more than min to load. you have serious problem on the server, That needs to be fixed. No user will wait for min to reload or get the new data.
I am not expert on the servers/networking but it should not take more than 10-15 secs.
Think about search logic as very similar to the browsing all data.
Search/Browse both needs paging
Browse returns all the data in pages
Search returns specific data in pages
Search/Browse proloads data after user reaches certain point
Related
I know how to implement pagination with UITableview but my question is we always append data of next page with existing complete data array so every next page array is increasing array size.
For example - We get 50 records in first page and we request for next page and we again get 50 records and then we will append that records in existing complete array so complete array is now having 100 records. I am requesting data with around 100 pages so my array will have 5000 records as we know holding some starting page array data is not good idea as we hardly come back for starting page after visited 100 pages .
Is there any way to optimize array size? please help me on this as i searched a lot but didn't find good answer for this.
I would be very grateful for help and sorry for my bad english.
I think you can achieve that by writing the "old" data to a local storage, and retrieve and insert back into your array.
So, imagine that you've already fetched, lets say 200 items. So when the user scrolls down, and you fetched the next page (the next 20 items), you "cut" from your array the items from 0 to 99 and write to a file. Now your array has 120 items. Then, when the user continues scrolling and again reached 220 (array.count >= 220), repeat the same logic, and so on.
Now the most interesting part. If the user scrolls back and the index of the top visible cell is <100, you read the previously written data from the file (and remove from the file) and insert into your array at 0 position.
And of course it'd be better to clear all that kind of files on the app launch.
Of course the numbers I wrote below are magic numbers and you should play with them to find the right ones that best fit your needs.
I'm using Firebase for my iOS application and I'm having trouble implement infinite scroll and filtering data together.
What I need to do is:
Display items with order/filter on multiple property (location, category, status . . .)
Implement infinite scroll when the user scrolled to bottom of the screen.
I tried to think about some solutions:
The first, I think that I'll query the data with the necessary conditions then limit the number of records by use queryLimitedToFirst(N), and increase N when need to load the next items. But because Firebase can only filter on one property at a time and it's also a waste to reload data. So, I was thinking about the second solution.
As approaches are suggested from Frank van Puffelen (Query based on multiple where clauses in firebase):
filter most on the server, do the rest on the client
Yes, exactly like that. I'll execute queryOrderedByKey, queryStartingAtValue, queryEndingAtValue to implement infinite scroll, pull down the remaining data and filter that on client. But there is one problem that is I would not have enough items to display for the user if execute filter on the client.
For example: each time run the query, I receive 10 items. After data filtering process on the client, I just left 5 (can be 0) items meet the conditions to display to the user.
I don't want this because user may think there is a problem
Can I please get some pointers on this? If I didn't structured the data properly, can I also get some tips there?
I'm trying to load the first 100 videos of a YouTube channel and I can only load 50 videos. I always get this error message
Max-results value is too high. Only up to 50 results can be returned per query.
and I'm using this URL.
http://gdata.youtube.com/feeds/api/users/shaytards/uploads?&max-results=100
I'm not sure if I need any kind of developer key to load 100 results. Even if you know a way to load videos 75-100 would be great but any help would be appreciated.
The Youtube data API requires you to set up your own pagination if you need to get more than 50 results. In this case, then, you would set max-results to 50 and do the query, then on your page have some sort of "next" button (or however you want to implement your pagination) that would trigger a new call (either through ajax, a new page, or whatever your own workflow is). This new call would be to the same data feed, but you'd append the parameter &start-index=51 so that you'd get 50 more results, numbered 51-100. You can continue on like this up to 1000 results.
See https://developers.google.com/youtube/2.0/reference#Paging_through_Results for more details.
In the YouTube Data API V3 it is a little different. Each search query returns a "nextPageToken" that can be used in a subsequent query for more results. Documentation: https://developers.google.com/youtube/v3/guides/implementation/pagination
I am working jqgrid pagination .And stucked at a very basic problem but it is really irritating me.
There are two main aspects of what i am doing.
1. Server side pagination for server side data.
2. Client side pagination for server side searching
In 1st case i am fetching 50 records for every pager button(next,previous,last,first) and also if user enter page number then also correct service call is fetching perfect 50 records for me and setting data. Also as per my requirements i want jqgrid to show total records on the server at the bottom-right side of grid even if grid contains only 50 at current and accordingly total pages should get updated. this is also working properly.
Actual conflict is here .If i search with some criteria service will return me whole data for search say 300 records. Now all the 300 records are fetched in single service call. So i want client side pagination for this .I am able set 300 records and page number also but view {} to {} and page number in textbox at the center does not get updated.
Is there any way to reset the value of page textbox and view{} to {} to default value ?
Please help
It seems, that you post wrong values in total, page and records of the server response. I suppose that you switched the values of records and total. The value of total should be the total number of pages in the dataset. The value of records should be the number of rows (records, items) in the dataset. You should either adjust your server code of specify jsonReader which get (calculate) in correct way the data total, page and records from the data returned from the server.
I recommend you to read the answer to understand why jqGrid send to the server additional parameters and why the server require to return total, page and records.
jqGrid is powered by remote json data in ASP .NET MVC2 application.
On page load two requests are sent to server: one to retrieve whole html page with colmodel and second invoked by jqgrid to retrieve data.
colmodel is stored in database and depends on user rights and user configuration. Creating colmodel requires number of sql server calls which take a while.
Both request require building colmodel in server. For data retrieval colmodel is required to get correct number of columns to build select statement.
Currently this colmodel is built two times for every request. Also total number of recods is required to be returned which is slow on large data (causes whole result scan in PostgreSql server).
How to speed the things up ?
How to build colmodel only once and send it and data in same request?
I agree that extension of jqGrid to support the loading of some parts of colModel per one Ajax will be very helpful. For about a year I posted the feature request for example.
What you can do now:
If I correct understand your requirements you need to show the user the subset of the columns depend on the user's permissions. One can implement the requirement in one Ajax request. What you can do is: first don't send the "hidden" data or send there as empty string. Seconds you can send the list of columns, which should be hidden, to the client. In the case you can implement the variable number of columns in jqGrid. You can send the information inside of userData part of JSON response for example. To have better performance with many hidden columns I would recommend you to call showCol or hideCol inside of beforeProcessing and hide/show the columns on the empty grid. It will speed up the performance of showCol or hideCol dramatically. If it's needed you can include additional call of clearGridData.
You have to optimize the retrieval colModel. I don't understand why it should be slow. All depends from your implementation. In any way I am sure that one can make the retrieval really quickly.
To improve the performance of request of data from the database you can consider don't fill records field of the JSON response and set total to page + 1. It enebles the "Next" button of the pager. You should set total equal to page only if you returns less rows as the rows (number of rows per page). In the most cases it will be good criteria to detect the last page. You can additionally hide some field on the pager lake the "Last" button and the sp_1_... span which shows the total number of pages. You can do this by the usage of pgtext : "Page {0}" option or the usage of pginput: false to have no pager input at all. The viewrecords should be false (its default value). After all the customization you will don't need to calculate the total number of records and in the way improve performance of the database request in case of large data.