I'm just a newbie, I'm loading the data to display in the view within a datatable, my big problem is that I have more than 1000 records and it takes too long to load so much data, what I would like is to load a certain amount of data and That by means of a paginacion are loading the other data but without losing the dynamic search of the datatable or sorting of the data by columns. Is there any way to do it? No matter if it is with datatable or with some other tool
Use kaminari, will_paginate, or simply .limit(n).offset(m) where n, m are numbers > 0
Related
I know how to implement pagination with UITableview but my question is we always append data of next page with existing complete data array so every next page array is increasing array size.
For example - We get 50 records in first page and we request for next page and we again get 50 records and then we will append that records in existing complete array so complete array is now having 100 records. I am requesting data with around 100 pages so my array will have 5000 records as we know holding some starting page array data is not good idea as we hardly come back for starting page after visited 100 pages .
Is there any way to optimize array size? please help me on this as i searched a lot but didn't find good answer for this.
I would be very grateful for help and sorry for my bad english.
I think you can achieve that by writing the "old" data to a local storage, and retrieve and insert back into your array.
So, imagine that you've already fetched, lets say 200 items. So when the user scrolls down, and you fetched the next page (the next 20 items), you "cut" from your array the items from 0 to 99 and write to a file. Now your array has 120 items. Then, when the user continues scrolling and again reached 220 (array.count >= 220), repeat the same logic, and so on.
Now the most interesting part. If the user scrolls back and the index of the top visible cell is <100, you read the previously written data from the file (and remove from the file) and insert into your array at 0 position.
And of course it'd be better to clear all that kind of files on the app launch.
Of course the numbers I wrote below are magic numbers and you should play with them to find the right ones that best fit your needs.
I am using Delphi 10.1 with FireDac and am using an SQLite Database. I am opening a Query from a timer.Event after the form shows because it takes a bit to load the Query with all records on startup and I want to let the user see something is happening, so showing the form first then loading data make sense to me.
I want to use a progress bar to show the progress as the data is loaded into the Query. I can get the progress using the GetRecords event of the Query and the Query.RecordCount which shows how many records have been fetched so far, but is there a way to get the total records of an SQLite table prior to calling Query.Open or within the GetRecords Event? This way I will be able to show progress as records load.
Thanks for the SQL Tip, SELECT COUNT(*) FROM TABLE Marc.
What actually fixed the slowness of the opening of the Query was not the Query at all, but the populating of the StringGrid. The StringGrid was only there for me and won't be displayed to the user. Disconnecting it from the Query made all the difference in speed.
Thanks, all.
In my app I have 800 000 data in server which I have to display to user. User can also search from those data. I really got confused what to do here now. How to achieve this functionality.? I am trying to load first 50 data to table and then at top part there is search bar from that user can search data but user can search by writing approximate word also (i.e if user wrote "bcd" then it will return all data having "bcd" combination). Can anyone suggest me something that will help me to get out of this situation.
You have to do pagination here without it you can't get that much data if you do that then your application will be crash. Fetch some data from the server like 30 or 40 and when you reach at 30 request for next 30 data. Then you can meet the application need.
You need to use pagination in your application .without pagination if you got 8 lakhs data in one shot then your application might be crash.
every time send request to server like"abc"
server get first 10 data from result and return those data.
now for second request server will return 11 to 20 records from resultant data
I am developer with SIMpalm. i would like to suggest you below answer.
why can't you take two array on for displaying in table view other contain all results ,when you search then search result in the Array which contains all results.and add them to the array which shown in the table.
You will have to use pagination, I don't see any other way you can do this without eating up lot of memory or the elegant way and worst case sporadic crash.
You can do the pagination in the browse and the search both. To avoid delay's for user you can preload data. e.g. for pages of 200 records, when user reaches to 150 you start fetching data for next page.
Also if your local/web server is taking more than min to load. you have serious problem on the server, That needs to be fixed. No user will wait for min to reload or get the new data.
I am not expert on the servers/networking but it should not take more than 10-15 secs.
Think about search logic as very similar to the browsing all data.
Search/Browse both needs paging
Browse returns all the data in pages
Search returns specific data in pages
Search/Browse proloads data after user reaches certain point
I am working jqgrid pagination .And stucked at a very basic problem but it is really irritating me.
There are two main aspects of what i am doing.
1. Server side pagination for server side data.
2. Client side pagination for server side searching
In 1st case i am fetching 50 records for every pager button(next,previous,last,first) and also if user enter page number then also correct service call is fetching perfect 50 records for me and setting data. Also as per my requirements i want jqgrid to show total records on the server at the bottom-right side of grid even if grid contains only 50 at current and accordingly total pages should get updated. this is also working properly.
Actual conflict is here .If i search with some criteria service will return me whole data for search say 300 records. Now all the 300 records are fetched in single service call. So i want client side pagination for this .I am able set 300 records and page number also but view {} to {} and page number in textbox at the center does not get updated.
Is there any way to reset the value of page textbox and view{} to {} to default value ?
Please help
It seems, that you post wrong values in total, page and records of the server response. I suppose that you switched the values of records and total. The value of total should be the total number of pages in the dataset. The value of records should be the number of rows (records, items) in the dataset. You should either adjust your server code of specify jsonReader which get (calculate) in correct way the data total, page and records from the data returned from the server.
I recommend you to read the answer to understand why jqGrid send to the server additional parameters and why the server require to return total, page and records.
I have a current implementation of will_paginate that uses the paginate_by_sql method to build the collection to be paginated. We have a custom query for total_entries that's very complicated and puts a large load on our DB. Therefore we would like to cut total_entries from the pagination altogether.
In other words, instead of the typical pagination display of 'previous 1 [2] 3 4 5 next', we would simply like a 'next - previous' button only. But we need to know a few things.
Do we display the previous link? This would only occur of course if records existing prior to the ones displayed in the current selection
Do we display the next link? This would not be displayed if the last record in the collection is being displayed
From the docs
A query for counting rows will
automatically be generated if you
don’t supply :total_entries. If you
experience problems with this
generated SQL, you might want to
perform the count manually in your
application.
So ultimately the ideal situation is the following.
Remove the total_entries count because it's causing too much load on the database
Display 50 records at a time with semi-pagination using only next/previous buttons to navigate and not needing to display all page numbers available
Only display the next button and previous button accordingly
Has anyone worked with a similar issue or have thoughts on a resolution?
There are many occasions where will_paginate does a really awful job of calculating the number of entries, especially if there are joins involved that confuse the count SQL generator.
If all you need is a simple prev/next method, then all you need to do is attempt to retrieve N+1 entries from the database, and if you only get N or less than you're on the last page.
For example:
per_page = 10
page = 2
#entries = Thing.with_some_scope.find(:all, :limit => per_page + 1, :offset => (page - 1) * per_page)
#next_page = #entries.slice!(per_page, 1)
#prev_page = page > 1
You can easily encapsulate this in some module that can be included in the various models that require it, or make a controller extension.
I've found that this works significantly better than the default will_paginate method.
The only performance issue is a limitation of MySQL that may be a problem depending on the size of your tables.
For whatever reason, the amount of time it takes to perform a query with a small LIMIT in MySQL is proportional to the OFFSET. In effect, the database engine reads through all rows leading up to the particular offset value, then returns the next LIMIT number rows, not skipping ahead as you'd expect.
For large data-sets, where you're having OFFSET values in the 100,000 plus range, you may find performance degrades significantly. How this will manifest is that loading page 1 is very fast, page 1000 is somewhat slow, but page 2000 is extremely slow.