Let's say I have an ActiveRecord query which has 1 million records as result and I'm using WillPaginate to (of course) paginate this result.
So, my doubt is:
All these 1 million records are stored somehow in memory or the will_paginate query gets executed again to retrieve more and more records ?
I hope I'm being clear :)
No, it won't repeatedly grab all 1 million records. The queries look something like this:
SELECT ... FROM ... LIMIT X OFFSET Y
...where X is the maximum per page, and Y is the number of records to skip from the top of the stack, as determined by X * (current_page - 1).
Will_paginate will run separate queries to populate the data and will not fetch all the data at once. You can check how will paginate works in the terminal also. Lets say you are using rails default webrick server. Then in your terminal you will see the sql queries made by will_paginate like so:
Processing by InvoicesController#index as HTML
Invoice Load (0.1ms) SELECT "invoices".* FROM "invoices" ORDER BY id DESC LIMIT 5 OFFSET 0
Related
I want to be able to limit the activerecord objects to 20 being returned, then perform a where() that returns a subset of the limited objects which I currently know only 10 will fulfil the second columns criteria.
e.g. of ideal behaviour:
o = Object.limit(20)
o.where(column: criteria).count
=> 10
But instead, activerecord still looks for 20 objects that fulfil the where() criteria, but looks outside of the original 20 objects that the limit() would have returned on its own.
How can I get the desired response?
One way to decrease the search space is to use a nested query. You should search the first N records rather than all records which match a specific condition. In SQL this would be done like this:
select * from (select * from table order by ORDERING_FIELD limit 20) where column = value;
The query above will only search for the condition in 20 rows from the table. Notice how I have added a ORDERING_FIELD, this is required because each query could give you a different order each time you run it.
To do something similar in Rails, you could try the following:
Object.where(id: Object.order(:id).limit(20).select(:id)).where(column: criteria)
This will execute a query similar to the following:
SELECT [objects].* FROM [objects] WHERE [objects].[id] IN (SELECT TOP (20) [objects].[id] FROM [objects] ORDER BY [objects].id ASC) AND [objects].[column] = criteria
I know I can find the first user in my database in the command line with,
User.first
And I can find the last with
User.last
my question is how would I call the 11th user in a database.
You can use offset with order:
User.offset(10).order(:id).first
You can do:
User.limit(1).offset(10)
That reduces the work to a SQL statement that looks like this:
SELECT `users`.* FROM `users` LIMIT 1 OFFSET 10
Using all will require loading all the users into memory and then finding the 11th one in that array. Quite pricey.
You can do
User.all[10]
User.all gives you an array or objects with index starting from 0. To access 11th user you can do that.
I am using sqlite3 database in my project. for that I can retrive the data from the Database using following query "select * from tablename"..
But I want to take the hundred sequence records from the database, like If I scroll the UITableView based on the I want to take 100 100 records.
I have tried the following things,
SELECT * FROM mytable ORDER BY record_date DESC LIMIT 100; - It retrives only 100 records.When I scroll the table i want to fetch the next 100 records and show it.
Is it possible to do it
Please Guide me.
You could simply use the OFFSET clause, but this would still force the database to compute all the records that you're skipping over, so it would become inefficient for a larger table.
What you should do is to save the last record_date value of the previous page, and continue with the following ones:
SELECT *
FROM MyTable
WHERE record_date < ?
ORDER BY record_date DESC
LIMIT 100
See https://www.sqlite.org/cvstrac/wiki?p=ScrollingCursor for details.
I have a table with paginated data and this is the way I select data for each page:
#visitors = EventsVisitor
.select('visitors.*, events_visitors.checked_in, events_visitors.checkin_date, events_visitors.source, events_visitors.id AS ticket_id')
.joins(:visitor)
.order(order)
.where(:event_id => params[:event_id])
.where(filter_search)
.where(mode)
.limit(limit)
.offset(offset)
Also to build table pagination I need to know total count of records. Currently my solution for this is very rough:
total = EventsVisitor
.select('count(*) as count, events_visitors.*')
.joins(:visitor)
.order(order)
.where(:event_id => params[:event_id])
.where(filter_search)
.where(mode)
.first()
.count
So my question is as follows - What is the optimal ruby way to select limited data for the current page and total count of records?
I noticed that if I do #visitors.count - additional sql query will be generated:
SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM `events_visitors` INNER JOIN `visitors` ON `visitors`.`id` = `events_visitors`.`visitor_id` WHERE `events_visitors`.`event_id` = 1 LIMIT 15 OFFSET 0) subquery_for_count
First of all, I do not understand what is the reason to send an additional query to get a count of data that we already have, I mean that after we got data from database in #visitors we can count it with ruby without need to send additional queries to DB.
Second - I thought that maybe there are some ways to use something like .total_count that will generate similar count(*) query but without that useless limit/offset?
you should except limit and offset
http://guides.rubyonrails.org/active_record_querying.html#except .
See how kaminari does it
https://github.com/kaminari/kaminari/blob/92052eedf047d65df71cc0021a9df9df1e2fc36e/lib/kaminari/models/active_record_relation_methods.rb#L11
So it might be something like
total = #visitors.except(:offset, :limit, :order).count
I've got a Rails ActiveRecord query that find all the records where the name is some token.
records = Market.where("lower(name) = ?", name.downcase );
rec = records.first;
count = records.count;
The server shows that the calls for .first and .count were BOTH hitting the database.
←[1m←[35mCACHE (0.0ms)←[0m SELECT "markets".* FROM "markets" WHERE (lower(nam
e) = 'my market') LIMIT 1
←[1m←[36mCACHE (0.0ms)←[0m ←[1mSELECT COUNT(*) FROM "markets" WHERE (lower(na
me) = 'my market')←[0m
Why is it going to the database to get the count when it can use the results already queried?
I'm concerned about future performance. Today there are 1000 records. When that table holds 8 million rows, doing two queries one for data, and one for count, it will be expensive.
How do I get the count from the collection, not the database?
RactiveRecord use lazy query to fetch data from database. If you want to simple count the records, you can only call size of the retrun array.
records = Market.where("lower(name) = ?", name.downcase ).all
records.size
So, records is an ActiveRelation. You would think it's an array of all your Market records that match your where criteria, but it's not. Each time you reference something like first or count on that relation, it performs the query retrieve what you're asking for.
To get the actual records into an array, just add .all to the relation to actually retrieve them. Like:
records = Market.where("lower(name) = ?", name.downcase).all
count = records.count
For Rails 6.0.1 and Ruby 2.6.5
You will need to store the results into an array by using the to_a.
records = Market.where("lower(name) = ?", name.downcase).to_a
This will create the SQL query and store the results in the array records.
Then, when you call either records.first or records.count it will only return the data or do the calculation, not rerun a query. This is the same for records.size and records.length.
Another Example
I was needing to do this for a blog I am developing. I was trying to run a query to find all of the tags associated with a post, and I wanted to count how many tags there were. This was causing multiple queries until I came across the to_a suffix.
So, my SQL query looks like this:
#tags = TagMap.where(post_id: #post).joins(:tag).select(:id, '"tags"."name"').to_a
This looks through my TagMap table for all records that have post_id equal to the id of the post that I am viewing. It then joins to the Tags table and pulls only the id of the TagMap record and the name of the tag from the Tags table. Then it puts them all into an array. I can then run #tags.count and it will return the number of TagMap records for that post without doing another query.
I hope that this helps anyone using Rails 6+