I have an active record query like this one:
House.where.not(id: Person.ill.pluck(:house_id))
When I run explain on this query, I see that all the id's get loaded into the query:
Seq Scan on houses (cost=0.00..4274.07 rows=1102 width=77)
Filter: id <> ALL ('{1117,487,842,126,127,459,458,515,332,55,54,10,1697,449,448,1555,13,510,1986,8,9,7,6,5,1865,519,157,513,512,56,103,28,27,97,25,23,385,138,278,92,1,435,196,195,61,363,229,230,238,237,231,160,158,749,748,518,517,174,173,172,395,153,1170,207,206,276,199,198, ....
Is there a more performent method to achieve the same query? Without loading all the ids?
pluck runs its query immediately, because it returns an array of IDs.
You can use select to keep the result as a Relation, which will then be used in-place as a subquery:
House.where.not(id: Person.ill.select(:house_id))
Try to use SQL JOIN, but you will cannot reuse ill scope here. If for example, this scope looks like this:
scope :ill, -> { where(status: 'ill') }
then your query could be like this:
House.joins("JOIN persons ON houses.id = persons.house_id AND persons.status <> 'ill'")
Related
I have output as result in code having queries in it (only showing basic one here)
So basically I need sum of the custom line items as well as all line items
results = Order.includes(:customer, :line_items).where('completed_at IS NOT NULL')
results.each do |result|
custom_items_sum = result.line_items.where(line_item_type: 'custom').sum(:amount)
total_sum = result.line_items.sum(:amount)
end
In this code, there is n+1 query issue, I have tried adding includes but for sure it is not going to work as we have another query inside the loop, Any help will be appreciated??
If you don't want to trigger other queries in the loop you need to avoid methods which work on relations and use that ones which work on collections. Try
custom_items_sum = result.line_items.
select { |line_item| line_item.line_item_type == 'custom' }.
sum(&:amount)
This should work without n+1 queries.
Note that it's possible to write just one query and avoid this computation anyway but that's beyond the scope of your question :)
Rails was never known to be robust enough as ORM. Use plain SQL instead:
results =
Order.connection.execute <<-SQL
SELECT order.id, SUM(line_items.amount)
FROM orders
JOIN line_items
ON (line_items.order_id = orders.id)
WHERE orders.completed_at IS NOT NULL
GROUP BY orders.id
HAVING line_items.line_item_type = 'custom'
SQL
That way you’ll get all the intermediate sums in a single query, which is way faster than performing all the calculations in ruby.
Just because #AlekseiMatiushkin says write it in raw SQL let's do the same with rails
order_table = Order.arel_table
line_items_table = LineItem.arel_table
custom_items = Arel::Table.new(:custom_items)
Order.select(
order_table[Arel.star],
line_items_table[:amount].sum.as('total_sum'),
custom_items[:amount].sum.as('custom_items_sum')
).joins(
order_table.join(line_items_table).on(
line_items_table[:order_id].eq(order_table[:id])
).join(
Arel::Nodes::As.new(line_items_table,:custom_items),
Arel::Nodes::OuterJoin
).on(
custom_items[:order_id].eq(order_table[:id]).and(
custom_items[:line_item_type].eq('custom')
)
).join_sources
).where(
order_table[:completed_at].not_eq(nil)
).group(:id)
This will produce an ActiveRecord::Relation of Order objects with a virtual attributes of total_sum and custom_items_sum using the following query
SELECT
orders.*,
SUM(line_items.amount) AS total_sum,
SUM(custom_items.amount) As custom_items_sum
FROM
orders
INNER JOIN line_items ON line_items.order_id = orders.id
LEFT OUTER JOIN line_items AS custom_items ON custom_items.order_id = orders.id
AND custom_items.line_item_type = 'custom'
WHERE
orders.completed_at IS NOT NULL
GROUP BY
orders.id
This should handle the request in a single query by using 2 joins to aggregate the needed data.
Try to use the scoping block. The following code generates very clean SQL queries.
Order.includes(:line_items).where.not(completed_at: nil).scoping do
#custom_items_sum = Order.where(line_items: { line_item_type: 'custom' })
.sum(:amount)
#total_sum = Order.sum(:amount)
end
There's not that much documentation about the scoping block but it scopes your model to the ActiveRecord requests made before (here : where('completed IS NOT NULL') and with the :line_items included).
Hope this helps! :)
I am trying to query my PostgreSQL database to get the latest (by created_at) and distinct (by user_id) Activity objects, where each user has multiple activities in the database. The activity object is structured as such:
Activity(id, user_id, created_at, ...)
I first tried to get the below query to work:
Activity.order('created_at DESC').select('DISTINCT ON (activities.user_id) activities.*')
however, kept getting the below error:
ActiveRecord::StatementInvalid: PG::InvalidColumnReference: ERROR: SELECT DISTINCT ON expressions must match initial ORDER BY expressions
According to this post: PG::Error: SELECT DISTINCT, ORDER BY expressions must appear in select list, it looks like The ORDER BY clause can only be applied after the DISTINCT has been applied. This does not help me, as I want to get the distinct activities by user_id, but also want the activities to be the most recently created activities. Thus, I need the activities to be sorted before getting the distinct activities.
I have come up with a solution that works, but first grouping the activities by user id, and then ordering the activities within the groups by created_at. However, this takes two queries to do.
I was wondering if what I want is possible in just one query?
This should work, try the following
Solution 1
Activity.select('DISTINCT ON (activities.user_id) activities.*').order('created_at DESC')
Solution 2
If not work Solution 1 then this is helpful if you create a scope for this
activity model
scope :latest, -> {
select("distinct on(user_id) activities.user_id,
activities.*").
order("user_id, created_at desc")
}
Now you can call this anywhere like below
Activity.latest
Hope it helps
I have a long SQL query, which i am using in rails Model.find_by_sql, I have to add couple of conditions in that query like i wanted to fetch all the records whose parent.is_allowed is true.
That means i am fetching all the child elements and i wanted to use association which is between child and parent.And in the Child table we have the parent_id.
Thanks
You could try something like this:
Model.find_by_sql("select * from models where parent_id in (select id from parents where is_allowed='true') and ... ")
Just replace the ... with your original SQL query.
Edit
Based on your comment, I think this line is more what you need:
Model.find_by_sql("select * from models where parent_id in (select id from users where is_allowed='true')")
I have a relationship between two models, Registers and Competitions. I have a very complicated dynamic query that is being built and if the conditions are right I need to limit Registration records to only those where it's Competition parent meets a certain criteria. In order to do this without select from the Competition table I was thinking of something along the lines of...
Register.where("competition_id in ?", Competition.where("...").collect {|i| i.id})
Which produces this SQL:
SELECT "registers".* FROM "registers" WHERE (competition_id in 1,2,3,4...)
I don't think PostgreSQL liked the fact that the in parameters aren't surrounded by parenthesis. How can I compare the Register foreign key to a list of competition ids?
you can make it a bit shorter and skip the collect (this worked for me in 3.2.3).
Register.where(competition_id: Competition.where("..."))
this will result in the following sql:
SELECT "registers".* FROM "registers" WHERE "registers"."competition_id" IN (SELECT "competitions"."id" FROM "competitions" WHERE "...")
Try this instead:
competitions = Competition.where("...").collect {|i| i.id}
Register.where(:competition_id => competitions)
I've got a Rails ActiveRecord query that find all the records where the name is some token.
records = Market.where("lower(name) = ?", name.downcase );
rec = records.first;
count = records.count;
The server shows that the calls for .first and .count were BOTH hitting the database.
←[1m←[35mCACHE (0.0ms)←[0m SELECT "markets".* FROM "markets" WHERE (lower(nam
e) = 'my market') LIMIT 1
←[1m←[36mCACHE (0.0ms)←[0m ←[1mSELECT COUNT(*) FROM "markets" WHERE (lower(na
me) = 'my market')←[0m
Why is it going to the database to get the count when it can use the results already queried?
I'm concerned about future performance. Today there are 1000 records. When that table holds 8 million rows, doing two queries one for data, and one for count, it will be expensive.
How do I get the count from the collection, not the database?
RactiveRecord use lazy query to fetch data from database. If you want to simple count the records, you can only call size of the retrun array.
records = Market.where("lower(name) = ?", name.downcase ).all
records.size
So, records is an ActiveRelation. You would think it's an array of all your Market records that match your where criteria, but it's not. Each time you reference something like first or count on that relation, it performs the query retrieve what you're asking for.
To get the actual records into an array, just add .all to the relation to actually retrieve them. Like:
records = Market.where("lower(name) = ?", name.downcase).all
count = records.count
For Rails 6.0.1 and Ruby 2.6.5
You will need to store the results into an array by using the to_a.
records = Market.where("lower(name) = ?", name.downcase).to_a
This will create the SQL query and store the results in the array records.
Then, when you call either records.first or records.count it will only return the data or do the calculation, not rerun a query. This is the same for records.size and records.length.
Another Example
I was needing to do this for a blog I am developing. I was trying to run a query to find all of the tags associated with a post, and I wanted to count how many tags there were. This was causing multiple queries until I came across the to_a suffix.
So, my SQL query looks like this:
#tags = TagMap.where(post_id: #post).joins(:tag).select(:id, '"tags"."name"').to_a
This looks through my TagMap table for all records that have post_id equal to the id of the post that I am viewing. It then joins to the Tags table and pulls only the id of the TagMap record and the name of the tag from the Tags table. Then it puts them all into an array. I can then run #tags.count and it will return the number of TagMap records for that post without doing another query.
I hope that this helps anyone using Rails 6+