In my Rails app I have the following models:
class Person < ApplicationRecord
has_many :payments
end
class Payment < ApplicationRecord
belongs_to :person
end
How can I get the payments for each person and order them by sum?
This is my controller:
class SalesController < ApplicationController
def index
#people = current_account.people.includes(:payments).where(:payments => { :date => #range }).order("payments.amount DESC")
end
end
It gives me the correct numbers but the order is wrong. I want it to start with the person having the highest sum of payments within a range.
This is the current Payments table:
How can this be done?
This should work for you:
payments = Payment.arel_table
sum_payments = Arel::Table.new('sum_payments')
payments_total = payments.join(
payments.project(
payments[:person_id],
payments[:amount].sum.as('total')
)
.where(payments[:date].between(#range))
.group( payments[:person_id])
.as('sum_payments'))
.on(sum_payments[:person_id].eq(Person.arel_table[:id]))
This will create broken SQL (selects nothing from payments which is syntactically incorrect and joins to people which does not even exist in this query) but we really only need the join e.g.
payments_total.join_sources.first.to_sql
#=> INNER JOIN (SELECT payments.person_id,
# SUM(payments.amount) AS total
# FROM payments
# WHERE
# payments.date BETWEEN ... AND ...
# GROUP BY payments.person_id) sum_payments
# ON sum_payments.id = people.id
So knowing this we can pass the join_sources to ActiveRecord::QueryMethods#joins and let rails and arel handle the rest like so
current_account
.people
.includes(:payments)
.joins(payments_total.join_sources)
.where(:payments => { :date => #range })
.order("sum_payments.total DESC")
Which should result in SQL akin to
SELECT
-- ...
FROM
people
INNER JOIN payments ON payments.person_id = people.id
INNER JOIN ( SELECT payments.person_id,
SUM(payments.amount) as total
FROM payments
WHERE
payments.date BETWEEN -- ... AND ...
GROUP BY payments.person_id) sum_payments ON
sum_payments.person_id = people.id
WHERE
payments.date BETWEEN -- ... AND ..
ORDER BY
sum_payments.total DESC
This will show all the people having made payments in a given date range (along with those payments) sorted by the sum of those payments in descending order.
This is untested as I did not bother to set up a whole rails application but it should be functional.
Related
Room has_many bookings
Booking belongs_to room
I have this code and it works fine:
available_rooms = Room.select {|room| room.bookings.where("day = ?", date).count < 3 || room.bookings.empty?}
but I wonder if it is possible to rewrite it like a lambda with left join
scope :available, lambda {|date| joins('LEFT OUTER JOIN bookings on bookings.room_id = rooms.id').......
I tried this but got back only Rooms WITH at least one Booking, so totally empty rooms WITHOUT Bookings at all were excluded:
def self.available(date)
# You can use `Arel.star` instead of `:id` on postgres.
b = Booking.arel_table[:id]
group(:id)
.left_joins(:bookings)
.where(bookings: { day: date })
.having(b.count.lt(3)) # COUNT(bookings.id) < 3
end
UPDATE
To satisfy the requirement you are actually going to need a complex join please try the following instead
class Room < ApplicationRecord
def self.available(date)
b = Booking.arel_table
join_statement = Arel::Nodes::OuterJoin.new(b,
Arel::Nodes::On.new(
arel_table[:id].eq(b[:room_id])
.and(b[:day].eq(Arel::Nodes.build_quoted(date)))
))
group(:id)
.joins(join_statement)
.having(b[:id].count.lt(3))
end
end
Here we are LEFT JOINing bookings based on the relation to room and the date requested. Now if a room is not booked for that date or it has less than 3 bookings for that date it should show up as requested.
This is the query I've got at present:
SELECT t1.discipline_id AS discipline1,
t2.discipline_id AS discipline2,
COUNT(DISTINCT t1.product_id) as product_count
FROM (SELECT "product_disciplines".* FROM "product_disciplines") t1
INNER JOIN (SELECT "product_disciplines".* FROM "product_disciplines") t2
ON t1.product_id = t2.product_id
WHERE (t1.discipline_id < t2.discipline_id)
GROUP BY t1.discipline_id, t2.discipline_id
ORDER BY "product_count" DESC
Basically, I've got a list of Products and Disciplines, and each Product may be associated with one or more Disciplines. This query lets me figure out, for each possible (distinct) pair of disciplines, how many products are associated with them. I'll use this as input to a dependency wheel in Highcharts.
The problem arises when I involve Active Model Serializers. This is my controller:
class StatsController < ApplicationController
before_action :get_relationships, only: [:relationships]
def relationships
x = #relationships
.select('t1.discipline_id AS discipline1, t2.discipline_id AS discipline2, COUNT(DISTINCT t1.product_id) as product_count')
.order(product_count: :DESC)
.group('t1.discipline_id, t2.discipline_id')
render json: x, each_serializer: RelationshipSerializer
end
private
def get_relationships
query = ProductDiscipline.all
#relationships = ProductDiscipline
.from(query, :t1)
.joins("INNER JOIN (#{query.to_sql}) t2 on t1.product_id = t2.product_id")
.where('t1.discipline_id < t2.discipline_id')
end
end
each_serializer points to this class:
class RelationshipSerializer < ApplicationSerializer
has_many :disciplines do
Discipline.where(id: [object.discipline1, object.discipline2])
end
attribute :product_count
end
When I query the database, there are ~1300 possible pairs, which translates my single query in ~1300 Discipline lookups.
Is there a way to avoid the N+1 queries problem with this structure?
I ended up splitting this in two separate API queries. RelationshipSerializer saves just the discipline IDs,
class RelationshipSerializer < ApplicationSerializer
# has_many :disciplines do
# # Discipline.where(id: [object.discipline1, object.discipline2])
# [object.discipline1, object.discipline2].to_json
# end
attributes :discipline1, :discipline2
attribute :product_count
end
Since in my app I already need the list of available disciplines, I chose to correlate them client-side.
I have a model Lap:
class Lap < ActiveRecord::Base
belongs_to :car
def self.by_carmodel(carmodel)
scoped = joins(:car_model).where(:car_models => {:name => carmodel})
scoped
end
def self.fastest_per_car
scoped = select("laps.id",:car_id, :time, :mph).group("laps.id", :car_id, :time, :mph).order("time").limit(1)
scoped
end
end
I want to only return the fastest lap for each car.
So, I need to group the Laps by the Lap.car_id and then only return the fastest lap time based on that car, which would determined by the column Lap.time
Basically I would like to stack my methods in my controller:
#corvettes = Lap.by_carmodel("Corvette").fastest_per_car
Hopefully that makes sense...
When trying to run just Lap.fastest_per_car I am limiting everything to 1 result, rather than 1 result per each Car.
Another thing I had to do was add "laps.id" as :id was showing up empty in my results as well. If i just select(:id) it was saying ambiguous
I think a decent approach to this would be to add a where clause based on an efficient SQL syntax for returning the single fastest lap.
Something like this correlated subquery ...
select ...
from laps
where id = (select id
from laps laps_inner
where laps_inner.car_id = laps.car_id
order by time asc,
created_at desc
limit 1)
It's a little complex because of the need to tie-break on created_at.
The rails scope would just be:
where("laps.id = (select id
from laps laps_inner
where laps_inner.car_id = laps.car_id
order by time asc,
created_at desc
limit 1)")
An index on car_id would be pretty essential, and if that was a composite index on (car_id, time asc) then so much the better.
You are using limit which will return you one single value. Not one value per car. To return one car value per lap you just have to join the table and group by a group of columns that will identify one lap (id is the simplest).
Also, you can have a more ActiveRecord friendly friendly with:
class Lap < ActiveRecord::Base
belongs_to :car
def self.by_carmodel(carmodel)
joins(:car_model).where(:car_models => {:name => carmodel})
end
def self.fastest_per_car
joins(:car_model)
.select("laps.*, MIN(car_models.time) AS min_time")
.group("laps.id")
.order("min_time ASC")
end
end
This is what I did and its working. If there is a better way to go about these please post your answer:
in my model:
def self.fastest_per_car
select('DISTINCT ON (car_id) *').order('car_id, time ASC').sort_by! {|ts| ts.time}
end
I have a class Agreement that has two belongs_to associations to a Member class - referenced as primary and secondary:
class Agreement < ActiveRecord::Base
belongs_to :primary, class_name: 'Member'
belongs_to :secondary, class_name: 'Member'
...
def self.by_member(member_attribute_hash)
# returns any agreements that has a primary OR secondary member that matches any of the values
# in the member_attribute_hash
...
end
end
The Member class has no knowledge of the association with the Agreement class - it does not need to:
class Member < ActiveRecord::Base
# contains surname, given_names, member_number
...
def self.by_any(member_attribute_hash)
# returns any member where the member matches on surname OR given_names OR member_number
...
end
end
What I would like to do is search for all agreements where the primary or secondary member matches a set of criteria.
From previous work (see question #14139609), I've sorted out how to build the conditional where clause for Member.by_any().
Trying to reuse that method while search for Agreements led me to try this:
class Agreement < ActiveRecord::Base
...
def self.by_member(member_attribute_hash)
Agreement.joins{primary.outer}.merge(Member.by_any(member_attribute_hash)).joins{secondary.outer}.merge(Member.by_any(member_attribute_hash))
end
end
On running this in the console, with a member_attribute_hash = {surname: 'Freud'}, the generated SQL fails to honour the alias generated for the second join to member:
SELECT "agreements".*
FROM "agreements"
LEFT OUTER JOIN "members"
ON "members"."id" = "agreements"."primary_id"
LEFT OUTER JOIN "members" "secondarys_agreements"
ON "secondarys_agreements"."id" = "agreements"."secondary_id"
WHERE "members"."surname" ILIKE 'Freud%'
AND "members"."surname" ILIKE 'Freud%'
Notice the duplicate conditions in the WHERE clause. This will return Agreements where the primary Member has a surname like 'Freud', but ignores the secondary Member condition because the alias is not flowing through the merge.
Any ideas?
After struggling to understand this, I ended up replacing the Member.by_any scope with a Squeel sifter:
class Member < ActiveRecord::Base
# contains surname, given_names, member_number
...
def self.by_any(member_attribute_hash)
# returns any member where the member matches on surname OR given_names OR member_number
squeel do
[
(surname =~ "#{member[:surname]}%" if member[:surname].present?),
(given_names =~ "#{member[:given_names]}%" if member[:given_names].present?),
(member_number == member[:member_number] if member[:member_number].present?)
].compact.reduce(:|)
# compact to remove the nils, reduce to combine the cases with |
end
end
end
The only difference (code-wise), bewteen the sifter and the scope is the replacement of the where in the scope with squeel in the sifter.
So, instead of using a merge to access the Member.by_any scope from the Agreement model, I was now able to reference the Member :by_any sifter from the Agreement model. It looked like:
class Agreement < ActiveRecord::Base
...
def self.by_member(member_attribute_hash)
Agreement.joins{primary.outer}.where{primary.sift :by_any, member_attribute_hash}.joins{secondary.outer}.where{secondary.sift :by_any, member_attribute_hash}
end
end
This fixed the aliasing issue - begin celebrating!:
SELECT "agreements".*
FROM "agreements"
LEFT OUTER JOIN "members"
ON "members"."id" = "agreements"."primary_id"
LEFT OUTER JOIN "members" "secondarys_agreements"
ON "secondarys_agreements"."id" = "agreements"."secondary_id"
WHERE "members"."surname" ILIKE 'Freud%'
AND "secondarys_agreements"."surname" ILIKE 'Freud%'
but I still wasn't getting the results I expected - celebration put on hold. What was wrong? The AND in the where clause was wrong. After a bit more digging (and a night away from the computer), a refreshed mind decided to try this:
class Agreement < ActiveRecord::Base
...
def self.by_member(member_attribute_hash)
Agreement.joins{primary.outer}.joins{secondary.outer}.where{(primary.sift :by_any, member_attribute_hash) | (secondary.sift :by_any, member_attribute_hash)}
end
end
Producing this:
SELECT "agreements".*
FROM "agreements"
LEFT OUTER JOIN "members"
ON "members"."id" = "agreements"."primary_id"
LEFT OUTER JOIN "members" "secondarys_agreements"
ON "secondarys_agreements"."id" = "agreements"."secondary_id"
WHERE ((("members"."surname" ILIKE 'Freud%')
OR ("secondarys_agreements"."surname" ILIKE 'Freud%')))
Ah sweet... restart celebration... now I get Agreements that have a primary or secondary Member that matches according to the rules defined in a single sifter.
And for all the work he's done on Squeel, a big shout-out goes to Ernie Miller.
I am trying to achieve by reducing the numbers of queries using ActiveRecord 3.0.9. I generated about 'dummy' 200K customers and 500K orders.
Here's Models:
class Customer < ActiveRecord::Base
has_many :orders
end
class Orders < ActiveRecord::Base
belongs_to :customer
has_many :products
end
class Product < ActiveRecord::Base
belongs_to :order
end
when you are using this code in the controller:
#customers = Customer.where(:active => true).paginate(page => params[:page], :per_page => 100)
# SELECT * FROM customers ...
and use this in the view (I removed HAML codes for easier to read):
#order = #customers.each do |customer|
customer.orders.each do |order| # SELECT * FROM orders ...
%td= order.products.count # SELECT COUNT(*) FROM products ...
%td= order.products.sum(:amount) # SELECT SUM(*) FROM products ...
end
end
However, the page is rendered the table with 100 rows per page. The problem is that it kinda slow to load because its firing about 3-5 queries per customer's orders. thats about 300 queries to load the page.
There's alternative way to reduce the number of queries and load the page faster?
Notes:
1) I have attempted to use the includes(:orders), but it included more than 200,000 order_ids. that's issue.
2) they are already indexed.
If you're only using COUNT and SUM(amount) then what you really need is to retrieve only that information and not the orders themselves. This is easily done with SQL:
SELECT customer_id, order_id, COUNT(id) AS order_count, SUM(amount) AS order_total FROM orders LEFT JOIN products ON orders.id=products.order_id GROUP BY orders.customer_id, products.order_id
You can wrap this in a method that returns a nice, orderly hash by re-mapping the SQL results into a structure that fits your requirements:
class Order < ActiveRecord::Base
def self.totals
query = "..." # Query from above
result = { }
self.connection.select_rows(query).each do |row|
# Build out an array for each unique customer_id in the results
customer_set = result[row[0].to_i] ||= [ ]
# Add a hash representing each order to this customer order set
customer_set << { :order_id => row[1].to_i, :count => row[2].to_i, :total => row[3].to_i } ]
end
result
end
end
This means you can fetch all order counts and totals in a single pass. If you have an index on customer_id, which is imperative in this case, then the query will usually be really fast even for large numbers of rows.
You can save the results of this method into a variable such as #order_totals and reference it when rendering your table:
- #order = #customers.each do |customer|
- #order_totals[customer.id].each do |order|
%td= order[:count]
%td= order[:total]
You can try something like this (yes, it looks ugly, but you want performance):
orders = Order.find_by_sql([<<-EOD, customer.id])
SELECT os.id, os.name, COUNT(ps.amount) AS count, SUM(ps.amount) AS amount
FROM orders os
JOIN products ps ON ps.order_id = os.id
WHERE os.customer_id = ? GROUP BY os.id, os.name
EOD
%td= orders.name
%td= orders.count
%td= orders.amount
Added: Probably it is better to create count and amount cache in Orders, but you will have to maintain it (count can be counter-cache, but I doubt there is a ready recipe for amount).
You can join the tables in with Arel (I prefer to avoid writing raw sql when possible). I believe that for your example you would do something like:
Customer.joins(:orders -> products).select("id, name, count(products.id) as count, sum(product.amount) as total_amount")
The first method--
Customer.joins(:orders -> products)
--pulls in the nested association in one statement. Then the second part--
.select("id, name, count(products.id) as count, sum(product.amount) as total_amount")
--specifies exactly what columns you want back.
Chain those and I believe you'll get a list of Customer instances only populated with what you've specified in the select method. You have to be careful though because you now have in hand read only objects that are possibly in in invalid state.
As with all the Arel methods what you get from those methods is an ActiveRecord::Relation instance. It's only when you start to access that data that it goes out and executes the SQL.
I have some basic nervousness that my syntax is incorrect but I'm confident that this can be done w/o relying on executing raw SQL.