I have users, problems, and attempts which is a join table between users and problems. I'm looking to show an index of all the problems along with the current user's most recent attempt for each, if they have one.
I've tried four things to get a left join with conditions and none of them have worked.
The naive approach is something like...
#problems = Problem.enabled
#problems.each do { |prob|
prob.last_attempt = prob.attempts
.where(user_id: current_user.id)
.last
end
This gets all the problems and the attempts I want but is N+1 queries. So...
#problems = Problem.enabled
.includes(:attempts)
This does the left join (or the equivalent two queries) getting all the problems but also all the attempts, not just those for the current user. So...
#problems = Problem.enabled
.includes(:attempts)
.where(attempts: {user_id: current_user.id})
This gets only those problems that the current user has already attempted.
So...
//problem.rb
has_many :user_attempts,
-> (user) { where(user_id: user.id) },
class_name: 'Attempt'
//problem_controller.index
#problems = Problem.enabled
.includes(:user_attempts, current_user)
And this gives an error message from rails saying joins with instance
arguments are not supported.
So I'm stuck. What is the best way to do this? Is Arel the right tool? Can I skip active record and just get back a JSON blob? Am I just being dumb?
This question is quite similar to this one but I'd need a argument to the joined scope which isn't supported. And I'm hoping rails added something in last couple years.
Thanks so much for your help.
The way I solved this was to use raw sql. It's ugly and a security risk but I didn't find better.
results = Problem.connection.exec_query(%(
SELECT *
FROM problems
LEFT JOIN (
SELECT *
//etc.
)
))
And then manipulating the results array in memory.
Related
Assuming this simplified schema:
users has_many discount_codes
discount_codes has_many orders
I want to grab all users, and if they happen to have any orders, only include the orders that were created between two dates. But if they don't have orders, or have orders only outside of those two dates, still return the users and do not exclude any users ever.
What I'm doing now:
users = User.all.includes(discount_codes: :orders)
users = users.where("orders.created_at BETWEEN ? AND ?", date1, date2).
or(users.where(orders: { id: nil })
I believe my OR clause allows me to retain users who do not have any orders whatsoever, but what happens is if I have a user who only has orders outside of date1 and date2, then my query will exclude that user.
For what it's worth, I want to use this orders where clause here specifically so I can avoid n + 1 issues later in determining orders per user.
Thanks in advance!
It doesn't make sense to try and control the orders that are loaded as part of the where clause for users. If you were to control that it'd have to be part of the includes (which I think means it'd have to be a part of the association).
Although technically it can combine them into a single query in some cases, activerecord is going to do this as two queries.
The first query will be executed when you go to iterate over the users and will use that where clause to limit the users found.
It will then run a second query behind the scenes based on that includes statement. This will simply be a query to get all orders which are associated with the users that were found by the previous query. As such the only way to control the orders that are found through the user's where clause is to omit users from the result set.
If I were you I would create an instance method in User model for what you are looking for but instead of using where use a select block:
def orders_in_timespan(start, end)
orders.select{ |o| o.between?(start, end) }
end
Because of the way ActiveRecord will cache the found orders from the includes against the instance then if you start off with an includes in your users query then I believe this will not result in n queries.
Something like:
render json: User.includes(:orders), methods: :orders_in_timespan
Of course, the easiest way to confirm the number of queries is to look at the logs. I believe this approach should have two queries regardless of the number of users being rendered (as likely does your code in the question).
Also, I'm not sure how familiar you are with sql but you can call .to_sql on the end of things such as your users variable in order to see the sql that would be generated which might help shed some light on the discrepancies between what you're getting and what you're looking for.
Option 1: Write a custom query in SQL (ugly).
Option 2: Create 2 separate queries like below...
#users = User.limit(10)
#orders = Order.joins(:discount_code)
.where(created_at: [10.days.ago..1.day.ago], discount_codes: {user_id: users.select(:id)})
.group_by{|order| order.discount_code.user_id}
Now you can use it like this ...
#users.each do |user|
orders = #orders[user.id]
puts user.name
puts user.id
puts orders.count
end
I hope this will solve your problem.
You need to use joins instead of includes. Rails joins use inner joins and will reject all the records which don't have associations.
User.joins(discount_codes: :orders).where(orders: {created_at: [10.days.ago..1.day.ago]}).distinct
This will give you all distinct users who placed orders in a given period of time.
user = User.joins(:discount_codes).joins(:orders).where("orders.created_at BETWEEN ? AND ?", date1, date2) +
User.left_joins(:discount_codes).left_joins(:orders).group("users.id").having("count(orders.id) = 0")
I'm using Rails 5. I have the following model ...
class Order < ApplicationRecord
...
has_many :line_items, :dependent => :destroy
The LineItem model has an attribute, "discount_applied." I would like to return all orders where there are zero instances of a line item having the "discount_applied" field being not nil. How do I write such a finder method?
First of all, this really depends on whether or not you want to use a pure Arel approach or if using SQL is fine. The former is IMO only advisable if you intend to build a library but unnecessary if you're building an app where, in reality, it's highly unlikely that you're changing your DBMS along the way (and if you do, changing a handful of manual queries will probably be the least of your troubles).
Assuming using SQL is fine, the simplest solution that should work across pretty much all databases is this:
Order.where("(SELECT COUNT(*) FROM line_items WHERE line_items.order_id = orders.id AND line_items.discount_applied IS NULL) = 0")
This should also work pretty much everywhere (and has a bit more Arel and less manual SQL):
Order.left_joins(:line_items).where(line_items: { discount_applied: nil }).group("orders.id").having("COUNT(line_items.id) = 0")
Depending on your specific DBMS (more specifically: its respective query optimizer), one or the other might be more performant.
Hope that helps.
Not efficient but I thought it may solve your problem:
orders = Order.includes(:line_items).select do |order|
order.line_items.all? { |line_item| line_item.discount_applied.nil? }
end
Update:
Instead of finding orders which all it's line items have no discount, we can exclude all the orders which have line items with a discount applied from the output result. This can be done with subquery inside where clause:
# Find all ids of orders which have line items with a discount applied:
excluded_ids = LineItem.select(:order_id)
.where.not(discount_applied: nil)
.distinct.map(&:order_id)
# exclude those ids from all orders:
Order.where.not(id: excluded_ids)
You can combine them in a single finder method:
Order.where.not(id: LineItem
.select(:order_id)
.where.not(discount_applied: nil))
Hope this helps
A possible code
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
I advice to get familiar with AR documentation for Query Methods.
Update
This seems to be more interested than I initially though. And more complicated, so I will not be able to give you a working code. But I would look into a solution using LineItem.group(order_id).having(discount_applied: nil), which should give you a collection of line_items and then use it as sub-query to find related orders.
If you want all the records where discount_applied is nil then:
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
(use includes to avoid n+1 problem)
or
Order.joins(:line_items).where.not(line_items: {discount_applied: nil})
Here is the solution to your problem
order_ids = Order.joins(:line_items).where.not(line_items: {discount_applied: nil}).pluck(:id)
orders = Order.where.not(id: order_ids)
First query will return ids of Orders with at least one line_item having discount_applied. The second query will return all orders where there are zero instances of a line_item having the discount_applied.
I would use the NOT EXISTS feature from SQL, which is at least available in both MySQL and PostgreSQL
it should look like this
class Order
has_many :line_items
scope :without_discounts, -> {
where("NOT EXISTS (?)", line_items.where("discount_applied is not null")
}
end
If I understood correctly, you want to get all orders for which none line item (if any) has a discount applied.
One way to get those orders using ActiveRecord would be the following:
Order.distinct.left_outer_joins(:line_items).where(line_items: { discount_applied: nil })
Here's a brief explanation of how that works:
The solution uses left_outer_joins, assuming you won't be accessing the line items for each order. You can also use left_joins, which is an alias.
If you need to instantiate the line items for each Order instance, add .eager_load(:line_items) to the chain which will prevent doing an additional query for every order (N+1), i.e., doing order.line_items.each in a view.
Using distinct is essential to make sure that orders are only included once in the result.
Update
My previous solution was only checking that discount_applied IS NULL for at least one line item, not all of them. The following query should return the orders you need.
Order.left_joins(:line_items).group(:id).having("COUNT(line_items.discount_applied) = ?", 0)
This is what's going on:
The solution still needs to use a left outer join (orders LEFT OUTER JOIN line_items) so that orders without any associated items are included.
Groups the line items to get a single Order object regardless of how many items it has (GROUP BY recipes.id).
It counts the number of line items that were given a discount for each order, only selecting the ones whose items have zero discounts applied (HAVING (COUNT(line_items.discount_applied) = 0)).
I hope that helps.
You cannot do this efficiently with a classic rails left_joins, but sql left join was build to handle thoses cases
Order.joins("LEFT JOIN line_items AS li ON li.order_id = orders.id
AND li.discount_applied IS NOT NULL")
.where("li.id IS NULL")
A simple inner join will return all orders, joined with all line_items,
but if there are no line_items for this order, the order is ignored (like a false where)
With left join, if no line_items was found, sql will joins it to an empty entry in order to keep it
So we left joined the line_items we don't want, and find all orders joined with an empty line_items
And avoid all code with where(id: pluck(:id)) or having("COUNT(*) = 0"), on day this will kill your database
I have following method in a model named CashTransaction.
def is_refundable?
self.amount > self.total_refunded_amount
end
def total_refunded_amount
self.refunds.sum(:amount)
end
Now I need to extract all the records which satisfy the above function i.e records which return true.
I got that working by using following statement:
CashTransaction.all.map { |x| x if x.is_refundable? }
But the result is an Array. I am looking for ActiveRecord_Relation object as I need to perform join on the result.
I feel I am missing something here as it doesn't look that difficult. Anyways, it got me stuck. Constructive suggestions would be great.
Note: Just amount is a CashTransaction column.
EDIT
Following SQL does the job. If I can change that to ORM, it will still do the job.
SELECT `cash_transactions`.* FROM `cash_transactions` INNER JOIN `refunds` ON `refunds`.`cash_transaction_id` = `cash_transactions`.`id` WHERE (cash_transactions.amount > (SELECT SUM(`amount`) FROM `refunds` WHERE refunds.cash_transaction_id = cash_transactions.id GROUP BY `cash_transaction_id`));
Sharing Progress
I managed to get it work by following ORM:
CashTransaction
.joins(:refunds)
.group('cash_transactions.id')
.having('cash_transactions.amount > sum(refunds.amount)')
But what I was actually looking was something like:
CashTransaction.joins(:refunds).where(is_refundable? : true)
where is_refundable? being a model function. Initially I thought setting is_refundable? as attr_accesor would work. But I was wrong.
Just a thought, can the problem be fixed in an elegant way using Arel.
There are two options.
1) Finish, what you have started (which is extremely inefficient when it comes to bigger amount of data, since it all is taken into the memory before processing):
CashTransaction.all.map(&:is_refundable?) # is the same to what you've written, but shorter.
SO get the ids:
ids = CashTransaction.all.map(&:is_refundable?).map(&:id)
ANd now, to get ActiveRecord Relation:
CashTransaction.where(id: ids) # will return a relation
2) Move the calculation to SQL:
CashTransaction.where('amount > total_refunded_amount')
Second option is in every possible way faster and efficient.
When you deal with database, try to process it on the database level, with smallest Ruby involvement possible.
EDIT
According to edited question here is how you would achieve the desired result:
CashTransaction.joins(:refunds).where('amount > SUM(refunds.amount)')
EDIT #2
As to your updates in question - I don't really understand, why you have latched onto is_refundable? as an instance method, which could be used in query, which is basically not possible in AR, but..
My suggestion is to create a scope is_refundable:
scope :is_refundable, -> { CashTransaction
.joins(:refunds)
.group('cash_transactions.id')
.having('cash_transactions.amount > sum(refunds.amount)')
}
Now it is available in as short notation as
CashTransaction.is_refundable
which is shorter and more clear than aimed
CashTransaction.where('is_refundable = ?', true)
You can do it this way:
cash_transactions = CashTransaction.all.map { |x| x if x.is_refundable? } # Array
CashTransaction.where(id: cash_transactions.map(&:id)) # ActiveRecord_Relation
But, this is an in-efficient way of doing it as the other answerers also mentioned.
You can do it using SQL if amount and total_refunded_amount are the columns of the cash_transactions table in the database which will be much more efficient and performant:
CashTransaction.where('amount > total_refunded_amount')
But, if amount or total_refunded_amount are not the actual columns in the database, then you can't do it this way. Then, I guess you have do it the other way which is in-efficient than using raw SQL.
I think you should pre-compute is_refundable result (in a new column) when a CashTransaction and his refunds (supposed has_many ?) are updated by using callbacks :
class CashTransaction
before_save :update_is_refundable
def update_is_refundable
is_refundable = amount > total_refunded_amount
end
def total_refunded_amount
self.refunds.sum(:amount)
end
end
class Refund
belongs_to :cash_transaction
after_save :update_cash_transaction_is_refundable
def update_cash_transaction_is_refundable
cash_transaction.update_is_refundable
cash_transaction.save!
end
end
Note : The above code must certainly be optimized to prevent some queries
They you can query is_refundable column :
CashTransaction.where(is_refundable: true)
I think it's not bad to do this on two queries instead of a join table, something like this
def refundable
where('amount < ?', total_refunded_amount)
end
This will do a single sum query then use the sum in the second query, when the tables grow larger you might find that this is faster than doing a join in the database.
I have some Events, People. There is a many-to-many relationship between them so there is a PersonEvent connecting Events to People.
Event has a date and type
PersonEvent has an event_id and a person_id
Person has a name
I'm trying to build a search form that allows the user to search by the type of an Event, and then returns a list of People who attended a Event of that type in the past and the last date they attended such an Event. This should be in the controller.
The only solution I can think of involves nested loops and will probably run very slowly. I'm definitely looping through a lot of things I don't need to be.
For each person in Person.all
For each personevent in PersonEvent.all
Add the personevent to an array if the person_event.event.type is correct
Now, loop through the array and find the event with the latest date. That's the date of the last Event attendance.
Can anyone suggest a better algorithm?
In RoR, it would be:
Person.joins(:events).where(events: { type: params[:type] })
Rails joins will create an INNER JOIN, which will discard people who don't have an associated event that meets the criteria in where.
You don't explain how your keeping the date of attendance information, so I'll leave that bit up to you.
As you have the associations already set up you should be able to do something like:
f = Person.joins(:events)
f = f.where(:events => { :type => "the_type_you_are_searching_for" })
f = f.group('people.id')
f = f.select('people.*, max(events.date) as last_event_date')
people = f.all # though you probably want to paginate really
I've done it line by line to make it easier to read in here but often you'd see the where, group and select chained together one after the other on the same line.
You need the group otherwise you'll get people returned multiple times if they have been to multiple events.
The custom select is to include the last_event_date in the results.
Why not just write a custom SQL query? It would look something like this:
SELECT * FROM person_events
INNER JOIN people ON people.id = person_events.person_id
INNER JOIN events ON events.id = person_events.event_id AND events.type = 'EventType'
i am trying to query my postgres db from rails with the following query
def is_manager(team)
User.where("manager <> 0 AND team_id == :team_id", {:team_id => team.id})
end
this basically is checking that the manager is flagged and the that team.id is the current id passed into the function.
i have the following code in my view
%td= is_manager(team)
error or what we are getting return is
#<ActiveRecord::Relation:0xa3ae51c>
any help on where i have gone wrong would be great
Queries to ActiveRecord always return ActiveRecord::Relations. Doing so essentially allows the lazy loading of queries. To understand why this is cool, consider this:
User.where(manager: 0).where(team_id: team_id).first
In this case, we get all users who aren't managers, and then we get all the non-manager users who are on team with id team_id, and then we select the first one. Executing this code will give you a query like:
SELECT * FROM users WHERE manager = 0 AND team_id = X LIMIT 1
As you can see, even though there were multiple queries made in our code, ActiveRecord was able to squish all of that down into one query. This is done through the Relation. As soon as we need to actual object (i.e. when we call first), then ActiveRecord will go to the DB to get the records. This prevents unnecessary queries. ActiveRecord is able to do this because they return Relations, instead of the queried objects. The best way to think of the Relation class is that it is an instance of ActiveRecord with all the methods of an array. You can call queries on a relation, but you can also iterate over it.
Sorry if that isn't clear.
Oh, and to solve your problem. %td = is_manager(team).to_a This will convert the Relation object into an array of Users.
Just retrieve first record with .first, this might help.
User.where("manager <> 0 AND team_id == :team_id", {:team_id => team.id}).first