I've got a Rails app with models called Item, ItemTag, and Tag. Items have many Tags through ItemTags. The associations are working correctly.
I want to build a search interface for Items that allows filtering by multiple Tags. I have a bunch of named scopes for various conditions, and they chain without problems. I figure I'll build the query based on user input. So I made a scope that returns the set of Items that has a given tag:
scope :has_tag, -> (tag_id) { joins(:tags).where(tags: {id: tag_id}) }
It works!
> Item.has_tag(73).count
=> 6
> Item.has_tag(81).count
=> 5
But:
> Item.has_tag(73).has_tag(81).count
=> 0
There are two items that have both tags, but chaining the scopes together produces the following SQL, which is always going to return empty, for obvious reasons; it's looking for a tag that has two ids, rather than an item that has two tags.
SELECT [items].*
FROM [items]
INNER JOIN [item_tags] ON [item_tags].[item_id] = [items].[id]
INNER JOIN [tags] ON [tags].[id] = [item_tags].[tag_id]
WHERE [tags].[id] = 81 AND [tags].[id] = 73
I know I can get the intersection of the collections after they're returned, but this seems inefficient, so I'm wondering if there is a standard practice here. (It seems like a common task.)
> (Item.has_tag(73) & Item.has_tag(81)).count
=> 2
Is there a way to write the scope to make this work, or will this need to be done another way?
I think the input tags need to be processed as an array, not chained. With this statement, you're getting a subset under tag(83) of tag(71), which is not what you want. The query confirms this.
Item.has_tag(73).has_tag(81).count
You might try a scope that processes the tags as an array and then constructs the query using lambda
scope :has_tags, lambda { |tag_ids| includes(:tag_ids)
.where("tag_id IN (?)", tag_ids.collect { |tag_id| } )
.group('"items"."id"')
.having('COUNT(DISTINCT "tag_ids"."id") = ?', tag_ids.count) }
I don't have full knowledge of your model relationship setups, etc, and I'm the typo queen, so the snippet above may or may not work out of the box, but I think the information here is that yes, you can process a set of tags in a single scope and query once to get a collection.
Related
I'm using Rails 5. I have the following model ...
class Order < ApplicationRecord
...
has_many :line_items, :dependent => :destroy
The LineItem model has an attribute, "discount_applied." I would like to return all orders where there are zero instances of a line item having the "discount_applied" field being not nil. How do I write such a finder method?
First of all, this really depends on whether or not you want to use a pure Arel approach or if using SQL is fine. The former is IMO only advisable if you intend to build a library but unnecessary if you're building an app where, in reality, it's highly unlikely that you're changing your DBMS along the way (and if you do, changing a handful of manual queries will probably be the least of your troubles).
Assuming using SQL is fine, the simplest solution that should work across pretty much all databases is this:
Order.where("(SELECT COUNT(*) FROM line_items WHERE line_items.order_id = orders.id AND line_items.discount_applied IS NULL) = 0")
This should also work pretty much everywhere (and has a bit more Arel and less manual SQL):
Order.left_joins(:line_items).where(line_items: { discount_applied: nil }).group("orders.id").having("COUNT(line_items.id) = 0")
Depending on your specific DBMS (more specifically: its respective query optimizer), one or the other might be more performant.
Hope that helps.
Not efficient but I thought it may solve your problem:
orders = Order.includes(:line_items).select do |order|
order.line_items.all? { |line_item| line_item.discount_applied.nil? }
end
Update:
Instead of finding orders which all it's line items have no discount, we can exclude all the orders which have line items with a discount applied from the output result. This can be done with subquery inside where clause:
# Find all ids of orders which have line items with a discount applied:
excluded_ids = LineItem.select(:order_id)
.where.not(discount_applied: nil)
.distinct.map(&:order_id)
# exclude those ids from all orders:
Order.where.not(id: excluded_ids)
You can combine them in a single finder method:
Order.where.not(id: LineItem
.select(:order_id)
.where.not(discount_applied: nil))
Hope this helps
A possible code
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
I advice to get familiar with AR documentation for Query Methods.
Update
This seems to be more interested than I initially though. And more complicated, so I will not be able to give you a working code. But I would look into a solution using LineItem.group(order_id).having(discount_applied: nil), which should give you a collection of line_items and then use it as sub-query to find related orders.
If you want all the records where discount_applied is nil then:
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
(use includes to avoid n+1 problem)
or
Order.joins(:line_items).where.not(line_items: {discount_applied: nil})
Here is the solution to your problem
order_ids = Order.joins(:line_items).where.not(line_items: {discount_applied: nil}).pluck(:id)
orders = Order.where.not(id: order_ids)
First query will return ids of Orders with at least one line_item having discount_applied. The second query will return all orders where there are zero instances of a line_item having the discount_applied.
I would use the NOT EXISTS feature from SQL, which is at least available in both MySQL and PostgreSQL
it should look like this
class Order
has_many :line_items
scope :without_discounts, -> {
where("NOT EXISTS (?)", line_items.where("discount_applied is not null")
}
end
If I understood correctly, you want to get all orders for which none line item (if any) has a discount applied.
One way to get those orders using ActiveRecord would be the following:
Order.distinct.left_outer_joins(:line_items).where(line_items: { discount_applied: nil })
Here's a brief explanation of how that works:
The solution uses left_outer_joins, assuming you won't be accessing the line items for each order. You can also use left_joins, which is an alias.
If you need to instantiate the line items for each Order instance, add .eager_load(:line_items) to the chain which will prevent doing an additional query for every order (N+1), i.e., doing order.line_items.each in a view.
Using distinct is essential to make sure that orders are only included once in the result.
Update
My previous solution was only checking that discount_applied IS NULL for at least one line item, not all of them. The following query should return the orders you need.
Order.left_joins(:line_items).group(:id).having("COUNT(line_items.discount_applied) = ?", 0)
This is what's going on:
The solution still needs to use a left outer join (orders LEFT OUTER JOIN line_items) so that orders without any associated items are included.
Groups the line items to get a single Order object regardless of how many items it has (GROUP BY recipes.id).
It counts the number of line items that were given a discount for each order, only selecting the ones whose items have zero discounts applied (HAVING (COUNT(line_items.discount_applied) = 0)).
I hope that helps.
You cannot do this efficiently with a classic rails left_joins, but sql left join was build to handle thoses cases
Order.joins("LEFT JOIN line_items AS li ON li.order_id = orders.id
AND li.discount_applied IS NOT NULL")
.where("li.id IS NULL")
A simple inner join will return all orders, joined with all line_items,
but if there are no line_items for this order, the order is ignored (like a false where)
With left join, if no line_items was found, sql will joins it to an empty entry in order to keep it
So we left joined the line_items we don't want, and find all orders joined with an empty line_items
And avoid all code with where(id: pluck(:id)) or having("COUNT(*) = 0"), on day this will kill your database
I am in a situation to filter the records based on some conditions(conditions are in the form of scopes). in user model
scope :has_email, -> { where('email IS NOT NULL') }
scope :email_contains, (email) -> { where("email ILIKE :email'", email: email)}
If I want both conditions to be combined with 'AND' operator, We can do something like,
User.has_email.email_contains
The query generated would be
SELECT "user".* FROM "user" WHERE (email ILIKE '%gmail.com%') AND (email IS NOT NULL)
How can I proceed if I need scopes to be combined with OR operators? I found that rails 5 added support to or method(https://blog.bigbinary.com/2016/05/30/rails-5-adds-or-support-in-active-record.html), But this won't work if we use includes or joins
Eg: User.has_email.or(User.some_scope).or(User.joins(:event).temp)
How do I join scopes with OR?
The bit you are missing is that a join is forcing the association to exist. To prevent that, you use left_joins:
User.left_joins(:event).where(event: {foo: bar})
Still it won't solve the issue because the .or method will work (by documentation) only on structurally equivalent relations.
You can actually overcome it by going one step lower, to Arel:
rels = [User.foo, User.bar(baz), User.joins(:event).temp]
cond = rels.map { |rel| rel.where_values.reduce(&:and) }.reduce(&:or)
User.left_joins(:event).where(cond)
The where_values property is an array of Arel::Nodes::Node instances, all of which are normally and-ed to get your query. You have to and them by hand, and then or the results.
If something does not work as expected, check the output of cond.to_sql in case you have missed something.
for a data analysis i need both results into one set.
a.follower_trackings.pluck(:date, :new_followers, :deleted_followers)
a.data_trackings.pluck(:date, :followed_by_count)
instead of ugly-merging an array (they can have different starting dates and i obv. need only those values where the date exists in both arrays) i thought about mysql
SELECT
followers.new_followers,
followers.deleted_followers,
trackings.date,
trackings.followed_by_count
FROM
instagram_user_follower_trackings AS followers,
instagram_data_trackings AS trackings
WHERE
followers.date = trackings.date
AND
followers.user_id=5
AND
trackings.user_id=5
ORDER
BY trackings.date DESC
This is Working fine, but i wonder if i can write the same with ActiveRecord?
You can do the following which should render the same query as your raw SQL, but it's also quite ugly...:
a.follower_trackings.
merge(a.data_trackings).
from("instagram_user_follower_trackings, instagram_data_trackings").
where("instagram_user_follower_trackings.date = instagram_data_trackings.date").
order(:date => :desc).
pluck("instagram_data_trackings.date",
:new_followers, :deleted_followers, :followed_by_count)
There are a few tricks turned out useful while playing with the scopes: the merge trick adds the data_trackings.user_id = a.id condition but it does not join in the data_trackings, that's why the from clause has to be added, which essentially performs the INNER JOIN. The rest is pretty straightforward and leverages the fact that order and pluck clauses do not need the table name to be specified if the columns are either unique among the tables, or are specified in the SELECT (pluck).
Well, when looking again, I would probably rather define a scope for retrieving the data for a given user (a record) that would essentially use the raw SQL you have in your question. I might also define a helper instance method that would call the scope with self, something like:
def Model
scope :tracking_info, ->(user) { ... }
def tracking_info
Model.tracking_info(self)
end
end
Then one can use simply:
a = Model.find(1)
a.tracking_info
# => [[...], [...]]
In Rails 3 I can perform query on associated models:
EXAMPLE 1:
model.associated_models.where(:attribute => 1)
associated_models is an array of models.
Is it possible to perform activerecord query on manualy created array of models?
EXAMPLE 2:
[Model.create!(attribute: 1), Model.create!(attribute: 2)].where(:attribute => 1)
Just like associated_models in first example its and array of models, but I guess there is something going on backstage when calling associated_models.
Can I simmulate this behaviour to get example 2 working?
short answer is no, you cannot. Activerecord scope chains construct queries for the db and this cannot be interpreted for arbitrary arrays, even if it is array of AR objects like in your example.
You can 'simulate' it by either a proper db scope
Model.where(:id => array_of_ar_objects.map(&:id), :attribute => 1)
(but this is wrong, since you want to do db calls only if needed) or by using array search:
array_of_ar_objects.select { |model| model.attribute == 1 }
Also note that model.associated_models is not an Array, but a ActiveRecord::Associations::HasManyAssociation, a kind of association proxy. It is quite tricky cause even its 'class' method is delegated to the array it is coerced to, this is why you were misled I guess.
model.associated_models.class == Array
-> true
I'd recommend Array#keep_if for this task rather than, squeezing an array into an an ActiveRecord::Relation.
[Model.create!(attribute: 1), Model.create!(attribute: 2)].keep_if { |m| m.attribute == 1 }
http://www.ruby-doc.org/core-1.9.3/Array.html#method-i-keep_if
(Note, Array#select! does the same thing, but I prefer keep_if to avoid confusion when reading it later, thinking that it might be related to an sql select)
I'm wondering if there's a way to fetch objects from the DB via ActiveRecord, without having Rails build the whole objects (just a few fields).
For example,
I sometimes need to check whether a certain object contains a certain field.
Let's say I have a Student object referencing a Bag object (each student has one bag).
I need to check if a female student exists that her bag has more than 4 pencils.
In ActiveRecord, I would have to do something like this:
exists = Student.female.find(:all, conditions => 'bags.pencil_count > 4', :include => :bag).size > 0
The problem is that if there are a 1000 students complying with this condition,
a 1000 objects would be built by AR including their 1000 bags.
This reduces me to using plain SQL for this query (for performance reasons), which breaks the AR.
I won't be using the named scopes, and I would have to remember to update them all around the code,
if something in the named scope changes.
This is an example, but there are many more cases that for performance reasons,
I would have to use SQL instead of letting AR build many objects,
and this breaks encapsulation.
Is there any way to tell AR not to build the objects, or just build a certain field (also in associations)?
If you're only testing for the existence of a matching record, just use Model.count from ActiveRecord::Calculations, e.g.:
exists = Student.female.count( :conditions => 'bags.pencil_count > 4',
:joins => :bag
) > 0
count simply (as the name of the class implies), does the calculation and doesn't build any objects.
(And for future reference it's good to know the difference between :include and :joins. The former eager-loads the associated model, whereas the latter does not, but still lets you use those fields in your :conditions.)
Jordan gave the best answer here - especially re: using joins instead of include (because join won't actually create the bag objects)
I'll just add to it by saying that if you do actually still need the "Student" objects (just with the small amount of info on it) you can also use the :select keyword - which works just like in mysql and means the db I/O will be reduced to just the info you put in the select - and you can also add derived fields form the other tables eg:
students = Student.female.all(
:select => 'students.id, students.name, bags.pencil_count AS pencil_count',
:conditions => 'students.gender = 'F' AND bags.pencil_count > 4',
:joins => :bag
)
students.each do |student|
p "#{student.name} has #{student.pencil_count} pencils in her bag"
end
would give eg:
Jenny has 5 pencils in her bag
Samantha has 14 pencils in her bag
Jill has 8 pencils in her bag
(though note that a derived field (eg pencil_count) will be a string - you may need to cast eg with student.pencil_count.to_i )