Problem
I have a Post and a Comment and I want to select posts and use a .joins() and a .where() on the Comment that contains an OR and has 2 IN conditions.
I want something that generates this:
SELECT * FROM posts
INNER JOIN comments ON comments.post_id = posts.id
WHERE comments.id IN (1,2,3) OR comments.user_id IN (4,5,6)
I would use the .or() method but it cannot take a hash.
Post.joins(Comment)
.where({ comments: { id: [1, 2, 3] } })
.or({ comments: { user_id: [4, 5, 6] } }) # <-- raises exception
Possible Solution
I simplified this for readability. In reality I need this to work across database adapters so I'd use Comment.connection.quote_table_name and Comment.connection.quote_column_name to correctly quote the table and column names.
ids = [1,2,3]
user_ids = [4,5,6]
clause = ""
clause += Comment.sanitize_sql_for_conditions(["comments.id IN (?)", ids]) if ids.any?
clause += " OR " if ids.any? and user_ids.any?
clause += Comment.sanitize_sql_for_conditions(["comments.user_id IN (?)", user_ids]) if user_ids.any?
Post.joins(Comment).where(clause)
Question
This works but it seems like there should be a better way... is there?
I assume you have a comments relation on your Post class with has_many :comments, Rails is clever enough to know that when you uses .where with the relation name, then you are thinking about the id of each comment, then you can simply write the ids.
To use the OR you must use the same class wich will be used to contruct the main query, it like a "subquery" inside the or, like follow.
Please try with the next code:
Post.joins(:comments)
.where(comments: [1, 2, 3])
.or(Post.where('comments.user_id IN ?', [4, 5, 6]))
RAILS OR: https://zaiste.net/rails_5_or_operator_active_record/
JOINS: https://apidock.com/rails/ActiveRecord/QueryMethods/joins
EDIT:
Due the known issue referencing on this answer, you should use raw SQL like follow.
Post.joins(:comments)
.where('comments.id in ? OR comments.user_id in ?', [1, 2, 3], [4, 5, 6])
Each ? inside the raw sql will be replaced with the parameter passed to .where from left to right in same order.
I'm trying to find the cleanest way to select records based on its associations and a search array.
I have Recipes which have many Ingredients (through a join table)
I have a search form field for an array of Ingredient.ids
To find any recipe which contains any of the ids in the search array, I can use
eg 1.
filtered_meals = Recipe.includes(:ingredients).where("ingredients.id" => ids)
BUT, I want to only match recipes where ALL of it's ingredients are found in the search array.
eg 2.
search_array = [1, 2, 3, 4, 5]
Recipe1 = [1, 4, 5, 6]
Recipe2 = [1, 3, 4]
# results => Recipe2
I am aware that I can use an each loop, something like this;
eg 3.
filtered_meals = []
Recipes.each do |meal|
meal_array = meal.ingredients.ids
variable = meal_array-search_array
if variable.empty?
filtered_meals.push(meal)
end
end
end
return filtered_meals
The problem here is pagination. In the first example I can use .limit() and .offset() to control how many results are shown, but in the third example I would need to add an extra counter, submit that with the results, and then on a page change, re-send the counter and use .drop(counter) on the each.do loop.
This seems way too long winded, is there any better way to do this??
Assuming you are using has_many through & recipe_id, ingredient_id combination are unique.
recipe_ids = RecipeIngredient.select(:recipe_id)
.where(ingredient_id: ids)
.group(:recipe_id)
.having("COUNT(*) >= ?", ids.length)
filtered_meals = Recipe.find recipe_ids
How about
filtered_meals = Recipe.joins(:ingredients)
.group(:recipe_id)
.order("ingredients.id ASC")
.having("array_agg(ingredients.id) = ?", ids)
You'll need to make sure your ids parameter is listed in ascending order so the order of the elements in the arrays will match too.
Ruby on Rails Guide 2.3.3 - Subset Conditions
Recipe.all(:ingredients => { :id => search_array })
Should result in:
SELECT * FROM recipes WHERE (recipes.ingredients IN (1,2,3,4,5))
in SQL.
Would the array & operator work for you here?
Something like:
search_array = [1, 2, 3, 4, 5]
recipe_1 = [1, 4, 5, 6]
recipe_2 = [1, 3, 4]
def contains_all_ingredients?(search_array, recipe)
(search_array & recipe).sort == recipe.sort
end
contains_all_ingredients(search_array, recipe_1) #=> false
contains_all_ingredients(search_array, recipe_2) #=> true
This method compares the arrays and returns only the elements present in both, so if the result of the comparison equals the recipe array, all are present. (And obviously you could have a little refactor to have the method sit in the recipe model.)
You could then do:
Recipes.all.select { |recipe| contains_all_ingredients?(search_array, recipe) }
I'm not sure it passes your example three, but might help on your way? Let me know if that starts off OK, and I'll have more of a think in the meantime / if it's useful :)
I had a similar need and solved it using the pattern below. This is what the method looks like in my Recipe model.
def self.user_has_all_ingredients(ingredient_ids)
# casts ingredient_ids to postgres array syntax
ingredient_ids = '{' + ingredient_ids.join(', ') + '}'
return Recipe.joins(:ingredients)
.group(:id)
.having('array_agg(ingredients.id) <# ?', ingredient_ids)
end
This returns every recipe where all of the required ingredients are included in an ingredients array.
The Postgres '<#' operator was the magic solution. The array_agg function creates an array of each recipe's ingredient ids and then the left-pointing bird operator asks whether all of the unique ids in that array are contained in the array on the right.
Using the array_agg function required me to cast my search_array into Postgres syntax.
My Recipes model has many Ingredients through Portions.
I'd love to know if anyone has any better optimizations or knows how to avoid the casting to Postgres syntax that I needed to do.
Knowing that by default Rails orders data by ID, how can I order by ids given to the where clause?
ids = Bookmark.where(user_id: 7).order(created_at: :desc).pluck(:company_id)
Result:
[146, 140, 128, 4, 2]
Now, when I try to get the companies in the same order from ids
Company.where(id: ids).pluck(:id)
Result:
[2, 4, 128, 140, 146]
Expected Result:
[146, 140, 128, 4, 2]
My pretended result will be the same in both cases (same order).
The companies should be returned in the same order that the Bookmarks on that company where created.
Company.includes(:bookmarks) .where(id: ids).order('bookmarks.created_at desc').pluck(:id)
So it looks like given a user ID, you want a list of companies sorted by the created_at date of the bookmarks that join Users and Companies.
Am I correct in assuming that Users and Companies share a many-to-many relationship through Bookmarks?
If that's the case, the following "join" could work for you!
Company.joins(:bookmarks).where('bookmarks.user_id = ?', 7).order('bookmarks.created_at')
Of course, in your code, this could be generalized to grab companies for any user!
user_id = 42
companies = Company.joins(:bookmarks).where('bookmarks.user_id = ?', user_id).order('bookmarks.created_at')
ActiveRecord "joins" reference
What exactly are you trying to order by... the company_id?
pluck returns an array, which is why this does NOT work:
ids = Bookmark.where(user_id: 7).pluck(:company_id).order(company_id: :desc)
## undefined method `order' for Array
Instead, you can call sort on the array.
ids = Bookmark.where(user_id: 7).pluck(:company_id).sort
That should do the trick
You are explicitly ordering by created_at in order(created_at: :desc)
You want to order by company_id ids = Bookmark.where(user_id: 7).order(:company_id).pluck(:company_id)
This is how I have solved the question:
Bookmark.includes(:company).where(user_id: current_user).order(created_at: :desc)
and when iterating over the elements I use:
record.company instead of record.
This way I have the companies from the same order that the Bookmarks where created.
Probably you can try this:
company_by_id = Company.find(ids).index_by(&:id) # Gives you a hash indexed by ID
ids.collect {|id| company_by_id[id].id }
I have prepare these two arrays:
list_of_students = Student.where('class = ?', param[:given_class])
list_of_teachers = Teacher.where(...)
Student belongs_to Teacher and Teacher has_many students.
And now, I'd need to remove from the list_of_students all items (students), where teacher_id is not included in list_of_teachers.
I found some tips and HOWTO's on comparing arrays, but none of that helped to figure out this case.
Thank you in advance
You can use the IN SQL statement.
list_of_students = Student.where('class = ? AND teacher_id IN (?)', param[:given_class], list_of_teachers.map(&:id))
If the list_of_teachers is an ActiveRecord::Relation (and not an array), you can also use #pluck(:id) or (from Rails 4) #ids
Student.where('class = ? AND teacher_id IN (?)', param[:given_class], list_of_teachers.ids)
There are several ways to write the IN statement. Given you already have a where, I joined it to the main where. But you could also write
Student.where('class = ?', param[:given_class]).where(teacher_id: list_of_teachers)
or
Student.where(class: param[:given_class], teacher_id: list_of_teachers)
Also note you don't need to assign the list of teachers to a variable.
Student.where(class: param[:given_class], teacher_id: Teacher.where(...).ids)
Last but not least, if your Teacher query is simple, you may want to use a single query and a JOIN. Assume you want to get all the Teachers with name Rose.
Student.where(class: param[:given_class], teacher_id: Teacher.where(name: 'Rose').ids)
You can rewrite the same query to
Student.where(class: param[:given_class]).joins(:teacher).where(teacher: { name: 'Rose' })
or (the final and shorter expression)
Student.joins(:teacher).where(class: param[:given_class], teacher: { name: 'Rose' })
You can try something like
a = [1,2,3]
b = [1,4,5]
pry(main)> a.delete_if {|a1| !b.include? a1}
=> [1]
it checks each value in a is in b or not. If not it deletes the value from a and gives you a array finally.
This is an example. You can use this accordingly
I have a query to get the IDs of people in a particular order, say:
ids = [1, 3, 5, 9, 6, 2]
I then want to fetch those people by Person.find(ids)
But they are always fetched in numerical order, I know this by performing:
people = Person.find(ids).map(&:id)
=> [1, 2, 3, 5, 6, 9]
How can I run this query so that the order is the same as the order of the ids array?
I made this task more difficult as I wanted to only perform the query to fetch people once, from the IDs given. So, performing multiple queries is out of the question.
I tried something like:
ids.each do |i|
person = people.where('id = ?', i)
But I don't think this works.
Editor's note:
As of Rails 5, find returns the records in the same order as the provided IDs (docs).
Note on this code:
ids.each do |i|
person = people.where('id = ?', i)
There are two issues with it:
First, the #each method returns the array it iterated on, so you'd just get the ids back. What you want is a collect
Second, the where will return an Arel::Relation object, which in the end will evaluate as an array. So you'd end up with an array of arrays. You could fix two ways.
The first way would be by flattening:
ids.collect {|i| Person.where('id => ?', i) }.flatten
Even better version:
ids.collect {|i| Person.where(:id => i) }.flatten
A second way would by to simply do a find:
ids.collect {|i| Person.find(i) }
That's nice and simple
You'll find, however, that these all do a query for each iteration, so not very efficient.
I like Sergio's solution, but here's another I would have suggested:
people_by_id = Person.find(ids).index_by(&:id) # Gives you a hash indexed by ID
ids.collect {|id| people_by_id[id] }
I swear that I remember that ActiveRecord used to do this ID ordering for us. Maybe it went away with Arel ;)
As I see it, you can either map the IDs or sort the result. For the latter, there already are solutions, though I find them inefficient.
Mapping the IDs:
ids = [1, 3, 5, 9, 6, 2]
people_in_order = ids.map { |id| Person.find(id) }
Note that this will cause multiple queries to be executed, which is potentially inefficient.
Sorting the result:
ids = [1, 3, 5, 9, 6, 2]
id_indices = Hash[ids.map.with_index { |id,idx| [id,idx] }] # requires ruby 1.8.7+
people_in_order = Person.find(ids).sort_by { |person| id_indices[person.id] }
Or, expanding on Brian Underwoods answer:
ids = [1, 3, 5, 9, 6, 2]
indexed_people = Person.find(ids).index_by(&:id) # I didn't know this method, TIL :)
people_in_order = indexed_people.values_at(*ids)
Hope that helps
If you have ids array then it is as simple as -
Person.where(id: ids).sort_by {|p| ids.index(p.id) }
OR
persons = Hash[ Person.where(id: ids).map {|p| [p.id, p] }]
ids.map {|i| persons[i] }
With Rails 5, I've found that this approach works (with postgres, at least), even for scoped queries, useful for working with ElasticSearch:
Person.where(country: "France").find([3, 2, 1]).map(&:id)
=> [3, 2, 1]
Note that using where instead of find does not preserve the order.
Person.where(country: "France").where(id: [3, 2, 1]).map(&:id)
=> [1, 2, 3]
There are two ways to get entries by given an array of ids. If you are working on Rails 4, dynamic method are deprecated, you need to look at the Rails 4 specific solution below.
Solution one:
Person.find([1,2,3,4])
This will raise ActiveRecord::RecordNotFound if no record exists
Solution two [Rails 3 only]:
Person.find_all_by_id([1,2,3,4])
This will not cause exception, simply return empty array if no record matches your query.
Based on your requirement choosing the method you would like to use above, then sorting them by given ids
ids = [1,2,3,4]
people = Person.find_all_by_id(ids)
# alternatively: people = Person.find(ids)
ordered_people = ids.collect {|id| people.detect {|x| x.id == id}}
Solution [Rails 4 only]:
I think Rails 4 offers a better solution.
# without eager loading
Person.where(id: [1,2,3,4]).order('id DESC')
# with eager loading.
# Note that you can not call deprecated `all`
Person.where(id: [1,2,3,4]).order('id DESC').load
You can get users sorted by id asc from the database and then rearrange them in the application any way you want. Check this out:
ids = [1, 3, 5, 9, 6, 2]
users = ids.sort.map {|i| {id: i}} # Or User.find(ids) or another query
# users sorted by id asc (from the query)
users # => [{:id=>1}, {:id=>2}, {:id=>3}, {:id=>5}, {:id=>6}, {:id=>9}]
users.sort_by! {|u| ids.index u[:id]}
# users sorted as you wanted
users # => [{:id=>1}, {:id=>3}, {:id=>5}, {:id=>9}, {:id=>6}, {:id=>2}]
The trick here is sorting the array by an artificial value: index of object's id in another array.
I here summarise the solutions, plus adding recent (9.4+) PostgreSQL-specific solution. The following is based on Rails 6.1 and PostgreSQL 12. Though I mention solutions for earlier versions of Rails and PostgreSQL, I haven't actually tested them with earlier versions.
For reference, this question "ORDER BY the IN value list" gives various ways of sorting/ordering with the database.
Here, I assume the model is guaranteed to have all the records specified by the Array of IDs, ids. Otherwise, an exception like ActiveRecord::RecordNotFound may be raised (or may not, depending on the way).
What does NOT work
Person.where(id: ids)
The order of the returned Relation is either arbitrary or that of the numerical values of the primary IDs; whichever, it usually does not agree with that of ids.
Simple solution to get an Array
(Rails 5+ only(?))
Person.find ids
which returns a Ruby Array of Person models in the order of the given ids.
A downside is you cannot further modify the result with SQL.
In Rails 3, the following is the way apparently, though this may not work (certainly does not in Rails 6) in the other versions of Rails.
Person.find_all_by_id ids
Pure Ruby solution to get an Array
Two ways. Either works regardless of Rails versions (I think).
Person.where(id: ids).sort_by{|i| ids.index(i.id)}
Person.where(id: ids).index_by(&:id).values_at(*ids)
which returns a Ruby Array of Person models in the order of the given ids.
DB-level solution to get a Relation
All of the following return Person::ActiveRecord_Relation, to which you can apply more filters if you like.
In the following solutions, all records are preserved, including those whose IDs are not included in the given array ids. You can filter them out any time by adding where(id: ids) (this sort of flexibility is a beauty of ActiveRecord_Relation).
For any Database
Based on user3033467's answer but updated to work with Rails 6 (which has disabled some features with order() due to a security concern; see "Updates for SQL Injection in Rails 6.1" by Justin for the background).
order_query = <<-SQL
CASE musics.id
#{ids.map.with_index { |id, index| "WHEN #{id} THEN #{index}" } .join(' ')}
ELSE #{ids.length}
END
SQL
Person.order(Arel.sql(order_query))
For MySQL specific
From Koen's answer (I haven't tested it).
Person.order(Person.send(:sanitize_sql_array, ['FIELD(id, ?)', ids])).find(ids)
For PostgreSQL specific
PostgreSQL 9.4+
join_sql = "INNER JOIN unnest('{#{ids.join(',')}}'::int[]) WITH ORDINALITY t(id, ord) USING (id)"
Person.joins(join_sql).order("t.ord")
PostgreSQL 8.2+
Based on Jerph's answer, but LEFT JOIN is replaced with INNER JOIN:
val_ids = ids.map.with_index.map{|id, i| "(#{id}, #{i})"}.join(", ")
Person.joins("INNER JOIN (VALUES #{val_ids}) AS persons_id_order(id, ordering) ON persons.id = persons_id_order.id")
.order("persons_id_order.ordering")
To get lower-level objects
The following is solutions to get lower-level objects.
In a vast majority of cases, the solutions described above must be superior to these, but am putting there here for the sake of completeness (and record before I found better solutions)…
In the following solutions, the records that do not match IDs in ids are filtered out, unlike the solutions described in the previous section (where all records can be chosen to be preserved).
To get an ActiveRecord::Result
This is a solution to get ActiveRecord::Result with PostgreSQL 9.4+.
ActiveRecord::Result is similar to an Array of Hash.
str_sql = "select persons.* from persons INNER JOIN unnest('{#{ids.join(',')}}'::int[]) WITH ORDINALITY t(id, ord) USING (id) ORDER BY t.ord;"
Person.connection.select_all(str_sql)
Person.connection.exec_query returns the same (alias?).
To get a PG::Result
This is a solution to get PG::Result with PostgreSQL 9.4+. Very similar to above, but replace exec_query with execute (the first line is identical to the solution above):
str_sql = "select persons.* from persons INNER JOIN unnest('{#{ids.join(',')}}'::int[]) WITH ORDINALITY t(id, ord) USING (id) ORDER BY t.ord;"
Person.connection.execute(str_sql)
Old question, but the sorting can be done by ordering using the SQL FIELD function. (Only tested this with MySQL.)
So in this case something like this should work:
Person.order(Person.send(:sanitize_sql_array, ['FIELD(id, ?)', ids])).find(ids)
Which results in the following SQL:
SELECT * FROM people
WHERE id IN (1, 3, 5, 9, 6, 2)
ORDER BY FIELD(id, 1, 3, 5, 9, 6, 2)
Most of the other solutions don't allow you to further filter the resulting query, which is why I like Koen's answer.
Similar to that answer but for Postgres, I add this function to my ApplicationRecord (Rails 5+) or to any model (Rails 4):
def self.order_by_id_list(id_list)
values_clause = id_list.each_with_index.map{|id, i| "(#{id}, #{i})"}.join(", ")
joins("LEFT JOIN (VALUES #{ values_clause }) AS #{ self.table_name}_id_order(id, ordering) ON #{ self.table_name }.id = #{ self.table_name }_id_order.id")
.order("#{ self.table_name }_id_order.ordering")
end
The query solution is from this question.
This is most efficiently handled in SQL via ActiveRecord and not in Ruby.
ids = [3,1,6,7,12,2]
Post.where(id: ids).order("FIELD(id, #{ids.join(',')})")
This simple solution costs less than joining on values:
order_query = <<-SQL
CASE persons.id
#{ids.map.with_index { |id, index| "WHEN #{id} THEN #{index}" } .join(' ')}
ELSE #{ids.length}
END
SQL
Person.where(id: ids).order(order_query)
To get the IDs of people in a particular order, say: ids = [1, 3, 5, 9, 6, 2]
In older version of rails, find and where fetch data in numerical order, but rails 5 fetch data in the same order in which you query it
Note: find preserve the order and where don't preserve it
Person.find(ids).map(&:id)
=> [1, 3, 5, 9, 6, 2]
Person.where(id: ids).map(&:id)
=> [1, 2, 3, 5, 6, 9]
But they are always fetched in numerical order, I know this by performing:
I tried the answers recommending the FIELD method on Rails6 but was encountering errors. However, I discovered that all one has to do is wrap the sql in Arel.sql().
# Make sure it's a known-safe values.
user_ids = [3, 2, 1]
# Before
users = User.where(id: user_ids).order("FIELD(id, 2, 3, 1)")
# With warning.
# After
users = User.where(id: user_ids).order(Arel.sql("FIELD(id, 2, 3, 1)"))
# No warning
[1] https://medium.com/#mitsun.chieh/activerecord-relation-with-raw-sql-argument-returns-a-warning-exception-raising-8999f1b9898a
Use find:
Thing.find([4, 2, 6])
For Rails 7:
Thing.where(id: [4, 2, 6]).in_order_of(:id, [4, 2, 6])
See https://hashrocket.com/blog/posts/return-results-using-a-specific-order-in-rails