I realised its quite difficult to explain my problem with only words, so i'm going to use an example to describe what i am trying to do instead.
So for example:
#model Book
has_many: book_genres
has_many: genres, through: :book_genres
#model Genre
has_many: book_genres
has_many: books, through: :book_genres
So finding books that belong to one genre only would be relatively straightforward, such as:
#method in books model
def self.find_books(genre)
#g = Genre.where('name LIKE ?' , "#{genre}").take
#b = #g.books
#get all the books that are of that genre
end
So in rails console i can do Book.find_books("Fiction") and then i would get all the books that are of fiction genre.
But how can i find all the books that are both "Young Adult" and "Fiction" ? Or what if i would like to query for books that have 3 genres, such as "Young Adult", "Fiction" and "Romance" ?
I could do g = Genre.where(name: ["Young Adult", "Fiction", "Romance"]) but subsequent to that i cannot do g.books and get all the books that are related to this 3 genres.
I am actually quite bad with active record so im not even sure if theres a better way to query through Books directly instead of finding Genre then finding all books that are associated with it.
But what i cannot wrap my head around is how do i get all the books that have multiple (specific)genres?
UPDATE:
So the current answers provided Book.joins(:genres).where("genres.name" => ["Young Adult", "Fiction", "Romance"]) works, but the problem is it returns all books that has the genre of Young Adult OR Fiction OR Romance.
What query do i pass so that the books return has ALL 3 Genres and not only 1 or 2 out of the 3?
Matching any of the given genres
The following should work for both an Array and a String:
Book.joins(:genres).where("genres.name" => ["Young Adult", "Fiction", "Romance"])
Book.joins(:genres).where("genres.name" => "Young Adult")
In general, it's better to pass a Hash to where, rather than trying to write a SQL snippet yourself.
See the Rails Guides for more details:
http://guides.rubyonrails.org/active_record_querying.html#hash-conditions
http://guides.rubyonrails.org/active_record_querying.html#specifying-conditions-on-the-joined-tables
Matching all of the given genres with one query
A single query could be built and then passed to .find_by_query:
def self.in_genres(genres)
sql = genres.
map { |name| Book.joins(:genres).where("genres.name" => name) }.
map { |relation| "(#{relation.to_sql})" }.
join(" INTERSECT ")
find_by_sql(sql)
end
This means that calling Book.in_genres(["Young Adult", "Fiction", "Romance"]) will run a query that looks something like this:
(SELECT books.* FROM books INNER JOIN … WHERE genres.name = 'Young Adult')
INTERSECT
(SELECT books.* FROM books INNER JOIN … WHERE genres.name = 'Fiction')
INTERSECT
(SELECT books.* FROM books INNER JOIN … WHERE genres.name = 'Romance');
It has the upside of letting the database do the heavy lifting of combining the result sets.
The downside is that we're using raw SQL, so we can't chain this with other ActiveRecord methods, for example Books.order(:title).in_genres(["Young Adult", "Fiction"]) will ignore the ORDER BY clause we've tried to add.
We're also manipulating SQL queries as strings. It's possible we could avoid this using Arel, but the way Rails and Arel handle binding query values makes this pretty complicated.
Matching all of the given genres with multiple query
It's also possible to use multiple queries:
def self.in_genres(genres)
ids = genres.
map { |name| Book.joins(:genres).where("genres.name" => name) }.
map { |relation| relation.pluck(:id).to_set }.
inject(:intersection).to_a
where(id: ids)
end
This means that calling Book.in_genres(["Young Adult", "Fiction", "Romance"]) will run four queries that look something like this:
SELECT id FROM books INNER JOIN … WHERE genres.name = 'Young Adult';
SELECT id FROM books INNER JOIN … WHERE genres.name = 'Fiction';
SELECT id FROM books INNER JOIN … WHERE genres.name = 'Romance';
SELECT * FROM books WHERE id IN (1, 3, …);
The downside here is that for N genres, we're making N+1 queries. The upside is that this can be combined with other ActiveRecord methods; Books.order(:title).in_genres(["Young Adult", "Fiction"]) will do our genre filtering, and sort by title.
I didn't try this but I think it will work
Book.joins(:genres).where("genres.name IN (?)", ["Young Adult", "Fiction", "Romance"])
Here is how I would do it in SQL:
SELECT *
FROM books
WHERE id IN (
SELECT bg.book_id
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE g.name LIKE 'Young Adult'
INTERSECT
SELECT bg.book_id
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE g.name LIKE 'Fiction'
INTERSECT
...
)
The inner query will contain only books belonging to all the genres you ask about.
Here is how I'd do it in ActiveRecord:
# book.rb
def self.in_genres(genre_names)
subquery = genre_names.map{|n|
<<-EOQ
SELECT bg.book_id
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE g.name LIKE ?
EOQ
}.join("\nINTERSECT\n")
where(<<-EOQ, *genre_names)
id IN (
#{subquery}
)
EOQ
end
Note that I am using ? to avoid sql injection vulnerabilities, which is a problem in the code you proposed in your question.
Another approach would be to use multiple EXISTS conditions with correlated sub-queries:
SELECT *
FROM books
WHERE EXISTS (SELECT 1
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE g.name LIKE 'Young Adult'
AND bg.book_id = books.id)
AND EXISTS (SELECT 1
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE g.name LIKE 'Fiction'
AND bg.book_id = books.id)
AND ...
You'd construct this query in ActiveRecord similarly to the first approach. I'm not sure which would be faster, so you could try both if you like.
Here is yet another way to do the SQL---possibly fastest:
SELECT *
FROM books
WHERE id IN (
SELECT bg.book_id
FROM book_genres bg
INNER JOIN genres g
ON g.id = bg.genre_id
WHERE (g.name LIKE 'Young Adult' OR g.name LIKE 'Fiction' OR ...)
GROUP BY bg.book_id
HAVING COUNT(DISTINCT bg.genre_id) >= 2 -- or 3, or whatever
)
Related
In the following book club example with associations:
class User
has_and_belongs_to_many :clubs
has_and_belongs_to_many :books
end
class Club
has_and_belongs_to_many :users
has_and_belongs_to_many :books
end
class Book
has_and_belongs_to_many :users
has_and_belongs_to_many :clubs
end
given a specific club record:
club = Club.find(params[:id])
how can I find all the users in the club who have all books in array of books?
club.users.where_has_all_books(books)
In PostgreSQL it can be done with a single query. (Maybe in MySQL too, I'm just not sure.)
So, some basic assumptions first. 3 tables: clubs, users and books, every table has id as a primary key. 3 join tables, books_clubs, books_users, clubs_users, each table contains pairs of ids (for books_clubs it will be [book_id, club_id]), and those pairs are unique within that table. Quite reasonable conditions IMO.
Building a query:
First, let's get ids of books from given club:
SELECT book_id
FROM books_clubs
WHERE club_id = 1
ORDER BY book_id
Then get users from given club, and group them by user.id:
SELECT CU.user_id
FROM clubs_users CU
JOIN users U ON U.id = CU.user_id
JOIN books_users BU ON BU.user_id = CU.user_id
WHERE CU.club_id = 1
GROUP BY CU.user_id
Join these two queries by adding having to 2nd query:
HAVING array_agg(BU.book_id ORDER BY BU.book_id) #> ARRAY(##1##)
where ##1## is the 1st query.
What's going on here: Function array_agg from the left part creates a sorted list (of array type) of book_ids. These are books of user. ARRAY(##1##) from the right part returns the sorted list of books of the club. And operator #> checks if 1st array contains all elements of the 2nd (ie if user has all books of the club).
Since 1st query needs to be performed only once, it can be moved to WITH clause.
Your complete query:
WITH club_book_ids AS (
SELECT book_id
FROM books_clubs
WHERE club_id = :club_id
ORDER BY book_id
)
SELECT CU.user_id
FROM clubs_users CU
JOIN users U ON U.id = CU.user_id
JOIN books_users BU ON BU.user_id = CU.user_id
WHERE CU.club_id = :club_id
GROUP BY CU.user_id
HAVING array_agg(BU.book_id ORDER BY BU.book_id) #> ARRAY(SELECT * FROM club_book_ids);
It can be verified in this sandbox: https://www.db-fiddle.com/f/cdPtRfT2uSGp4DSDywST92/5
Wrap it to find_by_sql and that's it.
Some notes:
ordering by book_id is not necessary; #> operator works with unordered arrays too. I just have a suspicion that comparison of ordered array is faster.
JOIN users U ON U.id = CU.user_id in 2nd query is only necessary for fetching user properties; in case of fetching user ids only it can be removed
It appears to work by grouping and counting.
club.users.joins(:books).where(books: { id: club.books.pluck(:id) }).group('users.id').having('count(*) = ?', club.books.count)
If anyone knows how to run the query without intermediate queries that would be great and I will accept the answer.
This looks like a situation where you'd make two queries, one to get all the ids you need, the other select perform a WHERE IN.
I have two tables users and posts and they have association of has_many. I want to fetch details of both users and posts in a single query. I'm able to manage the sql query but I don't want to use the raw query in the code (using execute method) as i think it is kind of simple thing and can be written using active record.
Here is the sql query
SELECT a.id, a.name, a.timestamp, b.id, b.user_id, b.title
FROM users a
INNER JOIN (SELECT id, user_id, title, from, to FROM posts) b on b.user_id = a.id
where id IN ( 1, 2, 3);
I think includes does not help here because i'm dealing with large data.
Can any one help me ?
If you just want those specific columns and nothing else then this will work
User.joins(:post)
.where(id: [1,2,3])
.select("users.id, users.name, users.timestamp,
posts.id as post_id, posts.user_id as post_user_id,
posts.title as post_title")
This will return an ActiveRecord::Relation of User objects with virtual attributes for post_id, post_user_id (Not sure why you need this one since you already selected users.id), and post_title.
The query produced will be
SELECT users.id,
users.name,
users.timestamp,
posts.id as post_id,
posts.user_id as post_user_id,
posts.title as post_title
FROM users
INNER JOIN posts on posts.user_id = users.id
where users.id IN ( 1, 2, 3);
Please note you may have multiple User objects, one for each Post, just as the SQL query does.
You can execute your exact query using the string version of joins e.g.
User.joins("INNER JOIN (SELECT id, user_id, title, from, to FROM posts) b on b.user_id = users.id")
.where(id: [1,2,3])
.select("users.id, users.name, users.timestamp,
b.id as post_id, b.user_id as post_user_id,
b.title as post_title")
Additionally to avoid some of the overhead you can use arel instead e.g.
users_table = User.arel_table
posts_table = Post.arel_table
query = users_table.project(Arel.star)
.join(posts_table)
.on(posts_table[:user_id].eq(users_table[:id]))
.where(users_table[:id].in([1,2,3]))
ActiveRecord::Base.connection.exec_query(query.to_sql)
This will return an ActiveRecord::Result with 2 useful methods columns (the columns selected) and rows. You can convert this to a Hash(#to_hash) but note that any columns with duplicate names (id for instance) will overwrite one another.
You could fix this by specifying the colums you want selected in the project portion. e.g. your current query would be:
query = users_table.project(
users_table[:id],
users_table[:name],
users_table[:timestamp],
posts_table[:id].as('post_id'),
posts_table[:user_id].as('post_user_id'),
posts_table[:title].as('post_title')
).join(posts_table)
.on(posts_table[:user_id].eq(users_table[:id]))
.where(users_table[:id].in([1,2,3]))
ActiveRecord::Base.connection.exec_query(query.to_sql).to_hash
Since none of the names collide now it can be structured into a nice Hash where the keys are the column names and the values or the row value for that record.
users = User.joins(:posts).includes(:posts).where(id: [1, 2, 3])
Will give you all the users with theirs posts.
then you can do whatever you want with them, but to access posts data for first retrieved user
first_user_posts = users.first.posts # this will not make additional DB queries as you used includes and data is already added
We use joins to have INNER JOIN statement in the SQL
We use includes to load all posts in the memory
I have two tables users and posts and they have association of
has_many. I want to fetch details of both users and posts in a single
query.
can be done with includes like
users = User.includes(:posts).where({posts: {user_id: [1,2,3]}})
other is eager_load and preload you can use as per your requirements, for more https://blog.arkency.com/2013/12/rails4-preloading/
I'm quite new to RoR and creating a student project for a course I'm taking. I'm wanting to construct a type of query we didn't cover in the course and which I know I could do in a snap in .NET and SQL. I'm having a heck of a time though getting it implemented the Ruby way.
What I'd like to do: Display a list on a user's page of all "posts" by that user's friends.
"Posts" are found in both a questions table and in a blurbs table that users contribute to. I'd like to UNION these two into a single recordset to sort by updated_at DESC.
The table column names are not the same however, and this is my sticking point since other successful answers I've seen have hinged on column names being the same between the two.
In SQL I'd write something like (emphasis on like):
SELECT b.Blurb AS 'UserPost', b.updated_at, u.username as 'Author'
FROM Blurbs b
INNER JOIN Users u ON b.User_ID = u.ID
WHERE u.ID IN
(SELECT f.friend_id FROM Friendships f WHERE f.User_ID = [current user])
ORDER BY b.updated_at DESC
UNION
SELECT q.Question, q.updated_at, u.username
FROM Questions q
INNER JOIN Users u ON q.User_ID = u.ID
WHERE u.ID IN
(SELECT f.friend_id FROM Friendships f WHERE f.User_ID = [current user])
ORDER BY b.updated_at DESC
The User model's (applicable) relationships are:
has_many :friendships
has_many :friends, through: :friendships
has_many :questions
has_many :blurbs
And the Question and Blurb models both have belongs_to :user
In the view I'd like to display the contents of the 'UserPost' column and the 'Author'. I'm sure this is possible, I'm just too new still to ActiveRecord and how statements are formed. Happy to have some input or review any relevant links that speak to this specifically!
Final Solution
Hopefully this will assist others in the future with Ruby UNION questions. Thanks to #Plamena's input the final implementation ended up as:
def friend_posts
sql = "...the UNION statement seen above..."
ActiveRecord::Base.connection.select_all(ActiveRecord::Base.send("sanitize_sql_array",[sql, self.id, self.id] ) )
end
Currently Active Record lacks union support. You can use SQL:
sql = <<-SQL
# your sql query goes here
SELECT b.created_at ...
UNION(
SELECT q.created_at
....
)
SQL
posts = ActiveRecord::Base.connection.select_all(sql)
Then you can iterate the result:
posts.each do |post|
# post is a hash
p post['created_at']
end
Your best way to do this is to just use the power of Rails
If you want all of something belonging to a user's friend:
current_user.friends.find(id_of_friend).first.questions
This would get all of the questions from a certain friend.
Now, it seems that you have writings in multiple places (this is hard to visualise without your providing a model of how writings is connected to everywhere else). Can you provide this?
#blurbs = Blurb.includes(:user)
#blurbs.each do |blurb|
p blurb.blurb, blurb.user.username
end
Given these models:
class Farm < ActiveRecord::Base
has_and_belongs_to_many :animals
end
Class Animal < ActiveRecord::Base
has_and_belongs_to_many :farms
end
I need to search for farms that have ducks, pigs and cows but none of cats and dogs.
This kind of query doesn't work:
Animal.joins(:farms)
.where('animals.name IN ? AND animals.name NOT IN ?',
good_animal_names, bad_animal_names)
As it searches farms with ANY of the animals. I need to search farms that have ALL of the desired ones and none of the others.
I also tried with SQL with something like this:
SELECT farms.id, farms.name
FROM farms
INNER JOIN animals_farms ON animals_farms.farm_id = farms.id
INNER JOIN animals ON animals_farms.animal_id = animals.id
WHERE animals.name IN (['ducks', 'pigs', 'cows'])
AND animals.name NOT IN (['dogs', 'cats'])
GROUP BY farms.id, farms.name
HAVING COUNT(unique(animals.name)) = 3
But I'm not sure if animals.name NOT IN will really exclude farms that have none of the animals or only farms that don't have one of them. The real database is very difficult to verify the results.
Also, it should be great to be able to do the query in Active Record or Arel but any recommendation in SQL is more than welcome.
The database is Oracle, I don't have much experience with Oracle but most of the queries I use with PostgreSQL are working here.
Assuming that you have only two categories of animals -(good/bad), and you need only good animals. Can you try this?
Farm.joins(:animals).where.not(animals: {name: bad_animal_names})
Hope it helps!
Well, I think I have a possible solution in SQL. Please, comment if you can see any issue or know a better way to do it.
WITH included_animals as (SELECT farms.id, farms.name
FROM farms
INNER JOIN animals_farms ON animals_farms.farm_id = farms.id
INNER JOIN animals ON animals_farms.animal_id = animals.id
WHERE animals.name IN ('ducks', 'pigs', 'cows')
GROUP BY farms.id, farms.name
HAVING COUNT(unique(animals.name)) = 3),
not_included_animals as (SELECT farms.id, farms.name
FROM farms
INNER JOIN animals_farms ON animals_farms.farm_id = farms.id
INNER JOIN animals ON animals_farms.animal_id = animals.id
WHERE animals.name IN ('dogs', 'cats')
GROUP BY farms.id, farms.name)
SELECT id, name
FROM included_animals
MINUS --use EXCEPT if riding PostgreSQL
SELECT id, name
FROM not_included_animals
I am looking for the most Rails-ish way to retrieve a distinct list of categories for available songs on a given album.
Here it is in SQL for album_id = 1
-- Using subselects
select * from categories where id in (
select distinct category_id
from categorizations
where song_id in (select song_id from album_songs
where album_id = 1 and available = 't')
)
order by name asc;
-- Using joins
select distinct c.* from categories c
inner join categorizations cz on c.id = cz.category_id
left join album_songs a on cz.song_id = a.song_id
where a.album_id = 1 and a.available = 't'
order by c.name asc;
My working (albeit naive!) attempts to port this to ActiveRecord
## attempting to do it like subselects (although they're not really
## subselects, it executes them individually -- from what i've read
## ActiveRecord won't do subselects?)
Category.where('id IN (?)',
Categorization.select('DISTINCT category_id').where('song_id IN (?)',
Album.find(1).songs.available.map(&:song_id)
).map(&:category_id)
).order('name ASC')
## joins - although at this point it's pretty much all sql
## as i couldn't find a way to do the left join in pure AR
## i'm also duplicating my AlbumSongs.available scope -- is
## that scope reusable here? (outside the AlbumSongs model?)
Category.select('DISTINCT categories.*')
.joins(:categorizations,
'LEFT OUTER JOIN album_songs ON categorizations.song_id = album_songs.song_id')
.where('album_songs.album_id = ? and available', 1)
I am going with the final one but it seems like I might as well just write it in SQL?
Is there any way to improve this to be more Rails-ish?
Well, it would certainly help if you post your model set up. But assuming that:
* song has_many :categories, :through => categorizations
* an album does not have a huge amount of songs on it
Why not just do:
Album.includes({:songs => {:categorizations => :categories}}).find(1).songs.collect {|s| s.category}.flatten.uniq