Can I add some condition to the LEFT JOIN sql that Rails generate for the includes method? (Rails 4.2.1, postresql).
I need to get all(!) the users with preloading ( not N+1 when I will puts in a view count of comments, posts and etc) of associations, but associations need to be filtered by some conditions.
Example:
User.includes(:comments)
# => SELECT * FROM users LEFT JOIN comments ON ...
This will return all the users and preload comments if they exists.
If I will add some conditions for the "comments" association in where, then SQL doesn't return ALL the users, for example:
User.includes(:comments).where(comments: {published_at: Date.today})
# => SELECT * FROM users LEFT JOIN comments ON ... WHERE comments.published_at = ...
This will return only users, that have comments, published today.
I need to put conditions inside LEFT JOIN AND save preloading (load objects to the memory - simple left join with joins method doesn't preload associations).
SELECT * FROM users LEFT JOIN comments ON (... AND comments.published_at = ...)
Those SQL will return right what I need (all the users, and their comments, published in requested date, if they exists)! But ... I cant generate it with the Rails includes method, and `joins' doesn't preload associations.
What do you advice me? Thanks!
Rails doesn't have methods in the framework library to do what you want.
This might work, though
class User < ActiveRecord::Base
has_many :comments
has_many :recent_comments, -> { where(published_at: Date.today) }, class_name: "Comment"
end
Then query for Users and preload recent_comments
#users = User.preload(:recent_comments)
Related
In my Ruby on Rails app I have the following model:
class SlideGroup < ApplicationRecord
has_many :survey_group_lists, foreign_key: 'group_id'
has_many :surveys, through: :survey_group_lists
end
I want to find all orphaned slide groups. Orphaned slide group is slide group which is not connected to any survey. I've been trying following query but it does not return anything and I'm sure that I have orphaned records in my test database:
SlideGroup.joins(:surveys).group("slide_groups.id, surveys.id").having("count(surveys.id) = ?",0)
this generates following sql query:
SlideGroup Load (9.3ms) SELECT "slide_groups".* FROM "slide_groups" INNER JOIN "survey_group_lists" ON "survey_group_lists"."group_id" = "slide_groups"."id" INNER JOIN "surveys" ON "surveys"."id" = "survey_group_lists"."survey_id" GROUP BY slide_groups.id, surveys.id HAVING (count(surveys.id) = 0)
You're using joins, which is INNER JOIN, whereas what you need is an OUTER JOIN -
includes:
SlideGroup.includes(:surveys).group("slide_groups.id, surveys.id").having("count(surveys.id) = ?",0)
A bit cleaner query:
SlideGroup.includes(:surveys).where(surveys: { id: nil })
Finding orphan records has been explained by others.
I see problems with this approach:
There should not be any orphan in the first place
The presence of a survey.id does not guarantee the presence of a Survey
What about SurveyGroupList that are orphan?
So the proper solution would be to ensure that no orphans are left in the DB. By implementing the proper logic AND adding foreign keys with on delete cascade to the DB. You can also add dependent: :destroy option to your associations but this only works if you use #destroy on your models (not delete) and of course does not work if you delete directly via SQL.
Need advice, how to write complex query in Ruby.
Query in PHP project:
$get_trustee = db_query("SELECT t.trustee_name,t.secret_key,t.trustee_status,t.created,t.user_id,ui.image from trustees t
left join users u on u.id = t.trustees_id
left join user_info ui on ui.user_id = t.trustees_id
WHERE t.user_id='$user_id' AND trustee_status ='pending'
group by secret_key
ORDER BY t.created DESC")
My guess in Ruby:
get_trustee = Trustee.find_by_sql('SELECT t.trustee_name, t.secret_key, t.trustee_status, t.created, t.user_id, ui.image FROM trustees t
LEFT JOIN users u ON u.id = t.trustees_id
LEFT JOIN user_info ui ON ui.user_id = t.trustees_id
WHERE t.user_id = ? AND
t.trustee_status = ?
GROUP BY secret_key
ORDER BY t.created DESC',
[user_id, 'pending'])
Option 1 (Okay)
Do you mean Ruby with ActiveRecord? Are you using ActiveRecord and/or Rails? #find_by_sql is a method that exists within ActiveRecord. Also it seems like the user table isn't really needed in this query, but maybe you left something out? Either way, I'll included it in my examples. This query would work if you haven't set up your relationships right:
users_trustees = Trustee.
select('trustees.*, ui.image').
joins('LEFT OUTER JOIN users u ON u.id = trustees.trustees_id').
joins('LEFT OUTER JOIN user_info ui ON ui.user_id = t.trustees_id').
where(user_id: user_id, trustee_status: 'pending').
order('t.created DESC')
Also, be aware of a few things with this solution:
I have not found a super elegant way to get the columns from the join tables out of the ActiveRecord objects that get returned. You can access them by users_trustees.each { |u| u['image'] }
This query isn't really THAT complex and ActiveRecord relationships make it much easier to understand and maintain.
I'm assuming you're using a legacy database and that's why your columns are named this way. If I'm wrong and you created these tables for this app, then your life would be much easier (and conventional) with your primary keys being called id and your timestamps being called created_at and updated_at.
Option 2 (Better)
If you set up your ActiveRecord relationships and classes properly, then this query is much easier:
class Trustee < ActiveRecord::Base
self.primary_key = 'trustees_id' # wouldn't be needed if the column was id
has_one :user
has_one :user_info
end
class User < ActiveRecord::Base
belongs_to :trustee, foreign_key: 'trustees_id' # relationship can also go the other way
end
class UserInfo < ActiveRecord::Base
self.table_name = 'user_info'
belongs_to :trustee
end
Your "query" can now be ActiveRecord goodness if performance isn't paramount. The Ruby convention is readability first, reorganizing code later if stuff starts to scale.
Let's say you want to get a trustee's image:
trustee = Trustee.where(trustees_id: 5).first
if trustee
image = trustee.user_info.image
..
end
Or if you want to get all trustee's images:
Trustee.all.collect { |t| t.user_info.try(:image) } # using a #try in case user_info is nil
Option 3 (Best)
It seems like trustee is just a special-case user of some sort. You can use STI if you don't mind restructuring you tables to simplify even further.
This is probably outside of the scope of this question so I'll just link you to the docs on this: http://api.rubyonrails.org/classes/ActiveRecord/Base.html see "Single Table Inheritance". Also see the article that they link to from Martin Fowler (http://www.martinfowler.com/eaaCatalog/singleTableInheritance.html)
Resources
http://guides.rubyonrails.org/association_basics.html
http://guides.rubyonrails.org/active_record_querying.html
Yes, find_by_sql will work, you can try this also:
Trustee.connection.execute('...')
or for generic queries:
ActiveRecord::Base.connection.execute('...')
I don't want to use join
I want to manually compare any field with other table field
for example
SELECT u.user_id, t.task_id
FROM tasks t, users u
WHERE u.user_id = t.user_id
how can i write this query in Rails ??
Assuming you have associations in your models, you can simply do as follow
User.joins(:tasks).select('users.user_id, tasks.task_id')
you can also do as follow
User.includes(:tasks).where("user.id =tasks.user_id")
includes will do eager loading check the example below or read eager loading at here
users = User.limit(10)
users.each do |user|
puts user.address.postcode
end
This will run 11 queries, it is called N+1 query problem(first you query to get all the rows then you query on each row again to do something). with includes Active Record ensures that all of the specified associations are loaded using the minimum possible number of queries.
Now when you do;
users = User.includes(:address).limit(10)
user.each do |user|
puts user.address.postcode
end
It will generate just 2 queries as follow
SELECT * FROM users LIMIT 10
SELECT addresses.* FROM addresses
WHERE (addresses.user_id IN (1,2,3,4,5,6,7,8,9,10))
Plus if you don't have associations then read below;
you should be have to look at http://guides.rubyonrails.org/association_basics.html
Assuming your are trying to do inner join, by default in rails when we associate two models and then query on them then we are doing inner join on those tables.
You have to create associations between the models example is given below
class User
has_many :reservations
...# your code
end
And in reservations
class Reservations
belongs_to :user
... #your code
end
Now when you do
User.joins(:reservations)
the generated query would look like as follow
"SELECT `users`.* FROM `users` INNER JOIN `reservations` ON `reservations`.`user_id` = `users`.`id`"
you can check the query by doing User.joins(:reservations).to_sql in terminal
Hopefully it would answer your question
User.find_by_sql("YOUR SQL QUERY HERE")
You can use as follows..
User.includes(:tasks).where("user.id =tasks.user_id").order(:user.id)
The problem is that when a Restaurant does not have any MenuItems that match the condition, ActiveRecord says it can't find the Restaurant. Here's the relevant code:
class Restaurant < ActiveRecord::Base
has_many :menu_items, dependent: :destroy
has_many :meals, through: :menu_items
def self.with_meals_of_the_week
includes({menu_items: :meal}).where(:'menu_items.date' => Time.now.beginning_of_week..Time.now.end_of_week)
end
end
And the sql code generated:
Restaurant Load (0.0ms)←[0m ←[1mSELECT DISTINCT "restaurants".id FROM "restaurants"
LEFT OUTER JOIN "menu_items" ON "menu_items"."restaurant_id" = "restaurants"."id"
LEFT OUTER JOIN "meals" ON "meals"."id" = "menu_items"."meal_id" WHERE
"restaurants"."id" = ? AND ("menu_items"."date" BETWEEN '2012-10-14 23:00:00.000000'
AND '2012-10-21 22:59:59.999999') LIMIT 1←[0m [["id", "1"]]
However, according to this part of the Rails Guides, this shouldn't be happening:
Post.includes(:comments).where("comments.visible", true)
If, in the case of this includes query, there were no comments for any posts, all the posts would still be loaded.
The SQL generated is a correct translation of your query. But look at it,
just at the SQL level (i shortened it a bit):
SELECT *
FROM
"restaurants"
LEFT OUTER JOIN
"menu_items" ON "menu_items"."restaurant_id" = "restaurants"."id"
LEFT OUTER JOIN
"meals" ON "meals"."id" = "menu_items"."meal_id"
WHERE
"restaurants"."id" = ?
AND
("menu_items"."date" BETWEEN '2012-10-14' AND '2012-10-21')
the left outer joins do the work you expect them to do: restaurants
are combined with menu_items and meals; if there is no menu_item to
go with a restaurant, the restaurant is still kept in the result, with
all the missing pieces (menu_items.id, menu_items.date, ...) filled in with NULL
now look aht the second part of the where: the BETWEEN operator demands,
that menu_items.date is not null! and this
is where you filter out all the restaurants without meals.
so we need to change the query in a way that makes having null-dates ok.
going back to ruby, you can write:
def self.with_meals_of_the_week
includes({menu_items: :meal})
.where('menu_items.date is NULL or menu_items.date between ? and ?',
Time.now.beginning_of_week,
Time.now.end_of_week
)
end
The resulting SQL is now
.... WHERE (menu_items.date is NULL or menu_items.date between '2012-10-21' and '2012-10-28')
and the restaurants without meals stay in.
As it is said in Rails Guide, all Posts in your query will be returned only if you will not use "where" clause with "includes", cause using "where" clause generates OUTER JOIN request to DB with WHERE by right outer table so DB will return nothing.
Such implementation is very helpful when you need some objects (all, or some of them - using where by base model) and if there are related models just get all of them, but if not - ok just get list of base models.
On other hand if you trying to use conditions on including tables then in most cases you want to select objects only with this conditions it means you want to select Restaurants only which has meals_items.
So in your case, if you still want to use only 2 queries (and not N+1) I would probably do something like this:
class Restaurant < ActiveRecord::Base
has_many :menu_items, dependent: :destroy
has_many :meals, through: :menu_items
cattr_accessor :meals_of_the_week
def self.with_meals_of_the_week
restaurants = Restaurant.all
meals_of_the_week = {}
MenuItems.includes(:meal).where(date: Time.now.beginning_of_week..Time.now.end_of_week, restaurant_id => restaurants).each do |menu_item|
meals_of_the_week[menu_item.restaurant_id] = menu_item
end
restaurants.each { |r| r.meals_of_the_week = meals_of_the_week[r.id] }
restaurants
end
end
Update: Rails 4 will raise Deprecation warning when you simply try to do conditions on models
Sorry for possible typo.
I think there is some misunderstanding of this
If there was no where condition, this would generate the normal set of two queries.
If, in the case of this includes query, there were no comments for any
posts, all the posts would still be loaded. By using joins (an INNER
JOIN), the join conditions must match, otherwise no records will be
returned.
[from guides]
I think this statements doesn't refer to the example Post.includes(:comments).where("comments.visible", true)
but refer to one without where statement Post.includes(:comments)
So all work right! This is the way LEFT OUTER JOIN work.
So... you wrote: "If, in the case of this includes query, there were no comments for any posts, all the posts would still be loaded." Ok! But this is true ONLY when there is NO where clause! You missed the context of the phrase.
I have two models in my rails app with a has many and belongs to association.
Category has many items and Item belongs to category.
These models are associated in the normal way through a category_id column in the Item model.
I'm looking for a quick way of finding all elements in the database with broken associations.
i.e. find all categories that exist with no associated items and items that exist with no associated category.
For example, if I have an item with a category_id of 7, but the category with id 7 has been deleted then this would be considered broken.
For your example, to find items with category_id's for categories which don't exist any more:
Item.where('NOT EXISTS (SELECT * FROM categories where category.id = item.category_id)')
You might want to look at this as well:
A rake task to track down missing database indexes (not foreign keys though, but indexes): https://github.com/eladmeidar/rails_indexes
A very effective way is using find_by_sql to let the database do the heavy lifting:
uncategorized_items = Item.find_by_sql("select * from items where category_id IS NULL")
Another way is a named scope:
class Item < ActiveRecord::Base
scope :uncategorized, where(:category_id => nil) # rails 3
# or...
named_scope :uncategorized, :conditions => 'category_id IS NULL'
end
These are just a couple of ideas. I assume that once you've found these broken associations you plan to fix them, right? You might want to use validates_associated in both models if it's important to you that this not happen again.
You can use find_by_sql and a left outer join to find all the items in one table but not another. Here, I use a downloads table and an image_files table (I've only included the SQL):
SELECT d.*, d.image_file_id
from downloads as d
LEFT OUTER JOIN image_files as i
ON i.id = d.image_file_id
WHERE d.image_file_id IS NULL