What is the "rails way" to efficiently grab all rows of a parent table along with a count of the number of children each row has?
I don't want to use counter_cache as I want to run these counts based on some time conditions.
The cliche blog example:
Table of articles. Each article has 0 or more comments.
I want to be able to pull how many comments each article has in the past hour, day, week.
However, ideally I don't want to iterate over the list and make separate sql calls for each article nor do I want to use :include to prefetch all of the data and process it on the app server.
I want to run one SQL statement and get one result set with all the info.
I know I can hard code out the full SQL, and maybe could use a .find and just set the :joins, :group, and :conditions parameters... BUT I am wondering if there is a "better" way... aka "The Rails Way"
This activerecord call should do what you want:
Article.find(:all, :select => 'articles.*, count(posts.id) as post_count',
:joins => 'left outer join posts on posts.article_id = articles.id',
:group => 'articles.id'
)
This will return a list of article objects, each of which has the method post_count on it that contains the number of posts on the article as a string.
The method executes sql similar to the following:
SELECT articles.*, count(posts.id) AS post_count
FROM `articles`
LEFT OUTER JOIN posts ON posts.article_id = articles.id
GROUP BY articles.id
If you're curious, this is a sample of the MySQL results you might see from running such a query:
+----+----------------+------------+
| id | text | post_count |
+----+----------------+------------+
| 1 | TEXT TEXT TEXT | 1 |
| 2 | TEXT TEXT TEXT | 3 |
| 3 | TEXT TEXT TEXT | 0 |
+----+----------------+------------+
Rails 3 Version
For Rails 3, you'd be looking at something like this:
Article.select("articles.*, count(comments.id) AS comments_count")
.joins("LEFT OUTER JOIN comments ON comments.article_id = articles.id")
.group("articles.id")
Thanks to Gdeglin for the Rails 2 version.
Rails 5 Version
Since Rails 5 there is left_outer_joins so you can simplify to:
Article.select("articles.*, count(comments.id) AS comments_count")
.left_outer_joins(:comments)
.group("articles.id")
And because you were asking about the Rails Way: There isn't a way to simplify/railsify this more with ActiveRecord.
From a SQL perspective, this looks trivial - Just write up a new query.
From a Rails perspective, The values you mention are computed values. So if you use find_by_sql, the Model class would not know about the computed fields and hence would return the computed values as strings even if you manage to translate the query into Rails speak. See linked question below.
The general drift (from the responses I got to that question) was to have a separate class be responsible for the rollup / computing the desired values.
How to get rails to return SUM(columnName) attributes with right datatype instead of a string?
A simple way that I used to solve this problem was
In my model I did:
class Article < ActiveRecord::Base
has_many :posts
def count_posts
Post.where(:article_id => self.id).count
end
end
Now, you can use for example:
Articles.first.count_posts
Im not sure if it can be more efficient way, But its a solution and in my opinion more elegant than the others.
I made this work this way:
def show
section = Section.find(params[:id])
students = Student.where( :section_id => section.id ).count
render json: {status: 'SUCCESS', section: students},status: :ok
end
In this I had 2 models Section and Student. So I have to count the number of students who matches a particular id of section.
Related
Forgive me if this has been asked before, I had a hard time thinking of good search queries.
Lets say I have 2 models, Posts and Tags. Posts have many tags through a pivot model, PostTags.
What I'd like to do is group posts that have the exact same combination of tags. I know how to group posts that have any of the same tags, but I've been having a harder time with this.
For example, if I have a post with and ID of 1, and the post has two Tags- one with an ID of 5, another with an ID of 7. I would have 2 PostTags, one with a post_id of 1, and a tag_id of 5, and then another with a post_id of 1 and a tag_id of 7. I have another Post with an id of 3, and it also has 2 PostTags - one with a post_id of 3 and a tag_id of 5, and another with a post_id of 3 and a tag_id of 7. I'd like to group these together so that I can get a count of how many posts have both of these tags, and no others.
Thanks, and I hope I was able to explain this properly.
I think you could probably do something like this in a nested query:
SELECT tag_ids,
string_agg(post_id, ',')
FROM (
SELECT post_id,
string_agg(tag_id, ',') as tag_ids
FROM post_tags
GROUP BY post_id)
GROUP BY tag_ids;
Explanation:
First in the inside query, you concatenate tag_ids grouped by post_id, so you can get the combination of tags for each post.
Then in the outside query, you concatenate post_ids by the combination of tag_ids, so you get all the post_ids for each tag combination.
This might not be the end yet, you could further process the post ids, or modify the query to fetch whatever data you need.
Hope this help!
Hope the Model relationships are sets properly.
# Post Model
class Post
has_many :post_tags
has_many :tags, through: :post_tags
end
# Fetch Tags to match with posts collection
tag_ids = []
# Query to fetch posts
Post.joins(:tags).where(tags: { id: tag_ids }).
group("posts.id").having("count(posts.id) >= ?", tag_ids.size)
First line ensure that only posts having tags included in the specified
tag list are fetched
Second line ensure that posts are having both of the tags.
If you want to match for exact tags (no other tags should present), then
change the condition to = instead of >=
Happy Hacking!
I want to show a time link mixing comments and post so I have this objects
#posts = Post::all()
#comments = Comment::all()
If I do this
#post.each ...
... end
#comments.each ...
... end
I will get first posts and after this, the comments. But I want a timeline, how i can create this?
I need to combine both object to create just one ordered list, example:
In post:
id | name | date
1 | post1 | 2015-01-01
2 | post2 | 2013-01-01
In comments:
id | name | date
1 | comment1 | 2014-01-01
2 | comment2 | 2016-01-01
if I do this
post.each ...
comments.each ...
the result will that:
-post1
-post2
-comment1
-comment2
But i need order by date to get
-post2
-comment1
-post1
-comment2
Thanks, and sorry for my ugly english.
Posts and comments are different models (and different tables), so we can't write SQL to get sorted collection, with pagination etc.
Usually I use next approach when I need mixed timeline.
I have TimelineItem model with source_id, source_type and timeline_at fields.
class TimelineItem < ApplicationRecord
belongs_to :source, polymorphic: true
end
Then I add in models logic to create timeline_item instance when needed:
has_many :timeline_items, as: :source
after_create :add_to_timeline
def add_to_timeline
timeline_items.create timeline_at: created_at
end
Then search and output as simple as
TimelineItem.includes(:source).order(:timeline_at).each { |t| pp t.source }
Solution#1
You can achieve this using UNION query.
sql= 'SELECT id, name, date FROM posts UNION ALL SELECT id, name, date FROM comments ORDER BY date'
ActiveRecord::Base.connection.execute(sql)
but union query will only work when you have same column names in both tables.
Solution#2 Add ordering logic in your view.
If you are simply displaying these records on html page then let them load on page without any specific order i.e.(first posts and then comments). Write javascript code to sort DOM elements which will run after page load.
for ref: Jquery - sort DIV's by innerHTML of children
Solution#3 Refactor your DB schema and put posts and comments in same database table. Then you will be able to query on single table.
Something like this,
class Text < ActiveRecord::Base
end
class Post < Text
end
class Comment < Text
end
Query will be Text.order(:date)
Refactoring your DB schema is too much to solve this problem. Do it if it makes sense for your application.
I want to show a time link mixing comments and post so I have this objects
#posts = Post::all()
#comments = Comment::all()
If I do this
#post.each ...
... end
#comments.each ...
... end
I will get first posts and after this, the comments. But I want a timeline, how i can create this?
I need to combine both object to create just one ordered list, example:
In post:
id | name | date
1 | post1 | 2015-01-01
2 | post2 | 2013-01-01
In comments:
id | name | date
1 | comment1 | 2014-01-01
2 | comment2 | 2016-01-01
if I do this
post.each ...
comments.each ...
the result will that:
-post1
-post2
-comment1
-comment2
But i need order by date to get
-post2
-comment1
-post1
-comment2
Thanks, and sorry for my ugly english.
Posts and comments are different models (and different tables), so we can't write SQL to get sorted collection, with pagination etc.
Usually I use next approach when I need mixed timeline.
I have TimelineItem model with source_id, source_type and timeline_at fields.
class TimelineItem < ApplicationRecord
belongs_to :source, polymorphic: true
end
Then I add in models logic to create timeline_item instance when needed:
has_many :timeline_items, as: :source
after_create :add_to_timeline
def add_to_timeline
timeline_items.create timeline_at: created_at
end
Then search and output as simple as
TimelineItem.includes(:source).order(:timeline_at).each { |t| pp t.source }
Solution#1
You can achieve this using UNION query.
sql= 'SELECT id, name, date FROM posts UNION ALL SELECT id, name, date FROM comments ORDER BY date'
ActiveRecord::Base.connection.execute(sql)
but union query will only work when you have same column names in both tables.
Solution#2 Add ordering logic in your view.
If you are simply displaying these records on html page then let them load on page without any specific order i.e.(first posts and then comments). Write javascript code to sort DOM elements which will run after page load.
for ref: Jquery - sort DIV's by innerHTML of children
Solution#3 Refactor your DB schema and put posts and comments in same database table. Then you will be able to query on single table.
Something like this,
class Text < ActiveRecord::Base
end
class Post < Text
end
class Comment < Text
end
Query will be Text.order(:date)
Refactoring your DB schema is too much to solve this problem. Do it if it makes sense for your application.
I'm trying to implement search over tags as part of a Texticle search. Since texticle doesn't search over multiple tables from the same model, I ended up creating a new model called PostSearch, following Texticle's suggestion about System-Wide Searching
class PostSearch < ActiveRecord::Base
# We want to reference various models
belongs_to :searchable, :polymorphic => true
# Wish we could eliminate n + 1 query problems,
# but we can't include polymorphic models when
# using scopes to search in Rails 3
# default_scope :include => :searchable
# Search.new('query') to search for 'query'
# across searchable models
def self.new(query)
debugger
query = query.to_s
return [] if query.empty?
self.search(query).map!(&:searchable)
#self.search(query) <-- this works, not sure why I shouldn't use it.
end
# Search records are never modified
def readonly?; true; end
# Our view doesn't have primary keys, so we need
# to be explicit about how to tell different search
# results apart; without this, we can't use :include
# to avoid n + 1 query problems
def hash
id.hash
end
def eql?(result)
id == result.id
end
end
In my Postgres DB I created a view like this:
CREATE VIEW post_searches AS
SELECT posts.id, posts.name, string_agg(tags.name, ', ') AS tags
FROM posts
LEFT JOIN taggings ON taggings.taggable_id = posts.id
LEFT JOIN tags ON taggings.tag_id = tags.id
GROUP BY posts.id;
This allows me to get posts like this:
SELECT * FROM post_searches
id | name | tags
1 Intro introduction, funny, nice
So it seems like that should all be fine. Unfortunately calling
PostSearch.new("funny") returns [nil] (NOT []). Looking through the Texticle source code, it seems like this line in the PostSearch.new
self.search(query).map!(&:searchable)
maps the fields using some sort of searchable_columns method and does it ?incorrectly? and results in a nil.
On a different note, the tags field doesn't get searched in the texticle SQL query unless I cast it from a text type to a varchar type.
So, in summary:
Why does the object get mapped to nil when it is found?
AND
Why does texticle ignore my tags field unless it is varchar?
Texticle maps objects to nil instead of nothing so that you can check for nil? - it's a safeguard against erroring out checking against non-existent items. It might be worth asking tenderlove himself as to exactly why he did it that way.
I'm not completely positive as to why Texticle ignores non-varchars, but it looks like it's a performance safeguard so that Postgres does not do full table scans (under the section Creating Indexes for Super Speed):
You will need to add an index for every text/string column you query against, or else Postgresql will revert to a full table scan instead of using the indexes.
Is this doable?
I have the following scope:
class Thing < ActiveRecord::Base
scope :with_tag, lambda{ |tag| joins(:tags).where('tags.name = ?', tag.name)
.group('things.id') }
def withtag_search(tags)
tags.inject(scoped) do |tagged_things, tag|
tagged_things.with_tag(tag)
end
end
I get a result if there's a single tag in the array of tags passed in with Thing.withtag_search(array_of_tags) but if I pass multiple tags in that array I get an empty relation as the result. In case it helps:
Thing.withtag_search(["test_tag_1", "test_tag_2"])
SELECT "things".*
FROM "things"
INNER JOIN "things_tags" ON "things_tags"."thing_id" = "things"."id"
INNER JOIN "tags" ON "tags"."id" = "things_tags"."tag_id"
WHERE (tags.name = 'test_tag_1') AND (tags.name = 'test_tag_2')
GROUP BY things.id
=> [] # class is ActiveRecord::Relation
whereas
Thing.withtag_search(["test_tag_1"])
SELECT "things".*
FROM "things"
INNER JOIN "things_tags" ON "things_tags"."thing_id" = "things"."id"
INNER JOIN "tags" ON "tags"."id" = "things_tags"."tag_id"
WHERE (tags.name = 'test_tag_1')
GROUP BY things.id
=> [<Thing id:1, ... >, <Thing id:2, ... >] # Relation including correctly all
# Things with that tag
I want to be able to chain these relations together so that (among other reasons) I can use the Kaminari gem for pagination which only works on relations not arrays - so I need a scope to be returned.
I also ran into this problem. The problem is not Rails, the problems is definitely MySQL:
Your SQL will create following temporary JOIN-table (only neccesary fields are shown):
+-----------+-------------+---------+------------+
| things.id | things.name | tags.id | tags.name |
+-----------+-------------+---------+------------+
| 1 | ... | 1 | test_tag_1 |
+-----------+-------------+---------+------------+
| 1 | ... | 2 | test_tag_2 |
+-----------+-------------+---------+------------+
So instead joining all Tags to one specific Thing, it generates one row for each Tag-Thing combination (If you don't believe, just run COUNT(*) on this SQL statement).
The problem is that you query criteria looks like this: WHERE (tags.name = 'test_tag_1') AND (tags.name = 'test_tag_2') which will be checked against each of this rows, and never will be true. It's not possible for tags.name to equal both test_tag_1 and test_tag_2 at the same time!
The standard SQL solution is to use the SQL statement INTERSECT... but unfortunately not with MySQL.
The best solution is to run Thing.withtag_search for each of your tags, collect the returning objects, and select only objects which are included in each of the results, like so:
%w[test_tag_1 test_tag_2].collect do |tag|
Thing.withtag_search(tag)
end.inject(&:&)
If you want to get this as an ActiveRecord relation you can probably do this like so:
ids = %w[test_tag_1 test_tag_2].collect do |tag|
Thing.withtag_search(tag).collect(&:id)
end.inject(&:&)
Things.where(:id => ids)
The other solution (which I'm using) is to cache the tags in the Thing table, and do MySQL boolean search on it. I will give you more details on this solution if you want.
Anyways I hope this will help you. :)
This is rather complicated at a glance, but based on your SQL, you want:
WHERE (tags.name IN ( 'test_tag_1', 'test_tag_2'))
I haven't dealt much with Rails 3, but if you can adjust your JOIN appropriately, this should fix your issue. Have you tried a solution akin to:
joins(:tag).where('tags.name IN (?), tags.map { |tag| tag.name })
This way, you will JOIN the way you are expecting (UNION instead of INTERSECTION). I hope this is a helpful way of thinking about this problem.
Don't seem to be able to find a solution to this problem. So, instead of using Kaminari and rolling my own tagging I've switched to Acts-as-taggable-on and will-paginate