active_model_serializer returning more than one result - ruby-on-rails

I'm trying to return the another_id for a related record. I would just add a has_many and belongs_to relation for each project, but I need to have the user id in order to return the correct results. However, with the code I have below, it returns all of the possible another_ids for the current_user.
If I enter this into psql, it works fine:
WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = 595
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = 2;
another_id
-----------
52
(1 row)
But if I run the code from the serializer, it returns: [52, 51]:
class ProjectSerializer < ActiveModel::Serializer
attributes :id, :another_id
def another_id__sql
"(WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = projects.thing_id
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = #{options[:current_user].id})"
end
end
class API::V1::ProjectsController < API::APIController
def index
render json: Project.all
end
private
def default_serializer_options
{ current_user: #current_user }
end
end
From what I can gather, I'm not understanding how active_model_serializers serializes more than one record.
I'm using rails 4.2.3 and active_model_serializers 0.8.3. I'm afraid I can't change the schema. Also, it probably doesn't matter, but this is the API for an Ember app.
Thanks in advance. I'm a bit embarrassed that I'm having trouble with this.
Edit:
I should probably mention that this is what my project model looks like:
class Project < ActiveRecord::Base
belongs_to :thing
has_many :user_thing, through: :thing
attr_accessor :another_id
def set_another_id(user)
connection = ActiveRecord::Base.connection
result = connection.execute("(WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = #{thing_id}
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = #{user.id})")
#another_id = result[0]["another_id"].to_i
end
end
And this is the show action in the controller:
def show
#project = Project.find(params[:id])
#project.set_another_id(#current_user)
render json: #project
end
The show action does return the correct id.
Also, I know what I have is incorrect. The thing is that I can't just use the activerecord associations, because it depends on that session's current user.
Edit 2:
I thought I was able to get it to work if I just rendered it using: render json: Project.all.to_json, and got rid of the another_id__sql method in the serializer. That does work if it does have another_id. However, if that's nil, I get the error: "NoMethodError in API::V1::ProjectsController#index undefined method []' for nil:NilClass". It looks like this is a possible bug in 0.8, so I'll either have to ask another Stack Overflow question, or I'll have to see if I can upgrade theactive_model_serializers` gem. I was wrong! See my answer below.

All the DB logic belongs in your model, not in your serializer. The serializers simply state what is supposed to be exposed, but it should not be responsible for computing it.
So here, I'd advise to make this another_id a method on your model, which won't solve your issue (as it seems it is more of an SQL issue than anything else), but it will make it so that you don't have a problem with AMS anymore.

Serializers take a record and return a serialized representation suitable for JSON or XML encoding.
They are meant as an alternative to littering your controllers with this:
render json: #users, except: [:foo, :bar, :baz], include: [..........]
And the mental flatulence that is jbuilder.
SQL queries and scopes instead belong in your models.
You can set the serializer by using the each_serializer option. But in this case it will not do you much good the objects you serialize must at least implement the base methods for a serializable model.
So you need to re-write your query so that it returns a collection or array of records.
see:
http://apidock.com/rails/ActiveRecord/Base/find_by_sql/class
https://github.com/rails-api/active_model_serializers/blob/master/lib/active_model_serializers/model.rb
https://github.com/rails-api/active_model_serializers

Got it! It appears that I needed one more method in the serializer:
project_serializer.rb
def another_id
object.another_id
end
def another_id__sql
# sql from before
end
I'm not 100% sure why this works, but I had noticed that, if I left out the another_id__sql, I would get the error column.projects.another_id does not exist. So, I'm guessing that the another_id__sql is called when it's returning an array, but uses the another_id method when the object is a single project record.
I'd still love to hear better ways to do this!

Related

How to use find_by_sql properly?

I have the following model
class Backup < ActiveRecord::Base
belongs_to :component
belongs_to :backup_medium
def self.search(value)
join_tables = "backups, components, backup_media"
joins = "backups.backup_medium_id = backup_media.id and components.id = backups.component_id"
c = Backup.find_by_sql "select * from #{join_tables} where components.name like '%#{value}%' and #{joins}"
b = Backup.find_by_sql "select * from #{join_tables} where backup_media.name like '%#{value}%' and #{joins}"
c.count > 0 ? c : b
end
end
In pry, when I run Backup.all.class, I get
=> Backup::ActiveRecord_Relation
but when I run Backup.search('xxx').class, I get
=> Array
Since the search should return a subset of all, I think I need to return an Active Record_Relation. What am I missing?
From the documentation:
Executes a custom SQL query against your database and returns all the
results. The results will be returned as an array with columns
requested encapsulated as attributes of the model you call this method
from. If you call Product.find_by_sql then the results will be
returned in a Product object with the attributes you specified in the
SQL query.
So you will get an array of Backup instances.
Note that you probably should not do it this way. Using string interpolation in a query opens you up to SQL injection attacks and gains you nothing. Also, you can get quite a bit more flexibility using ActiveRecord scopes for this.
def self.my_includes
includes(:components, :backup_media)
end
def self.by_component_name(name)
media_includes.where("components.name like ?", "'%#{name}%'")
end
def self.by_media_name(name)
media_includes.where("backup_media.name like ?", "'%#{value}%'")
end
def self.search(name)
by_component(name).any? ? by_component_name : by_media_name
end
You can then call
Backup.search(name)
as well as
Backup.by_component_name(name)
or
Backup.by_media_name(name)
find_by_sql returns an array of objects, not a Relation. If you want to return relation for consistency try to rewrite your search to use ActiveRecord api:
def self.search(value)
query = Backup.includes(:component, :backup_medium)
by_component_name = query.where("components.name like ?", "'%#{value}%'")
by_media_name = query.where("backup_media.name like ?", "'%#{value}%'")
by_component_name.any? ? by_component_name : by_media_name
end
or, if you still want to use sql, you can try to fetch record ids and then make a second query:
def self.search(value)
# ...
c = Backup.find_by_sql "select id from #{join_tables} where components.name like '%#{value}%' and #{joins}"
b = Backup.find_by_sql "select id from #{join_tables} where backup_media.name like '%#{value}%' and #{joins}"
ids = c.count > 0 ? c : b
Backup.where(id: ids)
end
So I am unable to get the syntax right for the media_includes, but inspired by your solution I have succeeded by using joins.
I created a small demo project which just shows the code related to search. You can take a look at https://github.com/pamh09/rails-search-demo. If you want to collaborate on a solution, I think this would be more efficient than trying to paste all the code here. That said, I do have a working solution if you'd rather not bother. But I would like to see what the right syntax is.
Below is the model code. It's very possible that I just have some kind of syntactic mismatch since I am not very familiar with how rails does its database magic (obviously).
class Backup < ApplicationRecord
belongs_to :component
belongs_to :backup_medium
#---- code below does not work ---
# in pry
# pry(Backup):1> by_media('bak').any?
# (0.0ms) SELECT COUNT(*) FROM "backups" WHERE (backup_media = 'bak')
# ActiveRecord::StatementInvalid: SQLite3::SQLException: no such column: backup_media.name: SELECT COUNT(*) FROM "backups" WHERE (backup_media.name = 'bak')
def self.my_includes
includes(:component, :backup_medium)
end
def self.by_component(name)
my_includes.where("components.name = ?", name)
end
def self.by_media(name)
my_includes.where("backup_media.name = ?", name)
end
def self.search_by(name)
by_component(name).any? ? by_component_name : by_media_name
end
# ----- code below works ... call search('string') -----
# I was unable to get the like query to work without using #{name}
def self.by_component_like(name)
# Note: joins (singular).where (plural.column ...)
joins(:component).where("components.name like '%#{name}%'")
end
def self.by_media_like(name)
joins(:backup_medium).where("backup_media.name like '%#{name}%'")
end
def self.search(name)
by_component_like(name).any? ? by_component_like(name) : by_media_like(name)
end
end
And, as noted in the code. I could not figure you how to use the ? with LIKE as the query would come in as LIKE '%'xxx'%' instead of '%xxx%'.

Returning a list with ActiveRecord query interface

My Program class has_many Patients
and Patient class has a field like salary
And the json I pass to JBuilder with respond_to :json , respond_with(#org) is like this:
#org = Org.includes(programs: [{hospitals: :nurses}, :patients]).find(params[:id])
Well now if my database has 200 programs that meed the org_id = params[:id] condition it will return them all. BUT this is Not what I want. I want to tell it to return "5" programs and those "five" programs that the "salary" field of their :'patients" table is the highest.
How can I implement this limitation in active record query?
Something like
Program.includes([{hospitals: :nurses}, :patients]).where("organization_id = ?", params[:id]).order("patients.salary desc").limit(5)
Or you can do:
#program_ids = Program.select("id").includes([{hospitals: :nurses}, :patients]).where("organization_id = ?", params[:id]).order("patients.salary desc").limit(5)
#org = Org.includes(programs: [{hospitals: :nurses}, :patients]).where("programs.id = ?", program_ids.collect(&:id)).find(params[:id])
You can also refactor with a subquery in one step (still 2 queries), by checking this question:
subqueries in activerecord
I'm not sure if you can get that information in just one single query

Rails active record query

How would i do a query like this.
i have
#model = Model.near([latitude, longitude], 6.8)
Now i want to filter another model, which is associated with the one above.
(help me with getting the right way to do this)
model2 = Model2.where("model_id == :one_of_the_models_filtered_above", {:one_of_the_models_filtered_above => only_from_the_models_filtered_above})
the model.rb would be like this
has_many :model2s
the model2.rb
belongs_to :model
Right now it is like this (after #model = Model.near([latitude, longitude], 6.8)
model2s =[]
models.each do |model|
model.model2s.each do |model2|
model2.push(model2)
end
end
I want to accomplish the same thing, but with an active record query instead
i think i found something, why does this fail
Model2.where("model.distance_from([:latitude,:longitude]) < :dist", {:latitude => latitude, :longitude => longitude, :dist => 6.8})
this query throws this error
SQLite3::SQLException: near "(": syntax error: SELECT "tags".* FROM "tags" WHERE (model.distance_from([43.45101666666667,-80.49773333333333]) < 6.8)
, why
use includes. It will eager-load associated models (only two SQL queries instead of N+1).
#models = Model.near( [latitude, longitude], 6.8 ).includes( :model2s )
so when you will do #models.first.model2s, associated model2s will already be loaded (see RoR guides for more info).
If you want to get an array of all model2s belonging to your collection of models, you can do :
#models.collect( &:model2s )
# add .flatten at the end of the chain if you want a one level deep array
# add .uniq at the end of the chain if you don't want duplicates
collect (also called map) will gather in an array the result of any block passed to each of the caller's elements (this does exactly the same as your code, see Enumerable's doc for more info). The & before the symbol converts it into a Proc passed to each element of the collection, so this is the same as writing
#models.collect {|model| model.model2s }
one more thing : #mu is right, seems SQLite does not know about your distance_from stored procedure. As i suspect this is a GIS related question, you may ask about this particular issue on gis.stackexchange.com

ActiveRecord Query Union

I've written a couple of complex queries (at least to me) with Ruby on Rail's query interface:
watched_news_posts = Post.joins(:news => :watched).where(:watched => {:user_id => id})
watched_topic_posts = Post.joins(:post_topic_relationships => {:topic => :watched}).where(:watched => {:user_id => id})
Both of these queries work fine by themselves. Both return Post objects. I would like to combine these posts into a single ActiveRelation. Since there could be hundreds of thousands of posts at some point, this needs to be done at the database level. If it were a MySQL query, I could simply user the UNION operator. Does anybody know if I can do something similar with RoR's query interface?
Here's a quick little module I wrote that allows you to UNION multiple scopes. It also returns the results as an instance of ActiveRecord::Relation.
module ActiveRecord::UnionScope
def self.included(base)
base.send :extend, ClassMethods
end
module ClassMethods
def union_scope(*scopes)
id_column = "#{table_name}.id"
sub_query = scopes.map { |s| s.select(id_column).to_sql }.join(" UNION ")
where "#{id_column} IN (#{sub_query})"
end
end
end
Here's the gist: https://gist.github.com/tlowrimore/5162327
Edit:
As requested, here's an example of how UnionScope works:
class Property < ActiveRecord::Base
include ActiveRecord::UnionScope
# some silly, contrived scopes
scope :active_nearby, -> { where(active: true).where('distance <= 25') }
scope :inactive_distant, -> { where(active: false).where('distance >= 200') }
# A union of the aforementioned scopes
scope :active_near_and_inactive_distant, -> { union_scope(active_nearby, inactive_distant) }
end
I also have encountered this problem, and now my go-to strategy is to generate SQL (by hand or using to_sql on an existing scope) and then stick it in the from clause. I can't guarantee it's any more efficient than your accepted method, but it's relatively easy on the eyes and gives you a normal ARel object back.
watched_news_posts = Post.joins(:news => :watched).where(:watched => {:user_id => id})
watched_topic_posts = Post.joins(:post_topic_relationships => {:topic => :watched}).where(:watched => {:user_id => id})
Post.from("(#{watched_news_posts.to_sql} UNION #{watched_topic_posts.to_sql}) AS posts")
You can do this with two different models as well, but you need to make sure they both "look the same" inside the UNION -- you can use select on both queries to make sure they will produce the same columns.
topics = Topic.select('user_id AS author_id, description AS body, created_at')
comments = Comment.select('author_id, body, created_at')
Comment.from("(#{comments.to_sql} UNION #{topics.to_sql}) AS comments")
Based on Olives' answer, I did come up with another solution to this problem. It feels a little bit like a hack, but it returns an instance of ActiveRelation, which is what I was after in the first place.
Post.where('posts.id IN
(
SELECT post_topic_relationships.post_id FROM post_topic_relationships
INNER JOIN "watched" ON "watched"."watched_item_id" = "post_topic_relationships"."topic_id" AND "watched"."watched_item_type" = "Topic" WHERE "watched"."user_id" = ?
)
OR posts.id IN
(
SELECT "posts"."id" FROM "posts" INNER JOIN "news" ON "news"."id" = "posts"."news_id"
INNER JOIN "watched" ON "watched"."watched_item_id" = "news"."id" AND "watched"."watched_item_type" = "News" WHERE "watched"."user_id" = ?
)', id, id)
I'd still appreciate it if anybody has any suggestions to optimize this or improve the performance, because it's essentially executing three queries and feels a little redundant.
You could also use Brian Hempel's active_record_union gem that extends ActiveRecord with an union method for scopes.
Your query would be like this:
Post.joins(:news => :watched).
where(:watched => {:user_id => id}).
union(Post.joins(:post_topic_relationships => {:topic => :watched}
.where(:watched => {:user_id => id}))
Hopefully this will be eventually merged into ActiveRecord some day.
Could you use an OR instead of a UNION?
Then you could do something like:
Post.joins(:news => :watched, :post_topic_relationships => {:topic => :watched})
.where("watched.user_id = :id OR topic_watched.user_id = :id", :id => id)
(Since you are joins the watched table twice I'm not too sure what the names of the tables will be for the query)
Since there are a lot of joins, it might also be quite heavy on the database, but it might be able to be optimized.
How about...
def union(scope1, scope2)
ids = scope1.pluck(:id) + scope2.pluck(:id)
where(id: ids.uniq)
end
Arguably, this improves readability, but not necessarily performance:
def my_posts
Post.where <<-SQL, self.id, self.id
posts.id IN
(SELECT post_topic_relationships.post_id FROM post_topic_relationships
INNER JOIN watched ON watched.watched_item_id = post_topic_relationships.topic_id
AND watched.watched_item_type = "Topic"
AND watched.user_id = ?
UNION
SELECT posts.id FROM posts
INNER JOIN news ON news.id = posts.news_id
INNER JOIN watched ON watched.watched_item_id = news.id
AND watched.watched_item_type = "News"
AND watched.user_id = ?)
SQL
end
This method returns an ActiveRecord::Relation, so you could call it like this:
my_posts.order("watched_item_type, post.id DESC")
There is an active_record_union gem.
Might be helpful
https://github.com/brianhempel/active_record_union
With ActiveRecordUnion, we can do:
the current user's (draft) posts and all published posts from anyone
current_user.posts.union(Post.published)
Which is equivalent to the following SQL:
SELECT "posts".* FROM (
SELECT "posts".* FROM "posts" WHERE "posts"."user_id" = 1
UNION
SELECT "posts".* FROM "posts" WHERE (published_at < '2014-07-19 16:04:21.918366')
) posts
In a similar case I summed two arrays and used Kaminari:paginate_array(). Very nice and working solution. I was unable to use where(), because I need to sum two results with different order() on the same table.
Heres how I joined SQL queries using UNION on my own ruby on rails application.
You can use the below as inspiration on your own code.
class Preference < ApplicationRecord
scope :for, ->(object) { where(preferenceable: object) }
end
Below is the UNION where i joined the scopes together.
def zone_preferences
zone = Zone.find params[:zone_id]
zone_sql = Preference.for(zone).to_sql
region_sql = Preference.for(zone.region).to_sql
operator_sql = Preference.for(Operator.current).to_sql
Preference.from("(#{zone_sql} UNION #{region_sql} UNION #{operator_sql}) AS preferences")
end
Less problems and easier to follow:
def union_scope(*scopes)
scopes[1..-1].inject(where(id: scopes.first)) { |all, scope| all.or(where(id: scope)) }
end
So in the end:
union_scope(watched_news_posts, watched_topic_posts)
gem 'active_record_extended'
Also has a set of union helpers among many others.
I would just run the two queries you need and combine the arrays of records that are returned:
#posts = watched_news_posts + watched_topics_posts
Or, at the least test it out. Do you think the array combination in ruby will be far too slow? Looking at the suggested queries to get around the problem, I'm not convinced that there will be that significant of a performance difference.
Elliot Nelson answered good, except the case where some of the relations are empty. I would do something like that:
def union_2_relations(relation1,relation2)
sql = ""
if relation1.any? && relation2.any?
sql = "(#{relation1.to_sql}) UNION (#{relation2.to_sql}) as #{relation1.klass.table_name}"
elsif relation1.any?
sql = relation1.to_sql
elsif relation2.any?
sql = relation2.to_sql
end
relation1.klass.from(sql)
end
When we add UNION to the scopes, it breaks at time due to order_by clause added before the UNION.
So I changed it in a way to give it a UNION effect.
module UnionScope
def self.included(base)
base.send(:extend, ClassMethods)
end
module ClassMethods
def union_scope(*scopes)
id_column = "#{table_name}.id"
sub_query = scopes.map { |s| s.pluck(:id) }.flatten
where("#{id_column} IN (?)", sub_query)
end
end
end
And then use it like this in any model
class Model
include UnionScope
scope :union_of_scopeA_scopeB, -> { union_scope(scopeA, scopeB) }
end
Tim's answer is great. It uses the ids of the scopes in the WHERE clause. As shosti reports, this method is problematic in terms of performance because all ids need to be generated during query execution. This is why, I prefer joeyk16 answer. Here a generalized module:
module ActiveRecord::UnionScope
def self.included(base)
base.send :extend, ClassMethods
end
module ClassMethods
def self.union(*scopes)
self.from("(#{scopes.map(&:to_sql).join(' UNION ')}) AS #{self.table_name}")
end
end
end
If you don't want to use SQL syntax inside your code, here's solution with arel
watched_news_posts = Post.joins(:news => :watched).where(:watched => {:user_id => id}).arel
watched_topic_posts = Post.joins(:post_topic_relationships => {:topic => :watched}).where(:watched => {:user_id => id}).arel
results = Arel::Nodes::Union.new(watched_news_posts, watched_topic_posts)
from(Post.arel_table.create_table_alias(results, :posts))

Rails 3 Conditions on Eager Loaded Association

I'm having trouble with Rails 3 using conditions on an associated table while eager loading. It appears that Rails is applying the condition when it loads the original model data, so it won't load the parent model unless a non-zero number of the child/associated models match the condition. This is easier to explain in code (simplified for example):
#post = Post.includes(:comments).where(:comments => { :approved => true }).find(1)
This would generate a SQL query similar to:
SELECT DISTINCT `posts`.id FROM `posts`
LEFT OUTER JOIN `comments` ON `comments`.`post_id` = `posts`.`id`
WHERE (`comments`.`approved` = 1) AND (`posts`.`id` = '1')
LIMIT 1
In the case that there aren't any comments that meet the approved = 1 condition, no rows are returned, and thus the Post never gets loaded at all.
What is the right way to load a post and the associated comments eagerly with a condition on the comments?
Update
I'd stil love to hear a better way of doing this, but for now I'm using the following to work around it (works with deeply nested eager loading):
#post = Post.find(1)
#comments = #post.comments.where(:approved => true).all
# allows deeper/more complex nesting without getting into SQL:
#post = Post.includes(:author => [ :websites, :photo ]).find(1)
#comments = #post.comments.includes(:editor).where(:approved => true).all
I guess what you are looking for is joins method, it will let you put your condition within join definition, not outside of it. For example:
#post = Post.joins("LEFT JOIN comments on posts.id = comments.post_id AND comments.approved = 1").first
Not sure about the correctness of the condition itself but you get my point.
Unfortunately you have to use that ugly string as joins is using INNER JOIN if you pass array/hash.
There's more about joins at rails guides
Update: There might be some nugget of wisdom in this post on includes vs eager_load vs preload.
I'd still love to hear a better way of doing this, but for now I'm using the following to work around it (works with deeply nested eager loading, unlike using joins):
#post = Post.find(1)
#comments = #post.comments.where(:approved => true).all
# allows deeper/more complex nesting without getting into SQL:
#post = Post.includes(:author => [ :websites, :photo ]).find(1)
#comments = #post.comments.includes(:editor).where(:approved => true).all

Resources