Controller
#communities = current_user.get_up_voted(Community).pluck(:id)
#codes = Code.includes(:user).where(:community_id => #communities).order('users.last_active_at DESC').page(params[:page]).per(10)
I get this error
ActiveRecord::StatementInvalid (Mysql2::Error: Column 'id' in field list is ambiguous: SELECT id FROM communities` INNER JOIN votes ON votes.votable_id = communities.id AND votes.votable_type = 'Community' WHERE votes.voter_id = 2 AND votes.voter_type = 'User' AND votes.vote_flag = 1 AND (communities.deleted_at IS NULL)):
UPDATE1(SQL Query with Patriot's code)
SELECT `codes`.`id` AS t0_r0, `codes`.`user_id` AS t0_r1, `codes`.`community_id` AS t0_r2, `codes`.`invisible` AS t0_r3, `codes`.`code` AS t0_r4, `codes`.`greeting` AS t0_r5, `codes`.`created_at` AS t0_r6, `codes`.`updated_at` AS t0_r7, `codes`.`deleted_at` AS t0_r8, `codes`.`notification` AS t0_r9, `codes`.`wanted` AS t0_r10, `users`.`id` AS t1_r0, `users`.`email` AS t1_r1, `users`.`encrypted_password` AS t1_r2, `users`.`username` AS t1_r3, `users`.`reset_password_token` AS t1_r4, `users`.`reset_password_sent_at` AS t1_r5, `users`.`remember_created_at` AS t1_r6, `users`.`sign_in_count` AS t1_r7, `users`.`current_sign_in_at` AS t1_r8, `users`.`last_sign_in_at` AS t1_r9, `users`.`current_sign_in_ip` AS t1_r10, `users`.`last_sign_in_ip` AS t1_r11, `users`.`banned` AS t1_r12, `users`.`confirmation_token` AS t1_r13, `users`.`confirmed_at` AS t1_r14, `users`.`confirmation_sent_at` AS t1_r15, `users`.`unconfirmed_email` AS t1_r16, `users`.`created_at` AS t1_r17, `users`.`updated_at` AS t1_r18, `users`.`deleted_at` AS t1_r19, `users`.`last_active_at` AS t1_r20, `users`.`comments_count` AS t1_r21, `users`.`follows_count` AS t1_r22, `users`.`codes_count` AS t1_r23, `users`.`communities_count` AS t1_r24, `users`.`nomail` AS t1_r25, `users`.`point_added_at` AS t1_r26, `user_profiles`.`id` AS t2_r0, `user_profiles`.`user_id` AS t2_r1, `user_profiles`.`language_id` AS t2_r2, `user_profiles`.`country_id` AS t2_r3, `user_profiles`.`prefecture_id` AS t2_r4, `user_profiles`.`gender_id` AS t2_r5, `user_profiles`.`nickname` AS t2_r6, `user_profiles`.`introduction` AS t2_r7, `user_profiles`.`picture_url` AS t2_r8, `user_profiles`.`created_at` AS t2_r9, `user_profiles`.`updated_at` AS t2_r10, `user_profiles`.`deleted_at` AS t2_r11, `user_profiles`.`user_avatar_file_name` AS t2_r12, `user_profiles`.`user_avatar_content_type` AS t2_r13, `user_profiles`.`user_avatar_file_size` AS t2_r14, `user_profiles`.`user_avatar_updated_at` AS t2_r15, `user_profiles`.`age` AS t2_r16, `user_profiles`.`activity_invisible` AS t2_r17, `user_profiles`.`total_point` AS t2_r18, `user_profiles`.`bonus_point` AS t2_r19, `user_profiles`.`introduction_html` AS t2_r20, `user_profiles`.`title` AS t2_r21, `user_profiles`.`next_level` AS t2_r22, `user_profiles`.`invitation` AS t2_r23, `user_profiles`.`notification` AS t2_r24, `user_profiles`.`notification_time` AS t2_r25, `user_profiles`.`wall_time` AS t2_r26, `user_profiles`.`wall_flag` AS t2_r27, `user_profiles`.`wanted_at` AS t2_r28, `user_profiles`.`wanted_message` AS t2_r29 FROM `codes` LEFT OUTER JOIN `users` ON `users`.`id` = `codes`.`user_id` LEFT OUTER JOIN `user_profiles` ON `user_profiles`.`user_id` = `users`.`id` WHERE `codes`.`community_id` IN (6, 2, 9, 1, 8, 16, 18, 19, 20, 21, 22, 23, 24, 30, 29, 67, 66, 5, 87) AND (`codes`.`deleted_at` IS NULL) ORDER BY users.last_active_at DESC LIMIT 10 OFFSET 0
UPDATE2
code
#communities = current_user.get_up_voted(Community)
#codes = Code.includes(:user).where(:community_id => #communities.collect(&:id)).order('created_at DESC').page(params[:page]).per(10)
SQL Query
ActiveRecord::StatementInvalid (Mysql2::Error: Column 'id' in field list is ambiguous: SELECT id FROM `communities` INNER JOIN `votes` ON `votes`.`votable_id` = `communities`.`id` AND `votes`.`votable_type` = 'Community' WHERE `votes`.`voter_id` = 2 AND `votes`.`voter_type` = 'User' AND `votes`.`vote_flag` = 1 AND (`communities`.`deleted_at` IS NULL)):
just because you called that column id which is no way to access it. it happens when the column name appears many times in resulting table (usually in case of join)
e.g: like this if i join users table and user_ranks table.
id | name | email | id | rank | user_id
1 | 'pari' | 'x#g.com' | `rank_id`| 12 | 1
You can solve this problem by calling the field name explicitly something like
current_user.get_up_voted(Community).pluck('communities.id')
Do you have a scope on Community like upvoted_by? E.g. Community.upvoted_by(user)? If so (or if you add one) then you can do:
Community.upvoted_by(user).pluck(:id)
to get a list of these community ids. But, I think the best answer is to let ActiveRecord and ARel shoulder some of the burden for you. You can .merge a scope like this:
# community.rb
scope :upvoted_by, -> user { where(user_id: user) }
# controller
Code.includes(user: { :communities }).merge(Community.upvoted_by(user)).order('users.last_active_at DESC').page(params[:page]).per(10)
Related
This works...
a = User.select("users.*, 1 as car_id").limit(2)
b = User.select("users.*, 1 as car_id").limit(2)
a.or(b)
>> returns a combined AR Relation
However,
a = User.select("users.*, 1 as car_id").limit(2)
b = User.select("users.*, 2 as car_id").limit(2)
a.or(b)
>> ArgumentError (Relation passed to #or must be structurally compatible. Incompatible values: [:select])
If I do a+b, it combines, but then converts to an array, and I would like to keep an AR relation to continue to do queries on.
When the select has different values for car_id, it can not union. I would have thought the same column name would allow the union.
I am creating virtual attribute (car_id) on the model, as each set of records needs car_id defined with a different value.
How do I solve the error and union with virtual attributes?
I want an ActiveRecord Relation, and interesting enough:
SELECT users.*, 1 as car_id
FROM users
UNION
SELECT users.*, 2 as car_id
FROM users
works fine when running raw sql.
SQL Example:
a.mail_for(:inbox).name
=> Mailboxer::Conversation::ActiveRecord_Relation
a.mail_for(:inbox).to_sql
"SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM \"mailboxer_conversations\" INNER JOIN \"mailboxer_notifications\" ON \"mailboxer_notifications\".\"conversation_id\" = \"mailboxer_conversations\".\"id\" AND \"mailboxer_notifications\".\"type\" IN ('Mailboxer::Message') INNER JOIN \"mailboxer_receipts\" ON \"mailboxer_receipts\".\"notification_id\" = \"mailboxer_notifications\".\"id\" WHERE \"mailboxer_notifications\".\"type\" = 'Mailboxer::Message' AND \"mailboxer_receipts\".\"receiver_id\" = 102 AND \"mailboxer_receipts\".\"receiver_type\" = 'Listing' AND \"mailboxer_receipts\".\"mailbox_type\" = 'inbox' AND \"mailboxer_receipts\".\"trashed\" = FALSE AND \"mailboxer_receipts\".\"deleted\" = FALSE ORDER BY \"mailboxer_conversations\".\"updated_at\" DESC"
So I pass an array of relations to union_scope...
ar= a.mail_for(:inbox)
br= b.mail_for(:inbox)
cr= c.mail_for(:inbox)
combined = union_scope([[a,ar],[b, br],[c, cr])
def union_scope(*relation)
combined = relation.first[1].none
relation.each do |relation_set|
mailer = relation_set[0]
scope = relation_set[1].select("#{relation_set[1].table_name}.*, \'#{mailer.id}\' as mailer_id, \'#{mailer.class.name}\' as mailer_type")
combined = combined.or(scope)
end
combined
end
Update:
def union_scope(*relation)
combined = relation.first[1].none
relation.each do |relation_set|
mailer = relation_set[0]
scope = relation_set[1].select("#{relation_set[1].table_name}.*, \'#{mailer.id}\' as mailer_id, \'#{mailer.class.name}\' as mailer_type")
combined = combined.union(scope)
end
conv = ::Mailboxer::Conversation.arel_table
::Mailboxer::Conversation.from(conv.create_table_alias(combined, :conversations).to_sql)
end
Resulted in this error:
Mailboxer::Conversation Load (5.8ms) SELECT "mailboxer_conversations".* FROM ( SELECT DISTINCT "mailboxer_conversations".* FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $1 AND "mailboxer_receipts"."receiver_id" = $2 AND "mailboxer_receipts"."receiver_type" = $3 AND (1=0) ORDER BY "mailboxer_conversations"."updated_at" DESC UNION SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $4 AND "mailboxer_receipts"."receiver_id" = $5 AND "mailboxer_receipts"."receiver_type" = $6 ORDER BY "mailboxer_conversations"."updated_at" DESC ) "conversations"
ActiveRecord::StatementInvalid (PG::SyntaxError: ERROR: syntax error at or near "UNION")
LINE 1: ...ER BY "mailboxer_conversations"."updated_at" DESC UNION SELE...
^
: SELECT "mailboxer_conversations".* FROM ( SELECT DISTINCT "mailboxer_conversations".* FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $1 AND "mailboxer_receipts"."receiver_id" = $2 AND "mailboxer_receipts"."receiver_type" = $3 AND (1=0) ORDER BY "mailboxer_conversations"."updated_at" DESC UNION SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $4 AND "mailboxer_receipts"."receiver_id" = $5 AND "mailboxer_receipts"."receiver_type" = $6 ORDER BY "mailboxer_conversations"."updated_at" DESC ) "conversations"
While I cannot answer your actual question as to why these are incompatible I can offer a solution akin to your raw SQL example.
You can perform the same operation like so
user_table = User.arel_table
a = user_table.project(Arel.star, Arel.sql("1 as car_id")).take(2)
b = user_table.project(Arel.star, Arel.sql("2 as car_id")).take(2)
union = Arel::Nodes::UnionAll.new(a,b)
User.from(Arel::Nodes::As.new(union,user_table))
This will result in the following query.
SELECT
users.*
FROM
( (SELECT *, 1 as car_id
FROM users
ORDER BY
users.id ASC
LIMIT 2) UNION ALL (
SELECT *, 2 as car_id
FROM users
ORDER BY
users.id ASC
LIMIT 2)) AS users
Since this will still return an ActiveRecord::Relation you can still act on this object as you would any other we have simply substituted the normal data source (the users table) with your custom union data source of the same name.
Update based on extreme revision and some general assumptions as to what you actually meant to do
a= a.mail_for(:inbox)
b= b.mail_for(:inbox)
c= c.mail_for(:inbox)
combined = union_scope(a,b,c)
def union_scope(*relations)
base = build_scope(*relations.shift)
combined = relations.reduce(base) do |memo, relation_set|
Arel::Nodes::UnionAll.new(memo,build_scope(*relation_set))
end
union = Arel::Nodes::As.new(combined,::Mailboxer::Conversation.arel_table)
::Mailboxer::Conversation.from(union)
end
def build_scope(mailer,relation)
relation.select(
"#{relation.table_name}.*,
'#{mailer.id}' as mailer_id,
'#{mailer.class.name}' as mailer_type"
).arel
end
I have 2 models: Deal and User with a many_to_many relations and a user_deal joined table.
On the homepage i display a list of Deals. For each Deal I use inside the view n attribute that is inside the user_deal table.
How can I eager load this ?
I tried to do this:
homepage controller
def home
#deals = Deal.includes(:user_deals)
respond_to do |format|
format.html # home.html.erb
format.json { render json: #deals }
format.xml { render xml: #deals }
# format.atom
end
end
end
Models
class Deal < ActiveRecord::Base
has_many :user_deals, dependent: :destroy
has_many :users, through: :user_deals
end
class User < ActiveRecord::Base
has_many :user_deals
has_many :deals, through: :user_deals
end
class UserDeal < ActiveRecord::Base
belongs_to :user, :foreign_key => 'user_id'
belongs_to :deal, :foreign_key => 'deal_id'
end
But I feel it's not working as it's preloading things but then loading each object a second time...
Processing by StaticPagesController#home as HTML
(0.5ms) SELECT COUNT(*) FROM "Deals" AND "Deals"."featured" = 't'
Deal Load (0.7ms)
SELECT "Deals".* FROM "Deals" WHERE "Deals"."country" = $1 AND "Deals"."featured" = 't' ORDER BY "Deals"."Deal_end_date" ASC
// it seems here below to implement the preloading
UserDeal Load (1.2ms) SELECT "user_deals".* FROM "user_Deals" WHERE "user_Deals"."Deal_id" IN (30, 339, 341, 337, 353, 31, 354, 260)
//but then it's loading things individually again
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 30) LIMIT 1
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 339) LIMIT 1
UserDeal Load (0.7ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 341) LIMIT 1
UserDeal Load (0.7ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 337) LIMIT 1
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 353) LIMIT 1
UserDeal Load (0.7ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 31) LIMIT 1
UserDeal Load (0.9ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 354) LIMIT 1
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 260) LIMIT 1
My assumption of where the problem comes from (I'm not sure)
The problem might be inside the User_deal table. It works this way: if a user participated in a deal, a new line is created with user_id = 5, deal_id= 4, number_of_participations = 6
So if the user 4 never participated in a deal 7, there is no line user_id= 4 and deal_id = 7
So on the homepage, Rails go to fetch the list of deals: deal 1, deal 2...and deal 7.
But he does not find the line where deal_7 as there is no such line.
Ideally, I should tell Rails to includes(:user_deals) but only those where inside the joined table UserDeals where user_id= current_user.id OR where the line with user_id = current_user.id does NOT exist...which I did not manager to implement anyway
but I'm a RoR rookie so I'm not sure of my ideas here above
EDIT
Following suggestion to try eager_load instead of includes, I get:
User Load (0.6ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT 1 [["id", 625]]
Processing by StaticPagesController#home as HTML
(0.7ms) SELECT COUNT(DISTINCT "deals"."id") FROM "deals" LEFT OUTER JOIN "user_deals" ON "user_deals"."deal_id" = "deals"."id" WHERE "deals"."country" = $1 AND (deal_launch_date <= '2016-05-02 17:21:59.547029' AND deal_end_date >= '2016-05-02 17:21:59.547087') AND "deals"."featured" = 't' [["country", "France"]]
SQL (3.0ms) SELECT "deals"."id" AS t0_r0, "deals"."country" AS t0_r1, "deals"."title" AS t0_r2, "deals"."deal_card_edito" AS t0_r3, "deals"."twitter_msg" AS t0_r4, "deals"."image_url" AS t0_r5, "deals"."prelaunch_date" AS t0_r6, "deals"."featured" AS t0_r9, "deals"."admin_user_id" AS t0_r10, "deals"."created_at" AS t0_r11, "deals"."updated_at" AS t0_r12,........................................................................................................
"user_deals"."id" AS t1_r0, "user_deals"."user_id" AS t1_r1, "user_deals"."deal_id" AS t1_r2, "user_deals"."number_participations" AS t1_r3, "user_deals"."nb_new_clicks" AS t1_r4, "user_deals"."created_at" AS t1_r5, "user_deals"."updated_at" AS t1_r6 FROM "deals" LEFT OUTER JOIN "user_deals" ON "user_deals"."deal_id" = "deals"."id" WHERE "deals"."country" = $1 AND (deal_launch_date <= '2016-05-02 17:21:59.547029' AND deal_end_date >= '2016-05-02 17:21:59.547087') AND "deals"."featured" = 't' ORDER BY "deals"."deal_end_date" ASC [["country", "France"]]
UserDeal Load (0.9ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 30) LIMIT 1
UserDeal Load (1.0ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 339) LIMIT 1
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 341) LIMIT 1
UserDeal Load (0.6ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 337) LIMIT 1
UserDeal Load (0.9ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 353) LIMIT 1
UserDeal Load (1.0ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 31) LIMIT 1
UserDeal Load (0.5ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 354) LIMIT 1
UserDeal Load (0.9ms) SELECT "user_deals".* FROM "user_deals" WHERE (user_id = 625 AND deal_id = 260) LIMIT 1
Short version:
I'd like to query the result of another query, in order to select a more limited result set. However, adding a where clause rewrites the first query rather than work on the results, so I don't get the answers I need.
The detail:
I have two models, checks and ticks. Checks has_many ticks.
The first query uses DISTINCT ON and gathers all of the 'checks' and all of the related ticks but only returns the most recent tick. I have that working as a scope in the model.
In my controller,
def checklist
#Filter the results by scope or return all checks with latest tick
case params[:filter]
when "duebylastresult"
#checks = Check.mostrecenttickonly.duebylastresult
when "duebydate"
#checks = Check.mostrecenttickonly.duebydate
else
#checks = Check.mostrecenttickonly
end
end
In the model, the first scope (working):
scope :mostrecenttickonly, -> {
includes(:ticks)
.order("checks.id, ticks.created_at DESC")
.select("DISTINCT ON (checks.id) *").references(:ticks)
}
Generates the following SQL:
Parameters: {"filter"=>""}
SQL (1.0ms) SELECT DISTINCT ON (checks.id) *,
"checks"."id" AS t0_r0,
"checks"."area" AS t0_r1, "checks"."frequency" AS t0_r2,
"checks"."showinadvance" AS t0_r3, "checks"."category" AS t0_r4,
"checks"."title" AS t0_r5, "checks"."description" AS t0_r6,
"checks"."created_at" AS t0_r7, "checks"."updated_at" AS t0_r8,
"ticks"."id" AS t1_r0, "ticks"."result" AS t1_r1,
"ticks"."comments" AS t1_r2, "ticks"."created_at" AS t1_r3,
"ticks"."updated_at" AS t1_r4, "ticks"."check_id" AS t1_r5
FROM "checks" LEFT OUTER JOIN "ticks"
ON "ticks"."check_id" = "checks"."id"
ORDER BY checks.id, ticks.created_at DESC
Having got that result, I want to show only the ticks that have a value equal or greater than 3, so the scope:
scope :duebylastresult, -> { where("ticks.result >= 3") }
Generates the SQL
Parameters: {"filter"=>"duebylastresult"}
SQL (1.0ms) SELECT DISTINCT ON (checks.id) *,
"checks"."id" AS t0_r0,
"checks"."area" AS t0_r1, "checks"."frequency" AS t0_r2,
"checks"."showinadvance" AS t0_r3, "checks"."category" AS t0_r4,
"checks"."title" AS t0_r5, "checks"."description" AS t0_r6,
"checks"."created_at" AS t0_r7, "checks"."updated_at" AS t0_r8,
"ticks"."id" AS t1_r0, "ticks"."result" AS t1_r1,
"ticks"."comments" AS t1_r2, "ticks"."created_at" AS t1_r3,
"ticks"."updated_at" AS t1_r4, "ticks"."check_id" AS t1_r5
FROM "checks" LEFT OUTER JOIN "ticks"
ON "ticks"."check_id" = "checks"."id"
WHERE (ticks.result >= 3)
ORDER BY checks.id, ticks.created_at DESC
As best I can tell, the WHERE statement is acting before the DISTINCT ON clause, so I now have the 'latest tick where the result is >= 3', whilst I'm looking for 'latest tick THEN only where the result is >= 3'.
Hope that makes sense & Thanks in advance!
Edit - Example of what I get and what I need:
The Data:
Table Checks:
ID: 98 Title: Eire
ID: 99 Title: Land
Table Ticks:
ID: 1 CheckID: 98 Result:1 Date: Jan12
ID: 2 CheckID: 98 Result:5 Date: Feb12
ID: 3 CheckID: 98 Result:1 Date: Mar12
ID: 4 CheckID: 99 Result:4 Date: Apr12
First query returns the most recent result, like;
Check.ID: 98 Tick.ID: 3 Tick.Result: 1 Tick.Date: Mar12
Check.ID: 99 Tick.ID: 4 Tick.Result: 4 Tick.Date: Apr12
Second query currently returns the most recent result where the result is =>3, like;
Check.ID: 98 Tick.ID: 2 Tick.Result: 5 Tick.Date: Feb12
Check.ID: 99 Tick.ID: 4 Tick.Result: 5 Tick.Date: Apr12
When I really want:
Check.ID: 99 Tick.ID: 4 Tick.Result: 5 Tick.Date: Apr12
(ID 98 doesn't show as the last Tick.Result is 1).
Could you try the following to see if it starts you in the right direction:
scope :just_a_test, -> {
includes(:ticks)
.order("checks.id")
.where("ticks.created_at = (SELECT MAX(ticks.created_at) FROM ticks WHERE ticks.check_id = checks.id)")
.where("ticks.result >= 3")
.group("checks.id")
}
I'm not sure I really understand the point of the :mostrecenttickonly scope since you're just loading the checks.
That being said, if you want to get only those checks whose most recent ticks have a result greater than three, I think the best way to do that would be a window function:
check.rb
...
scope :duebylastresult, -> {
find_by_sql(
'SELECT *
FROM (SELECT checks.*,
ticks.id AS tick_ids,
ticks.date AS tick_date,
ticks.result AS tick_result,
dense_rank() OVER (
PARTITION BY checks.id
ORDER BY ticks.date DESC
) AS tick_rank
FROM checks
LEFT OUTER JOIN ticks ON checks.id = ticks.check_id) AS ranked_ticks
WHERE tick_rank = 1 AND tick_result >= 3;'
)
}
...
Basically, we're just joining everything in the checks and ticks tables, then adding another attribute called tick_rank that is ranking each row in the result set according to its date versus the other rows with the same checks.id value.
The way SQL works is that the predicates (the conditions in the WHERE clause) are evaluated prior to the evaluation of the SELECT fields, meaning we can't just write tick_rank = 1 in this statement.
So we have to go the extra step of wrapping the results (which we alias as ranked_ticks) and then just select everything and apply the predicates we want to this outer select statement. The tick_rank has to be 1, meaning it's the most recent tick, and the result has to be >= 3.
edit: I was using that article I linked as a refresher since I often forget SQL syntax, but after looking at it, I think this would be somewhat more performant (basically just wait to join checks until after the partitioning is done, that way I believe it will do fewer full scans):
scope :duebylastresult, -> {
find_by_sql(
'SELECT *
FROM checks
LEFT OUTER JOIN
(SELECT id AS tick_id,
check_id AS check_id,
date AS tick_date,
result AS tick_result,
dense_rank() OVER (
PARTITION BY ticks.check_id
ORDER BY ticks.date DESC
) AS tick_rank
FROM ticks) AS ranked_ticks ON checks.id = ranked_ticks.check_id
WHERE tick_rank = 1 AND tick_result >= 3;'
)
}
This is really nothing more than just looking for some explanation around why the joins clause is necessary for what I am doing. Both return everything I want properly, but the queries are different and one is less optimal:
TL;DR: What is Rails doing exactly when I combine the joins with the includes, I wouldn't have thought I needed the joins really. Thanks for anyone willing to actually read through this monster.
Note: object is a ToDoListsUser object, and this is all happening in an exporter for what it's worth. And ProgressItem is a joins table between User and ToDoItem.
With the joins clause (queries look good and as expected):
object.to_do_list.progress_items.where(user_id: object.user_id)
.joins(:to_do_item)
.includes(:to_do_item)
Server output:
ToDoListsUser Load (0.5ms) SELECT "to_do_lists_users".* FROM "to_do_lists_users" WHERE "to_do_lists_users"."user_id" = $1 ORDER BY "to_do_lists_users"."id" DESC OFFSET 0 [["user_id", 543]]
ToDoList Load (0.3ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 144]]
SQL (1.6ms) SELECT "progress_items"."id" AS t0_r0, "progress_items"."completed" AS t0_r1, "progress_items"."user_id" AS t0_r2, "progress_items"."to_do_item_id" AS t0_r3, "progress_items"."created_at" AS t0_r4, "progress_items"."updated_at" AS t0_r5, "progress_items"."hidden" AS t0_r6, "to_do_items_progress_items"."id" AS t1_r0, "to_do_items_progress_items"."to_do_list_id" AS t1_r1, "to_do_items_progress_items"."description" AS t1_r2, "to_do_items_progress_items"."date_due" AS t1_r3, "to_do_items_progress_items"."created_at" AS t1_r4, "to_do_items_progress_items"."updated_at" AS t1_r5, "to_do_items_progress_items"."name" AS t1_r6, "to_do_items_progress_items"."element_id" AS t1_r7, "to_do_items_progress_items"."linkable_id" AS t1_r8, "to_do_items_progress_items"."linkable_type" AS t1_r9, "to_do_items_progress_items"."action" AS t1_r10 FROM "progress_items" INNER JOIN "to_do_items" "to_do_items_progress_items" ON "to_do_items_progress_items"."id" = "progress_items"."to_do_item_id" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 144], ["user_id", 543]]
ToDoList Load (0.4ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 133]]
SQL (3.3ms) SELECT "progress_items"."id" AS t0_r0, "progress_items"."completed" AS t0_r1, "progress_items"."user_id" AS t0_r2, "progress_items"."to_do_item_id" AS t0_r3, "progress_items"."created_at" AS t0_r4, "progress_items"."updated_at" AS t0_r5, "progress_items"."hidden" AS t0_r6, "to_do_items_progress_items"."id" AS t1_r0, "to_do_items_progress_items"."to_do_list_id" AS t1_r1, "to_do_items_progress_items"."description" AS t1_r2, "to_do_items_progress_items"."date_due" AS t1_r3, "to_do_items_progress_items"."created_at" AS t1_r4, "to_do_items_progress_items"."updated_at" AS t1_r5, "to_do_items_progress_items"."name" AS t1_r6, "to_do_items_progress_items"."element_id" AS t1_r7, "to_do_items_progress_items"."linkable_id" AS t1_r8, "to_do_items_progress_items"."linkable_type" AS t1_r9, "to_do_items_progress_items"."action" AS t1_r10 FROM "progress_items" INNER JOIN "to_do_items" "to_do_items_progress_items" ON "to_do_items_progress_items"."id" = "progress_items"."to_do_item_id" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 133], ["user_id", 543]]
vs.
Without the joins clause (queries are not optimal):
object.to_do_list.progress_items.where(user_id: object.user_id)
.includes(:to_do_item)
Server output:
ToDoListsUser Load (0.3ms) SELECT "to_do_lists_users".* FROM "to_do_lists_users" WHERE "to_do_lists_users"."user_id" = $1 ORDER BY "to_do_lists_users"."id" DESC OFFSET 0 [["user_id", 543]]
ToDoList Load (0.4ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 144]]
ProgressItem Load (2.0ms) SELECT "progress_items".* FROM "progress_items" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 144], ["user_id", 543]]
ToDoList Load (0.5ms) SELECT "to_do_lists".* FROM "to_do_lists" WHERE "to_do_lists"."id" = $1 LIMIT 1 [["id", 133]]
ProgressItem Load (0.5ms) SELECT "progress_items".* FROM "progress_items" INNER JOIN "to_do_items" ON "progress_items"."to_do_item_id" = "to_do_items"."id" WHERE "to_do_items"."to_do_list_id" = $1 AND "progress_items"."user_id" = $2 ORDER BY to_do_items.created_at [["to_do_list_id", 133], ["user_id", 543]]
ToDoItem Load (0.4ms) SELECT "to_do_items".* FROM "to_do_items" WHERE "to_do_items"."id" IN (193, 194, 195, 196)
From ActiveRecord documentation:
conditions
If you want to add conditions to your included models you’ll have to explicitly reference them. For example:
User.includes(:posts).where('posts.name = ?', 'example')
Will throw an error, but this will work:
User.includes(:posts).where('posts.name = ?', 'example').references(:posts)
Note that includes works with association names while references needs the actual table name.
That happens, because actually Rails with include doesn't do join, it fetches records with two requests.
For example:
class List < ActiveRecord::Base
has_many :tasks
end
List.includes(:tasks)
# List Load (1.9ms) SELECT "lists".* FROM "lists"
# Task Load (0.8ms) SELECT "tasks".* FROM "tasks" WHERE "tasks"."list_id" IN (1, 2, 3)
But with references it falls back to join:
List.includes(:tasks).references(:tasks)
# SQL (0.6ms) SELECT "lists"."id" AS t0_r0, "lists"."name" AS t0_r1,
# "tasks"."id" AS t1_r0, "tasks"."name" AS t1_r1 FROM
# "lists" LEFT OUTER JOIN "tasks" ON "tasks"."list_id" = "lists"."id"
Using Rails 4 and Ruby 1.9.3.
I have this view in usuarios#show I'm rendering. The controller demands a lot of data, but I have managed to reduce the queries with eager_load and akin from 300ms odd to a 44 ms.
My problem is that the database call is ok, but the views are taking close to 30 seconds to render.
I have made sure that I'm getting everything I use from the controller variables.
EDIT: I have added the query itself from the controller
This is the controller method, using eager_loading
#app/controllers/usuarios_controller.rb
before_action :show_usuario, only: [:show]
def show
if !#usuario.country_id.blank?
#country = #usuario.country
end
respond_to do |format
format.html
format.js
end
end
private
def show_usuario
if usuario_signed_in?
id = current_usuario.id
#usuario = Usuario.eager_load(:profile).find(id)
else
#usuario = Usuario.eager_load(:profile, textos: [:likes, :text_medals, :tags]).find(params[:id])
end
#textos = Texto.eager_load(:likes, :text_medals, :tags).where(:usuario_id => #usuario.id).order("textos.created_at DESC").paginate(:page => params[:page]).per_page(6)
end
Here is my log
Started GET "/" for 127.0.0.1 at 2014-03-25 12:55:39 +0100
Usuario Load (1.4ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = 12 ORDER BY "usuarios"."id" ASC LIMIT 1
Processing by UsuariosController#show as HTML
Notification Load (1.2ms) SELECT "notifications".* FROM "notifications" INNER JOIN "textos" ON "textos"."id" = "notifications"."text" WHERE (textos.usuario_id = 12 AND notifications.user_related != 12) ORDER BY "notifications"."id" DESC LIMIT 10
SQL (1.2ms) SELECT "usuarios"."id" AS t0_r0, "usuarios"."email" AS t0_r1, "usuarios"."encrypted_password" AS t0_r2, "usuarios"."reset_password_token" AS t0_r3, "usuarios"."reset_password_sent_at" AS t0_r4, "usuarios"."remember_created_at" AS t0_r5, "usuarios"."sign_in_count" AS t0_r6, "usuarios"."current_sign_in_at" AS t0_r7, "usuarios"."last_sign_in_at" AS t0_r8, "usuarios"."current_sign_in_ip" AS t0_r9, "usuarios"."last_sign_in_ip" AS t0_r10, "usuarios"."created_at" AS t0_r11, "usuarios"."updated_at" AS t0_r12, "usuarios"."nombre" AS t0_r13, "usuarios"."fecha_nac" AS t0_r14, "usuarios"."country_id" AS t0_r15, "usuarios"."is_admin" AS t0_r16, "usuarios"."publish_fbviews" AS t0_r17, "usuarios"."accept_terms" AS t0_r18, "profiles"."id" AS t1_r0, "profiles"."usuario_id" AS t1_r1, "profiles"."quote" AS t1_r2, "profiles"."quote_author" AS t1_r3, "profiles"."fb_account" AS t1_r4, "profiles"."twt_account" AS t1_r5, "profiles"."gpls_account" AS t1_r6, "profiles"."biografia" AS t1_r7, "profiles"."created_at" AS t1_r8, "profiles"."updated_at" AS t1_r9, "profiles"."hide_email" AS t1_r10, "profiles"."pic_file_name" AS t1_r11, "profiles"."pic_content_type" AS t1_r12, "profiles"."pic_file_size" AS t1_r13, "profiles"."pic_updated_at" AS t1_r14 FROM "usuarios" LEFT OUTER JOIN "profiles" ON "profiles"."usuario_id" = "usuarios"."id" WHERE "usuarios"."id" = $1 LIMIT 1 [["id", 12]]
USER CARGADO
Country Load (0.7ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT 1 [["id", 246]]
TEXTOS CARGADOS
Rendered usuarios/_main_frame.html.erb (12082.9ms)
(1.6ms) SELECT COUNT(DISTINCT "textos"."id") FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12
SQL (1.9ms) SELECT DISTINCT "textos".id, textos.created_at AS alias_0 FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12 ORDER BY textos.created_at DESC LIMIT 6 OFFSET 0
SQL (5.6ms) SELECT "textos"."id" AS t0_r0, "textos"."usuario_id" AS t0_r1, "textos"."titulo" AS t0_r2, "textos"."contenido" AS t0_r3, "textos"."idioma" AS t0_r4, "textos"."created_at" AS t0_r5, "textos"."updated_at" AS t0_r6, "textos"."is_borrador" AS t0_r7, "textos"."is_on_contest" AS t0_r8, "textos"."portada_file_name" AS t0_r9, "textos"."portada_content_type" AS t0_r10, "textos"."portada_file_size" AS t0_r11, "textos"."portada_updated_at" AS t0_r12, "textos"."contest_id" AS t0_r13, "textos"."views" AS t0_r14, "textos"."has_draft" AS t0_r15, "textos"."license_type" AS t0_r16, "likes"."id" AS t1_r0, "likes"."usuario_id" AS t1_r1, "likes"."texto_id" AS t1_r2, "likes"."created_at" AS t1_r3, "likes"."updated_at" AS t1_r4, "text_medals"."id" AS t2_r0, "text_medals"."name" AS t2_r1, "text_medals"."slogan" AS t2_r2, "text_medals"."created_at" AS t2_r3, "text_medals"."updated_at" AS t2_r4, "text_medals"."image_file_name" AS t2_r5, "text_medals"."image_content_type" AS t2_r6, "text_medals"."image_file_size" AS t2_r7, "text_medals"."image_updated_at" AS t2_r8, "tags"."id" AS t3_r0, "tags"."nombre" AS t3_r1, "tags"."created_at" AS t3_r2, "tags"."updated_at" AS t3_r3 FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12 AND "textos"."id" IN (75, 74, 73, 70, 69, 68) ORDER BY textos.created_at DESC
CACHE (0.0ms) SELECT COUNT(DISTINCT "textos"."id") FROM "textos" LEFT OUTER JOIN "likes" ON "likes"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals_textos" ON "text_medals_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "text_medals" ON "text_medals"."id" = "text_medals_textos"."text_medal_id" LEFT OUTER JOIN "tags_textos" ON "tags_textos"."texto_id" = "textos"."id" LEFT OUTER JOIN "tags" ON "tags"."id" = "tags_textos"."tag_id" WHERE "textos"."usuario_id" = 12
Usuario Load (0.8ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
Profile Load (0.7ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.1ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.0ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.0ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
CACHE (0.1ms) SELECT "usuarios".* FROM "usuarios" WHERE "usuarios"."id" = $1 ORDER BY "usuarios"."id" ASC LIMIT 1 [["id", 12]]
CACHE (0.1ms) SELECT "profiles".* FROM "profiles" WHERE "profiles"."usuario_id" = $1 ORDER BY "profiles"."id" ASC LIMIT 1 [["usuario_id", 12]]
Rendered usuarios/_user_texts.html.erb (16760.5ms)
Rendered usuarios/show.html.erb within layouts/application (28889.5ms)
Completed 200 OK in 29246ms (Views: 28898.8ms | ActiveRecord: 44.1ms)
As you can see, the views take ages to render.
This the "_user_texts" partial.
<div class="ui grid">
<% #textos.each do |texto| %>
<div class="text row" text-pages="<%= #textos.total_pages%>">
<div class="five wide column">
<%= image_tag texto.portada.url(:medium), :class => "ui rounded medium left floated image text-cover" %>
</div>
<div class="eleven wide column">
<div class="row">
<h1 class="ui header">
<%= link_to text_name(texto), usuario_texto_path(texto.usuario.id, texto.id), :class => "title-format"%>
<div class="sub header subheader-format">
escrito por <%= text_author_image texto %><%= text_author_name texto %> el <%= humanize_creation_date(texto.created_at) %>
</div>
</h1>
</div>
<div class="tag-list row">
<div class="ui horizontal list">
<% if not texto.tags.empty? %>
<% texto.tags.each do |tag| %>
<div class="tag item">
<%= link_to tag.nombre, tag_path(tag.id), :class => "ui red large label" %>
</div>
<% end %>
<% end %>
</div>
</div>
</div>
</div>
<% end %>
</div>
<% if #textos.total_pages > #textos.current_page %>
<a id="more_texts" href="javascript:void(0)">
<span class="vertical"></span><span>Ver más</span>
</a>
<% end %>
EDIT: I am using pagination. Specifically the will_paginate gem. When I get to the bottom of the page, there's a button for fetching more results.
Can somebody give me insights on how could I make it faster? This is driving me nuts.
Thanks!
I would benchmark your query, seems pretty heavy to me. I think the focus should be on enhancing the performance of your sql query. I have no knowledge of the exact content, but I doubt you need this much intelligence in a single query.
For instance, you could add indices within your table to improve performance. You should also consider what attributes you really need and use select to define them. Less columns in your query yield faster results.
Finally, you are doing a lot of joins. It seems like you are pretty much joining all of your models. Left joins are really slow, because you go over all records. If you really need all these joins, it would be great if you could turn some of those left joins to inner joins . This would mean a massive improvement, because you have to check only relevant records.
The query is not actually run until you call .all or .each in the view even if you store the results in an instance variable in the controller. This can make it seem like the views are slow when actually it's something in the database.
Try putting some calls to benchmark different places in your view to see where the time is being used.
http://api.rubyonrails.org/classes/ActiveSupport/Benchmarkable.html#method-i-benchmark
For seeing what you're query is doing and spotting missing indexes etc. explain is handy.
http://guides.rubyonrails.org/active_record_querying.html#running-explain
Solved. With the help of the Bullet gem, I reformulate the way I was eager loading.
It turns out that everything done was, in practice, well done. The issue was that my partner was storing the images for paperclip in Dropbox.
Dropbox is most likely not made for this kind of requests, so the delay was because of this. We will change to Amazon's S3.
For the reader from the future: be advised! And check where are you storing your images.
Thanks to everybody!