I have two simular codes:
1 code:
User.chats.first // execution time 17 SEC!
2 code
User.chats[0] // execution time is 2 ms
SQL log - 1 code:
**(17018.8ms)** SELECT "chats".* FROM "chats" WHERE "chats"."user_id" = $1 AND
("chats"."blocked" = 'f' OR "chats"."blocked" IS NULL) ORDER BY "chats"."id" ASC LIMIT $2
SQL log - 2 code:
**(2.2ms)** SELECT "chats".* FROM "chats" WHERE "chats"."user_id" = $1 AND
("chats"."blocked" = 'f' OR "chats"."blocked" IS NULL) ORDER BY "chats"."id" ASC
What is diffirent beetwen first method and [0]?
Why execution speed so different 17 sec and 2 ms?
Related
I'm using ActiveRecord to build a query and I got a parenthesis issue with generated SQL.
Here is a query creating my problem
joined_users = User.joins(:consumer)
users = joined_users.where(id: (1..3), name: 'foo').or(joined_users.where(consumer: {id: (1...4))
It creates the following SQL
SELECT "users".* FROM "users" INNER JOIN "consumers" "consumer" ON "consumer"."user_id" = "users"."id" WHERE ("users"."id" BETWEEN 1 AND 3 AND "users"."name" = 'foo' OR "consumer"."id" >= 1 AND "consumer"."id" < 4)
But actually, it should be
SELECT "users".* FROM "users" INNER JOIN "consumers" "consumer" ON "consumer"."user_id" = "users"."id" WHERE (("users"."id" BETWEEN 1 AND 3 AND "users"."name" = 'foo') OR ("consumer"."id" >= 1 AND "consumer"."id" < 4))
Do you get my point?
There are missing parenthesis separating the first where to the second with the or.
Bool logic is not the same at all with and without these parenthesis.
What am I missing?
I your particular case there should be no difference in query result, because logical AND operator has higher priority than logical OR, so:
A && B || C && D equals (A && B) || (C && D)
ActiveRecord will add parenthesis when it is needed, for example:
User.where(id: 1..3).or(User.where(id: 4...5)).where(email: 'some')
Will result in:
SELECT "users".* FROM "users" WHERE ("users"."id" BETWEEN 1 AND 3 OR "users"."id" >= 4 AND "users"."id" < 5) AND "users"."email" = 'some'
This works...
a = User.select("users.*, 1 as car_id").limit(2)
b = User.select("users.*, 1 as car_id").limit(2)
a.or(b)
>> returns a combined AR Relation
However,
a = User.select("users.*, 1 as car_id").limit(2)
b = User.select("users.*, 2 as car_id").limit(2)
a.or(b)
>> ArgumentError (Relation passed to #or must be structurally compatible. Incompatible values: [:select])
If I do a+b, it combines, but then converts to an array, and I would like to keep an AR relation to continue to do queries on.
When the select has different values for car_id, it can not union. I would have thought the same column name would allow the union.
I am creating virtual attribute (car_id) on the model, as each set of records needs car_id defined with a different value.
How do I solve the error and union with virtual attributes?
I want an ActiveRecord Relation, and interesting enough:
SELECT users.*, 1 as car_id
FROM users
UNION
SELECT users.*, 2 as car_id
FROM users
works fine when running raw sql.
SQL Example:
a.mail_for(:inbox).name
=> Mailboxer::Conversation::ActiveRecord_Relation
a.mail_for(:inbox).to_sql
"SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM \"mailboxer_conversations\" INNER JOIN \"mailboxer_notifications\" ON \"mailboxer_notifications\".\"conversation_id\" = \"mailboxer_conversations\".\"id\" AND \"mailboxer_notifications\".\"type\" IN ('Mailboxer::Message') INNER JOIN \"mailboxer_receipts\" ON \"mailboxer_receipts\".\"notification_id\" = \"mailboxer_notifications\".\"id\" WHERE \"mailboxer_notifications\".\"type\" = 'Mailboxer::Message' AND \"mailboxer_receipts\".\"receiver_id\" = 102 AND \"mailboxer_receipts\".\"receiver_type\" = 'Listing' AND \"mailboxer_receipts\".\"mailbox_type\" = 'inbox' AND \"mailboxer_receipts\".\"trashed\" = FALSE AND \"mailboxer_receipts\".\"deleted\" = FALSE ORDER BY \"mailboxer_conversations\".\"updated_at\" DESC"
So I pass an array of relations to union_scope...
ar= a.mail_for(:inbox)
br= b.mail_for(:inbox)
cr= c.mail_for(:inbox)
combined = union_scope([[a,ar],[b, br],[c, cr])
def union_scope(*relation)
combined = relation.first[1].none
relation.each do |relation_set|
mailer = relation_set[0]
scope = relation_set[1].select("#{relation_set[1].table_name}.*, \'#{mailer.id}\' as mailer_id, \'#{mailer.class.name}\' as mailer_type")
combined = combined.or(scope)
end
combined
end
Update:
def union_scope(*relation)
combined = relation.first[1].none
relation.each do |relation_set|
mailer = relation_set[0]
scope = relation_set[1].select("#{relation_set[1].table_name}.*, \'#{mailer.id}\' as mailer_id, \'#{mailer.class.name}\' as mailer_type")
combined = combined.union(scope)
end
conv = ::Mailboxer::Conversation.arel_table
::Mailboxer::Conversation.from(conv.create_table_alias(combined, :conversations).to_sql)
end
Resulted in this error:
Mailboxer::Conversation Load (5.8ms) SELECT "mailboxer_conversations".* FROM ( SELECT DISTINCT "mailboxer_conversations".* FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $1 AND "mailboxer_receipts"."receiver_id" = $2 AND "mailboxer_receipts"."receiver_type" = $3 AND (1=0) ORDER BY "mailboxer_conversations"."updated_at" DESC UNION SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $4 AND "mailboxer_receipts"."receiver_id" = $5 AND "mailboxer_receipts"."receiver_type" = $6 ORDER BY "mailboxer_conversations"."updated_at" DESC ) "conversations"
ActiveRecord::StatementInvalid (PG::SyntaxError: ERROR: syntax error at or near "UNION")
LINE 1: ...ER BY "mailboxer_conversations"."updated_at" DESC UNION SELE...
^
: SELECT "mailboxer_conversations".* FROM ( SELECT DISTINCT "mailboxer_conversations".* FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $1 AND "mailboxer_receipts"."receiver_id" = $2 AND "mailboxer_receipts"."receiver_type" = $3 AND (1=0) ORDER BY "mailboxer_conversations"."updated_at" DESC UNION SELECT DISTINCT mailboxer_conversations.*, '102' as mailer_id, 'Listing' as mailer_type FROM "mailboxer_conversations" INNER JOIN "mailboxer_notifications" ON "mailboxer_notifications"."conversation_id" = "mailboxer_conversations"."id" AND "mailboxer_notifications"."type" IN ('Mailboxer::Message') INNER JOIN "mailboxer_receipts" ON "mailboxer_receipts"."notification_id" = "mailboxer_notifications"."id" WHERE "mailboxer_notifications"."type" = $4 AND "mailboxer_receipts"."receiver_id" = $5 AND "mailboxer_receipts"."receiver_type" = $6 ORDER BY "mailboxer_conversations"."updated_at" DESC ) "conversations"
While I cannot answer your actual question as to why these are incompatible I can offer a solution akin to your raw SQL example.
You can perform the same operation like so
user_table = User.arel_table
a = user_table.project(Arel.star, Arel.sql("1 as car_id")).take(2)
b = user_table.project(Arel.star, Arel.sql("2 as car_id")).take(2)
union = Arel::Nodes::UnionAll.new(a,b)
User.from(Arel::Nodes::As.new(union,user_table))
This will result in the following query.
SELECT
users.*
FROM
( (SELECT *, 1 as car_id
FROM users
ORDER BY
users.id ASC
LIMIT 2) UNION ALL (
SELECT *, 2 as car_id
FROM users
ORDER BY
users.id ASC
LIMIT 2)) AS users
Since this will still return an ActiveRecord::Relation you can still act on this object as you would any other we have simply substituted the normal data source (the users table) with your custom union data source of the same name.
Update based on extreme revision and some general assumptions as to what you actually meant to do
a= a.mail_for(:inbox)
b= b.mail_for(:inbox)
c= c.mail_for(:inbox)
combined = union_scope(a,b,c)
def union_scope(*relations)
base = build_scope(*relations.shift)
combined = relations.reduce(base) do |memo, relation_set|
Arel::Nodes::UnionAll.new(memo,build_scope(*relation_set))
end
union = Arel::Nodes::As.new(combined,::Mailboxer::Conversation.arel_table)
::Mailboxer::Conversation.from(union)
end
def build_scope(mailer,relation)
relation.select(
"#{relation.table_name}.*,
'#{mailer.id}' as mailer_id,
'#{mailer.class.name}' as mailer_type"
).arel
end
Sorry for long query but its what im working with.
I got this query in rails
#posts = Cama::PostType.first.posts.joins(:custom_field_values).where("(status = ?) AND (cama_custom_fields_relationships.custom_field_slug = ? AND
CAST(cama_custom_fields_relationships.value AS INTEGER) >= ? AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= ? ) OR
(cama_custom_fields_relationships.custom_field_slug = ? AND
CAST(cama_custom_fields_relationships.value AS INTEGER) >= ? AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= ? ) AND
(LOWER(title) LIKE ? OR LOWER(content_filtered) LIKE ?)","published", "filtry-
powierzchnia", params['size-start'].to_i, params['size-end'].to_i,"filtry-
cena", params['price-start'].to_i, params['price-end'].to_i
,"%#{params[:q]}%",
"%#{params[:q]}%").group("cama_posts.id").having("COUNT(cama_custom_fields_rel
ationships.objectid) >= 2")
that make this sql
CamaleonCms::Post Load (0.9ms) SELECT "cama_posts".* FROM "cama_posts"
INNER JOIN "cama_custom_fields_relationships" ON
"cama_custom_fields_relationships"."objectid" = "cama_posts"."id" AND
"cama_custom_fields_relationships"."object_class" = $1 WHERE
"cama_posts"."post_class" = $2 AND "cama_posts"."taxonomy_id" = $3 AND
((status = 'published') AND
(cama_custom_fields_relationships.custom_field_slug = 'filtry-powierzchnia'
AND CAST(cama_custom_fields_relationships.value AS INTEGER) >= 1 AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000 ) OR
(cama_custom_fields_relationships.custom_field_slug = 'filtry-cena' AND
CAST(cama_custom_fields_relationships.value AS INTEGER) >= 0 AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000000 ) AND
(LOWER(title) LIKE '%%' OR LOWER(content_filtered) LIKE '%%')) GROUP BY
cama_posts.id HAVING (COUNT(cama_custom_fields_relationships.objectid) >= 2)
ORDER BY "cama_posts"."post_order" ASC, "cama_posts"."created_at" DESC LIMIT
$4 OFFSET $5 [["object_class", "Post"], ["post_class", "Post"],
["taxonomy_id", 2], ["LIMIT", 6], ["OFFSET", 0]]
As you can see Im using CAST, I can't change column just to integer, because its CMS. I created custom_field that suppoust to be integer but table in db is still text so im forced to do it that way.
Now everything is ok if my
..CAST(cama_custom_fields_relationships.value AS INTEGER) >= 1..
but when this value will be 0
so when my query will look like
CamaleonCms::Post Load (0.9ms) SELECT "cama_posts".* FROM "cama_posts"
INNER JOIN "cama_custom_fields_relationships" ON
"cama_custom_fields_relationships"."objectid" = "cama_posts"."id" AND
"cama_custom_fields_relationships"."object_class" = $1 WHERE
"cama_posts"."post_class" = $2 AND "cama_posts"."taxonomy_id" = $3 AND
((status = 'published') AND
(cama_custom_fields_relationships.custom_field_slug = 'filtry-powierzchnia'
AND ****CAST(cama_custom_fields_relationships.value AS INTEGER) >= 0**** AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000 ) OR
(cama_custom_fields_relationships.custom_field_slug = 'filtry-cena' AND
CAST(cama_custom_fields_relationships.value AS INTEGER) >= 0 AND
CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000000 ) AND
(LOWER(title) LIKE '%%' OR LOWER(content_filtered) LIKE '%%')) GROUP BY
cama_posts.id HAVING (COUNT(cama_custom_fields_relationships.objectid) >= 2)
ORDER BY "cama_posts"."post_order" ASC, "cama_posts"."created_at" DESC LIMIT
$4 OFFSET $5 [["object_class", "Post"], ["post_class", "Post"],
["taxonomy_id", 2], ["LIMIT", 6], ["OFFSET", 0]]
Im getting error:
ActionView::Template::Error (PG::InvalidTextRepresentation: ERROR: invalid input syntax for integer: "/media/1/asd.jpg"
: SELECT "cama_posts".* FROM "cama_posts" INNER JOIN "cama_custom_fields_relationships" ON "cama_custom_fields_relationships"."objectid" = "cama_posts"."id" AND "cama_custom_fields_relationships"."object_class" = $1 WHERE "cama_posts"."post_class" = $2 AND "cama_posts"."taxonomy_id" = $3 AND ((status = 'published') AND (cama_custom_fields_relationships.custom_field_slug = 'filtry-powierzchnia' AND CAST(cama_custom_fields_relationships.value AS INTEGER) >= 0 AND CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000 ) OR (cama_custom_fields_relationships.custom_field_slug = 'filtry-cena' AND CAST(cama_custom_fields_relationships.value AS INTEGER) >= 0 AND CAST(cama_custom_fields_relationships.value AS INTEGER) <= 10000000 ) AND (LOWER(title) LIKE '%%' OR LOWER(content_filtered) LIKE '%%')) GROUP BY cama_posts.id HAVING (COUNT(cama_custom_fields_relationships.objectid) >= 2) ORDER BY "cama_posts"."post_order" ASC, "cama_posts"."created_at" DESC LIMIT $4 OFFSET $5):
and #posts return it on the query lvl so its not error that occurs somewhere else.
I tried CASTing to numeric, same problem. It only happens when its starting condition, when there is another condition before it, it accepts 0 like in my example, the second condition after this one works with >= 0
params = {device_id: "u100", device_point: "1", point_no: "1", max_date: "2016-12-01 12:00", min_date: "2016-12-01 11:40"}
I am working in a ruby/rails environment with a PG database.
I am trying to add to my device controller to use ActiveRecord to get 10 minute averages from a some dummy sensor data.
I can group by minute fine with;
.group("date_trunc('minute', device_time)").average('point_data_val')
e.g
device_data = DeviceDatum.select('device_time, point_data_val').filter(params.slice(:device_id, :device_point, :point_no, :iot_version, :iot_time, :device_time, :point_data_type, :point_data_val, :point_bat_val,:min_val, :max_val, :min_date, :max_date)).limit(1008).group("date_trunc('minute', device_time)").average('point_data_val')
(240.3ms) SELECT AVG("device_data"."point_data_val") AS average_point_data_val, date_trunc('minute', device_time) AS date_trunc_minute_device_time FROM "device_data" WHERE "device_data"."device_id" = $1 AND "device_data"."device_point" = $2 AND "device_data"."point_no" = $3 AND (device_time >= '2016-12-01 11:40') AND (device_time <= '2016-12-01 12:00') GROUP BY date_trunc('minute', device_time) LIMIT 1008 [["device_id", "u100"], ["device_point", 1], ["point_no", 1]]
=> {2016-12-01 11:41:00 +0900=>#<BigDecimal:7f91599cfbc0,'-0.4816E1',18(27)>, 2016-12-01 11:51:00 +0900=>#<BigDecimal:7f91599cf8f0,'-0.4868E1',18(27)>}
I found this method to do it
But when I try and use AS in the group method
DeviceDatum.select('device_time, point_data_val')
.filter(params.slice(:device_id, :device_point, :point_no, :iot_version, :iot_time, :device_time,
:point_data_type, :point_data_val, :point_bat_val,:min_val, :max_val, :min_date,
:max_date)).limit(1008)
.group("date_trunc('hour', device_time) AS hour_stump,
(extract(minute FROM device_time)::int / 10)
AS min10_slot,count(*)").average('point_data_val')
I am getting this error.
ActiveRecord::StatementInvalid: PG::SyntaxError: ERROR: syntax error at or near "AS"
LINE 1: ... 12:00') GROUP BY date_trunc('hour', device_time) AS hour_st... : SELECT AVG("device_data"."point_data_val") AS average_point_data_val, date_trunc('hour', device_time) AS hour_stump,(extract(minute FROM device_time)::int / 10) AS min10_slot,count(*) AS date_trunc_hour_device_time_as_hour_stump_extract_minute_from_d FROM "device_data" WHERE "device_data"."device_id" = $1 AND "device_data"."device_point" = $2 AND "device_data"."point_no" = $3 AND (device_time >= '2016-12-01 11:40') AND (device_time <= '2016-12-01 12:00') GROUP BY date_trunc('hour', device_time) AS hour_stump,(extract(minute FROM device_time)::int / 10) AS min10_slot,count(*) LIMIT 1008
from ...../bundle/ruby/2.1.0/gems/activerecord-4.2.6/lib/active_record/connection_adapters/postgresql_adapter.rb:637:in `prepare'
ActiveRecord seems to be using the options from the group method in sql it generates for average as well.
SELECT
AVG("device_data"."point_data_val") AS average_point_data_val,
date_trunc('hour', device_time) AS hour_stump,
(
extract(minute
FROM
device_time)::int / 10
)
AS min10_slot,
count(*) AS date_trunc_hour_device_time_as_hour_stump_extract_minute_from_d
FROM
"device_data"
WHERE
"device_data"."device_id" = $1
AND "device_data"."device_point" = $2
AND "device_data"."point_no" = $3
AND
(
device_time >= '2016-12-01 11:40'
)
AND
(
device_time <= '2016-12-01 12:00'
)
GROUP BY
date_trunc('hour', device_time) AS hour_stump,
(
extract(minute
FROM
device_time)::int / 10
)
AS min10_slot,
count(*) LIMIT 1008
What am I missing here?
I think that your group clause should look like this:
group("hour_stump, min10_slot,
date_trunc_hour_device_time_as_hour_stump_extract_minute_from_d")
if you use aliases in select clause. As #Fallenhero and #Iceman said, aliases in group by are not allowed.
If aliases are substituted in select automatically, I think the only way is to use functions as is
group("date_trunc('hour', device_time),
(extract(minute FROM device_time)::int / 10), count(*)")
.
I have two models: show_request and show. A show_Request belongs_to a show and a show has_many show_requests. On the show_request page in active_admin, I want to order show_requests by the show's created_at value. Here is my code so far:
ActiveAdmin.register ShowRequest do
controller do
def scoped_collection
end_of_association_chain.includes(:show)
#I also tried ShowRequest.includes(:show)
end
end
index do
column 'Show', sortable: "shows.created_at_asc" do |show_req|
link_to show_req.show.name, admin_show_path(show_req.show)
end
end
end
Here are the server logs:
Started GET "/admin/show_requests" for 127.0.0.1 at 2015-09-18 09:35:36 -0400
Processing by Admin::ShowRequestsController#index as HTML
AdminUser Load (0.3ms) SELECT "admin_users".* FROM "admin_users" WHERE "admin_users"."id" = 1 ORDER BY "admin_users"."id" ASC LIMIT 1
(1.2ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't')
(0.2ms) SELECT COUNT(*) FROM "show_requests"
(0.2ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 't' AND i_want_my_horse_to_compete = 'f')
(0.3ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
CACHE (0.0ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
CACHE (0.0ms) SELECT COUNT(*) FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't')
CACHE (0.0ms) SELECT COUNT(count_column) FROM (SELECT 1 AS count_column FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') LIMIT 30 OFFSET 0) subquery_for_count
ShowRequest Load (2.0ms) SELECT "show_requests".* FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') ORDER BY "show_requests"."id" desc LIMIT 30 OFFSET 0
Show Load (9.7ms) SELECT "shows".* FROM "shows" WHERE "shows"."id" IN (2, 1)
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 2]]
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 1]]
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 1]]
Show Load (0.2ms) SELECT "shows".* FROM "shows"
User Load (0.2ms) SELECT "users".* FROM "users"
This is not working. It is not affecting the order of the columns at all. How do I fix this?
Take a look at this part of the log:
ShowRequest Load (2.0ms) SELECT "show_requests".* FROM "show_requests" WHERE (not_going_to_show = 'f' OR i_want_my_horse_to_compete = 't') ORDER BY "show_requests"."id" desc LIMIT 30 OFFSET 0
Show Load (9.7ms) SELECT "shows".* FROM "shows" WHERE "shows"."id" IN (2, 1)
You can see that the ShowRequests and Shows are loaded in separate queries. sortable is not going to work here, because you can't order one table by a field of another without a join. The fix should be to tell ActiveRecord that you need to reference the shows table in your query:
controller do
def scoped_collection
super.includes(:show).references(:shows)
end
end
index do
column :show, sortable: 'shows.created_at'
end
references only works with includes, and forces Rails to perform a join when eager loading.