Common Table Expression is a fairly common practice in different RDBMS (PostgreSQL, MySQL, Oracle, SQLite3 etc.) to perform the same calculation multiple times over across multiple query components or for some other purposes
I found old gem postgres_ext with such functionality. But it is not maintained. And it is Postgres specific
There are some old questions about it, but they are about specific rails version or specific RDBMS or about Arel
How do you use the postgresql WITH in activerecord?
Rails 5.2.2 (active record) WITH statement
Postgres Common Table Expression query with Ruby on Rails
Arel and CTE to Wrap Query
Multiple CTEs with Arel
Is it possible to use WITH clause in Rails using AR some common way?
Rails 7.1 introduce with method
It returns ActiveRecord::Relation
Post.with(posts_with_tags: Post.where("tags_count > ?", 0))
# WITH posts_with_tags AS (
# SELECT * FROM posts WHERE (tags_count > 0)
# )
# SELECT * FROM posts
Once you define Common Table Expression you can use custom FROM value or JOIN to reference it
Post.with(posts_with_tags: Post.where("tags_count > ?", 0)).from("posts_with_tags AS posts")
# WITH posts_with_tags AS (
# SELECT * FROM posts WHERE (tags_count > 0)
# )
# SELECT * FROM posts_with_tags AS posts
Post.with(posts_with_tags: Post.where("tags_count > ?", 0)).joins("JOIN posts_with_tags ON posts_with_tags.id = posts.id")
# WITH posts_with_tags AS (
# SELECT * FROM posts WHERE (tags_count > 0)
# )
# SELECT * FROM posts JOIN posts_with_tags ON posts_with_tags.id = posts.id
It's possible to pass not only AR but also SQL literal using Arel.
NB: Great caution should be taken to avoid SQL injection vulnerabilities. This method should not be used with unsafe values that include unsanitized input
Post.with(popular_posts: Arel.sql("... complex sql to calculate posts popularity ..."))
To add multiple CTEs just pass multiple key-value pairs
Post.with(
posts_with_comments: Post.where("comments_count > ?", 0),
posts_with_tags: Post.where("tags_count > ?", 0)
)
or chain multiple .with calls
Post
.with(posts_with_comments: Post.where("comments_count > ?", 0))
.with(posts_with_tags: Post.where("tags_count > ?", 0))
Related
I have a query with a lot of joins and I'm eager_loading some of associations at the time. And I need to compute some value as attribute of one of models.
So, I'm trying this code:
ServiceObject
.joins([{service_days: :ou}, :address])
.eager_load(:address, :service_days)
.where(ous: {id: OU.where(sector_code: 5)})
.select('SDO_CONTAINS(ous.service_area_shape, SDO_GEOMETRY(2001, 8307, sdo_point_type(addresses.lat, addresses.lng, NULL), NULL, NULL) ) AS in_zone')
Where SQL function call in select operates data from associated addresses and ous tables.
I'm getting next SQL (so my in_zone column getting calculated and returned as first column before other columns for all eager_loaded models):
SELECT SDO_CONTAINS(ous.service_area_shape, SDO_GEOMETRY(2001, 8307, sdo_point_type(addresses.lat, addresses.lng, NULL), NULL, NULL) ) AS in_zone, "SERVICE_OBJECTS"."ID" AS t0_r0, "SERVICE_OBJECTS"."TYPE" AS t0_r1, <omitted for brevity> AS t2_r36 FROM "SERVICE_OBJECTS" INNER JOIN "SERVICE_DAYS" ON "SERVICE_DAYS"."SERVICE_OBJECT_ID" = "SERVICE_OBJECTS"."ID" INNER JOIN "OUS" ON "OUS"."ID" = "SERVICE_DAYS"."OU_ID" INNER JOIN "ADDRESSES" ON "ADDRESSES"."ID" = "SERVICE_OBJECTS"."ADDRESS_ID" WHERE "OUS"."ID" IN (SELECT "OUS"."ID" FROM "OUS" WHERE "OUS"."SECTOR_CODE" = :a1) [["sector_code", "5"]]
But it seems like that in_zone isn't accessible from either model used in query.
I need to have calculated in_zone as attribute of ServiceObject model object, how I can accomplish that?
Ruby on Rails 4.2.6, Ruby 2.3.0, oracle_enhanced adapter 1.6.7, Oracle 12.1
I have successfully replicated your issue and it turns out that this is a known issue in Rails. The problem is that when using eager_load, Rails maps the columns of all eager-loaded tables into table and column aliases in the form of t0_r0, t0_r1, etc... (you can see these in the SQL that you pasted in the question). And while doing that, it simply ignores the custom columns in the select, probably because it cannot determine which eager-loaded table it should attribute the custom column to. It is sad that this issue is open for more than 2 years now...
Nevertheless I think I found a workaround. It seems that if you don't eager load the tables but manually join them (with joins), you can as well include them (with includes) and the custom columns will be returned as there will be no column aliasing taking place. The point is that you must not use associations in the joins clauses but you have to specify the joins yourself. Also note that you must specify all columns from the main table in the select manually too (see the service_objects.* in the select).
Try the following approach:
ServiceObject
.joins('INNER JOIN "SERVICE_DAYS" ON "SERVICE_DAYS"."SERVICE_OBJECT_ID" = "SERVICE_OBJECTS"."ID"')
.joins('INNER JOIN "OUS" ON "OUS"."ID" = "SERVICE_DAYS"."OU_ID"')
.joins('INNER JOIN "ADDRESSES" ON "ADDRESSES"."ID" = "SERVICE_OBJECTS"."ADDRESS_ID"')
.includes(:service_days, :address)
.where(ous: {id: OU.where(sector_code: 5)})
.select('service_objects.*, SDO_CONTAINS(ous.service_area_shape, SDO_GEOMETRY(2001, 8307, sdo_point_type(addresses.lat, addresses.lng, NULL), NULL, NULL) ) AS in_zone')
The computation in the select should still work as the related tables are joined together but there should be no column aliasing present.
Of course this approach means that you'll get three queries instead of just one but unless you return a huge amount of records, the following two queries run by the includes clause should be very fast as they simply load the relevant records using foreign keys.
That monkey patch helped #Envek:
module ActiveRecord
Base.send :attr_accessor, :_row_
module Associations
class JoinDependency
JoinBase && class JoinPart
def instantiate_with_row(row, *args)
instantiate_without_row(row, *args).tap { |i| i._row_ = row }
end; alias_method_chain :instantiate, :row
end
end
end
end
then it is possible to do:
ServiceObject
.joins([{service_days: :ou}, :address])
.eager_load(:address, :service_days)
.where(ous: {id: OU.where(sector_code: 5)})
.select('SDO_CONTAINS(ous.service_area_shape, SDO_GEOMETRY(2001, 8307, sdo_point_type(addresses.lat, addresses.lng, NULL), NULL, NULL) ) AS in_zone')
.first
._row_['in_zone']
I'm trying to join two different conditions on my Active Record Database.
Usually I'll join them with a comma (,) or AND, however in this case, putting a comma or AND between conditions is returning an error.
I wonder how to join those conditions together in one statement:
MyTable.where("created_at < ?", Time.now - 86400) # 86400 represent a day
MyTable.where(my_column: my_variable)
Thanks.
UPDATE
I'm using rails 4.
MyTable.where("created_at < ? AND my_column = ?", Time.now - 86400, my_variable)
I have these two pieces of code that each return a relation inside the Micropost model.
scope :including_replies, lambda { |user| where("microposts.in_reply_to = ?", user.id)}
def self.from_users_followed_by(user)
followed_user_ids = user.followed_user_ids
where("user_id IN (?) OR user_id = ?", followed_user_ids, user)
end
When I run r1 = Micropost.including_replies(user) I get a relation with two results with the following SQL:
SELECT `microposts`.* FROM `microposts` WHERE (microposts.in_reply_to = 102) ORDER BY
microposts.created_at DESC
When I run r2 = Micropost.from_users_followed_by(user) I get a relation with one result with the following SQL:
SELECT `microposts`.* FROM `microposts` WHERE (user_id IN (NULL) OR user_id = 102) ORDER
BY microposts.created_at DESC
Now when I merge the relations like so r3 = r1.merge(r2) I got zero results but was expecting three. The reason for this is that the SQL looks like this:
SELECT `microposts`.* FROM `microposts` WHERE (microposts.in_reply_to = 102) AND
(user_id IN (NULL) OR user_id = 102) ORDER BY microposts.created_at DESC
Now what I need is (microposts.in_reply_to = 102) OR (user_id IN (NULL) OR user_id = 102)
I need an OR instead of an AND in the merged relation.
Is there a way to do this?
Not directly with Rails. Rails does not expose any way to merge ActiveRelation (scoped) objects with OR. The reason is that ActiveRelation may contain not only conditions (what is described in the WHERE clause), but also joins and other SQL clauses for which merging with OR is not well-defined.
You can do this either with Arel directly (which ActiveRelation is built on top of), or you can use Squeel, which exposes Arel functionality through a DSL (which may be more convenient). With Squeel, it is still relevant that ActiveRelations cannot be merged. However Squeel also provides Sifters, which represent conditions (without any other SQL clauses), which you can use. It would involve rewriting the scopes as sifters though.
There are 3 tables payment_logs, sourcings and purchasings in our rails app. A payment_log belongs to either sourcing or purchasing but not both at the same time. There is a col project_id in both sourcing and purchasing. We want to pick up all payment_logs with its project_id = project_id_search (project_id_search passed from a search page). Also we need a ActiveRecord as resultset returned. Here is the individual query, assuming payment_logs holds the ActiveRecord result set:
pick all payment_logs with its sourcing's project_id = project_id_search
payment_logs = payment_logs.joins(:sourcing).where("sourcings.project_id = ?", project_id_search)
pick all payment_logs with its purchasing's project_id = project_id_search
payment_logs = payment_logs.(:purchasing).where("purchasings.project_id = ?", project_id_search)
We need to union 1 and 2 in order to pick up all the payment_logs whose project_id = project_id_search. What's the right way to accomplish it? We did not find union in rails and find_by_sql returns an array which is not what we want. Thanks.
payment_logs.where(["
payment_logs.sourcing_id IN (
SELECT id FROM sourcings WHERE sourcings.project_id = ?
)
OR payment_logs.purchasing_id IN
(
SELECT id FROM purchasings WHERE purchasings.project_id = ?
)", project_id_search, project_id_search])
Lot of SQL, but it should work
Option 2 (two SQL requests ...) :
payment_logs = []
payment_logs << PaymentLog.joins(:sourcing).where("sourcings.project_id" => project_id_search)
payment_logs << PaymentLog.joins(:purchasing).where("purchasings.project_id" => project_id_search)
payment_logs.uniq! #In case some records have both a sourcing and a purchasing
Option 3, with the squeel gem : https://github.com/ernie/squeel
PaymentLog.where{(source_id.in Sourcing.where(:project_id => project_id_search)) | (purchasing_id.in Purchasing.where(:project_id => project_id_search))}
I like this solution :)
Also, whenever you have a doubt on the generated SQL, from the console or anywhere else, you can add .to_sql at the end of an ActiveRecord query to double check the generated SQL
How do you do an OR query in Rails 3 ActiveRecord. All the examples I find just have AND queries.
Edit: OR method is available since Rails 5. See ActiveRecord::QueryMethods
If you want to use an OR operator on one column's value, you can pass an array to .where and ActiveRecord will use IN(value,other_value):
Model.where(:column => ["value", "other_value"]
outputs:
SELECT `table_name`.* FROM `table_name` WHERE `table_name`.`column` IN ('value', 'other_value')
This should achieve the equivalent of an OR on a single column
in Rails 3, it should be
Model.where("column = ? or other_column = ?", value, other_value)
This also includes raw sql but I dont think there is a way in ActiveRecord to do OR operation. Your question is not a noob question.
Rails 5 added or, so this is easier now in an app with Rails version greater than 5:
Model.where(column: value).or(Model.where(other_column: other_value)
this handles nil values as well
Use ARel
t = Post.arel_table
results = Post.where(
t[:author].eq("Someone").
or(t[:title].matches("%something%"))
)
The resulting SQL:
ree-1.8.7-2010.02 > puts Post.where(t[:author].eq("Someone").or(t[:title].matches("%something%"))).to_sql
SELECT "posts".* FROM "posts" WHERE (("posts"."author" = 'Someone' OR "posts"."title" LIKE '%something%'))
An updated version of Rails/ActiveRecord may support this syntax natively. It would look similar to:
Foo.where(foo: 'bar').or.where(bar: 'bar')
As noted in this pull request https://github.com/rails/rails/pull/9052
For now, simply sticking with the following works great:
Foo.where('foo= ? OR bar= ?', 'bar', 'bar')
Update: According to https://github.com/rails/rails/pull/16052 the or feature will be available in Rails 5
Update: Feature has been merged to Rails 5 branch
Rails has recently added this into ActiveRecord. It looks to be released in Rails 5. Committed to master already:
https://github.com/rails/rails/commit/9e42cf019f2417473e7dcbfcb885709fa2709f89
Post.where(column: 'something').or(Post.where(other: 'else'))
# => SELECT * FROM posts WHERE (column = 'something') OR (other = 'else)
Rails 5 comes with an or method. (link to documentation)
This method accepts an ActiveRecord::Relation object. eg:
User.where(first_name: 'James').or(User.where(last_name: 'Scott'))
If you want to use arrays as arguments, the following code works in Rails 4:
query = Order.where(uuid: uuids, id: ids)
Order.where(query.where_values.map(&:to_sql).join(" OR "))
#=> Order Load (0.7ms) SELECT "orders".* FROM "orders" WHERE ("orders"."uuid" IN ('5459eed8350e1b472bfee48375034103', '21313213jkads', '43ujrefdk2384us') OR "orders"."id" IN (2, 3, 4))
More information: OR queries with arrays as arguments in Rails 4.
The MetaWhere plugin is completely amazing.
Easily mix OR's and AND's, join conditions on any association, and even specify OUTER JOIN's!
Post.where({sharing_level: Post::Sharing[:everyone]} | ({sharing_level: Post::Sharing[:friends]} & {user: {followers: current_user} }).joins(:user.outer => :followers.outer}
Just add an OR in the conditions
Model.find(:all, :conditions => ["column = ? OR other_column = ?",value, other_value])
With rails + arel, a more clear way:
# Table name: messages
#
# sender_id: integer
# recipient_id: integer
# content: text
class Message < ActiveRecord::Base
scope :by_participant, ->(user_id) do
left = arel_table[:sender_id].eq(user_id)
right = arel_table[:recipient_id].eq(user_id)
where(Arel::Nodes::Or.new(left, right))
end
end
Produces:
$ Message.by_participant(User.first.id).to_sql
=> SELECT `messages`.*
FROM `messages`
WHERE `messages`.`sender_id` = 1
OR `messages`.`recipient_id` = 1
You could do it like:
Person.where("name = ? OR age = ?", 'Pearl', 24)
or more elegant, install rails_or gem and do it like:
Person.where(:name => 'Pearl').or(:age => 24)
I just extracted this plugin from client work that lets you combine scopes with .or., ex. Post.published.or.authored_by(current_user). Squeel (newer implementation of MetaSearch) is also great, but doesn't let you OR scopes, so query logic can get a bit redundant.
I'd like to add this is a solution to search multiple attributes of an ActiveRecord. Since
.where(A: param[:A], B: param[:B])
will search for A and B.
Using the activerecord_any_of gem, you can write
Book.where.any_of(Book.where(:author => 'Poe'), Book.where(:author => 'Hemingway')