How do i write this query in rails active record style
SELECT COUNT(o.id) no_of_orders,
SUM(o.total) total,
SUM(o.shipping) shipping
FROM orders o JOIN
(
SELECT DISTINCT order_id
FROM designer_orders
WHERE state IN('pending', 'dispatched', 'completed')
) d
ON o.id = d.order_id
You can do with an explicit query, you have 2 manner to do that:
Model.where("MYSQL_QUERY")
or
Model.find_by_sql("MYSQL_QUERY")
http://apidock.com/rails/ActiveRecord/Base/find_by_sql/class
OR
In Rails Style with a little more steps (probably can be done with less):
order_ids = DesignerOrder.where("state IN (?)", ['pending', 'dispatched', 'completed']).select(:order_id).distinct
partial_result = Order.where("id IN (?)", order_ids)
no_of_orders = partial_result.count
total_sum = partial_result.sum(:total)
shipping_sum = partial_result.sum(:shipping)
You can also do it like this
Order
.select('
COUNT(o.id) no_of_orders,
SUM(o.total) total,
SUM(o.shipping) shipping
')
.from('orders o')
.joins("
(#{
DesignerOrders
.select("DISTINCT order_id")
.where("state IN('pending', 'dispatched', 'completed')")
}) d on o.id = d.order_id
")
I didn't actually run this but the concept is valid. You don't even need an active record model if you use 'from'. We've used techniques like this to do AR style queries for extremely complex SQL and it's made our lives a lot easier.
Related
Using active record, I want to perform a lookup that returns a collection of items that have ALL matching id's.
Given that the below example matches on ANY id in the array, I am trying to figure out the syntax so that it will match when ALL of the id's match. (given that in this example there is a many to many relationship).
The array length of the id's is also variable which prohibits chaining .where()
x.where(id: [1,2])
Note: this question got removed before and there are a lot of answers for performing a sql "where in" but this question is about performing a sql "where and"
You can use exec_query and execute your own bound query:
values = [1, 2]
where_condition = values.map.with_index(1) { |_, index| "id = $#{index}" }.join(" AND ")
sql = "SELECT * FROM table WHERE #{ where_condition }"
binds = values.map { |i| ActiveRecord::Relation::QueryAttribute.new(nil, i, ActiveRecord::Type::Integer.new) }
ActiveRecord::Base.connection.exec_query(sql, nil, binds)
I completely agree with #muistooshort's comment
where(id: [1,2]) doesn't make sense unless you're joining to an association table and in that case,..."where in" combined with HAVING [solves your problem].
But for the sake of answering the question and the assumption that id was just and example.
While #SebastianPalma's answer will work it will return an ActiveRecord::Result whereas most of the time the desire is an ActiveRecord::Relation.
We can achieve this by using Arel to build the where clause like so:
(I modified the example to use description rather than id so that it makes more logical sense)
table = MyObject.arel_table
values = ['Jamesla','Example']
where_clause = values.map {|v| table[:description].matches("%{v}%")}.reduce(&:and)
# OR
where_clause = table[:description].matches_all(values.map {|v| "%#{v}%"})
MyObject.where(where_clause)
This will result in the following SQL query:
SELECT
my_objects.*
FROM
my_objects
WHERE
my_objects.description LIKE '%Jamesla%'
AND my_objects.description LIKE '%Example%'
Is there a way to programatically create a where clause in Arel where the columns and values are specified separately?
SELECT users.*
WHERE (country, occupation) IN (('dk', 'nurse'), ('ch', 'doctor'), ...
Say the input is a really long list of pairs that we want to match.
I'm am NOT asking how to generate a WHERE AND OR clause which is really simple to do with ActiveRecord.
So far I just have basic string manipulation:
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
User.where(
"(#{columns.join(', ')}) IN (#{ pairs.map { '(?, ?)' }.join(', ')})",
*pairs
)
Its not just about the length of the query WHERE (columns) IN (values) will also perform much better on Postgres (and others as well) as it can use an index only scan where OR will cause a bitmap scan.
I'm only looking for answers that can demonstrate generating a WHERE (columns) IN (values) query with Arel. Not anything else.
All the articles I have read about Arel start building of a single column:
arel_table[:foo].eq...
And I have not been able to find any documentation or articles that cover this case.
The trick to this is to build the groupings correctly and then pass them through to the Arel In Node, for example:
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
User.where(
Arel::Nodes::In.new(
Arel::Nodes::Grouping.new( columns.map { |column| User.arel_table[column] } ),
pairs.map { |pair| Arel::Nodes::Grouping.new(
pair.map { |value| Arel::Nodes.build_quoted(value) }
)}
)
)
The above will generate the following SQL statement (for MySQL):
"SELECT users.* FROM users WHERE (users.country,
users.occupation) IN (('dk', 'nurse'), ('ch', 'doctor'))"
This will still generate long query with 'OR' in between. But I felt this is lil elegant/different approach to achieve what you want.
ut = User.arel_table
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
where_condition = pairs.map do |pair|
"(#{ut[columns[0]].eq(pair[0]).and(ut[columns[1]].eq(pair[1])).to_sql})"
end.join(' OR ')
User.where(where_condition)
I have tried this different approach at my end. Hope it will work for you.
class User < ActiveRecord::Base
COLUMNS = %i(
country
occupation
)
PAIRS = [['dk', 'nurse'], ['ch', 'doctor']]
scope :with_country_occupation, -> (pairs = PAIRS, columns = COLUMNS) { where(filter_country_occupation(pairs, columns)) }
def self.filter_country_occupation(pairs, columns)
pairs.each_with_index.reduce(nil) do |query, (pair, index)|
column_check = arel_table[columns[0]].eq(pair[0]).and(arel_table[columns[1]].eq(pair[1]))
if query.nil?
column_check
else
query.or(column_check)
end
end.to_sql
end
end
Call this scope User.with_country_occupation let me know if it works for you.
Thanks!
I think we can do this with Array Conditions as mentioned here
# notice the lack of an array as the last argument
Model.where("attribute = ? OR attribute2 = ?", value, value)
Also, as mentioned here we can use an SQL in statement:
Model.where('id IN (?)', [array of values])
Or simply, as kdeisz pointed out (Using Arel to create the SQL query):
Model.where(id: [array of values])
I have not tried myself, but you can try exploring with these examples.
Always happy to help!
I'm resolving all the SQL Injections in a system and I've found something that I don't know how to treat.
Can somebody help me?
Here is my method
def get_structure()
#build query
sql = %(
SELECT pc.id AS "product_id", pc.code AS "code", pc.description AS "description", pc.family AS "family",
p.code AS "father_code", p.description AS "father_description",
p.family AS "father_family"
FROM products pc
LEFT JOIN imported_structures imp ON pc.id = imp.product_id
LEFT JOIN products p ON imp.product_father_id = p.id
WHERE pc.enable = true AND p.enable = true
)
#verify if there is any filter
if !params[:code].blank?
sql = sql + " AND UPPER(pc.code) LIKE '%#{params[:code].upcase}%'"
end
#many other parameters like the one above
#execute query
str = ProductStructure.find_by_sql(sql)
end
Thank you!
You could use Arel which will escape for you, and is the underlying query builder for ActiveRecord/Rails. eg.
products = Arel::Table.new("products")
products2 = Arel::Table.new("products", as: 'p')
imported_structs = Arel::Table.new("imported_structures")
query = products.project(
products[:id].as('product_id'),
products[:code],
products[:description],
products[:family],
products2[:code].as('father_code'),
products2[:description].as('father_description'),
products2[:family].as('father_family')).
join(imported_structs,Arel::Nodes::OuterJoin).
on(imported_structs[:product_id].eq(products[:id])).
join(products2,Arel::Nodes::OuterJoin).
on(products2[:id].eq(imported_structs[:product_father_id])).
where(products[:enable].eq(true).and(products2[:enable].eq(true)))
if !params[:code].blank?
query.where(
Arel::Nodes::NamedFunction.new('UPPER',[products[:code]])
.matches("%#{params[:code].to_s.upcase}%")
)
end
SQL result: (with params[:code] = "' OR 1=1 --test")
SELECT
[products].[id] AS product_id,
[products].[code],
[products].[description],
[products].[family],
[p].[code] AS father_code,
[p].[description] AS father_description,
[p].[family] AS father_family
FROM
[products]
LEFT OUTER JOIN [imported_structures] ON [imported_structures].[product_id] = [products].[id]
LEFT OUTER JOIN [products] [p] ON [p].[id] = [imported_structures].[product_father_id]
WHERE
[products].[enable] = true AND
[p].[enable] = true AND
UPPER([products].[code]) LIKE N'%'' OR 1=1 --test%'
To use
ProductStructure.find_by_sql(query.to_sql)
I prefer Arel, when available, over String queries because:
it supports escaping
it leverages your existing connection adapter for sytnax (so it is portable if you change databases)
it is built in code so statement order does not matter
it is far more dynamic and maintainable
it is natively supported by ActiveRecord
you can build any complex query you can possibly imagine (including complex joins, CTEs, etc.)
it is still very readable
You need to turn that into a placeholder value (?) and add the data as a separate argument. find_by_sql can take an array:
def get_structure
#build query
sql = %(SELECT...)
query = [ sql ]
if !params[:code].blank?
sql << " AND UPPER(pc.code) LIKE ?"
query << "%#{params[:code].upcase}%"
end
str = ProductStructure.find_by_sql(query)
end
Note, use << on String in preference to += when you can as it avoids making a copy.
I have a Company model that has_many Statement.
class Company < ActiveRecord::Base
has_many :statements
end
I want to get statements that have most latest date field grouped by fiscal_year_end field.
I implemented the function like this:
c = Company.first
c.statements.to_a.group_by{|s| s.fiscal_year_end }.map{|k,v| v.max_by(&:date) }
It works ok, but if possible I want to use ActiveRecord query(SQL), so that I don't need to load unnecessary instance to memory.
How can I write it by using SQL?
select t.username, t.date, t.value
from MyTable t
inner join (
select username, max(date) as MaxDate
from MyTable
group by username
) tm on t.username = tm.username and t.date = tm.MaxDate
For these kinds of things, I find it helpful to get the raw SQL working first, and then translate it into ActiveRecord afterwards. It sounds like a textbook case of GROUP BY:
SELECT fiscal_year_end, MAX(date) AS max_date
FROM statements
WHERE company_id = 1
GROUP BY fiscal_year_end
Now you can express that in ActiveRecord like so:
c = Company.first
c.statements.
group(:fiscal_year_end).
order(nil). # might not be necessary, depending on your association and Rails version
select("fiscal_year_end, MAX(date) AS max_date")
The reason for order(nil) is to prevent ActiveRecord from adding ORDER BY id to the query. Rails 4+ does this automatically. Since you aren't grouping by id, it will cause the error you're seeing. You could also order(:fiscal_year_end) if that is what you want.
That will give you a bunch of Statement objects. They will be read-only, and every attribute will be nil except for fiscal_year_end and the magically-present new field max_date. These instances don't represent specific statements, but statement "groups" from your query. So you can do something like this:
- #statements_by_fiscal_year_end.each do |s|
%tr
%td= s.fiscal_year_end
%td= s.max_date
Note there is no n+1 query problem here, because you fetched everything you need in one query.
If you decide that you need more than just the max date, e.g. you want the whole statement with the latest date, then you should look at your options for the greatest n per group problem. For raw SQL I like LATERAL JOIN, but the easiest approach to use with ActiveRecord is DISTINCT ON.
Oh one more tip: For debugging weird errors, I find it helpful to confirm what SQL ActiveRecord is trying to use. You can use to_sql to get that:
c = Company.first
puts c.statements.
group(:fiscal_year_end).
select("fiscal_year_end, MAX(date) AS max_date").
to_sql
In that example, I'm leaving off order(nil) so you can see that ActiveRecord is adding an ORDER BY clause you don't want.
for example you want to get all statements by start of the months you should use this
#companey = Company.first
#statements = #companey.statements.find(:all, :order => 'due_at, id', :limit => 50)
then group them as you want
#monthly_statements = #statements.group_by { |statement| t.due_at.beginning_of_month }
Building upon Bharat's answer you can do this type of query in Rails using find_by_sql in this way:
Statement.find_by_sql ["Select t.* from statements t INNER JOIN (
SELECT fiscal_year_end, max(date) as MaxDate GROUP BY fiscal_year_end
) tm on t.fiscal_year_end = tm.fiscal_year_end AND
t.created_at = tm.MaxDate WHERE t.company_id = ?", company.id]
Note the last where part to make sure the statements belong to a specific company instance, and that this is called from the class. I haven't tested this with the array form, but I believe you can turn this into a scope and use it like this:
# In Statement model
scope :latest_from_fiscal_year, lambda |enterprise_id| {
find_by_sql[..., enterprise_id] # Query above
}
# Wherever you need these statements for a particular company
company = Company.find(params[:id])
latest_statements = Statement.latest_from_fiscal_year(company.id)
Note that if you somehow need all the latest statements for all companies then this most likely leave you with a N+1 queries problem. But that is a beast for another day.
Note: If anyone else has a way to have this query work on the association without using the last where part (company.statements.latest_from_year and such) let me know and I'll edit this, in my case in rails 3 it just pulled em from the whole table without filtering.
There are 3 tables payment_logs, sourcings and purchasings in our rails app. A payment_log belongs to either sourcing or purchasing but not both at the same time. There is a col project_id in both sourcing and purchasing. We want to pick up all payment_logs with its project_id = project_id_search (project_id_search passed from a search page). Also we need a ActiveRecord as resultset returned. Here is the individual query, assuming payment_logs holds the ActiveRecord result set:
pick all payment_logs with its sourcing's project_id = project_id_search
payment_logs = payment_logs.joins(:sourcing).where("sourcings.project_id = ?", project_id_search)
pick all payment_logs with its purchasing's project_id = project_id_search
payment_logs = payment_logs.(:purchasing).where("purchasings.project_id = ?", project_id_search)
We need to union 1 and 2 in order to pick up all the payment_logs whose project_id = project_id_search. What's the right way to accomplish it? We did not find union in rails and find_by_sql returns an array which is not what we want. Thanks.
payment_logs.where(["
payment_logs.sourcing_id IN (
SELECT id FROM sourcings WHERE sourcings.project_id = ?
)
OR payment_logs.purchasing_id IN
(
SELECT id FROM purchasings WHERE purchasings.project_id = ?
)", project_id_search, project_id_search])
Lot of SQL, but it should work
Option 2 (two SQL requests ...) :
payment_logs = []
payment_logs << PaymentLog.joins(:sourcing).where("sourcings.project_id" => project_id_search)
payment_logs << PaymentLog.joins(:purchasing).where("purchasings.project_id" => project_id_search)
payment_logs.uniq! #In case some records have both a sourcing and a purchasing
Option 3, with the squeel gem : https://github.com/ernie/squeel
PaymentLog.where{(source_id.in Sourcing.where(:project_id => project_id_search)) | (purchasing_id.in Purchasing.where(:project_id => project_id_search))}
I like this solution :)
Also, whenever you have a doubt on the generated SQL, from the console or anywhere else, you can add .to_sql at the end of an ActiveRecord query to double check the generated SQL