There are 3 tables payment_logs, sourcings and purchasings in our rails app. A payment_log belongs to either sourcing or purchasing but not both at the same time. There is a col project_id in both sourcing and purchasing. We want to pick up all payment_logs with its project_id = project_id_search (project_id_search passed from a search page). Also we need a ActiveRecord as resultset returned. Here is the individual query, assuming payment_logs holds the ActiveRecord result set:
pick all payment_logs with its sourcing's project_id = project_id_search
payment_logs = payment_logs.joins(:sourcing).where("sourcings.project_id = ?", project_id_search)
pick all payment_logs with its purchasing's project_id = project_id_search
payment_logs = payment_logs.(:purchasing).where("purchasings.project_id = ?", project_id_search)
We need to union 1 and 2 in order to pick up all the payment_logs whose project_id = project_id_search. What's the right way to accomplish it? We did not find union in rails and find_by_sql returns an array which is not what we want. Thanks.
payment_logs.where(["
payment_logs.sourcing_id IN (
SELECT id FROM sourcings WHERE sourcings.project_id = ?
)
OR payment_logs.purchasing_id IN
(
SELECT id FROM purchasings WHERE purchasings.project_id = ?
)", project_id_search, project_id_search])
Lot of SQL, but it should work
Option 2 (two SQL requests ...) :
payment_logs = []
payment_logs << PaymentLog.joins(:sourcing).where("sourcings.project_id" => project_id_search)
payment_logs << PaymentLog.joins(:purchasing).where("purchasings.project_id" => project_id_search)
payment_logs.uniq! #In case some records have both a sourcing and a purchasing
Option 3, with the squeel gem : https://github.com/ernie/squeel
PaymentLog.where{(source_id.in Sourcing.where(:project_id => project_id_search)) | (purchasing_id.in Purchasing.where(:project_id => project_id_search))}
I like this solution :)
Also, whenever you have a doubt on the generated SQL, from the console or anywhere else, you can add .to_sql at the end of an ActiveRecord query to double check the generated SQL
Related
I have a Company model that has_many Statement.
class Company < ActiveRecord::Base
has_many :statements
end
I want to get statements that have most latest date field grouped by fiscal_year_end field.
I implemented the function like this:
c = Company.first
c.statements.to_a.group_by{|s| s.fiscal_year_end }.map{|k,v| v.max_by(&:date) }
It works ok, but if possible I want to use ActiveRecord query(SQL), so that I don't need to load unnecessary instance to memory.
How can I write it by using SQL?
select t.username, t.date, t.value
from MyTable t
inner join (
select username, max(date) as MaxDate
from MyTable
group by username
) tm on t.username = tm.username and t.date = tm.MaxDate
For these kinds of things, I find it helpful to get the raw SQL working first, and then translate it into ActiveRecord afterwards. It sounds like a textbook case of GROUP BY:
SELECT fiscal_year_end, MAX(date) AS max_date
FROM statements
WHERE company_id = 1
GROUP BY fiscal_year_end
Now you can express that in ActiveRecord like so:
c = Company.first
c.statements.
group(:fiscal_year_end).
order(nil). # might not be necessary, depending on your association and Rails version
select("fiscal_year_end, MAX(date) AS max_date")
The reason for order(nil) is to prevent ActiveRecord from adding ORDER BY id to the query. Rails 4+ does this automatically. Since you aren't grouping by id, it will cause the error you're seeing. You could also order(:fiscal_year_end) if that is what you want.
That will give you a bunch of Statement objects. They will be read-only, and every attribute will be nil except for fiscal_year_end and the magically-present new field max_date. These instances don't represent specific statements, but statement "groups" from your query. So you can do something like this:
- #statements_by_fiscal_year_end.each do |s|
%tr
%td= s.fiscal_year_end
%td= s.max_date
Note there is no n+1 query problem here, because you fetched everything you need in one query.
If you decide that you need more than just the max date, e.g. you want the whole statement with the latest date, then you should look at your options for the greatest n per group problem. For raw SQL I like LATERAL JOIN, but the easiest approach to use with ActiveRecord is DISTINCT ON.
Oh one more tip: For debugging weird errors, I find it helpful to confirm what SQL ActiveRecord is trying to use. You can use to_sql to get that:
c = Company.first
puts c.statements.
group(:fiscal_year_end).
select("fiscal_year_end, MAX(date) AS max_date").
to_sql
In that example, I'm leaving off order(nil) so you can see that ActiveRecord is adding an ORDER BY clause you don't want.
for example you want to get all statements by start of the months you should use this
#companey = Company.first
#statements = #companey.statements.find(:all, :order => 'due_at, id', :limit => 50)
then group them as you want
#monthly_statements = #statements.group_by { |statement| t.due_at.beginning_of_month }
Building upon Bharat's answer you can do this type of query in Rails using find_by_sql in this way:
Statement.find_by_sql ["Select t.* from statements t INNER JOIN (
SELECT fiscal_year_end, max(date) as MaxDate GROUP BY fiscal_year_end
) tm on t.fiscal_year_end = tm.fiscal_year_end AND
t.created_at = tm.MaxDate WHERE t.company_id = ?", company.id]
Note the last where part to make sure the statements belong to a specific company instance, and that this is called from the class. I haven't tested this with the array form, but I believe you can turn this into a scope and use it like this:
# In Statement model
scope :latest_from_fiscal_year, lambda |enterprise_id| {
find_by_sql[..., enterprise_id] # Query above
}
# Wherever you need these statements for a particular company
company = Company.find(params[:id])
latest_statements = Statement.latest_from_fiscal_year(company.id)
Note that if you somehow need all the latest statements for all companies then this most likely leave you with a N+1 queries problem. But that is a beast for another day.
Note: If anyone else has a way to have this query work on the association without using the last where part (company.statements.latest_from_year and such) let me know and I'll edit this, in my case in rails 3 it just pulled em from the whole table without filtering.
I am trying to find the best way to include a referenced model on what is essentially a compound key.
I have ChecklistItem (a list of things to do daily) and then ChecklistChecks (which ties the ChecklistItem together with a User for a particular day. These checklists can either be for all Stores (with a null store_id) or for a particular Store.
This pulls all of the ChecklistItems and their associated checks:
ChecklistItem.includes(:checklist_checks).where(store_id: [nil,#store.id], list_type: 'open')
The problem is that there would be checks from multiple days in there. What I need is to pull all of the ChecklistItems and any checks from a specific day.
I tried adding conditions like this:
ChecklistItem.includes(:checklist_checks).where(store_id: [nil,#store.id], list_type: 'open', checklist_checks: {store_id: #store.id, report_date: #today})
The problem is that will only pull ChecklistItems that have an associated ChecklistCheck.
It is generating SQL that is essentially:
SELECT
checklist_items.*,
checklist_checks.*
FROM
checklist_items
LEFT OUTER JOIN
checklist_checks
ON
checklist_checks.checklist_item_id = checklist_items.id
WHERE
checklist_items.list_type = 'open'
AND
checklist_checks.store_id = 1
AND
checklist_checks.report_date = '2015-05-03'
AND
(checklist_items.store_id = 1 OR checklist_items.store_id IS NULL)
I think the problem is that the conditions on checklist_checks is in the WHERE clause. If I could move them to the ON clause of the join, everything would work.
Is there a Rails way to end up with something like this?
SELECT
checklist_items.*,
checklist_checks.*
FROM
checklist_items
LEFT OUTER JOIN
checklist_checks
ON
checklist_checks.checklist_item_id = checklist_items.id
AND
checklist_checks.store_id = 1
AND
checklist_checks.report_date = '2015-05-03'
WHERE
checklist_items.list_type = 'open'
AND
(checklist_items.store_id = 1 OR checklist_items.store_id IS NULL)
UPDATE:
I found this: enter link description here
It suggests using find_by_sql and then passing the result array and model to be included to ActiveRecord::Associations::Preloader.new.preload
I tried that, and my find_by_sql pulls the right stuff, but the id column is nil in the resulting objects.
#store = Store.find(1)
#today = Date.today - 1.days
#open_items = ChecklistItem.find_by_sql(["SELECT checklist_items.*, checklist_checks.* FROM checklist_items LEFT OUTER JOIN checklist_checks ON checklist_checks.checklist_item_id = checklist_items.id AND checklist_checks.store_id = ? AND checklist_checks.report_date = ? WHERE checklist_items.list_type='open' AND (checklist_items.store_id=? OR checklist_items.store_ID IS NULL)", #store.id, #today, #store_id])
ActiveRecord::Associations::Preloader.new.preload(#open_items, :checklist_checks)
> #open_items.first.name
=> "Turn on the lights"
> #open_items.first.id
=> nil
A solution using Arel to generate a custom join clause:
class ChecklistItem < ActiveRecord::Base
has_many :checklist_checks
# ...
def self.superjoin(date, store_id)
# build the ON clause for the join
on = Arel::Nodes::On.new(
Arel::Nodes::Equality.new(ChecklistChecks.arel_table[:checklist_item_id], ChecklistItem.arel_table[:id]).\
and(ChecklistItem.arel_table[:store_id].eq(1)).\
and(ChecklistChecks.arel_table[:report_date].eq(date))
)
joins(Arel::Nodes::OuterJoin.new(ChecklistChecks.arel_table, on))
.where(store_id: [nil, store_id], list_type: 'open' )
end
end
I bundled it up into a model method to make it easier to test in the rails console.
irb(main):117:0> ChecklistItem.superjoin(1,2)
ChecklistItem Load (0.5ms) SELECT "checklist_items".* FROM "checklist_items" LEFT OUTER JOIN "checklist_checks" ON "checklist_checks"."checklist_item_id" = "checklist_items"."id" AND "checklist_items"."store_id" = 1 AND "checklist_checks"."report_date" = 1 WHERE (("checklist_items"."store_id" = 2 OR "checklist_items"."store_id" IS NULL)) AND "checklist_items"."list_type" = 'open'
=> #<ActiveRecord::Relation []>
How do i write this query in rails active record style
SELECT COUNT(o.id) no_of_orders,
SUM(o.total) total,
SUM(o.shipping) shipping
FROM orders o JOIN
(
SELECT DISTINCT order_id
FROM designer_orders
WHERE state IN('pending', 'dispatched', 'completed')
) d
ON o.id = d.order_id
You can do with an explicit query, you have 2 manner to do that:
Model.where("MYSQL_QUERY")
or
Model.find_by_sql("MYSQL_QUERY")
http://apidock.com/rails/ActiveRecord/Base/find_by_sql/class
OR
In Rails Style with a little more steps (probably can be done with less):
order_ids = DesignerOrder.where("state IN (?)", ['pending', 'dispatched', 'completed']).select(:order_id).distinct
partial_result = Order.where("id IN (?)", order_ids)
no_of_orders = partial_result.count
total_sum = partial_result.sum(:total)
shipping_sum = partial_result.sum(:shipping)
You can also do it like this
Order
.select('
COUNT(o.id) no_of_orders,
SUM(o.total) total,
SUM(o.shipping) shipping
')
.from('orders o')
.joins("
(#{
DesignerOrders
.select("DISTINCT order_id")
.where("state IN('pending', 'dispatched', 'completed')")
}) d on o.id = d.order_id
")
I didn't actually run this but the concept is valid. You don't even need an active record model if you use 'from'. We've used techniques like this to do AR style queries for extremely complex SQL and it's made our lives a lot easier.
I am using ruby 1.8.6 , rails 2.3.8.
Here , I have a problem with multiple combo box in Rails,
Product drop down list
Sku's drop down list ( depends on production selection )
Product tables fields are
id name
In Sku's tables fields are
id name product_id alias_id
Alias tables fields are
id name
For example I have Sku's tables data like below
id name product_id alias_id
1. 100-m 1 10
2. 10-ml 1 NULL
3. 150 1 2
4. 200-m 1 10
5. 300-m 1 10
in Controller I written query like,
#skus = Sku.all(:conditions => ["product_id = ? ",
params[:id]],:select=>"skus.id,
CASE when skus.alias_id IS NOT NULL then (SELECT alias.name FROM alias WHERE
alias.id = skus.alias_id group by alias.name) END AS 'skus_name'",
:order=>"skus_name" ,:include=>[:alias])
This query written output like,
id skus_name
1. 100gms
2. 10-ml
3. 150-ml
4. 100gms
5. 100gms
Can any one help me how to get the distinct results?
Thanks in advance
You can either call uniq on the #sku variable that is returned.
#skus = Sku.all(:conditions => ["product_id = ? ",
params[:id]],:select=>"skus.id,
CASE when skus.alias_id IS NOT NULL then (SELECT alias.name FROM alias WHERE
alias.id = skus.alias_id group by alias.name) END AS 'skus_name'",
:order=>"skus_name" ,:include=>[:alias]).uniq
This will perform the same DB select but get unique results in ruby.
The alternative is to use DISTINCT in the select
#skus = Sku.all(:conditions => ["product_id = ? ",
params[:id]],:select=>"skus.id,
CASE when skus.alias_id IS NOT NULL then (SELECT DISTINCT alias.name FROM alias WHERE
alias.id = skus.alias_id group by alias.name) END AS 'skus_name'",
:order=>"skus_name" ,:include=>[:alias])
This will only get unique results in the database.
I'd go with the second option as it should be quicker than doing uniq in ruby :)
Is there a way to sanitize sql in rails method find_by_sql?
I've tried this solution:
Ruby on Rails: How to sanitize a string for SQL when not using find?
But it fails at
Model.execute_sql("Update users set active = 0 where id = 2")
It throws an error, but sql code is executed and the user with ID 2 now has a disabled account.
Simple find_by_sql also does not work:
Model.find_by_sql("UPDATE user set active = 0 where id = 1")
# => code executed, user with id 1 have now ban
Edit:
Well my client requested to make that function (select by sql) in admin panel to make some complex query(joins, special conditions etc). So I really want to find_by_sql that.
Second Edit:
I want to achieve that 'evil' SQL code won't be executed.
In admin panel you can type query -> Update users set admin = true where id = 232 and I want to block any UPDATE / DROP / ALTER SQL command.
Just want to know, that here you can ONLY execute SELECT.
After some attempts I conclude sanitize_sql_array unfortunatelly don't do that.
Is there a way to do that in Rails??
Sorry for the confusion..
Try this:
connect = ActiveRecord::Base.connection();
connect.execute(ActiveRecord::Base.send(:sanitize_sql_array, "your string"))
You can save it in variable and use for your purposes.
I made a little snippet for this that you can put in initializers.
class ActiveRecord::Base
def self.escape_sql(array)
self.send(:sanitize_sql_array, array)
end
end
Right now you can escape your query with this:
query = User.escape_sql(["Update users set active = ? where id = ?", true, params[:id]])
And you can call the query any way you like:
users = User.find_by_sql(query)
Slightly more general-purpose:
class ActiveRecord::Base
def self.escape_sql(clause, *rest)
self.send(:sanitize_sql_array, rest.empty? ? clause : ([clause] + rest))
end
end
This one lets you call it just like you'd type in a where clause, without extra brackets, and using either array-style ? or hash-style interpolations.
User.find_by_sql(["SELECT * FROM users WHERE (name = ?)", params])
Source: http://blog.endpoint.com/2012/10/dont-sleep-on-rails-3-sql-injection.html
Though this example is for INSERT query, one can use similar approach for UPDATE queries. Raw SQL bulk insert:
users_places = []
users_values = []
timestamp = Time.now.strftime('%Y-%m-%d %H:%M:%S')
params[:users].each do |user|
users_places << "(?,?,?,?)" # Append to array
users_values << user[:name] << user[:punch_line] << timestamp << timestamp
end
bulk_insert_users_sql_arr = ["INSERT INTO users (name, punch_line, created_at, updated_at) VALUES #{users_places.join(", ")}"] + users_values
begin
sql = ActiveRecord::Base.send(:sanitize_sql_array, bulk_insert_users_sql_arr)
ActiveRecord::Base.connection.execute(sql)
rescue
"something went wrong with the bulk insert sql query"
end
Here is the reference to sanitize_sql_array method in ActiveRecord::Base, it generates the proper query string by escaping the single quotes in the strings. For example the punch_line "Don't let them get you down" will become "Don\'t let them get you down".
I prefer to do it with key parameters. In your case it may looks like this:
Model.find_by_sql(["UPDATE user set active = :active where id = :id", active: 0, id: 1])
Pay attention, that you pass ONLY ONE parameter to :find_by_sql method - its an array, which contains two elements: string query and hash with params (since its our favourite Ruby, you can omit the curly brackets).