I have two models with enum fields:
class TempAsset < ApplicationRecord
enum state: { running: 0, stopped: 1, terminated: 2 }
end
class AssetCredential < ApplicationRecord
enum map_status: { pending: 0, inprogress: 1, passed: 2, failed: 3 }
end
When I select column from the first table, it gives proper values from enum:
TempAsset
.joins('INNER JOIN asset_credentials
ON temp_assets.instance_id = asset_credentials.instance_id')
.pluck(:state)
.uniq
# ["stopped", "running"]
But, it gives numbers when I select column from the joined table:
TempAsset
.joins('INNER JOIN asset_credentials
ON temp_assets.instance_id = asset_credentials.instance_id')
.pluck(:map_status)
.uniq
# [0, 3, 2, 1]
So, should I do something like this:
AssetCredential.map_statuses.key(0) => "pending"
AssetCredential.map_statuses.key(1) => "inprogress"
Or is there any better way to do the same?
ActiveRecord::Enum is not designed to work across join table, therefore you cannot select or pluck other table enum column and expect the mapping value in return. As you said what you can do is to pluck the integer value and do the mapping by yourself.
Or in my case, I use enumerize gem which stores values in the database and gives you more options and customization such as validation and I18n. With this gem you can use your code above to pluck the expected values (because it stores exact value not mapping with integer).
See, the problem here is you are using active-record enums. These are stored as integers in the database, and mapping to some text is done on the application level. pluck works on the db level.
Whats happening here is when you do a inner join, it is collection columns from both tables. When you pluck from the table which you are joining-from, active record knows how to map it to the enum. In the second one, you are plucking column from another table, and active-record is unable to get the context.
Here, you can try using postgres enums for proper results.
TempAsset
.joins('
JOIN asset_credentials
ON temp_assets.instance_id = asset_credentials.instance_id
')
.where(
state: TempAsset.states[:terminated],
asset_credentials: { map_status: AssetCredentials.map_statuses[:passed] }
)
You can change the keys being passed in to whatever you need. I'm not sure what model you want returned but your could switch things around to suit your needs
I tried to simulate your problem. The interesting thing which I found here was this issue does not arise when we use 'joins' on 'one-to-many' relationship.
i.e., if one 'TempAsset' has many 'AssetCredentials', the query
TempAsset.joins(:asset_credentials)
gives out enum values as you expect. Whereas
AssetCredential.joins(:temp_asset) does produce the issue which you have mentioned.
So, If you can establish 'has_many' relationship between 'TempAsset' and 'AssetCredentials' through a 'has_one' relationship between 'TempAsset' and 'Instance', the issue could get fixed.
Given the rails associations are setup, can also try something like:
TempAsset
.joins('join asset_credentials on temp_assets.instance_id = asset_credentials.instance_id')
.map { |temp_asset| [temp_asset| [temp_asset.asset_credential.map_status] }
.uniq
Try this:
AssetCredential.joins('INNER JOIN temp_assets ON asset_credentials.instance_id = temp_assets.instance_id').pluck(:map_status).uniq
Related
I'm using Rails 5. I have the following model ...
class Order < ApplicationRecord
...
has_many :line_items, :dependent => :destroy
The LineItem model has an attribute, "discount_applied." I would like to return all orders where there are zero instances of a line item having the "discount_applied" field being not nil. How do I write such a finder method?
First of all, this really depends on whether or not you want to use a pure Arel approach or if using SQL is fine. The former is IMO only advisable if you intend to build a library but unnecessary if you're building an app where, in reality, it's highly unlikely that you're changing your DBMS along the way (and if you do, changing a handful of manual queries will probably be the least of your troubles).
Assuming using SQL is fine, the simplest solution that should work across pretty much all databases is this:
Order.where("(SELECT COUNT(*) FROM line_items WHERE line_items.order_id = orders.id AND line_items.discount_applied IS NULL) = 0")
This should also work pretty much everywhere (and has a bit more Arel and less manual SQL):
Order.left_joins(:line_items).where(line_items: { discount_applied: nil }).group("orders.id").having("COUNT(line_items.id) = 0")
Depending on your specific DBMS (more specifically: its respective query optimizer), one or the other might be more performant.
Hope that helps.
Not efficient but I thought it may solve your problem:
orders = Order.includes(:line_items).select do |order|
order.line_items.all? { |line_item| line_item.discount_applied.nil? }
end
Update:
Instead of finding orders which all it's line items have no discount, we can exclude all the orders which have line items with a discount applied from the output result. This can be done with subquery inside where clause:
# Find all ids of orders which have line items with a discount applied:
excluded_ids = LineItem.select(:order_id)
.where.not(discount_applied: nil)
.distinct.map(&:order_id)
# exclude those ids from all orders:
Order.where.not(id: excluded_ids)
You can combine them in a single finder method:
Order.where.not(id: LineItem
.select(:order_id)
.where.not(discount_applied: nil))
Hope this helps
A possible code
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
I advice to get familiar with AR documentation for Query Methods.
Update
This seems to be more interested than I initially though. And more complicated, so I will not be able to give you a working code. But I would look into a solution using LineItem.group(order_id).having(discount_applied: nil), which should give you a collection of line_items and then use it as sub-query to find related orders.
If you want all the records where discount_applied is nil then:
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
(use includes to avoid n+1 problem)
or
Order.joins(:line_items).where.not(line_items: {discount_applied: nil})
Here is the solution to your problem
order_ids = Order.joins(:line_items).where.not(line_items: {discount_applied: nil}).pluck(:id)
orders = Order.where.not(id: order_ids)
First query will return ids of Orders with at least one line_item having discount_applied. The second query will return all orders where there are zero instances of a line_item having the discount_applied.
I would use the NOT EXISTS feature from SQL, which is at least available in both MySQL and PostgreSQL
it should look like this
class Order
has_many :line_items
scope :without_discounts, -> {
where("NOT EXISTS (?)", line_items.where("discount_applied is not null")
}
end
If I understood correctly, you want to get all orders for which none line item (if any) has a discount applied.
One way to get those orders using ActiveRecord would be the following:
Order.distinct.left_outer_joins(:line_items).where(line_items: { discount_applied: nil })
Here's a brief explanation of how that works:
The solution uses left_outer_joins, assuming you won't be accessing the line items for each order. You can also use left_joins, which is an alias.
If you need to instantiate the line items for each Order instance, add .eager_load(:line_items) to the chain which will prevent doing an additional query for every order (N+1), i.e., doing order.line_items.each in a view.
Using distinct is essential to make sure that orders are only included once in the result.
Update
My previous solution was only checking that discount_applied IS NULL for at least one line item, not all of them. The following query should return the orders you need.
Order.left_joins(:line_items).group(:id).having("COUNT(line_items.discount_applied) = ?", 0)
This is what's going on:
The solution still needs to use a left outer join (orders LEFT OUTER JOIN line_items) so that orders without any associated items are included.
Groups the line items to get a single Order object regardless of how many items it has (GROUP BY recipes.id).
It counts the number of line items that were given a discount for each order, only selecting the ones whose items have zero discounts applied (HAVING (COUNT(line_items.discount_applied) = 0)).
I hope that helps.
You cannot do this efficiently with a classic rails left_joins, but sql left join was build to handle thoses cases
Order.joins("LEFT JOIN line_items AS li ON li.order_id = orders.id
AND li.discount_applied IS NOT NULL")
.where("li.id IS NULL")
A simple inner join will return all orders, joined with all line_items,
but if there are no line_items for this order, the order is ignored (like a false where)
With left join, if no line_items was found, sql will joins it to an empty entry in order to keep it
So we left joined the line_items we don't want, and find all orders joined with an empty line_items
And avoid all code with where(id: pluck(:id)) or having("COUNT(*) = 0"), on day this will kill your database
I have a Company model that has many Disclosures. The Disclosure has columns named title, pdf and pdf_sha256.
class Company < ActiveRecord::Base
has_many :disclosures
end
class Disclosure < ActiveRecord::Base
belongs_to :company
end
I want to make it unique by pdf_sha256 and if pdf_sha256 is nil that should be treated as unique.
If it is an Array, I'll write like this.
companies_with_sha256 = company.disclosures.where.not(pdf_sha256: nil).group_by(&:pdf_sha256).map do |key,values|
values.max_by{|v| v.title.length}
end
companies_without_sha256 = company.disclosures.where(pdf_sha256: nil)
companies = companies_with_sha256 + companeis_without_sha256
How can I get the same result by using ActiveRecord query?
It is possible to do it in one query by first getting a different id for each different pdf_sha256 as a subquery, then in the query getting the elements within that set of ids by passing the subquery as follows:
def unique_disclosures_by_pdf_sha256(company)
subquery = company.disclosures.select('MIN(id) as id').group(:pdf_sha256)
company.disclosures.where(id: subquery)
.or(company.disclosures.where(pdf_sha256: nil))
end
The great thing about this is that ActiveRecord is lazy loaded, so the first subquery will not be run and will be merged to the second main query to create a single query in the database. It will then retrieve all the disclosures unique by pdf_sha256 plus all the ones that have pdf_sha256 set to nil.
In case you are curious, given a company, the resulting query will be something like:
SELECT "disclosures".* FROM "disclosures"
WHERE (
"disclosures"."company_id" = $1 AND "disclosures"."id" IN (
SELECT MAX(id) as id FROM "disclosures" WHERE "disclosures"."company_id" = $2 GROUP BY "disclosures"."pdf_sha256"
)
OR "disclosures"."company_id" = $3 AND "disclosures"."pdf_sha256" IS NULL
)
The great thing about this solution is that the returned value is an ActiveRecord query, so it won't be loaded until you actually need. You can also use it to keep chaining queries. Example, you can select only the id instead of the whole model and limit the number of results returned by the database:
unique_disclosures_by_pdf_sha256(company).select(:id).limit(10).each { |d| puts d }
You can achieve this by using uniq method
Company.first.disclosures.to_a.uniq(&:pdf_sha256)
This will return you the disclosures records uniq by cloumn "pdf_sha256"
Hope this helps you! Cheers
Assuming you are using Rails 5 you could chain a .or command to merge both your queries.
pdf_sha256_unique_disclosures = company.disclosures.where(pdf_sha256: nil).or(company.disclosures.where.not(pdf_sha256: nil))
Then you can proceed with your group_by logic.
However, in the example above i'm not exactly sure what is the objective but I am curious to better understand how you would use the resulting companies variable.
If you wanted to have a hash of unique pdf_sha256 keys including nil, and its resultant unique disclosure document you could try the following:
sorted_disclosures = company.disclosures.group_by(&:pdf_sha256).each_with_object({}) do |entries, hash|
hash[entries[0]] = entries[1].max_by{|v| v.title.length}
end
This should give you a resultant hash like structure similar to the group_by where your keys are all your unique pdf_sha256 and the value would be the longest named disclosure that match that pdf_sha256.
Why not:
ids = Disclosure.select(:id, :pdf_sha256).distinct.map(&:id)
Disclosure.find(ids)
The id sill be distinct either way since it's the primary key, so all you have to do is map the ids and find the Disclosures by id.
If you need a relation with distinct pdf_sha256, where you require no explicit conditions, you can use group for that -
scope :unique_pdf_sha256, -> { where.not(pdf_sha256: nil).group(:pdf_sha256) }
scope :nil_pdf_sha256, -> { where(pdf_sha256: nil) }
You could have used or, but the relation passed to it must be structurally compatible. So even if you get same type of relations in these two scopes, you cannot use it with or.
Edit: To make it structurally compatible with each other you can see #AlexSantos 's answer
Model.select(:rating)
Result of this is an array of Model objects. Not plain ratings. And from uniq's point of view, they are completely different. You can use this:
Model.select(:rating).map(&:rating).uniq
or this (most efficient)
Model.uniq.pluck(:rating)
Model.distinct.pluck(:rating)
Update
Apparently, as of rails 5.0.0.1, it works only on "top level" queries, like above. Doesn't work on collection proxies ("has_many" relations, for example).
Address.distinct.pluck(:city) # => ['Moscow']
user.addresses.distinct.pluck(:city) # => ['Moscow', 'Moscow', 'Moscow']
In this case, deduplicate after the query
user.addresses.pluck(:city).uniq # => ['Moscow']
I have a model Company that have columns pbr, market_cap and category.
To get averages of pbr grouped by category, I can use group method.
Company.group(:category).average(:pbr)
But there is no method for weighted average.
To get weighted averages I need to run this SQL code.
select case when sum(market_cap) = 0 then 0 else sum(pbr * market_cap) / sum(market_cap) end as weighted_average_pbr, category AS category FROM "companies" GROUP BY "companies"."category";
In psql this query works fine. But I don't know how to use from Rails.
sql = %q(select case when sum(market_cap) = 0 then 0 else sum(pbr * market_cap) / sum(market_cap) end as weighted_average_pbr, category AS category FROM "companies" GROUP BY "companies"."category";)
ActiveRecord::Base.connection.select_all(sql)
returns a error:
output error: #<NoMethodError: undefined method `keys' for #<Array:0x007ff441efa618>>
It would be best if I can extend Rails method so that I can use
Company.group(:category).weighted_average(:pbr)
But I heard that extending rails query is a bit tweaky, now I just want to know how to run the result of sql from Rails.
Does anyone knows how to do it?
Version
rails: 4.2.1
What version of Rails are you using? I don't get that error with Rails 4.2. In Rails 3.2 select_all used to return an Array, and in 4.2 it returns an ActiveRecord::Result. But in either case, it is correct that there is no keys method. Instead you need to call keys on each element of the Array or Result. It sounds like the problem isn't from running the query, but from what you're doing afterward.
In any case, to get the more fluent approach you've described, you could do this:
class Company
scope :weighted_average, lambda{|col|
select("companies.category").
select(<<-EOQ)
(CASE WHEN SUM(market_cap) = 0 THEN 0
ELSE SUM(#{col} * market_cap) / SUM(market_cap)
END) AS weighted_average_#{col}
EOQ
}
This will let you say Company.group(:category).weighted_average(:pbr), and you will get a collection of Company instances. Each one will have an extra weighted_average_pbr attribute, so you can do this:
Company.group(:category).weighted_average(:pbr).each do |c|
puts c.weighted_average_pbr
end
These instances will not have their normal attributes, but they will have category. That is because they do not represent individual Companies, but groups of companies with the same category. If you want to group by something else, you could parameterize the lambda to take the grouping column. In that case you might as well move the group call into the lambda too.
Now be warned that the parameter to weighted_average goes straight into your SQL query without escaping, since it is a column name. So make sure you don't pass user input to that method, or you'll have a SQL injection vulnerability. In fact I would probably put a guard inside the lambda, something like raise "NOPE" unless col =~ %r{\A[a-zA-Z0-9_]+\Z}.
The more general lesson is that you can use select to include extra SQL expressions, and have Rails magically treat those as attributes on the instances returned from the query.
Also note that unlike with select_all where you get a bunch of hashes, with this approach you get a bunch of Company instances. So again there is no keys method! :-)
I am developing a rails 4 app using ActiveRecord models for my db tables.
The main issue is that my model is quite complicated, and I would like to display a lot of information when I do an index of the main object.
For example, let's assume I have the following tables and columns:
Person: name(string)
Address: address(string), person_id(int)
EmailAddress: email(string), person_id(int)
Email: spam (boolean), email_address_id(int)
and the relations:
person has_many: :email_addresses
person has_one: :address
email_address has_many: :emails
Now I would like to display the following information
person.name
person.address.name
person.email_addresses.count
person.email_addresses.map do |email_address|
email_address.email.where(spam: false).count
end
The main issue is that I have a big amount of records, and I don't want to instantiate all of them (I have some memory issues because of that). Therefore, I was wondering how to do this kind of thing directly to get either an array of hashes or of arrays.
I managed to get the beginning using pluck:
Person.joins(:address).pluck('persons.name, addresses.address')
The problem begins with the count part.
Has someone already encountered such a situation? And is there a way to do this without writing the complete SQL query?
You can't use pluck for complex queries, but you can always use select to fetch the columns you want. First you join all the tables you need. Note I joined emails table twice, the second one with the spam: false condition. Then you define your columns, directly from the table or COUNT'ed, in your select statement:
persons = Person.joins(:address, email_addresses: :emails).
joins('INNER JOIN emails not_spammy_email_addresses ON emails.email_address_id = email_addresses.id AND emails.spam = 0').
select('persons.name, addresses.address AS address_address,
COUNT(email_addresses.id) AS email_addresses_count,
COUNT(not_spammy_email_addresses.id) AS not_spammy_email_addresses_count')
And then call your result's columns like this:
person = persons.first
person.name
person.address_address # note I'm not using *address* to prevent conflict with the model Adress
person.email_addresses_count
person.not_spammy_email_addresses_count
I believe this is as far as you can get with active_record and a single query, but I'd love to see other approaches. For instance, if you use Arel this query would feel less SQLish.
I am new to Activerecord. I want to do sum on multiple columns of a model Student. My model student is like following:
class Student < ActiveRecord::Base
attr_accessible :class, :roll_num, :total_mark, :marks_obtained, :section
end
I want something like that:
total_marks, total_marks_obtained = Student.where(:id=>student_id).sum(:total_mark, :marks_obtained)
But it is giving following error.
NoMethodError: undefined method `except' for :marks_obtained:Symbol
So I am asking whether I have to query the model two times for the above, i.e. one to find total marks and another to find marks obtained.
You can use pluck to directly obtain the sum:
Student.where(id: student_id).pluck('SUM(total_mark)', 'SUM(marks_obtained)')
# SELECT SUM(total_mark), SUM(marks_obtained) FROM students WHERE id = ?
You can add the desired columns or calculated fields to pluck method, and it will return an array with the values.
If you just want sum of columns total_marks and marks_obtained, try this
Student.where(:id=>student_id).sum('total_mark + marks_obtained')
You can use raw SQL if you need to. Something like this to return an object where you'll have to extract the values... I know you specify active record!
Student.select("SUM(students.total_mark) AS total_mark, SUM(students.marks_obtained) AS marks obtained").where(:id=>student_id)
For rails 4.2 (earlier unchecked)
Student.select("SUM(students.total_mark) AS total_mark, SUM(students.marks_obtained) AS marks obtained").where(:id=>student_id)[0]
NB the brackets following the statement. Without it the statement returns an Class::ActiveRecord_Relation, not the AR instance. What's significant about this is that you CANNOT use first on the relation.
....where(:id=>student_id).first #=> PG::GroupingError: ERROR: column "students.id" must appear in the GROUP BY clause or be used in an aggregate function
Another method is to ActiveRecord::Calculations.pluck then Enumerable#sum on the outer array and again on the inner array pair:
Student
.where(id: student_id)
.pluck(:total_mark, :marks_obtained)
.map(&:sum)
.sum
The resulting SQL query is simple:
SELECT "students"."total_mark",
"students"."marks_obtained"
FROM "students"
WHERE "students"."id" = $1
The initial result of pluck will be an array of array pairs, e.g.:
[[10, 5], [9, 2]]
.map(&:sum) will run sum on each pair, totalling the pair and flattening the array:
[15, 11]
Finally .sum on the flattened array will result in a single value.
Edit:
Note that while there is only a single query, your database will return a result row for each record matched in the where. This method uses ruby to do the totalling, so if there are many records (i.e. thousands), this may be slower than having SQL do the calculations itself like noted in the accepted answer.
Similar to the accepted answer, however, I'd suggest using arel as follows to avoid string literals (apart from renaming columns, if needed).
Student
.where(id: student_id).
.where(Student.arel_table[:total_mark].sum, Student.arel_table[:marks_obtained].sum)
which will give you an ActiveRecord::Relation result over which you can iterate, or, as you'll only get one row, you can use .first (at least for mysql).
Recently, I also had the requirement to sum up multiple columns of a ActiveRecord relation. I ended up with the following (reusable) scope:
scope :values_sum, ->(*keys) {
summands = keys.collect { |k| arel_table[k].sum.as(k.to_s) }
select(*summands)
}
So, having a model e.g. Order with columns net_amount and gross_amount you could use it as follows:
o = Order.today.values_sum(:net_amount, :gross_amount)
o.net_amount # -> sum of net amount
o.gross_amount # -> sum of gross amount