I do the following so I am able to group all LineItem's together by count and display the LineItem by count along with the vendor_name
line_items = LineItem.all
vendor_line_items = line_items.group(:vendor_name).select('COUNT(*) as count', 'vendor_name').order('count desc')
My issue is that I am only able to receive the following params: id: nil, vendor_name: "name_here"
Is there a way to accomplish the same thing but allow all params from the model to be passed?
You can't select the rest of the columns since you have different values for each coulmn inside the group (like... if you have 2 LineItem in the same group, which ID do you expect to have?)
You could apply aggregate functions (like COUNT, MAX, MIN, etc) to other columns on the SELECT to tell the database which columns you want for each column I guess.
Personally, I would first get the groups ordered by count and then do more queries when needed to fetch the actual record for the groups.
counts = LineItem.group(:vendor_name).count
# counts should be something like: {vendor_1: X, vendor_2, Y, vendor_3: Z}
# order the vendors using the count for each vendor
ordered_vendors = counts.keys.sort_by { |ven| counts[ven] }
ordered_vendors.each do |vendor|
# do something with each vendor, fetch LineItems, etc
end
The reason why you only see the count and the vendor name is because that is all you are grouping by. Suppose in the database, you have 5 different Vendor A shown below.
vendor_name | product_name
-----------------------------
Vendor A | test
Vendor A | test2
Vendor A | test3
Vendor A | test4
Vendor A | test5
...
When you run your query, SQL will not know what to display for product_name as the group_by will only show 1 row instead of 5. Have a read about it here.
To achieve this you will need to either to group by the other columns too or use a min/max select to pick a value to display. Here is an example:
vendor_line_items = LineItem.select('COUNT(*) AS count', 'vendor_name', 'MAX(product_name)').group(:vendor_name).order('count DESC')
Now each of those results, you can call the attributes method.
Which will give you the following hash:
vendor_line_items.each do |x|
result = x.attributes
# Here result will be a hash.
# {"count" => 5, "vendor_name" => "Vendor A", "product_name" => "test5"}
end
(Not accepted answer unless a better way is received)
I did:
vendor_line_items = Vendor.joins(:line_items).group(:id).order('COUNT(line_items.id) DESC')
This gives me what I want by ordering the results by vendor.line_items.count and allowing me to get all of the associations to display any param I want.
I assume this way is much slower than what I was previously doing as it fetches all records and then on the front end goes through associations to get more records.
In the original way I was doing this. It is what I want minus an extra parameter that I would want the SUM of. The parameter is a decimal attribute. In the same way I count the LineItem that have the same vendor_name, I want to sum of the LineItem.attribute that share the same vendor_name.
Better Answer:
LineItem.select(:vendor_name, 'sum(line_item_revenue) as line_item_revenue', 'COUNT(*) as count').group(:vendor_name)
This seems to get me what I want with less queries (i believe) --- correct me if I am wrong on the queries.
I am quite confused about your code and your expectation. You are selecting the COUNT but the expected result is id instead of count?
If you want to group by vendor_name and show the count of group_by you can try
line_items.group(:vendor_name).count
Related
I have a Company model that has many Disclosures. The Disclosure has columns named title, pdf and pdf_sha256.
class Company < ActiveRecord::Base
has_many :disclosures
end
class Disclosure < ActiveRecord::Base
belongs_to :company
end
I want to make it unique by pdf_sha256 and if pdf_sha256 is nil that should be treated as unique.
If it is an Array, I'll write like this.
companies_with_sha256 = company.disclosures.where.not(pdf_sha256: nil).group_by(&:pdf_sha256).map do |key,values|
values.max_by{|v| v.title.length}
end
companies_without_sha256 = company.disclosures.where(pdf_sha256: nil)
companies = companies_with_sha256 + companeis_without_sha256
How can I get the same result by using ActiveRecord query?
It is possible to do it in one query by first getting a different id for each different pdf_sha256 as a subquery, then in the query getting the elements within that set of ids by passing the subquery as follows:
def unique_disclosures_by_pdf_sha256(company)
subquery = company.disclosures.select('MIN(id) as id').group(:pdf_sha256)
company.disclosures.where(id: subquery)
.or(company.disclosures.where(pdf_sha256: nil))
end
The great thing about this is that ActiveRecord is lazy loaded, so the first subquery will not be run and will be merged to the second main query to create a single query in the database. It will then retrieve all the disclosures unique by pdf_sha256 plus all the ones that have pdf_sha256 set to nil.
In case you are curious, given a company, the resulting query will be something like:
SELECT "disclosures".* FROM "disclosures"
WHERE (
"disclosures"."company_id" = $1 AND "disclosures"."id" IN (
SELECT MAX(id) as id FROM "disclosures" WHERE "disclosures"."company_id" = $2 GROUP BY "disclosures"."pdf_sha256"
)
OR "disclosures"."company_id" = $3 AND "disclosures"."pdf_sha256" IS NULL
)
The great thing about this solution is that the returned value is an ActiveRecord query, so it won't be loaded until you actually need. You can also use it to keep chaining queries. Example, you can select only the id instead of the whole model and limit the number of results returned by the database:
unique_disclosures_by_pdf_sha256(company).select(:id).limit(10).each { |d| puts d }
You can achieve this by using uniq method
Company.first.disclosures.to_a.uniq(&:pdf_sha256)
This will return you the disclosures records uniq by cloumn "pdf_sha256"
Hope this helps you! Cheers
Assuming you are using Rails 5 you could chain a .or command to merge both your queries.
pdf_sha256_unique_disclosures = company.disclosures.where(pdf_sha256: nil).or(company.disclosures.where.not(pdf_sha256: nil))
Then you can proceed with your group_by logic.
However, in the example above i'm not exactly sure what is the objective but I am curious to better understand how you would use the resulting companies variable.
If you wanted to have a hash of unique pdf_sha256 keys including nil, and its resultant unique disclosure document you could try the following:
sorted_disclosures = company.disclosures.group_by(&:pdf_sha256).each_with_object({}) do |entries, hash|
hash[entries[0]] = entries[1].max_by{|v| v.title.length}
end
This should give you a resultant hash like structure similar to the group_by where your keys are all your unique pdf_sha256 and the value would be the longest named disclosure that match that pdf_sha256.
Why not:
ids = Disclosure.select(:id, :pdf_sha256).distinct.map(&:id)
Disclosure.find(ids)
The id sill be distinct either way since it's the primary key, so all you have to do is map the ids and find the Disclosures by id.
If you need a relation with distinct pdf_sha256, where you require no explicit conditions, you can use group for that -
scope :unique_pdf_sha256, -> { where.not(pdf_sha256: nil).group(:pdf_sha256) }
scope :nil_pdf_sha256, -> { where(pdf_sha256: nil) }
You could have used or, but the relation passed to it must be structurally compatible. So even if you get same type of relations in these two scopes, you cannot use it with or.
Edit: To make it structurally compatible with each other you can see #AlexSantos 's answer
Model.select(:rating)
Result of this is an array of Model objects. Not plain ratings. And from uniq's point of view, they are completely different. You can use this:
Model.select(:rating).map(&:rating).uniq
or this (most efficient)
Model.uniq.pluck(:rating)
Model.distinct.pluck(:rating)
Update
Apparently, as of rails 5.0.0.1, it works only on "top level" queries, like above. Doesn't work on collection proxies ("has_many" relations, for example).
Address.distinct.pluck(:city) # => ['Moscow']
user.addresses.distinct.pluck(:city) # => ['Moscow', 'Moscow', 'Moscow']
In this case, deduplicate after the query
user.addresses.pluck(:city).uniq # => ['Moscow']
In a rails 4 app, in one model I have a column containing multiple ids as a string with comma separated values.
"123,4568,12"
I have a "search" engine that I use to retrieve the records with one or many values using the full text search of postgresql I can do something like this which is very useful:
records = MyModel.where("my_models.col_name ## ?", ["12","234"])
This return all the records that have both 12 and 234 in the targeted column. The array comes from a form with a multiple select.
Now I'm trying to make a query that will find all the records that have either 12 or 234 in there string.
I was hopping to be able to do something like:
records = MyModel.where("my_models.col_name IN (?)", ["12","234"])
But it's not working.
Should I iterate through all the values in the array to build a query with multiple OR ? Is there something more appropriate to do this?
EDIT / TL;DR
#BoraMa answer is a good way to achieve this.
To find all the records containing one or more ids referenced in the request use:
records = MyModel.where("my_models.col_name ## to_tsquery(?)", ["12","234"].join('|'))
You need the to_tsquery(?) and the join with a single pipe |to do a OR like query.
To find all the records containing exactly all the ids in the query use:
records = MyModel.where("my_models.col_name ## ?", ["12","234"])
And of course replace ["12","234"] with something like params[:params_from_my_form]
Postgres documentation for full text search
If you already started to use the fulltext search in Postgres in the first place,I'd try to leverage it again. I think you can use a fulltext OR query which can be constructed like this:
records = MyModel.where("my_models.col_name ## to_tsquery(?)", ["12","234"].join(" | "));
This uses the | operator for ORing fulltext queries in Postgres. I have not tested this and maybe you'll need to do to_tsvector('my_models.col_name') for this to work.
See the documentation for more info.
Suppose your ids are :
a="1,2,3,4"
You can simply use:
ModelName.find(a)
This will give you all the record of that model whose id is present in a.
I just think a super simple solution, we just sort the ids in saving callback of MyModel, then the query must be easier:
class MyModel < ActiveRecord::Base
before_save :sort_ids_in_col_name, if: :col_name_changed?
private
def sort_ids_in_col_name
self.col_name = self.col_name.to_s.split(',').sort.join(',')
end
end
Then the query will be easy:
ids = ["12","234"]
records = MyModel.where(col_name: ids.sort.join(',')
Rails: 4.1.2
Database: PostgreSQL
For one of my queries, I am using methods from both the textacular gem and Active Record. How can I chain some of the following queries with an "OR" instead of an "AND":
people = People.where(status: status_approved).fuzzy_search(first_name: "Test").where("last_name LIKE ?", "Test")
I want to chain the last two scopes (fuzzy_search and the where after it) together with an "OR" instead of an "AND." So I want to retrieve all People who are approved AND (whose first name is similar to "Test" OR whose last name contains "Test"). I've been struggling with this for quite a while, so any help would be greatly appreciated!
I digged into fuzzy_search and saw that it will be translated to something like:
SELECT "people".*, COALESCE(similarity("people"."first_name", 'test'), 0) AS "rankxxx"
FROM "people"
WHERE (("people"."first_name" % 'abc'))
ORDER BY "rankxxx" DESC
That says if you don't care about preserving order, it will just filter the result by WHERE (("people"."first_name" % 'abc'))
Knowing that and now you can simply write the query with similar functionality:
People.where(status: status_approved)
.where('(first_name % :key) OR (last_name LIKE :key)', key: 'Test')
In case you want order, please specify what would you like the order will be after joining 2 conditions.
After a few days, I came up with the solution! Here's what I did:
This is the query I wanted to chain together with an OR:
people = People.where(status: status_approved).fuzzy_search(first_name: "Test").where("last_name LIKE ?", "Test")
As Hoang Phan suggested, when you look in the console, this produces the following SQL:
SELECT "people".*, COALESCE(similarity("people"."first_name", 'test'), 0) AS "rank69146689305952314"
FROM "people"
WHERE "people"."status" = 1 AND (("people"."first_name" % 'Test')) AND (last_name LIKE 'Test') ORDER BY "rank69146689305952314" DESC
I then dug into the textacular gem and found out how the rank is generated. I found it in the textacular.rb file and then crafted the SQL query using it. I also replaced the "AND" that connected the last two conditions with an "OR":
# Generate a random number for the ordering
rank = rand(100000000000000000).to_s
# Create the SQL query
sql_query = "SELECT people.*, COALESCE(similarity(people.first_name, :query), 0)" +
" AS rank#{rank} FROM people" +
" WHERE (people.status = :status AND" +
" ((people.first_name % :query) OR (last_name LIKE :query_like)))" +
" ORDER BY rank#{rank} DESC"
I took out all of quotation marks in the SQL query when referring to tables and fields because it was giving me error messages when I kept them there and even if I used single quotes.
Then, I used the find_by_sql method to retrieve the People object IDs in an array. The symbols (:status, :query, :query_like) are used to protect against SQL injections, so I set their values accordingly:
# Retrieve all the IDs of People who are approved and whose first name and last name match the search query.
# The IDs are sorted in order of most relevant to the search query.
people_ids = People.find_by_sql([sql_query, query: "Test", query_like: "%Test%", status: 1]).map(&:id)
I get the IDs and not the People objects in an array because find_by_sql returns an Array object and not a CollectionProxy object, as would normally be returned, so I cannot use ActiveRecord query methods such as where on this array. Using the IDs, we can execute another query to get a CollectionProxy object. However, there's one problem: If we were to simply run People.where(id: people_ids), the order of the IDs would not be preserved, so all the relevance ranking we did was for nothing.
Fortunately, there's a nice gem called order_as_specified that will allow us to retrieve all People objects in the specific order of the IDs. Although the gem would work, I didn't use it and instead wrote a short line of code to craft conditions that would preserve the order.
order_by = people_ids.map { |id| "people.id='#{id}' DESC" }.join(", ")
If our people_ids array is [1, 12, 3], it would create the following ORDER statement:
"people.id='1' DESC, people.id='12' DESC, people.id='3' DESC"
I learned from this comment that writing an ORDER statement in this way would preserve the order.
Now, all that's left is to retrieve the People objects from ActiveRecord, making sure to specify the order.
people = People.where(id: people_ids).order(order_by)
And that did it! I didn't worry about removing any duplicate IDs because ActiveRecord does that automatically when you run the where command.
I understand that this code is not very portable and would require some changes if any of the people table's columns are modified, but it works perfectly and seems to execute only one query according to the console.
I have a site that fetches the ID and SCORE of records for a user from an external source.
I want to then fetch the records that match the ID's, ordered by each ID's SCORE.
The ID's and SCORES come to the app as XML which I'm currently parsing into an array (of hashes):
[{90=>279}, {32=>400}]
How can I use that score to order the records returned?
src_data = [{90=>279}, {32=>400}]
ids = src_data.sort_by{|hash| hash.values.first}.reverse.collect(&:keys).flatten
# => [32,90]
ids should now be an array of ids, in order of highest score to lowest score. Now we can do the find.
#users = User.where("id in (?)", ids).order("ORDER BY FIELD(ID,#{ids.join(',')})")
This should generate sql like
select * from users where id in (32,90) ORDER BY FIELD(ID,32,90);
which should give you the users back in the right order.
EDIT - for postgres
Postgresql doesn't have the field() function. You can define your own, see Simulating MySQL's ORDER BY FIELD() in Postgresql, but you might prefer to not bother and just reorder the results after you get them:
#users = User.where("id in (?)", ids).sort_by{|user| ids.index(user.id)}
If it's a standard ruby array you should be able to sort it by using "sort_by".
Two examples of sorting based on the keys / values:
results = [{90=>279}, {32=>400}]
results.sort_by{|result| result.keys[0]}
# => [{32=>400}, {90=>279}]
results.sort_by{|result| result.values[0]}
# => [{90=>279}, {32=>400}]
Check this blogpost on more information regarding sort_by and sort in Ruby:
http://brandon.dimcheff.com/2009/11/18/rubys-sort-vs-sort-by.html
I'm having trouble figuring out how to loop over the results of a ThinkingSphinx search that has been set to group_by. I currently have the following:
search = Event.search(
{
group_by: 'category_id',
group_function: :attr
}
)
search.each_with_groupby_and_count do |event, group, count|
puts [event, group, count].join(' - ')
end
This, however, only returns one record per category. It seems like the group and count values are correct, but I only get the first Event of each category, which I would have expected to be all the events in the group. Is it possible to get an array of Hashes or similar? Furthermore, if this is possible, would the per_page option be per group?
I would expect each_with_group_and_count to iterate over something like this:
[
{group: 1, hits: [Event1, Event2], count: 2},
{group: 2: hits: [Event3], count: 1}
]
I'm afraid Sphinx's grouping functionality doesn't behave in that matter - it only returns one document (in this situation, one event) per group value.
It may be more appropriate to just sort by category_id instead, and track when it changes as you iterate over it (or use Enumerable#group_by to group all events by category_id) - keep in mind that Sphinx paginates results, so you may want to increase the default page size (with :per_page) depending on how you're using these results.