I'm building a query using ransack predicates, so that means I need a lot of conditions for a WHERE clause SQL-wise. Here's what I have as a Hash of conditions for ransanck's search method:
This is the console output of params[:q]
{"price_gteq"=>"0", "rooms_gteq"=>"1", "baths_gteq"=>"1", "has_photos_true"=>"0", "zone_id"=>"1", "deal_type_eq"=>"1", "s"=>{"0"=>{"name"=>"price", "dir"=>"asc"}}, "price_lteq"=>"2500000", "rooms_lteq"=>"2", "baths_lteq"=>"7", "sector_id_in"=>nil}
And this is the SQL query it generates:
Property.search(params[:q]).result.to_sql
Which gives:
SELECT "properties".* FROM "properties" LEFT OUTER JOIN "sectors" ON "sectors"."id" = "properties"."sector_id" WHERE (("properties"."price" >= 0.0 AND "properties"."price" <= 2500000.0)) ORDER BY "properties"."price" ASC
How can I include ALL the fields inside the where clause?
As you can see, only the :price field remains inside the where clause. I need :baths and :rooms to be included for the where clause too.
How can I fix it?
I've solved my own problem fortunately, What happened was that I was using the UNRANSACKABLE_ATTRS array for my model to exclude the aforementioned fields from sorting (The default Sorting methodology provided by Ransack), but I didn't know this would affect search as well. Anyways I removed the UNRANSACKABLE_ATTRS array and every condition in search started to work as expected. :)
Related
I'm using Rails 5. I have the following model ...
class Order < ApplicationRecord
...
has_many :line_items, :dependent => :destroy
The LineItem model has an attribute, "discount_applied." I would like to return all orders where there are zero instances of a line item having the "discount_applied" field being not nil. How do I write such a finder method?
First of all, this really depends on whether or not you want to use a pure Arel approach or if using SQL is fine. The former is IMO only advisable if you intend to build a library but unnecessary if you're building an app where, in reality, it's highly unlikely that you're changing your DBMS along the way (and if you do, changing a handful of manual queries will probably be the least of your troubles).
Assuming using SQL is fine, the simplest solution that should work across pretty much all databases is this:
Order.where("(SELECT COUNT(*) FROM line_items WHERE line_items.order_id = orders.id AND line_items.discount_applied IS NULL) = 0")
This should also work pretty much everywhere (and has a bit more Arel and less manual SQL):
Order.left_joins(:line_items).where(line_items: { discount_applied: nil }).group("orders.id").having("COUNT(line_items.id) = 0")
Depending on your specific DBMS (more specifically: its respective query optimizer), one or the other might be more performant.
Hope that helps.
Not efficient but I thought it may solve your problem:
orders = Order.includes(:line_items).select do |order|
order.line_items.all? { |line_item| line_item.discount_applied.nil? }
end
Update:
Instead of finding orders which all it's line items have no discount, we can exclude all the orders which have line items with a discount applied from the output result. This can be done with subquery inside where clause:
# Find all ids of orders which have line items with a discount applied:
excluded_ids = LineItem.select(:order_id)
.where.not(discount_applied: nil)
.distinct.map(&:order_id)
# exclude those ids from all orders:
Order.where.not(id: excluded_ids)
You can combine them in a single finder method:
Order.where.not(id: LineItem
.select(:order_id)
.where.not(discount_applied: nil))
Hope this helps
A possible code
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
I advice to get familiar with AR documentation for Query Methods.
Update
This seems to be more interested than I initially though. And more complicated, so I will not be able to give you a working code. But I would look into a solution using LineItem.group(order_id).having(discount_applied: nil), which should give you a collection of line_items and then use it as sub-query to find related orders.
If you want all the records where discount_applied is nil then:
Order.includes(:line_items).where.not(line_items: {discount_applied: nil})
(use includes to avoid n+1 problem)
or
Order.joins(:line_items).where.not(line_items: {discount_applied: nil})
Here is the solution to your problem
order_ids = Order.joins(:line_items).where.not(line_items: {discount_applied: nil}).pluck(:id)
orders = Order.where.not(id: order_ids)
First query will return ids of Orders with at least one line_item having discount_applied. The second query will return all orders where there are zero instances of a line_item having the discount_applied.
I would use the NOT EXISTS feature from SQL, which is at least available in both MySQL and PostgreSQL
it should look like this
class Order
has_many :line_items
scope :without_discounts, -> {
where("NOT EXISTS (?)", line_items.where("discount_applied is not null")
}
end
If I understood correctly, you want to get all orders for which none line item (if any) has a discount applied.
One way to get those orders using ActiveRecord would be the following:
Order.distinct.left_outer_joins(:line_items).where(line_items: { discount_applied: nil })
Here's a brief explanation of how that works:
The solution uses left_outer_joins, assuming you won't be accessing the line items for each order. You can also use left_joins, which is an alias.
If you need to instantiate the line items for each Order instance, add .eager_load(:line_items) to the chain which will prevent doing an additional query for every order (N+1), i.e., doing order.line_items.each in a view.
Using distinct is essential to make sure that orders are only included once in the result.
Update
My previous solution was only checking that discount_applied IS NULL for at least one line item, not all of them. The following query should return the orders you need.
Order.left_joins(:line_items).group(:id).having("COUNT(line_items.discount_applied) = ?", 0)
This is what's going on:
The solution still needs to use a left outer join (orders LEFT OUTER JOIN line_items) so that orders without any associated items are included.
Groups the line items to get a single Order object regardless of how many items it has (GROUP BY recipes.id).
It counts the number of line items that were given a discount for each order, only selecting the ones whose items have zero discounts applied (HAVING (COUNT(line_items.discount_applied) = 0)).
I hope that helps.
You cannot do this efficiently with a classic rails left_joins, but sql left join was build to handle thoses cases
Order.joins("LEFT JOIN line_items AS li ON li.order_id = orders.id
AND li.discount_applied IS NOT NULL")
.where("li.id IS NULL")
A simple inner join will return all orders, joined with all line_items,
but if there are no line_items for this order, the order is ignored (like a false where)
With left join, if no line_items was found, sql will joins it to an empty entry in order to keep it
So we left joined the line_items we don't want, and find all orders joined with an empty line_items
And avoid all code with where(id: pluck(:id)) or having("COUNT(*) = 0"), on day this will kill your database
I have a model A associated to model B via INNER JOIN:
class A
has_many :bees, as: :bable
scope :bees, -> () {
joins("INNER JOIN bees AS b ON id = b.bable_id .......")
}
end
class B
table_name = "bees"
belongs_to :bable, polymorphic: true
end
I need to filter using B's datetime field (created_at), so I declared a new attribute thus:
has bees.created_at, as: :b_created_at
The sphinx query statement generated now includes:
GROUP_CONCAT(DISTINCT UNIX_TIMESTAMP(bees.`created_at`) SEPARATOR ',') AS `b_created_at`
After indexing, my sphinx index file size exploded.
How much is the "GROUP_CONCAT" part of the query causing the problem, and is there a better way to filter by this attribute?
How can I debug the indexer and find other causes of the large index file being generated?
Thanks
It appears that the indexer is creating, within the index file, a comma separated list of all created timestamps of all bees - as created timestamps are generally unique (!), this indexing is going to create one item for every bee. If you have a lot of bees then this is going to be big.
I would be looking at some way to bypass Sphinx for this part of the query if that is possible and get it to add a direct SQL BETWEEN LowDateTs AND HighDateTs against the built in created_at instead. I hope this is possible - it will definitely be better than using a text index to find it.
Hope this is of some help.
Edit:
Speed reading Sphinx' docs:
[...] WHERE clause. This clause will map both to fulltext query and filters. Comparison operators (=, !=, <, >, <=, >=), IN, AND, NOT, and BETWEEN are all supported and map directly to filters [...]
So the key is to stop it treating the timestamp as a text search and use a BETWEEN, which will be vastly more efficient and hopefully stop it trying to use text indexing on this field.
I am working on an asp.net mvc web application, and I am using Sql server 2008 R2 + Entity framework.
Now on the sql server I have added a unique index on any column that might be ordered by . for example I have created a unique index on the Sql server on the Tag colum and I have defined that the sort order for the index to be Ascending. Now I have some queries inside my application that order the tag ascending while other queries order the Tag descending, as follow:-
LatestTechnology = tms.Technologies.Where(a=> !a.IsDeleted && a.IsCompleted).OrderByDescending(a => a.Tag).Take(pagesize).ToList(),;
TechnologyList = tms.Technologies.Where(a=> !a.IsDeleted && a.IsCompleted).OrderBy (a => a.Tag).Take(pagesize).ToList();
So my question is whether the two OrderByDescending(a => a.Tag). & OrderBy(a => a.Tag), can benefit from the asending unique index on the sql server on the Tag colum ? or I should define two unique indexes on the sql server one with ascending sort order while the other index with decedning sort order ?
THanks
EDIT
the following query :-
LatestTechnology = tms.Technologies.Where(a=> !a.IsDeleted && a.IsCompleted).OrderByDescending(a => a.Tag).Take(pagesize).ToList();
will generate the following sql statement as mentioned by the sql server profiler :-
SELECT TOP (15)
[Extent1].[TechnologyID] AS [TechnologyID],
[Extent1].[Tag] AS [Tag],
[Extent1].[IsDeleted] AS [IsDeleted],
[Extent1].[timestamp] AS [timestamp],
[Extent1].[TypeID] AS [TypeID],
[Extent1].[StartDate] AS [StartDate],
[Extent1].[IT360ID] AS [IT360ID],
[Extent1].[IsCompleted] AS [IsCompleted]
FROM [dbo].[Technology] AS [Extent1]
WHERE ([Extent1].[IsDeleted] <> cast(1 as bit)) AND ([Extent1].[IsCompleted] = 1)
ORDER BY [Extent1].[Tag] DESC
To answer your question:
So my question is whether the two OrderByDescending(a => a.Tag). &
OrderBy(a => a.Tag), can benefit from the asending unique index on the
sql server on the Tag colum ?
Yes, SQL Server can read an index in both directions: as in index definition or in the exact opposite direction.
However, from your intro I suspect that you still have a wrong impression how indexing works for order by. If you have both, a where clause and an order by clause, you must make sure to have a single index that covers both clauses! It does not help to have on index for the where clause (like on isDeleted and isCompleted — whatever that is in your example) and another index on tag. You need to have a single index that first has the columns of the where clause followed by the columns of the order by clause (multi-column index).
It can be tricky to make it work correctly, but it's worth the effort especially if your are only fetching the first few rows (like in your example).
If it doesn't work out right away, please have a look at this:
http://use-the-index-luke.com/sql/sorting-grouping/indexed-order-by
It is generally best to show the actual SQL query—not the .NET source code—when asking for performance advice. Then I could tell you which index to create exactly. At the moment I'm unsure about isDeleted and isCompleted — are these table columns or expressions that evaluate upon other columns?
EDIT (after you added the SQL query)
There are two ways to make your query work as indexed top-n query:
http://sqlfiddle.com/#!6/260fb/4
The first option is a regular index on the columns from the where clause followed by those from the order by clause. However, as you query uses this filter IsDeleted <> cast(1 as bit) it cannot use the index in a order-preserving way. If, however, you re-phrase the query so that it reads like this IsDeleted = cast(0 as bit) then it works. Please look at the fiddle, I've prepared everything there. Yes, SQL Server could be smart enough to know that, but it seems like it isn't.
I don't know how to tweak EF to produce the query in the above described way, sorry.
However, there is a second option using a so called filtered index — that is an index that only contains a sub-set of the table rows. It's also in the SQL Fiddle. Here it is important that you add the where clause to the index definition in the very same way as it appears in your query.
In both ways it still works if you change DESC to ASC.
The important part is that the execution plan doesn't show a sort operation. You can also verify this in SQL Fiddle (click on 'View execution plan').
I have a set-up with multiple contests and objects. They are tied together with a has_many :through arrangement with contest_objs. contest_objs also has votes so I can have several contests including several objects. I have a complex SQL setup to calculate the current ranking. However, I need to specify the contest in the SQL select statement for the ranking. I am having difficulty doing this. This is what I got so far:
#objects = #contest.objects.select('"contest_objs"."votes" AS v, name, "objects"."id" AS id,
(SELECT COUNT(DISTINCT "oi"."object_id")
FROM contest_objs oi
WHERE ("oi"."votes") > ("contest_objs"."votes"))+1 AS vrank')
Is there any way in the selection of vrank to specify that WHERE also includes "oi"."contest_id" = #contest.id ?
Since #contest.id is an integer and does not present any risk of an SQL Injection, you could do the following using string interpolation :
Model.select("..... WHERE id = #{#contest.id}")
Another possible solution would be to build your subquery using ActiveRecord, and then call .to_sql in order to get the generated SQL, and insert it in your main query.
Use sanitize_sql_array:
sanitize_sql_array('select ? from foo', 'bar')
If you're outside a model, because the method is protected you have to do this:
ActiveRecord::Base.send(:sanitize_sql_array, ['select ? from foo', 'bar'])
http://apidock.com/rails/ActiveRecord/Sanitization/ClassMethods/sanitize_sql_array
You can insert variables into sql commands like this:
Model.select("...... WHERE id = ?", #contest.id)
Rails will escape the values for you.
Edit:
This does not work as stated by Intrepidd in the comments, use string interpolation like he suggested in his answer. That is safe for integer parameters.
If you find yourself inserting several strings in a query, you could consider using find_by_sql, which gives you the above mentioned ? replacement, but you can't use it with chaining, so rewriting the whole query would be needed.
I already have a working solution, but I would really like to know why this doesn't work:
ratings = Model.select(:rating).uniq
ratings.each { |r| puts r.rating }
It selects, but don't print unique values, it prints all values, including the duplicates. And it's in the documentation: http://guides.rubyonrails.org/active_record_querying.html#selecting-specific-fields
Model.select(:rating)
The result of this is a collection of Model objects. Not plain ratings. And from uniq's point of view, they are completely different. You can use this:
Model.select(:rating).map(&:rating).uniq
or this (most efficient):
Model.uniq.pluck(:rating)
Rails 5+
Model.distinct.pluck(:rating)
Update
Apparently, as of rails 5.0.0.1, it works only on "top level" queries, like above. Doesn't work on collection proxies ("has_many" relations, for example).
Address.distinct.pluck(:city) # => ['Moscow']
user.addresses.distinct.pluck(:city) # => ['Moscow', 'Moscow', 'Moscow']
In this case, deduplicate after the query
user.addresses.pluck(:city).uniq # => ['Moscow']
If you're going to use Model.select, then you might as well just use DISTINCT, as it will return only the unique values. This is better because it means it returns less rows and should be slightly faster than returning a number of rows and then telling Rails to pick the unique values.
Model.select('DISTINCT rating')
Of course, this is provided your database understands the DISTINCT keyword, and most should.
This works too.
Model.pluck("DISTINCT rating")
If you want to also select extra fields:
Model.select('DISTINCT ON (models.ratings) models.ratings, models.id').map { |m| [m.id, m.ratings] }
Model.uniq.pluck(:rating)
# SELECT DISTINCT "models"."rating" FROM "models"
This has the advantages of not using sql strings and not instantiating models
Model.select(:rating).uniq
This code works as 'DISTINCT' (not as Array#uniq) since rails 3.2
Model.select(:rating).distinct
Another way to collect uniq columns with sql:
Model.group(:rating).pluck(:rating)
If I am going right to way then :
Current query
Model.select(:rating)
is returning array of object and you have written query
Model.select(:rating).uniq
uniq is applied on array of object and each object have unique id. uniq is performing its job correctly because each object in array is uniq.
There are many way to select distinct rating :
Model.select('distinct rating').map(&:rating)
or
Model.select('distinct rating').collect(&:rating)
or
Model.select(:rating).map(&:rating).uniq
or
Model.select(:name).collect(&:rating).uniq
One more thing, first and second query : find distinct data by SQL query.
These queries will considered "london" and "london " same means it will neglect to space, that's why it will select 'london' one time in your query result.
Third and forth query:
find data by SQL query and for distinct data applied ruby uniq mehtod.
these queries will considered "london" and "london " different, that's why it will select 'london' and 'london ' both in your query result.
please prefer to attached image for more understanding and have a look on "Toured / Awaiting RFP".
If anyone is looking for the same with Mongoid, that is
Model.distinct(:rating)
Some answers don't take into account the OP wants a array of values
Other answers don't work well if your Model has thousands of records
That said, I think a good answer is:
Model.uniq.select(:ratings).map(&:ratings)
=> "SELECT DISTINCT ratings FROM `models` "
Because, first you generate a array of Model (with diminished size because of the select), then you extract the only attribute those selected models have (ratings)
You can use the following Gem: active_record_distinct_on
Model.distinct_on(:rating)
Yields the following query:
SELECT DISTINCT ON ( "models"."rating" ) "models".* FROM "models"
In my scenario, I wanted a list of distinct names after ordering them by their creation date, applying offset and limit. Basically a combination of ORDER BY, DISTINCT ON
All you need to do is put DISTINCT ON inside the pluck method, like follow
Model.order("name, created_at DESC").offset(0).limit(10).pluck("DISTINCT ON (name) name")
This would return back an array of distinct names.
Model.pluck("DISTINCT column_name")