I have a model with an has_many association: Charts has_many ChartConditions
charts model has fields for:
name (title)
table_name (model)
The chart_conditions model has fields for
assoc_name (to .joins)
name (column)
value (value)
operator (operator
Basically my Chart tells us which model (using the table_name field) I want to run a dynamic query on. Then the chart_conditions for the Chart will tell us which fields in that model to sort on.
So In my models that will be queried, i need to dynamically build a where clause using multiple chart_conditions.
Below you can see that i do a joins first based on all the object's assoc_name fields
Example of what I came up with. This works, but not with a dynamic operator for the name/value and also throws a deprecation warning.
def self.dynamic_query(object)
s = joins(object.map{|o| o.assoc_name.to_sym})
#WORKS BUT GIVES DEPRECATED WARNING (RAILS4)
object.each do |cond|
s = s.where(cond.assoc_name.pluralize.to_sym => {cond.name.to_sym => cond.value})
end
end
How can i then add in my dynamic operator value to this query? Also why can I not say:
s = s.where(cond.assoc_name.pluralize : {cond.name : cond.value})
I have to use the => and .to_sym to get it to work. The above syntax errors: syntax error, unexpected ':' ...ere(cond.assoc_name.pluralize : {cond.name : cond.value}) ... ^
What if you store the query in a variable and append to that?
def self.dynamic_query(object)
q = joins(object.map{|o| o.assoc_name.to_sym})
object.each do |cond|
q = q.where(cond.assoc_name.pluralize : {cond.name : cond.value})
end
q # returns the full query
end
Another approach might be the merge(other) method. From the API Docs:
Merges in the conditions from other, if other is an ActiveRecord::Relation. Returns an array representing the intersection of the resulting records with other, if other is an array.
Post.where(published: true).joins(:comments).merge( Comment.where(spam: false) )
# Performs a single join query with both where conditions.
That could be useful to knot all the conditions together.
Related
I have a Deal model with a lot of associations. One of the associations is Currency. The deals table and the currencies table both have a name column. Now, I have the following ActiveRecord query:
Deal.
joins(:currency).
where("privacy = ? or user_id = ?", false, doorkeeper_token.resource_owner_id).
select("deals.name as deal_name, deals.date as deal_creation_date, deals.amount as deal_amount, currencies.name as currency_name, currencies.symbol as currency_symbol")
This query doesn't work, its result is an array of Deal objects with no attributes. According to someone on IRC, the "as" parts are incorrect because the ORM doesn't know how to assign the columns to which attribute (or something like that) which is fair enough. I tried to add attr_accessor and attr_accessible clauses though but it didn't work.
How can I make the above query work please? What I expect the result to be is an Array of Deal objects with deal_name, deal_creation_date, etc. virtual attributes.
Most likely the query is working correctly, but the returned Deal objects appear not to have any attributes when you print them out because of the way that Deal implements the inspect method.
So if you assign the result of the query to a variable you will see that this appears empty:
v.each do |deal| ; puts deal.inspect ; end
While this shows the attributes you want:
v.each do |deal| ; puts deal.to_yaml ; end
More details are in this question.
I'm trying to pass a string into the .order method, such as
Item.order(orderBy)
I was wondering if orderBy gets sanitized by default and if not, what would be the best way to sanitize it.
The order does not get sanitized. This query will actually drop the Users table:
Post.order("title; drop table users;")
You'll want to check the orderBy variable before running the query if there's any way orderBy could be tainted from user input. Something like this could work:
items = Item.scoped
if Item.column_names.include?(orderBy)
items = items.order(orderBy)
end
They are not sanitized in the same way as a .where clause with ?, but you can use #sanitize_sql_for_order:
sanitize_sql_for_order(["field(id, ?)", [1,3,2]])
# => "field(id, 1,3,2)"
sanitize_sql_for_order("id ASC")
# => "id ASC"
http://api.rubyonrails.org/classes/ActiveRecord/Sanitization/ClassMethods.html#method-i-sanitize_sql_for_order
Just to update this for Rails 5+, as of this writing, passing an array into order will (attempt to) sanitize the right side inputs:
Item.order(['?', "'; DROP TABLE items;--"])
#=> SELECT * FROM items ORDER BY '''; DROP TABLE items;--'
This will trigger a deprecation warning in Rails 5.1 about a "Dangerous query method" that will be disallowed in Rails 6. If you know the left hand input is safe, wrapping it in an Arel.sql call will silence the warning and, presumably, still be valid in Rails 6.
Item.order([Arel.sql('?'), "'; DROP TABLE items;--"])
#=> SELECT * FROM items ORDER BY '''; DROP TABLE items;--'
It's important to note that unsafe SQL on the left side will be sent to the database unmodified. Exercise caution!
If you know your input is going to be an attribute of your model, you can pass the arguments as a hash:
Item.order(column_name => sort_direction)
In this form, ActiveRecord will complain if the column name is not valid for the model or if the sort direction is not valid.
I use something like the following:
#scoped = #scoped.order Entity.send(:sanitize_sql, "#{#c} #{#d}")
Where Entity is the model class.
Extend ActiveRecord::Relation with sanitized_order.
Taking Dylan's lead I decided to extend ActiveRecord::Relation in order to add a chainable method that will automatically sanitize the order params that are passed to it.
Here's how you call it:
Item.sanitized_order( params[:order_by], params[:order_direction] )
And here's how you extend ActiveRecord::Relation to add it:
config/initializers/sanitized_order.rb
class ActiveRecord::Relation
# This will sanitize the column and direction of the order.
# Should always be used when taking these params from GET.
#
def sanitized_order( column, direction = nil )
direction ||= "ASC"
raise "Column value of #{column} not permitted." unless self.klass.column_names.include?( column.to_s )
raise "Direction value of #{direction} not permitted." unless [ "ASC", "DESC" ].include?( direction.upcase )
self.order( "#{column} #{direction}" )
end
end
It does two main things:
It ensures that the column parameter is the name of a column name of the base klass of the ActiveRecord::Relation.
In our above example, it would ensure params[:order_by] is one of Item's columns.
It ensures that the direction value is either "ASC" or "DESC".
It can probably be taken further but I find the ease of use and DRYness very useful in practice when accepting sorting params from users.
I am struggling with the best way to meta program a dynamic method, where I'll be limiting results based on conditions... so for example:
class Timeslip < ActiveRecord::Base
def self.by_car_trans(car, trans)
joins(:car)
.where("cars.trans IN (?) and cars.year IN (?) and cars.model ILIKE ?", trans, 1993..2002, car)
.order('et1320')
end
end
Let's say instead of passing in my arguments, i pass in an array of conditions with key being the fieldname, and value being the field value. so for example, I'd do something like this:
i'd pass in [["field", "value", "operator"],["field", "value", "operator"]]
def self.using_conditions(conditions)
joins(:car)
conditions.each do |key, value|
where("cars.#{key} #{operator} ?", value)
end
end
However, that doesn't work, and it's not very flexible... I was hoping to be able to detect if the value is an array, and use IN () rather than =, and maybe be able to use ILIKE for case insensitive conditions as well...
Any advice is appreciated. My main goal here is to have a "lists" model, where a user can build their conditions dynamically, and then save that list for future use. This list would filter the timeslips model based on the associated cars table... Maybe there is an easier way to go about this?
First of all, you might find an interest in the Squeel gem.
Other than that, use arel_table for IN or LIKE predicates :
joins( :car ).where( Car.arel_table[key].in values )
joins( :car ).where( Car.arel_table[key].matches value )
you can detect the type of value to select an adequate predicate (not nice OO, but still):
column = Car.arel_table[key]
predicate = value.respond_to?( :to_str ) ? :in : :matches # or any logic you want
joins( :car ).where( column.send predicate, value )
you can chain as many as those as you want:
conditions.each do |(key, value, predicate)|
scope = scope.where( Car.arel_table[key].send predicate, value )
end
return scope
So, you want dynamic queries that end-users can specify at run-time (and can be stored & retrieved for later use)?
I think you're on the right track. The only detail is how you model and store your criteria. I don't see why the following won't work:
def self.using_conditions(conditions)
joins(:car)
crit = conditions.each_with_object({}) {|(field, op, value), m|
m["#{field} #{op} ?"] = value
}
where crit.keys.join(' AND '), *crit.values
end
CAVEAT The above code as is is insecure and prone to SQL injection.
Also, there's no easy way to specify AND vs OR conditions. Finally, the simple "#{field} #{op} ?", value for the most part only works for numeric fields and binary operators.
But this illustrates that the approach can work, just with a lot of room for improvement.
The question
I'm trying to wrap my head around arel and squeel, but I feel like I lack the vocabulary to ask Google what I'm looking for.
TL;DR: Does anyone know how to mimic Squeel's Model.where{related.objects.field.matches string} syntax out of composed Squeel::Node objects?
The problem
This question demonstrates how to build a Squeel KeyPath from a string, like so:
search_relation = 'person.pets'
User.joins{Squeel::Nodes::KeyPath.new(search_relation.split('.'))}
# Mimics User.joins{person.pets}
The plot thickens in this post, where I learned how to metaprogram a search across multiple columns. Cleaned up with a documented Squeel trick (the last line on the page), it looks like this:
search_columns = %w[first_name last_name]
search_term = 'chris'
User.where{
search_columns.map do |column|
column_stub = Squeel::Nodes::Stub.new(column)
Squeel::Nodes::Predicate.new(column_stub, :matches, "%#{search_term}%")
end.compact.inject(&:|)
}
# Mimics User.where{(last_name.matches '%chris%') | (first_name.matches '%chris%')}
What I want is to be able to combine the two, mimicking the syntax of
User.joins{person.pets}.where{person.pets.name.matches '%polly%'}
by building the squeel objects myself.
The first part is easy, demonstrated in the first example. The second part I can do for columns directly on the table being queried, demonstrated in the second example.
However, I can't figure out how to do the second part for columns on joined in tables. I think I need to use the KeyPath object from the first example to scope the Predicate object in the second example. The Predicate object only seems to work with a Stub object that represents a single column on a single table, not a KeyPath.
Previous attempts
Assumming
search_relation = 'key.path'
search_column = 'field'
search_term = '%search%'
keypath = Squeel::Nodes::KeyPath.new(search_relation.split('.') << search_column)
I've tried calling the matches method on the KeyPath:
Model.where{keypath.matches search_term}
#=> NoMethodError: undefined method `matches' for Squeel::Nodes::KeyPath
I've tried using the matches operator, =~:
Model.where{keypath =~ search_term}
#=> TypeError: type mismatch: String given
And I've tried building the Predicate manually, passing in a :matches parameter:
Model.where{Squeel::Nodes::Predicate.new(keypath, :matches, search_term)}
#=> ActiveRecord::StatementInvalid: Mysql2::Error: Unknown column
#=> 'model.key.path.field' in 'where clause':
#=> SELECT `model`.* FROM `model`
#=> INNER JOIN `key` ON `key`.`id` = `model`.`key_id`
#=> INNER JOIN `path` ON `path`.`id` = `key`.`path_id`
#=> WHERE `model`.`key.path.field` LIKE '%search%'
Here, it successfully generates SQL (correctly joining using Model.joins{search_relation} like above, omitted for readability) then assumes the KeyPath is a field instead of traversing it.
Chris,
You're on the right track. Simply calling the matches method on the KeyPath object will create a matches predicate with the keypath on the left side and the argument to matches on the right. Squeel's visitor will traverse the keypath on the left, arrive at the attribute in question, and build the matches ARel predicate against it.
While the KeyPath in the first example is sufficient for joining tables, a KeyPath to be used in WHERE clauses needs to be constructed a little more carefully, by ensuring the last value is indeed a Stub.
search_relation = 'key.path'
search_column = 'field'
search_stub = Squeel::Nodes::Stub(search_column)
search_term = '%search%'
keypath = Squeel::Nodes::KeyPath.new(search_relation.split('.') << search_stub)
Model.where{Squeel::Nodes::Predicate.new(keypath, :matches, search_term)}
#=> SELECT `model`.* FROM `model`
#=> INNER JOIN `key` ON `key`.`id` = `model`.`key_id`
#=> INNER JOIN `path` ON `path`.`id` = `key`.`path_id`
#=> WHERE `path`.`field` LIKE '%search%'
I am trying to build a query that compares two object, and if they have the same id the record is not fetched. What I have is this:
#channels.each do |channel|
unless #guide_channels.where(:id => channel.id).exists?
#dropdown_channels = #dropdown_channels.where(:id => channel.id)
end
end
This creates the query, but puts a AND between every value which isn't what I have in mind. I want the "or" operator. Is there a 'orwhere' function I can use or is there a better way to do this with some compare function?
The point is that the .where() method of a AR::Relation objects adds the condition to a set of conditions that are then AND-ed together when the query is executed.
What you have to do is a query like NOT IN:
# select all the ids for related #guide_channels
# if #channels comes from a query (so it's a ActiveRecord::Relation):
guide_ids = #guide_channels.where(:id => #channels.pluck(:id)).pluck(:id)
# use .where(:id => #channels.map {|c| c.id}) instead if #channels is just an array
# then use the list to exclude results from the following.
#dropdown_channels = #dropdown_channels.where("id NOT IN (?)", guide_ids.to_a)
The first query will accumulate all the IDs for channels that have an entry in #guide_channels. The second one will use the result of the first to exclude the found channels from tthe results for the dropdown.
This strange behavior is due to ActiveRecord's lazy evaluation of scopes.
What happens is that the line
#dropdown_channels = #dropdown_channels.where(:id => channel.id)
Does not send a query to the DB until you actually use the value of #dropdown_channels, and when you do it concatenates all the conditions into one big query, this is why you get the AND between the conditions.
In order to force ActiveRecord to eager load the scope you can use either the all scope or the first scope, for example:
#dropdown_channels = #dropdown_channels.where(:id => channel.id).first
This will force ActiveRecord to calculate the query on spot returning the result immediately and not to accumulate the scopes for lazy evaluation.
Another approach could be to accumulate all those channels_ids and get them later in one query, instead of making a query for each one. This approach is more cost-effective relating to DB resources.
In order to achieve this :
dropdown_channels_ids = []
#channels.each do |channel|
unless #guide_channels.where(:id => channel.id).exists?
dropdown_channels_ids << channel.id
end
end
#dropdown_channels = #dropdown_channels.where(:id => dropdown_channels_ids)