I am attempting to order by a column of type "character varying []" and cast it to an integer[] while sorting (instead of using the default ASCII comparison sort). I am using a Postgresql database.
I've found that the following query works as expected:
select <col> from <table> order by <col>::integer[] desc
Unfortunately, when I attempt to programmatically do this in rails it is adding quotes around the column and casting suffix. This results in it thinking "::integer[]" is part of the column name - and of course there is no such column. Thus the query fails.
Here is the rails code:
scope.order([ '<col>::integer[]', 'desc', 'NULLS LAST' ].join(' '))
And this is the query it produces:
select <col> from <table> order by "<table>"."<col>::integer[]" desc
How can I implement this properly with rails?
Thanks to #engineersmnky comment, I found the solution that I need in my code.
In the code I'm actually processing an array of columns (and directions) to sort by. It turns out the solution was indeed to use the Arel.sql() function to process the order by parameters prior to calling scope.order(), with the end result looking something like this:
def sort(scope, sorts)
str = \
sorts.map |sort| do
col = get_sort_column_alias(sort[0])
dir = sort[1]
nullpos = (dir == 'asc') ? 'FIRST' : 'LAST'
"#{col} #{dir} NULL #{nullpos}"
end
scope.order(Arel.sql(str))
end
def get_sort_column_alias(col)
case col
when 'target' then 'target::integer[]'
...
else col
end
end
Related
Using active record, I want to perform a lookup that returns a collection of items that have ALL matching id's.
Given that the below example matches on ANY id in the array, I am trying to figure out the syntax so that it will match when ALL of the id's match. (given that in this example there is a many to many relationship).
The array length of the id's is also variable which prohibits chaining .where()
x.where(id: [1,2])
Note: this question got removed before and there are a lot of answers for performing a sql "where in" but this question is about performing a sql "where and"
You can use exec_query and execute your own bound query:
values = [1, 2]
where_condition = values.map.with_index(1) { |_, index| "id = $#{index}" }.join(" AND ")
sql = "SELECT * FROM table WHERE #{ where_condition }"
binds = values.map { |i| ActiveRecord::Relation::QueryAttribute.new(nil, i, ActiveRecord::Type::Integer.new) }
ActiveRecord::Base.connection.exec_query(sql, nil, binds)
I completely agree with #muistooshort's comment
where(id: [1,2]) doesn't make sense unless you're joining to an association table and in that case,..."where in" combined with HAVING [solves your problem].
But for the sake of answering the question and the assumption that id was just and example.
While #SebastianPalma's answer will work it will return an ActiveRecord::Result whereas most of the time the desire is an ActiveRecord::Relation.
We can achieve this by using Arel to build the where clause like so:
(I modified the example to use description rather than id so that it makes more logical sense)
table = MyObject.arel_table
values = ['Jamesla','Example']
where_clause = values.map {|v| table[:description].matches("%{v}%")}.reduce(&:and)
# OR
where_clause = table[:description].matches_all(values.map {|v| "%#{v}%"})
MyObject.where(where_clause)
This will result in the following SQL query:
SELECT
my_objects.*
FROM
my_objects
WHERE
my_objects.description LIKE '%Jamesla%'
AND my_objects.description LIKE '%Example%'
I have a form that has nested field (habtm and accepts_nested_attributes_for). That form contains with a field "keywords", that autocompletes keywords that come from a postgresql table.
All that works well. This is in params :
"acte"=>{"biblio_id"=>"1", "keywords"=>{"keywords"=>"judge, ordeal, "}
What I now need to do is take those keywords and get their keywords_id out of the table keywords. Those id must be added to the join table.
I'm doing this :
q = params[:acte][:keywords].fetch(:keywords).split(",")
a = q.map {|e| Keyword.find_by keyword: e }
As per the guides, find_by returns only the first matching field. I guess I would need to use find_each but I'm not certain about that and I can't get it to word. I have tried this:
q = params[:acte][:motclefs].fetch(:motclefs).split(",")
a = Array.new
Motclef.where(motcle: q).find_each do |mot|
a << mot.id
end
This also finds only the first result like : [251].
What I'm looking to get is something like [1453, 252, 654]
thanks !
Putting find_by in a loop means you will be executing a separate SQL query for each SQL keyword.
You can instead just get all the ids in a single SQL call by doing keyword in.
After you do q = params[:acte][:keywords].fetch(:keywords).split(","), your q will be an array of keywords. So q will be ["judge", " ordeal"].
You can simply do Keyword.where(keyword: q).select(:id) which will generate a query like SELECT keywords.id FROM keywords where keyword in ('judge', 'ordeal').
I'm building a generalized reporting tool for rails, and I'd like to not only get the column names of a raw SQL query, but the converted ruby type as well. I'm going to use the type to make the interface a little better.
The following works, but surely there's a more "rails"-way to approach this? The query is just an example, it could potentially span every table using whatever dynamic SQL the user wants.
sql = "SELECT * FROM SOME_TABLE"
results = ActiveRecord::Base.connection.raw_connection.exec(sql)
results.nfields.times do |i|
puts results.fname(i)
name = results.fname(i)
typename = DataSet.connection.raw_connection.
exec( "SELECT format_type($1,$2)", [results.ftype(i), results.fmod(1)] ).
getvalue( 0, 0 )
column = ActiveRecord::ConnectionAdapters::Column.new(name, nil, typename)
puts column.klass # gives a decent assumption of type
end
ModelX.columns.each do |column|
puts column.sql_type # int(11)
puts column.type # :integer
end
This assumes you have an ActiveRecord model for each table.
I'm trying to pass a string into the .order method, such as
Item.order(orderBy)
I was wondering if orderBy gets sanitized by default and if not, what would be the best way to sanitize it.
The order does not get sanitized. This query will actually drop the Users table:
Post.order("title; drop table users;")
You'll want to check the orderBy variable before running the query if there's any way orderBy could be tainted from user input. Something like this could work:
items = Item.scoped
if Item.column_names.include?(orderBy)
items = items.order(orderBy)
end
They are not sanitized in the same way as a .where clause with ?, but you can use #sanitize_sql_for_order:
sanitize_sql_for_order(["field(id, ?)", [1,3,2]])
# => "field(id, 1,3,2)"
sanitize_sql_for_order("id ASC")
# => "id ASC"
http://api.rubyonrails.org/classes/ActiveRecord/Sanitization/ClassMethods.html#method-i-sanitize_sql_for_order
Just to update this for Rails 5+, as of this writing, passing an array into order will (attempt to) sanitize the right side inputs:
Item.order(['?', "'; DROP TABLE items;--"])
#=> SELECT * FROM items ORDER BY '''; DROP TABLE items;--'
This will trigger a deprecation warning in Rails 5.1 about a "Dangerous query method" that will be disallowed in Rails 6. If you know the left hand input is safe, wrapping it in an Arel.sql call will silence the warning and, presumably, still be valid in Rails 6.
Item.order([Arel.sql('?'), "'; DROP TABLE items;--"])
#=> SELECT * FROM items ORDER BY '''; DROP TABLE items;--'
It's important to note that unsafe SQL on the left side will be sent to the database unmodified. Exercise caution!
If you know your input is going to be an attribute of your model, you can pass the arguments as a hash:
Item.order(column_name => sort_direction)
In this form, ActiveRecord will complain if the column name is not valid for the model or if the sort direction is not valid.
I use something like the following:
#scoped = #scoped.order Entity.send(:sanitize_sql, "#{#c} #{#d}")
Where Entity is the model class.
Extend ActiveRecord::Relation with sanitized_order.
Taking Dylan's lead I decided to extend ActiveRecord::Relation in order to add a chainable method that will automatically sanitize the order params that are passed to it.
Here's how you call it:
Item.sanitized_order( params[:order_by], params[:order_direction] )
And here's how you extend ActiveRecord::Relation to add it:
config/initializers/sanitized_order.rb
class ActiveRecord::Relation
# This will sanitize the column and direction of the order.
# Should always be used when taking these params from GET.
#
def sanitized_order( column, direction = nil )
direction ||= "ASC"
raise "Column value of #{column} not permitted." unless self.klass.column_names.include?( column.to_s )
raise "Direction value of #{direction} not permitted." unless [ "ASC", "DESC" ].include?( direction.upcase )
self.order( "#{column} #{direction}" )
end
end
It does two main things:
It ensures that the column parameter is the name of a column name of the base klass of the ActiveRecord::Relation.
In our above example, it would ensure params[:order_by] is one of Item's columns.
It ensures that the direction value is either "ASC" or "DESC".
It can probably be taken further but I find the ease of use and DRYness very useful in practice when accepting sorting params from users.
Is there a way to sanitize sql in rails method find_by_sql?
I've tried this solution:
Ruby on Rails: How to sanitize a string for SQL when not using find?
But it fails at
Model.execute_sql("Update users set active = 0 where id = 2")
It throws an error, but sql code is executed and the user with ID 2 now has a disabled account.
Simple find_by_sql also does not work:
Model.find_by_sql("UPDATE user set active = 0 where id = 1")
# => code executed, user with id 1 have now ban
Edit:
Well my client requested to make that function (select by sql) in admin panel to make some complex query(joins, special conditions etc). So I really want to find_by_sql that.
Second Edit:
I want to achieve that 'evil' SQL code won't be executed.
In admin panel you can type query -> Update users set admin = true where id = 232 and I want to block any UPDATE / DROP / ALTER SQL command.
Just want to know, that here you can ONLY execute SELECT.
After some attempts I conclude sanitize_sql_array unfortunatelly don't do that.
Is there a way to do that in Rails??
Sorry for the confusion..
Try this:
connect = ActiveRecord::Base.connection();
connect.execute(ActiveRecord::Base.send(:sanitize_sql_array, "your string"))
You can save it in variable and use for your purposes.
I made a little snippet for this that you can put in initializers.
class ActiveRecord::Base
def self.escape_sql(array)
self.send(:sanitize_sql_array, array)
end
end
Right now you can escape your query with this:
query = User.escape_sql(["Update users set active = ? where id = ?", true, params[:id]])
And you can call the query any way you like:
users = User.find_by_sql(query)
Slightly more general-purpose:
class ActiveRecord::Base
def self.escape_sql(clause, *rest)
self.send(:sanitize_sql_array, rest.empty? ? clause : ([clause] + rest))
end
end
This one lets you call it just like you'd type in a where clause, without extra brackets, and using either array-style ? or hash-style interpolations.
User.find_by_sql(["SELECT * FROM users WHERE (name = ?)", params])
Source: http://blog.endpoint.com/2012/10/dont-sleep-on-rails-3-sql-injection.html
Though this example is for INSERT query, one can use similar approach for UPDATE queries. Raw SQL bulk insert:
users_places = []
users_values = []
timestamp = Time.now.strftime('%Y-%m-%d %H:%M:%S')
params[:users].each do |user|
users_places << "(?,?,?,?)" # Append to array
users_values << user[:name] << user[:punch_line] << timestamp << timestamp
end
bulk_insert_users_sql_arr = ["INSERT INTO users (name, punch_line, created_at, updated_at) VALUES #{users_places.join(", ")}"] + users_values
begin
sql = ActiveRecord::Base.send(:sanitize_sql_array, bulk_insert_users_sql_arr)
ActiveRecord::Base.connection.execute(sql)
rescue
"something went wrong with the bulk insert sql query"
end
Here is the reference to sanitize_sql_array method in ActiveRecord::Base, it generates the proper query string by escaping the single quotes in the strings. For example the punch_line "Don't let them get you down" will become "Don\'t let them get you down".
I prefer to do it with key parameters. In your case it may looks like this:
Model.find_by_sql(["UPDATE user set active = :active where id = :id", active: 0, id: 1])
Pay attention, that you pass ONLY ONE parameter to :find_by_sql method - its an array, which contains two elements: string query and hash with params (since its our favourite Ruby, you can omit the curly brackets).