I have a situation where I need to allow building up of SQL manually from a form. I have something like this:
SomeModel.where("id in (#{custom_sql})")
where custom_sql is a select statement like so:
SELECT u.id FROM some_model u WHERE country_iso = 'AU'
I want to be able to catch the StatementInvalid exception that is thrown when there is invalid SQL in the where clause, but I cannot figure out how to do so.
begin
SomeModel.where("id in (#{custom_sql})")
rescue
puts "Error"
end
But it keeps falling through without an error. Yet in rails c, when I do User.where("id in (#{custom_sql})"), it will correctly error. Any ideas?
Check the validity of your SQL statement using ActiveRecord::Base.connection.execute. It will throw an error if your statement is invalid.
begin
ActiveRecord::Base.connection.execute custom_sql
User.where("id in (#{custom_sql})")
rescue
puts "Error"
end
First of all your example is unclear or misleading (probably due to your efforts to mask your actual model names)
Because if you really wanted to run the scenario you are proposing, your resulting query would be
select * from some_model where id in (select u.id from some_model u where country_iso = "AU"
(ignoring the fact that your country_iso field is ambiguous) You are just running
SomeModel.where country_iso: "AU"
So you'll need to refine your example and properly illustrate your problem, because while #zeantsoi suggests a way to help you validate your statement(s), I agree with #phoet that you are probably approaching the situation incorrectly.
firstly, this is sql injection at it's best. you should reconsider whatever you want to accomplish.
secondly, User.where returns a relation. relations are lazy. lazy statements are not yet executed. so if you are doing User.where(...).all this should help.
Related
Following the principle of fail-fast:
When querying the database where there should only ever be one record, I want an exception if .first() (first) encounters more than one record.
I see that there is a first! method that throws if there's less records than expected but I don't see anything for if there's two or more.
How can I get active record to fail early if there are more records than expected?
Is there a reason that active record doesn't work this way?
I'm used to C#'s Single() that will throw if two records are found.
Why would you expect activerecord's first method to fails if there are more than 1 record? it makes no sense for it to work that way.
You can define your own class method the count the records before getting the first one. Something like
def self.first_and_only!
raise "more than 1" if size > 1
first!
end
That will raise an error if there are more than 1 and also if there's no record at all. If there's one and only one it will return it.
It seems ActiveRecord has no methods like that. One useful method I found is one?, you can call it on an ActiveRecord::Relation object. You could do
users = User.where(name: "foo")
raise StandardError unless users.one?
and maybe define your own custom exception
If you care enough about queries performance, you have to avoid ActiveRecord::Relation's count, one?, none?, many?, any? etc, which spawns SQL select count(*) ... query.
So, your could use SQL limit like:
def self.single!
# Only one fast DB query
result = limit(2).to_a
# Array#many? not ActiveRecord::Calculations one
raise TooManySomthError if result.many?
# Array#first not ActiveRecord::FinderMethods one
result.first
end
Also, when you expect to get only one record, you have to use Relation's take instead of first. The last one is for really first record, and can produce useless SQL ORDER BY.
find_sole_by (Rails 7.0+)
Starting from Rails 7.0, there is a find_sole_by method:
Finds the sole matching record. Raises ActiveRecord::RecordNotFound if no record is found. Raises ActiveRecord::SoleRecordExceeded if more than one record is found.
For example:
Product.find_sole_by(["price = %?", price])
Sources:
ActiveRecord::FinderMethods#find_sole_by.
Rails 7 adds ActiveRecord methods #sole and #find_sole_by.
Rails 7.0 adds ActiveRecord::FinderMethods 'sole' and 'find_sole_by'.
I want to do non-standard query with join based on the parameters.
For example I have 2 tables: a_examples and b_examples both with fields: field_x and field_y.
I want to join rows when both tables have the same values on field_x(or field_y).
Example query can looks like this:
AExample.joins('INNER JOIN b_examples ON b_examples.field_x = a_examples.field_x')
The problem occurs when I have field name based on parameter.
For example I have variable field_name and want to use it for query. I can do it like this:
AExample.joins("INNER JOIN b_examples ON b_examples.#{field_name} = a_examples.#{field_name}")
This query works, but is susceptible for sql injection.
For where clause we have special syntax with ? to avoid sql injection but there isnt any such thing for joins. How can I make this query safe?
Do not attempt this:
(explanation below)
You can use the ActiveRecord::Sanitization module and write something like the following, inside your ActiveRecord model:
AExample.joins("INNER JOIN b_examples ON b_examples.#{sanitize_sql(field_name)} = a_examples.#{sanitize_sql(field_name)}")
Or you can include the module somewhere else and use it there (e.g. your controller).
Do use this, instead: (included from another answer)
AExample.joins("INNER JOIN b_examples ON b_examples.#{ActiveRecord::Base.connection.quote_column_name(field_name)} = a_examples.#{ActiveRecord::Base.connection.quote_column_name(field_name)}")
It will raise an error if the column is not found, preventing malicious code from entering your query.
However I wouldn't do this in my app as it looks suspicious, other programers may not understand what is happening, it may be implemented wrong, solid testing should be included, it may have bugs and such. In your problem, you only need to construct two different queries, with that information I would write something like:
case dynamic_field
when 'field_x'
AExample.joins('INNER JOIN b_examples ON b_examples.field_x = a_examples.field_x')
when 'field_y'
AExample.joins('INNER JOIN b_examples ON b_examples.field_y = a_examples.field_y')
else
raise "Some suspicious parameter was sent!: #{dynamic_field}"
end
Or even write scopes on your model and avoid this code to be flying around.
With problems of this nature, as with encryption, try to find a workaround and avoid implementing your own solutions as much as possible.
EDIT:
The method sanitize_sql is intended to sanitize conditions for a WHERE clause (ActiveRecord::Sanitization):
Accepts an array or string of SQL conditions and sanitizes them into a valid SQL fragment for a WHERE clause.
It is not an option as you try to sanitize for an INNER JOIN, or an ON clause.
Note that the ActiveRecord::Sanitization module only has options for WHERE, SET, ORDER and LIKE clauses. I was unable to find a sanitization method for a column name, an INNER JOIN or an ON clause. Perhaps is a useful funcionality that should be added on Rails on further version.
Using sanitize_sql with a string passes it almost unfiltered, so if the field_name variable has some malicious code as:
"field_x = a_examples.field_x; DROP TABLE a_examples; --"
It will be included in your query, without any error being raised.
This solution is not safe, and is for reasons like these that we should avoid writing code of this nature. Perhaps you find something helpful with Arel or other gems, but I would strongly advice not to.
EDIT 2:
Added the working solution to escape a column name. It raises an error if malicious code is being entered, as the column with that name will not be found.
You can sanitize parameters using ActiveRecord::Base.connection.quote(string) or even .quote_column
I have this index in my Model's table:
UNIQUE KEY `index_panel_user_offer_visits_on_offer_id_and_panel_user_id` (`offer_id`,`panel_user_id`)
And this code:
def get_offer_visit(offer_id, panel_user_id)
PanelUserOfferVisit.where(:offer_id => offer_id, :panel_user_id => panel_user_id).first_or_create!
end
is randomly causing an ActiveRecord::RecordNotUnique exception in our application,
the issue is:
Duplicate entry for key
'index_panel_user_offer_visits_on_offer_id_and_panel_user_id'
I've read on the rails doc that first_or_create! is not an atomic method and can cause this kind of issues in a high concurrency environment and that a solution would be just to catch the exception and retry.
I tried different approaches, including Retriable gem to retry a certain number of times to repeat the operation but RecordNotUnique is still raised.
I even tried to change the logic:
def get_offer_visit(offer_id, panel_user_id)
begin
offer_visit = PanelUserOfferVisit.where(:offer_id => offer_id, :panel_user_id => panel_user_id).first_or_initialize
offer_visit.save!
offer_visit
rescue ActiveRecord::RecordNotUnique
offer_visit = PanelUserOfferVisit.where(:offer_id => offer_id, :panel_user_id => panel_user_id).first
raise Exception.new("offer_visit not found") if offer_visit.nil?
offer_visit
end
end
But the code still raise the custom exception I made and I really don't understand how it can fails to create the record on the first try because the record exists but then when it tries to find the record it doesn't find it again.
What am I missing?
You seem to be hitting rails query cache for PanelUserOfferVisit.where(...).first query: rails have just performed identical SQL query, it thinks that result will remain the same for the duration of the request.
ActiveRecord::Base.connection.clear_query_cache before the second call to db should help
Also you do not have to save! the record if it is not new_record?
If I understand properly your problem here is due to a race condition. I didn't found the exact piece of code from Github but lets say the method find_or_create is something like (edit: here is the code):
def first_or_create!(attributes = nil, &block)
first || create!(attributes, &block)
end
You should try to solve the race conditions using optimistic or pessimistic locking
Once locking implemented within your custom method, you should continue capturing possible NotUnique exceptions but try to clean cache or use uncached block before to perform the find again.
I experienced a similar issue but the root cause was not related to a race condition.
With first_or_create!, the INSERT remove surrounding spaces in the value but the SELECT don't.
So if you try this:
Users.where(email: 'user#mail.com ')
(mind the space at the end of the string), the resulting sql query will be:
SELECT "users".* FROM "users" WHERE "users"."email" = 'user#mail.com ' ORDER BY "users"."id";
which result in not finding the potentially existing record, then the insert query will be:
INSERT INTO "users" ("email") VALUES ('user#mail.com');
which can result in ActiveRecord::RecordNotUnique error if a record with 'user#mail.com' as an email already exists.
I do not have enough details about your problem so it might be unrelated but it might helps others.
I want to achieve the following:
Model.where("asdasd = ? AND to <= ?", nil, Time.now).each do |model|
I get the following error:
ActiveRecord::StatementInvalid: SQLite3::SQLException: near "to": syntax error: SELECT "bla".* FROM "table" WHERE (asdasd = NULL AND to <= '2015-11-09 10:18:14.777643')
I also use the same in another controller, where I get the same error. What would be the correct way to achieve what I want? I am also quite sure that it worked already like that, could there be any other thing which causes this issue?
Thanks!!!
TO is an SQL keyword, and needs to be quoted in a query. the list of SQLite keywords is avaiable in the SQLite documentation.
Other database engines have similar (but not identical) rules and lists of keywords, so if you change from SQLite then check the rules for whatever the new engine is.
I'm having a really confusing problem. Here it is:
[12] pry(EstimatedTime)> EstimatedTime.where(user_id: User.current.id, plan_on: date).pluck(:hours) => []
[13] pry(EstimatedTime)> EstimatedTime.where(user_id: User.current.id, plan_on: date).sum(:hours) => 3.0
What kind of magic is this?
This statement resides in model method, that is being called from view. Before that, in controller action, i'm calling another method of the same model, that is performing bulk create of records within transaction.
def self.save_all(records)
transaction do
records.each do |record|
record.save!
end
end
rescue ActiveRecord::RecordInvalid
return false
end
The exception is being thrown, method returns false, view is rendered and this happens.
UPD
I found a workaround, replacing .sum with .pluck(:hours).sum, but I still have no idea why my first way of doing this fails.
As David Aldrige pointed out in comments to the question, the problem was that .sum(:hours) was using cached data, while .pluck(:hours) was actually looking into database.
Why database and query cache contained different data? Well, seems like in Rails 3 (I should probably have specified my rails version when asking question) failed transaction would rollback the database, but leave query cache intact. This led to temporary data inconsistency between database and cache. This was corrected in Rails 4 as pointed out in this issue.
Solution 1 aka the one I chose for myself
Clear the query cache after transaction fails. So, the method performing the mass insert within transaction now looks like this:
def self.save_all(records)
transaction do
records.each do |record|
record.save!
end
end
rescue ActiveRecord::RecordInvalid
self.connection.clear_query_cache
return false
end
Solution 2
I personally find this much less elegant, though I cannot compare these solutions performance-wise, so I'll post them both. One can simply write the sum statement like this:
EstimatedTime.where(user_id: User.current.id, plan_on: date).pluck(:hours).sum
It will use the database data to calculate sum, bypassing the inconsistent cache problem.