I need to run sql query like
sql = 'SELECT * FROM users WHERE id != ' + self.id.to_s + ' AND id NOT IN (SELECT artner_id FROM encounters WHERE user_id = ' + self.id.to_s + ')'
sql += ' AND id NOT IN (SELECT user_id FROM encounters WHERE partner_id = ' + self.id.to_s + ' AND predisposition = ' + Encounter::Negative.to_s + ')'
sql += ' AND cfg_sex = ' + self.sex.to_s + ' AND cfg_country = ' + self.country.to_s + ' AND cfg_city = ' + self.city.to_s
sql += ' ORDER BY rand() LIMIT 1'
It can be executed by AR.find_by_sql, but the code before is bad readable.
Are there any query builder, which can build that query?
For example, Kohana (it is PHP framework, I am php developer, but I want to change that kid-language to ruby/rails) have a query builder, which works like this:
$sql = DB::select('*')->from('users');
$sql->where('id', 'NOT_IN', DB::expr('SELECT partner_id FROM encounters WHERE user_id = '.$user->id));
$sql->where('id', 'NOT_IN', DB::expr('SELECT user_id FROM encounters WHERE partner_id = '.$user->id.' AND predisposition = '.Encounter::Negative));
....
etc
...
Query which was builded with query builder like a Kohana query builder is more readable and understandable.
Are there any gem to solve this problem?
You need the squeel gem. It extends AR with blocks and makes very complicated queries with ease.
Just few features:
# not_in == cool! )
Product.where{id.not_in LineItem.select{product_id}}
# SELECT "products".* FROM "products" WHERE "products"."id" NOT IN
# (SELECT "line_items"."product_id" FROM "line_items" )
# outer joins on pure Ruby:
LineItem.joins{product.outer}
# LineItem Load (0.0ms) SELECT "line_items".* FROM "line_items"
# LEFT OUTER JOIN "products" ON "products"."id" = "line_items"."product_id"
# calcs, aliasing:
Product.select{[avg(price).as(middle)]}
# SELECT avg("products"."price") AS middle FROM "products"
# comparison
Product.where{id != 100500}
Product.where{price<10}
# logical OR
Product.where{(price<10) | (title.like '%rails%')}
# SELECT "products".* FROM "products" WHERE (("products"."price" < 10 OR
# "products"."title" LIKE '%rails%'))
# xxx_any feature (also available xxx_all)
Product.where{title.like_any %w[%ruby% %rails%]}
# SELECT "products".* FROM "products" WHERE (("products"."title" LIKE '%ruby%' OR
# "products"."title" LIKE '%rails%'))
Note the using blocks: {...} here aren't hashes. Also note the absence of symbols.
If you decide to pick it, read the section that starts with "This carries with it an important implication"
There's a ruby library that utilizes relational algebra. It is called ARel. If you are using Rails 3.x, then you already have.
ids = Partner.where(user_id: self.id).pluck(:partner_id) << self.id
users = User.where("id NOT IN #{ ids.join(',') }")
Here's the same query cast into rails AREL terms. It's not pretty yet -- it's a complicated query in general.
User.where("id = ? AND "
"id NOT IN (SELECT artner_id FROM encounters WHERE user_id = ?) AND " +
"id NOT IN (SELECT user_id FROM encounters WHERE partner_id = ? AND predisposition = ? ) AND " +
"cfg_sex = ? AND cfg_country = ? AND cfg_city = ?)",
self.id, self.id, self.id, Encounter::Negative,
self.sex, self.country, self.city).order(" rand() ").limit(1)
(I've not tested this, so it's possible there could be typo's in it.)
I'd recommend a couple things:
When you have complex where clauses they can be chained together and AREL will put them back together generally pretty well. This allows you to use scopes in your model classes and chain them together.
For example, you could do this:
class User < ActiveRecord::Base
def self.in_city_state_country(city, state, country)
where("cfg_sex = ? AND cfg_country = ? AND cfg_city = ?", city, state, country)
end
def self.is_of_sex(sex)
where("cfg_sex = ?", sex)
end
end
Then you could rewrite these portions of the query this way:
User.is_of_sex(user.sex).in_city_state_country(user.city, user.state, user.country)
and so on.
Breaking the queries down into smaller parts also makes it easier to test specific pieces of it with your rspecs. It results in more modular, maintainable code.
For more details, check out the Rails Guide - Active Record Query Interface
Related
I'm resolving all the SQL Injections in a system and I've found something that I don't know how to treat.
Can somebody help me?
Here is my method
def get_structure()
#build query
sql = %(
SELECT pc.id AS "product_id", pc.code AS "code", pc.description AS "description", pc.family AS "family",
p.code AS "father_code", p.description AS "father_description",
p.family AS "father_family"
FROM products pc
LEFT JOIN imported_structures imp ON pc.id = imp.product_id
LEFT JOIN products p ON imp.product_father_id = p.id
WHERE pc.enable = true AND p.enable = true
)
#verify if there is any filter
if !params[:code].blank?
sql = sql + " AND UPPER(pc.code) LIKE '%#{params[:code].upcase}%'"
end
#many other parameters like the one above
#execute query
str = ProductStructure.find_by_sql(sql)
end
Thank you!
You could use Arel which will escape for you, and is the underlying query builder for ActiveRecord/Rails. eg.
products = Arel::Table.new("products")
products2 = Arel::Table.new("products", as: 'p')
imported_structs = Arel::Table.new("imported_structures")
query = products.project(
products[:id].as('product_id'),
products[:code],
products[:description],
products[:family],
products2[:code].as('father_code'),
products2[:description].as('father_description'),
products2[:family].as('father_family')).
join(imported_structs,Arel::Nodes::OuterJoin).
on(imported_structs[:product_id].eq(products[:id])).
join(products2,Arel::Nodes::OuterJoin).
on(products2[:id].eq(imported_structs[:product_father_id])).
where(products[:enable].eq(true).and(products2[:enable].eq(true)))
if !params[:code].blank?
query.where(
Arel::Nodes::NamedFunction.new('UPPER',[products[:code]])
.matches("%#{params[:code].to_s.upcase}%")
)
end
SQL result: (with params[:code] = "' OR 1=1 --test")
SELECT
[products].[id] AS product_id,
[products].[code],
[products].[description],
[products].[family],
[p].[code] AS father_code,
[p].[description] AS father_description,
[p].[family] AS father_family
FROM
[products]
LEFT OUTER JOIN [imported_structures] ON [imported_structures].[product_id] = [products].[id]
LEFT OUTER JOIN [products] [p] ON [p].[id] = [imported_structures].[product_father_id]
WHERE
[products].[enable] = true AND
[p].[enable] = true AND
UPPER([products].[code]) LIKE N'%'' OR 1=1 --test%'
To use
ProductStructure.find_by_sql(query.to_sql)
I prefer Arel, when available, over String queries because:
it supports escaping
it leverages your existing connection adapter for sytnax (so it is portable if you change databases)
it is built in code so statement order does not matter
it is far more dynamic and maintainable
it is natively supported by ActiveRecord
you can build any complex query you can possibly imagine (including complex joins, CTEs, etc.)
it is still very readable
You need to turn that into a placeholder value (?) and add the data as a separate argument. find_by_sql can take an array:
def get_structure
#build query
sql = %(SELECT...)
query = [ sql ]
if !params[:code].blank?
sql << " AND UPPER(pc.code) LIKE ?"
query << "%#{params[:code].upcase}%"
end
str = ProductStructure.find_by_sql(query)
end
Note, use << on String in preference to += when you can as it avoids making a copy.
I can chain scopes like so:
scoped = User.where(sex: 'F')
scoped = scoped.where(color: 'blue')
ActiveRecord generates this SQL for the 1st line:
SELECT COUNT(*) FROM "users" WHERE "users"."sex" = 'F'
And this SQL for the 2nd line:
SELECT COUNT(*) FROM "users" WHERE "users"."sex" = 'F' AND "users"."color" = 'blue'
While keeping the initial query so it's built over 2 lines how can I change it from AND to OR so the end result looks like:
SELECT COUNT(*) FROM "users" WHERE "users"."sex" = 'F' OR "users"."color" = 'blue'
Without extensions, this is currently impossible with Rails unless you use raw SQL:
User.where('sex = ? OR color = ?', 'F', 'blue')
In Rails 5, these types of ORs will be supported out of the box.
If you use the squeel gem, you can implement this like so:
User.where(
{ sex: 'F' } |
{ color: 'blue' }
)
However, since it sounds like you need to build the query dynamically, I'd recommend doing something like this:
queries = [
"sex = 'F'",
"color = 'blue'"
]
User.where(queries.join(" OR "))
With this method, you can dynamically add as many queries as you need to the queries array. Obviously, you'll have to write the queries in raw SQL but this is the only option I'm aware of for accomplishing this with ActiveRecord.
UPDATE
To answer #Thomas question in the comments:
I'm wondering if there is a way to utilize join data with this. Like
user.sex instead of sex.
queries = [
"users.sex = 'F'",
"color = 'blue'"
]
Car.joins(:user).where(queries.join(" OR "))
I am learning ActiveRecord. Can I build this query?
#sales_by_product = ActiveRecord::Base.connection.execute("SELECT
it.name,
it.id,
it.seller_id,
pur.volume,
pur.sales
FROM items it
INNER JOIN (SELECT
item_id,
COUNT(*) AS volume,
SUM(price) AS sales,
workflow_state AS state
FROM purchases
WHERE workflow_state = 'payment_successful'
GROUP BY item_id,
workflow_state) pur
ON pur.item_id = it.id
WHERE it.seller_id = '" + current_user.id.to_s + "'")
I would like to use the AR api as much as possible but I have not yet gotten the above to work using just AR.
Thanks!
I don't think it is a good idea to use AR for this query. It seems fun at first, but becomes annoying. And, it will be difficult to change later.
You can create your own query builder:
def query_for current_user
<<-SQL
SELECT
it.name,
it.id,
it.seller_id,
pur.volume,
pur.sales
FROM items it
INNER JOIN (SELECT
item_id,
COUNT(*) AS volume,
SUM(price) AS sales,
workflow_state AS state
FROM purchases
WHERE workflow_state = 'payment_successful'
GROUP BY item_id,
workflow_state) pur
ON pur.item_id = it.id
WHERE it.seller_id = '" + current_user.id.to_s + "'")
SQL
end
#sales_by_product = ActiveRecord::Base.connection.execute( query_for( current_user ))
I am trying to find the best way to include a referenced model on what is essentially a compound key.
I have ChecklistItem (a list of things to do daily) and then ChecklistChecks (which ties the ChecklistItem together with a User for a particular day. These checklists can either be for all Stores (with a null store_id) or for a particular Store.
This pulls all of the ChecklistItems and their associated checks:
ChecklistItem.includes(:checklist_checks).where(store_id: [nil,#store.id], list_type: 'open')
The problem is that there would be checks from multiple days in there. What I need is to pull all of the ChecklistItems and any checks from a specific day.
I tried adding conditions like this:
ChecklistItem.includes(:checklist_checks).where(store_id: [nil,#store.id], list_type: 'open', checklist_checks: {store_id: #store.id, report_date: #today})
The problem is that will only pull ChecklistItems that have an associated ChecklistCheck.
It is generating SQL that is essentially:
SELECT
checklist_items.*,
checklist_checks.*
FROM
checklist_items
LEFT OUTER JOIN
checklist_checks
ON
checklist_checks.checklist_item_id = checklist_items.id
WHERE
checklist_items.list_type = 'open'
AND
checklist_checks.store_id = 1
AND
checklist_checks.report_date = '2015-05-03'
AND
(checklist_items.store_id = 1 OR checklist_items.store_id IS NULL)
I think the problem is that the conditions on checklist_checks is in the WHERE clause. If I could move them to the ON clause of the join, everything would work.
Is there a Rails way to end up with something like this?
SELECT
checklist_items.*,
checklist_checks.*
FROM
checklist_items
LEFT OUTER JOIN
checklist_checks
ON
checklist_checks.checklist_item_id = checklist_items.id
AND
checklist_checks.store_id = 1
AND
checklist_checks.report_date = '2015-05-03'
WHERE
checklist_items.list_type = 'open'
AND
(checklist_items.store_id = 1 OR checklist_items.store_id IS NULL)
UPDATE:
I found this: enter link description here
It suggests using find_by_sql and then passing the result array and model to be included to ActiveRecord::Associations::Preloader.new.preload
I tried that, and my find_by_sql pulls the right stuff, but the id column is nil in the resulting objects.
#store = Store.find(1)
#today = Date.today - 1.days
#open_items = ChecklistItem.find_by_sql(["SELECT checklist_items.*, checklist_checks.* FROM checklist_items LEFT OUTER JOIN checklist_checks ON checklist_checks.checklist_item_id = checklist_items.id AND checklist_checks.store_id = ? AND checklist_checks.report_date = ? WHERE checklist_items.list_type='open' AND (checklist_items.store_id=? OR checklist_items.store_ID IS NULL)", #store.id, #today, #store_id])
ActiveRecord::Associations::Preloader.new.preload(#open_items, :checklist_checks)
> #open_items.first.name
=> "Turn on the lights"
> #open_items.first.id
=> nil
A solution using Arel to generate a custom join clause:
class ChecklistItem < ActiveRecord::Base
has_many :checklist_checks
# ...
def self.superjoin(date, store_id)
# build the ON clause for the join
on = Arel::Nodes::On.new(
Arel::Nodes::Equality.new(ChecklistChecks.arel_table[:checklist_item_id], ChecklistItem.arel_table[:id]).\
and(ChecklistItem.arel_table[:store_id].eq(1)).\
and(ChecklistChecks.arel_table[:report_date].eq(date))
)
joins(Arel::Nodes::OuterJoin.new(ChecklistChecks.arel_table, on))
.where(store_id: [nil, store_id], list_type: 'open' )
end
end
I bundled it up into a model method to make it easier to test in the rails console.
irb(main):117:0> ChecklistItem.superjoin(1,2)
ChecklistItem Load (0.5ms) SELECT "checklist_items".* FROM "checklist_items" LEFT OUTER JOIN "checklist_checks" ON "checklist_checks"."checklist_item_id" = "checklist_items"."id" AND "checklist_items"."store_id" = 1 AND "checklist_checks"."report_date" = 1 WHERE (("checklist_items"."store_id" = 2 OR "checklist_items"."store_id" IS NULL)) AND "checklist_items"."list_type" = 'open'
=> #<ActiveRecord::Relation []>
What are the Rails methods that are vulnerable to SQL injection, and in what form?
For example, I know that where with a string argument is vulnerable:
Model.where("name = #{params[:name]}") # unsafe
But a parameterized string or hash is not:
Model.where("name = ?", params[:name]) # safe
Model.where(name: params[:name]) # safe
I'm mostly wondering about where, order, limit and joins, but would like to know about any other methods that might be attack vectors.
In Rails, where, order, limit and joins all have vulnerable forms. However, Rails limits the number of SQL operations performed to 1 so vulnerability is limited. An attacker cannot end a statement and execute a new arbitrary one.
Where
Where has one vulnerable form: string.
# string, unsafe
Model.where("name = '#{params[:name]}'")
# hash/parameterized string/array, safe
Model.where(name: params[:name])
Model.where("name = ?", params[:name])
Model.where(["name = ?", params[:name]])
Order
String form is vulnerable:
# unsafe
params[:order] = "1; --\n drop table users;\n --"
Model.order("#{params[:order]} ASC")
# safe
order_clause = sanitize(params[:order])
Model.order(order_clause)
Limit
Limit has no vulnerable forms, since Rails casts input to Integer beforehand.
Model.limit("1; -- \n SELECT password from users; -- ")
=> ArgumentError: invalid value for Integer(): "1; -- \n SELECT password from users; -- "
Joins
String form is vulnerable:
params[:table] = "WHERE false <> $1; --"
Model.where(:user_id => 1).joins(params[:table])
=> SELECT "models".* FROM "models" WHERE false <> $1 -- WHERE "models"."user_id" = $1 [["user_id", 1]]
Much more comprehensive information can be found at rails-sqli.org.
Generally: If you let the user input and save any text into your database, without escaping code, it could harm your system. Especially if these texts may contain tags/code snippets.