I am working on a legacy Rails project that relies on Ruby version 1.8
I have a string looks like this:
my_str = "a,b,c"
I would like to convert it to
value_list = "('a','b','c')"
so that I can directly use it in my SQL statement like:
"SELECT * from my_table WHERE value IN #{value_list}"
I tried:
my_str.split(",")
but it returns "abc" :(
How to convert it to what I need?
To split the string you can just do
my_str.split(",")
=> ["a", "b", "c"]
The easiest way to use that in a query, is using where as follows:
Post.where(value: my_str.split(","))
This will just work as expected. But, I understand you want to be able to build the SQL-string yourself, so then you need to do something like
quoted_values_str = my_str.split(",").map{|x| "'#{x}'"}.join(",")
=> "'a','b','c'"
sql = ""SELECT * from my_table WHERE value IN (#{quoted_values_str})"
Note that this is a naive approach: normally you should also escape quotes if they should be contained inside your strings, and makes you vulnerable for sql injection. Using where will handle all those edge cases correctly for you.
Under no circumstances should you reinvent the wheel for this. Rails has built-in methods for constructing SQL strings, and you should use them. In this case, you want sanitize_sql_for_assignment (aliased to sanitize_sql):
my_str = "a,b,c"
conditions = sanitize_sql(["value IN (?)", my_str.split(",")])
# => value IN ('a','b','c')
query = "SELECT * from my_table WHERE #{conditions}"
This will give you the result you want while also protecting you from SQL injection attacks (and other errors related to badly formed SQL).
The correct usage may depend what version of Rails you're using, but this method exists as far back as Rails 2.0 so it will definitely work even with a legacy app; just consult the docs for the version of Rails you're using.
value_list = "('#{my_str.split(",").join("','")}')"
But this is a very bad way to query. You better use:
Model.where(value: my_str.split(","))
The string can be manipulated directly; there is no need to convert it to an array, modify the array then join the elements.
str = "a,b,c"
"(%s)" % str.gsub(/([^,]+)/, "'\\1'")
#=> "('a','b','c')"
The regular expression reads, "match one or more characters other than commas and save to capture group 1. \\1 retrieves the contents of capture group 1 in the formation of gsub's replacement string.
couple of use cases:
def full_name
[last_name, first_name].join(' ')
end
or
def address_line
[address[:country], address[:city], address[:street], address[:zip]].join(', ')
end
Related
When Ecto queries get more complex and require clauses like CASE...WHEN...ELSE...END, we tend to depend on Ecto's fragment to solve it.
e.g. query = from t in <Model>, select: fragment("SUM(CASE WHEN status = ? THEN 1 ELSE 0 END)", 2)
In fact the most popular Stack Overflow post about this topic suggests to create a macro like this:
defmacro case_when(condition, do: then_expr, else: else_expr) do
quote do
fragment(
"CASE WHEN ? THEN ? ELSE ? END",
unquote(condition),
unquote(then_expr),
unquote(else_expr)
)
end
end
so you can use it this way in your Ecto queries:
query = from t in <Model>,
select: case_when t.status == 2
do 1
else 0
end
at the same time, in another post, I found this:
(Ecto.Query.CompileError) to prevent SQL injection attacks, fragment(...) does not allow strings to be interpolated as the first argument via the `^` operator, got: `"exists (\n SELECT 1\n FROM #{other_table} o\n WHERE o.column_name = ?)"
Well, it seems Ecto's team figured out people are using fragment to solve complex queries, but they don't realize it can lead to SQL injection, so they don't allow string interpolation there as a way to protect developers.
Then comes another guy who says "don't worry, use macros."
I'm not an elixir expert, but that seems like a workaround to DO USE string interpolation, escaping the fragment protection.
Is there a way to use fragment and be sure the query was parameterized?
SQL injection, here, would result of string interpolation usage with an external data. Imagine where: fragment("column = '#{value}'") (instead of the correct where: fragment("column = ?", value)), if value comes from your params (usual name of the second argument of a Phoenix action which is the parameters extracted from the HTTP request), yes, this could result in a SQL injection.
But, the problem with prepared statement, is that you can't substitute a paremeter (the ? in fragment/1 string) by some dynamic SQL part (for example, a thing as simple as an operator) so, you don't really have the choice. Let's say you would like to write fragment("column #{operator} ?", value) because operator would be dynamic and depends on conditions, as long as operator didn't come from the user (harcoded somewhere in your code), it would be safe.
I don't know if you are familiar with PHP (PDO in the following examples), but this is exactly the same with $bdd->query("... WHERE column = '{$_POST['value']}'") (inject a value by string interpolation) in opposite to $stmt = $bdd->prepare('... WHERE column = ?') then $stmt->execute([$_POST['value']]); (a correct prepared statement). But, if we come back to my previous story of dynamic operator, as stated earlier, you can't dynamically bind some random SQL fragment, the DBMS would interpret "WHERE column ? ?" with > as operator and 'foo' as value like (for the idea) WHERE column '>' 'foo' which is not syntactically correct. So, the easiest way to turn this operator dynamic is to write "WHERE column {$operator} ?" (inject it, but only it, by string interpolation or concatenation). If this variable $operator is defined by your own code (eg: $operator = some_condition ? '>' : '=';), it's fine but, in the opposite, if it involves some superglobal variable which comes from the client like $_POST or $_GET, this creates a security hole (SQL injection).
TL;DR
Then comes another guy who says "don't worry, use macros."
The answer of Aleksei Matiushkin, in the mentionned post, is just a workaround to the disabled/forbidden string interpolation by fragment/1 to dynamically inject a known operator. If you reuse this trick (and can't really do otherwise), as long as you don't blindly "inject" any random value coming from the user, you'll be fine.
UPDATE:
It seems, after all, that fragment/1 (which I didn't inspect the source) doesn't imply a prepared statement (the ? are not placeholder of a true prepared statement). I tried some simple and stupid enough query like the following:
from(
Customer,
where: fragment("lastname ? ?", "LIKE", "%")
)
|> Repo.all()
At least with PostgreSQL/postgrex, the generated query in console appears to be in fact:
SELECT ... FROM "customers" AS c0 WHERE (lastname 'LIKE' '%') []
Note the [] (empty list) at the end for the parameters (and absence of $1 in the query) so it seems to act like the emulation of prepared statement in PHP/PDO meaning Ecto (or postgrex?) realizes proper escaping and injection of values directly in the query but, still, as said above LIKE became a string (see the ' surrounding it), not an operator so the query fails with a syntax error.
I'm attempting to write a Ruby method which accepts an array of strings (for example, ["EG", "K", "C"], and returns all records from a database table where the icao_code field starts with any of those strings (for example, KORD, EGLL, and CYVR would all match). The length of the array will vary, and it will be input by a user, so it needs to be sanitized.
If I were only searching for a single string, I could do something like Airport.where("icao_code LIKE ?", "#{icao_start}%"). However, since I need to search against an arbitrary number of strings, I can't use that syntax.
Right now, I've got it working as follows:
def in_region(icao_starts)
where_clause = icao_starts.map{|i| "icao_code LIKE '#{i}%'"}.join(" OR ")
return Airport.where(where_clause)
end
However, I'm a bit worried using a setup like this with untrusted user input, since I suspect it would be vulnerable to SQL injection.
Is there a better way to get the same result in a more secure way?
You could consider something like this:
def in_region(icao_starts)
where_clause = "icao_code LIKE '#?%' OR " * icao_starts.length
return Airport.where(where_clause.sub(/\ OR\ $/, ''), *icao_starts)
end
This will build up a (potentially very long?) string with ? placeholders. The *icao_starts will expand that array into arguments to the where clause, so each ? will end up getting safely replaced. The sub(/\ OR\ $/, '') simply trims off the final OR (you could append 1=0 instead if you wanted).
If I were you I would also perform a .uniq on icao_starts before you do anything, truncate the array at some sensible upper length limit, and also have a whitelist of permitted values (oh, forget that, I thought users were searching by airport code). That should be pretty much infallible.
You are right about not interpolating user input into your SQL query. This is dangerous and makes your code vulnerable for SQLI attacks.
def in_region(icao_starts)
conditions = icao_starts.map { "icao_code LIKE ?"}
Airport.where(conditions.join(' OR '), *icao_starts.map { |name| "#{name}%"})
end
It is pretty similar than the solution of bogardpd but does not use a Regexp to get rid of the last " OR"
I have array of strings:
a = ['*#foo.com', '*#bar.com', '*#baz.com']
I would like to query my model so I will get all the records where email isn't in any of above domains.
I could do:
Model.where.not(email: a)
If the list would be a list of strings but the list is more of a regexp.
It depends on your database adapter. You will probably be able to use raw SQL to write this type of query. For example in postgres you could do:
Model.where("email NOT SIMILAR TO '%#foo.com'")
I'm not saying thats exactly how you should be doing it but it's worth looking up your database's query language and see if anything matches your needs.
In your example you would have to join together your matchers as a single string and interpolate it into the query.
a = ['%#foo.com', '%#bar.com', '%#baz.com']
Model.where("email NOT SIMILAR TO ?", a.join("|"))
Use this code:
a = ['%#foo.com', '%#bar.com', '%#baz.com']
Model.where.not("email like ?",a.join("|"))
Replace * to % in array.
I need to store a regular expression related to other fields in a database table with ActiveRecord.
I found the to_s method in Regexp class which states
Returns a string containing the regular expression and its options
(using the (?opts:source) notation. This string can be fed back in to
Regexp::new to a regular expression with the same semantics as the
original. (However, Regexp#== may not return true when comparing the
two, as the source of the regular expression itself may differ, as the
example shows). Regexp#inspect produces a generally more readable
version of rxp.
So it seems a working solution, but it will store the exp with an unusual syntax and in order to get the string to store I need to build it manually with /my-exp/.to_s. Also I may not be able to edit to regexp directly. For instance a simple regexp produces:
/foo/i.to_s # => "(?i-mx:foo)"
The other option is to eval the field content so I might store the plain expression in the db column and then doing an eval(record.pattern) to get the actual regexp. This is working and since I'm the only one who will be responsible to manage the regexp records there should be no issues in doing that, except application bugs ;-)
Do I have other options? I'd prefer to not doing eval on db fields but on the other side I don't want to work with a syntax which I don't know.
use serialize to store your regex 'as-is'
class Foo < ActiveRecord::Base
serialize :my_regex, Regexp
end
see the API doc to learn more about this.
Not sure I understand your constraints exactly.
If you store a string in db, you could make a Regexp from it:
a = 'foo'
=> "foo"
/#{a}/
=> /foo/
Regexp.new('dog', Regexp::EXTENDED + Regexp::MULTILINE + Regexp::IGNORECASE)
=> /dog/mix
There are other constructors, see doc.
The very best solution to not use eval'd code is to store the regexp part in a string column and flags in a separate integer column. In this way the regexp can be built with:
record = Record.new pattern: 'foo', flags: Regexp::IGNORECASE
Regexp.new record.pattern, record.flags # => /foo/i
You can use #{} within regular expressions to insert variables, so you could insert a carefully cleaned regexp by storing "foo" in the db under record.pattern as a string, and then evaluating it with:
/#{record.pattern}/
So, in the db, you would store:
"pattern"
in your code, you could do:
if record.other_field =~ /#{record.pattern}/
# do something
end
This compiles the regexp from a dynamic string in the db that you can change, and allows you to use it in code. I wouldn't recommend it for security reasons though, see below:
Obviously this could be dangerous, as the regex can contain ruby code, so this is simpler, but in terms of danger, it is similar to eval:
a = "foo"
puts a
=> foo
b = "#{a = 'bar'}"
a =~ /#{b}/
puts a
=> bar
You might be better to consider whether for security it is worth decomposing your regex tests into something you can map to methods which you write in the code, so you could store keys in the db for constraints, something like:
'alpha,numeric' etc.
And then have hard-coded tests which you run depending on the keys stored. Perhaps look at rails validations for hints here, although those are stored in code, it's probably the best approach (generalise your requirements, and keep the code out of the db). Even if you don't think you need security now, you might want it later, or forget about this and grant access to someone malicious.
I want to create a search function on my website, and I don't want to use a plugin for this thing, because it's very simple, but I can't solve this problem:
I give the keyword to the model which creates a query, but I couldn't figure out how to put joker characters in this query.
I'm using Propel
Dennis
The filterByXXX() query functions will use LIKE when your query contains wildcards:
$books = BookQuery::create()
->filterByTitle('War%')
->find();
// example Query generated for a MySQL database
$query = 'SELECT book.* from `book` WHERE book.TITLE LIKE :p1'; // :p1 => 'War%'
Remember, the wildcards you can use in SQL are _ for exactly one and % for zero or more characters. So not ? or *.