I am working on reports page for a ZF2 project. Now I need to generate dynamic query depends on filters (which can be '=', '>', '>=', '<', '<=', 'IN' ). I am using DB select closure for generate where statement. But I am afraid if it could be a bottleneck in coming days ( by performance or by limitations ).
Can any body suggest if my approach is Ok or need to generate string where statements like
->where('A > 12 AND B < 12 AND C IN (1,2,3)')
instead of
->where(function(Where $where){
$where->equalTo('A', 10)->equalTo('B', 12)->IN('C', array(1,2,3));
});
Or any better idea ?
I got a better solution. Instead of using closure I using direct Where Object something like that
$where = new \Zend\Db\Sql\Where();
$where->equalTo('A', 10)->equalTo('B', 12)->IN('C', array(1,2,3));
$sql->select()->where($where);
It is more dynamic as $where can be dynamically updated by other values. Still if anybody have other ideas please share.
Related
When Ecto queries get more complex and require clauses like CASE...WHEN...ELSE...END, we tend to depend on Ecto's fragment to solve it.
e.g. query = from t in <Model>, select: fragment("SUM(CASE WHEN status = ? THEN 1 ELSE 0 END)", 2)
In fact the most popular Stack Overflow post about this topic suggests to create a macro like this:
defmacro case_when(condition, do: then_expr, else: else_expr) do
quote do
fragment(
"CASE WHEN ? THEN ? ELSE ? END",
unquote(condition),
unquote(then_expr),
unquote(else_expr)
)
end
end
so you can use it this way in your Ecto queries:
query = from t in <Model>,
select: case_when t.status == 2
do 1
else 0
end
at the same time, in another post, I found this:
(Ecto.Query.CompileError) to prevent SQL injection attacks, fragment(...) does not allow strings to be interpolated as the first argument via the `^` operator, got: `"exists (\n SELECT 1\n FROM #{other_table} o\n WHERE o.column_name = ?)"
Well, it seems Ecto's team figured out people are using fragment to solve complex queries, but they don't realize it can lead to SQL injection, so they don't allow string interpolation there as a way to protect developers.
Then comes another guy who says "don't worry, use macros."
I'm not an elixir expert, but that seems like a workaround to DO USE string interpolation, escaping the fragment protection.
Is there a way to use fragment and be sure the query was parameterized?
SQL injection, here, would result of string interpolation usage with an external data. Imagine where: fragment("column = '#{value}'") (instead of the correct where: fragment("column = ?", value)), if value comes from your params (usual name of the second argument of a Phoenix action which is the parameters extracted from the HTTP request), yes, this could result in a SQL injection.
But, the problem with prepared statement, is that you can't substitute a paremeter (the ? in fragment/1 string) by some dynamic SQL part (for example, a thing as simple as an operator) so, you don't really have the choice. Let's say you would like to write fragment("column #{operator} ?", value) because operator would be dynamic and depends on conditions, as long as operator didn't come from the user (harcoded somewhere in your code), it would be safe.
I don't know if you are familiar with PHP (PDO in the following examples), but this is exactly the same with $bdd->query("... WHERE column = '{$_POST['value']}'") (inject a value by string interpolation) in opposite to $stmt = $bdd->prepare('... WHERE column = ?') then $stmt->execute([$_POST['value']]); (a correct prepared statement). But, if we come back to my previous story of dynamic operator, as stated earlier, you can't dynamically bind some random SQL fragment, the DBMS would interpret "WHERE column ? ?" with > as operator and 'foo' as value like (for the idea) WHERE column '>' 'foo' which is not syntactically correct. So, the easiest way to turn this operator dynamic is to write "WHERE column {$operator} ?" (inject it, but only it, by string interpolation or concatenation). If this variable $operator is defined by your own code (eg: $operator = some_condition ? '>' : '=';), it's fine but, in the opposite, if it involves some superglobal variable which comes from the client like $_POST or $_GET, this creates a security hole (SQL injection).
TL;DR
Then comes another guy who says "don't worry, use macros."
The answer of Aleksei Matiushkin, in the mentionned post, is just a workaround to the disabled/forbidden string interpolation by fragment/1 to dynamically inject a known operator. If you reuse this trick (and can't really do otherwise), as long as you don't blindly "inject" any random value coming from the user, you'll be fine.
UPDATE:
It seems, after all, that fragment/1 (which I didn't inspect the source) doesn't imply a prepared statement (the ? are not placeholder of a true prepared statement). I tried some simple and stupid enough query like the following:
from(
Customer,
where: fragment("lastname ? ?", "LIKE", "%")
)
|> Repo.all()
At least with PostgreSQL/postgrex, the generated query in console appears to be in fact:
SELECT ... FROM "customers" AS c0 WHERE (lastname 'LIKE' '%') []
Note the [] (empty list) at the end for the parameters (and absence of $1 in the query) so it seems to act like the emulation of prepared statement in PHP/PDO meaning Ecto (or postgrex?) realizes proper escaping and injection of values directly in the query but, still, as said above LIKE became a string (see the ' surrounding it), not an operator so the query fails with a syntax error.
I have a plain SQL query written by a trusted administrator that is to be run in a Rails (4.2) app. I am sanitizing it with ActiveRecord::Base.send(:sanitize_sql, ...) to allow user inputs to act as conditions, using the ? character for bind variables. The code has to allow arbitrary SQL, so I'm not interested in the arguments about why this is not the Rails way, etc.
The problem is that I can not include ? in a result field in the SQL without the underlying replace_bind_variables method replacing an intended literal ? in the result.
A simple query for example would be:
select 'http://www.google.com?q=' || res from some_table where a = ?;
To sanitize:
ActiveRecord::Base.send(:sanitize_sql, [sql, 'not me'], :some_table)
The sanitization fails because the ? in the URL gets replaced with the data intended for the condition, leading to the exception:
ActiveRecord::PreparedStatementInvalid: wrong number of bind variables (1 for 2)
The question is, does sanitize_sql or some variant allow literal ? characters to be included in a query so that they are not replaced? Is there some way of escaping them?
In the end I read through the ActiveRecord source and couldn't identify a way to handle this situation without a lot of code changes. There doesn't appear to be a way to escape the ? characters.
To resolve it for this one query I ended up using the SQL chr() function to generate a character that would pass the santization step untouched:
select 'http://www.google.com' || chr(63) || 'q=' || res from some_table where a = ?;
ASCII character 63 is ?.
Although not a perfect solution, I could at least get this one SQL query into the system without having to make massive code changes.
I want to avoid using injection of parms in the query statement. Therefore we used the following instructions from the NEO4J .NET client class:
var queryClassRelationshipsNodes = client.Cypher
.Start("a", (NodeReference)sourceReference.Id)
.Match("a-[Rel: ***{relationshipType***} ]->foundClass")
.Where("Rel.RelationStartNode =" + "\'" + relationshipStart + "\'")
.AndWhere("Rel.RelationDomainNode =" + "\'" + relationshipDomain + "\'")
.AndWhere("Rel.RelationClassNode =" + "\'" + relationshipClass + "\'")
.WithParam("relationshipType", relationshipType)
.Return<Node<Dictionary<string, string>>>("foundClass")
.Results;
However this code does not work once executed by the server. For some reason the PARM: relationshipType is not connected with the variable which we put in between {}.
Can someone please help us debug the problem with this code? We would prefer to use WithParms rather than injecting variables inside the statement.
Thanks a lot!
Can someone please help us debug the problem with this code?
There's a section on https://bitbucket.org/Readify/neo4jclient/wiki/cypher titled "Debugging" which describes how to do this.
As for your core problem though, your approach is hitting a Cypher restriction. Parameters are for parts of the query that aren't compiled into the query plan. The match clause is however.
From the Neo4j documentation:
Parameters can be used for literals and expressions in the WHERE clause, for the index key and index value in the START clause, index queries, and finally for node/relationship ids. Parameters can not be used as for property names, since property notation is part of query structure that is compiled into a query plan.
You could do something like:
.Match("a-[Rel:]->foundClass")
.Where("type(Rel) = {relationshipType}")
.WithParam("relationshipType", relationshipType)
(Disclaimer: I've just typed that here. I haven't tested it at all.)
That will likely be slower though, because you need to retrieve all relationships, then test their types. You should test this. There's a reason why the match clause is compiled into the query plan.
Given I got User.attachments and Attachment.visits as an integer with the number count.
How can I easily count all the visits of all images of that user?
Use ActiveRecord::Base#sum:
user.attachments.sum(:visits)
This should generate an efficient SQL query like this:
SELECT SUM(attachments.visits) FROM attachments WHERE attachments.user_id = ID
user.attachments.map{|a| a.visits}.sum
There's also inject:
user.attachments.inject(0) { |sum, a| sum + a.visits }
People generally (and quite rightly) hate inject, but since the two other main ways of achieving this have been mentioned, I thought I may as well throw it out there. :)
The following works with Plain Old Ruby Objects, and I suspect the following is marginally faster than using count += a.visits, plus it has an emoticon in it:
user.attachments.map(&:visits).inject(:+)
I want to create a search function on my website, and I don't want to use a plugin for this thing, because it's very simple, but I can't solve this problem:
I give the keyword to the model which creates a query, but I couldn't figure out how to put joker characters in this query.
I'm using Propel
Dennis
The filterByXXX() query functions will use LIKE when your query contains wildcards:
$books = BookQuery::create()
->filterByTitle('War%')
->find();
// example Query generated for a MySQL database
$query = 'SELECT book.* from `book` WHERE book.TITLE LIKE :p1'; // :p1 => 'War%'
Remember, the wildcards you can use in SQL are _ for exactly one and % for zero or more characters. So not ? or *.