I am using Ruby on Rails and I have a location in my database with the name:
1A J#ck$on & S0n's #{10}
I am receiving this name via a webhook and then searching my database with it however it does not find the location name ( it is instead searching for the interpolated name:
1A J#ck$on & S0n's 10
How can I receive this string via a webhook like this:
#location = inbound_webhook_request['location']
And then put it in a pg "like" query as shown below:
Location.where("name ~* ?", #location['name'])
Without it being interpolated along the way?
The string is not being interpolated. I'm not sure what led you to that assumption. However:
Location.where("name ~* ?", #location['name'])
This is not a LIKE operation, it's a POSIX regexp (case insensitive) operation.
Assuming you actually did want to perform a LIKE operation, not a regular expression search, you can do this:
Location.where("name LIKE ?", "%#{#location['name']}%")
or, using the shorthand syntax from the above linked documentation:
Location.where("name ~~ ?", "%#{#location['name']}%")
For a case-insensitive LIKE, you can use ILIKE or ~~*.
If the user input needs to be further sanitised, see this answer.
Related
It used in this part of code:
some_list.where("'#{#new_params[:email]}' ~~* name").any?
I tried to use google search, but i found only description of ~ rxp and this same unclear for me (especially in example). I had no experience with Ruby earlier, sorry if question is stupid.
~~* doen't actually have anything to with Ruby. Its the Postgres specific ILIKE operator for pattern matching.
This code is also a textbook example of a SQL injection vulnerability. The user input should be parameterized.
some_list.where("? ~~* name", #new_params[:email]).any?
This code is also pretty bizarre in that it has a Yoda condition. Normally you would write it as:
some_list.where("name ~~* ?", #new_params[:email]).any?
That has nothing do do with ruby. You construct a sql query and pass it into the #where method therefor it is a PostgreSQL operator.
The operator ~~ is equivalent to LIKE, and ~~* corresponds to ILIKE. There are also !~~ and !~~* operators that represent NOT LIKE and NOT ILIKE, respectively. All of these operators are PostgreSQL-specific.
That's what you are passing into it:
"foo#bar.com ILIKE name"
Any string in a where clause will be put into the SQL query which is then handed off to the database. So the ~~* syntax is not ruby, but SQL. My guess would be, that you are using Postgres as a DB, because:
The operator ~~ is equivalent to LIKE, and ~~* corresponds to ILIKE. There are also !~~ and !~~* operators that represent NOT LIKE and NOT ILIKE, respectively. All of these operators are PostgreSQL-specific.
Taken from: https://www.postgresql.org/docs/current/functions-matching.html#FUNCTIONS-LIKE
NOTE: The following uses a PG database.
I have setup wherein model Foo belongs to model Bar. In trying to return all entries Foo in which bar.name ends with "abc" (e.g. "def abc" but not "def abc-2"), I wrote the query:
Foo.joins(:bar).where('bar.name ~* ?', "abc\z")
This query returns no results. However, this query:
Foo.joins(:bar).where('bar.name ~* ?', "abc$")
returns the results I expected.
Digging a little deeper, I tested the same queries, but omitting the "abc" (i.e. querying solely on the regex anchors). For my current database, the "\z" query returns 94 results, whereas the "$" query returns 2,016 results.
I'd like to understand (a) if I've done something fundamentally wrong with my query, and (b) what's going on under the hood that leads to this behavior. It seems like the database is being queried like a bunch of chunked multi-line strings rather than each entry field being considered its own string, but I don't know why, or even if that's the case.
I am having trouble with a DB query in a Rails app. I want to store various search terms (say 100 of them) and then evaluate against a value dynamically. All the examples of SIMILAR TO or ~ (regex) in Postgres I can find use a fixed string within the query, while I want to look the query up from a row.
Example:
Table: Post
column term varchar(256)
(plus regular id, Rails stuff etc)
input = "Foo bar"
Post.where("term ~* ?", input)
So term is VARCHAR column name containing the data of at least one row with the value:
^foo*$
Unless I put an exact match (e.g. "Foo bar" in term) this never returns a result.
I would also like to ideally use expressions like
(^foo.*$|^second.*$)
i.e. multiple search terms as well, so it would match with 'Foo Bar' or 'Search Example'.
I think this is to do with Ruby or ActiveRecord stripping down something? Or I'm on the wrong track and can't use regex or SIMILAR TO with row data values like this?
Alternative suggestions on how to do this also appreciated.
The Postgres regular expression match operators have the regex on the right and the string on the left. See the examples: https://www.postgresql.org/docs/9.3/static/functions-matching.html#FUNCTIONS-POSIX-TABLE
But in your query you're treating term as the string and the 'Foo bar' as the regex (you've swapped them). That's why the only term that matches is the exact match. Try:
Post.where("? ~* term", input)
So let's say I search for 'Blerg'. And I have a item with the name SomethingblergSomething.
If I do an ILIKE search in postgres (and rails) like this:
where("name ILIKE ?", "%#{ 'Blerg' }%")
It will return the result 'SomethingBlergSomething' because it contains Blerg.
Is there a way to make the faster tsvector do a similar style of searching inside a word:
where("(to_tsvector('english', name) ## to_tsquery(?))", ('Blerg' + ':*'))
The above query will not return 'SomethingBlergSomething'.
So how do I make tsvector act like ILIKE when searching inside words.
Are you aware of trigram search, provided by the additional module pg_trgm? That seems more appropriate for your use case than text search.
With a trigram index in place (GIN or GiST) you can use your original ILIKE predicate and get index support for it. You need Postgres 9.1+ for that.
Details:
PostgreSQL LIKE query performance variations
Pattern matching with LIKE, SIMILAR TO or regular expressions in PostgreSQL
I created this regex in normal Regex
/(first|last)\s(last|first)/i
It matches the first three of
first last
Last first
First Last
First name
I am trying to get all the records where the full_name matches with the regex I wrote. I'm using PostgreSQL
Person.where("full_name ILIKE ?", "%(first|last)%(last|first)%")
This is my attempt. I also tried SIMILAR TO and ~ with no luck
Your LIKE query:
full_name ilike '%(first|last)%(last|first)%'
won't work because LIKE doesn't understand regex grouping ((...)) or alternation (|), LIKE only understands _ for a single character (like . in a regex) and % for any sequence of zero or more characters (like .* in a regex).
If you hand that pattern to SIMILAR TO then you'll find 'first last' but none of the others due to case problems; however, this:
lower(full_name) similar to '%(first|last)%(last|first)%'
will take care of the case problems and find the same ones as your regex.
If you want to use a regex (which you probably do because LIKE is very limited and cumbersome and SIMILAR TO is, well, a strange product of the fevered minds of some SQL standards subcommittee) then you'll want to use the case-insensitive matching operator and your original regex:
full_name ~* '(first|last)\s+(last|first)'
That translates to this bit of AR:
Person.where('full_name ~* :pat', :pat => '(first|last)\s+(last|first)')
# or this
Person.where('full_name ~* ?', '(first|last)\s+(last|first)')
There's a subtle change in my code that you need to take note of: I'm using single quotes for my Ruby strings, you're using double quotes. Backslashes mean more in double quoted strings than they do in single quoted strings so '\s' and "\s" are different things. Toss in a couple to_sql calls and you might see something interesting:
> puts Person.where('full_name ~* :pat', :pat => 'a\s+b').to_sql
SELECT "people".* FROM "people" WHERE (full_name ~* 'a\s+b')
> puts Person.where('full_name ~* :pat', :pat => "a\s+b").to_sql
SELECT "people".* FROM "people" WHERE (full_name ~* 'a +b')
That difference probably isn't causing you any problems but you need to be very careful with your strings when everyone wants to use the same escape character. Personally, I use single quoted strings unless I specifically need the extra escapes and string interpolation functionality of double quoted strings.
Some demos: http://sqlfiddle.com/#!15/99a2c/6