How can I modify a where/like condition on a search query in Rails:
find(:all, :conditions => ["lower(name) LIKE ?", "%#{search.downcase}%"])
so that the results are matched irrespective of accents? (eg métro = metro). Because I'm using utf8, I can't use "to_ascii". Production is running on Heroku.
Proper solution
Since PostgreSQL 9.1 you can just:
CREATE EXTENSION unaccent;
Provides a function unaccent(), doing what you need (except for lower(), just use that additionally if needed). Read the manual about this extension.
More about unaccent and indexes:
Does PostgreSQL support "accent insensitive" collations?
Poor man's solution
If you can't install unacccent, but are able to create a function. I compiled the list starting here and added to it over time. It is comprehensive, but hardly complete:
CREATE OR REPLACE FUNCTION lower_unaccent(text)
RETURNS text
LANGUAGE sql IMMUTABLE STRICT AS
$func$
SELECT lower(translate($1
, '¹²³áàâãäåāăąÀÁÂÃÄÅĀĂĄÆćčç©ĆČÇĐÐèéêёëēĕėęěÈÊËЁĒĔĖĘĚ€ğĞıìíîïìĩīĭÌÍÎÏЇÌĨĪĬłŁńňñŃŇÑòóôõöōŏőøÒÓÔÕÖŌŎŐØŒř®ŘšşșߊŞȘùúûüũūŭůÙÚÛÜŨŪŬŮýÿÝŸžżźŽŻŹ'
, '123aaaaaaaaaaaaaaaaaaacccccccddeeeeeeeeeeeeeeeeeeeeggiiiiiiiiiiiiiiiiiillnnnnnnooooooooooooooooooorrrsssssssuuuuuuuuuuuuuuuuyyyyzzzzzz'
));
$func$;
Your query should work like that:
find(:all, :conditions => ["lower_unaccent(name) LIKE ?", "%#{search.downcase}%"])
For left-anchored searches, you can use an index on the function for very fast results:
CREATE INDEX tbl_name_lower_unaccent_idx
ON fest (lower_unaccent(name) text_pattern_ops);
For queries like:
SELECT * FROM tbl WHERE (lower_unaccent(name)) LIKE 'bob%';
Or use COLLATE "C". See:
PostgreSQL LIKE query performance variations
Is there a difference between text_pattern_ops and COLLATE "C"?
For those like me who are having trouble on add the unaccent extension for PostgreSQL and get it working with the Rails application, here is the migration you need to create:
class AddUnaccentExtension < ActiveRecord::Migration
def up
execute "create extension unaccent"
end
def down
execute "drop extension unaccent"
end
end
And, of course, after rake db:migrate you will be able to use the unaccent function in your queries: unaccent(column) similar to ... or unaccent(lower(column)) ...
First of all, you install postgresql-contrib. Then you connect to your DB and execute:
CREATE EXTENSION unaccent;
to enable the extension for your DB.
Depending on your language, you might need to create a new rule file (in my case greek.rules, located in /usr/share/postgresql/9.1/tsearch_data), or just append to the existing unaccent.rules (quite straightforward).
In case you create your own .rules file, you need to make it default:
ALTER TEXT SEARCH DICTIONARY unaccent (RULES='greek');
This change is persistent, so you need not redo it.
The next step would be to add a method to a model to make use of this function.
One simple solution would be defining a function in the model. For instance:
class Model < ActiveRecord::Base
[...]
def self.unaccent(column,value)
a=self.where('unaccent(?) LIKE ?', column, "%value%")
a
end
[...]
end
Then, I can simply invoke:
Model.unaccent("name","text")
Invoking the same command without the model definition would be as plain as:
Model.where('unaccent(name) LIKE ?', "%text%"
Note: The above example has been tested and works for postgres9.1, Rails 4.0, Ruby 2.0.
UPDATE INFO
Fixed potential SQLi backdoor thanks to #Henrik N's feedback
There are 2 questions related to your search on the StackExchange:
https://serverfault.com/questions/266373/postgresql-accent-diacritic-insensitive-search
But as you are on Heroku, I doubt this is a good match (unless you have a dedicated database plan).
There is also this one on SO: Removing accents/diacritics from string while preserving other special chars.
But this assumes that your data is stored without any accent.
I hope it will point you in the right direction.
Assuming Foo is the model you are searching against and name is the column. Combining Postgres translate and ActiveSupport's transliterate. You can do something like:
Foo.where(
"translate(
LOWER(name),
'âãäåāăąÁÂÃÄÅĀĂĄèééêëēĕėęěĒĔĖĘĚìíîïìĩīĭÌÍÎÏÌĨĪĬóôõöōŏőÒÓÔÕÖŌŎŐùúûüũūŭůÙÚÛÜŨŪŬŮ',
'aaaaaaaaaaaaaaaeeeeeeeeeeeeeeeiiiiiiiiiiiiiiiiooooooooooooooouuuuuuuuuuuuuuuu'
)
LIKE ?", "%#{ActiveSupport::Inflector.transliterate("%qué%").downcase}%"
)
Related
I'm using Rails4's Store feature. I've added a new stored attributes namely "friends" with four accessors, first ... fourth.
The problem is how to utilize it in the "where" condition. When I use it as:
#persons = Person.where(friends.has_value?#user.id)
I receive this error:
NameError in UsersController#myfrineds
undefined local variable or method `friends'
I tried some other different ways but still I get error. Could you please help me to solve it? Or please let me know if you have any better idea to implement it (storing a dynamic hash of key/values)?
As stated by Uzbekjon, store is not made for this kind of things. Some workarounds to your problem:
Use a custom query (would be quite slow depending on table size so be careful):
#persons = Person.where('friends LIKE ? OR friends LIKE ? OR friends LIKE ? OR friends LIKE ?', "%first: #{#person.id}\n%", "%second: #{#person.id}\n%", "%third: #{#person.id}\n%", "%fourth: #{#person.id}\n%")
This assumes you used yaml for the serialization of friends attribute (it's the default). If you used json you'll have to change the query accordingly.
If you're using PostgreSQL you can use array attribute instead of store. Queries would have better timings since PostgreSQL supports this datatype.
Migration:
def change
add_column :people, :friends, :text, array: true, default: []
# if you want to add index:
add_index :people, :friends, using: 'gin'
end
Creation of records:
Person.create(..., friends: [friend_id_1, friend_id_2, friend_id_3, friend_id_4])
Query:
#persons = Person.where('? = ANY(friends)', #person.id)
You may also need to add to your Gemfile:
gem 'postgres_ext'
Hope it helps!
Short answer - You can't! Because, ActiveRecord stores your "hash" as a string in a single column. The only way I can think of is to use .where("friends LIKE :friend", friend: 'BFF') syntax. Don't forget to index your column though.
It is mentioned in the docs as well:
It's like a simple key/value store baked into your record when you don't care about being able to query that store outside the context of a single record.
I used to do this with an array condition inside the where method:
Article.where('title ILIKE ?','%today%')
This worked in Postgres but ILIKE is not present in MySQL and other DBMS.
What I need is to be able to perform case insensitive queries using a code like
Article.ilike(title:'%today%',author:'%john%')
Even if there's not builtin method to perform case insensitive queries, you can use the Arel library and the matches method, like in:
Article.where(Article.arel_table[:title].matches('%today%'))
This is DB agnostic and SQL Injection proof.
I've written an ilike method in my common scope file, that allows you to call it with a list of attributes and values, that's it:
module CommonScopes
extend ActiveSupport::Concern
module ClassMethods
def ilike( options={} )
raise ArgumentError unless options.is_a? Hash or options.empty?
if options.one?
where(arel_table[options.keys.first].matches(options.values.first))
else
key, value = options.shift
ilike( {key=>value} ).merge( ilike( options ) )
end
end
end
end
You can place this inside app/models/concerns/common_scopes.rb and include where you need it.
No, there isn't. You need to write driver-specific SQL to achieve this.
ActiveRecord's goal is to make database access fast and easy for 90% of usecases, not to make your models completely database-agnostic. Switching your entire database backend from one system to another is not something they optimize for.
You might consider looking at another gem like DataMapper which provides a Ruby-syntax for wrapping things like like (but which may or may not provide an equivalent to ilike):
# If the value of a pair is an Array, we do an IN-clause for you.
Person.all(:name.like => 'S%', :id => [ 1, 2, 3, 4, 5 ])
Rails don't have the direct case sensitive search. It's dependent on the DB level. For MySQL you can use LOWER method.
YourModel.where('lower(column_name) = ?', str.downcase)
I need to select random records from db. In Sqlite3, which I use on development, there is a function called Random(). However, in Postgresql it's called Rand(). I don't remember about MySql, but probably it's called so there.
So if I have a code of (for Sqlite3)
data = Items.where(pubshied: is_pubshied).order("RANDOM()").limit(count)
how do I ensure that it will work with different databases?
Rails doesn't support this out of the box. I believe I achieved this with a model extension (I dont use it anymore because I force the use of Postgresql), but something like this could work:
module Randomize
extend ActiveSupport::Concern
included do
scope :random, -> { order(rand_cmd) }
end
module ClassMethods
def rand_cmd
if connection.adapter_name =~ /mysql/i
'rand()'
else
'random()'
end
end
end
end
You can then do
class Item
include Randomize
end
Item.where(...).random.limit(...)
For a performant, non-adapter-specific way to order randomly, populate a random column, put an index on it and call it something like:
Foo.order("random_column > #{rand}").limit(1)
From the comments from the post that waldyr.ar mentions in his comment: https://stackoverflow.com/a/12038506/16784.
Tl;dr: you can use Items.all.sample(count). Of course that retrieves the entire table and may not be useful for large tables.
I'm trying to update a field in using update_all. However I need the value to be taken from another field which is re-written to my specific format.
If I have something like this in my model:
def self.clean_mac_address()
clean_mac_address = :macaddress.gsub(/[^0-9a-z]/i, '')
end
When I run this:
Radacct.update_all("mac_clean = #{clean_mac_address}")
I get an error:
NoMethodError: undefined method `gsub' for :macaddress:Symbol
Any thoughts how I can do this? Or is there a simpler way to update the field?
update_all generates a single SQL query to run - it can't do clever stuff like change arbitrary bits of ruby into equivalent SQL.
You either need to load all you instances (via find_each for example) and fix them one by one (ie don't use update_all), for example
Foo.find_each do |foo|
# update foo here
foo.save!
end
Or find a way of expressing that cleaning operation in SQL. For example Postgres has a regexp_replace function
Foo.update_all("some_column = regexp_replace(some_column, 'your_regexp_here', '','g')")
Which would remove everything replacing that regexp. Obviously you'll need to check the documentation for your database to see whether it supports such a feature.
While the accepted answer provides a nice way to update_all, what I'd use is
read_with_clean_addr = Radacct.where(mac_clean: :macaddress.gsub(/[^0-9a-z]/i, ''))
read_with_clean_add.update_all(mac_clean: "#{clean_mac_address}")
I want to execute one update raw sql like below:
update table set f1=? where f2=? and f3=?
This SQL will be executed by ActiveRecord::Base.connection.execute, but I don't know how to pass the dynamic parameter values into the method.
Could someone give me any help on it?
It doesn't look like the Rails API exposes methods to do this generically. You could try accessing the underlying connection and using it's methods, e.g. for MySQL:
st = ActiveRecord::Base.connection.raw_connection.prepare("update table set f1=? where f2=? and f3=?")
st.execute(f1, f2, f3)
st.close
I'm not sure if there are other ramifications to doing this (connections left open, etc). I would trace the Rails code for a normal update to see what it's doing aside from the actual query.
Using prepared queries can save you a small amount of time in the database, but unless you're doing this a million times in a row, you'd probably be better off just building the update with normal Ruby substitution, e.g.
ActiveRecord::Base.connection.execute("update table set f1=#{ActiveRecord::Base.sanitize(f1)}")
or using ActiveRecord like the commenters said.
ActiveRecord::Base.connection has a quote method that takes a string value (and optionally the column object). So you can say this:
ActiveRecord::Base.connection.execute(<<-EOQ)
UPDATE foo
SET bar = #{ActiveRecord::Base.connection.quote(baz)}
EOQ
Note if you're in a Rails migration or an ActiveRecord object you can shorten that to:
connection.execute(<<-EOQ)
UPDATE foo
SET bar = #{connection.quote(baz)}
EOQ
UPDATE: As #kolen points out, you should use exec_update instead. This will handle the quoting for you and also avoid leaking memory. The signature works a bit differently though:
connection.exec_update(<<-EOQ, "SQL", [[nil, baz]])
UPDATE foo
SET bar = $1
EOQ
Here the last param is a array of tuples representing bind parameters. In each tuple, the first entry is the column type and the second is the value. You can give nil for the column type and Rails will usually do the right thing though.
There are also exec_query, exec_insert, and exec_delete, depending on what you need.
None of the other answers showed me how to use named parameters, so I ended up combining exec_update with sanitize_sql:
User.connection.exec_update(
User.sanitize_sql(
[
"update users set name = :name where id = :id and name <> :name",
{
id: 123,
name: 'My Name'
}
]
)
)
This works for me on Rails 5, and it executes this SQL:
update users set name = 'My Name' where id = 123 and name <> 'My Name'
You need to use an existing Rails model instead of User if you don't have that.
I wanted to use named parameters to avoid issues with the ordering when I use ? or $1/$2,etc. Positional ordering is kind of frustrating when I have more than a handful of parameters, but named parameters allow me to refactor the SQL command without having to update the parameters.
You should just use something like:
YourModel.update_all(
ActiveRecord::Base.send(:sanitize_sql_for_assignment, {:value => "'wow'"})
)
That would do the trick. Using the ActiveRecord::Base#send method to invoke the sanitize_sql_for_assignment makes the Ruby (at least the 1.8.7 version) skip the fact that the sanitize_sql_for_assignment is actually a protected method.
Sometime would be better use name of parent class instead name of table:
# Refers to the current class
self.class.unscoped.where(self.class.primary_key => id).update_all(created _at: timestamp)
For example "Person" base class, subclasses (and database tables) "Client" and "Seller"
Instead using:
Client.where(self.class.primary_key => id).update_all(created _at: timestamp)
Seller.where(self.class.primary_key => id).update_all(created _at: timestamp)
You can use object of base class by this way:
person.class.unscoped.where(self.class.primary_key => id).update_all(created _at: timestamp)
Here's a trick I recently worked out for executing raw sql with binds:
binds = SomeRecord.bind(a_string_field: value1, a_date_field: value2) +
SomeOtherRecord.bind(a_numeric_field: value3)
SomeRecord.connection.exec_query <<~SQL, nil, binds
SELECT *
FROM some_records
JOIN some_other_records ON some_other_records.record_id = some_records.id
WHERE some_records.a_string_field = $1
AND some_records.a_date_field < $2
AND some_other_records.a_numeric_field > $3
SQL
where ApplicationRecord defines this:
# Convenient way of building custom sql binds
def self.bind(column_values)
column_values.map do |column_name, value|
[column_for_attribute(column_name), value]
end
end
and that is similar to how AR binds its own queries.
I needed to use raw sql because I failed at getting composite_primary_keys to function with activerecord 2.3.8. So in order to access the sqlserver 2000 table with a composite primary key, raw sql was required.
sql = "update [db].[dbo].[#{Contacts.table_name}] " +
"set [COLUMN] = 0 " +
"where [CLIENT_ID] = '#{contact.CLIENT_ID}' and CONTACT_ID = '#{contact.CONTACT_ID}'"
st = ActiveRecord::Base.connection.raw_connection.prepare(sql)
st.execute
If a better solution is available, please share.
In Rails 3.1, you should use the query interface:
new(attributes)
create(attributes)
create!(attributes)
find(id_or_array)
destroy(id_or_array)
destroy_all
delete(id_or_array)
delete_all
update(ids, updates)
update_all(updates)
exists?
update and update_all are the operation you need.
See details here: http://m.onkey.org/active-record-query-interface