I have a column called "meta" in a table called "Notifications". The data in the column looks something like this:
---
:full_name: Dude
:user_id: 1
:update_id: 10
:update_type: want
:update_name: Some name here
and I want it to be changed to something like this:
---
:full_name: Dude
:user_id: 1
:post_id: 10
:post_type: want
:post_name: Some name here
What would a migration to achieve this look like?
def up
execute "update notifications set meta = replace(meta, 'update', 'post')"
end
I'm not entirely sure about the double quotes / single quotes, as I've seen both sometimes working sometimes failing even within different versions of the same SQL engine.
It is usually a good idea to use SQL to migrate data, as your ruby models might change over time and render your migration unusable, e.g. due to altered validation constraints. The same in ruby would look like
def up
Notification.find_each{ | n | n.update_attribute(:meta, n.meta.gsub('update','post')) }
end
Related
In our Rails app, the user (or we on his behalf) load some data or even insert it manually using a crud.
After this step the user must validate all the configuration (the data) and "accept and agree" that it's all correct.
On a given day, the application will execute some tasks according the configuration.
Today, we already have a "freeze" flag, where we can prevent changes in the data, so the user cannot mess the things up...
But we also would like to do something like hash the data and say something like "your config is frozen and the hash is 34FE00...".
This would give the user a certain that the system is running with the configuration he approved.
How can we do that? There are 7 or 8 tables. The total of records created would be around 2k or 3k.
How to hash the data to detect changes after the approval? How would you do that?
I'm thinking about doing a find_by_user in each table, loop all records and use some fields (or all) to build a string and hash it at the end of the current loop.
After loop all tables, I would have 8 hash strings and would concatenate and hash them in a final hash.
How does it looks like? Any ideas?
Here's a possible implementation. Just define object as an Array of all the stuff you'd like to hash :
require 'digest/md5'
def validation_hash(object, len = 16)
Digest::MD5.hexdigest(object.to_json)[0,len]
end
puts validation_hash([Actor.first,Movie.first(5)])
# => 94eba93c0a8e92f8
# After changing a single character in the first Actors's biography :
# => 35f342d915d6be4e
How can I modify a where/like condition on a search query in Rails:
find(:all, :conditions => ["lower(name) LIKE ?", "%#{search.downcase}%"])
so that the results are matched irrespective of accents? (eg métro = metro). Because I'm using utf8, I can't use "to_ascii". Production is running on Heroku.
Proper solution
Since PostgreSQL 9.1 you can just:
CREATE EXTENSION unaccent;
Provides a function unaccent(), doing what you need (except for lower(), just use that additionally if needed). Read the manual about this extension.
More about unaccent and indexes:
Does PostgreSQL support "accent insensitive" collations?
Poor man's solution
If you can't install unacccent, but are able to create a function. I compiled the list starting here and added to it over time. It is comprehensive, but hardly complete:
CREATE OR REPLACE FUNCTION lower_unaccent(text)
RETURNS text
LANGUAGE sql IMMUTABLE STRICT AS
$func$
SELECT lower(translate($1
, '¹²³áàâãäåāăąÀÁÂÃÄÅĀĂĄÆćčç©ĆČÇĐÐèéêёëēĕėęěÈÊËЁĒĔĖĘĚ€ğĞıìíîïìĩīĭÌÍÎÏЇÌĨĪĬłŁńňñŃŇÑòóôõöōŏőøÒÓÔÕÖŌŎŐØŒř®ŘšşșߊŞȘùúûüũūŭůÙÚÛÜŨŪŬŮýÿÝŸžżźŽŻŹ'
, '123aaaaaaaaaaaaaaaaaaacccccccddeeeeeeeeeeeeeeeeeeeeggiiiiiiiiiiiiiiiiiillnnnnnnooooooooooooooooooorrrsssssssuuuuuuuuuuuuuuuuyyyyzzzzzz'
));
$func$;
Your query should work like that:
find(:all, :conditions => ["lower_unaccent(name) LIKE ?", "%#{search.downcase}%"])
For left-anchored searches, you can use an index on the function for very fast results:
CREATE INDEX tbl_name_lower_unaccent_idx
ON fest (lower_unaccent(name) text_pattern_ops);
For queries like:
SELECT * FROM tbl WHERE (lower_unaccent(name)) LIKE 'bob%';
Or use COLLATE "C". See:
PostgreSQL LIKE query performance variations
Is there a difference between text_pattern_ops and COLLATE "C"?
For those like me who are having trouble on add the unaccent extension for PostgreSQL and get it working with the Rails application, here is the migration you need to create:
class AddUnaccentExtension < ActiveRecord::Migration
def up
execute "create extension unaccent"
end
def down
execute "drop extension unaccent"
end
end
And, of course, after rake db:migrate you will be able to use the unaccent function in your queries: unaccent(column) similar to ... or unaccent(lower(column)) ...
First of all, you install postgresql-contrib. Then you connect to your DB and execute:
CREATE EXTENSION unaccent;
to enable the extension for your DB.
Depending on your language, you might need to create a new rule file (in my case greek.rules, located in /usr/share/postgresql/9.1/tsearch_data), or just append to the existing unaccent.rules (quite straightforward).
In case you create your own .rules file, you need to make it default:
ALTER TEXT SEARCH DICTIONARY unaccent (RULES='greek');
This change is persistent, so you need not redo it.
The next step would be to add a method to a model to make use of this function.
One simple solution would be defining a function in the model. For instance:
class Model < ActiveRecord::Base
[...]
def self.unaccent(column,value)
a=self.where('unaccent(?) LIKE ?', column, "%value%")
a
end
[...]
end
Then, I can simply invoke:
Model.unaccent("name","text")
Invoking the same command without the model definition would be as plain as:
Model.where('unaccent(name) LIKE ?', "%text%"
Note: The above example has been tested and works for postgres9.1, Rails 4.0, Ruby 2.0.
UPDATE INFO
Fixed potential SQLi backdoor thanks to #Henrik N's feedback
There are 2 questions related to your search on the StackExchange:
https://serverfault.com/questions/266373/postgresql-accent-diacritic-insensitive-search
But as you are on Heroku, I doubt this is a good match (unless you have a dedicated database plan).
There is also this one on SO: Removing accents/diacritics from string while preserving other special chars.
But this assumes that your data is stored without any accent.
I hope it will point you in the right direction.
Assuming Foo is the model you are searching against and name is the column. Combining Postgres translate and ActiveSupport's transliterate. You can do something like:
Foo.where(
"translate(
LOWER(name),
'âãäåāăąÁÂÃÄÅĀĂĄèééêëēĕėęěĒĔĖĘĚìíîïìĩīĭÌÍÎÏÌĨĪĬóôõöōŏőÒÓÔÕÖŌŎŐùúûüũūŭůÙÚÛÜŨŪŬŮ',
'aaaaaaaaaaaaaaaeeeeeeeeeeeeeeeiiiiiiiiiiiiiiiiooooooooooooooouuuuuuuuuuuuuuuu'
)
LIKE ?", "%#{ActiveSupport::Inflector.transliterate("%qué%").downcase}%"
)
I am using Hirb in my Rails console to try to display mongoid records in a nicer looking table, but the _id, created_at, and updated_at fields take too much space and, even though I really have one meaning field, name, when I do a Project.first it returns a rather disintegrated table:
Is there any chance that there's a way to 'ignore' or 'hide' certain fields?
When you open up hirb, run this:
table projects, :fields => [:name]
Then Project.all and enjoy! ;-)
Also see this: Hirb - Irb On The Good Stuff.
UPDATE:
"Hirb also supports a yaml configuration file (config/hirb.yml or ~/.hirb.yml)"
---
:output:
Project:
:options:
:fields:
- name
Stuff I've already figured out
I'm learning how to create a multi-tenant application in Rails that serves data from different schemas based on what domain or subdomain is used to view the application.
I already have a few concerns answered:
How can you get subdomain-fu to work with domains as well? Here's someone that asked the same question which leads you to this blog.
What database, and how will it be structured? Here's an excellent talk by Guy Naor, and good question about PostgreSQL and schemas.
I already know my schemas will all have the same structure. They will differ in the data they hold. So, how can you run migrations for all schemas? Here's an answer.
Those three points cover a lot of the general stuff I need to know. However, in the next steps I seem to have many ways of implementing things. I'm hoping that there's a better, easier way.
Finally, to my question
When a new user signs up, I can easily create the schema. However, what would be the best and easiest way to load the structure that the rest of the schemas already have? Here are some questions/scenarios that might give you a better idea.
Should I pass it on to a shell script that dumps the public schema into a temporary one, and imports it back to my main database (pretty much like what Guy Naor says in his video)? Here's a quick summary/script I got from the helpful #postgres on freenode. While this will probably work, I'm gonna have to do a lot of stuff outside of Rails, which makes me a bit uncomfortable.. which also brings me to the next question.
Is there a way to do this straight from Ruby on Rails? Like create a PostgreSQL schema, then just load the Rails database schema (schema.rb - I know, it's confusing) into that PostgreSQL schema.
Is there a gem/plugin that has these things already? Methods like "create_pg_schema_and_load_rails_schema(the_new_schema_name)". If there's none, I'll probably work at making one, but I'm doubtful about how well tested it'll be with all the moving parts (especially if I end up using a shell script to create and manage new PostgreSQL schemas).
Thanks, and I hope that wasn't too long!
Update Dec 5, 2011
Thanks to Brad Robertson and his team, there's the Apartment gem. It's very useful and does a lot of the heavy lifting.
However, if you'll be tinkering with schemas, I strongly suggest knowing how it actually works. Familiarize yourself with Jerod Santo's walkthrough , so you'll know what the Apartment gem is more or less doing.
Update Aug 20, 2011 11:23 GMT+8
Someone created a blog post and walks though this whole process pretty well.
Update May 11, 2010 11:26 GMT+8
Since last night I've been able to get a method to work that creates a new schema and loads schema.rb into it. Not sure if what I'm doing is correct (seems to work fine, so far) but it's a step closer at least. If there's a better way please let me know.
module SchemaUtils
def self.add_schema_to_path(schema)
conn = ActiveRecord::Base.connection
conn.execute "SET search_path TO #{schema}, #{conn.schema_search_path}"
end
def self.reset_search_path
conn = ActiveRecord::Base.connection
conn.execute "SET search_path TO #{conn.schema_search_path}"
end
def self.create_and_migrate_schema(schema_name)
conn = ActiveRecord::Base.connection
schemas = conn.select_values("select * from pg_namespace where nspname != 'information_schema' AND nspname NOT LIKE 'pg%'")
if schemas.include?(schema_name)
tables = conn.tables
Rails.logger.info "#{schema_name} exists already with these tables #{tables.inspect}"
else
Rails.logger.info "About to create #{schema_name}"
conn.execute "create schema #{schema_name}"
end
# Save the old search path so we can set it back at the end of this method
old_search_path = conn.schema_search_path
# Tried to set the search path like in the methods above (from Guy Naor)
# [METHOD 1]: conn.execute "SET search_path TO #{schema_name}"
# But the connection itself seems to remember the old search path.
# When Rails executes a schema it first asks if the table it will load in already exists and if :force => true.
# If both true, it will drop the table and then load it.
# The problem is that in the METHOD 1 way of setting things, ActiveRecord::Base.connection.schema_search_path still returns $user,public.
# That means that when Rails tries to load the schema, and asks if the tables exist, it searches for these tables in the public schema.
# See line 655 in Rails 2.3.5 activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
# That's why I kept running into this error of the table existing when it didn't (in the newly created schema).
# If used this way [METHOD 2], it works. ActiveRecord::Base.connection.schema_search_path returns the string we pass it.
conn.schema_search_path = schema_name
# Directly from databases.rake.
# In Rails 2.3.5 databases.rake can be found in railties/lib/tasks/databases.rake
file = "#{Rails.root}/db/schema.rb"
if File.exists?(file)
Rails.logger.info "About to load the schema #{file}"
load(file)
else
abort %{#{file} doesn't exist yet. It's possible that you just ran a migration!}
end
Rails.logger.info "About to set search path back to #{old_search_path}."
conn.schema_search_path = old_search_path
end
end
Change line 38 to:
conn.schema_search_path = "#{schema_name}, #{old_search_path}"
I presume that postgres is trying to lookup existing table names when loading schema.rb and since you've set the search_path to only contain the new schema, it fails. This of course, is presuming you still have the public schema in your database.
Hope that helps.
Is there a gem/plugin that has these things already?
pg_power provides this functionality to create/drop PostgreSQL schemas in migration, like this:
def change
# Create schema
create_schema 'demography'
# Create new table in specific schema
create_table "countries", :schema => "demography" do |t|
# columns goes here
end
# Drop schema
drop_schema 'politics'
end
Also it takes care about correctly dumping schemas into schema.rb file.
Is there a way to do this in Rails:
I have an activerecord query
#posts = Post.find_by_id(10)
Anytime the query is called, SQL is generated and executed at the DB that looks like this
SELECT * FROM 'posts' WHERE id = 10
This happens every time the AR query is executed. Similarly with a helper method like this
<%= f.textarea :name => 'foo' %>
#=> <input type='textarea' name='foo' />
I write some Railsy code that generates some text that is used by some other system (database, web browser). I'm wondering if there's a way to write an AR query or a helper method call that generates the text in the file. This way the text rendering is only done once (each time the code changes) instead of each time the method is called?
Look at the line, it may be going to the database for the first one but ones after it could be saying CACHE at the start of the line meaning it's going to ActiveRecord's query cache.
It also sounds to me like you want to cache the page, not the query. And even if it were the query, I don't think it's as simple as find_by_id(10) :)
Like Radar suggested you should probably look into Rails caching. You can start with something simple like the memory store or file cache and then move to something better like memcached if necessary. You can throw in some caching into the helper method which will cache the result after it is queried once. So for example you can do:
id = 10 # id is probably coming in as a param/argument
cache_key = "Post/#{id}"
#post = Rails.cache.read(cache_key)
if #post.nil?
#post = Post.find_by_id(id)
# Write back to the cache for the next time
Rails.cache.write(cache_key,#post)
end
The only other thing left to do is put in some code to expire the cache entry if the post changes. For that take a look at using "Sweepers" in Rails. Alternatively you can look at some of the caching gems like Cache-fu and Cached-model.
I'm not sure I understand your question fully.
If you're asking about the generated query, you can just do find_by_sql and write your own SQL if you don't want to use the active record dynamic methods.
If you're asking about caching the resulset to a file, it's already in the database, I don't know that if it was in a file it would be much more efficient.