Use of Migrations in Ruby on Rails - ruby-on-rails

I would like to confirm that the following analysis is correct:
I am building a web app in RoR. I have a data structure for my postgres db designed (around 70 tables; this design may need changes and additions during development to reflect Rails ways of doing things. EG, I designed some user and role tables - but if it makes sense to use Restful Authentication, I will scrub them and replace with whatever RA requires. ).
I have a shellscript which calls a series of .sql files to populate the empty database with tables and initial data (eg, Towns gets pre-filled with post towns) as well as test data (eg, Companies gets a few dummy companies so I have data to play with).
for example:
CREATE TABLE towns (
id integer PRIMARY KEY DEFAULT nextval ('towns_seq'),
county_id integer REFERENCES counties ON DELETE RESTRICT ON UPDATE CASCADE,
country_id integer REFERENCES countries ON DELETE RESTRICT ON UPDATE CASCADE NOT NULL,
name text NOT NULL UNIQUE
);
Proposition 0: Data lasts longer than apps, so I am convinced that I want referential integrity enforced at the DB level as well as validations in my RoR models, despite the lack of DRYNESS.
Proposition 1: If I replace the script and sql files with Migrations, it is currently impossible to tell my Postgres database about the Foreign Key and other constraints I currently set in SQL DDL files within the migration code.
Proposition 2: The touted benefit of migrations is that changes to the schema are versioned along with the RoR model code. But if I keep my scripts and .sql files in railsapp/db, I can version them just as easily.
Proposition 3: Given that migrations lack functionality I want, and provide benefits I can replicate, there is little reason for me to consider using them. So I should --skipmigrations at script/generate model time.
My question: If Proposition 0 is accepted, are Propositions 1,2,3 true or false, and why?
Thanks!

Proposition 1 is false in at least two situations - you can use plugins like foreign_key_migrations to do the following:
def self.up
create_table :users do |t|
t.column :department_id, :integer, :references => :departments
end
end
which creates the appropriate foreign key constraint in your DB.
Of course, you might have other things that you want to do in your DDL, in which case the second situation becomes more compelling: you're not forced to use the Ruby DSL in migrations. Try the execute method, instead:
def self.up
execute 'YOUR SQL HERE'
end
With that, you can keep the contents of your SQL scripts in migrations, gaining the benefits of the latter (most prominently the down methods, which you didn't address in your original question) and retaining the lower-level control you prefer.

Proposition 1 is mistaken : you can definitely define referential integrity using migrations if only by using direct SQL inside the migration, see this post for more details.
Proposition 2: The touted interest of migrations is to be able to define your database model incrementally while keeping track of what each change added and be able to easily rollback any such change at a later time.
You have to be careful with the order you create/modify things in but you can do it.
One thing to keep in mind : rails is better suited for application-centri design. in the Rails Way(tm) the database is only ever accessed through the application active record layer and exposes data to the outside using webservices

1: You may want to try out this plugin. I didn't try it myself though, but it seems to be able to add foreign key constraints through migrations.
2: The real benefit of migration is the ability to go back and forth in the history of your database. That's not as easy with your .sql files.
3: See if the above-mentioned plugin works for you, then decide :) At any rate, it's not a capital sin if you don't use them!

Since you are using Postgres and may not want to install the foreign_key_migrations plugin, here is what I do when I want to use both migrations and foreign key constraints.
I add a SchemaStatements method to ActiveRecord::SchemaStatements called "add_fk_constraint".
This could go in some centralized file, but in the example migration file below, I have just put it inline.
module ActiveRecord
module ConnectionAdapters # :nodoc:
module SchemaStatements
# Example call:
# add_fk_constraint 'orders','advertiser_id','advertisers','id'
# "If you want add/alter a 'orders' record, then its 'advertiser_id' had
# better point to an existing 'advertisers' record with corresponsding 'id'"
def add_fk_constraint(table_name, referencing_col, referenced_table, referenced_col)
fk_name = "#{table_name}_#{referencing_col}"
sql = <<-ENDSQL
ALTER TABLE #{table_name}
ADD CONSTRAINT #{fk_name}
FOREIGN KEY (#{referencing_col}) REFERENCES #{referenced_table} (#{referenced_col})
ON UPDATE NO ACTION ON DELETE CASCADE;
CREATE INDEX fki_#{fk_name} ON #{table_name}(#{referencing_col});
ENDSQL
execute sql
end
end
end
end
class AdvertisersOrders < ActiveRecord::Migration
def self.up
create_table :advertisers do |t|
t.column :name, :string, :null => false
t.column :net_id, :integer, :null => false
t.column :source_service_id, :integer, :null => false, :default => 1
t.column :source_id, :integer, :null => false
end
create_table :orders do |t|
t.column :name, :string, :null => false
t.column :advertiser_id, :integer, :null => false
t.column :source_id, :integer, :null => false
end
add_fk_constraint 'orders','advertiser_id','advertisers','id'
end
def self.down
drop_table :orders
drop_table :advertisers
end
end
I hopes this helps someone. It has been very useful to me since I need to load a lot of externally supplied data with SQL "COPY" calls, yet I find the migrations system very convenient.

Related

Active Record adding plain sql index fails with case sensitivity [duplicate]

I need to create a case-insensitive index on a column in rails. I did this via SQL:
execute(
"CREATE UNIQUE INDEX index_users_on_lower_email_index
ON users (lower(email))"
)
This works great, but in my schema.rb file I have:
add_index "users", [nil],
:name => "index_users_on_lower_email_index",
:unique => true
Notice the "nil". So when I try to clone the database to run a test, I get an obvious error. Am I doing something wrong here? Is there some other convention that I should be using inside rails?
Thanks for the help.
Since MySQL indexes are already case-insensitive, I'm guessing you're dealing with PostgreSQL, which creates case-sensitive indexes by default. I'm answering here based on Rails 3.2.3 and PostgreSQL 8.4.
It seems functional indexes are one more example of things that ActiveRecord can't generate. Foreign keys and UUID columns are two more that come to mind. So there is no choice (other than monkey-patching ActiveRecord) but to use execute statements.
This means for an accurate dump of your database, you'll need to abandon the DB-agnostic schema.rb in favor of DB-specific structure.sql. See the Rails Guide on Migrations, section 6.2 Types of Schema Dumps. This is set as follows:
config/application.rb
config.active_record.schema_format = :sql
db/structure.sql should be updated automatically when you run a migration. You can generate it manually with this command:
rake db:structure:dump
The file is pure Postgres SQL. Although not documented when you use rake -T to list rake tasks, it seems that you can use this command to load the database from the structure.sql dump:
rake db:structure:load
There's nothing magic here: the source code (shown here from Rails 3.2.16) just calls psql on structure.sql.
Finally, here is my migration to drop an old, case-sensitive email constraint and add the case-sensitive functional index:
class FixEmailUniqueIndexOnUsers < ActiveRecord::Migration
def up
remove_index :users, :email
execute "CREATE UNIQUE INDEX index_users_on_lowercase_email
ON users USING btree (lower(email));"
end
def down
execute "DROP INDEX index_users_on_lowercase_email;"
add_index :users, :email, :unique => true
end
end
If you are using PostgreSQL you can change your column type to citext - case-insensitive string. It also makes search independent from the register.
def change
enable_extension :citext
change_column :users, :email, :citext
add_index :users, :email, unique: true
end
I would simplify this...
In your model:
before_validation :downcase_email
def downcase_email
self.email = email.downcase
end
That way, the index is database agnostic, and your emails are all lowercase in the database.
Have you considered using schema_plus (https://github.com/lomba/schema_plus)? Among other things (support for enforcing foreign keys in the database and for views), it supports setting case-insensitive indexes for PostgreSQL databases and handles dumping them in the schema. From the Readme, "If you’re using Postgresql, SchemaPlus provides support for conditions, expressions, index methods, and case-insensitive indexes."
The documentation is unclear on how to do this but the source looks like this:
def add_index(table_name, column_name, options = {})
index_name, index_type, index_columns = add_index_options(table_name, column_name, options)
execute "CREATE #{index_type} INDEX #{quote_column_name(index_name)} ON #{quote_table_name(table_name)} (#{index_columns})"
end
So, if your database's quote_column_name is the default implementation (which does nothing at all), then this might work:
add_index "users", ['lower(email)'], :name => "index_users_on_lower_email_index", :unique => true
You note that you tried that one but it didn't work (adding that to your question might be a good idea). Looks like ActiveRecord simply doesn't understand indexes on a computed value. I can think of an ugly hack that will get it done but it is ugly:
Add an email_lc column.
Add a before_validation or before_save hook to put a lower case version of email into email_lc.
Put your unique index on email_lc.
That's pretty ugly and you might feel dirty for doing it but that's the best I can think of right now.
I would suggest (just as an opportunity to consider among others) to use two separate fields:
email
email_original
The first one is always down cased and thus can be uniquely indexed in a database agnostic manner, while the second field keeps verbatim to what user enetered and can have upper characters.
Obviously in User model one then needs to set email based on email_original on each save and prohibit direct mangling with email field.
For Rails 4.2, create case-insensitive unique index in users table on name column.
Create new migration file with empty change method:
$ rails generate migration add_index_in_users_on_name
Add call of add_index method to empty change method:
add_index :users, 'lower(name)', name: 'index_users_on_lower_name', unique: true
Run Rake db:migrate task:
$ rake db:migrate
In result, index will be added correctly and file db/schema.rb contains correct add_index:
add_index "users", ["LOWER(\"NAME\")"], name: "index_users_on_lower_name", unique: true
This tested only with RDB Oracle.
I think you need column name(s) as below
add_index "users", [email], {:name => "index_users_on_lower_email_index", :unique => true }
And you have to make email field in database with proper Case Insensitive collation, that way your index will be also case insensitive.
depending of db engine which you are using it syntax may be different but
alter table [users] alter column [email] varchar(250) collate utf8_general_ci ...
and when you add index to this column it will be case insensitive.

How do I database level contraints to my rails application long after I've migrated everything?

1) I've created an entire Rails JSON API as my first project. It's fairly sizeable, and I put off database level constraints at the time. It's time to do it now.
I have 30+ or so migrations. For example
class CreateItems < ActiveRecord::Migration
def change
create_table :items do |t|
t.string :name
t.text :description
t.timestamps
end
end
end
When I go in and edit
t.string :name
to be
t.string :name, null: false
and run a bundle exec db:reset db:migrate it doesn't work. What do I need to do to add db level constraints to my migrations files, and re migrate them?
2) Also, my 30+ migrations all have a "def change" and that's it. I believe somewhere a long time ago I saw something like def up, def down or something like that. It looks like it was used for rollbacks. Do I need to have "def down" for all of my migrations? Is that how I solve #1? Put "def down" in all of my 30 migrations, rollback the entire thing, then put in my db level constraints, and migrate the whole thing again?
You have two options: you can write new migrations to make changes to the existing columns. In console:
rails g migration add_null_to_name
In the migration file:
class ChangeNumbericFieldInMyTable < ActiveRecord::Migration
def self.up
change_column :my_table, :name, :string, :null => false
end
def self.down
change_column :my_table, :name, :string
end
Rinse and repeat for all the changes you need to make. Note that you can define them in the same migration file, but then if you had to rollback in the future it would roll back all of them.
Your second option is to roll everything back (or just start over), fix your migration methods and remigrate. This might be a cleaner approach, but that's not necessarily a good thing: it would also delete the existing data, if you have any.
I know I said you have two options, but you actually also have a third option, which is to to rely on model validations instead. These are more appropriate in most situations and can make testing much easier. Database validations are more reserved for complex scenarios where the database is used by other applications in addition to yours, and a few other edge cases.
As for your second question, every migration is a two-way street. When you migrate something, you should also be able to roll it back. In a lot of cases Rails can guess the reverse operation. For example, if you add_column in a migration, the opposite of that is obviously remove_column so you don't need to define it explicitly. But certain operations, such as change_column in the example above, don't have an obvious reverse. Rails can understand that something changed, but it can't tell what it changed from. So in those situations you need to explicitly define the down operation. In the case above, it's changing the column back to a state where it doesn't have a validation.

Rollback and change or create a new migration

I am just curious,
Suppose I create this migration:
def change
create_table :pages do |t|
t.string :title
t.text :content
t.timestamps
end
end
and then I run the migration.
Now after a couple of hours I remember that I should have added a slug column too.
Now (supposing I haven't created any other migrations after this one), should I rollback and add the new field here (in this migration), or should I create a new migration and add the filed there?
If you haven't pushed it out to production, I'd say just roll back and add it into the existing migration. If you have deployed it to production, then I'd make another migration.
This is, however, mostly a matter of preference, if there's no data that will be lost as a result. I just like to get into the habit of doing things the safer way in production.

Case-insensitive unique index in Rails/ActiveRecord?

I need to create a case-insensitive index on a column in rails. I did this via SQL:
execute(
"CREATE UNIQUE INDEX index_users_on_lower_email_index
ON users (lower(email))"
)
This works great, but in my schema.rb file I have:
add_index "users", [nil],
:name => "index_users_on_lower_email_index",
:unique => true
Notice the "nil". So when I try to clone the database to run a test, I get an obvious error. Am I doing something wrong here? Is there some other convention that I should be using inside rails?
Thanks for the help.
Since MySQL indexes are already case-insensitive, I'm guessing you're dealing with PostgreSQL, which creates case-sensitive indexes by default. I'm answering here based on Rails 3.2.3 and PostgreSQL 8.4.
It seems functional indexes are one more example of things that ActiveRecord can't generate. Foreign keys and UUID columns are two more that come to mind. So there is no choice (other than monkey-patching ActiveRecord) but to use execute statements.
This means for an accurate dump of your database, you'll need to abandon the DB-agnostic schema.rb in favor of DB-specific structure.sql. See the Rails Guide on Migrations, section 6.2 Types of Schema Dumps. This is set as follows:
config/application.rb
config.active_record.schema_format = :sql
db/structure.sql should be updated automatically when you run a migration. You can generate it manually with this command:
rake db:structure:dump
The file is pure Postgres SQL. Although not documented when you use rake -T to list rake tasks, it seems that you can use this command to load the database from the structure.sql dump:
rake db:structure:load
There's nothing magic here: the source code (shown here from Rails 3.2.16) just calls psql on structure.sql.
Finally, here is my migration to drop an old, case-sensitive email constraint and add the case-sensitive functional index:
class FixEmailUniqueIndexOnUsers < ActiveRecord::Migration
def up
remove_index :users, :email
execute "CREATE UNIQUE INDEX index_users_on_lowercase_email
ON users USING btree (lower(email));"
end
def down
execute "DROP INDEX index_users_on_lowercase_email;"
add_index :users, :email, :unique => true
end
end
If you are using PostgreSQL you can change your column type to citext - case-insensitive string. It also makes search independent from the register.
def change
enable_extension :citext
change_column :users, :email, :citext
add_index :users, :email, unique: true
end
I would simplify this...
In your model:
before_validation :downcase_email
def downcase_email
self.email = email.downcase
end
That way, the index is database agnostic, and your emails are all lowercase in the database.
Have you considered using schema_plus (https://github.com/lomba/schema_plus)? Among other things (support for enforcing foreign keys in the database and for views), it supports setting case-insensitive indexes for PostgreSQL databases and handles dumping them in the schema. From the Readme, "If you’re using Postgresql, SchemaPlus provides support for conditions, expressions, index methods, and case-insensitive indexes."
The documentation is unclear on how to do this but the source looks like this:
def add_index(table_name, column_name, options = {})
index_name, index_type, index_columns = add_index_options(table_name, column_name, options)
execute "CREATE #{index_type} INDEX #{quote_column_name(index_name)} ON #{quote_table_name(table_name)} (#{index_columns})"
end
So, if your database's quote_column_name is the default implementation (which does nothing at all), then this might work:
add_index "users", ['lower(email)'], :name => "index_users_on_lower_email_index", :unique => true
You note that you tried that one but it didn't work (adding that to your question might be a good idea). Looks like ActiveRecord simply doesn't understand indexes on a computed value. I can think of an ugly hack that will get it done but it is ugly:
Add an email_lc column.
Add a before_validation or before_save hook to put a lower case version of email into email_lc.
Put your unique index on email_lc.
That's pretty ugly and you might feel dirty for doing it but that's the best I can think of right now.
I would suggest (just as an opportunity to consider among others) to use two separate fields:
email
email_original
The first one is always down cased and thus can be uniquely indexed in a database agnostic manner, while the second field keeps verbatim to what user enetered and can have upper characters.
Obviously in User model one then needs to set email based on email_original on each save and prohibit direct mangling with email field.
For Rails 4.2, create case-insensitive unique index in users table on name column.
Create new migration file with empty change method:
$ rails generate migration add_index_in_users_on_name
Add call of add_index method to empty change method:
add_index :users, 'lower(name)', name: 'index_users_on_lower_name', unique: true
Run Rake db:migrate task:
$ rake db:migrate
In result, index will be added correctly and file db/schema.rb contains correct add_index:
add_index "users", ["LOWER(\"NAME\")"], name: "index_users_on_lower_name", unique: true
This tested only with RDB Oracle.
I think you need column name(s) as below
add_index "users", [email], {:name => "index_users_on_lower_email_index", :unique => true }
And you have to make email field in database with proper Case Insensitive collation, that way your index will be also case insensitive.
depending of db engine which you are using it syntax may be different but
alter table [users] alter column [email] varchar(250) collate utf8_general_ci ...
and when you add index to this column it will be case insensitive.

How might you clone a database table via Rails migration?

I want a migration to create a clone of an existing table by just suffixing the name, including all the indexes from the original table.
So there's a "snapshots" table and I want to create "snapshots_temp" as an exact copy of the table (not the data, just the table schema, but including the indexes).
I could just copy and paste the block out of the schema.rb file and manually rename it.
But I'm not sure by the time this migration is applied if the definition from schema.rb will still be accurate. Another developer might have changed the table and I don't want to have to update my migration script.
So how might I get the schema of the table at runtime? Essentially, how does 'rake schema:dump' reverse engineer the table so I can do the same in my migration? (but changing the table name).
Try doing it with pure SQL. This will do what you want:
CREATE TABLE new_tbl LIKE orig_tbl;
In Rails 4 & PostgreSQL, create a new migration and insert:
ActiveRecord::Base.connection.execute("CREATE TABLE clone_table_name AS SELECT * FROM source_table_name;")
This will create the clone with the exact structure of the original table, and populate the new table with old values.
More info: http://www.postgresql.org/docs/9.0/static/sql-createtableas.html
This will do. It's not perfect, because it won't copy table options or indices. If you do have any table options set, you will have to add them to this migration manually.
To copy indices you'll have to formulate a SQL query to select them, and then process them into new add_index directives. That's a little beyond my knowledge. But this works for copying the structure.
class CopyTableSchema < ActiveRecord::Migration
def self.up
create_table :new_models do |t|
Model.columns.each do |column|
next if column.name == "id" # already created by create_table
t.send(column.type.to_sym, column.name.to_sym, :null => column.null,
:limit => column.limit, :default => column.default, :scale => column.scale,
:precision => column.precision)
end
end
# copy data
Model.all.each do |m|
NewModel.create m.attributes
end
end
def self.down
drop_table :new_models
end
end
Copy the tables entry from your projects db/schema.rb straight into your migration. Just change the table name and your good to go.
It looks like this logic is wrapped up in ActiveRecord::SchemaDumper but it only exposes an all-in-one "dump" method and you can't dump just a specific table (the "table" method is private).

Resources