I want a migration to create a clone of an existing table by just suffixing the name, including all the indexes from the original table.
So there's a "snapshots" table and I want to create "snapshots_temp" as an exact copy of the table (not the data, just the table schema, but including the indexes).
I could just copy and paste the block out of the schema.rb file and manually rename it.
But I'm not sure by the time this migration is applied if the definition from schema.rb will still be accurate. Another developer might have changed the table and I don't want to have to update my migration script.
So how might I get the schema of the table at runtime? Essentially, how does 'rake schema:dump' reverse engineer the table so I can do the same in my migration? (but changing the table name).
Try doing it with pure SQL. This will do what you want:
CREATE TABLE new_tbl LIKE orig_tbl;
In Rails 4 & PostgreSQL, create a new migration and insert:
ActiveRecord::Base.connection.execute("CREATE TABLE clone_table_name AS SELECT * FROM source_table_name;")
This will create the clone with the exact structure of the original table, and populate the new table with old values.
More info: http://www.postgresql.org/docs/9.0/static/sql-createtableas.html
This will do. It's not perfect, because it won't copy table options or indices. If you do have any table options set, you will have to add them to this migration manually.
To copy indices you'll have to formulate a SQL query to select them, and then process them into new add_index directives. That's a little beyond my knowledge. But this works for copying the structure.
class CopyTableSchema < ActiveRecord::Migration
def self.up
create_table :new_models do |t|
Model.columns.each do |column|
next if column.name == "id" # already created by create_table
t.send(column.type.to_sym, column.name.to_sym, :null => column.null,
:limit => column.limit, :default => column.default, :scale => column.scale,
:precision => column.precision)
end
end
# copy data
Model.all.each do |m|
NewModel.create m.attributes
end
end
def self.down
drop_table :new_models
end
end
Copy the tables entry from your projects db/schema.rb straight into your migration. Just change the table name and your good to go.
It looks like this logic is wrapped up in ActiveRecord::SchemaDumper but it only exposes an all-in-one "dump" method and you can't dump just a specific table (the "table" method is private).
Related
with what migration can I add a column to every model at once? This would be handy almost every time you forgot to add a field anyway because most of the time a field you want to add is not limited to one model only. Like "email" In this case I forgot to add an "order" field.
Thanks
You can do as follows -
def change
tables = [:users, :products, :addresses]
tables.each do |table_name|
add_column table_name, :order, :integer
end
end
It's called a group migration
Answering your first question "With what migration can I add a column to every model at once?". Answer: None. Rails migrations are a way to alter database schemas over time in a consistent way.
Rails migrations are Ruby classes using Rails methods as instructions to modify your database as needed. So your question could be better formulated as "How can I create a migration to add a column to every model at once?"
IMHO I don't think there's going to be a specific method to do this, as the requeriment is pretty custom, but, depending in your Rails version you can get all the ApplicationRecord.descendants:
Zeitwerk::Loader.eager_load_all
ApplicationRecord.descendants.map { |table| table.name.downcase.pluralize }.each do |table|
add_column table, :logdate, :datetime
end
Or all those tables from the database that can be safe_constantized:
ActiveRecord::Base.connection.tables.map do |table|
table.classify.safe_constantize
end.reject(&:nil?).each do |table|
add_column table, :logdate, :datetime
end
That way you get the name of each table/model and use it as the first argument for add_column.
The difference is that in Rails 6 the default code loader is Zeitwerk, so you can eager load all the project dependencies. In other versions you could do the same but using Rails.application.eager_load!.
The second version would work without having to load the models as dependencies as it makes a query asking for their tables and then maps them as constants.
The question
I am using Rails 2.3 with the Spatial Adapter plugin and a Postgres 8.4 database with PostGIS extension.
A while ago, a geometry field has been added to a table via a migration like this:
add_column :my_table, :the_geom, :geometry, :srid => 4326
Along with the field, a table dimension constraint is created. In this case it makes sure the geometry entered into the the_geom field does have exactly two dimensions.
Now, I'd like to add geometries with more than 2 dimensions. What Rails migration code could I use to either change the constraint to allow more than two dimensions or remove it altogether?
What didn't work
When I create a new geometry field via the add_column method, I know I can use the :with_z and :with_m options to increase the number of dimensions allowed (the constraint would allow up to four dimension when using both options).
However, when using these options with the change_column command, it doesn't seem to have any effect on the constraint.
I don't want to remove the column and re-add it, because then I lose all data.
I know, I can use raw SQL to remove the constraint (I could also use a migration or rake task as a wrapper). The problem with this is, that it doesn't have an effect on the test database, because it is always created from scratch when I run all tests.
I'm not familiar with the spatial adapter fields/options so this code might be a little off, but how about something like this?
class UpgradeGeometryMigration < ActiveRecord::Migration
def self.up
rename_column :my_table, :the_geom, :the_geom_old
add_column :my_table, :the_geom, :geometry, :srid => 4326, :with_z => true, :with_m => true
# Add some code here to iterate through all of your records in the
# :the_geom_old table and copy them into the new :the_geom field
remove_column :my_table, :the_geom_old
# Add any indices required for :the_geom
end
end
There is no way to alter a geometry column without turning to raw SQL via ActiveRecord::Base.connection.execute.
Unfortunately raw SQL statements do not make it into the Rails database schema and so the constraints remain in the DB when it is recreated (for tests for example).
Some raw SQL statements end up in the database schema. The following modifies the column type from geometry to geography:
class ChangePostGisGeometryToGeography < ActiveRecord::Migration
def up
execute <<-SQL
ALTER TABLE mytable ALTER COLUMN geom TYPE geography(MULTIPOLYGON,4326);
SQL
end
end
The migration modifies db/schema.rb correctly :
- t.spatial "geom", :limit => {:srid=>4326, :type=>"multi_polygon"}
+ t.spatial "geom", :limit => {:srid=>4326, :type=>"multi_polygon", :geographic=>true}
(Tested under Rails 3.2.13 and activerecord-postgis-adapter 0.6.6)
Let's say you have "lineitems" and you used to define a "make" off of a line_item.
Eventually you realize that a make should probably be on its own model, so you create a Make model.
You then want to remove the make column off of the line_items table but for every line_item with a make you want to find_or_create_by(line_item.make).
How would I effectively do this in a rails migration? I'm pretty sure I can just run some simple find_or_create_by for each line_item but I'm worried about fallback support so I was just posting this here for any tips/advice/right direction.
Thanks!
I guess you should check that the Make.count is equal to the total unique makes in lineitems before removing the column, and raise an error if it does not. As migrations are transactional, if it blows up, the schema isn't changed and the migration isn't marked as executed. Therefore, you could do something like this:
class CreateMakesAndMigrateFromLineItems < ActiveRecord::Migration
def self.up
create_table :makes do |t|
t.string :name
…
t.timestamps
end
makes = LineItem.all.collect(:&make).uniq
makes.each { |make| Make.find_or_create_by_name make }
Make.count == makes.length ? remove_column(:line_items, :make) : raise "Boom!"
end
def self.down
# You'll want to put logic here to take you back to how things were before. Just in case!
drop_table :makes
add_column :line_items, :make
end
end
You can put regular ruby code in your migration, so you can create the new table, run some code across the old model moving the data into the new model, and then delete the columns from the original model. This is even reversible so your migration will still work in both directions.
So for your situation, create the Make table and add a make_id to the lineitem. Then for each line item, find_or_create with the make column on lineitem, setting the returned id to the new make_id on lineitem. When you are done remove the old make column from the lineitem table.
I'm trying to convert a column in my Rails app, for arguments sake let's pretend I'm trying to change the age column in my users table to a string representation rather than an int.
In my migration I have this;
def.self up
add_column :users, :age_text, :string
users = User.find(:all)
users.each do |u|
u.age_text = convert_to_text(u.age)
u.save
end
end
def self.convert_to_text(number)
#code here to convert 1 to 'one' etc
end
But it doesn't seem to be working, is what I'm attempting here even possible with migrations?
What you're trying to do is possible, and I would say the correct thing to do.
You need, though, to reload the column info for the model classes you're updating in the migration, so that Rails knows about the new columns. Try this:
def.self up
add_column :users, :age_text, :string
User.reset_column_information
users = User.find(:all)
users.each do |u|
u.age_text = convert_to_text(u.age)
u.save
end
end
On a separate note, please note that if your table is large, doing updates one by one will take a looong time.. Be careful with that.
Since I'm new here I can't comment on the above so I'll add my own answer.
GENERALLY manipulating data in migrations is a BAD idea. Migrations with direct model access can get stuck if the model logic changes.
Imagine in your second migration you've added a new column. You want to seed that column with new data.
Let's also say a few weeks later you add a new validation to the model - a validation that operates on a field that does not yet exist in your second migration. if you ever were to construct the database from migration 0, you'd have some problems.
I strongly suggest using migrations to alter columns and other means to manage database data, especially when moving to production.
Here is an example migration I ran to convert data. You can easily convert it to use integers instead of strings. Making the conversion in SQL is much faster than loading each row in Rails.
class ConvertCommentTextToText < ActiveRecord::Migration
def up
add_column :comments, :text_tmp, :text
# copy text column
execute <<-SQL
update comments set text_tmp = text
SQL
remove_column :comments, :text
rename_column :comments, :text_tmp, :text
end
def down
add_column :comments, :text_tmp, :string
# copy text column
execute <<-SQL
update comments set text_tmp = text
SQL
remove_column :comments, :text
rename_column :comments, :text_tmp, :text
end
end
And to test it:
rake db:migrate
rake db:rollback
rake db:migrate
I would say that if you can "undo" the imported data when rolling back the migration version, then it's appropriate to put imports into the migration.
For example, I have a migration which sets up a lot of lookup tables and other meta-data. The data for these tables are populated during this phase. As the data for these lookup tables changes, I create new YAML files storing the meta-data and load those files in subsequent migrations (and un-do those YAMLS, re-loading the previous YAML file when backing out of a migration version). This is pretty clean. I have files (in different well-defined folders in my case) with these files:
002_setup_meta_data.rb
002_meta_data.yaml
007_change_meta_data.rb
007_meta_data.yaml
If you're importing "production" data from another system into transactional (non-static) tables, then I would say using migrations is not appropriate. Then I would follow Brian Hogan's advice of using rake tasks.
I would like to confirm that the following analysis is correct:
I am building a web app in RoR. I have a data structure for my postgres db designed (around 70 tables; this design may need changes and additions during development to reflect Rails ways of doing things. EG, I designed some user and role tables - but if it makes sense to use Restful Authentication, I will scrub them and replace with whatever RA requires. ).
I have a shellscript which calls a series of .sql files to populate the empty database with tables and initial data (eg, Towns gets pre-filled with post towns) as well as test data (eg, Companies gets a few dummy companies so I have data to play with).
for example:
CREATE TABLE towns (
id integer PRIMARY KEY DEFAULT nextval ('towns_seq'),
county_id integer REFERENCES counties ON DELETE RESTRICT ON UPDATE CASCADE,
country_id integer REFERENCES countries ON DELETE RESTRICT ON UPDATE CASCADE NOT NULL,
name text NOT NULL UNIQUE
);
Proposition 0: Data lasts longer than apps, so I am convinced that I want referential integrity enforced at the DB level as well as validations in my RoR models, despite the lack of DRYNESS.
Proposition 1: If I replace the script and sql files with Migrations, it is currently impossible to tell my Postgres database about the Foreign Key and other constraints I currently set in SQL DDL files within the migration code.
Proposition 2: The touted benefit of migrations is that changes to the schema are versioned along with the RoR model code. But if I keep my scripts and .sql files in railsapp/db, I can version them just as easily.
Proposition 3: Given that migrations lack functionality I want, and provide benefits I can replicate, there is little reason for me to consider using them. So I should --skipmigrations at script/generate model time.
My question: If Proposition 0 is accepted, are Propositions 1,2,3 true or false, and why?
Thanks!
Proposition 1 is false in at least two situations - you can use plugins like foreign_key_migrations to do the following:
def self.up
create_table :users do |t|
t.column :department_id, :integer, :references => :departments
end
end
which creates the appropriate foreign key constraint in your DB.
Of course, you might have other things that you want to do in your DDL, in which case the second situation becomes more compelling: you're not forced to use the Ruby DSL in migrations. Try the execute method, instead:
def self.up
execute 'YOUR SQL HERE'
end
With that, you can keep the contents of your SQL scripts in migrations, gaining the benefits of the latter (most prominently the down methods, which you didn't address in your original question) and retaining the lower-level control you prefer.
Proposition 1 is mistaken : you can definitely define referential integrity using migrations if only by using direct SQL inside the migration, see this post for more details.
Proposition 2: The touted interest of migrations is to be able to define your database model incrementally while keeping track of what each change added and be able to easily rollback any such change at a later time.
You have to be careful with the order you create/modify things in but you can do it.
One thing to keep in mind : rails is better suited for application-centri design. in the Rails Way(tm) the database is only ever accessed through the application active record layer and exposes data to the outside using webservices
1: You may want to try out this plugin. I didn't try it myself though, but it seems to be able to add foreign key constraints through migrations.
2: The real benefit of migration is the ability to go back and forth in the history of your database. That's not as easy with your .sql files.
3: See if the above-mentioned plugin works for you, then decide :) At any rate, it's not a capital sin if you don't use them!
Since you are using Postgres and may not want to install the foreign_key_migrations plugin, here is what I do when I want to use both migrations and foreign key constraints.
I add a SchemaStatements method to ActiveRecord::SchemaStatements called "add_fk_constraint".
This could go in some centralized file, but in the example migration file below, I have just put it inline.
module ActiveRecord
module ConnectionAdapters # :nodoc:
module SchemaStatements
# Example call:
# add_fk_constraint 'orders','advertiser_id','advertisers','id'
# "If you want add/alter a 'orders' record, then its 'advertiser_id' had
# better point to an existing 'advertisers' record with corresponsding 'id'"
def add_fk_constraint(table_name, referencing_col, referenced_table, referenced_col)
fk_name = "#{table_name}_#{referencing_col}"
sql = <<-ENDSQL
ALTER TABLE #{table_name}
ADD CONSTRAINT #{fk_name}
FOREIGN KEY (#{referencing_col}) REFERENCES #{referenced_table} (#{referenced_col})
ON UPDATE NO ACTION ON DELETE CASCADE;
CREATE INDEX fki_#{fk_name} ON #{table_name}(#{referencing_col});
ENDSQL
execute sql
end
end
end
end
class AdvertisersOrders < ActiveRecord::Migration
def self.up
create_table :advertisers do |t|
t.column :name, :string, :null => false
t.column :net_id, :integer, :null => false
t.column :source_service_id, :integer, :null => false, :default => 1
t.column :source_id, :integer, :null => false
end
create_table :orders do |t|
t.column :name, :string, :null => false
t.column :advertiser_id, :integer, :null => false
t.column :source_id, :integer, :null => false
end
add_fk_constraint 'orders','advertiser_id','advertisers','id'
end
def self.down
drop_table :orders
drop_table :advertisers
end
end
I hopes this helps someone. It has been very useful to me since I need to load a lot of externally supplied data with SQL "COPY" calls, yet I find the migrations system very convenient.