The situation I have is that we have multiple schemas on SQL Server that we need to be able to do schema:dump and migrations against. One schema is for our new Rails application, the other schema is for a legacy system that we have dependencies on.
When running rake db:schema:dump our new schema tables are correctly created in the schema.rb file. The legacy schema tables do not end up in schema.rb. I'm wondering how others are dealing with this issue.
Another consideration I have given to this is since our legacy schema tables are fairly static would be to add these to a separate file once and then create a before hook for rake db:schema:load that would run that file prior to the schema.rb. Is there a before hook for rake db:schema:load; if so what is that?
Here is how I ended up solving this issue.
I added a before hooks into schema load and schema dump within hooks.rake as described below.
namespace :project do
namespace :db do
task :before_schema_load => :environment do
add_tables
end
task :before_schema_dump => :environment do
add_ignored_tables
end
end
end
Rake::Task['db:schema:dump'].enhance(['project:db:before_schema_dump'])
Rake::Task['db:schema:load'].enhance(['project:db:before_schema_load'])
Within the add_tables functionality I've manually created what is essentially a static schema.rb equivalent for my legacy tables since these will change infrequently (possibly never).
Within the add_ignored_tables I've added tables to the ActiveRecord::SchemaDumper.ignore_tables array to indicate tables that are outside of my schema that I don't want dumped to schema.rb. In my case this is everything that isn't under my current app's schema. In my situation everything that I want outside of my app's schema is specified within the add_tables so those tables as well should not end up in the schema.rb.
There is some material about multi-tenant databases using Postgres, the one I've referenced before is http://blog.jerodsanto.net/2011/07/building-multi-tenant-rails-apps-with-postgresql-schemas/. There is also a gem (https://github.com/influitive/apartment), which may also be inspiration, or even a solution for you.
Related
I have a question regarding my migrations in rails.
Normally when i want to add a colum to a model for i dont make extra migrations but instead i perform this steps:
rake db:rollback
next i change the migration file in db/migrations and rerune:
rake db:migrate
The biggest problem is that when i do this i loose my data.
Previous i wrote migrations from the command line with for example
rake g migration Add_Column_House_to_Users house:string
The problem with this approach is that my db/migrations folder afterwards get very large and not very clear! I mean at the end i dont know wich variables the object has! Im not an expert in rails and would like to ask you how to keep the overview over the migrations!Thanks
Just a minor thought - I just use the file db/migrate/schema.rb to determine whats in the database as opposed to tracking through the migrations
You definitely shouldn't use db:rollback with a table with existing data.
I have a few production RonR apps with a ton of data and there are 100+ entries in the migrations table and adding new migrations to tweak tables is the rails way to do things. Not sure what you mean by lucid, but your schema and data model are going to change over time and that is ok and expected.
One tip. The migrations are great, but they are just the beginning, you can include complex logic as needed to fix your existing data (like so)
Changing data in existing table:
def up
add_column :rsvps, :column_name_id, :integer
update_data
end
def update_data
rsvps = Rsvp.where("other_column is not null")
rsvps.each do |rsvp|
invite = Blah.find(rsvp.example_id)
...
rsvp.save
end
end
Another tip: backup your production database often (should do this anyway), but use it to test all of your migrations before deploying. I run scripts like this all the time for local testing:
mysql -u root -ppassword
drop database mydatabase_dev;
create database mydatabase_dev;
use mydatabase_dev;
source /var/www/bak/mydatabase_backup_2013-10-04-16.28.06.sql
exit
rake db:migrate
I have an application that requires a sequence to be present in the database. I have a migration that does the following:
class CreateSequence < ActiveRecord::Migration
def self.up
execute "CREATE SEQUENCE sequence"
end
def self.down
execute "DROP SEQUENCE sequence"
end
end
This does not modify the schema.rb and thus breaks rake db:setup. How can I force the schema to include the sequence?
Note: The sequence exists after running rake db:migrate.
Rails Migrations because they aim toward a schema of tables and fields, instead of a complete database representation including stored procedures, functions, seed data.
When you run rake db:setup, this will create the db, load the schema and then load the seed data.
A few solutions for you to consider:
Choice 1: create your own rake task that does these migrations independent of the Rails Migration up/down. Rails Migrations are just normal classes, and you can make use of them however you like. For example:
rake db:create_sequence
Choice 2: run your specific migration after you load the schema like this:
rake db:setup
rake db:migrate:up VERSION=20080906120000
Choice 3: create your sequence as seed data, because it's essentially providing data (rather than altering the schema).
db/seeds.rb
Choice 4 and my personal preference: run the migrations up to a known good point, including your sequence, and save that blank database. Change rake db:setup to clone that blank database. This is a bit trickier and it sacrifices some capabilities - having all migrations be reversible, having migrations work on top of multiple database vendors, etc. In my experience these are fine tradeoffs. For example:
rake db:fresh #=> clones the blank database, which you store in version control
All the above suggestions are good. however, I think I found a better solution. basically in your development.rb put
config.active_record.schema_format = :sql
For more info see my answer to this issue -
rake test not copying development postgres db with sequences
Check out the pg_sequencer gem. It manages Pg sequences for you as you wish. The one flaw that I can see right now is that it doesn't play nicely with db/schema.rb -- Rails will generate a CREATE SEQUENCE for your tables with a serial field, and pg_sequencer will also generate a sequence itself. (Working to fix that.)
You'll have to admit, to a newbie to rails and databases, the official explanation on rubyonrails.org makes all four of these tasks sound exactly the same. Quote:
rake db:test:clone Recreate the test database from
the current environment’s database schema
rake db:test:clone_structure Recreate the test database from the
development structure
rake db:test:load Recreate the test database from the current schema.rb
rake db:test:prepare Check for pending migrations and load the test schema
I don't even know the difference between structure and schema. And what's the difference between loading the current environment's schema and just loading schema.rb?
Just how similar (or different) are these tasks?
Very good question. Had me stumped so I dove into the rails source and pulled up database.rake. Now it's more clear:
db:test:clone is just a combination of db:schema:dump and db:test:load:
task :clone => %w(db:schema:dump db:test:load)
db:test:clone_structure uses the {rails_env}_structure.sql file:
task :clone_structure => [ 'db:structure:dump', 'db:test:purge' ] do
# skipped some code, here's what happens for MySQL:
ActiveRecord::Base.establish_connection(:test)
# ...
IO.readlines("#{Rails.root}/db/#{Rails.env}_structure.sql").join.split("\n\n").each do |table|
ActiveRecord::Base.connection.execute(table)
end
end
db:test:load is the same as db:schema:load, but invokes it on the test database:
task :load => 'db:test:purge' do
ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
# ...
db_namespace['schema:load'].invoke
end
db:test:prepare alerts you if any migrations are pending, and if not, either runs db:test:clone_structure (using the {rails_env}_structure.sql file) or db:test:load (using the schema.rb file), depending on the schema format (this is a little confusing to me, maybe someone else can expand on it):
task :prepare => 'db:abort_if_pending_migrations' do
# ...
db_namespace[{ :sql => 'test:clone_structure', :ruby => 'test:load' }[ActiveRecord::Base.schema_format]].invoke
end
Hope this clears it up! Again, going through the database.rake file is easy and will clear up any other questions you might have. That link goes to the line that is the beginning of the :test namespace.
They are actually not quite the same thing. Any of those tasks that contain the word 'schema' act on the .../db/schema.rb file. schema.rb is effectively the state of your schema after applying all migrations. It can be executed to restore your schema rather than running all of the db migrations (which can take a long time if you have lots of migrations).
Any of the tasks with the word 'structure', act on the {Rails.env}_structure.sql file. This file is used when your schema contains constructs that can't be expressed in the schema.rb file. For example, if you use features specific to a particular RDBMS. Under the covers, rails produces this file using whatever schema dump utility it appropriate for your RDBMS. To restore the schema, it reads the file in and executes the SQL statements agains using an RDBMS-specific tool.
Rails knows whether to go the schema.rb route or the structure.sql route based on whether or not you've set
config.active_record.schema_format = :sql
in your .../config/application.rb
I have a migration in Rails that inserts a record into the database. The Category model depends on this record. Since RSpec clears the database before each example, this record is lost and furthermore never seems to be created since RSpec does not seem to generate the database from migrations. What is the best way to create/recreate this record in the database? Would it be using before(:all)?
It's not that RSpec clears the database, it's that Rails's rake:db:prepare task copies the schema (but not the contents) of your dev database into your *_test db.
Yes, you can use before(:all), as transactions are wrapped around each individual example - but a simple fixture file would also do the same job.
(There's a more complicated general solution to this issue: moving to a service-oriented architecture, where your 'dev' and 'test' services are going to be completely separate instances. You can then point your test db config to the development database in your test service, disable rake:db:prepare, and build your test service from migrations as you regenerate it. Then you can test your migrations and data transformations.)
What I like to do is create a folder in db/migration called data, and then put yml fixtures in there, in your case categories.yml
Then I create a migration with the following
def self.up
down
directory = File.join( File.dirname(__FILE__), "data" )
Fixtures.create_fixtures( directory, "categories" )
end
def self.down
Category.delete_all
end
I have a migration that runs an SQL script to create a new Postgres schema. When creating a new database in Postgres by default it creates a schema called 'public', which is the main schema we use. The migration to create the new database schema seems to be working fine, however the problem occurs after the migration has run, when rails tries to update the 'schema_info' table that it relies on it says that it does not exist, as if it is looking for it in the new database schema and not the default 'public' schema where the table actually is.
Does anybody know how I can tell rails to look at the 'public' schema for this table?
Example of SQL being executed: ~
CREATE SCHEMA new_schema;
COMMENT ON SCHEMA new_schema IS 'this is the new Postgres database schema to sit along side the "public" schema';
-- various tables, triggers and functions created in new_schema
Error being thrown: ~
RuntimeError: ERROR C42P01 Mrelation "schema_info" does not exist
L221 RRangeVarGetRelid: UPDATE schema_info SET version = ??
Thanks for your help
Chris Knight
Well that depends what your migration looks like, what your database.yml looks like and what exactly you are trying to attempt. Anyway more information is needed change the names if you have to and post an example database.yml and the migration. does the migration change the search_path for the adapter for example ?
But know that in general rails and postgresql schemas don't work well together (yet?).
There are a few places which have problems. Try and build and app that uses only one pg database with 2 non-default schemas one for dev and one for test and tell me about it. (from thefollowing I can already tell you that you will get burned)
Maybe it was fixed since the last time I played with it but when I see http://rails.lighthouseapp.com/projects/8994/tickets/390-postgres-adapter-quotes-table-name-breaks-when-non-default-schema-is-used or this http://rails.lighthouseapp.com/projects/8994/tickets/918-postgresql-tables-not-generating-correct-schema-list or this in postgresql_adapter.rb
# Drops a PostgreSQL database
#
# Example:
# drop_database 'matt_development'
def drop_database(name) #:nodoc:
execute "DROP DATABASE IF EXISTS #{name}"
end
(yes this is wrong if you use the same database with different schemas for both dev and test, this would drop both databases each time you run the unit tests !)
I actually started writing patches. the first one was for the indexes methods in the adapter which didn't care about the search_path ending up with duplicated indexes in some conditions, then I started getting hurt by the rest and ended up abandonning the idea of using schemas: I wanted to get my app done and I didn't have the extra time needed to fix the problems I had using schemas.
I'm not sure I understand what you're asking exactly, but, rake will be expecting to update the version of the Rails schema into the schema_info table. Check your database.yml config file, this is where rake will be looking to find the table to update.
Is it a possibility that you are migrating to a new Postgres schema and rake is still pointing to the old one? I'm not sure then that a standard Rails migration is what you need. It might be best to create your own rake task instead.
Edit: If you're referencing two different databases or Postgres schemas, Rails doesn't support this in standard migrations. Rails assumes one database, so migrations from one database to another is usually not possible. When you run "rake db:migrate" it actually looks at the RAILS_ENV environment variable to find the correct entry in database.yml. If rake starts the migration looking at the "development" environment and database config from database.yml, it will expect to update to this environment at the end of the migration.
So, you'll probably need to do this from outside the Rails stack as you can't reference two databases at the same time within Rails. There are attempts at plugins to allow this, but they're majorly hacky and don't work properly.
You can use pg_power. It provides additional DSL for migration to create PostgreSQL schemas and not only.