Rails separate DB migration not working - ruby-on-rails

I am trying to run a migration to a separate DB in Rails, using establish_connection. I have something like this in the migration file:
def change
ActiveRecord::Base.establish_connection("user_#{Rails.env}")
ActiveRecord::Base.initialize_schema_migrations_table
create_table "users", :force => true do |t|
t.string "email",
end
In the yml I have this config:
user_development:
adapter: postgresql
encoding: unicode
database: MyApp_user_development
pool: 5
host: localhost
username: xxx
password: xxx
Also I have defined the User Model like this:
class User < ActiveRecord::Base
establish_connection "user_#{Rails.env}"
[...]
end
Now, the DB exists but when running rake db:migrate I get this error:
== CreateUserOnSeparateDb: migrating =========================================
rake aborted!
An error has occurred, this and all later migrations canceled:
connection is closed
Does any of you have any idea what is going on?
I have also tried to move the establish_connection call into a connection method in the migration file. In this case the table is created on migrate but then I get the same error (so somehow failing on other migrations) and by the way it does not create the schema_migrations table...
Any help?

Resolved above issue by following below steps.
class AddCustomTbl < ActiveRecord::Migration
def connection
ActiveRecord::Base.establish_connection(Rails.env).connection
end
def up
# Connection with user_#{Rails.env}
oldEnv = Rails.env
Rails.env = "user_#{Rails.env}"
ActiveRecord::Base.establish_connection(Rails.env)
create_table "users", :force => true do |t|
t.string "email"
end
# Connection with Rails.env
Rails.env = oldEnv
ActiveRecord::Base.establish_connection ActiveRecord::Base.configurations[Rails.env]
end
def down
# Connection with user_#{Rails.env}
oldEnv = Rails.env
Rails.env = "user_#{Rails.env}"
ActiveRecord::Base.establish_connection(Rails.env).connection
drop_table :users
# Connection with Rails.env
Rails.env = oldEnv
ActiveRecord::Base.establish_connection ActiveRecord::Base.configurations[Rails.env]
end
end

Related

Rails migration stopped, even though table does not exist

Rails 6
environment: local development on Mac OS
DB server: MySQL
I deleted all the tables from the DB, and the only tables left in the DB are:
schema_migrations
ar_internal_metadata
I made sure that schema_migrations has no data in it and looked into ar_internal_metadata, and that table has a single row in it, with the following values:
key: environment, value: development
I have several migrations, the most recent one, is devise_create_users.rb.
I am trying to run:
rake db:migrate
But I am getting the error message:
=> rake db:migrate
== 20200317184535 DeviseCreateUsers: migrating ================================
-- create_table(:users)
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Mysql2::Error: Table 'users' already exists
/Users/dev/rails/myapp/db/migrate/20200317184535_devise_create_users.rb:5:in `change'
Caused by:
ActiveRecord::StatementInvalid: Mysql2::Error: Table 'users' already exists
/Users/dev/rails/myapp/db/migrate/20200317184535_devise_create_users.rb:5:in `change'
Caused by:
Mysql2::Error: Table 'users' already exists
/Users/dev/rails/myapp/db/migrate/20200317184535_devise_create_users.rb:5:in `change'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
Process finished with exit code 1
class DeviseCreateUsers < ActiveRecord::Migration[6.0]
def change
create_table :users do |t|
t.string :email, null: false, default: ""
t.string :encrypted_password, null: false, default: ""
t.string :reset_password_token
t.datetime :reset_password_sent_at
t.datetime :remember_created_at
t.integer :sign_in_count, default: 0, null: false
t.datetime :current_sign_in_at
t.datetime :last_sign_in_at
t.string :current_sign_in_ip
t.string :last_sign_in_ip
t.timestamps null: false
end
add_index :users, :email, unique: true
add_index :users, :reset_password_token, unique: true
# add_index :users, :confirmation_token, unique: true
# add_index :users, :unlock_token, unique: true
end
end
When I check the DB after this, I still don't see a users table, and the schema_migrations table is still empty. In addition, the DeviseCreateUsers migration is the most recent on, so why is it running first.
Any ideas?
Edit:
Based on a comment to the question, I looked at my database.yml file:
default: &default
host: localhost
database: <%= ENV['RAILS_DB_NAME'] %>
username: <%= ENV['RAILS_DB_USER'] %>
password: <%= ENV['RAILS_DB_PWD'] %>
adapter: mysql2
encoding: utf8mb4
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
socket: /tmp/mysql.sock
development:
<<: *default
I made that change last night, and I forgot that my local ENV settings, were for a different project, so Rails was picking up the settings for that project, and was indeed correct, in that the users table is already there. The fix was for me to create project specific ENV settings, for my local development environment
When things like this happen it is needed to check the database.yml. As in most cases it happens because of some faulty configuration.
Good thing that just mentioning the database.yml in the comment section of the question helped to find the problem.
Using rails console to make sure there is not any users table, try create a new instance of User User.new. If the object was instanciated, your User table is hidden in somewhere. It is not a problem, see all atributtes previously generated and just migrate new ones. I can see in your devise_create_user migration that you can just replace it with something like addColumnstoUser.rb. This will just add the apropriated columns necessary to let Devise runing well.
Choosing this way, inside your migration:
add_column :table_name, :fieldname, :type, in your case, using just one column as an example, it must be:
def change
add_column :users, :reset_password_token, :string
end.
If you are using version controll, such as git, the migration might not know where you can rollback or go ahead. All the migration and rollbacks must stay on the same branch.
For example:
1) Adding a new branch to handle user migration: git checkout -b generate-user-model, generating user model rails generate model User name:string email:string
and running the migration rails db:migrate will create the users table and will update scheema.rb (check scheema file). Adding it to version controll git add . and `git commit -m "Migrating user model"
2) Moving to a new branch git checkout -b comments-controller, generating a new controller rails generate model Comment. And then migrating it rails db:migrate.
This second migration only teachs scheema.rb how to go ahead and how to rollback on this specifc migration. This last git branch doesn't know nothing about how to rollback the User model, unless you merge a branch inside another git merge generate-user-model.
After many migrations has been used, it's usefull keep using migrations to add or delete tables.
Hence, If I had a table called users and I wanted to set it to handle Devise, I just need to generate a new migration with the columns I'd needed.
Use https://sqlitebrowser.org/ to help you cheking your database tables and columns.

Activerecord Migrations multiple postgresql database

I have multiple databases. All the migrations for the main database human_development runs fine. The second database is called animals and the migrations fail.
database.yml:
default: &default
adapter: postgresql
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000
encoding: unicode
user: blah
development:
<<: *default
database: human_development
animals:
<<: *default
database: animals
Migrations that are failing:
class SomeTable < ActiveRecord::Migration[5.2]
def change
ActiveRecord::Base.establish_connection("animals")
create_table :some_table, id: :uuid do |t|
t.string :type
t.timestamps
end
ActiveRecord::Base.establish_connection(Rails.env)
end
end
I have also tried the following, non worked:
def connection
ActiveRecord::Base.establish_connection("animals").connect
#ActiveRecord::Base.establish_connection("animals".to_sym).connect
end
establish connection out side of the change
ActiveRecord::Base.establish_connection("animals").connect
# also tried with to_sym
If I run "rails db:migrate", passing the database name as a string I get the following error:
rake aborted!
ActiveRecord::AdapterNotSpecified: database configuration does not specify adapter
.../db/migrate/2019_some_tables.rb:2:in
and if I run the rails db:migrate with to_sym I get the following error:
-- create_table(:some_table, {:id=>:uuid})
rake aborted!
ActiveRecord::StatementInvalid: PG::ConnectionBad: connection is closed: SELECT pg_advisory_unlock
Caused by:
PG::ConnectionBad: connection is closed
Caused by:
StandardError: An error has occurred, this and all later migrations canceled:
PG::ConnectionBad: connection is closed: ROLLBACK
Caused by:
ActiveRecord::StatementInvalid: PG::ConnectionBad: connection is closed: ROLLBACK
Caused by:
PG::ConnectionBad: connection is closed
The migration file should be the same for all databases
class SomeTable < ActiveRecord::Migration[5.2]
def change
create_table :some_table, id: :uuid do |t|
t.string :type
t.timestamps
end
end
end
then in your console you run
# for the main db
rails db:migrate
# for the animals db
RAILS_ENV=animals rails db:migrate
#Eyeslandic Thank you so much, that helped a lot!
In case anyone else runs into the same problems. Looks like rails 6 is going to have a better solution for this, but meanwhile, here are additional changes I had to make:
database.yml: I wanted to keep the migrations in a separate folder:
├── db
│   ├── migrate
│   ├── schema.rb
│   └── seeds.rb
├── animals_db
│   └── migrate
animals:
<<: *default
database: animals
migrations_paths: animals_db/migrate
Create a new environment: config/environments/animals.rb
Change config/secrets.yml to include new environment
Generate secret: RAILS_ENV=animals rake secret
Save the secret in .env file or export it
Create the database: rails db:create RAILS_ENV=animals
Create the migration in animals_db but here is the catch. I had to include the following in def change
def change
create_table :some_table, id: :uuid do |t|
enable_extension "uuid-ossp"
enable_extension "pgcrypto"
t.string :type
t.timestamps
end
end
Run the migration: rails db:migrate RAILS_ENV=animals
You may need to change config/cable.yml as well to include new environment
Hope this helps. Looking forward to see how rails 6 is going to improve this

Sinatra and ActiveRecord (no Rails) - Creating test database

I'm trying out a simpler setup for using Sinatra with ActiveRecord and I am running into some puzzling problems and would love another set of eyes on it. I will give all the relevant files:
database.yml:
development:
adapter: sqlite3
database: db/development.sqlite3
test:
adapter: sqlite3
database: db/test.sqlite3
production:
url: <%= ENV['DATABASE_URL'] %>
Rakefile:
require_relative "demo_app"
require 'sinatra/activerecord/rake'
require 'rake/testtask'
Rake::TestTask.new do |t|
t.pattern = "test/*_test.rb"
end
test_helper.rb:
ENV['RACK_ENV'] = 'test'
ENV["SINATRA_ENV"] = "test"
require_relative '../demo_app'
require 'minitest/autorun'
require 'rack/test'
ActiveRecord::Migration.maintain_test_schema!
Migration file:
class CreatePeople < ActiveRecord::Migration
def change
create_table :people do |t|
t.string :name
t.date :dob
t.string :gender
t.string :gender
t.integer :zipcode
end
end
end
Now, I want to run tests against the test database. When I try to do
rake db:create RAILS_ENV=test - it creates a development.sqlite3
rake db:migrate RAILS_ENV=test - same + runs migration
I know that I am using ActiveRecord outside of Rails so I suspect the problem lies somewhere relating to the processing of database.yml or some environment variable or something. But I can't figure it out.
Thoughts?
Try RACK_ENV instead of RAILS_ENV:
$ rake db:create RACK_ENV=test
Or:
$ RACK_ENV=test rake db:create

Debugging ActiveRecord Migrations based on delayed_job

I have been trying to run a migration based on the following database.yml
database.yml:
development:
adapter: postgresql
encoding: utf8
database: test_development
timeout: 5000
queue:
adapter: postgresql
encoding: utf8
database: test_queue_development
timeout: 5000
What I want to do is to create a table "delayed_jobs" in test_queue_development database.
Unfortuanately, the table gets created in test_development. The initializer for delayed_job.rb is the following.
config/initiallizers/delayed_job.rb:
db_queue_config = ActiveRecord::Base.configurations[Rails.env]['queue']
if db_queue_config
Delayed::Backend::ActiveRecord::Job.establish_connection(db_queue_config)
end
I am running the migration only in development
The migration should respect Delayed::Backend::ActiveRecord::Job.connection and create
the table in test_queue_development but it always creates the table in test_development.
db/migrate/create_delayed_jobs.rb
class CreateDelayedJobs < ActiveRecord::Migration
def self.connection
Delayed::Backend::ActiveRecord::Job.connection
end
def self.up
create_table :delayed_jobs do |table|
# Allows some jobs to jump to the front of the queue
table.integer :priority, :default => 0
# Provides for retries, but still fail eventually.
table.integer :attempts, :default => 0
# YAML-encoded string of the object that will do work
table.text :handler, :limit => 500.kilobytes
# reason for last failure (See Note below)
table.text :last_error
# The queue that this job is in
table.string :queue, :default => nil
# When to run.
# Could be Time.zone.now for immediately, or sometime in the future.
table.datetime :run_at
# Set when a client is working on this object
table.datetime :locked_at
# Set when all retries have failed
table.datetime :failed_at
# Who is working on this object (if locked)
table.string :locked_by
table.timestamps
end
add_index :delayed_jobs, [:priority, :run_at], :name => 'delayed_jobs_priority'
add_index :delayed_jobs, [:queue], :name => 'delayed_jobs_queue'
end
My question is: How can I introduce extra debugging into the migration so I can see what's going wrong? If i get into the rails console and type:
Delayed::Backend::ActiveRecord::Job.connection
i get info related to the queue database, which means self.connection runs fine and the initializer worked as expected. The code samples were take from canvas-lms
Thank you.

Can't Rails 3 run in MySQL MyISAM mode, without InnoDB?

I have a MySQL server running with InnoDB disabled (for reasons of performance), with this setup I can't seem to be able to use Rails 3 (with the mysql2 adapter).
Here's my test migration:
class CreateTxts < ActiveRecord::Migration
def change
create_table(:txts, :options => 'ENGINE=MyISAM') do |t|
t.timestamps
end
end
end
And here's the error:
>rake db:migrate
rake aborted!
Mysql2::Error: Unknown storage engine 'InnoDB': CREATE TABLE `schema_migrations`
(`version` varchar(255) NOT NULL) ENGINE=InnoDB
Tried the workaround described here, but it doesn't seem to work either (I did modify MysqlAdapter to Mysql2Adapter to match my setup).
Sorry I'm a newb in Rails. Any help will be much appreciated :o
Going to answer my own question. Here's a patch for environment.rb I ended up with that works with the native mysql driver as well as JRuby/JDBC-mysql:
# Load the rails application
require File.expand_path('../application', __FILE__)
# Patch Mysql adapter to default to MyISAM instead of InnoDB
require 'active_record/connection_adapters/mysql_adapter'
module ActiveRecord
module ConnectionAdapters
class MysqlAdapter
def create_table(table_name, options = {}) #:nodoc:
super(table_name, options.reverse_merge(:options => "ENGINE=MyISAM"))
end
end
end
end
# Initialize the rails application
.....
rake db:migrate now succeeds and creates all tables including schema_migrations with TYPE=MyISAM.
Note: For mysql2 adapter, rename mysql_adapter to mysql2_adapter and MysqlAdapter to Mysql2Adapter.
Try creating the table without specifying the type of engine being used like this
class CreateTxts < ActiveRecord::Migration
def change
create_table(:txts) do |t|
t.timestamps
end
end
end
and then in mysql cli, type this
ALTER TABLE txts ENGINE = MYISAM
hope it helps
The error is coming from creating the schema_migrations table (which rails uses to track which migrations have been run) rather than your table. You could create that table yourself (with a single varchar(255) column called version with an index on it).
If you do end up overwriting the create_table method, you need to preserve the method's signature - you're ignoring the block that it yields. I'd try something like
def create_table(name, options={})
super(name, options.merge(...)) {|t| yield t}
end
#rustyx nice one! &here's my slightly tweaked monkey-patch so the engine can be set from within the /config/database.yml configuration:
&Rather than in /config/environment.rb, i put the codes into an initializer, Eg: /config/initializers/mysql2adapter_default_engine.rb.
Either this for brevity:
require 'active_record/connection_adapters/abstract_mysql_adapter'
class ActiveRecord::ConnectionAdapters::Mysql2Adapter < ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter
def create_table(table_name, options = {}) #:nodoc:
super(table_name, #config[:engine].nil? ? options : options.reverse_merge(:options => "ENGINE="+#config[:engine]))
end
end
or this for more clarity:
require 'active_record/connection_adapters/abstract_mysql_adapter'
module ActiveRecord
module ConnectionAdapters
class Mysql2Adapter < AbstractMysqlAdapter
def create_table(table_name, options = {}) #:nodoc:
super(table_name, #config[:engine].nil? ? options : options.reverse_merge(:options => "ENGINE="+#config[:engine]))
end
end
end
end
Then in /config/database.yml we can have something like this:
production:
adapter: mysql2
encoding: utf8
database: ooook
username: ooook
password: ooook
host: ooook.tld
port: 3306
pool: 5
timeout: 5000
engine: MyISAM
If no engine is specified then the mysql2 adapter will use the default (InnoDB or whatever).

Resources