I am trying to update a gem called textacular to be compatible with latest rails version.
In the development Rake-tasks we need to run a few ActiveRecord migrations - without Rails.
Today it looks like:
namespace :migrate do
desc 'Run the test database migrations'
task :up => :'db:connect' do
ActiveRecord::Migrator.up 'db/migrate'
end
desc 'Reverse the test database migrations'
task :down => :'db:connect' do
ActiveRecord::Migrator.down 'db/migrate'
end
end
However when using ActiveRecord >= 5, it fails with:
NoMethodError: undefined method 'up' for ActiveRecord::Migrator:Class.
I have tried to look through the source code for ActiveRecord, tried a bunch of different methods but have not managed to run the migrations.
Does anyone have a hint of what to do?
Edit
Using ActiveRecord::Migration.up does nothing, probably just returns based on the method.
Using ActiveRecord::Migration.migrate(:up) gives output:
== ActiveRecord::Migration: migrating ========================================
== ActiveRecord::Migration: migrated (0.0000s) ===============================
All migrations are in the folder db/migrate.
You can also use this command:
ActiveRecord::MigrationContext.new(Rails.root.join('db', 'migrate'), ActiveRecord::SchemaMigration).migrate
I managed to solve this (until Rails 6 is release at least), the implementation can be seen on textacular
task :up => :'db:connect' do
migrations = if ActiveRecord.version.version >= '5.2'
ActiveRecord::Migration.new.migration_context.migrations
else
ActiveRecord::Migrator.migrations('db/migrate')
end
ActiveRecord::Migrator.new(:up, migrations, nil).migrate
end
Not very satisfied with this solution, but this was the only way I could get the migrations to run on Rails 5, 5.1 and 5.2.
By looking at the rails github page then starting from Rails 5.2 the Migrator class is cut down quite a bit. Much of its methods have moved to Migration class instead (including the .up method).
Therefore just replace ActiveRecord::Migrator with ActiveRecord::Migration.
Related
I am upgrading an app from 6.0 to 6.1, using the next_rails dual boot gem. As part of the upgrade I needed to execute rails active_storage:update which adds a service_name to the ActiveStorage::Blob table. When I run my tests with next rspec spec this works fine, but rspec spec fails with tests that use active_storage with messages like
ActiveRecord::NotNullViolation:
PG::NotNullViolation: ERROR: null value in column "service_name" violates not-null constraint
Is there a way of adding some conditional code (if Rails::VERSION::MINOR) that detects the version is 6.0 and somehow removes the null constraint for the service_name column.
There are two approaches.
Migrate down to the version before the update:
if Rails.version == "6.0.0"
ActiveRecord::Base.connection.migration_context.migrate(20220816031332)
else
ActiveRecord::Base.connection.migration_context.migrate
end
# or maybe invoke the db:migrate tasks instead
You might need to turn off pending migration check: config.active_record.migration_error = false
This approach is general and should work with any migration(s).
The second approach is to call migration helpers manually. This is specific to the problem posed in this question and just changes the columns without updating migration version in the database:
if Rails.version == "6.0.0"
ActiveRecord::Migration.change_column_null(:active_storage_blobs, :service_name, true)
else
ActiveRecord::Migration.change_column_null(:active_storage_blobs, :service_name, false)
end
The code can be placed in config/environment.rb after Rails.application.initialize!. If you wish to place it earlier, you would need to establish the connection to the database manually.
Note that both methods cause persistent changes to the database, e.g. will remain after the tests have run. The changes only affect the database of which ever environment the code is run in e.g. test or development.
Work is transitioning from Rails 3 to Rails 4. Everything seems to be running more or less smoothly on the development side, but testing results in a multitude of varying failures when all suites - units, functionals, and integration - are run in one go with rake:test.
What's interesting is that running each of these suites individually produces no failures. This strongly suggests that the database is not resetting for our tests in Rails 4 quite the way it was in Rails 3.
I've tried overriding the rake:test task to execute db:test:prepare prior to running each suite, but this apparently doesn't do what I think it does, or rather, what I want it to do, which is to work with a fresh set of data for each test and therefore succeed or fail independently of every other test - the way it (was or should have been) in our Rails 3 setup.
Why is this happening? How might I fix it?
(Note that the testing framework is vanilla Rails with fixtures and the like. The testing failures we're getting are usually foreign key errors or failures for data to change in expected ways that don't show up when the DB is reset before a testing suite.)
EDIT:
Here are the changes I'm making to my Rakefile. The idea is to get rake:test to perform as it would in Rails 3, with the exception of properly resetting the database between unit, functional, and integration suites.
# /Rakefile
task :test => [];
Rake::Task[:test].clear
task :test do
puts Rails.env
if Rails.env == "test"
puts "units"
# Rake::Task["db:drop"].execute - results in 'test database not configured' error
# Rake::Task["db:reset"].execute - results in 'relation 'table' does not exist' error
Rake::Task["db:create"].execute
Rake::Task["db:migrate"].execute
Rake::Task["test:units"].execute
puts "functionals"
Rake::Task["db:schema:load"].execute
Rake::Task["test:functionals"].execute
puts "integration"
Rake::Task["db:schema:load"].execute
Rake::Task["test:integration"].execute
end
end
EDIT 2:
Rails version 4.1.8
Here is my test_helper. I've omitted helper methods that don't work with our data since I'm probably not allowed to share them and they're irrelevant to how the testing database is loaded.
Note also that we have a custom schema.rb from which the testing data are generated, as well, since trying to create them from migrations does not support some of the custom functionality/behavior we want in the end product, like materialized views. Redacted stuff will be enclosed in <<>>.
#test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
include ActionDispatch::TestProcess
class ActiveSupport::TestCase
# Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
set_fixture_class <<cached_object>>: ResultsDb::<<DesiredObjectClass>>Cache
fixtures :all
# Add more helper methods to be used by all tests here...
def reload_contig(contig_sym)
contig = contigs(contig_sym)
src = contig.src
new_tig = Contig.new(
org: contig.org,
version: contig.version,
size: contig.size,
sha1: contig.sha1,
active: contig.active,
src_id: src.id,
src_type: contig.src_type)
new_tig.id = contig.id
contig.destroy
unless new_tig.save
raise new_tig.errors.full_messages.inspect
end
src.contigs << new_tig
src.save
new_tig
end
def reload_coord(coord_sym)
coord = coords(coord_sym)
new_coord = Coord.new(
:mapped_id => coord.mapped_id,
:mapped_type => coord.mapped_type,
:contig_id => coord.contig_id,
:contig_type => coord.contig_type,
:start => coord.start,
:stop => coord.stop,
:fwd_strand => coord.fwd_strand
)
new_coord.id = coord.id
coord.destroy
new_coord.save!
end
end
You might have a look at the DatabaseCleaner gem that is specifically designed to assist you with database consistency and cleanliness during testing in Rails.
I've only run into problems when my code bypasses ActiveRecord and manipulates the data directly. In all other cases this gem has worked very well for me for testing.
Assuming you are currently on Rails 4.0, your current approach is about the best you can do if you can't add the database_cleaner gem. (I would suggest that you prefer db:schema:load to db:migrate throughout—there's no reason ever to run through all the migrations from the start in the test environment, unless you've done something unusual with them.) But once you get to Rails 4.1 or higher, you've got a better solution available.
Starting with Rails 4.1, management of the test database schema was automated, and rake db:test:prepare was deprecated. If you regenerate your test_helper.rb at this point, everything should work magically.
Have you tried adding either of these in your overriding of rake:test?
rake db:reset db:migrate
or
rake db:drop db:create db:migrate
The question seems similar to this one. I know I've always built my own rake db:recreate that does this sort of thing for test data.
Where and how do I run a simple script that uses my rails environment. Specifically I have one column that holds multiple pieces of information, I've added columns now for each piece of information and need to run a ruby script that can run to call a method on each row of the database to extrapolate data and save it to the new column.
Using a migration sounds like the right way to go if I am understanding your use case.
However, if you really do want to write a standalone script that needs access to your Rails application's models, you can require the environment.rb file from inside your standalone script.
Example:
#!/bin/env ruby
ENV['RAILS_ENV'] = "production" # Set to your desired Rails environment name
require '/path/to/railsapp/config/environment.rb'
# After this point you have access to your models and other classes from your Rails application
model_instance = MyModel.find(7)
model_instance.some_attribute = "new value"
model_instance.save
I have to agree with David here. Use a migration for this. I'm not sure what you want to do, but running it from inside your environment is much, much more efficient then loading up the app environment manually. And since your initial post suggests you're only doing this once, a migration is the way to go:
rails g migration MigrateData
.. generates:
class MigrateData < ActiveRecord::Migration
def self.up
# Your migration code here
end
def self.down
# Rollback scenario
end
end
Of course, you will always want to perform this locally first, using some test data.
Agree with everyone, for this specific case it sounds like migration will be way to go, however, to do this regularly, or write some other task/script that interacts rails app environment make rails generate a rake task for you! This gets saved with your rails app, and can be run again and again :)
Easiest way to generate a rake task that interact with rails app/models is to make Rails generate Rake tasks for you!! :)
Here's an example
run rails g task my_namespace my_task
This will generate a file called lib/tasks/my_namespace.rake which looks like:
namespace :my_namespace do
desc "TODO: Describe your task here"
task :my_task1 => :environment do
#write any ruby code here and also work with your models
puts User.find(1).name
end
end
Run this task with rake my_namespace:my_task
Watch your ruby code task that interacts with rails modal run!
Seeding data:
http://railscasts.com/episodes/179-seed-data
Adding data with migrations
http://railscasts.com/episodes/23-counter-cache-column
Working with Rake Tasks
http://railscasts.com/episodes/66-custom-rake-tasks
I prefer to use migrations for adding some data in your case.
If it's a one-time thing, use a migration.
If this is something that needs to be done multiple times, use a rake task for it.
In order to override the table_exists? method in the Rails PostgreSQL adapter I have tried the following in an initializer file:
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.class_eval do
def table_exists?(name)
raise 'got here'
end
end
This will raise the the following error:
uninitialized constant ActiveRecord::ConnectionAdapters::PostgreSQLAdapter
I believe this would have worked in previous versions of rails and I even found a small plugin that did something like this in Rails 2.3.6. Also I only encounter this error when I am trying to run a rake task like db:migrate and not when I start my application server.
Could someone show me the right way to do this and / or explain why PostgreSQLAdapter doesn't seem to be loaded when I am in an initializer file?
Instead of config/initializers, place that code in lib/ folder.
While this means that the active_record is loaded after the rails initializers, which is unusual. I ll update this with more detail, once I am done investigating the whole flow. If you want some more details about the rails 3 initialization process, check out this link:
http://ryanbigg.com/guides/initialization.html
I had success by moving this code into a Rails plugin. It is a little bit more overhead, but it is working consistently when I run rails s and when I run rake db:migrate.
I just followed the rails guide page on the topic and ran
rails generate plugin rails_patches --with-generator
and moved my init.rb file into rails as recommended.
~vendor/
`~plugins/
`~rails_patches/
|~lib/
| `-rails_patches.rb
|~rails/
| `-init.rb
|+test/
|-install.rb
|-MIT-LICENSE
|-Rakefile
|-README
`-uninstall.rb
I put this code in init.rb:
require 'rails_patches'
I put this code in rails_patches.rb:
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.class_eval do
def table_exists?(name)
raise 'got here'
end
end
This now behaves as I expected.
Experienced with Rails / ActiveRecord 2.1.1
You create a first version with (for example) ruby script\generate scaffold product title:string description:text image_url:string
This create (for example) a migration file called 20080910122415_create_products.rb
You apply the migration with rake db:migrate
Now, you add a field to the product table with ruby script\generate migration add_price_to_product price:decimal
This create a migration file called 20080910125745_add_price_to_product.rb
If you try to run rake db:migrate, it will actually revert the first migration, not apply the next one! So your product table will get destroyed!
But if you ran rake alone, it would have told you that one migration was pending
Pls note that applying rake db:migrate (once the table has been destroyed) will apply all migrations in order.
The only workaround I found is to specify the version of the new migration as in:
rake db:migrate version=20080910125745
So I'm wondering: is this an expected new behavior?
You should be able to use
rake db:migrate:up
to force it to go forward, but then you risk missing interleaved migrations from other people on your team
if you run
rake db:migrate
twice, it will reapply all your migrations.
I encounter the same behavior on windows with SQLite, it might be a bug specific to such an environment.
Edit -- I found why. In the railstie database.rake task you have the following code :
desc "Migrate the database through scripts in db/migrate. Target specific version with VERSION=x. Turn off output with VERBOSE=false."
task :migrate => :environment do
ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
ActiveRecord::Migrator.migrate("db/migrate/", ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
Rake::Task["db:schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
end
Then in my environment variables I have
echo %Version% #=> V3.5.0f
in Ruby
ENV["VERSION"] # => V3.5.0f
ENV["VERSION"].to_i #=>0 not nil !
thus the rake task calls
ActiveRecord::Migrator.migrate("db/migrate/", 0)
and in ActiveRecord::Migrator we have :
class Migrator#:nodoc:
class << self
def migrate(migrations_path, target_version = nil)
case
when target_version.nil? then up(migrations_path, target_version)
when current_version > target_version then down(migrations_path, target_version)
else up(migrations_path, target_version)
end
end
Yes, rake db:migrate VERSION=0 is the long version for rake db:migrate:down
Edit - I would go update the lighthouse bug but I the super company proxy forbids that I connect there
In the meantime you may try to unset Version before you call migrate ...
This is not the expected behaviour. I was going to suggest reporting this as a bug on lighthouse, but I see you've already done so! If you provide some more information (including OS/database/ruby version) I will take a look at it.
I respectfully disagree Tom! this is a bug !! V3.5.0f is not a valid version for rake migrations. Rake should not use it to migrate:down just because ruby chose to consider that "V3.5.0f".to_i is 0 ...
Rake should loudly complain that VERSION is not valid so that users know what is up
(between you and me, checking that the version is a YYYYMMDD formated timestamp by converting to integer is a bit light)
[Damn IE6 that won't allow me to comment ! and no I can't change browser thanks corporate]
Jean,
Thanks a lot for your investigation. You're right, and actually I think you've uncovered a more severe bug, of species 'design bug'.
What's happening is that rake will grab whatever value you pass to the command line and store them as environment variables. The rake tasks that will eventually get called will just pull this values from the environment variable.
When db:migrate queries ENV["VERSION"], it actually requests the version parameter which you set calling rake. When you call rake db:migrate, you don't pass any version.
But we do have an environment variable called VERSION that has been set for other purposes by some other program (I don't which one yet). And the guys behind rake (or behind database.rake) haven't figured this would happen. That's a design bug. At least, they could have used more specific variable names like "RAKE_VERSION" or "RAKE_PARAM_VERSION" instead of just "VERSION".
Tom, I will definitely not close but edit my bug report on lighthouse to reflect these new findings.
And thanks again Jean for your help. I've posted this bug on lighthouse like 5 days agao and still got no answer!
Rollo