I want to create schema.sql instead of schema.rb. After googling around I found that it can be done by setting sql schema format in application.rb. So I set following in application.rb
config.active_record.schema_format = :sql
But if I set schema_format to :sql, schema.rb/schema.sql is not created at all. If I comment the line above it creates schema.rb but I need schema.sql. I am assuming that it will have database structure dumped in it and
I know that the database structure can be dumped using
rake db:structure:dump
But I want it to be done automatically when database is migrated.
Is there anything I am missing or assuming wrong ?
Five months after the original question the problem still exists. The answer is that you did everything correctly, but there is a bug in Rails.
Even in the guides it looks like all you need is to change the format from :ruby to :sql, but the migrate task is defined like this (activerecord/lib/active_record/railties/databases.rake line 155):
task :migrate => [:environment, :load_config] do
ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
db_namespace["schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
end
As you can see, nothing happens unless the schema_format equals :ruby.
Automatic dumping of the schema in SQL format was working in Rails 1.x. Something has changed in Rails 2, and has not been fixed.
The problem is that even if you manage to create the schema in SQL format, there is no task to load this into the database, and the task rake db:setup will ignore your database structure.
The bug has been noticed recently: https://github.com/rails/rails/issues/715 (and issues/715), and there is a patch at https://gist.github.com/971720
You may want to wait until the patch is applied to Rails (the edge version still has this bug), or apply the patch yourself (you may need to do it manually, since line numbers have changed a little).
Workaround:
With bundler it's relatively hard to patch the libraries (upgrades are so easy, that they are done very often and the paths are polluted with strange numbers - at least if you use edge rails ;-), so, instead of patching the file directly, you may want to create two files in your lib/tasks folder:
lib/tasks/schema_format.rake:
import File.expand_path(File.dirname(__FILE__)+"/schema_format.rb")
# Loads the *_structure.sql file into current environment's database.
# This is a slightly modified copy of the 'test:clone_structure' task.
def db_load_structure(filename)
abcs = ActiveRecord::Base.configurations
case abcs[Rails.env]['adapter']
when /mysql/
ActiveRecord::Base.establish_connection(Rails.env)
ActiveRecord::Base.connection.execute('SET foreign_key_checks = 0')
IO.readlines(filename).join.split("\n\n").each do |table|
ActiveRecord::Base.connection.execute(table)
end
when /postgresql/
ENV['PGHOST'] = abcs[Rails.env]['host'] if abcs[Rails.env]['host']
ENV['PGPORT'] = abcs[Rails.env]['port'].to_s if abcs[Rails.env]['port']
ENV['PGPASSWORD'] = abcs[Rails.env]['password'].to_s if abcs[Rails.env]['password']
`psql -U "#{abcs[Rails.env]['username']}" -f #{filename} #{abcs[Rails.env]['database']} #{abcs[Rails.env]['template']}`
when /sqlite/
dbfile = abcs[Rails.env]['database'] || abcs[Rails.env]['dbfile']
`sqlite3 #{dbfile} < #{filename}`
when 'sqlserver'
`osql -E -S #{abcs[Rails.env]['host']} -d #{abcs[Rails.env]['database']} -i #{filename}`
# There was a relative path. Is that important? : db\\#{Rails.env}_structure.sql`
when 'oci', 'oracle'
ActiveRecord::Base.establish_connection(Rails.env)
IO.readlines(filename).join.split(";\n\n").each do |ddl|
ActiveRecord::Base.connection.execute(ddl)
end
when 'firebird'
set_firebird_env(abcs[Rails.env])
db_string = firebird_db_string(abcs[Rails.env])
sh "isql -i #{filename} #{db_string}"
else
raise "Task not supported by '#{abcs[Rails.env]['adapter']}'"
end
end
namespace :db do
namespace :structure do
desc "Load development_structure.sql file into the current environment's database"
task :load => :environment do
file_env = 'development' # From which environment you want the structure?
# You may use a parameter or define different tasks.
db_load_structure "#{Rails.root}/db/#{file_env}_structure.sql"
end
end
end
and lib/tasks/schema_format.rb:
def dump_structure_if_sql
Rake::Task['db:structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
end
Rake::Task['db:migrate' ].enhance do dump_structure_if_sql end
Rake::Task['db:migrate:up' ].enhance do dump_structure_if_sql end
Rake::Task['db:migrate:down'].enhance do dump_structure_if_sql end
Rake::Task['db:rollback' ].enhance do dump_structure_if_sql end
Rake::Task['db:forward' ].enhance do dump_structure_if_sql end
Rake::Task['db:structure:dump'].enhance do
# If not reenabled, then in db:migrate:redo task the dump would be called only once,
# and would contain only the state after the down-migration.
Rake::Task['db:structure:dump'].reenable
end
# The 'db:setup' task needs to be rewritten.
Rake::Task['db:setup'].clear.enhance(['environment']) do # see the .clear method invoked?
Rake::Task['db:create'].invoke
Rake::Task['db:schema:load'].invoke if ActiveRecord::Base.schema_format == :ruby
Rake::Task['db:structure:load'].invoke if ActiveRecord::Base.schema_format == :sql
Rake::Task['db:seed'].invoke
end
Having these files, you have monkeypatched rake tasks, and you still can easily upgrade Rails. Of course, you should monitor the changes introduced in the file activerecord/lib/active_record/railties/databases.rake and decide whether the modifications are still necessary.
I'm using rails 2.3.5 but this may apply to 3.0 as well:
rake db:structure:dump does the trick for me.
It's possible you need to delete the schema.rb for the schema.sql to be created.
Related
I have a number of rake tasks for which I would like to implement around-hook-like behavior. Specifically, I'm looking for a way to ensure that all of my Rake tasks execute in a particular (complicated, derived) Time.use_zone block.
For analogy, I have this in my ApplicationController:
around_filter :use_time_zone
def use_time_zone
time_zone = non_trivial_derivation
Time.use_zone(time_zone) { yield }
end
And now all of my controller actions will appropriately execute in the specified time zone. I would like some mechanism like this for Rake. I'd be willing to change or modify the dependency chain for my rake tasks, but I don't want to insert the actual time zone derivation code at the top of each rake task, out of concerns that that would lead to maintenance fragility. I'm pretty sure that Rake dependencies hold the solution--after all, Rake dependencies allow me to execute code in the context of my Rails application. But I can't figure out how to get that done for this use case.
I came up with a simple solution that doesn't require any external dependencies or gems such as rake-hooks:
desc "rake around hook"
task :use_timezone, [:subtask] => :environment do |name, args|
puts "using timezone"
Rake::Task[args[:subtask]].invoke
puts "end using timezone"
end
task :testing do
puts "testing"
end
The idea is that you execute the main use_timezone task and pass in your actual task as an argument:
$ rake use_timezone[testing]
That outputs:
> using timezone
> testing
> end using timezone
For your case you can write it like this:
task :use_timezone, [:subtask] => :environment do |name, args|
time_zone = non_trivial_derivation
Time.use_zone(time_zone) { Rake::Task[args[:subtask]].invoke }
end
And use it like this:
$ rake use_timezone[your_task]
Hope that helps.
Seeking to move all my shared models to an engine which can be included in each of my micro apps.
This engine should provide a model layer to all our legacy data, including:
Model files
Schema files
Migrations (we're following Pivotal Labs' pattern, this isn't the issue)
Model files are being patched in automatically, that's fine.
Schema files are being monkey-patched in using Nikolay Strum's db.rake:
namespace :db do
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
filename = "#{Rails.root}/db/foo_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("foo_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
# like db:test:purge
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs['foo_test']['database'], mysql_creation_options(abcs['foo_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection('foo_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/foo_schema.rb")
end
end
end
We need rake db:create and rake db:schema:load to work,
The db.rake patches only affect db:schema:dump and db:test:load_schema (part of tests_prepare, I assume). I've attempted to patch them into db:schema:load using:
namespace :db do
# Helpers
def mysql_creation_options(config)
#charset = ENV['CHARSET'] || 'utf8'
#collation = ENV['COLLATION'] || 'utf8_unicode_ci'
{:charset => (config['charset'] || #charset), :collation => (config['collation'] || #collation)}
end
def load_schema(schema_name)
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs[schema_name+'_test']['database'], mysql_creation_options(abcs[schema_name+'_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection(schema_name+'_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/#{schema_name}_schema.rb")
end
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
dump_schema = -> (schema_name) {
filename = "#{Rails.root}/db/#{schema_name}_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("#{schema_name}_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
}
dump_schema.call('kiddom')
dump_schema.call('kiddom_warehouse')
end
# When loading from schema, load these files, too
task :load => [:environment, :load_config] do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
end
But this gives me the error NoMethodError: undefined method 'recreate_database' for #<ActiveRecord::ConnectionAdapters::SQLite3Adapter:0x007feb6bb43558>. Apparently, this only works on Oracle-type databases?
What are the Rails commands for the underlying DROP and CREATE DATABASE sql I'm trying to affect for the extra schema.rb's?
You are using SQLite as your database engine. Hope that is what you want to do.
Since you are creating SQLite Database , things differ a bit from other database adapters like MySQLAdpter or Postgress.
In case of MySQL, database has to be created prior to establish a connection by spending "CREATE DATABASE ... " SQL commands. So you must create the database before establishing connection.
But in case of SQLite, since the database reside in a file and one file can contain only one database, there is no separate step to create database. An attempt to establishing a connection to the database itself will cause the database file to be created.
Hence create_database method won't work when using SQLiteAdapter. You can simply remove that line from your code.
You may have a look at the source code for the Rake task db:create
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
Also, the source code for 'create' method in SQLiteDatabaseTasks. As you can see, it simply calls the establish_connection method
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
I have a large database with 4+ million addresses/records.
The rake command (below) worked fine when the database was a small test set, but now with the large database it just simply stalls.
rake geocode:all CLASS=YourModel
2 questions:
1. Is there any simple method to have geocoder code a null/nil lat and long when the records are called (on the fly). I have a feeling that this would be hard.
2. Anyone else have problems with geocode-ing a large dataset and using the rake command?
Thanks!
Update:
I create pull request based on this answer and now you can use batch in geocoder:
rake geocode:all CLASS=YourModel SLEEP=0.25 BATCH=100
I would use this solution for large database, i take it from geocoder gem rake task:
You can refine this for your needs.
Some example create rake task:
namespace :geocode_my_data do
desc "Geocode all objects in my databse."
task all: :environment do
klass = User
klass.where(geocoded: false).find_each(limit: 100) do |obj|
obj.geocode; obj.save
end
end
end
$> rake geocode_my_data:all
Used below code and put it into my lib/tasks folder as geocode_my_data.rake
To run:
rake geocode:all CLASS=YourModel
Works great!
namespace :geocode_my_data do
desc "Geocode all objects without coordinates."
task :all => :environment do
class_name = ENV['CLASS'] || ENV['class']
sleep_timer = ENV['SLEEP'] || ENV['sleep']
raise "Please specify a CLASS (model)" unless class_name
klass = class_from_string(class_name)
klass.not_geocoded.find_each(batch_size: 100) do |obj|
obj.geocode; obj.save
sleep(sleep_timer.to_f) unless sleep_timer.nil?
end
end
end
##
# Get a class object from the string given in the shell environment.
# Similar to ActiveSupport's +constantize+ method.
#
def class_from_string(class_name)
parts = class_name.split("::")
constant = Object
parts.each do |part|
constant = constant.const_get(part)
end
constant
end
In my rake task if I want to know the name of the file when that file is picked up for testing then how do I do that. Reason is that some of the files produce warning. I am not sure which of my 800 tests is producing warning.
My rake task is something like this. I am using rails3.
Rake::TestTask.new(:test_hr_module) do |t|
t.libs << 'test'
t.test_files = Dir.glob('test/{hr}/**/*_test.rb').sort
t.warning = true
t.verbose = true
end
You can always call single files:
ruby test/unit/model_test.rb
You can even use the name flag to only run specific tests
ruby test/unit/model_test.rb -n test_name
You'll probably need to change require 'test_helper' to require 'test/test_helper' in the test file though.
If you really need to do this in a rake task could you try:
Dir.glob('test/**/*_test.rb').each do |f|
puts `ruby #{f}`
end
end
Perhaps you can use the pseudo-variable __FILE__ to achieve this?
I wrote a little monkeypatch to the Rails MySQLAdapter and want to package it up to use it in my other projects. I am trying to write some tests for it but I am still new to testing and I am not sure how to test this. Can someone help get me started?
Here is the code I want to test:
unless RAILS_ENV == 'production'
module ActiveRecord
module ConnectionAdapters
class MysqlAdapter < AbstractAdapter
def select_with_explain(sql, name = nil)
explanation = execute_with_disable_logging('EXPLAIN ' + sql)
e = explanation.all_hashes.first
exp = e.collect{|k,v| " | #{k}: #{v} "}.join
log(exp, 'Explain')
select_without_explain(sql, name)
end
def execute_with_disable_logging(sql, name = nil) #:nodoc:
#Run a query without logging
#connection.query(sql)
rescue ActiveRecord::StatementInvalid => exception
if exception.message.split(":").first =~ /Packets out of order/
raise ActiveRecord::StatementInvalid, "'Packets out of order' error was received from the database. Please update your mysql bindings (gem install mysql) and read http://dev.mysql.com/doc/mysql/en/password-hashing.html for more information. If you're on Windows, use the Instant Rails installer to get the updated mysql bindings."
else
raise
end
end
alias_method_chain :select, :explain
end
end
end
end
Thanks.
General testing
You could start reading about testing.
After you are understanding the basics of testing, you should think what you have changed. Then make some tests which test for
the original situation, resulting in errors since you updated it. So reverse the test after it indeed is working for the original situation.
the new situation to see whether you have implemented your idea correctly
The hardest part is to be sure that you covered all situations. Finally, if both parts pass then you could say that your code it working as expected.
Testing gems
In order to test gems you can run
rake test:plugins
to test all plugins of your rails application (see more in chapter 6 of the testing guide), this only works when the gem is in the vendor directory of an application.
Another possibility is to modify the Rakefile of the gem by including a testing task. For example this
desc 'Test my custom made gem.'
Rake::TestTask.new(:test) do |t|
t.libs << 'lib'
t.libs << 'test'
t.pattern = 'test/**/*_test.rb'
t.verbose = true
end
would run all available tests in the test directory ending with _test.rb. To execute this test you can type rake test (from the gem directory!).
In order to run the tests for the gem by default (when typing just rake) you can add/modify this line:
task :default => :test
I used the second method in my ruby-bbcode gem, so you could take a look at it to see the complete example.