How do I regenerate all the YML fixture files? I accidentally deleted them.
#brian,
I'm using the following script to generate the fixtures from a given sql
This is under my lib/task directory as a rake task
namespace :fixture_generator do
desc "generate fixtures for a given sql query from the current development database"
task :fixture_generator, [:sql, :file_name] => :environment do |t, args|
args.with_defaults(:sql => nil, :file_name => nil)
i = "000"
p "creating fixture - #{args.file_name}"
File.open("#{Rails.root}/test/fixtures/#{args.file_name}.yml", 'a+') do |file|
data = ActiveRecord::Base.connection.select_all(args.sql)
file.write data.inject({}) { |hash, record|
number = i.succ!
hash["#{args.file_name}_#{number}"] = record
hash
}.to_yaml
end
end
end
Usage, Say I want to generate fixture for users table
rake fixture_generator:fixture_generator["select * from users","users"]
And also, If you run another query with the same fixture file name, it will append to the existing one
HTH
Related
Seeking to move all my shared models to an engine which can be included in each of my micro apps.
This engine should provide a model layer to all our legacy data, including:
Model files
Schema files
Migrations (we're following Pivotal Labs' pattern, this isn't the issue)
Model files are being patched in automatically, that's fine.
Schema files are being monkey-patched in using Nikolay Strum's db.rake:
namespace :db do
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
filename = "#{Rails.root}/db/foo_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("foo_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
# like db:test:purge
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs['foo_test']['database'], mysql_creation_options(abcs['foo_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection('foo_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/foo_schema.rb")
end
end
end
We need rake db:create and rake db:schema:load to work,
The db.rake patches only affect db:schema:dump and db:test:load_schema (part of tests_prepare, I assume). I've attempted to patch them into db:schema:load using:
namespace :db do
# Helpers
def mysql_creation_options(config)
#charset = ENV['CHARSET'] || 'utf8'
#collation = ENV['COLLATION'] || 'utf8_unicode_ci'
{:charset => (config['charset'] || #charset), :collation => (config['collation'] || #collation)}
end
def load_schema(schema_name)
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs[schema_name+'_test']['database'], mysql_creation_options(abcs[schema_name+'_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection(schema_name+'_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/#{schema_name}_schema.rb")
end
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
dump_schema = -> (schema_name) {
filename = "#{Rails.root}/db/#{schema_name}_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("#{schema_name}_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
}
dump_schema.call('kiddom')
dump_schema.call('kiddom_warehouse')
end
# When loading from schema, load these files, too
task :load => [:environment, :load_config] do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
end
But this gives me the error NoMethodError: undefined method 'recreate_database' for #<ActiveRecord::ConnectionAdapters::SQLite3Adapter:0x007feb6bb43558>. Apparently, this only works on Oracle-type databases?
What are the Rails commands for the underlying DROP and CREATE DATABASE sql I'm trying to affect for the extra schema.rb's?
You are using SQLite as your database engine. Hope that is what you want to do.
Since you are creating SQLite Database , things differ a bit from other database adapters like MySQLAdpter or Postgress.
In case of MySQL, database has to be created prior to establish a connection by spending "CREATE DATABASE ... " SQL commands. So you must create the database before establishing connection.
But in case of SQLite, since the database reside in a file and one file can contain only one database, there is no separate step to create database. An attempt to establishing a connection to the database itself will cause the database file to be created.
Hence create_database method won't work when using SQLiteAdapter. You can simply remove that line from your code.
You may have a look at the source code for the Rake task db:create
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
Also, the source code for 'create' method in SQLiteDatabaseTasks. As you can see, it simply calls the establish_connection method
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
I'm very new to the concept of importing data into a SQL database with CSV. I've followed some stackoverflow posts but I'm getting an error. The error states, Errno::ENOENT: No such file or directory # rb_sysopen - products.csv after running rake import:data. I have csv required in my application.rb as well as I have created a csv file and placed it in TMP. Here is my code so far. I understand I may be asking for a lot from the community but if someone were to answer this question, can you provide some more insight into CSV and rake functions. Thanks so much!!!
<b>import.rake</b>
namespace :import do
desc "imports data from a csv file"
task :data => :environment do
require 'csv'
CSV.foreach('tmp/products.csv') do |row|
name = row[0]
price = row[1].to_i
Product.create( name: name, price: price )
end
end
end
Specify the full path to the CSV file.
For example, if the file is in /tmp/ use:
CSV.foreach('/tmp/products.csv') do |row|
If the products.csv file is in your application's tmp directory use:
CSV.foreach(Rails.root.join('tmp', 'products.csv')) do |row|
I ran into something similar, it was forgetting to put both parenthesis with the braces so you might want to try going from:
Product.create( name: name, price: price )
to:
Product.create({ name: name, price: price })
Check out the smarter_csv Gem.
In it's simplest form you can do this:
SmarterCSV.process('tmp/products.csv').each do |hash|
Product.create( hash )
end
Add smarter_csv to your Gemfile, so it's auto-loaded when you require the environment in your Rake task
This gives you:
namespace :import do
desc 'imports data from given csv file'
task :data, [:filename] => :environment do |t, args|
fail "File not found" unless File.exists? args[:filename]
options = {} # add options if needed
SmarterCSV.process( args[:filename], options).each do |hash|
Product.create( hash )
end
end
end
Call it like this:
rake import:data['/tmp/products.csv']
See also: https://github.com/tilo/smarter_csv
I am writing a Ruby on Rails application which has a Rake task that can parse a CSV file.
Here is the code:
desc "Import Channels into DB\n Usage: rake channel_import"
task :import_channels, :path_to_channel_list do |t, args|
require "#{Rails.root}/app/models/channel"
require "csv"
filePath = args.path_to_channel_list
puts "Filepath received = #{filePath}"
csv = CSV.read("#{filePath}", :encoding => 'windows-1251:utf-8')
csv.each_with_index do |row, i|
if [0].include?(i)
puts "Skipping over row #{i}"
next
end
if(row.nil?)
puts "row[#{i}] was nil"
else
channelName = nil
classif = nil
owner = nil
channelName = row[0].force_encoding('UTF-8')
classif = row[1].force_encoding('UTF-8')
owner = row[2].force_encoding('UTF-8')
if (channelName.nil?)
puts "Channel name for row #{i} was nil"
#add entry to Log file or errors database
next #skip this row of the csv file and go to next row
else
channel_hash = Hash.new("name" =>"#{channelName}", "classification" => "#{classif}", "owner" => "#{owner}" )
end
puts "\nChannel Name = #{channelName}\nClassification = #{classif}\n Ownership = #{owner}"
#If channel name exists in the Database, update it
xisting_channel = nil
xisting_channel = Channel.find_by channel_name: '#{channelName}'
if(xisting_channel.nil?)
#create a new channel
#new_channel = Channel.create(channel_hash)
puts "Inserted....#{#new_channel.inspect}"
else
#update existing channel
Channel.update(xisting_channel.id, :classification => "#{classif}", :ownership => "#{owner}" )
puts "Updated...."
puts "channel_hash = #{channel_hash.inspect} "
end#end if/else
end#end if/else
end #end CSV.each
end
When I run this code I get the following error message:
MRMIOMP0903:am AM$ rake import_channels[/XXXXXXXX/Channellist.csv]
Filepath received = /XXXXXXX/Channellist.csv
Skipping over row 0
Channel Name = APTN HD+
Classification = Specialty
Ownership = Aboriginal Peoples Television Network
rake aborted!
ActiveRecord::ConnectionNotEstablished
I tried to create a Channel object using IRB and it worked just fine. The DB is created and I'm seeing it using MySQL WorkBench. All tables are there, however, I am unable to create the Channel object from the Rake task.
My hypothesis is, perhaps outside of a certain folder/hierarchy the app cannot access the ActiveRecord::Base class or something like this.
Any suggestions on how to make this work?
UPDATE:
BAsed on the answer by Phillip Hallstrom
I changed the top line to load the environment
task :import_channels => :environment do|t, args|
Then tried rake import_channels and got this error:
rake aborted!
undefined method `safe_constantize' for #<Hash:0x007fbeacd89920>
You need to load the environment prior to running your Rake task. I'm not familiar with your multiple task name options there, but for a simple example, you'd want this:
task : import_channels => :environment do
I have a Rake script similar to below,but I am wondering if there is a more efficient way to do this, without having to drop the database, run all the migrations, reseed the database and then add the sample data?
namespace :db do
desc 'Fill database with sample data'
task populate: :environment do
purge_database
create_researchers
create_organisations
add_survey_groups_to_organisations
add_members_to_survey_groups
create_survey_responses_for_members
end
end
def purge_database
puts 'about to drop and recreate database'
system('rake db:drop')
puts 'database dropped'
system('rake db:create')
system('rake db:migrate')
system('rake db:seed')
puts 'Database recreated...'
end
def create_researchers
10.times do
researcher = User.new
researcher.email = Faker::Internet.email
researcher.save!
end
end
You should not fill your database with sample data via db:seed. That's not the purpose of the seeds file.
db:seed is for initial data that your app needs in order to function. It's not for testing and/or development purposes.
What I do is to have one task that populates sample data and another task that drops the database, creates it, migrates, seeds and populates. The cool thing is that it's composed of other tasks, so you don't have to duplicate code anywhere:
# lib/tasks/sample_data.rake
namespace :db do
desc 'Drop, create, migrate, seed and populate sample data'
task prepare: [:drop, :create, "schema:load", :seed, :populate_sample_data] do
puts 'Ready to go!'
end
desc 'Populates the database with sample data'
task populate_sample_data: :environment do
10.times { User.create!(email: Faker::Internet.email) }
end
end
I would suggest making rake db:seed self sufficient. By which I mean, you should be able to run it multiple times without it doing any damage, while ensuring that whatever sample data you need loaded gets loaded.
So, for your researches, the db:seed task should do something like this:
User.destroy_all
10.times do
researcher = User.new
researcher.email = Faker::Internet.email
researcher.save!
end
You can run this over and over and over and are ensured you will always end up with 10 random users.
I see this is for development. In that case, I wouldn't put it in db:seed as that might get run in production. But you can put it in a similar rake task that you can re-run as often as needed.
I have a rails project where I need to add some default values to the database table. I want to know the best way of doing these (im using rails 2.3.3 which doesn't have seed.rb file :()
1 - Creating a sql script
2 - creating a migration
3 - creating a rake task
4 - other (Please explain)
thanks in advance
cheers
sameera
Take a look at seed-fu.
In current stable version of Rails (2.3.8) there is rake task db:seed, which execute code in db/seeds.rb file. In that file you can load you data by directly executing Rails code (News.create :title => "test" ...), or use any other method you prefer.
I prefer to load data from fixtures, cause fixtures can be used in tests later. I'm using rspec, so my fixtures stored in spec/fixtures/ directory.
You can use next code to make fixtures from existing sql tables:
def make_fixtures(tablenames, limit = nil)
sql = "SELECT * FROM %s"
sql += " LIMIT #{limit}" unless limit.nil?
dump_tables = tablenames.to_a
dump_tables.each do |table_name|
i = "000"
file_name = "#{RAILS_ROOT}/spec/fixtures/#{table_name}.yml"
puts "Fixture save for table #{table_name} to #{file_name}"
File.open(file_name, 'w') do |file|
data = ActiveRecord::Base.connection.select_all(sql % table_name )
file.write( data.inject({}) do |hash, record|
hash["#{table_name}_#{i.succ!}"] = record
hash
end.to_yaml )
end
end
end
In db/seeds.rb you can load from fixtures:
require 'active_record/fixtures'
[ "classifiers", "roles", "countries", "states", "metro_areas" ].each do |seed|
puts "Seeding #{seed}..."
Fixtures.create_fixtures(File.join(Rails.root, "spec", "fixtures"), seed)
end