I have a rails project where I need to add some default values to the database table. I want to know the best way of doing these (im using rails 2.3.3 which doesn't have seed.rb file :()
1 - Creating a sql script
2 - creating a migration
3 - creating a rake task
4 - other (Please explain)
thanks in advance
cheers
sameera
Take a look at seed-fu.
In current stable version of Rails (2.3.8) there is rake task db:seed, which execute code in db/seeds.rb file. In that file you can load you data by directly executing Rails code (News.create :title => "test" ...), or use any other method you prefer.
I prefer to load data from fixtures, cause fixtures can be used in tests later. I'm using rspec, so my fixtures stored in spec/fixtures/ directory.
You can use next code to make fixtures from existing sql tables:
def make_fixtures(tablenames, limit = nil)
sql = "SELECT * FROM %s"
sql += " LIMIT #{limit}" unless limit.nil?
dump_tables = tablenames.to_a
dump_tables.each do |table_name|
i = "000"
file_name = "#{RAILS_ROOT}/spec/fixtures/#{table_name}.yml"
puts "Fixture save for table #{table_name} to #{file_name}"
File.open(file_name, 'w') do |file|
data = ActiveRecord::Base.connection.select_all(sql % table_name )
file.write( data.inject({}) do |hash, record|
hash["#{table_name}_#{i.succ!}"] = record
hash
end.to_yaml )
end
end
end
In db/seeds.rb you can load from fixtures:
require 'active_record/fixtures'
[ "classifiers", "roles", "countries", "states", "metro_areas" ].each do |seed|
puts "Seeding #{seed}..."
Fixtures.create_fixtures(File.join(Rails.root, "spec", "fixtures"), seed)
end
Related
Seeking to move all my shared models to an engine which can be included in each of my micro apps.
This engine should provide a model layer to all our legacy data, including:
Model files
Schema files
Migrations (we're following Pivotal Labs' pattern, this isn't the issue)
Model files are being patched in automatically, that's fine.
Schema files are being monkey-patched in using Nikolay Strum's db.rake:
namespace :db do
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
filename = "#{Rails.root}/db/foo_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("foo_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
# like db:test:purge
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs['foo_test']['database'], mysql_creation_options(abcs['foo_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection('foo_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/foo_schema.rb")
end
end
end
We need rake db:create and rake db:schema:load to work,
The db.rake patches only affect db:schema:dump and db:test:load_schema (part of tests_prepare, I assume). I've attempted to patch them into db:schema:load using:
namespace :db do
# Helpers
def mysql_creation_options(config)
#charset = ENV['CHARSET'] || 'utf8'
#collation = ENV['COLLATION'] || 'utf8_unicode_ci'
{:charset => (config['charset'] || #charset), :collation => (config['collation'] || #collation)}
end
def load_schema(schema_name)
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.connection.recreate_database(abcs[schema_name+'_test']['database'], mysql_creation_options(abcs[schema_name+'_test']))
# like db:test:load_schema
ActiveRecord::Base.establish_connection(schema_name+'_test')
ActiveRecord::Schema.verbose = false
load("#{Rails.root}/db/#{schema_name}_schema.rb")
end
namespace :schema do
# desc 'Dump additional database schema'
task :dump => [:environment, :load_config] do
dump_schema = -> (schema_name) {
filename = "#{Rails.root}/db/#{schema_name}_schema.rb"
File.open(filename, 'w:utf-8') do |file|
ActiveRecord::Base.establish_connection("#{schema_name}_#{Rails.env}")
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, file)
end
}
dump_schema.call('kiddom')
dump_schema.call('kiddom_warehouse')
end
# When loading from schema, load these files, too
task :load => [:environment, :load_config] do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
namespace :test do
# desc 'Purge and load foo_test schema'
task :load_schema do
load_schema('kiddom')
load_schema('kiddom_warehouse')
end
end
end
But this gives me the error NoMethodError: undefined method 'recreate_database' for #<ActiveRecord::ConnectionAdapters::SQLite3Adapter:0x007feb6bb43558>. Apparently, this only works on Oracle-type databases?
What are the Rails commands for the underlying DROP and CREATE DATABASE sql I'm trying to affect for the extra schema.rb's?
You are using SQLite as your database engine. Hope that is what you want to do.
Since you are creating SQLite Database , things differ a bit from other database adapters like MySQLAdpter or Postgress.
In case of MySQL, database has to be created prior to establish a connection by spending "CREATE DATABASE ... " SQL commands. So you must create the database before establishing connection.
But in case of SQLite, since the database reside in a file and one file can contain only one database, there is no separate step to create database. An attempt to establishing a connection to the database itself will cause the database file to be created.
Hence create_database method won't work when using SQLiteAdapter. You can simply remove that line from your code.
You may have a look at the source code for the Rake task db:create
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
Also, the source code for 'create' method in SQLiteDatabaseTasks. As you can see, it simply calls the establish_connection method
https://github.com/rails/rails/blob/f47b4236e089b07cb683ee9b7ff8b06111a0ec10/activerecord/lib/active_record/railties/databases.rake
I have a large database with 4+ million addresses/records.
The rake command (below) worked fine when the database was a small test set, but now with the large database it just simply stalls.
rake geocode:all CLASS=YourModel
2 questions:
1. Is there any simple method to have geocoder code a null/nil lat and long when the records are called (on the fly). I have a feeling that this would be hard.
2. Anyone else have problems with geocode-ing a large dataset and using the rake command?
Thanks!
Update:
I create pull request based on this answer and now you can use batch in geocoder:
rake geocode:all CLASS=YourModel SLEEP=0.25 BATCH=100
I would use this solution for large database, i take it from geocoder gem rake task:
You can refine this for your needs.
Some example create rake task:
namespace :geocode_my_data do
desc "Geocode all objects in my databse."
task all: :environment do
klass = User
klass.where(geocoded: false).find_each(limit: 100) do |obj|
obj.geocode; obj.save
end
end
end
$> rake geocode_my_data:all
Used below code and put it into my lib/tasks folder as geocode_my_data.rake
To run:
rake geocode:all CLASS=YourModel
Works great!
namespace :geocode_my_data do
desc "Geocode all objects without coordinates."
task :all => :environment do
class_name = ENV['CLASS'] || ENV['class']
sleep_timer = ENV['SLEEP'] || ENV['sleep']
raise "Please specify a CLASS (model)" unless class_name
klass = class_from_string(class_name)
klass.not_geocoded.find_each(batch_size: 100) do |obj|
obj.geocode; obj.save
sleep(sleep_timer.to_f) unless sleep_timer.nil?
end
end
end
##
# Get a class object from the string given in the shell environment.
# Similar to ActiveSupport's +constantize+ method.
#
def class_from_string(class_name)
parts = class_name.split("::")
constant = Object
parts.each do |part|
constant = constant.const_get(part)
end
constant
end
I'm working on a project that is migrating data from a customers old_busted DB into rails objects to be worked on later. Similarly, I need to convert these objects into a CSV and upload it to a neutral FTP (this is to allow a coworker to build the example pages through Sugar CRM). I've created rake files to do all of this, and it was successful. Now, I'm going to continue this process for each object that I create in rails (relative to the previous DB) and, best case, wanted these generated when I run rake generate scaffold <object>.
Here is my import rake:
desc "Import Clients from db"
task :get_busted_clients => [:environment] do
#old_clients = Busted::Client.all
#old_clients.each do |row|
#client = Client.new();
#client.client_id = row.NUMBER
#client.save
end
end
Here is my CSV convert/FTP upload rake:
desc "Exports db's to local CSV and uploads them to FTP"
task :export_clients_CSV => [:environment] do
# Required libraries for CSV read/write and NET/FTP IO #
require 'csv'
require 'net/ftp'
# Pull all Editor objects into clients for reading #
clients = Client.all
puts "Creating CSV file for <Clients> and updating column names..."
# Open a new CSV file that uses the column headers from Client #
CSV.open("clients.csv", "wb",
:write_headers => true, :headers => Client.column_names) do |csv|
puts "--Loading each entry..."
# Load all entries from Client into the CSV file row by row #
clients.each do |client|
# This line specifically puts the attributes in the rows WITH RESPECT TO#
# THE COLUMNS
csv << client.attributes.values_at(*Client.column_names)
end
puts "--Done loading each entry..."
end
puts "...Data populated. Finished bulding CSV. Closing File."
puts "------------------------"
# Upload CSV File to FTP server by requesting new FTP connection, assigning credentials
# and informing the client what file to look for and what to name it
puts "Uploading <Clients>..."
ftp = Net::FTP.new('192.168.xxx.xxx')
ftp.login(user = "user", passwd = "passwd")
ftp.puttextfile("clients.csv", "clients.csv")
ftp.quit()
puts "...Finished."
end
I ran rake generate g get_busted and put this in my get_busted_generator.rb:
class GetBustedGenerator < Rails::Generators::NamedBase
source_root File.expand_path('../templates', __FILE__)
def generate_get_busted
copy_file "getbusted.rake", "lib/tasks/#{file_name}.rake"
end
end
After that, I got lost. I can't find anything on templating a rake file or the syntax included to do so.
Rails has been a recent endeavor and I may be overlooking something in terms of design of the solution to my problem.
TL;DR: Is templating a rake file a bad thing? Solution alternatives? If not, whats the syntax for generating either script custom to the object (or point me in the direction, please).
How do I regenerate all the YML fixture files? I accidentally deleted them.
#brian,
I'm using the following script to generate the fixtures from a given sql
This is under my lib/task directory as a rake task
namespace :fixture_generator do
desc "generate fixtures for a given sql query from the current development database"
task :fixture_generator, [:sql, :file_name] => :environment do |t, args|
args.with_defaults(:sql => nil, :file_name => nil)
i = "000"
p "creating fixture - #{args.file_name}"
File.open("#{Rails.root}/test/fixtures/#{args.file_name}.yml", 'a+') do |file|
data = ActiveRecord::Base.connection.select_all(args.sql)
file.write data.inject({}) { |hash, record|
number = i.succ!
hash["#{args.file_name}_#{number}"] = record
hash
}.to_yaml
end
end
end
Usage, Say I want to generate fixture for users table
rake fixture_generator:fixture_generator["select * from users","users"]
And also, If you run another query with the same fixture file name, it will append to the existing one
HTH
I am trying to run a ruby script that parses a text file with album name, artist name, year on each line. It should then save a new record into a Rails model called Album.
(buildDB.rb)
fh = File.open('albums.txt')
while line = fh.gets
if ( line =~ /^(.+)\~\s(.+) \(\'(\d\d)\)/ )
a = Album.new
a.name = $1
a.artist = $2
a.year = $3
a.save
end
end
I am running ruby buildDB.rb in terminal which produces the message
buildDb.rb:12:in '<main>': uninitialized constant Album (NameError)
This made me think that the script could not find the model. So I tried loading the rails environment by using
require "C:/ruby192/www/Project02/config/environment.rb"
at the top of the ruby script. The script will run without errors but nothing is committed to the sqlite database. I can also run find on the already existing Albums, it just seems I can't create new ones.
I am a rails noob so, there probably is a better way to do this (seeds.rb or a rake task maybe). Any help or a direction to look into would be greatly appreciated.
I would use a rake task:
task :create_albums => :environment do
fh = File.open('albums.txt')
while line = fh.gets
if ( line =~ /^(.+)\~\s(.+) \(\'(\d\d)\)/ )
a = Album.new
a.name = $1
a.artist = $2
a.year = $3
a.save!
end
end
fh.close
end
I added a ! to the save method so that any errors will throw an exeception. Also, make sure you close your files.
Rake task is the way ahead. I would use FasterCSV as it will handle some of the data import for you.
namespace :import do
desc "Import from a csv file"
task :album_csv, [:filename] => :environment do |task, args|
lines = FasterCSV.read(args[:filename]) rescue nil
if lines
lines.slice!(0) # remove the CSV header if there is one
puts "# Processing #{lines.count} records"
lines.each do |line|
a = Album.new
a.name = line[0]
a.artist = line[1]
a.year = line[2]
a.save!
end
end
end
You can then call your rake task as:
rake import:album_csv[filename]