My seeds file runs through quite a few csv files, does a few checks and creates various ActiveRecord records accordingly. While testing all these files, I finally think I have it and run rake db:seed but if something fails, I want what has been created so far to rollback.
Scenario that has already happened:
Seeds file requires 4 different CSV's
Only 3 of the 4 CSV's were uploaded to staging server
rake db:seed was run and the seeds file blew up half way through because it couldn't find a file, but over 1000 AR objects were created prior to that.
Ideally I'd like to do something like:
begin
CSV.readlines(file1)
CSV.readlines(file2)
CSV.readlines(file3)
CSV.readlines(file4)
rescue
# raise an error
# rollback all objects created prior to error
end
I suppose I could implement something custom but I can't find anything on the rails guides regarding this.
This is the purpose of Active Record Transactions:
Transactions are protective blocks where SQL statements are only
permanent if they can all succeed as one atomic action. The classic
example is a transfer between two accounts where you can only have a
deposit if the withdrawal succeeded and vice versa. Transactions
enforce the integrity of the database and guard the data against
program errors or database break-downs. So basically you should use
transaction blocks whenever you have a number of statements that must
be executed together or not at all.
Try this
ActiveRecord::Base.transaction do
...
end
Related
I don't want to load a seed file for every deployment, since it takes 2-3 mins.
I only want to load the seeds if it is needed.
I have the following code which will run a seed in the initializer of the application, so when the application starts, it will load the seeds. But is there a way to only do it if it is needed (i.e. something has changed?)
For migrations we have: if ActiveRecord::Migrator.needs_migration?
But what about seeds?
if ENV_PROPS['run_seeds_at_startup']
Common::Log.info 'Starting seed load'
seed_file = File.join('db/seeds.rb')
load(seed_file) if File.exist?(seed_file)
Common::Log.info 'Seed data loaded'
else
Common::Log.info 'Automatic Seeds are shutdown in the env-props.yml...'
end
Generally if there's data I only need to create once, I try to setup a custom rake task that can either be run on the server directly post deploy, or one can setup a capistrano recipe to run it once. If you want to use seeds.rb, really the only way I've found to do this is to check if the records exist already. But i'm not aware of any global check or wrapper around executing the seeds file.
I've added a dependency to both order and order_items fixtures (which already existed), but I'm receiving the following error every time I run my rspec worker test.
ActiveRecord::StatementInvalid:
Mysql2::Error: Table 'inventory_test10.order_packages' doesn't exist: SHOW FULL FIELDS FROM `order_packages` /*controller:,action:,line:*/
I have an order which has many order_items and many order_packages. order_items also belong to order_packages. Therefore, I am able to do:
order.order_items.each do |oi|
put oi.order_package.status
end
The original issue was that status wasn't recognized for nil class because an order_packages.yml fixture was never created. I've tried several rake tasks, but I'm not super familiar with fixtures, migrations, rake tasks, etc and I'm not sure if I accidentally caused the error running multiple taks. Below is a snippet from a blog that warned about running the command multiple times - http://brandonhilkert.com/blog/using-rails-fixtures-to-seed-a-database/:
rake db:fixtures:load FIXTURES=credit_card_types
A word of warning, if we run this command multiple times, it will seed
the table multiple times. It’s not idempotent.
Other tasks I ran:
FIXTURES=orders; rake db:fixtures:load
rake db:fixtures:dump (didn't work - error)
rake db:fixtures:drop (didn't work - error)
Thanks in advance for any suggestions!
Your test framework should automatically load fixtures at the beginning of the test run, and delete them at the end of the test run. You should not need to load fixtures yourself.
Fixtures load data into tables; they do not alter the database structure. Migrations can alter the database by creating/dropping tables, adding/removing columns, etc. If you are having an issue with a missing table, it is very like a migration problem.
I recommend a review of the Guide to Testing Rails Applications, and (if you are using RSpec) the rspec-rails documentation, which explain these concepts in greater depth.
I have a lot of spots in my code that actually call activerecord finders. For example, in a Blog engine, I might have a table of tags that correspond to an activerecord model Tag. Suppose, for some reason, that I want special logic to happen if a post is created with a tag where tag.description == 'humor'. Then I might have a method in the model:
class Tag < ActiveRecord::Base
def self.humor_tag
find_by_description('humor')
end
end
Whether or not this is poor design, it causes insane amounts of problems for me when using rake commands to build a database. Say that later on, I've finished my development and I want to deploy to production. So I take the dumped schema.rb file, and then I want to load a new database structure from that schema.rb, or alternatively, just run my migrations to create a production database.
RAILS_ENV=production rake db:schema:load
The problem is that, in the production environment, the rake command seems to load every model. When it tries to load the Tag#humor_tag method, it throws an error that stops the process:
rake aborted!
Table 'production_database.tags' doesn't exist
Well of course it doesn't exist, it hasn't been created yet! I've googled around and people seem to solve this problem by either cloning the database in SQL or moving around their code just so they can run the rake task.
What are you supposed to do? It seems like there might be some configuration somewhere to let you tell rake to freaking ignore calls to database tables before any tables are created.
I would suggest replacing queries by class methods with scopes: http://guides.rubyonrails.org/active_record_querying.html#scopes
and if you have an initializer that is causing the models to load, use a proc in the scope definition, such as
class Post < ActiveRecord::Base
scope :published, Proc.new { where(:published => true) }
end
to prevent the scope from running at initialization time.
I'm not completely satisfied with this answer, but if anyone gets to this question and has a similar problem, this may be helpful. In order to move over a database in a situation where you would usually rake db:schema:load or just create it and run the migrations, you can alternatively load the database from SQL (or presumably other database technologies).
rake db:structure:dump
That command will dump the structure of the database into a file that will then be able to recreate it. For me, it created a file db/development_dump.sql, that contained calls to create all of the tables and indices, but didn't copy any of the data like on a normal sql dump. Then, I moved that file to my production database, and ran it:
mysql prod_database < development_dump.sql
This doesn't answer the question at hand, but it may be relevant for someone facing a similar problem.
If I have the following code defined inside db/seeds.rb,
default_car=Car.create({:name=>'TOYOTA'})
User.create({:username=>'default_user', car_id=>default_car.id})
I know the default_car and the user instances will be stored into Database when I run "rake db:seed".
My question is, if I run 'rake db:seed' again, again and again(multiple times), will the same instances be stored to database with multiple copies or it only save the instance once into database no matter how many times I run rake db:seed?
Better solution:
default_car = Car.find_or_create_by_name 'TOYOTA'
user = User.find_or_create_by_username 'default_user'
user.car = default_car
user.save
That way you can run "rake db:seed" multiple times without having to drop the database manually every time.
This is a limitation of having a single seed file. I was finding this frustrating as the application grows you often want to add new seed data so you end up either doing what Pascal suggests or creating either migrations with data in them or rake tasks to load the seeds. To get round this I knocked up seedbank. So I combine this with Pascals approach so I can re-run the seeds but can also target specific ones if I want to.
depends on your models if you allow duplicate values. if you don't it will throw an error. what you do is to clear your db first before running seed via rake db:resetdb
In a Rails application, I need a table in my database to contain constant data.
This table content is not intended to change for the moment but I do not want to put the content in the code, to be able to change it whenever needed.
I tried filling this table in the migration that created it, but this does not seem to work with the test environment and breaks my unit tests. In test environment, my model is never able to return any value while it is ok in my development environment.
Is there a way to fill that database correctly even in test environment ? Is there another way of handling these kind of data that should not be in code ?
edit
Thanks all for your answers and especially Vlad R for explaining the problem.
I now understand why my data are not loaded in test. This is because the test environment uses the db:load rake command which directly loads the schema instead of running the migrations. Having put my values in the migration only and not in the schema, these values are not loaded for test.
What you are probably observing is that the test framework is not running the migrations (db:migrate), but loading db/schema.rb directly (db:load) instead.
You have two options:
continue to use the migration for production and development; for the test environment, add your constant data to the corresponding yml files in db/fixtures
leave the existing db/fixtures files untouched, and create another set of yml files (containing the constant data) in the same vein as db/fixtures, but usable by both test and production/development environments when doing a rake db:load schema initialization
To cover those scenarios that use db:load (instead of db:migrate - e.g. test, bringing up a new database on a new development machine using the faster db:load instead of db:migrate, etc.) is create a drop-in rakefile in RAILS_APP/lib/tasks to augment the db:load task by loading your constant intialization data from "seed" yml files (one for each model) into the database.
Use the db:seed rake task as an example. Put your seed data in db/seeds/.yml
#the command is: rake:db:load
namespace :db do
desc 'Initialize data from YAML.'
task :load => :environment do
require 'active_record/fixtures'
Dir.glob(RAILS_ROOT + '/db/seeds/*.yml').each do |file|
Fixtures.create_fixtures('db/seeds', File.basename(file, '.*'))
end
end
end
To cover the incremental scenarios (db:migrate), define one migration that does the same thing as the task defined above.
If your seed data ever changes, you will need to add another migration to remove the old seed data and load the new one instead, which may be non-trivial in case of foreign-key dependencies etc.
Take a look at my article on loading seed data.
There's a number of ways to do this. I like a rake task called db:populate which lets you specify your fixed data in normal ActiveRecord create statements. For getting the data into tests, I've just be loading this populate file in my test_helper. However, I think I am going to switch to a test database that already has the seed data populated.
There's also plugin called SeedFu that helps with this problem.
Whatever you do, I recommend against using fixtures for this, because they don't validate your data, so it's very easy to create invalid records.