What I want to do is to dump the database into a custom created .rb file.
I found a seed_dump gem that allows me to do this:
rails db:seed:dump FILE=db/seeds/my_db_file_name.rb
Then I noticed that my datebase is our of order, so I found this on SO to include the ids:
rails db:seed:dump FILE=db/seeds/my_db_file_name.rb EXCLUDE=[]
Seemed fine until I wanted to add new record to my database. Turned out that reseting primary keys solved the problem:
def reset_pk
ActiveRecord::Base.connection.tables.each do |t|
ActiveRecord::Base.connection.reset_pk_sequence!(t)
end
redirect_to root_url
end
What I am now want to do is to simplify the dumping process, as for now, every time I dump the data base the records are "out of order", what I will explain below.
Let's assume I have a two models: Lab and Offer. Lab can have many offers. So in order to create an Offer object I first have to create a Lab object. But when I dump the schema my file looks like this:
Offer.create...
Offer.create...
Offer.create
Lab.create...
Lab.create...
Lab.create...
and if I try to seed it, it won't do it as Offers are created before Labs, and it should be the other way.
My question is, is there a way to actually keep the relationships while dumping the database so Labs get created first?
EDIT
I managed to do something like this:
rails db:seed:dump FILE=db/seeds/my_db_file_name.rb EXCLUDE=[] MODELS="Lab, Offer"
This one keeps the order as I want it, but I wonder if there is a simple way (in case of having 15, 20 models instead of just 2).
You could try dumping/restoring one table at a time in the order you wish.
Related
I have a join table in rails that has a few entries that need to be deleted.
lets say the join table is named 'products_variants'
I found out i have a few entries in this join table that were created erroneously a while ago. I have their IDs, so i could go in phpmyadmin and delete them, but I want to make a migration to do it in case anyone uses an older database (which has happened before).
Since I don't have a ruby object representing this join table I cant do something like:
ProductsVariants.find(id_array)
How would i go about deleting these entries in a rails migration?
You can create AR class for this table inside of migration and use it for delete record.
How would you do it from the console? Whatever that is, put it in the migration. For example
class Cleanup < ActiveRecord::Migration
def up
execute("delete from product_variants where id in (1,2,3)")
end
def down
end
end
Barring a solution like maxd's answer, you can also delete them via plain 'ol SQL. If you already have the list of ids, you can do something like this:
ActiveRecord::Base.connection.execute("DELETE FROM products_variants WHERE id IN (...)")
Where ... is the list of ids to delete.
Semi-pointless side-note: Technically speaking, data manipulation is not typically done in the migrations for various reason; one of them being that you're usually not necessarily guaranteed that all (or even any) migrations will be run by your colleagues (speaking very generally here), or in the case of new local project setups (meaning, you've just pulled down the project code and are setting it up locally for the first time).
While it doesn't sound like this is an issue in your scenario, if you want to do this the Rails-y way, one alternative would be to put this in a Rake task, so that you or others can execute it as needed, rather than relying on the migrations.
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
I am working on an application that is already deployed to some test and staging systems and various developers workstations. I need to add some additional reference data but i'm not sure how to add it.
Most of the advice says use seed.rb, however my understanding is that this is only run once, when the application is initially deployed. Since we don't want to rebuild the test and staging databases just so that we can add 1 row of reference data, is there another way to add the data?
I'm thinking of using a db migration, is this the correct approach?
Thanks
Structure your seed.rb file to allow ongoing creation and updating of data. You are not limited to running a seed file only once and if you think it's only used for initial deployment you will miss out on the flexibility it can offer in setting reference data.
A seed file is just ruby so you can do things like:
user = User.find_or_initialize_by(email: 'bob#example.com')
user.name = 'Bob'
user.password = 'secret'
user.role = 'manager'
user.save!
This will create new data if it doesn't exist or update the data if it finds some.
If you structure your seed file correctly you can also create and update dependent objects.
I recommend using the bang save to ensure that exceptions are raised in the event that an object cannot be saved. This is the easiest method of debugging the seed.
I use the seedbank gem to provide more structure to my seed data, including setting data per environment, dependent seeds and more.
I don't recommend using migrations for seed data. There is a lack of flexibility (how do you target seed data to just one environment for instance) and no real way to build up a reusable set of data that can be run at any time to refresh a particular environment. You would also have a set of migrations which have no reference to your schema and you would have to create new migrations every time you wanted to generate new or vary current data.
You can use a migration, but that's not the safest option you have.
Say, for example, you add a record to a table via a migration, then in the future you change that table's schema. When you'll install the app somewhere, you won't be able to run rake db:migrate.
Seeds are always advisable because rake db:seed can be run on a completely migrated schema.
If it's just for a record, go for the rails console.
It's best to use an idempotent method like this in seed.rb or another task called by seed.rb:
Contact.find_by_email("test#example.com") || Contact.create(email: "test#example.com", phone: "202-291-1970", created_by: "System")
# This saves you an update to the DB if the record already exists.
Or similar to #nmott's:
Contact.find_or_initialize_by_email("test#example.com").update_attributes(phone: "202-291-1970", created_by: "System")
# this performs an update regardless, but it may be useful if you want to reset your data.
or use assign_attributes instead of update_attributes if you want to assign multiple attributes before saving.
I use the seed file to add instances to new or existing tables all the time. My solution is simple. I just comment out all the other seed data in the db/seeds.rb file so that only the new seed data is live code. Then run bin/rake db:seed.
I did something like this in seed.rb
users_list = [
{id: 1, name: "Diego", age: "25"},
{id: 2, name: "Elano", age: "27"}
]
while !users_list.empty? do
begin
User.create(users_list)
rescue
users_list = users_list.drop(1) #removing the first if the id already exist.
end
end
If a item in the list with the given id already exist it will return a exception, then we remove that item and try it again, until the users_list array is empty.
This way you don't need to search each object before include it, but you will not be able tho update the values already inserted like in #nmott code.
Instead of altering seeds.db, which you probably want to use for seeding new databases, you can create a custom Rake task (RailsCast #66 Custom Rake Tasks).
You can create as many Rake tasks as you want. For instance, lets say you have two servers, one running version 1.0 of your app, the other one running 1.1, and you want to upgrade both to 1.2. Then you can create lib/tasks/1-0-to-1-2.rake and lib/tasks`1-1-to-1-2.rake since you may need different code depending on the version of your app.
in the application i am currently creating in ruby on rails. I am trying to do some tests in rails console where i have to destroy data in the database and the database is connected to a server. I am importing an XML and parsing it and putting it into a database with scaffolding.
Now what i need: Basically what i am attempting to do is to destroy the data and replace it with a new one every week..but the problem i am getting, the userid is gone up to 700+ and there are only 50 records :S cause it doesnt reset...
To delete all records i am currently using "whatever.destroy_all" does the trick
Any help?
Btw i am using SQLITE
The ID column created in the table usually is set as unique and to increment by 1 for each new record, which is why each time you destroy and add new data the ID keeps getting higher.
The fact that the ID # is getting larger and larger is not an issue at all.
If you really want to start back at zero, I would think you could drop the table and recreate it, but that seems like overkill for a trivial issue.
Regarding the connection to the other scaffold, how are you connecting the two and what do they both represent?
Ideally the data population for testing should be done through fixtures (or easy tools such as factorygirl etc..)
The main advantage of having a fix data set is you can run your tests in any environment. But as per your requirement you can do something like this,
When you populate the date through the active records pass the id parameter as well
Ex: User.new(:id => 1, :name => "sameera").create
By this way you can have constant id's But make sure you increment the id accordingly.
I have a standard rails application, that uses a mysql database through Active Record, with data loaded through a separate parsing process from a rather large XML file.
This was all well and good, but now I need to load data from an Oracle database, rather than the XML file.
I have no control how the database looks, and only really need a fraction of the data it contains (maybe one or two columns out of a few tables). As such, what I really want to do is make a call to the database, get data back, and put the data in the appropriate locations in my existing, Rails friendly mysql database.
How would I go about doing this? I've heard* you can (on a model by model basis) specifiy different databases for Rails Models to use, but that sounds like they use them in their entirety, (that is, the database is Rails friendly). Can I make direct Oracle calls? Is there a process that makes this easier? Can Active Record itself handle this?
A toy example:
If I need to know color, price, and location for an Object, then normally I would parse a huge XML file to get this information. Now, with oracle, color, price, and location are all in different tables, indexed by some ID (there isn't actually an "Object" table). I want to pull all this information together into my Rails model.
Edit: Sounds like what I'd heard about was ActiveRecord's "establish_connection" method...and it does indeed seem to assume one model is mapped to one table in the target database, which isn't true in my case.
Edit Edit: Ah, looks like I might be wrong there. "establish_connection" might handle my situation just fine (just gotta get ORACLE working in the first place, and I'll know for sure... If anyone can help, the question is here)
You can create a connection to Oracle directly and then have ActiveRecord execute a raw SQL statement to query your tables (plural). Off the top of my head, something like this:
class OracleModel < ActiveRecord::Base
establish_connection(:oracle_development)
def self.get_objects
self.find_by_sql("SELECT...")
end
end
With this model you can do OracleModel.get_objects which will return a set of records whereby the columns specified in the SELECT SQL statement are attributes of each OracleModel. Obviously you can probably come up with a more meaningful model name than I have!
Create an entry named :oracle_development in your config/database.yml file with your Oracle database connection details.
This may not be exactly what you are looking for, but it seems to cover you situation pretty well: http://pullmonkey.com/2008/4/21/ruby-on-rails-multiple-database-connections/
It looks like you can make an arbitrarily-named database configuration in the the database.yml file, and then have certain models connect to it like so:
class SomeModel < ActiveRecord::Base
establish_connection :arbitrary_database
#other stuff for your model
end
So, the solution would be to make ActiveRecord models for just the tables you want data out of from this other database. Then, if you really want to get into some sql, use ActiveRecord::Base.connection.execute(sql). If you need it as a the actual active_record object, do SomeModel.find_by_sql(sql).
Hope this helps!
I don't have points enough to edit your question, but it sounds like what you really need is to have another "connection pool" available to the second DB -- I don't think Oracle itself will be a problem.
Then, you need to use these alternate connections to "simply" execute a custom query within the appropriate controller method.
If you only need to pull data from your Oracle database, and if you have any ability to add objects to a schema that can see the data you require . . . .
I would simplify things by creating a view on the Oracle table that projects the data you require in a nice friendly shape for ActiveRecord.
This would mean maintaining code to two layers of the application, but I think the gain in clarity on the client-side would outweigh the cost.
You could also use the CREATE OR REPLACE VIEW Object AS SELECT tab1., tab2. FROM tab1,tab2 syntax so the view returned every column in each table.
If you need to Insert or Update changes to your Rails model, then you need to read up on the restrictions for doing Updates through a view.
(Also, you may need to search on getting Oracle to work with Rails as you will potentially need to install the Oracle client software and additional Ruby modules).
Are you talking about an one-time data conversion or some permanent data exchange between your application and the Oracle database? I think you shouldn't involve Rails in. You could just make a SQL query to the Oracle database, extract the data, and then just insert it into the MySQL database.