Hi I have a sqlite3 database full of data from my previous version of the web app which WASN'T written in rails. Im now rewriting the web app in rails from scratch. But Id like to use the data from my previous app in the new rails app. Whats the best way to accomplish this?
This is what Ive tried so far and it didn't work very well:
1) I created a new rails app
2) replaced my sqlite3 database in the new app with the sqlite3 database from the old app
3) Created a model with the same schema as the old DB.
4) changed the databse.yml file with the updated DB file details
5) un my model added the "establish_connection" method
6) I could thus get it to show me all the details of the old database in my browser #"index.html"
7) However, I ran into problems when wanting to insert/edit records into the DB. Since the DB did not have a primary key column for each row, it didnt work.
8) So I tried to do a migration to add a column with a primary key . It didnt work
9) suddenly a new development.sqlite3 database had shown up in the app and it was trying to add a primary key to the NEW DB.
10) So I just deleted the new DB that had popped up and after that the app wasn't working
11) Now I want to start from scratch and so my question:
"Whats the best way to import data from a previous non-rails app (in a sqlite3 DB format) into a new rails app"
You want to be working with 2 databases, the old and the new. So, your database.yml needs to have 2 connections in it. One should be your normal connection called development or production and the other should be called legacy. Fill out their different connection information. By default, your ActiveRecord models will pull from the one not called legacy.
Create a model called LegacyBase. In the model put establish_connection "legacy". Now create a directory at app/models/legacy. Put all of the models represent data from the old DB inside this directory. All of those models should extend from LegacyBase. This will mean they're all reading from the old DB.
Create your new models. Do you want them to have a primary key column? If not, see this answer. Create an ActiveRecord database table with no :id column?. They will by default have an id column, though and I recommend leaving it that way.
For each of the legacy models, write a method called to_model. In this method, write the code that creates a new object and populate it with the old data and save it. Something like this:
class OldUser < LegacyBase
establish_connection "legacy"
def to_model
User.create!(self.attributes)
end
end
You can do any logic that needs to happen to make the old data fit your new app. Call this method on all of the old records like OldUser.all.map(&:to_model).
Do this for all of the tables that you want to move over.
Related
I have a join table in rails that has a few entries that need to be deleted.
lets say the join table is named 'products_variants'
I found out i have a few entries in this join table that were created erroneously a while ago. I have their IDs, so i could go in phpmyadmin and delete them, but I want to make a migration to do it in case anyone uses an older database (which has happened before).
Since I don't have a ruby object representing this join table I cant do something like:
ProductsVariants.find(id_array)
How would i go about deleting these entries in a rails migration?
You can create AR class for this table inside of migration and use it for delete record.
How would you do it from the console? Whatever that is, put it in the migration. For example
class Cleanup < ActiveRecord::Migration
def up
execute("delete from product_variants where id in (1,2,3)")
end
def down
end
end
Barring a solution like maxd's answer, you can also delete them via plain 'ol SQL. If you already have the list of ids, you can do something like this:
ActiveRecord::Base.connection.execute("DELETE FROM products_variants WHERE id IN (...)")
Where ... is the list of ids to delete.
Semi-pointless side-note: Technically speaking, data manipulation is not typically done in the migrations for various reason; one of them being that you're usually not necessarily guaranteed that all (or even any) migrations will be run by your colleagues (speaking very generally here), or in the case of new local project setups (meaning, you've just pulled down the project code and are setting it up locally for the first time).
While it doesn't sound like this is an issue in your scenario, if you want to do this the Rails-y way, one alternative would be to put this in a Rake task, so that you or others can execute it as needed, rather than relying on the migrations.
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
in the application i am currently creating in ruby on rails. I am trying to do some tests in rails console where i have to destroy data in the database and the database is connected to a server. I am importing an XML and parsing it and putting it into a database with scaffolding.
Now what i need: Basically what i am attempting to do is to destroy the data and replace it with a new one every week..but the problem i am getting, the userid is gone up to 700+ and there are only 50 records :S cause it doesnt reset...
To delete all records i am currently using "whatever.destroy_all" does the trick
Any help?
Btw i am using SQLITE
The ID column created in the table usually is set as unique and to increment by 1 for each new record, which is why each time you destroy and add new data the ID keeps getting higher.
The fact that the ID # is getting larger and larger is not an issue at all.
If you really want to start back at zero, I would think you could drop the table and recreate it, but that seems like overkill for a trivial issue.
Regarding the connection to the other scaffold, how are you connecting the two and what do they both represent?
Ideally the data population for testing should be done through fixtures (or easy tools such as factorygirl etc..)
The main advantage of having a fix data set is you can run your tests in any environment. But as per your requirement you can do something like this,
When you populate the date through the active records pass the id parameter as well
Ex: User.new(:id => 1, :name => "sameera").create
By this way you can have constant id's But make sure you increment the id accordingly.
Yesterday i was absolutely sertain that all migrations data for EF placed in classes, placed in my solution as nested from DbMigration. But today i was dig a slightly deeper(just try to fallback to old migration with enable data loss not with nu-get and visual studio, but with code())
DbMigrator fg = new DbMigrator(new Settings() { AutomaticDataLossEnabled = true});
fg.Update("MigrationName");
And get exception, smth like "string should be truncated", those means that migrator tried to update column from big to small MaxLength attribute. So, i had excluded migration that caused this update and move this changes to migration, those create tables. The error still was occured. I got to intellitrace and it said that those(deleted) migration still was called. Looking to requests told me things like this:
SELECT [Extent1].[MigrationId] AS [MigrationId] FROM [dbo].[__MigrationHistory] AS [Extent1]
Looking to a table __MigrationsHistory and get my deleted migration there with field model that contains crypted data(don't decrypt this yet) . I was realy shocked. Does this means that all code, have written in classes is just the fake and really executed code placed here? And does anyone know, how to work with this table, register projections of migration classes to it etc. Or the once way to work with migrations is nu-get console?
I am not fully sure what your primary question is, so I will first try to answer last part about __MigrationHistory table.
Code in classes is not fake, your code in classes is compiled and run.
This table, however, really contains your database model, but it is not encrypted, it is compressed. The reason why Migrations API needs to store your model, is to be able to compare it against your current actual model and track changes for you (for example when you add a new property it will be able to tell what property you added and to perform automatic db migration).
In previous version of EF there was an EdmMetadata table where hash of your model was stored, and EF was able to detect if you made some changes to model by comparing stored and current model hash value. New version when migrations are enabled stores entire model as compressed blob, so it can do diff between the model that was used to create database and current model you are using, and make automatic migrations accordingly.
You should not work directly with this table, it is automatically populated by migrations API, but nuget console is not the only way to do migrations, you can check this resource for some insights how to do it from code.
Now, regarding your question from question title (where they are stored?), migrations are stored in code, in a class inheriting from DbMigration class that migrations API creates for you when you do Add-Migration command in nuget console. When you perform an migration (Update-Database), either from nuget package manager console or from code, API will compare your current model with versions in __MigrationsHistory to find initial version (if you have not specified it) and perform all migrations in between initial and target version (if not specified otherwise target is latest version).
I'm not really clear how you did exclude your migration that causes problems, as you need to migrate your database to version before that migration, and then delete and recreate all subsequent migrations from there.
Maybe you could solve your fallback to old version problem by implementing public override void Down() method in your migration that causes problems when trying to rollback? This method can be used to execute code which performs inverse any operations for migration.
Not directly related to question but worth mentioning, there is also pretty detailed tutorial here for EF CF.
Let's say I'm within a rails project in some model named Apple.
I'm going to switch databases to another server so I'll call: ActiveRecord::Base.establish_connection()
I want to access the table 'Banana' in this particular database. I can see that the table exists by calling table_exists?
However, I'm not sure how to access the table... I want to just be able to do something like Banana.find(:all)
How should I work this out?
Have you tried connection_ninja gem? Pretty handy for multi-db connections:
https://github.com/cherring/connection_ninja