We are developing a large ASP MVC web application EF6 Code First (100+ entities) and we have some doubts about migrations.
If we enable migrations while we are developing but not yet in production, we are going to end up with tons of migration files, since it is recommendable to do small migrations. Is this a correct approach or we should enable DropCreateIfModelChanges for developing before the app is in any client?
Also, we need to seed the database with common data all the clients using the app are going to need, like countries, states, a few super users...
Where do we have to seed this data so it is always in every client we install the web app?
I have seen a few different approaches:
- Seed data in the Seed function of Migrations Configuration, with a AddOrUpdate command, but I have seen in some places that this is a bad approach.
- Seed data in a Add-Migration using Sql("INSERT INTO....");
Thanks in advance.
I usually start development just using the initializers. There is a seed method specifically for initializers that only runs when the database is created which is ideal for security and lookup tables. (Migration initializers run with every update-database).
When I get to the point where I don't want to lose other data, I switch to migrations. As you mention, these can accumulate so I use Chris's technique to roll them up before deployment.
You should also be aware of the issues working with migrations in a team environment.
Related
Perhaps a stupid question, and without code, so I'm not sure I'm on the right StackExchange site. If so, sorry and give me a comment please.
I begin programming in Sinatra (only intranet until now) and in the samples they almost always use migration with activerecord, the same with RoR.
I have experience enough with activerecord itself and it is very helpfull but I'm not sure why migration is always used ? If I create a project I just create a SQL script or a Ruby activerecord script that does the creation of the DB for me and that's it..
If I need the site or script somewhere else I just need to execute that script and ready.
Obviously I'm missing here a lot, so who can me explain the other benefits or point me to a good explanation ?
From Rails docs:
Migrations are a convenient way to alter your database schema over
time in a consistent and easy way. They use a Ruby DSL so that you
don't have to write SQL by hand, allowing your schema and changes to
be database independent.
So, the main two benefits are:
It's like Git for db schema, you won't know how that's useful until you are in a mid-size or big project with many contributors and someone makes a bobo :)
You write ruby code to build your db schema, this way if you decide to move from mysql to pg for example, you don't need to open up pg manual and check code compatibility
Update
In the api docs of migrations, you will find many nice use cases (to be honest i didn't know about half of them) ... check it out (http://api.rubyonrails.org/classes/ActiveRecord/Migration.html)
Building a creation script for your database is great, provided two things:
All your database deployments are on new machines
You get your database schema exactly right the first time
In our modern agile environment we not only don't believe it is possible for a project larger than a few lines of code, we also don't encourage aspiring to it.
Active Record's Migrations offer a way to describe your database incrementally. Each "migration" defines how to add its features to the database ("upgrade"), and how to remove them if necessary ("downgrade").
When running migrations, on top of actually running the scripts, Active Record also maintains its own table (schema_migrations), to remember which migrations have been applied to the database, and which are pending.
This enables you to build your database alongside the features as you develop them. It also facilitates working in teams, since each team member develops her own migrations, and Active Record "stitches" everything together, so you don't have to maintain a monolithic creation script.
I have a Rails app and a Sinatra app, sharing the same database. The Sinatra app uses ActiveRecord.
Can I run migrations from within each app, as if they were in the same app? Will this cause any problems?
The schema.rb file in the Rails app tracks the current migration via
ActiveRecord::Schema.define(:version => 20121108154656) do
but, how does the Sinatra app know the current version the database?
Rails 3.2.2, Ruby 1.9.3.
The version column in the schema_migrations table equate to the time stamp on the front of the ruby migration file example: 20130322151805_create_customers.rb So if two ore more applications are contributing to the schema_migrations table roll backs will not be possible if rails can't find the down() method (because it will not find a migration file contained in another app ie db/migrate/...)
I have a current situation that is exactly this and I have opted to have a master ActiveRecord app that manages migration and data conversions as our database evolves. Keep in mind that part of the deal is to keep models up to date as well. This has been time consuming so we are considering breaking apart the DB in to business domains and providing APIs (JSON) to query support data for another application. This way each application manages it domain and is responsible for exposing data via API.
regards.
If you connect both applications to the same database you should be able to run migrations on it but I strongly suggest you use another option since you will almost surely hit a wall at one time or another:
split the database in two if possible with each application responsible for its own database /migrations.
have one application considered the "master" database and use another database for the data specific to the second application but make it connects to both database (each application still only apply migrations to one database)
If you need to share data between multiple applications another option is to implement a REST service in one and use it on the other, you can have a look at the grape gem for a simple way of doing so.
Edit: I realize I forgot to speak about the activerecord migration, there is no longer any "version" of the schema, what activerecord does is that it read all your migration filename, extract their identifier (the starting part) and check if they have already been applied so in theory you can run migrations from two applications on the same database provided they don't interfere.
But if both migrations act on the same tables you will almost certainly run into big troubles at one point.
I disagree with Schmurfy, even if his presented options are valid, its a bit of an overkill to share data through REST (granted, its pretty easy to implement with ruby / rails).
If your case is simple you could just use one database from both apps, and since you use AR in both of them you have no problems with versioning, AR takes care of that.
Also i dont know what happens if you run db:migrate from both apps simultaniously if you use a inferior dbms like mysql which does not allow DDL in a transaction, certainly nothing good..
Also it would bother me to look which app needs what column and not have the migrations in one place. You could use a shared repository to manage the migrations from both apps.
Rails migrations store current database version in schema_migrations table in the database. So both of your apps will be able to check the current version.
The version numbers are timestamps, so there shouldn't be any problem with duplicate values, as it'll be almost impossible to generate two migrations at the exact same millisecond. So you should be fine here.
The only problem I see is that when you rollback a migration in one app, it'll set the db to the previous known version and I'm not sure if it will pick the previous one from the db (which could be from the other app), or the number from the previous migration file. You may want to test that scenario to make sure.
I decided to put all migrations in the Rails app because:
Since there is only one database
Rails manages migrations
This has worked well.
This simplifies the system because all migrations are stored in one place. And, the Sinatra app doesn't need to know about them anyway.
When developing a Rails app with a legacy schema that is in-use by an existing application, if tables have NOT NULL constraints on foreign id columns throughout the schema, in order to create/save models for tests, models need to exist for those associations, and their associations, etc. So, it isn't an easy as just creating one model at a time as you need it and testing with it.
As far as testing goes, this seems to be a problem if you are using FactoryGirl and want to create and save model instances to return from controllers, etc. when all of the association dependencies are involved. Another option is to mock, but mocking can be a little more time consuming and it doesn't allow you to do integration testing as easily. Another is to use fixtures, but those are time consuming and brittle. Another is to pre-populate the test database with production data, but that doesn't solve the need for factories, etc./known data to expect in tests, and Rails usually expects to start with a clean DB for the test environment.
What strategies for developing models, tests, etc. do you use when you have an existing complex schema that you are bolting a Rails app onto- not just for reading data, but also writing to the existing schema that is in use by an existing production application? (i.e. the "rebuild the ship while at sea" problem)
When we first started out on this problem, I was looking for something that could autogenerate models from the existing schema. I found Dr. Nic's magic model generator, but it only stated that worked in Rails 2.x. I got a Rails 2.x environment running (forget the specific version it worked in) but it didn't seem to help much, so I wrote a script to generate our models. However, when we started development we then had a load of models, so we started to try to move out the ones we didn't need, then needed to comment out associations to models that were no longer there, but some of those were required/NOT NULL foreign keys, so we're now having to move them back in and uncomment those more than we thought we would.
We did write and uncover some things that were helpful during the process, not all of which we've used, but they might be helpful to others:
https://github.com/influitive/apartment
https://github.com/karledurante/secondbase
https://github.com/drnic/composite_primary_keys
https://github.com/garysweaver/activerecord-define_nils
https://github.com/garysweaver/activerecord-native_db_types_override
https://github.com/garysweaver/modelist
https://github.com/garysweaver/stepford
https://github.com/garysweaver/structure_sql_to_migration
As for test data setup, I started writing a tool to automate the development of FactoryGirl factories called Stepford and wrote Modelist to help us test models, resolve circular dependencies, and identify model dependencies, so we don't have to include all automatically generated models.
So far really, the only answer I've come up with and have heard other say also is that rebuilding an existing application, even piecemeal, with different technologies is difficult, slow, and error-prone.
I'm a PHP programmer for over a decade and making the move to RoR. Here is what I'm used to from the PHP world:
Create DB schema in a tool like MySQL WorkBench -- and make fields precisely the size I want without wasting space (e.g. varchar(15) if it's ip_address).
Write models using Datamapper and place those exact field lengths and specifications in there so my app doesn't try to put in any larger values.
In the RoR world from what I've seen over the past two days, this seems to be the flow suggested:
Add fields / schema using the command line which creates a migration script and apparently created large ass fields (e.g. "ip_address string" is probably making the field varchar(255) in the db when I run the migration).
Put in validations during model creation.
Am I missing something here? What's the process in the RoR world for enterprise level applications where you actually want to create a highly customized schema? Do I manually write out migration scripts?
The scaffolding is what you use to get started quickly. But before running the migration, you can edit it and add constraints and specific column lengths.
Validations specified in the model (in the ruby code) does not carry the same level of security as validations /constraints specified on the database. So you still need to define those on the database.
While it is possible to work with Rails without migrations, I would strongly advice against it. In some cases it cannot be avoided (when working with legacy databases for instance).
The biggest advantage of using the migrations is that your database schema, accross different platforms, can be held in sync through different stages. E.g. your development and your production database. When deploying your code, the migrations will take care that the database is migrated correctly.
You can edit the migration scripts before you run the migration in order to customize the fields.
Yes, if you need to tweak the defaults, you edit the migration scripts.
Also note that you don't need to use migrations, they're a "convenience" while iterating through DB development. There's nothing that says you must use them. The active record pattern doesn't rely on how the DB tables/fields/etc. are created or defined.
For example, migrations are useless when dealing with legacy DBs, but you can still write a Rails app around them.
I'm switching to RoR from ASP.NET MVC. Yeah, migrations are cool, but I do not need to use different databases in my web applications. Postgresql will do just fine.
So is it okay if I use PGAdmin to create and administer my databases and schema and avoid all these fancy migrate, rake etc?
Update
Thanks everyone! Now I better understand what migrations are, and why I should use them.
I don't think that's what migration means.
Migrations in rails (and in other frameworks) is a method by which you can use to update your database schema when there are multiple versions of the same database running
For example, you may have two databases, one running on your production server, and another running locally for development. After a few days of coding, your local development database may looks a bit different. With migrations, you can simply push your code to the production server and then run the migrations to automatically update your production database so it is up-to-date with the one you use locally for development.
So, to answer your question, Yes it is OK but you might not get a few of the migrations niceties when the time comes that you'll have to maintain multiple versions of your database.
Have to agree with charkit but one (rather two) important note why you should use migrations: Migrations don't make up the model definitions. They are stored seperately in a file schema.rb. This defines the rows and tables of your database. When looking into the file, you find these lines:
This file is auto-generated from the current state of the database. Instead of editing this file, please use the migrations feature of Active Record to incrementally modify your database, and then regenerate this schema definition.
The second reason is for testing: you can easily set up a test database to run all your tests against without the need to touch the "real" database. I know when developing, this is not a big problem but this will get more important after some time.
So, yes, it is possible to use PGAdmin to create all your database related stuff but you should not forget to always keep the schema file up to date and come up with a solution for testing.
With migrations you're able to develop your database schema in Ruby and this is usually database indpendent.
In short, spend the 20 minutes or so to really get migrations and the value they add. Then determine whether or not you want to ditch them. Strangely for me I learned Rails before I started my first MVC project; one of the things I missed most was migrations.
From a technical standpoint you should be fine without them.