I'm currently learning Rails and building a blog app. I can't wrap my head around migrations. I'm assuming from what i've read, that you can use migrations to add columns, tables to a database. But why is that important/ can that not be accomplished by version control?
I appreciate any feedback/guidance. I am pretty new to rails/ActiveRecord and so please bear with me.
Migrations work for databases as a Git repo works for code.
It keeps track of the changes made to the database during the development of the app.
Without migrations, you would have to manually do all the changes on every device you share the code.
Migrations and version control are two entirely different things. Migrations do not maintain states. Migrations are a way of altering your database / model, and they do a lot more than 'add columns' as you said it.
Everyone should review rails guides when getting started.
Version control doesn't just maintain your database structure. Let's say you change your database, and views and controllers and decide, and then decide you only want to undo your database change. Reverting your version control would undo all your changes. That's bad I think
With Orchard CMS 1.6, I have four environments set up: Development, Testing, Staging, and Production.
I have made a lot of changes from the Orchard UI Dashboard in Development and I want to migrate those changes into other environments. A related question shows this can be done manually through the Orchard Dashboard using Import/Export modules, but I'm looking for a solution for data migrations that I can automate. Really, I'm looking to nail step three for SQL Server 2005/2008 database in the accepted answer for this related question: that states, "Migrate your db to production environment."
There is not much documentation out there when it comes to setting up and maintaining Orchard CMS in multiple environments outside of Azure and I have 88 tables in my current Orchard database. Obviously, anything I can do to automate data deployments would be a huge help.
I ran a Schema and Data compare between Development and Testing(which currently reflects Production). After backing up databases and replicating schema, I noticed there are a lot of data differences in almost every table. Before migrating data, I want to be sure I've isolated the tables I do not want changed. Environmental variables such as connection strings are going to have to stay unchanged.
What tables should persist in their environments?
Right now I am thinking they are:
Orchard_Users_UserPartRecord - I have users in Production I do not want to have anywhere else.
Environmental Data - I have connection strings I put in a table for a custom module that are different for each environment.
Am I missing anything? What other data should be retained in tables on destination environments?
I really wouldn't recommend a database migration: you are going to have huge problems with ids, relationships, etc. Content items are distributed onto many tables, sometimes in ways that are hard to predict, and there's no guarantee that the same id hasn't been attributed on different environments to different items or records. This is why a higher-level procedure like import/export is the recommended approach.
Automating it is absolutely something we want, and there is a feature branch doing exactly that for a future version of Orchard (feature/deployment IIRC).
In the meantime, I'm pretty sure import and export also exist as commands for the CLI, which should enable you to automate it to a degree.
I've heard before that Rails migrations are flawed, but I never really experienced any example of this firsthand until just recently. What I now realize is that if a migration relies on a particular state of the code, then you're in trouble if you try to clone the repo two years later and run all the migrations (as a lot of them will depend on older versions of the code).
I thought this guy had a good idea:
http://www.mattdipasquale.com/blog/2011/02/28/rails-model-safe-migrations/
Is there anything like this: a gem (say) to automatically check out the commit where each migration was added, all the way up to HEAD?
Obviously it wouldn't be a fool-proof system, as it does rely on every migration being legitimately possible from a clean slate at the point it was committed to the repo (I can imagine cases where teams have written migrations that only incidentally work based on things they've done with the database completely outside of version control). But it would certainly be better than nothing.
I think there are 2 cases that could happen, and in each case, the solution is different:
You develop the code and deliver it in small release increments. The solution is running somewhere, has data in it, and you have to migrate it every time you deliver a new version.
==> Rails is then a perfect answer how to develop and deliver the new releases. I do that all the time (with 2 applications where I am the only user), and never had a problem.
You develop the code, and deliver a lot of small releases (with the first approach). You then want to instantiate it on a new server, without any data stored there.
==> Then dumping your scheme and loading it on the new server is the best way, just to ensure that everything is in place.
I do not know Capistrano, perhaps there are options to do it differently. So if you have something like the scenario 1, use the Rails approach, in case of scenario 2, use the dump-and-load approach.
Great reason to version your database schema.rb.
The git changelog will show when the schema.rb changed, and give you the commit hash.
I am reasonably new to Ruby on Rails so I am not sure how to implement this. My understanding is that rails is not designed with multiple databases in mind, although I could use establish_connection etc to make it work.
My main problem is:
I have an SaaS/application that will serve several businesses. Each
business will have several database tables such as: users, comments,
messages, transfers, navigation history, logs etc. It seems I have 3
options:
1: Store everybody's data in one database with every object belonging_to a business or just tagging something like a businessID/name. Use this tag to fetch the appropriate data and worry about scaling/performance later as my app grows. (Would I have to worry about this pretty early on?)
2: One database per Business. No need to store associations, and db queries perform consistently throughout the application's life (possibly bad assumption here).
3: Have separate instances of my app each running some number of businesses (not sure this is any good).
What I have seen used in other frameworks/businesses is just (2) multiple dbs.
I am also really interested is what is the best practice in rails as well. I know several applications have this same problem and hearing how this has been solved will help.
Any help is much appreciated. Thank you so much.
Env.
Ruby 1.9.2
Rails 3.1
Production:Heroku or EY (still deciding, now running on heroku)
According to this page, You'd need to apply some metaprogramming for multiple databases.
Why not make your deployment script to deploy to different directories with different database settings? One branch per business? Might require some more maintenance, but allows for per-business code if you need it.
As I move through the iterations on my application*(s) I accumulate migrations. As of just now there are 48 such files, spanning about 24 months' activity.
I'm considering taking my current schema.rb and making that the baseline.
I'm also considering deleting (subject to source control, of course) the existing migrations and creating a nice shiny new single migration from my my current schema? Migrations tend to like symbols, but rake db:schema:dump uses strings: should I care?
Does that seem sensible?
If so, at what sort of interval would such an exercise make sense?
If not, why not?
And am I missing some (rake?) task that would do this for me?
* In my case, all apps are Rails-based, but anything that uses ActiveRecord migrations would seem to fit the question.
Yes, this makes sense. There is a practice of consolidating migrations. To do this, simply copy the current schema into a migration, and delete all the earlier migrations. Then you have fewer files to manage, and the tests can run faster. You need to be careful doing this, especially if you have migrations running automatically on production. I generally replace a migration that I know everyone has run with the new schema one.
Other people have slightly different ways to do this.
I generally haven't done this until we had over 100 migrations, but we can hit this after a few months of development. As the project matures, though, migrations come less and less often, so you may not have to do it again.
This does go against a best practice: Once you check in a migration to source control, don't alter it. I make a rare exception if there is a bug in one, but this is quite rare (1 in 100 maybe). The reason is that once they are out in the wild, some people may have run them. They are recorded as being completed in the db. If you change them and check in a new version, other people will not get the benefit of the change. You can ask people to roll back certain changes, and re-run them, but that defeats the purpose of the automation. Done often, it becomes a mess. It's better left alone.
I think that there are two kinds of migrations:
those you made during design/development, because you changed your mind on how your db should be like;
those you made between releases, reflecting some behaviour changes.
I get rid of the first kind of migrations as soon as I can, as they do not really represent working releases, and keep the second kind, so that it is possible, in theory, to update the app.
About symbols vs strings: many argue that only strings should be used in migrations: symbols are meant to be "handles" to objects, and should not be used to represent names (column and table names, in this case). This is a mere stylistic consideration, but convinced me, and I'm no more using symbols in migrations.
I've read of another point for using strings: "ruby symbols are memory leaks", meaning that, when you create a symbol, it never gets disposed for all the application life time. This seems quite pointless to me, as all your db columns will be used as symbols in a Rails (and ActiveRecord) app; the migrating task, also, will not last forever, so I don't think that this point actually makes sense.
The top of schema.rb declares:
# This file is auto-generated from the current state of the database. Instead of editing this file,
# please use the migrations feature of Active Record to incrementally modify your database, and
# then regenerate this schema definition.
#
# Note that this schema.rb definition is the authoritative source for your database schema. If you need
# to create the application database on another system, you should be using db:schema:load, not running
# all the migrations from scratch. The latter is a flawed and unsustainable approach (the more migrations
# you'll amass, the slower it'll run and the greater likelihood for issues).
#
# It's strongly recommended to check this file into your version control system.
I must endorse what [giorgian] said above about different migrations for different purposes. I recommend cleaning up development-oriented migrations along with other tasks you do when you branch for a release. That works for well for me, for myself and small teams. Of course my main app sits atop and between two other databases with their own schemas which I have to be careful of so we use migrations (rather than schema restore) for a new install and those need to survive release engineering.
Having lots of migrations are a good thing. Combined with your version control system, they allow you to see what developer made a change to the database and why. This helps with accountability. Removing them just makes this a big hassle.
If you really want to get a new database up and running quickly you can just load the schema with rake db:schema:load RAILS_ENV=your_environment and if you want to get your test database setup quick you can just use rake db:test:prepare
That being said, if you really want to consolidate your migrations then I'd create a new migration that checks to see if the very last migration in your set has been performed (ex: does the column you added exist?) and if not, then it will fire. Otherwise the migration will just add itself to the schema table as completed so it doesn't attempt to fire again.
Just communicate what you're doing to the rest of your team so that they understand what is going on lest they blindly fire off a rake db:migrate and screw up something they already had.
Although I'm sure everyone has their own practices, there's a few rules implied by the way the migration system works:
Never commit changes to migrations that may have been used by other developers or previous deployments. Instead, make an additional migration to adjust things as required.
Never put model-level dependencies in a migration. The model may be renamed or deleted at some point in the future and this would prevent the migration. Keep the migration as self-contained as possible, even if that means it's quite simplistic and low-level.
Of course there are exceptions. For example, if a migration doesn't work, for whatever reason, a patch may be required to bring it up to date. Even then, though, the nature of the changes effected by the migration shouldn't change, though the implementation of them may.
Any mature Rails project will likely have around 200 to 1000 migrations. In my experience it is unusual to see a project with less than 30 except in the planning stages. Each model, after all, typically needs its own migration file.
Collapsing multiple migrations into a single one is a bad habit to get into when working on an evolving piece of software. You probably don't collapse your source control history, so why worry about database schema history?
The only occasion I can see it as being reasonably practical is if you're forking an old project to create a new version or spin-off and don't want to have to carry forward with an extraordinary number of migrations.
You shouldn't be deleting migrations. Why create the extra work?
Migrations essentially are a set of instructions that define how to build the database to support your application. As you build your application the migrations record the iterative changes you make to the database.
IMHO by resetting the baseline periodically you are making changes that have the potential to introduce bugs/issues with your application, creating extra work.
In the case where a column is mistakenly added and then needs to be removed sometime later, just create a new migration to remove extra column. My main reason for this is that when working in a team you don't want your colleagues to have to keep rebuilding their databases from scratch. With this simple approach you (and they) can carry on working in an iterative manner.
As an aside - when building a new database from scratch (without any data) migrations tend to run very quickly. A project I am currently working on has 177 migrations, this causes no problems when building a new database.