I've been having a problem which doesn't seem uncommon (I've read a lot of stack overflow pages in the last 2 days) but every solution I've read hasn't worked for me.
I've been following this video tutorial
At 6:42 the tutor shows the tables in postgresql and mine don't show up.
When I try rake db:migrate the files migrate no problem. rake doesn't throw up any errors, the relevant .rb files are created in the models folder, my schema.rb looks right. It seems postgresql just isn't reading my schema file.
My problem sounds identical to
rake db:migrate doesn't seem to work in production
However typing rake db:migrate RAILS_ENV=production doesn't work. (I've tried this a few times after other rake commands like rake db:rollback STEP=3 in the past few days because I was paranoid it was the solution, this person on ruby forum, has the same problem and is offered the same answer).
This is my first attempt at programming anything and I'm loving the tutorial (and the learning curve this problem has turned out to be) Asking here is pretty much my last resort because I've tried everything I can understand online as a possible solution so please help me! Thanks a lot in advance
There is quite a bit of confusion going on here:
Migrations
Migrations are a convenient way to alter your database schema over
time in a consistent and easy way. They use a Ruby DSL so that you
don't have to write SQL by hand, allowing your schema and changes to
be database independent.
Postgres does not read your schema file or migrations - rather migrations run SQL queries against your database. In this case a CREATE TABLE ... query will be run when the migration is run.
Migrations are basically a more maintainable and sane way of doing what was classically done by opening a DB console and running SQL queries.
config/schema.rb is not actually used by the database or ActiveRecord - rather its created when you run migrations as a snapshot of what the database schema should look like. Its just a developer convenience. ActiveRecord gets its mappings by querying the database schema.
Migrations and generators
Migrations do not create model files either - those are generators such as:
rails g model Dude abides:boolean
Which creates a CreateDudes migration and a model at app/models/dude.rb.
In fact migrations are just concerned about altering the DB schema and don't care if the model file exists or not - the models is not actually used until you query the database for records.
ENV vars
RAILS_ENV=production sets a environmental variable.
rake db:migrate RAILS_ENV=test
Is the documented way to run a migration in a different environment. Some obscure shells require the ENV var to prefix the command.
However - if you are running a production server you should set the RAILS_ENV env var permanently - not on invocation! This prevents embarrassing misstakes when someone expected you to have configured the server properly and just ran rails s when restarting the server. See the documentation for your server OS on how to set env vars.
If you are still running the migration and do not see the expected results you most likely have not configured config/database.yml properly - the migrations are running. But not against the database you would expect.
http://guides.rubyonrails.org/active_record_migrations.html
Related
I'm doing my first project on RoR I decided to use Devise gem for the authentication everything is working fine in Localhost, right now i'm able to sing_up, log_in, reset pass and some other default features that comes with devise.
What I don't understand is how everything gets setup once i push the project into production environment. Let me explain better.
-I see that devise generates db/schema.rb file based on the user.rb model that devise auto generate, these schema i assume needs to be created in the production db.
I have to push the project into git repo but..
1) How the code gets executed once is published
2) How the user table gets created
3) where do I need to specify the db connection (db, user, pass)
In general how is the process to migrate into production env?
I appreciate your help.
In rails db schema is controlled using migrations. Migrations are usually very small files living in db/migrations folder. Each of them defines a single subclass of ActiveRecord::Migration class.
Each migration is supposed to go both-ways, so it has to implement both up and down method (for rollback). In majority of the changes those are obvious (like when up action creates a table, down action should destroy it). For migrations like this you can use change function - rails will create down function automatically (assuming that all actions are reversible - some of the methods even accept extra arguments to make it possible like remove_column :table, :field, :integer. :integer bit might seem redundant here, however the action is irreversible without that information).
You run your migrations using db:migrate rake task. Effectively it will run all the migration which has not been run against current database. When it is done, it will dump your current database into db/schema.rb file. This file is used only as a snapshot of your current database schema, which is of great help if you need to get back to previous version of your application (which is why schema.rb should always be checked into your source control). It is also possible to recreate your database directly from that file rake db:schema:load, however it will cost you all your data.
If you check your database (any environment), you will notice there is one extra table called schema_migrations which does not appear in your schema file. This table contains all the migrations that has been executed against this database. This way rails will not retry migrations that already has been run.
There are many advantages of using migration system:
It is easy for every developer to bring their local database up to the latest schema.
It is easy to rollback your latest changes if deployment failed. (Not always the case)
So how to use it in production? You simply has to point the rake task to the right database:
RAILS_ENV=production rake db:migrate
You can find your database configuration in config/database.yml file. Note that this is the best practice not to store that file in source control and in case of production it is not advisable to store your database password there. Normally you will need to set it through environment variable.
It is important to remember couple of things about migrations:
Never, ever change the existing migration if there is a chance that someone has already run it. Especially, do not modify migrations that has run in production unless you are 100% sure what you are doing (for example fixing rollback). Since database remembers which migrations has run, it might lead to getting your local db out of sync with production db and hence lead to bugs.
Never ever change database schema manually - every change has to be made through migrations.
Well depending on where you're hosting your code you should still do a rake db:migrate to make the changes to the production DB.
If deployed on Heroku, run the following in the console:
heroku run rake db:migrate
Hope that answers your question?
I'm working on deploying my first Rails application right now, and somewhere along the way, I botched a migration. When I try to push my application to the production server and run rake db:migrate, it fails somewhere with an error.
Now, I am way too lazy to work through my migrations individually to find out what went wrong, so I'm trying to avoid doing that. Given that my current development database works just fine, is there a way to "flatten" the current schema into a single, comprehensive migration?
I understand that this is sloppy, and I understand that I likely did a dumb thing to break the migration chain in the first place. (I probably edited the database schema directly somewhere, which I now understand is a no-no.) This is a pretty small project though, and I'm essentially the only developer involved, so I'm comfortable sweeping this issue under the rug if possible.
Is there a way to do that?
Thanks for lending your expertise.
This is what the db/schema.rb file is for. If you've only got structural changes in your migrations you will be able to run rake db:schema:load rather than running rake db:migrate to get the absolute structure for your tables.
If you edited the schema directly you will need to run:
rake db:schema:dump
This will take whatever is in the database and create a schema.rb file. Then you can run rake db:schema:load anytime you want. However it will mean that your migrations are still bad. You could delete all of them and recreate them from the schema.rb file.
Very simple question here - if migrations can get slow and cumbersome as an app gets more complex and if we have the much cleaner rake db:schema:load to call instead, why do migrations exist at all?
If the answer to the above is that migrations are used for version control (a stepwise record of changes to the database), then as an app gets more complex and rake db:schema:load is used more instead, do they continue to maintain their primary function?
Caution:
From the answers to this question: rake db:schema:load will delete data on a production server so be careful when using it.
Migrations provide forward and backward step changes to the database. In a production environment, incremental changes must be made to the database during deploys: migrations provide this functionality with a rollback failsafe. If you run rake db:schema:load on a production server, you'll end up deleting all your production data. This is a dangerous habit to get into.
That being said, I believe it is a decent practice to occasionally "collapse" migrations. This entails deleting old migrations, replacing them with a single migration (very similar to your schema.rb file) and updating the schema_migrations table to reflect this change. Be very careful when doing this! You can easily delete your production data if you aren't careful.
As a side note, I strongly believe that you should never put data creation in the migration files. The seed.rb file can be used for this, or custom rake or deploy tasks. Putting this into migration files mixes your database schema specification with your data specification and can lead to conflicts when running migration files.
Just stumbled across this post, that was long ago and didn't see the answer I was expecting.
rake db:schema:load is great for the first time you put a system in production. After that you should run migrations normally.
This also helps you cleaning your migrations whenever you like, since the schema has all the information to put other machines in production even when you cleaned up your migrations.
Migrations lets you add data to the db too. but db:schema:load only loads the schema .
Because migrations can be rolled back, and provide additional functionality. For example, if you need to modify some data as part of a schema change then you'll need to do that as a migration.
As a user of other ORM's, it always seemed strange to me that Rails didn't have a 'sync and update' feature. ie, by using the schema file (which represents the entire, up-to-date schema), go through the existing DB structure and add/remove tables, columns, indexes as required.
To me this would be a lot more robust, even if possibly a little slower.
I have already posted as a comment, but feels it is better to put the comments of the db/schema.rb file here:
# Note that this schema.rb definition is the authoritative source for your
# database schema. If you need to create the application database on another
# system, you should be using db:schema:load, not running all the migrations
# from scratch. The latter is a flawed and unsustainable approach (the more migrations
# you'll amass, the slower it'll run and the greater likelihood for issues).
#
# It's strongly recommended that you check this file into your version control system.
Actually, my experience is that it is better to put the migration files in git and not the schema.rb file...
rake db:migrate setup the tables in the database. When you run the migration command, it will look in db/migrate/ for any ruby files and execute them starting with the oldest. There is a timestamp at the beginning of each migration filename.
Unlike rake db:migrate that runs migrations that have not run yet, rake db:schema:load loads the schema that is already generated in db/schema.rbinto the database.
You can find out more about rake database commands here.
So schema:load takes the currently configured schema, derives the associated queries to match, and runs them all in one go. It's kind of a one-and-done situation. As you've seen, migrations make changes step-by-step. Loading the schema might make sense when working on a project locally, especially early in the lifetime of a project. But if we were to drop and recreate the production DB each time we do a deployment, we would lose production data each time. That's a no-go. So that's why we use migrations to make the required changes to the existing DB.
So. The deeper into a project you get, the more migrations you'll get stacked up as you make more changes to the DB. And with each migration, those migrations become more and more the source of truth of what's on production - what matters isn't what's in the schema, but what migrations have been run in production. The difference is effectively moot if we have both in sync. But as soon as one goes of out date from the other, you start to have discrepancies. Ideally this would not happen, but we live in the real world, and stuff happens. And if you're using schema:load to set up your DB locally, you might not be getting the actual state of the DB, as it is reflected via the migration history on production.
At some point in my rails development I started making database changes (e.g. dropping or altering columns/tables) without using rails migrations. So now I get errors when I try to deploy my rails app from scratch.
blaine#blaine-laptop ~/tmp/rbjacolyte $ rake db:migrate
(in /home/blaine/tmp/rbjacolyte)
== AddHashToTrack: migrating =================================================
-- add_column(:tracks, :hash, :string)
rake aborted!
An error has occurred, all later migrations canceled:
Mysql::Error: Table 'jacolyte_dev_tmp.tracks' doesn't exist: ALTER TABLE `tracks` ADD `hash` varchar(255)
(See full trace by running task with --trace)
How can I sync my production and development environments with migrations after I've mucked it up by using raw SQL? I want to deploy my rails application without database errors, and I don't want to start from scratch.
The data in the production and development environments match, but the migrations fail. I want a way to 'start from scratch.'
Could I simply delete all of the migrations that I have, and then just start using migrations from now on?
The shortcut way: manually add an entry to schema_migrations for a timestamp that represents a baseline. You can add migrations after that and as long as they don't make any bad assumptions about the db schema they should be able to run just fine. You won't be able to migrate backwards, but that's not a huge problem.
The bigger problem is that you won't be able to make a DB from scratch, which gets to be a pain longer term.
The fix for that is to delete all your existing migrations and create a new one that creates the existing schema. Manually delete everything from the schema_migrations table and put in an entry for this one new migration. After that, you can create new migrations that build on this new baseline and they should apply just fine. You should be able to bootstrap new databases in the normal fashion.
As long as your direct SQL is contained in Rails migrations, there's no problem with using it. Just make sure you implement both the #up and #down methods and you should be good. We've actually taken to using raw SQL as a best practice to avoid problems when models are changed later on. Something like
Foo.create(:name => 'bar')
seems innocuous, until the User model is modified to have
validates_presence_of :baz
At which point the new migration will run against an existing database, but that earlier migration that created the table and added the dummy entry will fail because User fails validation. Just using
execute("insert into foos (name) values ('bar')")
will work fine as long as the later migrations properly populate any new columns they add.
Maybe you could just get rid of all your current migrations, and use rake db:schema:dump to create a new schema.rb file, and manually edit your production database to reflect the changes you've made so far?
I like Veeti's suggestion, with a modification: rake db:schema:dump, then move that file to your development machine. Flatten your Rails migrations so far (see this SO thread on that), get rid of most of your migrations, and re-work your migrations to work, given your new schema.
Get this working on your dev machine, commit and deploy.
If the existing production data is compatible with the development database schema, then I would:
Dump the production data to a file using a program such as mysqldump
Drop the production database
Recreate the production database
Run the migrations against the production database, specifying VERSION=0
Import the production data from the file created at step one
If the schemas aren't compatible then you might be able to follow this process but you'll have to edit the SQL in the file created in the first step to take account of the schema differences.
I have two machines... a development machine and a production machine. When I first brought my rails app onto the production server, I had no problem. I simply imported schema.rb by running rake db:schema:load RAILS_ENV=production. All was well.
So, then on my development machine, I made some more changes and another migration, and then copy the new application over to the production machine. I then tried to update the database by running rake db:migrate RAILS_ENV=production. I get the following error:
"There is already an object named 'schema_migrations' in the database."
I'm thinking to myself, ya no kidding Rake... you created it! I ran trace on rake and it seems as if rake thinks it's the first time it's ever ran. However, by analyzing my 'schema_migrations' table on my development machine and my production machine you can see that there is a difference of one migration, namely the one that I want to migrate.
I have also tried to explicitly define the version number, but that doesn't work either.
Any ideas on how I can bring my production server up to date?
Update:
Let me start off by saying that I can't just 'drop' the database. It's a production server with a little over 100k records already in it. What happens if a similar problem occurs in the future? Am, I to just drop the table every time a database problem occurs? It might work this time, but it doesn't seem like a practical long term solution to every database problem. I doubt the problem I'm having now is unique to me.
It sounds like the 'schema_info' table and the 'schema_migrations' table are the same. In my setup, I only have 'schema_migrations'. As stated previously, the difference between the 'schema_migrations' table on the production server and the development machine is just one record. That is, the record containing the version number of the change I want to migrate.
From the book I read, 'Simply Rails 2', it states that when first moving to a production server, instead of running rake db:migrate, one should just run rake:db:schema:load.
If it matters, I'm using Rails version 2.1.
This is a guess, I admit: I think that because you first ran db:schema:load instead of db:migrate in your production environment, you got the structure of your db, but not the data that migrate populates into your schema_info table. So now, when you run migrate in the production environment, there is no data in schema_info which is why migrate believes that it hasn't run yet (because it hasn't).
That said... you say that you have looked in the "schema_migrations" table, and that there is a difference of one version from dev to production... I haven't heard of that table, although I'm a few months behind on my rails version. Maybe you could try creating a "schema_info" table in the production environment, with a single "version" column, and add a row with the version that you believe your production environment to be on.
If you get "There is already an object named 'schema_migrations' in the database." error message then I suspect that you are using MS SQLServer as your database? (As this seems like MS SQL Server error message)
If yes then which ActiveRecord database adapter you are using? (What is your database.yml file, what gems have you installed to access MS SQL Server database?)
Currently it seems that Rails does not find schema_migrations table in production schema and therefore tries to create it and this creation fails with database error message. Probably the reason is upper/lower case characters in schema_migrations table name - as far as I understand MS SQL Server identifiers are case sensitive.
Depending on the system used in production, I have seen instances where the below does not work:
rake db:migrate RAILS_ENV=production
But where this one does work:
RAILS_ENV=production rake db:migrate
Quirky, I know, but it's worth trying it to see if it makes a difference.
Regarding your update:
I don't understand what the difference is between your production schema_migrations and the dev version. Is there a record in both tables (there should be just 1 column, "version", right) or is there a single record in the dev DB and zero records in production? If there are zero records in the production table, then do this:
ActiveRecord::Base.connection.execute("INSERT schema_migrations (version) VALUES(#{my version number that production is supposedly on})")
Alternatively, you could try dropping the schema_migrations table totally on production:
ActiveRecord::Base.connection.execute("DROP TABLE schema_migrations")
Then, re-running rake db:migrate RAILS_ENV=production. That will run migrations from starting from version 1 though, which is probably not what you're after.
Alternatively alternatively, you could start an IRB session in your production environment, do either a "require" or "load" (I can never remember which, or if it matters) of the migration file that you want to load, and then call MyMigrationClass.up. You would need to manually set the version number in the schema_migrations table after that, as you would still have the problem going forward, but as a quick-fix type of hack, that would work.
I would just drop the DB, add it again and run rake rb:migrate. Brad is correct that when you ran the schema load, it didn't put any records in the schema_migrations table.
This is more complicated of course if there is data you can't lose on the production server. You could get the rake backup tasks (not sure if that is part of core or not) and then run rake db:backup:write on your production database, and then after you get the migrations up to date on production, run rake db:backup:read.
schema_info is from an old version of Rails. schema_migrations is the new kid on the block. You should be able to remove the schema_info table as it'll no longer be used. You'll probably want to search for any issues associated with this name change.
rake db:schema:load will load the database structure from schema.rb. This file is the current representation of the database structure. It's used when you have an empty schema (database) that needs all the tables and indexes creating. It saves you having to run all the migrations. If you have an existing production database with data in, you don't want to run it. As others have said that would be bad!
I know this post was some time ago, but I stumbled across it and it hasn't really been answered. As it comes up on google, here goes.
When you did a rake db:schema:dump (or when this was done for you by the build scripts) it will have put the definition of the migrations table into the schema.rb. At the end of the script, the process will try to create the table again, however it obviously exists already. Just remove the migrations table from the schema.rb before running rake:schema:load and there will be no error message.
You will need to set the version number in the migrations table to subsequently run migrations. So it is important to know what version your schema.rb relates too, or delete all the old migrations (they're safely in your SCM right?)
rake db:migrate RAILS_ENV=production
Use the db:schema:load task just for the first creation, incremental changes should be migrated.