I have a rails app and want to split off my Postgres database to a remote, managed one rather than the standard local one. It seemed easy enough to configure this— however, now I am trying to run my migrations against this new db and it’s proving more difficult. I’m using Mina to deploy, which calls rake db:migrate as part of the deployment. It does not run the migrations, however, as it says that all migrations are up to date, and my create calls can’t find the tables, so I assume the migrations have not run on the remote db.
Whst’s the best way to accomplish this? Every other answer I’ve found involves adding something like ActiveRecord::Base.establish_connection(db_params) command to the top of every migration and every model. This seems absurd— I have probably a 75 migrations at this point. Is there no better way? Is this even the right approach, or could I also use the generated schemes file somehow?
You can set up your database credentials in database.yml with something like this.
remote:
adapter: postgresql
host: your.remote.host
database: yourdb
username: user
password: pass
pool: 5
timeout: 5000
locale: en_US.UTF8
Then run your migrations like
RAILS_ENV=remote rails db:migrate
My goal is to have 2 databases and 2 deployments of rails on the same server. I want to have the regular production server using the production database. Then I want to be able to deploy to a different web address that will use a different database. My goal is to be able to push the backup first and make sure all the migrations etc. work in the full environment. I would then push it to the main server.
The issue I seem to run into is that the database.ml file only lists the 3 database types. The passenger environment will also assume that its running in production and would migrate the main MySQL database even if I deploy the code to a different directory. Whats the best way around this? Was wondering if it is simple or if it involves setting lots of variables in lots of places? Any suggestions would be great!
You can add other database types to database.yml as you see fit.
staging:
adapter: postgresql
host: mydb_host
database: mydb_staging
etc...
You can copy config/environments/production.rb to config/environments/staging.rb and leave it as is so the two environments are exactly the same, or tweak staging.rb as you see fit.
Now you have a staging environment! Use it where appropriate, e.g.:
rake RAILS_ENV=staging db:migrate
I am not a passenger expert, but know that my shop has both staging and production instances of apps running on the same server under passenger, so it can be done. Google can probably instruct you better on configuring that than I can.
Rails/PostgreSQL newbie here, having real problems dealing with creating PostgreSQL projects in Rails.
Briefly, I am following Ryans excellent Railscast episodes, in particular the episode on deployment. I create a new project like this:
rails new blog -d postgresql
This generates a database.yml file similar to the following (comments extracted):
development:
adapter: postgresql
encoding: unicode
database: blog_development
pool: 5
username: blog
password:
test:
adapter: postgresql
encoding: unicode
database: blog_test
pool: 5
username: blog
password:
production:
adapter: postgresql
encoding: unicode
database: blog_production
pool: 5
username: blog
password:
Looks good so far.
However, on my development machine, whenever I try to run rake db:create:all, I get a message similar to the following:
rake aborted!
FATAL: role "blog" does not exist
My understanding is that this is because I haven't (or, rather, Rails hasn't) created a user called "blog" when the application was created. Therefore, I need to either:
Change the username for all environments to my system's "super
user" I chose when I installed Homebrew (which works); OR
Create a new super user for each individual project I set up (e.g. a
super user "blog" in this instance).
Now, the problem is this - should I be creating a super user for each individual project I create? If so, how? The reason I ask is that Ryan's Railscasts episodes never actually mention this, so I'm not sure if this is the correct approach to be taking (it seems a bit long-winded), or if there is a problem with my PostgreSQL database installation (as I would have thought that the super user would be automatically set up along with the database when the application is created).
Additionally, so as to make deployment easier, I'd like to keep the overall database.yml file the same in both the development and production environments (although I admit to knowing even less about deployment, so maybe this isn't ideal).
Thanks!
Creating database users is up to you. Rails does not create database users.
Depending on your platform, you can use some kind of database management GUI to make it easier to create users and manage their rights. I use PGAdmin, which came with the Mac OS X executable for PostgreSQL. It is available for other platforms as well.
In development, many people use the same "superuser" for all projects. In production, I would advice you to have application-specific users with minimal privileges.
I don't exactly understand the last paragraph about the database.yml being the same for dev and prod envs, because that is precisely the point - one file for all environments. You should probably not use the same credentials for development and production.
At the moment I have Capistrano deploying to a server which is set up as a development environment.
However, everytime I run cap deploy, it doesn't keep the database at all, so every deployment ends up with a fresh database, completely empty. I have to run cap deploy:migrations to set up the DB, but the issue here is that there is an individual DB for each deployment.
I figure I could change database.yml to use a path such as ../../db/development.sqlite3 for the DB but this would mean I then have to copy that change locally too, and moving my DB out of the directory for my project on my own laptop would be very in-convenient.
Is there a way to tell Capistrano to use a single DB location for every deployment yet still keep my DB in the same place locally? Setting the server to a production environment isn't an option as this stage, unfortunately. Something like being able to do :
development:
adapter: sqlite3
:on local
database: db/development.sqlite3
:on server
database: /webapps/rails/shared/dev.sqlite3
pool: 5
timeout: 5000
(At this point it's probably also worth mentioning I'm very much still learning my way around Rails).
Any of your thoughts would be most appreciated, thank you. If the only option is to set the env to production then that will have to do, but if there's a way round it that lets me keep the server as a development server, that would be great.
Jack.
Add a step in capistrano that runs before any database stuff to create a symbolic link for whatever database file you want that points to the shared directory. This is how logs is set up for you. Something along the lines of this:
namespace :custom do
task :symlink, :roles => :app do
run "ln -nfs #{shared_path}/development.sqlite3 #{release_path}/db/development.sqlite3"
end
end
after "deploy:create_symlink", "customs:symlink"
I think you're after multiple application environments: one for staging on the server, one for development locally and eventually one for production. For a good run through, try here.
After significant troubleshooting, I figured out that I needed to run rake spec once (I can abort with control-c) before I can run rspec directly (e.g. on a subset of our specs). We are running Rails 3.0.7 and RSpec 2.5.0.
Clearly, rake is running some important database setup tasks / code (we have custom code in the root level rails Rakefile and possibly other places).
How can I run the rake test database setup tasks / code without running rake spec?
In addition to being able to run rspec on a subset of files, I am using specjour to spread our specs across multiple cores (haven't had success with spreading them across the LAN yet), but I see the same behavior as for running rspec directly: I need to run rake spec on each test database (assuming two cores) before specjour works:
rake spec TEST_ENV_NUMBER=1
control-c (after tests start)
rake spec TEST_ENV_NUMBER=2
control-c (after tests start)
specjour
Note: my config/database.yml has this entry for test (as is common for the parallel testing gems):
test:
adapter: postgresql
encoding: unicode
database: test<%=ENV['TEST_ENV_NUMBER']%>
username: user
password:
parallel_tests seems to set up its databases correctly, but many of our specs fail.
I should also mention that running specjour prepare causes Postgres to log errors that it can't find the databases, but it creates them (without tables). On a subsequent run, no errors are logged, but also no tables are created. It is possible that my whole issue is simply a bug in prepare, so I reported it on github.
I think that I can run arbitrary code on each specjour test database by setting Specjour::Configuration.prepare in .specjour/hooks.rb, so if there's any rake tasks or other code that I need to run, it may work there.
I would recommend dropping your test database, then re-create it and migrate:
bundle exec rake db:drop RAILS_ENV=test
bundle exec rake db:create RAILS_ENV=test
bundle exec rake db:schema:load RAILS_ENV=test
After these steps you can run your specs:
bundle exec rspec spec
gerry3 noted that:
A simpler solution is to just run rake db:test:prepare
However, if you're using PostgreSQL this wont work because the rails environment gets loaded, which opens a database connection. This causes the prepare call to fail, because the DB cannot be dropped. Tricky thing.
I had a similar problem setting up the CI system at work, so I gradually worked up a system to handle this. It may not be the best solution, but it works for me in my situation and I'm always on the lookout for better ways to do things.
I have a test database that I needed setup, but also needed seeded data loaded for our tests to work.
The basics of troubleshooting rake tasks is to run rake with the --trace option to see what is happening under the hood. When i did this, I found that running rake spec did a number of things that I could replicate (or modify as I saw fit) in a custom rake task.
Here's an example of what we do.
desc "Setup test database - drops, loads schema, migrates and seeds the test db"
task :test_db_setup => [:pre_reqs] do
Rails.env = ENV['RAILS_ENV'] = 'test'
Rake::Task['db:drop'].invoke
Rake::Task['db:create'].invoke
result = capture_stdout { Rake::Task['db:schema:load'].invoke }
File.open(File.join(ENV['CC_BUILD_ARTIFACTS'] || 'log', 'schema-load.log'), 'w') { |f| f.write(result) }
Rake::Task['db:seed:load'].invoke
ActiveRecord::Base.establish_connection
Rake::Task['db:migrate'].invoke
end
This is only an example, and specific to our situation, so you'll need to figure out what needs to be done to get your test db setup, but it is quite easy to determine using the --trace option of rake.
Additionally, if you find the test setup is taking too long (as it does in our case), you can also dump the database into .sql format and have the test database pipe it directly into mysql to load. We save several minutes off the test db setup that way. I don't show that here because it complicates things substantially -- it needs to be generated properly without getting stale, etc.
HTH
The provided solutions all require to load the Rails environment, which is, in most cases, not the desired behaviour due to very large overhead and very low speed. DatabaseCleaner gem is also rather slow, and it adds another dependency to your app.
After months of chagrin and vexation thanks to reasons vide supra, I have finally found the following solution to be exactly what I need. It's nice, simple and fast. In spec_helper.rb:
config.after :all do
ActiveRecord::Base.subclasses.each(&:delete_all)
end
The best part about this is: It will only clear those tables that you have effectively touched (untouched Models will not be loaded and thus not appear in subclasses, also the reason why this doesn't work before tests). Also, it executes after the tests, so the (hopefully) green dots will appear right away.
The only downside to this is that if you have a dirty database before running tests, it will not be cleaned. But I doubt that is a major issue, since the test database is usually not touched from outside tests.
Edit
Seeing as this answer has gained some popularity, I wanted to edit it for completeness: if you want to clear all tables, even the ones not touched, you should be able to do something like the "hacks" below.
Hack 1 - pre-loading all models for subclasses method
Evaluate this before calling subclasses:
Dir[Rails.root.join("app", "models", "**", "*.rb")].each(&method(:require))
Note that this method may take some time!
Hack 2 - manually truncating the tables
ActiveRecord::Base.connection.tables.keep_if{ |x| x != 'schema_migrations' }
will get you all table names, with those you can do something like:
case ActiveRecord::Base.configurations[Rails.env]["adapter"]
when /^mysql/, /^postgresql/
ActiveRecord::Base.connection.execute("TRUNCATE #{table_name}")
when /^sqlite/
ActiveRecord::Base.connection.execute("DELETE FROM #{table_name}")
ActiveRecord::Base.connection.execute("DELETE FROM sqlite_sequence where name='#{table_name}'")
end
It appears that in Rails 4.1+, the best solution is simply to add ActiveRecord::Migration.maintain_test_schema! in your rails_helper after require 'rspec/rails'.
i.e. you don't have to worry about having to prepare the database anymore.
https://relishapp.com/rspec/rspec-rails/docs/upgrade#pending-migration-checks
In a spring-ified Rails 4 app, my bin/setup is usually augmented to contain
puts "\n== Preparing test database =="
system "RAILS_ENV=test bin/rake db:setup"
This is very similar to leviathan's answer, plus seeding the test DB, as
rake db:setup # Create the database, load the schema, and initialize with the seed data
(use db:reset to also drop the database first)
As the comment mentions, if we want to drop the DB first, rake db:reset does just that.
I also find that this provides more feedback when compared to rake db:test:prepare.
I started by dropping my test database
rake db:drop RAILS_ENV=test
when trying to create a new test database I ran into an issue because my user account was not the same as the account that owns the databases so I created the database in PostgreSQL instead.
type psql in the command prompt and then run the below to create a test database that uses an account other than your own.
CREATE DATABASE your_database_name OWNER your_db_owner;
then run your migrations in the test environment.
rake db:migrate RAILS_ENV=test