As far as I can tell, the capistrano :db role is used only to run migrations.
(Thus, in most cases it probably shouldn't actually be the server that runs your database. Why would you have a ruby/rails stack there (or allow ssh logins there)? it's just whatever server you want to actually execute the rails migrations).
And only the server identified as db role with :primary => true is used to run migrations.
So any other servers identified as 'db' role but without :primary => true... are used for nothing at all? So why does the default deploy.rb created by capify . encourage you to list them? What would you even list here?
Anything I'm missing?
Obviously, the name of role :db is misleading. As you pointed out, Capistrano defines it (with :primary => true) as a host that we execute rake db:migrate on, but database servers are not always running on such hosts. We can alter the schema of remote database server through a Rails app. The correct role name for this kind of host is not :db.
Inferring from the comments on lib/capistrano/configuration/roles.rb, the original meaning of the role :db is a host on which database servers are running. We are expected to login to the :db hosts and do some tasks.
The designers of Capistrano should have defined :migration role or something else for the deploy:migrate task. But the association between the :db role with this task was defined six years ago with 9a6d2fb and has not been changed since then.
Generally speaking, the users of Capistrano can define roles and associate them with tasks freely. The deploy:migrate task is provided just as a recipe for Rails developers. Unfortunately, this recipe includes a misconception about how we do the database migration and is used widely for a long time.
You may have a setup that includes a master database server and multiple slave servers. In some cases, you could have an capistrano task that needs to be run on all database servers. In others, you may want to run a task (for instance, a migration) only on the master server and allow the changes to propagate to the slave instances.
Related
I have a rails app looking good on my localhost. Now I want to deploy it to multiple server (one load balancer, and two application server to be exact, with possible increase in the future), and somehow I'm lost. This would be my first time deploying a web by myself, so I'm sorry for my lack of knowledge.
I want all application server to run exactly same code.
And when I create a new content, I want the new content to be stored on each server's database instance (MySQL). So when I took down one server for maintenance and updating, the rest of the server could serve users with exact same content. I've read that capistrano could help me with this, but somehow I managed to get lost in learning how to do this. So, how should I proceed from here? How should the capistrano recipe look like, and do I have to tweak database.yml in my rails also?
Thank you very much for help.
You can use roles to deploy the same application to multiple servers.
Assuming you're using the multistage extension, define the roles in production.rb:
server1 = 'appserver1.tld'
server2 = 'appserver2.tld'
server3 = 'webserver1.tld'
role :app, server1, server2
role :web, server3
The web server will run on servers specified by the :web role.
The app layer will run on servers specified by the :app role.
If you run migrations or other DB operations during deploy, you should also specify a server under the :db role. For example:
role :db, 'dbserver.tld', :primary => true
You may have multiple DB servers, but by specifying one as the primary server capistrano will only run DB operations on that server.
In your deploy.rb, you can also create tasks that run only for certain roles. For example:
task :restart, :roles => :app, :except => { :no_release => true } do
run "touch #{current_path}/tmp/restart.txt"
end
In the above example, :except => { :no_release => true } means that it will only run if at least one release exists on the server being deployed to.
This wiki article may be of further help to you.
So, I'm diddling around with rails (ruby 1.9.3p392, rails 3.2, sqlite3 db) and I'm trying to deploy the ubiquitous blog tutorial code to a "production" server (apache, passenger, ubuntu). My deploy.rb looks like this:
require 'bundler/capistrano'
require 'rvm/capistrano'
load 'deploy/assets'
set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"")
set :rvm_type, :user
set :user, 'blah'
set :application, 'railsTest'
set :domain, 'www.blah.com'
set :applicationdir, "/home/sean/public/blah.com/public"
set :scm, 'git'
set :repository, "ssh://blah#1.1.1.1/home/blah/public/bla.com/public/capDep.git"
#set :git_enable_submodules, 1 # if you have vendored rails
set :branch, 'master'
set :git_shallow_clone, 1
set :scm_verbose, true
set :use_sudo, false
# roles (servers)
role :web, domain
role :app, domain
role :db, domain, :primary => true
# deploy config
set :deploy_to, applicationdir
set :deploy_via, :export
set :migrate_target, :latest
# additional settings
default_run_options[:pty] = true # Forgo errors when deploying from windows
#ssh_options[:keys] = %w(/home/blah/.ssh/id_rsa)
ssh_options[:forward_agent] = true
# if you want to clean up old releases on each deploy uncomment this:
# If you are using Passenger mod_rails uncomment this:
namespace :deploy do
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
#after "deploy:update_code", "deploy:migrate"
Now, I am sure that must look like a big hot mess to those who know what they are doing with capistrano, but I am an utter rube. In the end, despite my inadequacies, the deploy seems to work, because when I run the following
cap deploy:setup
cap deploy
my app is up and running and, just because I can, I add a few rows to a table in the db via the web ui that was created for me by rails. Now, I get bold and create a migration, adding a column to a table. I push my changes to git. To my horror, when I run
cap deploy
ALL the migrations are run, which recreates the tables, thus destroying all my data. I have repeated this painful process several times. My schema_migrations table looks like this:
20130620210004
20130620220229
20130628213331
20130628214946
20130628223002
What am I missing here?
UPDATE: I recently gave #TheMahrvin's suggestion regarding running deploy:migrations at the command line and removing it from the deploy.rb. It didn't work... once again, all migrations were run. My muse must have whispered something in my ear, because I decided to try running db:migrate on the server itself. I was astonished to see this output after running just "rake":
20130717230110 CreateHighScores
20130717230342 CreateGames
20130717231041 AddGameTypeToGame
20130717233707 AddGamePublisherToGame
20130717234124 AddGameRatingToGame
20130731210558 AddGameMechanicToGame
Only the last migrations should be pending. So, perhaps this isn't a problem with Capistrano at all (I've updated the title of this question to reflect that). So, why are the previous migrations still being flagged as pending? I know they were run in the past, both because I saw them in the output and verified the db schema after they ran.
UPDATE #2: Setup another migration and ssh'd into the server and cd'd my way to the "current" directory, which if I understand capistrano at all (fat chance) is where the current files are. Running
bundle exec rake db:migrate:status
got me:
Status Migration ID Migration Name
--------------------------------------------------
down 20130717230110 Create high scores
down 20130717230342 Create games
down 20130717231041 Add game type to game
down 20130717233707 Add game publisher to game
down 20130717234124 Add game rating to game
down 20130731210558 Add game mechanic to game
down 20130731212454 Add publish year to game
down 20130731214515 Add game rank to game
down 20130731214928 Add game abbr to game
down 20130731215749 Add crazy field to game
I can't help feeling that there is something profoundly wrong with what I am trying to do.
OK, figured it out... though how anybody else in the stackosphere was supposed to do the same based on the red herrings in my original question is beyond me.
The problem was that my production database was set to
db/production.sqlite3
Because it was a sqlite database in the main project directory, it was getting axed every time I ran
cap deploy
Then, when I would run
cap deploy:migrate
It would find an empty database and think all migrations needed to be run. I solved this by changing the database path to
/my_absolute_path/shared/db/production.sqlite3
Thanks to #TheMahvin and anyone else who attempted to take on the hopeless task of answering my poorly worded question!
H/T to this question, which made the scales fall from my eyes:
Capistrano Deploy Wipes Database?
I haven't seen:
after "deploy:update_code", "deploy:migrate"
before. Try deleting that line and run:
bundle exec cap deploy:migrations (deploys code and migrations)
or
bundle exec cap deploy:migrate (runs the migrate rake task on the server)
instead.
The rest of your deploy.rb seems OK to me, though I don't know anything about the rvm/capistrano integration or the windows adjustment.
How did you "add a few rows to a table in the db"?
I suspect your data loss results from mixing migrations and your own db changes.
Rails expects you to do all the database changes via migrations.
There's some debate on migrations in general in the Rails community, but right now (especially if you're a beginner) always use migrations to change your database. That way you have a complete blueprint for your db allowing you to deploy on several machines from scratch without fiddling with your db and to make sure other contributors have the same db to work with.
I don't know a lot about these kinds of internals, but from what I understand your data loss resulted something like that:
After your manual changes Rails couldn't match the db-layout to the result of any migration (via the timestamps in your migrations and your schema respectively) thus treating the db as if it was new. To get to the state defined by all migrations, all of them needed to be executed, including the migrations that create tables, thus discarding everything in them.
I hope this helps,
Andy
I don't really understand what model of unix accounts/permissions is intended with Capistrano.
Let's say I've got a rails app called Widget and I'll be deploying with passenger. In general, pre-capistrano, I want the entire ./widget directory to be owned by a user called 'widget'. And then by default default passenger will run the app process as user 'widget' too, because passenger runs as user that owns the file.
And the whole point of this is for that 'widget' account to have fairly limited permissions, right? Since a web app will be running under that account?
So since I want the files to be owned by 'widget', I tell cap
set :user, "widget"
But now when I run "cap deploy:setup", it wants to 'sudo' from that account. No way that 'widget' account gets sudo privileges, the whole point is keeping this account limited privs.
Okay, I can tell cap not to use sudo... but then it won't actually have privs to do what it needs, maybe.
I can find a workaround to this too. But I start thinking, why do I keep having to re-invent the wheel? I mistakenly thought the point of cap recipes was to give me some best practices here. Anyway... what do people actually do here?
Use one unix account for install, but then have cap somehow 'chown' it to something else? Use one unix account, but have something non-cap (puppet?) do enough setup so that account doesn't need to sudo to get things started? What? What am I missing?
You can avoid some of the headache by using Passenger most commonly with Nginx as your webserver.
Then to restart web services the unprivileged Widget user creates a file in his path and Passenger will automatically restart Nginx when it sees that file being present.
This is enabled via the following in your config/deploy.rb:
namespace :deploy do
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
As for other privileged tasks for MySQL/DB administration your database.yml provides the credentials necessary to handle rake migration tasks.
So really the only time you would need something more privileged would be for system wide installation of gems, ruby, or rails updates, but a lot of that depends on how your production environment was setup/installed.
Given Passenger+Nginx and separate credentials for DB you can disable sudo and see if you encounter any errors during your Capistrano deploy process and then pickup from there.
How do you ensure your migrations are run only once, when you deploy to multiple machines at the same time ?
What I have to do right now is select one machine to run the migrations when I do have a change of that sort. Ideally, deployment is brainless and the process would take care of that for me.
My idea currently is to have the migrator look for schemas to migrate and acquire a lock if it has something to do. If the lock is already acquired, it skips the migration alltogether. Reading the ActiveRecord code it doesn't seem to support such idea so it would need some patching.
What's your idea ?
Are you using Capistrano? You can specify a list of database servers and mark one as Primary. Migrations will only be run on that server:
role :app, 'example.com.com'
role :web, 'example.com'
role :db, 'db01.example.com', :primary => true
role :db, 'db02.example.com'
role :db, 'db03.example.com'
EDIT: the :db role is not intended to be used for a separate database server which is not running Rails application code. This probably isn't your setup.
We have a large deployment of around a dozen servers. We'd like to use the Whenever gem but I can't figure out a way to say which machine the cron jobs should go on! We only want these jobs to run on our server that does background jobs.
Is there a way to specify this?
If you deploy the project with Capistrano and you use the default Whenever recipe, you can create a new Capistrano role
role :whenever, "192.168.1.1"
and set the role in your deploy.rb file
set :whenever_roles, "whenever"
In this way, the task will be executed only on the specified server.
whenever is preconfigured to run against the db role, so if your db role is also the background machine's role you don't have to do the 'role :whenever, "192.168.0.1"' hack. see the codes
https://github.com/javan/whenever/blob/master/lib/whenever/capistrano.rb#L2