I used to be a .NET guy, and enjoyed using a nightly build system (continuous integration) called CruiseControl.NET to automatically deploy my applications to my staging environment each night.
Now that I've made the jump to Ruby on Rails and GitHub, I've found myself a little confused about how to setup an equivalent automated, nightly build system. I want to do things right, the Rails way, but I could use a push in the right direction.
Here's what I'm using...
Ruby on Rails 3.2.9 (with asset pipeline)
RVM
Apache + Passenger
MySQL
Ubuntu 12.04 (staging server OS)
GitHub (SCM)
I'm looking for a system/solution that fulfills these requirements: (Ideally utilizing Capistrano...)
Deploy the latest commit from my 'master' branch in my GitHub repository to my staging server.
One-click build on-demand: I want to click a button or link to force a deploy (or re-deploy) at any time
Have the capability to run custom commands on the staging server as a part of the deploy (i.e. 'bundle install', restart Apache, etc...)
Automatic deploy daily or after a commit on GitHub (optional)
And because I'd never ask this question without doing some research first, here are some relevant resources I've found, but haven't been able to decipher:
Automatic Deployment via Git
Deploying Rails 3 apps with Capistrano
Suggestions, anyone? Thanks!
David
Well, it turns out there is alot of good information out there regarding how to use Capistrano for this purpose (including Prakash's reference), but none of it seems to be fully comprehensive.
After many hours of picking through guide, after forum, after stack overflow question, I managed to fulfill most of my goals. To save time for others, I will try to provide answers and information that I found in the context of my original question.
UPDATE:
It turns out I was looking for Jenkins all along: it's a perfect (an even an improved) analog to the CruiseControl build server application I had used before. Jenkins is a web application that kicks off builds on a scheduled basis or, with plugins, commit events. It doesn't include functionality to actually deploy my Rails app, so that's where Capistrano steps in. Using Jenkins' "shell execute" build task to trigger Capistrano deploys, I was able to accomplish all of my goals above.
Take a look at this guide for more detail.
ORIGINAL POST:
First off, Capistrano can work for the purpose of build system, however, it is not at all similar to CruiseControl.
CruiseControl is:
A web application, that is run on a machine that serves as a 'build server'
Any user may visit the web app to run builds; the user does not have to have anything setup or configured on their end... it's just a webpage.
The interface also provides functions for logs, blame, scheduled builds, etc...
Capistrano is:
Not an application or service, but a gem that works like rake; it bears similar functionality to the Ant and Nant scripts used by CruiseControl for the purpose of deploying.
Capistrano is run on a user's local machine (typically the Rails app folder), and requires configuration on their end to run
It directly connects from the user's machine to the remote server via SSH to perform deploys (this may not be preferable if you don't want to grant users SSH access to the remote server)
Commands are run through a console i.e. cap deploy (this is as close as it gets to 'one click' deploys.)
It does produce logs, perform rollbacks etc.
It does not kick-off scheduled builds on its own (using cron was suggested)
In regards to my original requirements...
Deploy the latest commit from my 'master' branch in my GitHub repository to my staging server.
Configuring Capistrano's deploy.rb file will accomplish this.
One-click build on-demand: I want to click a button or link to force a deploy (or re-deploy) at any time
All Capistrano deploys are done through 'force': you run 'cap deploy' in your console manually
Have the capability to run custom commands on the staging server as a part of the deploy (i.e. 'bundle install', restart Apache, etc...)
Configuring Capistrano's deploy.rb file will accomplish this. Most of these commands are included out of the box.
Automatic deploy daily or after a commit on GitHub (optional)
I haven't figured this one out yet... a cron job might be the best way to do this.
Setting up Capistrano
Start with this tutorial from GitHub first. In your Rails App folder, you should end up with a Capfile and config/deploy.rb file.
To save you some time, copy and paste these files, and tweak the settings to your needs.
The files below are configured for:
Deployment to a test environment
GitHub
RVM based Rails
Using an explicit version of Ruby and gemset
Including untracked files (i.e. a database.yml file you didn't commit to SCM)
Using Rails asset pipeline
Running migrate and seed with each deploy
Restarting Apache based Passenger with each deploy
Capfile
# Set this if you use a particular version of Ruby or Gemset
set :rvm_ruby_string, 'ruby-1.9.3-p286#global'
#set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"") # Read from local system
require "bundler/capistrano"
# Uncomment this if you're using RVM
require "rvm/capistrano"
load 'deploy'
# Uncomment if you are using Rails' asset pipeline
load 'deploy/assets'
load 'config/deploy' # remove this line to skip loading any of the default tasks
config/deploy.rb
# BEGIN RUBY CONFIG
# You can manually override path variables here
# set :default_environment, {
# 'PATH' => "/usr/local/bin:/bin:/usr/bin:/bin:/<ruby-dir>/bin",
# 'GEM_HOME' => '<ruby-dir>/lib/ruby/gems/1.8',
# 'GEM_PATH' => '<ruby-dir>lib/ruby/gems/1.8',
# 'BUNDLE_PATH' => '<ruby-dir>/lib/ruby/gems/1.8/gems'
# }
# This changes the default RVM bin path
# set :rvm_bin_path, "~/bin"
# If your remote server doesn't have a ~/.rvm directory, but is installed
# at the /usr/local/rvm path instead, use this line
set :rvm_type, :system
# END RUBY CONFIG
default_run_options[:pty] = true # Must be set for the password prompt
# from git to work
# BEGIN MULTIPLE ENVIRONMENT DEPLOYS
# Read the following URL if you need to deploy to different environments (test, development, etc)
# https://github.com/capistrano/capistrano/wiki/2.x-Multistage-Extension
# set :stages, %w(production test)
# set :default_stage, "test"
# require 'capistrano/ext/multistage'
# END MULTIPLE ENVIRONMENT DEPLOYS
# BEGIN APPLICATION VARS
set :application, "yourapp"
set :rails_env, 'test'
# END APPLICATION VARS
# BEGIN PATH DEFINITIONS
set(:releases_path) { File.join(deploy_to, version_dir) }
set(:shared_path) { File.join(deploy_to, shared_dir) }
set(:current_path) { File.join(deploy_to, current_dir) }
set(:release_path) { File.join(releases_path, release_name) }
# END PATH DEFINITIONS
# BEGIN SCM VARS
set :repository, "git#github.com:yourgithubuser/yourrepository.git" # Your clone URL
set :scm, "git"
set :scm_username, "yourgithubuser"
set :scm_password, proc{Capistrano::CLI.password_prompt('GitHub password:')} # The deploy user's password
set :branch, "master"
# END SCM VARS
# BEGIN SERVER VARS
set :user, "ubuntu" # The server's user for deploys
role :web, "dev.#{application}" # The location of your web server i.e. dev.myapp.com
role :app, "dev.#{application}" # The location of your app server i.e. dev.myapp.com
role :db, "dev.#{application}", :primary => true # The location of your DB server i.e. dev.myapp.com
set :deploy_to, "/home/#{user}/www/#{application}"
set :deploy_via, :remote_cache
# Uncomment this if you want to store your Git SSH keys locally and forward
# Else, it uses keys in the remote server's .ssh directory
# ssh_options[:forward_agent] = true
# END SERVER VARS
# BEGIN ADDITIONAL TASKS
before "deploy:start" do
deploy.migrate
deploy.seed
end
before "deploy:restart" do
deploy.migrate
deploy.seed
end
# Some files that the Rails app needs are too sensitive to store in SCM
# Instead, manually upload these files to your <Rails app>/shared folder
# then the following code can be used to generate symbolic links to them,
# so that your rails app may use them.
# In this example below, I link 'shared/config/database.yml' and 'shared/db/seed_data/moderators.csv'
before "deploy:assets:precompile" do
run ["ln -nfs #{shared_path}/config/settings.yml #{release_path}/config/settings.yml",
"ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml",
"mkdir -p #{release_path}/db/seed_data",
"ln -nfs #{shared_path}/db/seed_data/moderators.csv #{release_path}/db/seed_data/moderators.csv",
"ln -fs #{shared_path}/uploads #{release_path}/uploads"
].join(" && ")
end
namespace :deploy do
# Define seed task (call 'cap deploy:seed')
desc "Reload the database with seed data"
task :seed do
run "cd #{current_path}; bundle exec rake db:seed RAILS_ENV=#{rails_env}"
end
# If you are using Passenger mod_rails uncomment this:
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
# END ADDITIONAL TASKS
Yes. Capistrano is the tool you are looking for. Using Capistrano with the system cron should enable you to implement a build/deploy system with all four of the mentioned requirements. Rather easily at that.
The best resource to learn about Capistrano is the Deploying Rails book. It has two chapters on Capistrano, the first one covering the basic usage, and the second covering some advanced usage concepts. There is also a Capistrano Case study at the end which explores further config options with a real life deploy script example.
FYI: I finished reading the two chapters yesterday in a couple of hours and tweeted about it ;-)
I'd say go with Capistrano for the deploy part.
The CI part (build when you do a git push) would be best implemented by using a CI server, such as travis (if you're on Github) or Jenkins if you have a private repo.
For the one-click-build, go with either capistrano directly (cap deploy from command line) or a simple CI hook which runs all tests, then runs cap deploy.
Related
I am trying to deploy my application using capistrano. But I want to deploy my application to multiple paths of the same server.For example If for the first run I want to deploy it to below path
set :deploy_to, '/home/a/some_path/
Once completed the first one it should run for the second path that will be
set :deploy_to, '/home/b/some_path/
and so on. Any suggestions how can I achieve this? Right now my single path deployment path is working AOK.
In your config file:
set :deploy_to, ENV["DEPLOY_PATH"]
Then, to deploy, run the command setting the DEPLOY_PATH variable:
DEPLOY_PATH="my/path" cap production deploy
Using capistrano 3.8.2, I monkeypatched lib/capistrano/dsl/paths.rb in my deploy.rb, but then I found that I needed more work to get git wrapper set up right when there where different deploy users.
The result is at: https://gist.github.com/mcr/49e8c7034658120013c1fe49da77c2ac
But, I'm leaving the essence of the content here:
module Capistrano
module DSL
module Paths
def deploy_to
dir = #host.properties.fetch(:deploy_to) || fetch(:deploy_to)
puts "For #{#host.hostname} deploy_to: #{dir}"
dir
end
end
end
end
(You can take the puts out, and shorten it to a one-liner, but I found the extra debug useful)
One then does:
server "server.client1.example.com", user: "client1", roles: %w{app db web}, deploy_to: '/client1/app/foobar'
server "server.client2.example.com", user: "client2", roles: %w{app db web}, deploy_to: '/client2/app/foobar'
where server.client1.example.com and server.client2.example.com are CNAMEs or duplicate A/AAAA records for the same server. This also isolates the question of where each client is to DNS.
I'm in the process of upgrading from Capistrano 2 to Capistrano 3. In Cap 2 I was using the following to take a command line argument as the branch name (otherwise default to master)
set :branch, fetch(:branch, "master")
If I called cap deploy it would deploy the master branch. But it also let me do something like this:
cap deploy -S branch=foo
Which would deploy the foo branch.
Now, in Capistrano 3, if I try to run the above I get an error: invalid option: -S.
What's the proper way to pass an argument via the command line now?
What I ended up doing was setting an ENV variable.
So now I can call
cap production deploy branch=mybranch
And it will deploy mybranch. If I run a simple cap production deploy it will deploy the default branch (master if you don't set one, but I've changed mine below to default to demonstrate)
This is the code I put in my deploy.rb file:
set :branch, "default"
if ENV['branch']
set :branch, ENV['branch']
end
Rake tasks (which cap is using) are supporting arguments.
namespace :test do
desc "Test task"
task :test, :arg1 do |t, args|
arg1 = args[:arg1]
puts arg1
end
end
cap -T outputs:
cap yiic:test[arg1] # Test task
Example of invocation:
cap production yiic:test[test1]
Also, here is a helpful post
P.S.: you should use env vars for "global" settings. Like common values for multiple tasks.
To give an updated and working solution for Capistrano 3 (as it took me a while to find and too many tests to make it works).
My files are like:
config/
deploy/
staging.rb
production.rb
deploy.rb
...
Capfile
In staging.rb I have:
server 'staging', roles: %w(db)
set :branch, ENV.fetch('REVISION', 'master')
set :use_sudo, false
set :user, 'toto'
set :deploy_to, '/var/www'
(In server 'staging', staging is a SSH connexion, defined in my .ssh/config)
Then, to deploy a specific revision, I just need to call:
cap staging deploy REVISION=3b2d9ab
Where 3b2d9ab is the Git commit hash (short or long version).
Is there a way to set a default stage in Capistrano 3?
I've tried putting set :stage, :production inside deploy.rb but that didn't work, it gives the error:
Stage not set, please call something such as `cap production deploy`,
where production is a stage you have defined
I only have one stage right now so I want to be able to just run cap deploy and have it execute on the default.
Capistrano v3 is somewhat of a wrapper around Rake, so you need to realize that what's really happening is that a production task is getting run first, followed by a deploy task.
If you debug it a little, you'll find that deploy.rb doesn't get loaded when you don't type in a stage. This is because the stage's task is where deploy.rb gets loaded: Looking at lib/setup.rb, a task is defined for each stage. When run, the stage's task sets :stage, loads up the capistrano defaults, and then finally loads your deploy.rb file.
So, an easy trick would be to tell Capistrano to invoke the stage task every time you run cap by adding this to the end of your Capfile (not your deploy.rb):
Rake::Task[:production].invoke
or, using the invoke method from Capistrano's DSL:
invoke :production
This may have some unintended consequences if you actually do use multiple stages, but if you only ever use the production stage, it should work fine.
Another easy solution could be a simple shell alias, such as alias cap='cap production', but it might not work great if you have multiple projects with different stage names.
After I cd into the RAILS Root directory, issuing the command:
cap development deploy
seems to work. Earlier I was in the app/models folder and issuing the command came back with this error:
Stage not set, please call something such as cap production deploy, where production is a stage you have defined.
The old solution works for me in Capistrano 3:
cap --version
#=> Capistrano Version: 3.3.5 (Rake Version: 10.4.2)
At the very top of the Capfile after these lines
# Load DSL and Setup Up Stages
require 'capistrano/setup'
add:
set :stage, :production
and then run you task as usual without the stage specified:
cap foo:bar
New answer for capistrano 3.6+:
It's better to use invoke :production unless Rake.application.options.show_tasks to avoid the warning which you would otherwise get with cap -T
You can add the following line to your deploy.rb, which will prevent Capistrano from expecting a stage:
set :stages, ["production"]
I searched all of the relevant Capistrano issues, but couldn't find something that even elucidated anything here for me.
git version 1.6.4.2
Capistrano v2.5.10
Basically, when I run my cap deploy.rb script, it connects to the server, starts executing the deploy:update task, then in the deploy:update_code task:
*** [deploy:update_code] rolling back
* executing "rm -rf /home/user_name/public_html/project_name/releases/20091223094358; true"
servers: ["project_name.com"]
It fails with the following error:
/Library/Ruby/Gems/1.8/gems/capistrano-2.5.10/lib/capistrano/recipes/deploy/scm/git.rb:231:in `query_revision': Unable to resolve revision for 'master' on repository 'ssh://git#slice_ip:path_to_git_repository'. (RuntimeError)
Here's my deploy script, I've tried including and omitting:
set :branch 'master'
I also just thought my path to the repository was off, but i've tried just about every permutation (absolute, not absolute, .git suffix, no suffix). There's definitely a bare git repository at the path i'm pointing to.
**I do have multiple projects being hosted on one slice. The other projects is also a rails project, but is running SVN. Capistrano deployments work fine.
Any pointers in the right direction or any ideas would help reduce the amount of drinking I am planning on doing if I can't figure this out. (Paths / IPs obfuscated, dont hack me bro!)
set :application, "project1"
set :user, "username"
set :repository, "ssh://git#67.24.9.133/home/git/project1.git"
set :branch, "master"
set :port, 696969
set :deploy_to, "/home/username/public_html/#{application}"
set :scm, :git
role :app, application
role :web, application
role :db, application, :primary => true
# deployment via remote client (workstation)
set :deploy_via, :copy
set :runner, user
# mod_rails
namespace :deploy do
desc "Restarting mod_rails with restart.txt"
task :restart, :roles => :app, :except => { :no_release => true } do
run "touch #{current_path}/tmp/restart.txt"
end
[:start, :stop].each do |t|
desc "#{t} task is a no-op with mod_rails"
task t, :roles => :app do ; end
end
end
This was the most relevant post (extremely relevant even), but I couldn't really figure out what they were saying the fix is. I'm pretty new with git / capistrano configs.
https://capistrano.lighthouseapp.com/projects/8716/tickets/56-query_revision-unable-to-resolve-revision-for-head-on-repository
Ok I seemed to have fixed it.
Basically, since I have 2 separate repositories on the remote server, I think the "git" user was failing because I hadn't registered an ssh keypair for the git user. That explains why one of my deploy.rb scripts was working properly, while this one wasn't.
In the link I posted in the question, one of the commenters pointed out the issue:
https://capistrano.lighthouseapp.com/projects/8716/tickets/56-query%5Frevision-unable-to-resolve-revision-for-head-on-repository
Note this error is also displayed if
you are using multiple github keys per
http://capistrano.lighthouseapp....
and you do not have these keys and a
corresponding entry in your
.ssh/config on the workstation you're
running the deploy from. so the
ls-remote is run locally. is there a
way to reference the repository at
github.com for this request while the
remote deploy uses
git#github-project1:user/project1.git
Also, see the following link for more details, since the whole ssh issue would apply even if you're not using github.
http://github.com/guides/multiple-github-accounts
Both your workstation and your server must be able to reach the repository at the address specified, if not then you may have to set :local_repository to how you access it from your workstaion, and :repository to be how your servers should access it.
For me Capistrano deployments with Git only seem to work when setting set :copy_cache, true
I've only used capistrano with git once, but never used or seen the use of ssh:// in the repository definition.
Try using set :repository, "git#67.24.9.133/home/git/project1.git" instead
Make sure the branch you are deploying from exists.
set :branch, "upgrade-to-2013.4.3"
is not equal to
set :branch, "upgrade-to-2013.3.4"
I have two config-files
/app/config/database.yml
and
/app/config/userconfig.yml
i don't want to put the database credentials and userconfig in the svn-repository, so i have database.yml.dist and userconfig.yml.dist checked in.
What is the best way to get copys of the dist-files in the shared-directory when deploying the app for the first time?
For later deploys i'll link to them from /app/current/config
You should place your config files in
/path/to/deployed_app/shared
Then in a capistrano task, sym link to those files:
namespace :deploy do
task :symlink_shared do
run "ln -s #{shared_path}/database.yml #{release_path}/config/"
end
end
before "deploy:restart", "deploy:symlink_shared"
In Capistrano v3, you can use a task called deploy:symlink:shared.
Provide a list of files you placed in the shared directory, so Capistrano knows which files to symlink when the task is run. This is typically done in deploy.rb:
set :linked_files, %w{
app/config/database.yml
app/config/userconfig.yml
}
Related: Capistrano - How to put files in the shared folder?