My project has a multi-step pipeline that includes some steps like Crawling, NLP, etc, but I'll just refer to them as Step1, Step2, etc. Additionally I want to be able to deploy to different environments (dev, prod, etc).
I figure I'll use multistage component for Capistrano in order to deploy to the different environments (e.g. cap dev deploy vs cap prod deploy).
It also seems to make intuitive sense to use roles for each pipeline step. However, each step runs fairly independently of the rest of the pipeline, so each step can be restarted/deployed independently of the other steps. It doesn't seem like Capistrano explicitly supports running tasks only for a specific role. What's a good way to do this?
Is the best way of doing this defining tasks specifically for each role? What if there are some tasks common between roles? Maybe helper methods is the answer there?
Not sure this is exactly what you are looking for, but when I want to do something just to a particular role, I use the capistrano shell. For example, let's say I want to deploy but only to my app servers, I might do the following:
cap production shell
cap>with app
cap>!deploy #or any other cap task you have
You can also scope by machine if you want. Again:
cap production shell
cap>on <machine name or ip>
cap>!deploy #or any other cap task you have
Hope it helps,
Scott
If you want to run a capistrano task from the command line, but only for a subset of defined roles, you can use the ROLES parameter.
The command below executes the task category:task only for role was:
cap ROLES=was category:task
The ROLES parameter is multivalued, so you can add more roles separated by a comma:
cap ROLES=was,db category:task
For more details, you can refer to the invoke documentation
Check this discussion Creating a Capistrano task that performs different tasks based on role
task :stop_memcached, :roles => :memcache do
...
end
Not sure at what version the feature was added, but cap will look in the ROLES environment variable to determine which roles to run so
$ ROLES=db cap deploy
Related
I've never worked with Capistrano before and currently I am fighting the urge to just scrap it and go back to my old manual ways.
As I understand, Capistrano V3 does not create the initial database because they feel it is the duty of the DB administrator.
So I must be missing something but I have followed their instructions but the initial cap staging deploy fails when it gets to the rake db:migrate step because the database does not exist.
Because of the failure, the symlink for current -> releases never gets created.
Is it just accepted general practice that we SSH into our boxes and cd into the first folder under releases and manually run rake db:create...?
And then from there, am I supposed to just run cap staging deploy again so that it finishes creating the symlinks?
Seems hacky for something that is supposed to make things easier and I am not sure if I am understanding this correctly or not.
Thanks.
It does make sense to leave certain things out of a deployment. As the initial set up and the routine deployments are very separate functions and require different specialties, or in large deployments even different skillsets. That said.. I'm totally with you - on the first deploy having to manually set up the database and certain files (specifically linked files like secrets.yml) is a step that just wastes my time.
I use this plugin:
https://github.com/capistrano-plugins/capistrano-postgresql
just add the require capistrano/postgresql to your capfile as you would any plugin
then run cap staging setup before the first time you run cap staging deploy
Capistrano on the whole is a very useful tool, but the definitions are so modular and distributed it can be difficult (or near impossible) to find the definition of a task when needed, or easily piece together the order of events.
I had only vaguely worked with Capistrano before v3, and I recall there being a "cold deploy" task.
However, I can't seem to find it anywhere within the capistrano repository, nor within any of the plugins (capistrano/rvm, capistrano/bundler, capistrano/rails, etc...). A simple repository search for the term 'cold' yields nothing
Where is this task defined? Does it exist in Capistrano v3? And is there an easy way to visualize all the tasks, in order, that run when I execute a certain command (e.g. bundle exec cap production deploy would list all 10,000+ deploy tasks)
Thanks!
actually there's no such task in capistrano 3.
You can see all task with command:
cap -T
for deploying i usually start with
cap production setup # Server setup tasks
cap production deploy:check # Check required files and directories exist
There is no such task like deploy:cold in capistrano 3 rather you can use the following command for the same thing
bundle exec cap production deploy setup
You can read the task definition in the lib files located in
lib/capistrano/tasks/deploy.rake
I have been thinking about this and searching this for ages without finding anything, so I am going to assume I hit the XY problem.
Let me describe my issue, this sounds common enough.
We use capistrano to deploy our web app and db. The relevant part is that we have a dedicated server for delayed job and we use capistrano to deploy to it and start/restart the processes. This is a custom number of workers with 2 different Gemfiles and 3 queues.
What I want to do is to start them up on server restart or, more to the point, on server clone + start.
I have tried calling cap production delayed_job:custom_start from the server itself.. didn't work. (This is the core of my non XY problem adjusted question). Not sure it even makes sense. But I want to know if it is possible. custom_start is a task that starts our set of workers.
Alternatively I am thinking of abstracting the code into a rake task or a script or something and calling it from both capistrano and where ever I would need to add it to start on startup. Does this make more sense?
Edit: Just found this post.. discouraging..
p.s. I just want to clarify that when I say server I mean my Machine/ec2 instance and not my web app restarting.
My Jenkins deploy jobs are littered with direct tasks developers can call such as cap dev app:fetch_logs, cap qa sanitize_environment, etc.
This feature of Capistrano is easy and verified.
What I am guessing you want to do is use Capistrano to setup rc.d files. Yes, you can do this. Should you use chef/puppet at this point? Consider. Scalr/Rightscale are funs things to look at too.
You can add a bash script as an .erb template for all your worker variables, then put upload the script into the deploy_to directory. Finally, you can setup another task (#{sudo}) to inject rc.d wrappers into rc.d. Or you rather than rc.d wrappers to you bash script, just call the bash script from rc.d-local. You can use sed to append to rc.d-local.
I ended up moving the delayed job related logic to its own script that accepts start/stop, and delegating to this script from capistrano. This means I can add this script to my rc.local.
We have a large deployment of around a dozen servers. We'd like to use the Whenever gem but I can't figure out a way to say which machine the cron jobs should go on! We only want these jobs to run on our server that does background jobs.
Is there a way to specify this?
If you deploy the project with Capistrano and you use the default Whenever recipe, you can create a new Capistrano role
role :whenever, "192.168.1.1"
and set the role in your deploy.rb file
set :whenever_roles, "whenever"
In this way, the task will be executed only on the specified server.
whenever is preconfigured to run against the db role, so if your db role is also the background machine's role you don't have to do the 'role :whenever, "192.168.0.1"' hack. see the codes
https://github.com/javan/whenever/blob/master/lib/whenever/capistrano.rb#L2
I'm trying to deploy the same rails app to two different locations with different app names, different logos, different stylesheets, etc.
I've got the code working based on an APP_NAME and a HOST_NAME variable I store in environments/production.rb. Now I need to actually deploy it, and I need a better solution than manually editing the environment file on the production machine.
The only way I can see to do it is to create a new production environment - e.g. production_app2 - and define APP_NAME and HOST_NAME differently in them. Is there a better way?
No no no! Don't edit the environment files. I mean, edit them as you need to for things that need to be configured the same for every deployment, but not for things that should be configurable between deployments.
For that, use configuration.
Throw a YAML file in config that looks something like this:
development:
:app_name: App 1
:host_name: something.com
test:
:app_name: App 1
:host_name: something.com
production:
:app_name: App 1
:host_name: something.com
Call it whatever makes sense. Let's say settings.yml.
Now load it with an initializer in config/initializers/settings.rb that looks like this:
SETTINGS = YAML.load_file("#{RAILS_ROOT}/config/settings.yml")[RAILS_ENV]
Now access your configuration like this:
SETTINGS[:app_name]
(If you don't want to change your existing code at all, inside config/initializers/settings.rb add lines that set your existing names like APP_NAME = SETTINGS[:app_name], etc.)
Note that this is one possible implementation of settings configuration, but even if another approach is taken it should be based on deployment-independent configuration. This can be much more easily and maintainably set up to persist between deployments and upgrades than mucking with environment files.
Again, to recap:
environment files are for configuration that is the same across all deployments
configuration files are for configuration that can change between deployments
Update
For Capistrano based deployments, this is what I use to symlink multiple configuration files in the new current from the shared directory (I think it originally came from an Ezra recipe from EngineYard):
after "deploy:update_code","deploy:symlink_configs"
namespace(:deploy) do
task :symlink_configs, :roles => :app, :except => {:no_symlink => true} do
configs = %w{ database settings }
configs.map! { |file| "ln -nfs #{shared_path}/config/#{file}.yml #{release_path}/config/#{file}.yml" }
run <<-CMD
cd #{release_path} && #{configs.join(' && ')}
CMD
end
end
I think that's a pretty good way.
Where we are we define different environments (e.g. 'staging', 'production', 'production_backup' - giving us a staging.rb, production.rb, production_backup.rb where you can define your specific APP_NAMEs and HOST_NAMEs) and can deploy to each of them using Capistrano. It works just fine.
This is a good link on it: http://www.egtheblog.com/?p=8
Because you are actually deploying to two different environments, it seems best to create two different environment files, each with their own settings. Make sure you pick descriptive names for your environment files, not just production2.
You could also store this information in the database, but I don't know if you're willing to accept such a dependency. I guess using a database would only make sense if the number of deployments is too large to manage easily with a few environment files.