We have a large deployment of around a dozen servers. We'd like to use the Whenever gem but I can't figure out a way to say which machine the cron jobs should go on! We only want these jobs to run on our server that does background jobs.
Is there a way to specify this?
If you deploy the project with Capistrano and you use the default Whenever recipe, you can create a new Capistrano role
role :whenever, "192.168.1.1"
and set the role in your deploy.rb file
set :whenever_roles, "whenever"
In this way, the task will be executed only on the specified server.
whenever is preconfigured to run against the db role, so if your db role is also the background machine's role you don't have to do the 'role :whenever, "192.168.0.1"' hack. see the codes
https://github.com/javan/whenever/blob/master/lib/whenever/capistrano.rb#L2
Related
I have a production server with two instances app1 and app2 to which I use capistrano to deploy my application, ie: cap app1 deploy, cap app2 deploy.
I am now configuring the clockwork to schedule my delayed jobs. But what happens the jobs are scheduled twice, because the clockwork process runs on both app1 and app2 after application is deployed.
How can I force the capistrano to run the clockwork process only on app2 ?
Thanks for a hint.
Miroslav
SOLUTION
Add the following settings into the deploy.rb:
set :clockwork_role, :clock
And then configure only one of the instances to use the role :clock, ie:
server 'URL', user: 'deploy', roles: %w{app db web clock}
In your case you can set some config variable to true in one case, and to false in the second, and check it when setting clockwork.
But want to mention that you are using different 'stages' (which are usually 'staging', 'production', 'qa', 'features', etc) for the different servers of one stage.
Capistrano has 'roles' (app, db, background jobs, etc) each can have several servers.
Also please mention which capistrano version you're using - there were big changes from 2 to 3
I have been thinking about this and searching this for ages without finding anything, so I am going to assume I hit the XY problem.
Let me describe my issue, this sounds common enough.
We use capistrano to deploy our web app and db. The relevant part is that we have a dedicated server for delayed job and we use capistrano to deploy to it and start/restart the processes. This is a custom number of workers with 2 different Gemfiles and 3 queues.
What I want to do is to start them up on server restart or, more to the point, on server clone + start.
I have tried calling cap production delayed_job:custom_start from the server itself.. didn't work. (This is the core of my non XY problem adjusted question). Not sure it even makes sense. But I want to know if it is possible. custom_start is a task that starts our set of workers.
Alternatively I am thinking of abstracting the code into a rake task or a script or something and calling it from both capistrano and where ever I would need to add it to start on startup. Does this make more sense?
Edit: Just found this post.. discouraging..
p.s. I just want to clarify that when I say server I mean my Machine/ec2 instance and not my web app restarting.
My Jenkins deploy jobs are littered with direct tasks developers can call such as cap dev app:fetch_logs, cap qa sanitize_environment, etc.
This feature of Capistrano is easy and verified.
What I am guessing you want to do is use Capistrano to setup rc.d files. Yes, you can do this. Should you use chef/puppet at this point? Consider. Scalr/Rightscale are funs things to look at too.
You can add a bash script as an .erb template for all your worker variables, then put upload the script into the deploy_to directory. Finally, you can setup another task (#{sudo}) to inject rc.d wrappers into rc.d. Or you rather than rc.d wrappers to you bash script, just call the bash script from rc.d-local. You can use sed to append to rc.d-local.
I ended up moving the delayed job related logic to its own script that accepts start/stop, and delegating to this script from capistrano. This means I can add this script to my rc.local.
As far as I can tell, the capistrano :db role is used only to run migrations.
(Thus, in most cases it probably shouldn't actually be the server that runs your database. Why would you have a ruby/rails stack there (or allow ssh logins there)? it's just whatever server you want to actually execute the rails migrations).
And only the server identified as db role with :primary => true is used to run migrations.
So any other servers identified as 'db' role but without :primary => true... are used for nothing at all? So why does the default deploy.rb created by capify . encourage you to list them? What would you even list here?
Anything I'm missing?
Obviously, the name of role :db is misleading. As you pointed out, Capistrano defines it (with :primary => true) as a host that we execute rake db:migrate on, but database servers are not always running on such hosts. We can alter the schema of remote database server through a Rails app. The correct role name for this kind of host is not :db.
Inferring from the comments on lib/capistrano/configuration/roles.rb, the original meaning of the role :db is a host on which database servers are running. We are expected to login to the :db hosts and do some tasks.
The designers of Capistrano should have defined :migration role or something else for the deploy:migrate task. But the association between the :db role with this task was defined six years ago with 9a6d2fb and has not been changed since then.
Generally speaking, the users of Capistrano can define roles and associate them with tasks freely. The deploy:migrate task is provided just as a recipe for Rails developers. Unfortunately, this recipe includes a misconception about how we do the database migration and is used widely for a long time.
You may have a setup that includes a master database server and multiple slave servers. In some cases, you could have an capistrano task that needs to be run on all database servers. In others, you may want to run a task (for instance, a migration) only on the master server and allow the changes to propagate to the slave instances.
I don't really understand what model of unix accounts/permissions is intended with Capistrano.
Let's say I've got a rails app called Widget and I'll be deploying with passenger. In general, pre-capistrano, I want the entire ./widget directory to be owned by a user called 'widget'. And then by default default passenger will run the app process as user 'widget' too, because passenger runs as user that owns the file.
And the whole point of this is for that 'widget' account to have fairly limited permissions, right? Since a web app will be running under that account?
So since I want the files to be owned by 'widget', I tell cap
set :user, "widget"
But now when I run "cap deploy:setup", it wants to 'sudo' from that account. No way that 'widget' account gets sudo privileges, the whole point is keeping this account limited privs.
Okay, I can tell cap not to use sudo... but then it won't actually have privs to do what it needs, maybe.
I can find a workaround to this too. But I start thinking, why do I keep having to re-invent the wheel? I mistakenly thought the point of cap recipes was to give me some best practices here. Anyway... what do people actually do here?
Use one unix account for install, but then have cap somehow 'chown' it to something else? Use one unix account, but have something non-cap (puppet?) do enough setup so that account doesn't need to sudo to get things started? What? What am I missing?
You can avoid some of the headache by using Passenger most commonly with Nginx as your webserver.
Then to restart web services the unprivileged Widget user creates a file in his path and Passenger will automatically restart Nginx when it sees that file being present.
This is enabled via the following in your config/deploy.rb:
namespace :deploy do
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
As for other privileged tasks for MySQL/DB administration your database.yml provides the credentials necessary to handle rake migration tasks.
So really the only time you would need something more privileged would be for system wide installation of gems, ruby, or rails updates, but a lot of that depends on how your production environment was setup/installed.
Given Passenger+Nginx and separate credentials for DB you can disable sudo and see if you encounter any errors during your Capistrano deploy process and then pickup from there.
My project has a multi-step pipeline that includes some steps like Crawling, NLP, etc, but I'll just refer to them as Step1, Step2, etc. Additionally I want to be able to deploy to different environments (dev, prod, etc).
I figure I'll use multistage component for Capistrano in order to deploy to the different environments (e.g. cap dev deploy vs cap prod deploy).
It also seems to make intuitive sense to use roles for each pipeline step. However, each step runs fairly independently of the rest of the pipeline, so each step can be restarted/deployed independently of the other steps. It doesn't seem like Capistrano explicitly supports running tasks only for a specific role. What's a good way to do this?
Is the best way of doing this defining tasks specifically for each role? What if there are some tasks common between roles? Maybe helper methods is the answer there?
Not sure this is exactly what you are looking for, but when I want to do something just to a particular role, I use the capistrano shell. For example, let's say I want to deploy but only to my app servers, I might do the following:
cap production shell
cap>with app
cap>!deploy #or any other cap task you have
You can also scope by machine if you want. Again:
cap production shell
cap>on <machine name or ip>
cap>!deploy #or any other cap task you have
Hope it helps,
Scott
If you want to run a capistrano task from the command line, but only for a subset of defined roles, you can use the ROLES parameter.
The command below executes the task category:task only for role was:
cap ROLES=was category:task
The ROLES parameter is multivalued, so you can add more roles separated by a comma:
cap ROLES=was,db category:task
For more details, you can refer to the invoke documentation
Check this discussion Creating a Capistrano task that performs different tasks based on role
task :stop_memcached, :roles => :memcache do
...
end
Not sure at what version the feature was added, but cap will look in the ROLES environment variable to determine which roles to run so
$ ROLES=db cap deploy