Rolling restart of process group in monit - monitoring

Question

Does anyone have any suggestions of how I might go about achieving a rolling restart of a process group using monit?
Thanks in advance,
fturtle

Answer 1

I'm not sure for which server you are talking about. But I can provide you an example for thin which supports rolling restart itself. (option onebyone: true)
So for monit you can use something like,
if ... then exec '/path/to/thin_restart.sh'
And thin_restart.sh would be something like,
source /path/to/scripts/rvm
rvm use your_gemset#some_ruby
thin -C thin.yml restart
And contents of thin.yml would look like,
port: 1337
pid: tmp/pids/thin.pid
rackup: /path/to/config.ru
daemonize: true
servers: 2
onebyone: true
There are other ways to fine tune this restarts based on pid. You can monitor files with pids and restart only those thin process based on conditions.
e.g.
check process app-1337
with pid /path/to/app.1337.pid
start = 'thin -d -p 1337 start'
stop = 'thin -d -p 1337 -P /path/to/thin.1337.pid stop'
if cpu usage > 50% then restart
check process app-1338
with pid /path/to/app.1338.pid
start = 'thin -d -p 1338 start'
stop = 'thin -d -p 1338 -P /path/to/thin.1338.pid stop'
if cpu usage > 50% then restart
The other way would be of using groups which monit provides.
Extending above example.
check process app-1337
with pid /path/to/app.1337.pid
group thin
group thin-odd
start = 'thin -d -p 1337 start'
stop = 'thin -d -p 1337 -P /path/to/thin.1337.pid stop'
if cpu usage > 50% then restart
check process app-1338
with pid /path/to/app.1338.pid
group thin
group thin-even
start = 'thin -d -p 1338 start'
stop = 'thin -d -p 1338 -P /path/to/thin.1338.pid stop'
if cpu usage > 50% then restart
check process app-1337
with pid /path/to/app.1339.pid
group thin
group thin-odd
start = 'thin -d -p 1339 start'
stop = 'thin -d -p 1339 -P /path/to/thin.1339.pid stop'
if cpu usage > 50% then restart
check process app-1340
with pid /path/to/app.1340.pid
group thin
group thin-even
start = 'thin -d -p 1340 start'
stop = 'thin -d -p 1340 -P /path/to/thin.1340.pid stop'
if cpu usage > 50% then restart
So now you can do following to restart all:
monit -g thin restart
or to achieve sort of rolling restart, restart odd ones then even.
To restart only odd ones:
monit -g thin-odd restart
and to restart even:
monit -g thin-even restart