Proper deployment of a Rails app with Mina and Foreman - ruby-on-rails

For production purposes I need three processes running. This is my procfile and I use Foreman to start them:
web: bundle exec rails s Puma -p $PORT
queuing: bundle exec clockwork clock.rb
workers: bundle exec rake resque:workers
For deployment I'm using Mina. What's the appropriate way to start Foreman at the end of deploy task? Currently I'm starting like this:
desc "Deploys the current version to the server."
task :deploy => :environment do
deploy do
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
to :launch do
queue "touch #{deploy_to}/tmp/restart.txt"
queue "bundle exec foreman start"
end
end
end
... but I don't think that's the proper way since the "mina deploy" command never successfully exits and the local console just starts outputting whatever these processes are doing.
Question number two: How do I initialize logging for each of these three processes separately in separate files?
And how do I prevent killing all of these three processes when one of them crashes? How do I make the process restart when it crashes?
Thanks!

OK, so that's 3 questions.
1) I think you want to detach foreman process from the terminal. That way the deployment process will finish and foreman process will be running even after you have disconnected from the server. nohup is great for that, e.g. this will launch your app and pipe all logs to server.log file:
nohup foreman start > server.log 2>&1 &
2) AFAIK, foreman doesn't let you do that. You should probably use another process management service (e.g. systemd, upstart). Thankfully, foreman lets you easily export your config to different process management formats (http://ddollar.github.io/foreman/#EXPORTING).
3) Again, you probably want to separate your processes and manage them separately via upstart, systemd, etc.

Related

Delayed job stopped working after changing server from passenger to puma

Recently I just changed server for my production environment from passenger to puma. Before puma delayed job was working fine without any error but after puma it stopped working.
I am running 3 Delayed job to handle incoming requests on two different servers by dividing them as server 1 : 1 Job and server 2 : 2 Jobs.
RAILS_ENV=production bin/delayed_job -i first --queues=job_1,job_2 start
This command I am using to start delayed job after deployment. I am using Capistrano 3 for deployment. Also I have upgrade version of Capistrano version 2 to 3.
In Capistrano 2 we had
require 'delayed/recepies'
but as per delayed job official documentation I have created a rake task and executing in order to start job again.
It creates a new entry in delayed_jobs table as Delayed::Backend::ActiveRecord::Job.
What's wrong I am doing here? Is there any worker not listening to my incoming calls? Should I need to start additional things?
----UPDATED----
This is the rake task I have created to restart the jobs.
task :restart do
on roles(delayed_job_roles) do
within release_path do
with rails_env: fetch(:rails_env) do
execute('cd #{current_path};RAILS_ENV=production bin/delayed_job -i first --queues=job_1,job_2 stop')
execute('cd #{current_path};RAILS_ENV=production bin/delayed_job -i second --queues=job_1,job_2 start')
end
end
end
end
after 'deploy:publishing', 'delayed_job:restart'
As I have mentioned that I am running two seperate servers to manage delayed_jobs. On server 2 delayed_job.log file is not updated with latest logs. When I try to run delayed_job it executes on server 1 instead of server 2. Queues are running over server 2 to handle those jobs and when it runs on server 1 it couldn't find the exact source, which I am looking for to get executed.

Run capistrano tasks locally

In my deploy.rb file I have a lot of tasks, one of which restarts the unicorn and some services. For example:
namespace :deploy do
task :restart do
invoke 'unicorn:restart' # using capistrano3-unicorn gem
invoke 'some_service:restart'
...
end
end
Now I need to run the same task locally through console on production server. I don't want to duplicate my code creating the same task as a rake task. I want to run this task, which is already exists and defined in the deploy.rb file, but I want to do it locally, not remotely. Is it possible ?
If I understand correctly, you have your Capistrano tasks which you run from your dev box as part of your deployment, and they execute on the server. You now want to run a specific command on the server, and you are logged into the server.
I'm not sure how to do it that way, but I'd suggest an alternative which might get you what you want. If you, on your dev box, run a specific task such as bundle exec cap production unicorn:restart, it will execute just that task on the server.

What's the better way to execute daemons when Rails server runing

I have some gems in my Rails App, such as resque, sunspot. I run the following command manually when the machines boots:
rake sunspot:solr:start
/usr/local/bin/redis-server /usr/local/etc/redis.conf
rake resque:work QUEUE='*'
Is there a better practice to run these daemon in the background? And is there any side-effect when run these tasks run in the background?
My solution to that is to use a mix of god, capistrano and whenever. A specific problem I have is that I want all app processes to be run as user, so initd scripts are not an option (this could be done, but it's quite a pain of user switching / environment loading).
God
The basic idea is to use god to start / restart / monitor processes. God may be difficult to get start with, but is very powerful :
running god alone will start all your processes (webserver, bg jobs, whatever)
it can detect a process crashed and restart it
you can group processes and batch restart them (staging, production, background, devops, etc)
Whenever
You still have to start god on server restart. A good mean to do so is to use user crontab. Most cron implementation have a special instruction called #reboot, which allows you to run a specific command on server restart :
#reboot /bin/bash -l -c 'cd /home/my_app && SERVER=true god -c production/current/config/app.god"
Whenever is a gem that allows easy management for crontab, including generating reboot command. While it's not absolutely necessary for achieving what I describe, it's really useful for its capistrano integration.
Capistrano
You not only want to start your processes on server restart, you also want to restart them on deploy. If your background jobs code is not up to date, problem will arise.
Capistrano allows to easily handle that, just ask god to restart the whole group (like : god restart production) in a post deploy capistrano task, and it will be handled seamlessly.
Whenever's capistrano integration also ensure your crontab is always up to date, updating it if you changed your config/schedule.rb file.
You can use something like foreman to manage these processes. You can define process types and other things in a Procfile and you can start and do whatever with them.

How to autostart Rails delayed jobs?

I'm using delayed job to create job queues such as 'mailer'
For this to work I have to run this:
$ RAILS_ENV=development QUEUE=mailer rake jobs:work
But if the server crashes and is restarted, I need the worker to start running again automatically.
What would be the recommended way to deal with this?
You need to use a third-party service like monit/bluepill/god/upstart to watch the process and restart it. I recommend using the combination of foreman and upstart. See here: http://blog.daviddollar.org/2011/05/06/introducing-foreman.html
Some time ago I wrote a patch for the DelayedJob to reload the classes for every job in development mode. Same patch should work for your requirement also.
betamatt's approach is definitely one way to do it if you have such a monitoring tool in place.
Another way to do it would simply be to add a script to your OS's startup which runs the RAILS_ENV=development QUEUE=mailer rake jobs:work command under a user who has the necessary permissions.
Here's an example of how to do it on Ubuntu using Upstart, but if you lookup similar init.d methods, or whatever is the relevant for your server OS, you'll find other ways. What you're looking for, basically, is "How to run a script on startup [your OS name]", and then wrap your command in an executable script.
I had the same issue with my application am working with. So i wrote a rake task which runs every minute(as a cron job). When delayed job starts it will create a .pid file in the temp folder. I used this to check the existence of a delayed job process. If the file doesn't exist i ran the console command through code.
delayed_job_status = File.file?("./tmp/pids/delayed_job.pid")
This will check the existence of process. If nil response go to next statement
./bundle exec script/delayed_job start production
This will start delayed job
My solution was creating the bash script in user's home "delayed_job_startup.sh"
which contain
#!/bin/bash
cd /home/deploy/project/current/
RAILS_ENV=production bin/delayed_job start
and in file /etc/rc.local I run this script from my user
su -s /bin/bash - deploy /home/deploy/delayed_job_startup.sh

Rails 3 delayed_job start after rebooting

I have my rails site deployed under apache. The apache is run as a service. Now I have added delayed_job there and things work fine.
Now I want to start the workers together with apache, e.g, After rebooting the server, my site and workers are up and ready so I don't have to log in and type "sudo RAILS_ENV=production script/delayed_job -n 2 start".
Another issue is that whenever I want to start the delayed_job I have to use "sudo"...
Any idea how to avoid those 2 issues?
Thanks for your help.
Use the whenever gem and its 'every :reboot' functionality. In schedule.rb:
environment = ENV['RAILS_ENV'] || 'production'
every :reboot do
command "cd #{path} && #{environment_variable}=#{environment} bin/delayed_job --pool=queue1:2, --pool=queue2,queue3:1 restart"
end
Could you just create a shell script to execute the commands you need?
#!/bin/sh
# stop delayed job
# restart apache
apachectl restart
# start delayed job
sudo RAILS_ENV=production script/delayed_job -n 2 start
It sounds like you want to have delayed_job automatically start after apache starts when you boot up the hardware. If that's the case you need to write an init script in /etc/init.d or /etc/rc.d/init.d (depending on your system). This page gives a decent primer on this:
http://www.philchen.com/2007/06/04/quick-and-dirty-how-to-write-and-init-script

Resources