Manage sidekiq with init.d script using RVM - ruby-on-rails

I'm using the init.d script provided (the init.d script from the sidekiq github repo), but I am on an ubuntu system with RVM installed system wide.
I cannot seem to figure out how to cd into my app directory and issue the command without there being some complaining in the log and nothing actually starting.
Question: What should the startup command for sidekiq look like in my init.d script when I am using RVM? My user is named ubuntu. Currently I have this in my init.d script:
START_CMD="$BUNDLE exec $SIDEKIQ"
# where bundle is /usr/local/rvm/gems/ruby-1.9.3-p385/bin/bundle
# and sidekiq is sidekiq
# I've also tried with the following args: -e $APP_ENV -P $PID_FILE -C $APP_CONFIG/sidekiq.yml -d -L $LOG_FILE"
RETVAL=0
start() {
status
if [ $? -eq 1 ]; then
[ `id -u` == '0' ] || (echo "$SIDEKIQ runs as root only .."; exit 5)
[ -d $APP_DIR ] || (echo "$APP_DIR not found!.. Exiting"; exit 6)
cd $APP_DIR
echo "Starting $SIDEKIQ message processor .. "
echo "in dir `pwd`"
su - ubuntu -c "$START_CMD >> $LOG_FILE 2>&1 &"
RETVAL=$?
#Sleeping for 8 seconds for process to be precisely visible in process table - See status ()
sleep 8
[ $RETVAL -eq 0 ] && touch $LOCK_FILE
return $RETVAL
else
echo "$SIDEKIQ message processor is already running .. "
fi
}
My sidekiq.log gives me this error:Could not locate Gemfile. However, I print the working directory and I am most definitely in my app's current directory, according to the echo pwd, at the time this command is executed.
When I take out the su - ubuntu -c [command here], I get this error:
/usr/bin/env: ruby_noexec_wrapper: No such file or directory
My solution is to just start the process manually. When I manually cd into my app directory and issue this command:
bundle exec sidekiq -d -L log/sidekiq.log -P tmp/pids/sidekiq.pid
things go as planned, and then
sudo /etc/init.d/sidekiq status
tells me things are up and running.
Also, sudo /etc/init.d/sidekiq stop and status work as expected.

I wrote a blog post a few months ago on my experience writing an init.d script for Sidekiq, however I was using rbenv rather than RVM.
https://cdyer.co.uk/blog/init-script-for-sidekiq-with-rbenv/
I think you should be able to use something almost identical except for modifying the username and app dir variables.

use wrappers:
BUNDLER=/usr/local/rvm/wrappers/ruby-1.9.3-p385/bundle
in case the bundler wrapper is not available generate it with:
rvm wrapper ruby-1.9.3-p385 --no-links bundle # OR:
rvm wrapper ruby-1.9.3-p385 --no-links --all
you can use aliases to make it easier:
rvm alias create my_app 1.9.3-p385
and then use it like this:
BUNDLER=/usr/local/rvm/wrappers/my_app/bundle
this way you will not have to change the script when application ruby changes - just update the alias, there is a good description/integration for this in rvm-capistrano => https://github.com/wayneeseguin/rvm-capistrano/#create-application-alias-and-wrappers

Related

ruby unicorn as service in docker - uses wrong rake

my problem is that i can start unicorn as a service in docker, though it works just fine if i start it from command line.
trying to build ruby with unicorn and nginx web server docker image.
using as a base FROM ruby:2.3 image. but with latest ubuntu saw same troubles.
this article explains pretty straight forward how to use unicorn with nginx.
everything seems to be working if i start it from bash like this
(cd /app && bundle exec unicorn -c /app/config/unicorn.rb -E uat -D)
but i see errors if start it s as service
service unicorn_appname start
the error is:
bundler: command not found: unicorn
after i did some investigation i've realized that the issue is most probably in env variables because service essentially tries to execute my command with su - root -c prefix:
su - root -c " cd /app && bundle exec unicorn -c config/unicorn.rb -E uat -D"
this command produces same error.
though i am logged in as root in my bash as well
after googling for a while i found partial solution - set PATH env variable like this:
su - root -c "PATH=\"$(ruby -e 'print Gem.default_dir')/bin:$PATH\" && cd /app && bundle exec unicorn -c config/unicorn.rb -E uat -D"
but now i see Could not find rake-12.0.0 in any of the sources.
and rake --version returns rake, version 12.0.0. Meanwhile su - root -c "rake --version" returns rake, version 10.4.2
which rake returns /usr/local/bundle/bin/rake, meanwhile su - root -c "which rake" returns /usr/local/bin/rake
so my guess is that service tries to use wrong path for rake.
how do i change default rake path? or any other suggestion where to dig into?
---------------- UPDATE - kinda solution ---------------------
i think i found the reason of all my issues with bundler in docker. looks like all env variables for bundler are set in shell startup part. thus they are not there if i run it as sudo su - appuser -c "...cmd..."
so i've tested it by running printenv right in bash. and another one like this sudo su - appuser -c "printenv". - found big difference.
since i was building docker i've set them through docker file, but it also works if just export them.
ENV PATH=/usr/local/bundle/bin:/usr/local/bundle/gems/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV RUBYGEMS_VERSION=3.0.3
ENV RUBY_VERSION=2.3.8
ENV GEM_HOME=/usr/local/bundle
ENV BUNDLE_PATH=/usr/local/bundle
ENV BUNDLE_SILENCE_ROOT_WARNING=1
ENV RUBY_MAJOR=2.3
ENV BUNDLE_APP_CONFIG=/usr/local/bundle
i also did
RUN bundle config app_config /usr/local/bundle && bundle config path /usr/local/bundle
and since the right way is to not use root for web app i rebuild everything in docker file so it creates and uses separate user (but this part i guess is optional):
RUN adduser --disabled-password --gecos "" appuser
....
#installing sudo
RUN apt-get update
RUN apt-get install -y sudo
....
# gives sudo to new user
RUN echo "appuser ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/appuser && chmod 0440 /etc/sudoers.d/appuser
....
#don't forget to give rights to your folder for new user like this:
#RUN sudo chown -R appuser:appuser /usr/local
....
#utilize new user
USER appuser
#bundle install and rest stuff is here
....
hope my update save someone time

AWS - Elastic Beanstalk Sidekiq Not Starting Automatically

Hey I'm running a rails application on an AWS Elastic Beanstalk instance and have run into a couple issues using Sidekiq for background jobs. I'm using Sidekiq to send welcome and password reset emails.
The application is running Rails 4.2.3. The Elastic Beanstalk instance is running 64bit Amazon Linux 2015.03 v2.0.1 running Ruby 2.2 (Puma). I used the following tutorial to set up a Redis node and Sidekiq on Elastic Beanstalk, and used the sidekiq.config from the following GIST to place inside my .ebextentions folder. I did not use any of the other recommended config files from the GIST.
Everything seemed to go off without a hitch. The application deployed correctly and there were no errors in any of the logs. However, when I created a new user, triggering a welcome email, the email failed to send. I checked the Sidekiq web application and it appeared that my jobs were scheduled but never executed. On further inspection I determined that Sidekiq wasn't actually running. I connected to the EC2 instance via SSH and manually ran bundle exec sidekiq and the email sent.
For some reason Sidekiq isn't getting run when the application is being deployed.
Here are the contents of my sidekiq.config:
# Sidekiq interaction and startup script
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/containerfiles/envvars
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid
cd $EB_CONFIG_APP_CURRENT
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -TERM `cat $PIDFILE`
sleep 10
rm -rf $PIDFILE
fi
fi
BUNDLE=/usr/local/bin/bundle
SIDEKIQ=/usr/local/bin/sidekiq
$BUNDLE exec $SIDEKIQ \
-e production \
-P /var/app/containerfiles/pids/sidekiq.pid \
-C /var/app/current/config/sidekiq.yml \
-L /var/app/containerfiles/logs/sidekiq.log \
-d
"/opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/containerfiles/envvars
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -USR1 `cat $PIDFILE`
sleep 10
fi
fi
What am I missing? Are there other files I need to add to my .ebextentions folder? Why isn't Sidekiq being initiated?

error while running ruby application at system startup in ubuntu

I am on Ubuntu 12.04 machine. Have a script file which runs when entered manually in terminal
gnome-terminal -e /home/precise/Desktop/cartodb/script.sh
The content of script file is
cd /home/ubuntupc/Desktop/cartodb20/
sh /home/ubuntupc/.rvm/scripts/rvm
bundle exec foreman start -p 3000
So what i tried to do is to run this script at every system start up. So on Startup Applications
command: gnome-terminal -e /home/precise/Desktop/cartodb/script.sh
On terminal Edit -> Profile Preferences -> Title and Command
Checked the "Run command as a login shell"
But this seems to be not working. When restarted the machine found these error in terminal
The child process exited normally with status 127.
ERROR: RVM Ruby not used, run `rvm use ruby` first.
Some info regarding the installed packages and system.
$ which ruby
/home/ubuntupc/.rvm/rubies/ruby-1.9.2-p320/bin/ruby
$ which rails
/home/ubuntupc/.rvm/gems/ruby-1.9.2-p320/bin/rails
$ which gem
/home/ubuntupc/.rvm/rubies/ruby-1.9.2-p320/bin/gem
$ cat ~/.bash_profile
[[ -s "$HOME/.profile" ]] && source "$HOME/.profile" # Load the default .profile
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function*
$ which -a ruby
/home/ubuntupc/.rvm/rubies/ruby-1.9.2-p320/bin/ruby
$ sudo update-alternatives --config ruby
update-alternatives: error: no alternatives for ruby.
$ sudo find / -name "rubygems" -print
/home/ubuntupc/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/site_ruby/1.9.1/rubygems
/home/ubuntupc/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/rubygems
/home/ubuntupc/.rvm/src/ruby-1.9.2-p320/lib/rubygems
/home/ubuntupc/.rvm/src/ruby-1.9.2-p320/test/rubygems
/home/ubuntupc/.rvm/src/ruby-1.9.2-p320/test/rubygems/rubygems
/home/ubuntupc/.rvm/src/ruby-1.9.2-p320/doc/rubygems
/home/ubuntupc/.rvm/src/rubygems-2.2.1/lib/rubygems
/home/ubuntupc/.rvm/src/rubygems-2.2.1/test/rubygems
/home/ubuntupc/.rvm/src/rubygems-2.2.1/test/rubygems/rubygems
/home/ubuntupc/.rvm/src/rvm/scripts/functions/rubygems
/home/ubuntupc/.rvm/src/rvm/scripts/rubygems
/home/ubuntupc/.rvm/scripts/functions/rubygems
/home/ubuntupc/.rvm/scripts/rubygems
/usr/lib/ruby/1.9.1/rubygems
/usr/local/rvm/rubies/ruby-1.9.2-p320/lib/ruby/site_ruby/1.9.1/rubygems
/usr/local/rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/rubygems
/usr/local/rvm/src/ruby-1.9.2-p320/lib/rubygems
/usr/local/rvm/src/ruby-1.9.2-p320/test/rubygems
/usr/local/rvm/src/ruby-1.9.2-p320/test/rubygems/rubygems
/usr/local/rvm/src/ruby-1.9.2-p320/doc/rubygems
/usr/local/rvm/src/rubygems-2.2.0/lib/rubygems
/usr/local/rvm/src/rubygems-2.2.0/test/rubygems
/usr/local/rvm/src/rubygems-2.2.0/test/rubygems/rubygems
/usr/local/rvm/src/rvm/scripts/functions/rubygems
/usr/local/rvm/src/rvm/scripts/rubygems
/usr/local/rvm/scripts/functions/rubygems
/usr/local/rvm/scripts/rubygems
Please point out what i am missing as i am new to the ruby applications.
Thanks in advance

Start Unicorn with Runit and User's RVM

I'm deploying my Rails App servers with Chef. Have just swapped to RVM from a source install of Ruby (because I was having issues with my deploy user).
Now I have my deploy sorted, assets compiled and bundler's installed all my gems.
The problem I have is supervising Unicorn with Runit..
RVM is not installed as root user - only as my deploy user has it, as follows:
$ rvm list
rvm rubies
=* ruby-2.0.0-p247 [ x86_64 ]
I can manually start Unicorn successfully from my deploy user. However, it won't start as part of runit.
My run file looks like this. I have also tried the solution in this SO question unsuccessfully..
#!/bin/bash
cd /var/www/html/deploy/production/current
exec 2>&1
exec chpst -u deploy:deploy /home/deploy/.rvm/gems/ruby-2.0.0-p247/bin/unicorn -E production -c config/unicorn_production.rb
If I run it manually, I get this error:
/usr/bin/env: ruby_noexec_wrapper: No such file or directory
I created a small script (gist here) which does run as root. However, if I call this from runit, I see the workers start but I get two processes for runit and I can't stop or restart the service:
Output of ps:
1001 29062 1 0 00:08 ? 00:00:00 unicorn master -D -E production -c /var/www/html/deploy/production/current/config/unicorn_production.rb
1001 29065 29062 9 00:08 ? 00:00:12 unicorn worker[0] -D -E production -c /var/www/html/deploy/production/current/config/unicorn_production.rb
root 29076 920 0 00:08 ? 00:00:00 su - deploy -c cd /var/www/html/deploy/production/current; export GEM_HOME=/home/deploy/.rvm/gems/ruby-2.0.0-p247; /home/deploy/.rvm/gems/ruby-2.0.0-p247/bin/unicorn -D -E production -c /var/www/html/deploy/production/current/config/unicorn_production.rb
1001 29083 29076 0 00:08 ? 00:00:00 -su -c cd /var/www/html/deploy/production/current; export GEM_HOME=/home/deploy/.rvm/gems/ruby-2.0.0-p247; /home/deploy/.rvm/gems/ruby-2.0.0-p247/bin/unicorn -D -E production -c /var/www/html/deploy/production/current/config/unicorn_production.rb
What should I do here? Move back to monit which worked nicely?
your run file is doing it wrong, you are using the binary without setting the environment, for that purpose you should use wrappers:
rvm wrapper ruby-2.0.0-p247 --no-links unicorn
To simplify the script use alias so it does not need to be changed when you decide which ruby should be used:
rvm alias create my_app_unicorn ruby-2.0.0-p247
And change the script to:
#!/bin/bash
cd /var/www/html/deploy/production/current
exec 2>&1
exec chpst -u deploy:deploy /home/deploy/.rvm/wrappers/my_app_unicorn/unicorn -E production -c config/unicorn_production.rb
This will ensure proper environment is used for execution of unicorn and any time you want change ruby used to run it just crate alias to a new ruby.

ROR, Redis, Resque, God & Cron on Ubuntu Server - Boot

I have made several jobs that god takes care of in my ruby application. However when the server reboots the job stops. I want to avoid this so I've made this script on my server. It looks like this.
my_app.sh
#!/bin/bash
# god tasks
#
case $1 in
start)
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/god
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/god start
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/god load /usr/local/Linux/apache2/www/hej.se/ruby/config/resque.god
/usr/local/rvm/gems/ruby-1.9.3-p194/bin/god load /usr/local/Linux/apache2/www/hej.se/ruby/config/resque_schedule.god
;;
esac
exit 0
If I log in manually and write
"/etc/init.d/my_app start"
it gives me
Sending 'start' command
No matching task or group
Sending 'load' command with action 'leave'
The following tasks were affected:
resque-0
resque-1
resque-2
resque-3
resque-4
Sending 'load' command with action 'leave'
The following tasks were affected:
resque_scheduler
And everything works, it does what I want it to do, i.e the jobs.
I have tried several ways to start this script on boot (Linux 10.4.4 LTS), rc.local, rc-default and now my latest attempt is crontab.
The script must be run under my user and not root, (it can't find the ruby installation if I run it under root).
Because of this I've configured the crontab under my user account:
#reboot /etc/init.d/my_app start
Sadly this doesn't work... I don't what I'm doing wrong. And this should probably not be necessary. I mean shouldn't you be able to this per auto when booting up the ruby application?
Im using passenger on this server, I don't know if this has something to do with it?
The solution below with the changes I made to the sh:
my_app.sh
bash -c "source /usr/local/rvm/scripts/rvm && /usr/local/rvm/gems/ruby-1.9.3-p194/bin/god"
bash -c "source /usr/local/rvm/scripts/rvm && /usr/local/rvm/gems/ruby-1.9.3-p194/bin/god start"
bash -c "source /usr/local/rvm/scripts/rvm && /usr/local/rvm/gems/ruby-1.9.3-p194/bin/god load /usr/local/Linux/apache2/www/hej.se/ruby/config/resque.god"
bash -c "source /usr/local/rvm/scripts/rvm && /usr/local/rvm/gems/ruby-1.9.3-p194/bin/god load /usr/local/Linux/apache2/www/hej.se/ruby/config/resque_schedule.god"
Forget the cronjob.
Centos/Fedora:
sudo chmod a+x /etc/init.d/my_app
sudo chkconfig --add my_app
sudo chkconfig my_app on
Ubuntu/Debian:
sudo update-rc.d my_app defaults
Both of these symlink the script to /etc/rc1.d, /etc/rc2.d, etc., and make the script available to run on boot for those runlevels.

Resources