AWS Elastic Beanstalk and Whenever Gem - ruby-on-rails

I have a Rails 4.2.1 app, using Ruby 2.2. I'm trying to use the Whenever Gem to update cron tasks on my Elastic Beanstalk from my code base. I've followed a few resources from AWS where you can add files to the .ebextensions folder and use EB's post deployment hook via shell files. Here are a couple resources:
https://forums.aws.amazon.com/thread.jspa?threadID=137136
http://www.dannemanne.com/posts/post-deployment_script_on_elastic_beanstalk_restart_delayed_job
http://blog.endpoint.com/2015/01/elastic-beanstalk-whenever.html
Following the blog articles, I added the file below, uncommented the gitignore line about files in the .ebextensions folder, and deployed my app. Unfortunatley, I have been able to see any changes. I've checked the log files (log/eb-tools.log, log/cron, etc.), and grepped all the log files for the shell file I create, whenever, and more. No luck though.
commands:
create-post-dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_update_cron.sh"
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Using similar syntax as the appdeploy pre hooks that is managed by AWS
# Loading environment data
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
EB_APP_CURRENT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
EB_APP_PIDS_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
# Setting up correct environment and ruby version so that bundle can load all gems
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
# Now we can do the actual restart of the worker. Make sure to have double quotes when using env vars in the command.
su -c "cd $EB_APP_CURRENT_DIR; bundle exec whenever --update-cron --set='environment=$RACK_ENV'" - $EB_APP_USER
How can I make sure this shell file is getting called? Can I test it without a new deployment each time? Also, I'm open to other options if the Whenever gem is not the best option. I mainly want to be able to have my cron tasks managed in code and under version control.
Thanks in advance!
UPDATE:
I had a type on the .ebextensions folder, which was causing it to not be added. After that was fixed, I was able to read error messages, and create a cron script that updated the crontab using the Whenever gem.

I was able to figure this out after fixing a typo in my .ebextensions folder name. Afterwards, the scripts got compiled, and log messages started to appear. After reading the log messages, I came up with the following config script (.ebextensions/01_cron.config):
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/01_cron.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Using similar syntax as the appdeploy pre hooks that is managed by AWS
set -xe
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
cd $EB_DEPLOY_DIR
su -c "bundle exec whenever --update-cron"
su -c "crontab -l"

Related

Whenever Gem AWS elastic beanstalk

Apologies if this is a duplicate - there are a few results with similar questions but most of the answers are quite dated and I am having trouble getting it to work.
I am using the whenever gem to run a single task every hour in my rails app but I can't seem to get AWS to configure the Cron settings right.
My first thought was to SSH into the instance and run the whenever update /write command which went through fine but my task didn't run.
After reviewing the old questions on here I created a an ebextensions folder with a config file to run the update command but still no joy. I used this config script but don't really understand it so I assume I have not set it up right - can someone please assist?
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_update_cron.sh"
mode: "000755"
owner: root
group: root
content: |
#! /usr/bin/env bash
. /opt/elasticbeanstalk/containerfiles/envvars
su -c "cd $EB_CONFIG_APP_CURRENT; bundle exec whenever --update-cron"
- $EB_CONFIG_APP_USER
EDIT Added Logs
2018-01-09T23:24:45.282Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AppDeployStage1/AppDeployPostHook] : Starting activity...
[2018-01-09T23:24:45.282Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AppDeployStage1/AppDeployPostHook/01_cron.sh] : Starting
activity...
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AppDeployStage1/AppDeployPostHook/01_cron.sh] : Completed
activity. Result:
[write] crontab file updated
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AppDeployStage1/AppDeployPostHook] : Completed activity.
Result:
Successfully execute hooks in directory
/opt/elasticbeanstalk/hooks/appdeploy/post.
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AppDeployStage1] : Completed activity. Result:
Application version switch - Command CMD-AppDeploy stage 1 completed
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter] : Starting activity...
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter/ConfigLogRotation] : Starting activity...
[2018-01-09T23:24:46.618Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter/ConfigLogRotation/10-config.sh] : Starting
activity...
[2018-01-09T23:24:46.778Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter/ConfigLogRotation/10-config.sh] : Completed
activity. Result:
Disabled forced hourly log rotation.
[2018-01-09T23:24:46.779Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter/ConfigLogRotation] : Completed activity.
Result:
Successfully execute hooks in directory
/opt/elasticbeanstalk/addons/logpublish/hooks/config.
[2018-01-09T23:24:46.779Z] INFO [1641] - [Application update app-5c64-
180109_162309#15/AddonsAfter] : Completed activity.
[2018-01-09T23:24:46.779Z] INFO [1641] - [Application update app-5c64-
EDIT 2 my new config code that now forms the following errors:
commands:
70.1-create-post-dir:
# "mkdir -p" ignores error if directory already exists
command: "mkdir -p /opt/elasticbeanstalk/hooks/appdeploy/post"
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/01_cron.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Load environment data
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
EB_APP_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
# Export EB_APP_LOG_DIR so we can access it when running "whenever" below,
# which accesses config/schedule.rb, which uses EB_APP_LOG_DIR.
export EB_APP_LOG_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_log_dir)
# Make sure the cron-whenever.log exists and is owned by $EB_APP_USER
touch $EB_APP_LOG_DIR/cron-whenever.log
chown $EB_APP_USER:$EB_APP_USER $EB_APP_LOG_DIR/cron-whenever.log
# cron requires a home directory. Make sure it exists and is owned by $EB_APP_USER.
mkdir -p /home/$EB_APP_USER
chown $EB_APP_USER:$EB_APP_USER /home/$EB_APP_USER
# Set up correct environment and ruby version so that bundle can load all gems
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
# Run the whenever --update command to update the cron job for $EB_APP_USER.
cd $EB_APP_DEPLOY_DIR
whenever --update -u $EB_APP_USER
# After setup, you can run "crontab -l -u webapp" to check the configuration.
First off - is that how your .config file is formatted? The file is in YAML format, so whitespace/indenting is important. Also, can you confirm what the file is called and where it is located? Should look like this:
.ebextensions/01-whenever.config
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_update_cron.sh"
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/containerfiles/envvars
su -c "cd $EB_CONFIG_APP_CURRENT; bundle exec whenever --update-cron" - $EB_CONFIG_APP_USER
If that doesn't fix it, check to make sure that the file /opt/elasticbeanstalk/hooks/appdeploy/post/99_update_cron.sh is being created correctly by connecting via eb ssh.
Then, check the log file at /var/log/eb-activity.log and look for the section where your 99_update_cron.sh hook is being executed. There may be some informative error messages.
If you have multiple EC2 instances running and want to run Whenever only in one instance, you probably want the following config:
files:
"/tmp/99_update_cron.template":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Using similar syntax as the appdeploy pre hooks that is managed by AWS
set -xe
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
cd $EB_DEPLOY_DIR
su -c "bundle exec whenever --update-cron"
su -c "crontab -l"
container_commands:
enable_cron:
command: "mv /tmp/99_update_cron.template /opt/elasticbeanstalk/hooks/appdeploy/post/99_update_cron.sh"
leader_only: true
2022 AWS EB CHANGED - LINUX 2
Note if you are running the newer LINUX 2 platform on elastic beanstalk, the custom platform hooks most people refer to here are deprecated and instead you need to do the following:
create a simple shell script in the new .platform/hooks/postdeploy directory (create it if it doesn't exist).
.platform/hooks/postdeploy/01_whenever.sh
#!/usr/bin/env bash
cd /var/app/current
bundle exec whenever --update-crontab
However there is a catch, if you need any environment variables that are not in rails credentials or .env files, the environment variables are not available to the system so when your cron task runs, if it needs to for example to run rails, rails will break since it doesn't have access to the env variables.
To solve this, you need to either, move those variables into rails credentials or .env or make the eb env vars available to the system (see below).
under .ebextensions create a file called 01_export_vars.config
commands:
setvars:
command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /etc/profile.d/sh.local
packages:
yum:
jq: []
Now, when cron runs rails it will have access to eb env vars and run correctly.
Do note, that if you add a new eb env var you need to redeploy so that this script gets run again and exports that eb env var.
This is the best solution I have found, but if someone knows a better way, please tell.

Elastic Beanstalk: what's the best way to create folders & set permissions after deploy?

I'm running a Rails 4.2 app on Elastic Beanstalk, and need to set log permissions and create the /tmp/uploads folder (plus permissions) after deploy.
I was running two ebextensions scripts to do this, but on some occasions they would fail because the folder /var/app/current/ didn't yet exist.
I'm presuming this is because the permissions and/or folders should be created on /app/ondeck/ first so that EB can copy the contents over to /var/app/current/, but I'm interested to see if there's a recommended and more foolproof approach to doing this?
For reference, my two ebextension scripts were:
commands:
01_set_log_permissions:
command: "chmod 755 /var/app/current/log/*"
and
commands:
01_create_uploads_folder:
command: "mkdir -p /var/app/current/tmp/uploads/"
02_set_folder_permission:
command: "chmod 755 /var/app/current/tmp/uploads/"
Thanks,
Dan
you should probably use files tag and not command:
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_make_changes.sh":
mode: "000777"
content: |
#!/bin/bash
mkdir -p /var/app/current/tmp/uploads/
chmod 755 /var/app/current/tmp/uploads/
it will be triggered after app deploy finished
I've used the below stpes:
Create a folder .ebextensions
Creta a file .config
Move .config to .ebextensions
Edit the file .config, it must have the below sintaxe
commands:
command1:
command: mkdir /opt/jenkins
command2:
command: chmod 644 /opt/jenkins
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-commands
* Pay Attention *
You cannot run again command1 "mkdir /opt/jenkins", you will have a error, so you must do a test before.
What about using Container Commands?
http://docs.aws.amazon.com/ko_kr/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-container-commands
You can use the container_commands key to execute commands for your container. The commands in container_commands are processed in alphabetical order by name. They run after the application and web server have been set up and the application version file has been extracted, but before the application version is deployed. They also have access to environment variables such as your AWS security credentials.
Container commands are run from the staging directory, where your source code is extracted prior to being deployed to the application server. Any changes you make to your source code in the staging directory with a container command will be included when the source is deployed to its final location.
container_commands:
01_set_log_permissions:
command: "chmod 755 log/*"
and
container_commands:
01_create_uploads_folder:
command: "mkdir -p tmp/uploads/"
02_set_folder_permission:
command: "chmod 755 tmp/uploads/"

Can you run a cronjob with Amazon SQS?

I have my app with Elastic Beanstalk and I need to create a cronjob that run a task in rails
rake "sitemap:generate"
and I wonder if I can do it with Amazon SQS, anyone knows how to do that?
I tried to do a crontab but it doesn't work in the beanstalk...
files:
"/tmp/cron_job.sh":
mode: "000777"
content: |
#!/usr/bin/env bash
*/2 * * * * cd /var/app/current/ && RACK_ENV=production bundle exec rake sitemap:generate
encoding: plain
container_commands:
01_delete_cron_jobs:
command: "crontab -r -u ec2-user || exit 0"
02_add_cron_jobs:
command: "crontab /tmp/cron_job.sh -u ec2-user"
leader_only: true
Is there another way to do a cronjob in Elastic Beanstalk?
Thank you.
You can run a rake task in a crontab in Elastic Beanstalk as follows.
You may need access to configuration that is set in your profile. If you put the following in a shell script like generate_map.sh, it will include the config that your app is using (this may be overkill for your application). Put this script file in the root of your application.
#!/bin/bash
source /etc/profile
cd /var/app/current
rake sitemap:generate
Put the cronjob you want to run in a text file (like file_with_cron_commands.txt) in your project in the .ebextensions directory. For example, if you want your task to run every minute, that file would include the line below. Make sure to leave a blank line at the end of your text file.
* * * * * root cd /var/app/current ; sh generate_map.sh
In your .config file, use a container command to copy it to the appropriate cron tab directory and set the permissions on it.
container_commands:
01_run_my_gen_map_cron:
command: "cat .ebextensions/file_with_cron_commands.txt > /etc/cron.d/my_genmap_cron && chmod 644 /etc/cron.d/my_genmap_cron"
Looks like an old question, but it pops up in google search.
So here is my attempt
You can create a periodic job (like cron jobs) using SQS with your rails app.
Create an "worker tier" env which uses SQS for queueing
in the home folder of your app create a file "cron.yaml"
sample content like
#cron.yaml
version: 1
cron:
— name: "schedule"
url: "/schedule"
schedule: "0 */12 * * *"
also checkout:
https://medium.com/#joelennon/running-cron-jobs-on-amazon-web-services-aws-elastic-beanstalk-a41d91d1c571
the gem "active-elastic-job"

Can you run a rails console or rake command in the elastic beanstalk environment?

I have set up a RoR environement on AWS' elastic beanstalk. I am able to ssh into my EC2 instance.
My home directory is /home/ec2-user, which is effectively empty.
If I move up a directory, there is also a /home/webapp directory that i do not have access to.
Is there a way to run a rake command or rails console on my elastic beanstalk instance?
If I type rails console I get Usage: rails new APP_PATH [options]
If I type RAILS_ENV=production bundle exec rails console, I get "Could not locate Gemfile"
For rails, jump to /var/app/current then as #juanpastas said, run RAILS_ENV=production bundle exec rails c
Don't know why, but since EBS runs everything as root, this worked for me:
sudo su
bundle exec rails c production
None of these solutions mentioned here worked for me, so I cooked up a little script that I put in script/aws-console.
You can run it from the /var/app/current directory as root:
eb ssh
cd /var/app/current
sudo script/aws-console
My script can be found as a Gist here.
None of the other answers worked for me so I went looking - this is working for me now on an elastic beanstalk 64bit amazon linux 2016.03 V2.1.2 ruby 2.2 (puma) stack
cd /var/app/current
sudo su
rake rails:update:bin
bundle exec rails console
that returns me the expected console
Loading production environment (Rails 4.2.6)
irb(main):001:0>
For Ruby 2.7:
if you don't need environment variables:
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
It looks like environment variables are not loaded automatically anymore, which might prevent rails console from starting.
I solved it by creating this .ebextensions file:
# Simply call `sudo /var/app/scripts/rails_c`
commands:
create_script_dir:
command: "mkdir -p /var/app/scripts"
ignoreErrors: true
files:
"/var/app/scripts/export_envvars":
mode: "000755"
owner: root
group: root
content: |
#!/opt/elasticbeanstalk/.rbenv/shims/ruby
if __FILE__ == $0
require 'json'
env_file = '/var/app/scripts/envvars'
env_vars = env_vars = JSON.parse(`/opt/elasticbeanstalk/bin/get-config environment`)
str = ''
env_vars.each do |key, value|
new_key = key.gsub(/\s/, '_')
str << "export #{new_key}=\"#{value}\"\n"
end
File.open(env_file, 'w') { |f| f.write(str) }
end
"/var/app/scripts/rails_c":
mode: "000755"
owner: root
group: root
content: |
. ~/.bashrc
/var/app/scripts/export_envvars
. /var/app/scripts/envvars
cd /var/app/current
/opt/elasticbeanstalk/.rbenv/shims/bundle exec rails c
Create a .ebextension file named setvars.config and add those lines to it
commands:
setvars:
command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /etc/profile.d/sh.local
packages:
yum:
jq: []
Then deploy your code again it should work.
reference: https://aws.amazon.com/ar/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-shell/
For Ruby 2.7:
As someone said, if you don't need env vars, run the following
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
However, if you need ENV, I recommend doing this as per AWS doc:
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-linux2/
tl;dr
On Amazon Linux 2, all environment properties are centralised into a single file called /opt/elasticbeanstalk/deployment/env. No user can access these outside the app. So, they recommend to add some hook scripts after deploy to basically create a copy.
#!/bin/bash
#Create a copy of the environment variable file.
cp /opt/elasticbeanstalk/deployment/env /opt/elasticbeanstalk/deployment/custom_env_var
#Set permissions to the custom_env_var file so this file can be accessed by any user on the instance. You can restrict permissions as per your requirements.
chmod 644 /opt/elasticbeanstalk/deployment/custom_env_var
#Remove duplicate files upon deployment.
rm -f /opt/elasticbeanstalk/deployment/*.bak
If because of some reason you don't want to run as root, do the following to pass env vars from root into new user environment:
sudo -u <user> -E env "PATH=$PATH" bash -c 'cd /var/app/current/ && <wtv you want to run>'
I like to create an eb_console file at the root of my rails app, then chmod u+x it. It contains the following:
ssh -t ec2-user#YOUR_EC2_STATION.compute.amazonaws.com 'cd /var/app/current && bin/rails c'
This way, I just have to run:
./eb_console
like I would have run heroku run bundle exec rails c.
#!/bin/sh
shell_join () {
ruby -r shellwords -e 'puts Shellwords.join(ARGV)' "$#"
}
command_str () {
printf 'set -e; . /etc/profile.d/eb_envvars.sh; . /etc/profile.d/use-app-ruby.sh; set -x; exec %s\n' "$(shell_join "$#")"
}
exec sudo su webapp -c "$(command_str "$#")"
Put above file somewhere in your source code, deploy, eb ssh into the eb instance, cd /var/app/current, and then execute path/to/above/script bin/rails whatever argumeents you usually use.
Reason why I have written above script is:
When using sudo, it drops some environment variables which might actually be needed for your rails app; so manually load the profiles which the Elastic Beanstalk platform provides.
Current Beanstalk ruby platform assumes you run rails application on user webapp, a non-login-able user, so it would be wise to run your command in this user.
For the latest ruby version, please use the following command:
BUNDLE_PATH=/opt/rubies/ruby-2.6.3/lib/ruby/gems/2.6.0/ bundle exec rails c production
Running it with sudo is not needed.
add an eb extension shortcut:
# .ebextensions/irb.config
files:
"/home/ec2-user/irb":
mode: "000777"
owner: root
group: root
content: |
sudo su - -c 'cd /var/app/current; bundle exec rails c'
then:
$ eb ssh
$ ./irb
irb(main):001:0>
None of these were working for me, including the aws-console script. I finally ended up creating a script directory in /var/app/current and then creating a rails file in that directory as outline by this answer on another SO question.
eb ssh myEnv
cd /var/app/current
sudo mkdir script
sudo vim script/rails
Add this to file and save:
echo #!/usr/bin/env ruby
# This command will automatically be run when you run "rails" with Rails 3 gems installed from the root of your application.
APP_PATH = File.expand_path('../../config/application', __FILE__)
require File.expand_path('../../config/boot', __FILE__)
require 'rails/commands'
Then make it executable and run it:
sudo chmod +x script/rails
sudo script/rails console
And it worked.
You have to find the folder with your Gemfile :p.
To do that, I would take a look in you web server config there should be a config that tells you where your app directory is.
Maybe you know where your app is.
But in case you don't know, I would give a try to:
grep -i your_app_name /etc/apache/*
grep -i your_app_name /etc/apache/sites-enabled/*
To search files containing your_app_name in Apache config.
Or if you are using nginx, replace apache above by nginx.
after you find application folder, cd into it and run RAILS_ENV=production bundle exec rails c.
Making sure that your application is configured to run in production in Apache or nginx configuration.

How to automatically restart delayed_job when deploying a rails project on Amazon Elastic Beanstalk?

I'm hosting a rails project on Amazon Elastic Beanstalk and I try to configure a container command to automatically restart my delayed_job worker on the server after each deployment.
I tried with this one :
container_commands:
restartdelayedjob:
command: "RAILS_ENV=production script/delayed_job --pid-dir=/home/ec2-user/pids start"
cwd: /var/app/current
But, it seems that the pushed version is deployed after the restarting of the worker so the jobs failed to be processed by the worker.
When I connect on my instance by ssh, kill the worker process and restart a new one from the deployed version folder, everything works fine.
Do you have any ideas of how I can handle this?
Thanks
As per the Amazon documentation for container_commands:
They run after the application and web server have been set up and the application version file has been extracted, but before the application version is deployed.
(emphasis mine)
This means at that point /var/app/current which you are setting as the cwd for your command is still pointing to the previous version. However by default, from the docs again, cwd:
is the directory of the unzipped application.
This means that if you want to run delayed_job from the directory of the app that just got extracted (but not yet deployed), don't override cwd and it should start the delayed_job for the app that's about to be deployed.
Ref: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#customize-containers-format-container_commands
Update:
I've now set this up myself and found there's limitations to doing it via the standard container_commands - basically delayed_job will be started while it is still in the /var/app/ondeck directory. Usually this is OK, but I had some issues with some jobs because that path had stuck around it would cause errors as the app was now in /var/app/current.
I found an undocumented (so warning!) approach that you can add scripts to be run AFTER your app server is restarted (and your new deploy is in /var/app/current).
Basically Elastic Beanstalk will execute any scripts in /opt/elasticbeanstalk/hooks/appdeploy/post after the web server is restarted. This means if you drop shell scripts in this directory they will be run.
I created a shell script like this:
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
cd $EB_CONFIG_APP_CURRENT
su -c "RAILS_ENV=production script/delayed_job --pid-dir=$EB_CONFIG_APP_SUPPORT/pids restart" $EB_CONFIG_APP_USER
I uploaded this script to an S3 bucket, and made sure it was "public". You can then use an options script in your .ebextensions directory (eg. 99delayed_job.config) to deploy this script as part of your app deploy, taking note that the post directory might not exist:
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_restart_delayed_job.sh":
mode: "000755"
owner: root
group: root
source: http://YOUR_BUCKET.s3.amazonaws.com/99_restart_delayed_job.sh
When you deploy you should see something like this in your /var/log/eb-tools.log:
2013-05-16 01:20:53,759 [INFO] (6467 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Executing directory: /opt/elasticbeanstalk/hooks/appdeploy/post/
2013-05-16 01:20:53,760 [INFO] (6467 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Executing script: /opt/elasticbeanstalk/hooks/appdeploy/post/99_restart_delayed_job.sh
2013-05-16 01:21:02,619 [INFO] (6467 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Output from script: delayed_job: trying to stop process with pid 6139...
delayed_job: process with pid 6139 successfully stopped.
2013-05-16 01:21:02,620 [INFO] (6467 MainThread) [directoryHooksExecutor.py-29] [root directoryHooksExecutor info] Script succeeded.
As I said, putting stuff in this "post" directory is undocumented - but hopefully at some point Amazon add actual support to the .options scripts to run commands post-deploy, in that case you could move this to the officially supported approach.
On 64bit Amazon Linux 2014.09 v1.1.0 running Ruby 2.1 (Passenger Standalone), got it working thanks to this post.
Note that this script is run as root, but your workers should be run as the webapp user.
# Adds a post-deploy hook such that after a new version is deployed
# successfully, restarts the delayed_job workers.
#
# http://stackoverflow.com/questions/14401204/how-to-automatically-restart-delayed-job-when-deploying-a-rails-project-on-amazo
# http://www.dannemanne.com/posts/post-deployment_script_on_elastic_beanstalk_restart_delayed_job
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_delayed_job.sh":
mode: "000755"
owner: root
group: root
encoding: plain
content: |
#!/usr/bin/env bash
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_APP_CURRENT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_APP_PIDS_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
cd $EB_APP_CURRENT_DIR
# Switch to the webapp user. Worker shouldn't be run as root.
su -s /bin/bash -c "bundle exec bin/delayed_job --pid-dir=$EB_APP_PIDS_DIR restart" $EB_APP_USER
In case anybody is looking to get delayed_job working in latest ElasticBeanstalk (64bit Amazon Linux 2014.09 v1.0.9 running Ruby 2.1 (Puma)): I got it to work using the below code (Thanks to damontorgerson). This file goes in ruby.config in .ebextensions folder.
# Install git in order to be able to bundle gems from git
packages:
yum:
git: []
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_delayed_job":
mode: "000777"
owner: root
group: root
content: |
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_APP_STAGING_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)
EB_CONFIG_APP_CURRENT=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
EB_CONFIG_APP_LOGS=$(/opt/elasticbeanstalk/bin/get-config container -k app_log_dir)
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)
EB_CONFIG_APP_PIDS=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
cd $EB_CONFIG_APP_CURRENT
. $EB_SUPPORT_DIR/envvars.d/sysenv
bin/delayed_job --pid-dir=/var/tmp restart
I got mine working like so with the "daemons" gem:
commands:
create_post_dir:
command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
ignoreErrors: true
webapp_pids:
command: "mkdir /home/webapp/pids"
ignoreErrors: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/99_restart_delayed_job.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
chown webapp:webapp /home/webapp/pids
su -l -c "$EB_CONFIG_APP_CURRENT/bin/delayed_job --pid-dir=/home/webapp/pids restart" $EB_CONFIG_APP_USER
echo "worker starting" >> /var/log/directory-hooks-executor.log

Resources