Crontab in Amazon Elastic Beanstalk - ruby-on-rails

I am doing a cron tab in AWS - Elastic Beanstalk with Ruby on Rails 3, but I don't know what is wrong.
I have this code in my .ebextensions/default.config
container_commands:
01remove_old_cron_jobs:
command: "crontab -r || exit 0"
02send_test_email:
command: crontab */2 * * * * rake send_email:test
leader_only: true
I receive this error:
Failed on instance with return code: 1 Output: Error occurred during build: Command 02send_test_email failed .
UPDATE 1
I tried next:
crontab.txt
*/2 * * * * rake send_email:test > /dev/null 2>&1
default.config
02_crontab:
command: "cat .ebextensions/crontab.txt | crontab"
leader_only: true
RESULT: No errors, but it does not work.
UPDATE 2
crontab.sh
crontab -l > /tmp/cronjob
#CRONJOB RULES
echo "*/2 * * * * /usr/bin/wget http://localhost/crontabs/send_test_email > /dev/null 2>&1" >> /tmp/cronjob
#echo "*/2 * * * * rake send_email:test > /dev/null 2>&1" >> /tmp/cronjob
crontab /tmp/cronjob
rm /tmp/cronjob
echo 'Script successful executed, crontab updated.'
default.config
02_crontab:
command: "/bin/bash .ebextensions/crontab.sh"
leader_only: true
RESULT: Works with url, but not with rake task.

Updated for 2018
In order to get this to work on the latest version of Elastic Beanstalk, I added the following to my .ebextensions:
.ebextensions/0005_cron.config
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
56 11 * * * root . /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
How I got there:
There are four main issues to confront when trying to cron a rake task in AWS EB:
The first hurdle is making sure all of your EB and Rails environment variables are loaded. I beat my head against the wall a while on this one, but then I discovered this AWS forum post (login may be required). Running . /opt/elasticbeanstalk/support/envvars loads all of your environment variables.
Then we need to make sure we cd into the current app directory using cd /var/app/current.
Next we need to know where to find the bundle and rake executables. They are not installed in the normal bin directories, but are located in a directory specific to your ruby version. To find out where your executables are located, ssh into your EB server (eb ssh) and then type the following:
$ cd /var/app/current
$ which bundle
/opt/rubies/ruby-2.3.4/bin/bundle
$ which rake
/opt/rubies/ruby-2.3.4/bin/rake
You could probably guess the directory based on your ruby version, but the above commands will let you know for sure. Based on the above, your can build your rake command as:
/opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test
NOTE: If you update your ruby version, you will likely need to update your cron config as well. This is a little brittle. I'd recommend making a note in your README on this. Trust me, six months from now, you will forget.
The fourth thing to consider is logging. I'd recommend logging to the same location as your other rails logs. We do this by tacking on >> /var/app/current/log/cron.log 2>&1 to the end of our command string.
Putting all of this together leads to a cron command string of:
. /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
Finally, I referenced the latest AWS documentation to build an .ebextensions config file for my cron command. The result was the .ebextensions/0005_cron.config file displayed at the top of this answer.

I am having the same issue. Though I figured out that the reason that rake task doesn't run correctly on eb is because of RACK_ENV, RAILS_ENV and BUNDLE_WITHOUT
Defaults of eb:
RACK_ENV: production
RAILS_ENV: production
BUNDLE_WITHOUT: test:development
When the cron runs rake task, it runs in development mode, and gives gem not found error, as gems grouped in development are not installed.
you can see this by changing your cron a bitfrom:
*/2 * * * * rake send_email:test > /dev/null 2>&1
to:
*/2 * * * * cd /var/app/current/ && /usr/bin/bundle exec /usr/bin/rake send_email:test > /tmp/cron_log 2>&1
and then checking the /tmp/cron_log file
To know the location of bundle and rake, run
which bundle
which rake
I tried setting RAILS_ENV in command in cron, but that didn't work aswell
One quick fix is to set
BUNDLE_WITHOUT to null
EDIT:
Finally I got it to work,
.ebextensions/.config
files:
"/tmp/cron_jobs" :
mode: "000777"
content: |
1 10 * * * cd /var/app/current/ && RACK_ENV=production rake some:task >> /var/app/current/log/cron_log 2>&1
encoding: plain
container_commands:
01_delete_cron_jobs:
command: "crontab -r -u webapp || exit 0"
02_add_cron_jobs:
command: "crontab /tmp/cron_jobs -u webapp"
leader_only: true
option_settings:
- option_name: RAILS_ENV
value: production
- option_name: RACK_ENV
value: production
Notice the '-u webapp' when removing and adding cron, this will run this cron under user webapp. The above will also run in production mode. And the output will be dumped in log/cron_log file.
If the above wont work then adding 'Bundle exec' before 'rake some:task' might work.

I've seen these used with separate files in .ebextensions, such as:
02send_test_email:
command: "cat .ebextensions/crontab | crontab"
leader_only: true
I haven't gotten around to it yet, but I took note of this along the way. Let us know if this works.
This stackoverflow post has much more information
After Update 1/2:
Cron doesn't know where rake is. Your application runs from /var/app/current, and you need to be running bundle exec rake from that directory.
Elastic beanstalk is horrible with logging errors, to get this right, ssh to the machine and experiment until you have the commands right, then put this back into your cron script. You can even try and re-run some of the eb scripts as found in the logs, then reverse that into your ebextensions files.

Related

ElasticBeanstalk Ruby PostDeploy Script Mission Impossible

We have recently updated our ruby/elasticbeanstalk platform to AWS Linux 2 / Ruby (Ruby 2.7 running on 64bit Amazon Linux 2/3.2.0)
A part of our Ruby deployment is a delayed_job (daemon gem)
After many attempts to have a bash script from the .platform/hooks/postdeploy/ folder, I have offically declared I am stuck. Here is the error from eb-engine.log:
2020/12/08 04:18:44.162454 [INFO] Running platform hook: .platform/hooks/postdeploy/restart_delayed_job.sh
2020/12/08 04:18:44.191301 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/restart_delayed_job.sh failed with error exit status 127
2020/12/08 04:18:44.191327 [INFO] Executing cleanup logic
2020/12/08 04:18:44.191448 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1607401124,"severity":"ERROR"}]}]}```
Here is one of many scripts I have attempted:
#!/bin/bash
#Using similar syntax as the appdeploy pre hooks that is managed by AWS
exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>delayed_job_err.out 2>&1
# Loading environment data
# source /etc/profile.d/sh.local #created from other .ebextension file
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config platformconfig -k AppUser)
EB_APP_CURRENT_DIR=$(/opt/elasticbeanstalk/bin/get-config platformconfig -k AppDeployDir)
#EB_APP_PIDS_DIR=/home/webapp/pids
/opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /tmp/envvars
source /tmp/envvars
cd /var/app
cd $EB_APP_CURRENT_DIR
su -s /bin/bash -c "bin/delayed_job restart" $EB_APP_USER```
Here is the delayed_job file:
#!/usr/bin/env ruby
require File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))
require 'delayed/command'
Delayed::Command.new(ARGV).daemonize
As you can see I'm doing my best to load up the env variables. The delayed_job seems to run just fine as root from within the EB Linux 2 host with the env vars loaded.
total 12
-rwxrwxr-x 1 webapp webapp 179 Dec 8 04:15 001_load_envs.sh
-rw-r--r-- 1 root root 251 Dec 8 04:46 delayed_job_err.out
-rwxrwxr-x 1 webapp webapp 1144 Dec 8 04:15 restart_delayed_job.sh
[root#ip-172-16-100-178 postdeploy]# cat delayed_job_err.out
/var/app/current/vendor/bundle/ruby/2.7.0/gems/json-1.8.6/lib/json/common.rb:155: warning: Using the last argument as keyword parameters is deprecated
delayed_job: warning: no instances running. Starting...
delayed_job: process with pid 5292 started.
Any help would be appreciated..
I am also using elasticbeanstalk on Amazon Linux 2
I am using resque which needs a restart postdeploy. Following is my postdeploy hook which restarts resque workers
.platform/hooks/postdeploy/0020_restart_resque_workers.sh
#!/usr/bin/env bash
. /opt/elasticbeanstalk/deployment/env
cd /var/app/current/
su -c "RAILS_ENV=production bundle exec rake resque:restart_workers" webapp ||
echo "resque workers restarted."
true
Notice the environment variable setup. It simply executes /opt/elasticbeanstalk/deployment/env which will give you the environment.
Hope you are able to use above script by simply replcaing command to restart delayed job instead of resque workers.

Can't get Ruby Whenever working

I'm having some trouble getting whenever to work in my local dev environment. I'm using mac OS 10.12, ruby 2.4, rails 5.1 and whenever 0.10.0.
I have my schedule.rb file setup as:
set :environment, "development"
set :job_template, "TZ=\"America/Los_Angeles\" bash -l -c ':job'"
set :output, Whenever.path + "/log/cron.log"
every 5.minutes do
runner "Test.new.run"
command "/bin/echo 'Is this working?'"
end
crontab -l outputs the following:
# Begin Whenever generated tasks for: /Users/me/apps/Test/config/schedule.rb at: 2018-05-25 12:59:14 -0700
0,5,10,15,20,25,30,35,40,45,50,55 * * * * TZ="America/Los_Angeles" bash -l -c 'cd /Users/me/apps/Test && bundle exec bin/rails runner -e development '\''Test.new.run'\'' >> /Users/me/apps/Test/log/cron.log 2>&1'
0,5,10,15,20,25,30,35,40,45,50,55 * * * * TZ="America/Los_Angeles" bash -l -c '/bin/echo '\''Is this working?'\'' >> /Users/me/apps/Test/log/cron.log 2>&1'
# End Whenever generated tasks for: /Users/me/apps/Test/config/schedule.rb at: 2018-05-25 12:59:14 -0700
and I'm receiving the following when I run the mail command in terminal:
Message 1:
From me#box.local Fri May 25 13:00:01 2018
X-Original-To: me
Delivered-To: me#box.local
From: me#box.local (Cron Daemon)
To: me#box.local
Subject: Cron <me#box> TZ="America/Los_Angeles" bash -l -c '/bin/echo '\''Is this working?'\'' >> /Users/me/apps/Test/log/cron.log 2>&1'
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=me>
X-Cron-Env: <USER=me>
X-Cron-Env: <HOME=/Users/me>
Date: Fri, 25 May 2018 13:00:00 -0700 (PDT)
However, I'm not seeing anything output to cron.log nor does the DB reflect any activity from Test.new.run
Not really sure what I'm doing wrong...
I think the problem may have had to do with rvm and the .rvmrc file I had set up which loaded the correct rvm settings when I cd into the directory.
In attempting to ensure that the command was working opened a new terminal window and ran: bash -l -c 'cd /Users/me/apps/Test && bundle exec bin/rails runner -e development '\''Test.new.run'\'' >> /Users/me/apps/Test/log/cron.log 2>&1'
The first thing that came up was the standard 1 off rvm message:
You are using '.rvmrc', it requires trusting, it is slower and it is not compatible with other ruby managers,
you can switch to '.ruby-version' using 'rvm rvmrc to ruby-version'
or ignore this warning with 'rvm rvmrc warning ignore /Users/me/apps/Test/.rvmrc',
'.rvmrc' will continue to be the default project file in RVM 1 and RVM 2,
to ignore the warning for all files run 'rvm rvmrc warning ignore all.rvmrcs'.
********************************************************************************
* NOTICE *
********************************************************************************
* RVM has encountered a new or modified .rvmrc file in the current directory, *
* this is a shell script and therefore may contain any shell commands. *
* *
* Examine the contents of this file carefully to be sure the contents are *
* safe before trusting it! *
* Do you wish to trust *
* '/Users/me/apps/Test/.rvmrc'? *
* Choose v[iew] below to view the contents *
********************************************************************************
y[es], n[o], v[iew], c[ancel]>
Once I typed y and hit enter, the cron jobs are now executing properly.
Unfortunately, I'm still not seeing anything in my cron.log file.

Cronjob is running every hour but not at a specific time

I have a cronjob on my server that suppose to run a rake script on my rails app everyday at specific time (eg. 8.00 am). I use whenever gem to automate this. This is how it appears currently on the cron file:
0 8 * * * /bin/bash -l -c 'cd /home/deploy/apps/myapp/releases/20160121092339 && RAILS_ENV=production bundle exec rake overdue_payments --silent >> /home/deploy/apps/myapp/releases/20160121092339/log/cron.log 2>&1'
And it's not working. But strangely enough, if I edit my cron to run for every hour, it is working just fine! Here is how I did it to make it run for every hour:
0 * * * * /bin/bash -l -c 'cd /home/deploy/apps/myapp/releases/20160121092339 && RAILS_ENV=production bundle exec rake overdue_payments --silent >> /home/deploy/apps/myapp/releases/20160121092339/log/cron.log 2>&1'
I have modified my timezone to the local timezone (UK) but still not working:
We need to specify the timezone to cronjob description. Here is my example:
test_cron_job:
cron: "45 11 * * * Asia/Bangkok"
class: "Schedulers::TestCronJob"
queue: default
Above cronjob will be run at 11:45 am in Bangkok time. Btw you may find out your timezone by using:
ActiveSupport::TimeZone.zones_map.values.map(&:tzinfo).map(&:identifier)
I finally solved it! For the timezone, i just need to add TZ before it. So for my case it will be: TZ="Europe/London".
0 8 * * * TZ="Europe/London" /bin/bash -l -c 'cd /home/deploy/apps/myapp/releases/20160121092339 && RAILS_ENV=production bundle exec rake overdue_payments --silent >> /home/deploy/apps/myapp/releases/20160121092339/log/cron.log 2>&1'
Hope this solution help you
every :day, :at => (6..8).to_a.map { |x| ["#{x}:00","#{x}:30", "#{x+1}:00"] }.flatten do
# Run rake task.
end
# => ["6:00", "6:30", "7:00", "7:00", "7:30", "8:00", "8:00", "8:30", "9:00"]
Adjust initial range and the inner block to cover any range. Its support 24hr clock.

Whenever rails gem not working with cronjob.. why?

I am using Whenever gem with rails. For some reason the first cronjob does not work. If i copy paste the command into the shell manually, it works as it should.
The second "touch testing123.txt" also works fine.
I have no idea why the GiftPackage.do_scheduled_deliveries rails method does not run. I'm really at a loss right now.. any help would be greatly appreciated!
# Begin Whenever generated tasks for: /var/www/mysite/releases/20130131200554/config/schedule.rb
*/1 * * * * /bin/bash -l -c 'cd /var/www/mysite/releases/20130131200554 && script/rails runner -e staging '\''GiftPackage.do_scheduled_deliveries'\'''
*/1 * * * * /bin/bash -l -c 'cd /var/www/mysite/releases/20130131200554 && touch testing123.txt'
# End Whenever generated tasks for: /var/www/mysite/releases/20130131200554/config/schedule.rb
Thanks!
:)
Most likely you are getting an error within the app itself.
First step would be, run the task in the console using the staging environment.
Second would be to check the cron logs and see if it is outputting any errors.
You can set whenever to log output by adding the following to your schedule.rb file;
set :output, "/[path to log]/cron.log"
This should help you get closer to finding the solution.
Try setting your PATH variable before the jobs, your cron is probably not able to find ruby :
For instance :
PATH=/usr/local/rvm/gems/ruby-1.9.3-p194/bin:/usr/local/rvm/gems/ruby-1.9.3-p194#global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p194/bin:/usr/local/rvm/bin:/usr/local/sbin:/usr/local/$
# Begin Whenever generated tasks for: /var/www/mysite/releases/20130131200554/config/schedule.rb
*/1 * * * * /bin/bash -l -c 'cd /var/www/mysite/releases/20130131200554 && script/rails runner -e staging '\''GiftPackage.do_scheduled_deliveries'\'''
*/1 * * * * /bin/bash -l -c 'cd /var/www/mysite/releases/20130131200554 && touch testing123.txt'
# End Whenever generated tasks for: /var/www/mysite/releases/20130131200554/config/schedule.rb

whenever gem: I set :output but the logfile doesn't show up where I'd expect it to

In my schedule.rb file I have the following lines:
set :output, '/log/cron_log.log'
every 5.minutes do
command 'echo "hello"'
end
I ran whenever -w as suggested in this question Rails, using whenever gem in development, and I assume the cronfile is written and running. (I tried restarting the Rails server as well.)
And when I run $ crontab -l I see the following:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash
-l -c 'echo "hello" >> /log/cron_log.log 2>&1'
But I can't find the log file. I checked in rails_project/log, ~/log and /log to no avail. On OSX btw.
How can I get the log file to be written to the rails project log folder?
Where is your log?
You're putting the output file at the highest directory level:
$ cd /log
To see if the file exists and if it has data in it:
$ ls -la cron_log.log
To create the log file if needed:
$ touch cron_log.log
To open up permissions for your own local debugging (do NOT do this in production!)
$ chmod +rw cron_log.log
Is your command running?
To run the command manually to find out if it works as you expect:
$ /bin/bash -l -c 'echo "hello" >> /log/cron_log.log 2>&1'
To improve your security and protect your path, use full paths:
wrong: command 'echo "hello"'
right: command '/bin/echo "hello"'
To find the command full path:
$ which echo
To verify the cron is running as you expect:
$ sudo grep CRON /var/log/syslog
The grep result should have lines that something like this:
Jan 1 12:00:00 example.com CRON[123]: (root) CMD (... your command here ...)
Are you on a Mac?
If you're not seeing output in the syslog, and you're on a Mac, you may want to read about the Mac OSX switching from cron to launchd.
See the cron plist (/System/Library/LaunchDaemons/com.vix.cron.plist) and use a stdout/stderr path to debug cron itself. I don't recall if launchctl unloading and launchctl loading the plist is sufficient, or since it's a system daemon if you'd have to restart entirely. (see where is the cron log file in lion)
How to log relative to Rails?
To put the log relative to a Rails app, omit the leading slash (and typically call it cron.log)
set :output, "log/cron.log"
To put the log in a specific fully-qualified directory:
set :output, '/abc/def/ghi/log/cron.log'
The Whenever wiki has some good examples about redirecting output:
https://github.com/javan/whenever/wiki/Output-redirection-aka-logging-your-cron-jobs
Examples:
every 3.hours do
runner "MyModel.some_process", :output => 'cron.log'
rake "my:rake:task", :output => {:error => 'error.log', :standard => 'cron.log'}
command "/usr/bin/cmd"
end

Resources