I'm trying to add the following line to the crontab for a rails app:
* * * * * lsof -i tcp | grep -v grep | grep -q puma || /home/me/app_name/bin/rails s
... in short, I have an error that occasionally takes down my rails server, and I would like to run this command every minute to make sure the server remains as available as possible. (The actual error is a question for another day.)
Initially, I got an error:
shared_helpers.rb: 34
in 'default_gemfile': could not locate Gemfile (Bundler::GemfileNotFound)
... which traced back to line 4 in bin/rails:
load File.expand_path('../spring', __FILE__)
I changed the ('../spring') to ('./spring'), because the spring script was located in the same directory as the rails script, not in the parent. (Oddly, running rails s from the command line works whether I use '..' or '.' in the path.)
This, at least got rid of the error above, but now I have the following:
/bin/sh: 1: lsof: not found
/bin/sh: 1: grep: not found
/bin/sh: 1: grep: not found
/usr/bin/env: 'ruby_executable_hooks': No such file or directory
I get these errors once per minute, so at least I know the time format my entry is correct.
The relevant part of my crontab looks like this:
PATH=$PATH:/usr/share/rvm/rubies/ruby-2.4.1/bin/
GEM_PATH=$GEM_PATH:/usr/share/rvm/gems/ruby-
2.4.1:/usr/share/rvm/gems/ruby-2.4.1#global
* * * * * lsof -i tcp | grep -v grep | grep -q puma || /home/me/app/bin/rails s
I also tried:
* * * * * lsof -i tcp | grep -v grep | grep -q puma || /usr/share/rvm/gems/ruby-2.4.1/bin/rails s
... but get the same error.
I can run rails s from the command line, as I mentioned. I know running the command from a logged-in shell has the benefit of login scripts being read and environment variables being set, but I don't know what I need to do to get the environment in this crontab to co-operate.
NOTE: I'm not using the whenever gem for this, because I'm using external commands like grep and lsof; if there is an equivalent rails-based solution (to test if the server is running, and start it if it isn't), I could do that instead of using the crontab. (But I don't know what that would be.)
Related
I need help with this one. An Artisan command is being run by crontab to analyze a logfile and generate a report into an HTML format.
Here is the line of code in the command that is being executed by crontab:
$today = date("d/M/Y");
exec('grep "$today" /path_to_logfile | goaccess -a > /path_of_generated_HTML_file');
exit();
Unfortunately, the file being generated by cron is empty but when you execute the same code directly in the terminal, it successfully generates an html file. I appreciate your help guys.
This is most likely because your PATH is configured incorrectly when running under crontab. For most implementations of crontab, you can simply specify the PATH on the first line on the crontab file opened with crontab -e:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
You could also set the path inside PHP if you so desire:
<?php
putenv('PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin');
$today = date("d/M/Y");
exec('grep "$today" /path_to_logfile | goaccess -a > /path_of_generated_HTML_file');
exit;
I am doing a cron tab in AWS - Elastic Beanstalk with Ruby on Rails 3, but I don't know what is wrong.
I have this code in my .ebextensions/default.config
container_commands:
01remove_old_cron_jobs:
command: "crontab -r || exit 0"
02send_test_email:
command: crontab */2 * * * * rake send_email:test
leader_only: true
I receive this error:
Failed on instance with return code: 1 Output: Error occurred during build: Command 02send_test_email failed .
UPDATE 1
I tried next:
crontab.txt
*/2 * * * * rake send_email:test > /dev/null 2>&1
default.config
02_crontab:
command: "cat .ebextensions/crontab.txt | crontab"
leader_only: true
RESULT: No errors, but it does not work.
UPDATE 2
crontab.sh
crontab -l > /tmp/cronjob
#CRONJOB RULES
echo "*/2 * * * * /usr/bin/wget http://localhost/crontabs/send_test_email > /dev/null 2>&1" >> /tmp/cronjob
#echo "*/2 * * * * rake send_email:test > /dev/null 2>&1" >> /tmp/cronjob
crontab /tmp/cronjob
rm /tmp/cronjob
echo 'Script successful executed, crontab updated.'
default.config
02_crontab:
command: "/bin/bash .ebextensions/crontab.sh"
leader_only: true
RESULT: Works with url, but not with rake task.
Updated for 2018
In order to get this to work on the latest version of Elastic Beanstalk, I added the following to my .ebextensions:
.ebextensions/0005_cron.config
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
56 11 * * * root . /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
How I got there:
There are four main issues to confront when trying to cron a rake task in AWS EB:
The first hurdle is making sure all of your EB and Rails environment variables are loaded. I beat my head against the wall a while on this one, but then I discovered this AWS forum post (login may be required). Running . /opt/elasticbeanstalk/support/envvars loads all of your environment variables.
Then we need to make sure we cd into the current app directory using cd /var/app/current.
Next we need to know where to find the bundle and rake executables. They are not installed in the normal bin directories, but are located in a directory specific to your ruby version. To find out where your executables are located, ssh into your EB server (eb ssh) and then type the following:
$ cd /var/app/current
$ which bundle
/opt/rubies/ruby-2.3.4/bin/bundle
$ which rake
/opt/rubies/ruby-2.3.4/bin/rake
You could probably guess the directory based on your ruby version, but the above commands will let you know for sure. Based on the above, your can build your rake command as:
/opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test
NOTE: If you update your ruby version, you will likely need to update your cron config as well. This is a little brittle. I'd recommend making a note in your README on this. Trust me, six months from now, you will forget.
The fourth thing to consider is logging. I'd recommend logging to the same location as your other rails logs. We do this by tacking on >> /var/app/current/log/cron.log 2>&1 to the end of our command string.
Putting all of this together leads to a cron command string of:
. /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
Finally, I referenced the latest AWS documentation to build an .ebextensions config file for my cron command. The result was the .ebextensions/0005_cron.config file displayed at the top of this answer.
I am having the same issue. Though I figured out that the reason that rake task doesn't run correctly on eb is because of RACK_ENV, RAILS_ENV and BUNDLE_WITHOUT
Defaults of eb:
RACK_ENV: production
RAILS_ENV: production
BUNDLE_WITHOUT: test:development
When the cron runs rake task, it runs in development mode, and gives gem not found error, as gems grouped in development are not installed.
you can see this by changing your cron a bitfrom:
*/2 * * * * rake send_email:test > /dev/null 2>&1
to:
*/2 * * * * cd /var/app/current/ && /usr/bin/bundle exec /usr/bin/rake send_email:test > /tmp/cron_log 2>&1
and then checking the /tmp/cron_log file
To know the location of bundle and rake, run
which bundle
which rake
I tried setting RAILS_ENV in command in cron, but that didn't work aswell
One quick fix is to set
BUNDLE_WITHOUT to null
EDIT:
Finally I got it to work,
.ebextensions/.config
files:
"/tmp/cron_jobs" :
mode: "000777"
content: |
1 10 * * * cd /var/app/current/ && RACK_ENV=production rake some:task >> /var/app/current/log/cron_log 2>&1
encoding: plain
container_commands:
01_delete_cron_jobs:
command: "crontab -r -u webapp || exit 0"
02_add_cron_jobs:
command: "crontab /tmp/cron_jobs -u webapp"
leader_only: true
option_settings:
- option_name: RAILS_ENV
value: production
- option_name: RACK_ENV
value: production
Notice the '-u webapp' when removing and adding cron, this will run this cron under user webapp. The above will also run in production mode. And the output will be dumped in log/cron_log file.
If the above wont work then adding 'Bundle exec' before 'rake some:task' might work.
I've seen these used with separate files in .ebextensions, such as:
02send_test_email:
command: "cat .ebextensions/crontab | crontab"
leader_only: true
I haven't gotten around to it yet, but I took note of this along the way. Let us know if this works.
This stackoverflow post has much more information
After Update 1/2:
Cron doesn't know where rake is. Your application runs from /var/app/current, and you need to be running bundle exec rake from that directory.
Elastic beanstalk is horrible with logging errors, to get this right, ssh to the machine and experiment until you have the commands right, then put this back into your cron script. You can even try and re-run some of the eb scripts as found in the logs, then reverse that into your ebextensions files.
In my schedule.rb file I have the following lines:
set :output, '/log/cron_log.log'
every 5.minutes do
command 'echo "hello"'
end
I ran whenever -w as suggested in this question Rails, using whenever gem in development, and I assume the cronfile is written and running. (I tried restarting the Rails server as well.)
And when I run $ crontab -l I see the following:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash
-l -c 'echo "hello" >> /log/cron_log.log 2>&1'
But I can't find the log file. I checked in rails_project/log, ~/log and /log to no avail. On OSX btw.
How can I get the log file to be written to the rails project log folder?
Where is your log?
You're putting the output file at the highest directory level:
$ cd /log
To see if the file exists and if it has data in it:
$ ls -la cron_log.log
To create the log file if needed:
$ touch cron_log.log
To open up permissions for your own local debugging (do NOT do this in production!)
$ chmod +rw cron_log.log
Is your command running?
To run the command manually to find out if it works as you expect:
$ /bin/bash -l -c 'echo "hello" >> /log/cron_log.log 2>&1'
To improve your security and protect your path, use full paths:
wrong: command 'echo "hello"'
right: command '/bin/echo "hello"'
To find the command full path:
$ which echo
To verify the cron is running as you expect:
$ sudo grep CRON /var/log/syslog
The grep result should have lines that something like this:
Jan 1 12:00:00 example.com CRON[123]: (root) CMD (... your command here ...)
Are you on a Mac?
If you're not seeing output in the syslog, and you're on a Mac, you may want to read about the Mac OSX switching from cron to launchd.
See the cron plist (/System/Library/LaunchDaemons/com.vix.cron.plist) and use a stdout/stderr path to debug cron itself. I don't recall if launchctl unloading and launchctl loading the plist is sufficient, or since it's a system daemon if you'd have to restart entirely. (see where is the cron log file in lion)
How to log relative to Rails?
To put the log relative to a Rails app, omit the leading slash (and typically call it cron.log)
set :output, "log/cron.log"
To put the log in a specific fully-qualified directory:
set :output, '/abc/def/ghi/log/cron.log'
The Whenever wiki has some good examples about redirecting output:
https://github.com/javan/whenever/wiki/Output-redirection-aka-logging-your-cron-jobs
Examples:
every 3.hours do
runner "MyModel.some_process", :output => 'cron.log'
rake "my:rake:task", :output => {:error => 'error.log', :standard => 'cron.log'}
command "/usr/bin/cmd"
end
I am trying to parse my mvn verify output to only show lines with INFO tags. Please note that maven outputs line to stdout in real time and not by batch. I do not think that it is a problem with maven.
At first I tried to do it with grep:
$ mvn verify | grep INFO
but didn't seem to output lines in real time, as I understand grep buffers its lines before outputting, so I have to wait a few seconds between each flush and then I have tens of lines being printed at the same time, not very convenient. Then I thought I would try with sed.
According to this link, the following command:
sed -n '/PATTERN/p' file
// is equivalent to
grep PATTERN file
and according to this link, the -l option should force sed to flush its output buffer after every newline. So now I am using this command:
$ mvn verify | sed -ln -e '/INFO/p'
but I'm still getting the same result as before, I get a ton of output flushed every 30s or so and I don't know what I've done wrong. Can someone point me in the right direction please?
Try this, if your grep supports it:
mvn verify | grep --line-buffered INFO
If you're doing this in a terminal and still seeing buffered results, it would probably be something earlier than grep doing the buffering, but I'm not familiar with mvn. (And, yes, the -l option to sed should have done the same thing, so the problem may be upstream.)
try this line:
mvn verify | while read line; do echo $line|grep INFO; done
I found what was the problem, I was using a script to colorise maven output (see here) and in fact it was that script that was buffering the output down the pipe. I forgot about it as I was using it as an alias, I guess this is a good lesson, I won't alias as easily in the future. Anyway here is the fix, I changed -e to -le in the last line of the sed call:
mvn $# | sed -e "s/\(\[INFO\]\ \-.*\)/${TEXT_BLUE}${BOLD}\1/g" \
-e "s/\(\[INFO\]\ \[.*\)/${RESET_FORMATTING}${BOLD}\1${RESET_FORMATTING}/g" \
-e "s/\(\[INFO\]\ BUILD SUCCESSFUL\)/${BOLD}${TEXT_GREEN}\1${RESET_FORMATTING}/g" \
-e "s/\(\[WARNING\].*\)/${BOLD}${TEXT_YELLOW}\1${RESET_FORMATTING}/g" \
-e "s/\(\[ERROR\].*\)/${BOLD}${TEXT_RED}\1${RESET_FORMATTING}/g" \
-le "s/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/${BOLD}${TEXT_GREEN}Tests run: \1${RESET_FORMATTING}, Failures: ${BOLD}${TEXT_RED}\2${RESET_FORMATTING}, Errors: ${BOLD}${TEXT_RED}\3${RESET_FORMATTING}, Skipped: ${BOLD}${TEXT_YELLOW}\4${RESET_FORMATTING}/g"
In effect this is telling sed to flush its output at every new line, which is what I wanted. I am sorry I didn't find another workaround that is more generic. I tried playing around with empty (see man page) and script but none of these solutions worked for me.
Getting permission denied error while executing shell command from ruby console.
And the same shell command is working from shell.
From Shell..
tests#tests-workstation:~$ "`grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='`/db_backups"
bash: /db_backups: is a directory
tests#tests-workstation:~$
From ruby console..
>> %x["`grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='`/db_backups"]
sh: /db_backups: Permission denied
=> ""
Any Idea !
You're trying to execute a directory and the shells are saying no; bash says no by saying "/db_backups: is a directory" whereas sh says "/db_backups: Permission denied". If you just execute the backedticked part:
grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='
You'll almost certainly see no output at all and the reason is probably that your regular expression is too tight, something like this:
grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='
Would serve you better; the character classes contain a space and a tab.
Now that you're looking for the right things we can move on to why it still won't work. The %x[] quoter tries to execute its argument using the shell. When you feed the backticked grep stuff:
`grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='`/db_backups
to the shell, you should get a directory name that ends with /db_backups but you can't execute a directory. I think you want this to produce the directory name:
d = %x[echo `grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='`/db_backups].strip
Note the leading echo and the .strip call on the returned string. The .strip is necessary to remove the newline from what echo produces.
I think you're going through a lot of trouble for something that could easily be done with just a couple lines of Ruby:
dir = nil
File.open('/etc/mysql/my.cnf').each do |line|
if(m = line.match(/^\s*datadir\s*=\s*(\S+)/))
dir = m[1] + '/db_backups'
break
end
end
You could probably tighten that up a bit if you wanted but I think that that's at least less confusing than putting shell backticks inside Ruby backticks.
It looks like you just want to get field 2 from the file. Then just do it in Ruby using split
File.open("file").each do |line|
if line[/^datadir/]
print line.split("=",2)[0]
end
end
There is no need to specifically shell out to call grep. This is inefficient and non-portable