Rails cron with whenever, setting the environment - ruby-on-rails

This question will probably only make sense if you know about the whenever gem for creating cron jobs. I have a task in my schedule.rb like
every 1.day, :at => '4am' do
command "cd #{RAILS_ROOT} && rake thinking_sphinx:stop RAILS_ENV=#{RAILS_ENV}"
command "cd #{RAILS_ROOT} && rake thinking_sphinx:index RAILS_ENV=#{RAILS_ENV}"
command "cd #{RAILS_ROOT} && rake thinking_sphinx:start RAILS_ENV=#{RAILS_ENV}"
end
However when I update my crontab using
whenever --update-crontab appname --set environment=production
the cron jobs still have RAILS_ENV=development. My tasks on production and development are the same right now, I just need to change the environment variable because thinking_sphinx needs to know the current environment. Any ideas on how to do this?
Thanks!

Whenever doesn't detect your environment, it just defaults to using production.
You can set the environment for all jobs using set:
set :environment, 'staging'
Or per job:
every 2.hours do
runner 'My.runner', :environment => 'staging'
end

Don't write the RAILS_ENV variable. It should set it automatically.
every 1.day, :at => '4am' do
command "cd #{RAILS_ROOT} && rake thinking_sphinx:stop"
command "cd #{RAILS_ROOT} && rake thinking_sphinx:index"
command "cd #{RAILS_ROOT} && rake thinking_sphinx:start"
end
It works in my app:
every 4.days do
runner "AnotherModel.prune_old_records"
end
$ whenever --set environment=production
0 0 1,5,9,13,17,21,25,29 * * /Users/weppos/Sites/git/app/script/runner -e production "AnotherModel.prune_old_records"
$ whenever --set environment=development
0 0 1,5,9,13,17,21,25,29 * * /Users/weppos/Sites/git/app/script/runner -e development "AnotherModel.prune_old_records"

For Whenever (0.9.2)
Use the #environment variable for environment check:
case #environment
when 'production'
every 1.minutes do
rake "user:take_sample"
end
when 'development'
every 1.minutes do
rake "user:dev_sample"
end
end

Something else you may want to try if you're using bundler and capistrano.
In your deploy.rb file, when you set the :whenever_command, DO NOT simply do this:
set :whenever_command, "bundle exec whenever"
Instead, do this:
set(:whenever_command) { "RAILS_ENV=#{rails_env} bundle exec whenever" }
Now, the RAILS_ENV environment variable will be available when the schedule.rb file is loaded, so in schedule.rb you can now do this:
set :environment, ENV['RAILS_ENV']
Voila! You're ready to go.

This questions has been open a long time so I thought I would share what worked with whenever 0.9.7, Ruby 2.4.0, and RAILS 5.0.1. In the previously mentioned answer there are a lot of close tries but syntax error plagues them. Below is what worked and is very simple approach.
schedule.rb
require File.expand_path(File.dirname(__FILE__) + '/environment')
set :output, {:standard => 'log/cron_log.log', :error => 'log/cron_error_log.log'}
env :PATH, ENV['PATH']
every :day, :at => '10am' do
rake "somejob:run", :environment => #environment
end
Update the crontab(dev)
whenever --set 'environment=development' --update-crontab
Results(dev)
0 10 * * * /bin/bash -l -c 'cd /my/rails/app && RAILS_ENV=development bundle exec rake somejob:run --silent >> log/cron_log.log 2>> log/cron_error_log.log'
Update the crontab(prod)
whenever --set 'environment=production' --update-crontab
Results(prod)
0 10 * * * /bin/bash -l -c 'cd /my/rails/app && RAILS_ENV=production bundle exec rake somejob:run --silent >> log/cron_log.log 2>> log/cron_error_log.log'
Hopefully this can help someone out.
Happy Coding this!

Watch out if you want to pass more than one param to whenever.
You have to do it like that:
whenever --update-crontab appname --set 'environment=production&cron_log=/path/to/log'

Latest whenever allows easy Capistrano integration. You can add following to deploy.rb:
set :whenever_environment, defer { stage }
set :whenever_identifier, defer { "#{application}-#{stage}" }
require "whenever/capistrano"

I was having an issue where environment wasn't being set up for whenever cron jobs - /usr/bin/bundle was being picked up instead of /usr/local/bin/bundle.
The solution was to add following to top of schedule.rb
env 'PATH', ENV['PATH']

Add the following line of code at top of config/schedule.rb.
ENV['RAILS_ENV'] = "#{#pre_set_variables[:environment]}"
and update the crontab using following command.
whenever --update-crontab pvcnxt --set 'environment=production'
and then finally restart crontab using command
service crond restart
Thats it!
Final config/schedule.rb looks this way
ENV['RAILS_ENV'] = "#{#pre_set_variables[:environment]}"
env :PATH, ENV['PATH']
require File.expand_path(File.dirname(__FILE__) + "/environment")
set :output, "#{Rails.root}/logs/cron_log_#{ENV['RAILS_ENV']}.log"
every 1.day, :at => '00:00 am' do
command "cd #{Rails.root}/lib/tasks && rake clean__posts_table_rake"
end

I would consider using the "rake" shortcut to make it even cleaner:
every 1.day, :at => '4am' do
rake "thinking_sphinx:stop"
rake "thinking_sphinx:index"
rake "thinking_sphinx:start"
end

Related

bundler: not executable: bin/rails in rails production while using whenever gem

I'm using whenever gem for cronjobs in rails application - production.
I'm getting an error bundler: not executable: bin/rails
scheduler.rb
every 15.minute do
runner 'TestJob.perform_later()'
end
crontab
0,15,30,45 * * * * /bin/bash -l -c 'cd /home/deploy/my-app/releases/20190719103116 && bundle exec bin/rails runner -e production '\''TestJob.perform_later()'\'''
but when i run /bin/bash -l -c 'cd /home/deploy/my-app/releases/20190719103116 && bundle exec bin/rails runner -e production '\''TestJob.perform_later()'\''' in my bash replacing bin/rails with just rails this work fine.How to fix this?
Try running it via a rake task:
config/schedule.rb
every 15.minutes do
rake 'testing:run_tests'
end
lib/tasks/testing.rake
namespace :testing do
desc 'Run tests'
task run_tests: :environment do
TestJob.perform_later
end
end
Quick Answer
With the intent of still using the original runner job type, simply add this to your schedule.rb:
set :runner_command, "rails runner"
Long Answer
The documentation isn't so clear about it, but you can see the default runner job type being set as such in Whenever's /lib/whenever/setup.rb:
job_type :runner, "cd :path && :bundle_command :runner_command -e :environment ':task' :output"
In that same file you can see that this :runner_command setting is set with:
set :runner_command, case
when Whenever.bin_rails?
"bin/rails runner"
when Whenever.script_rails?
"script/rails runner"
else
"script/runner"
end
So by default your cron command will be created using bin/rails runner unless you overide it with the above answer.

Whenever cronjobs does not run on staging

I am trying to make a simple cron job running this task:
Here is my schedule.rb:
set :environment, "staging"
set :path, "/var/www/my_app/current"
every 2.minutes do
# specify the task name as a string
rake 'cron_test'
end
Here is the task:
task :cron_test => :environment do
out_file=File.new("cron_test.txt", "w")
out_file.puts(Time.now.to_s)
out_file.close
end
I tried to do what is advised on this page but nothing works:
Cron job not working in Whenever gem
When I run crontab -l :
# Begin Whenever generated tasks for: /var/www/my_app/current/config/schedule.rb
0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58 * * * * /bin/bash -l -c 'cd /var/www/my_app/current && RAILS_ENV=staging bundle exec rake cron_test'
# End Whenever generated tasks for: /var/www/my_app/current/config/schedule.rb
When I run grep CRON /var/log/syslog:
Jun 29 17:22:01 my_server CRON[21164]: (root) CMD (/bin/bash -l -c 'cd /var/www/my_app/current && RAILS_ENV=staging bundle exec rake cron_test')
Any ideas?
Thanks
What happens when you manually run the command from crontab on the console? Try to run in. See if there are any errors.
/bin/bash -l -c 'cd /var/www/my_app/current && RAILS_ENV=staging bundle exec rake cron_test'
Might be something with local or cron environment. Also logs would be great, capturing any running errors. You can set output log to capture any problems with crontab running the command.
set :output, '/path/to/file.log'
for you example:
set :output, '/var/log/my_app.cron.log'
set :environment, "staging"
set :path, "/var/www/my_app/current"
every 2.minutes do
# specify the task name as a string
rake 'cron_test'
end
Here's the documentation whenever output redirection
Thanks it is working now!
When I created the /var/log/staging_cron.log I read this:
/bin/bash: bundle: command not found
I added env :PATH, ENV['PATH'] on the first line of my schedule.rb, updated the crontab (whenever --update-crontab) and now my test file is well created and updated.

crontab didn't run on ubuntu 14.04

I had a problem about set crontab Ubuntu 14.04 on DigitalOcean VPS.
Here are my steps:
First, create new crontab: crontab -e
Second, i set this command:
This is my crontab which i set it run at 10:26AM everyday:
26 10 * * * /bin/bash -l -c 'cd /var/www/my_app/current && RAILS_ENV=production bundle exec rake check:user >> log/cron_log.log'
Third, i checked crontab as crontab -l
Finally, i restart crontab as sudo service cron restart
But it didn't run anyway (i waited for 10:24 until 10:30), and i copied this command and ran it in my console, it worked very well.
So, i didn't understand why it didn't run same as crontab. Hope everybody can explain or give me some advises. Thank you very much.
EDIT: I tried to use gem whenever but it didn't run.
set :environment, "production"
env :PATH, ENV['PATH']
env :GEM_PATH, ENV['GEM_PATH']
set :output, {:error => "log/cron_error_log.log", :standard => "log/cron_log.log"}
# Learn more: http://github.com/javan/whenever
every :day, at: '10:26am' do
rake "check:user"
end
And i updated my crontab use whenever as whenever -w
But it don't work.
Update:
#Coderhs: I tried run this command bundle exec whenever --update-crontab RAILS_ENV=production, but it didn't work :(.
This is list my crontab when i use command crontab -l:
# Begin Whenever generated tasks for: RAILS_ENV=production
PATH=/var/www/my_app/shared/bundle/ruby/2.2.0/bin:/usr/local/rvm/gems/ruby-2.2.1/bin:/usr/local/rvm/gems/ruby-2.2.1#global/bin:/usr/local/rvm/rubies/ruby-2.2.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/rvm/bin
GEM_PATH=""
26 10 * * * /bin/bash -l -c 'cd /var/www/my_app/releases/20150710024713 && RAILS_ENV=production bundle exec rake user:check --silent >> log/cron_log.log 2>> log/cron_error_log.log'
# End Whenever generated tasks for: RAILS_ENV=production
It still didn't run.
UPDATE:
The problem was solved. Because in my local TIME was different with SERVER TIME.
So, i had just set SERVER TIME same as my LOCAL TIME. Thank you everybody supported me.
Like previously mentioned, I would use whenever gem.
Make sure you run whenever -w again after each capistrano deploy so its looking for the correct release directory.
Make sure you run whenever -w as the same user that your application is running as. Don't use root unless your app is running as root, which it shouldn't be.
Also, you should have your log dir set as a shared directory in Capistrano deploy config. Something like this in config/deploy.rb :
set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'vendor/bundle', 'public/system', 'pids', 'public/uploads', 'public/temp')
Also run the 'date' command in your terminal and see what the time there is. It might not be your local time, so the cron could work but at a totally different time than when you are expecting.
The issue is probably because the user which runs the cron, can't access the ruby command 'bundle'. You will have to use the full bath to the bundle executable file in the crontab.
The better solution for this problem would be to use the whenever gem https://github.com/javan/whenever
It would manage all the stuff related to crontab, to avoid issues like this.
schedule.rb for whenever
set :environment, "production"
set :output, {:error => "log/cron_error_log.log", :standard => "log/cron_log.log"}
env :PATH, ENV['PATH']
env :GEM_PATH, ENV['GEM_PATH']
# Learn more: github.com/javan/whenever
every :day, at: '10:26am' do
rake "check:user"
end
don't forget to run.
bundle exec whenever --update-crontab RAILS_ENV=production
after you set the shedule.rb

Missing required gems when execute cron created by whenever gem in production environment

I've ruby on rails application running in production, this apps have STOCK model and it should be updated every 1 minute.
For updating this model i create simple rake task db:populate:stocks, when it contain only two operation :
truncate the STOCK model
fill STOCK model with latest data fetch from webservice
this rake task should be execute every 1 minute, for this purpose i use whenever gem. Here my schedule.rb :
env :PATH, ENV['PATH']
set :job_template, nil
set :output, {:standard => '/home/admin/shared/log/cron.log', :error => '/home/admin/shared/log/cron-error.log'}
job_type :rake, "cd :path && rake :task RAILS_ENV=:environment --trace :output"
every 1.minute do
rake "db:populate:stocks"
end
In production i'm using rvm running ruby 1.9.2-p180, rails 3.1.0 and capistrano, here my capistrano task for update cron tab :
after "deploy:update_code" do
run "cd #{release_path} && whenever --clear-crontab RAILS_ENV=production"
run "cd #{release_path} && whenever --update-crontab RAILS_ENV=production"
end
And my schedule.rb create cron task like :
# Begin Whenever generated tasks for: RAILS_ENV=production
PATH=/home/admin/.rvm//gems/ruby-1.9.2-p180#admin/bin:/home/admin/.rvm//gems/ruby-1.9.2-p180#global/bin:/home/admin/.rvm//rubies/ruby-1.9.2-p180/bin:/home/admin/.rvm//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
* * * * * cd /home/admin/releases/20120904103658 && RAILS_ENV=production rake db:populate:stocks --trace >> /home/admin/shared/log/cron.log 2>> /home/admin/shared/log/cron-error.log
THE PROBLEM is the cron task failed to execute the rake task, from cron-error.log :
/home/admin/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems.rb:314:in `bin_path': can't find gem rake ([">= 0"]) with executable rake (Gem::GemNotFoundException)
from /home/admin/.rvm//gems/ruby-1.9.2-p180#admin/bin/rake:19:in `<main>'
What i'm missing here, why the cron job failed to load my env ? Is there any problem with my schedule.rb ?
Make sure your RVM defaults to the correct ruby version where the gems are.
Then try to use bundle exec rake ...
by doing so it will explicitly invoke whatever the gems in the bundle (assuming you are using RVM)
I have similar problem and I fix with
rvm cron setup
in server. This add rvm env variables (PATH, GEM_HOME,GEM_PATH...) to my cronjob.
I have to delete the set command
env :PATH, ENV['PATH']
modifying PATH is not enough, check rvm help cron - there are few options that will help manage rvm in crontab.

How do I start Thinking Sphinx delayed delta rake task from deploy script?

I have Thinking Sphinx setup and working however I am having a problem getting the Delayed Job rake tasks to start during deployment.
I have the following task in deploy.rb which appears to execute, however the delayed jobs are not processed - they stack up until I run rake ts:dd from the server command line:
namespace :thinkingsphinx do
desc 'Start Delayed Job Sphinx delta indexing'
task :dd do
run "cd #{current_path} && rake ts:dd RAILS_ENV=#{rails_env} &"
end
end
How can I get the delayed jobs to start running from the deployment script?
Thanks
Simon
The link Florian provided has code by Amit Solanki that works!
Here is what I did to get this to work with Capistrano:
Install gems
ts-delayed-delta
daemon_generator
whenever
Create a file called script/delayed_delta with the contents:
#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'
dir = File.expand_path(File.join(File.dirname(__FILE__), '..'))
daemon_options = {
:multiple => false,
:dir_mode => :normal,
:dir => File.join(dir, 'tmp', 'pids'),
:backtrace => true
}
puts 'delayed_delta'
Daemons.run_proc('job_runner', daemon_options) do
if ARGV.include?('--')
ARGV.slice! 0..ARGV.index('--')
else
ARGV.clear
end
Dir.chdir dir
RAILS_ENV = ARGV.first || ENV['RAILS_ENV'] || 'development'
require File.join('config', 'environment')
Delayed::Worker.new(
:min_priority => ENV['MIN_PRIORITY'],
:max_priority => ENV['MAX_PRIORITY']
).start
end
Configure Capistrano
Capistrano needs to start Sphinx and job_runner (with our script/delayed_delta).
Add something like this to the deploy.rb:
deploy.task :restart, :roles => :app do
run "export RAILS_ENV=production && cd #{deploy_to}/current && /usr/bin/rake ts:rebuild"
run "export RAILS_ENV=production && cd #{current_path} && /usr/bin/ruby script/delayed_delta start"
end
Configure whenever gem
In your config/schedule.rb add lines to update Sphinx's index and start job_runner if it isn't already running
every 30.minutes do
command "export RAILS_ENV=production && cd /path/to/rails/production && /usr/bin/rake ts:index && /usr/bin/ruby script/delayed_delta start"
end
This gets converted to a crontab that is run every 30 minutes to update sphinx
Final Notes and Lessons Learned
The script/delayed_delta uses the daemon_generator gem to start the job_runner background worker script. This is equivalent to running rake thinking_sphinx:delayed_deltas on the console, but persistent.
Make sure only one job_runner or rake thinking_sphinx:delayed_deltas process is running at one time
Let Capistrano start both Sphinx (rake ts:rebuild) and script/delayed_delta. I had problem when I started sphinx and delayed_deltas from different users or different environments
This link will be useful for anybody who need to restart the delayed_delta rake task from deploy script :
http://amitsolanki.com/2010/04/running-delayed-delta-daemon-in-background-for-thinking-sphinx/#comment-5802
I would put your delayed_job task in a separate script and run it from cron or have it started/monitored by your monitoring tool of choice (e.g., monit). Your deploy script can just kill it to make sure it restarts each time (killall job_runner). Here's the script I use:
#!/usr/bin/env ruby
## this script is for making sure and delayed_jobs get run
## it is used by thinking sphinx
require File.dirname(__FILE__) + '/../config/environment'
# you can also put the definition of this in config/environments/*.rb so it's different for test, production and development
JobRunnerPidFile = "#{RAILS_ROOT}/tmp/pids/job_runner.pid"
if File.exists?(JobRunnerPidFile)
old_pid = File.read(JobRunnerPidFile).to_i
begin
if Process.getpgid(old_pid) > 0
# still running, let's exit silently...
exit(0)
end
rescue
# looks like nothing is running, so let's carry on
end
end
File.open(JobRunnerPidFile, "w") {|f| f.write "#{$$}\n" }
Delayed::Worker.new.start

Resources