What is the best way to run cron jobs on rails, when different machines have different jobs to do?
For example, server 1 runs cron job A, while server 2 runs cron job B
Is there a way to deploy the cron files along when we do a regular cap deploy?
take a look at the whenever gem, http://github.com/javan/whenever
It is great for automating cron tasks with rails with a clear DSL. We have been using it production for several months now and it just works and is very lightweight. Some examples from their README:
every 3.hours do
runner "MyModel.some_process"
rake "my:rake:task"
command "/usr/bin/my_great_command"
end
every 1.day, :at => '4:30 am' do
runner "MyModel.task_to_run_at_four_thirty_in_the_morning"
end
every :hour do # Many shortcuts available: :hour, :day, :month, :year, :reboot
runner "SomeModel.ladeeda"
end
every :sunday, :at => '12pm' do # Use any day of the week or :weekend, :weekday
runner "Task.do_something_great"
end
The README is very thorough, but there is also a good screencast on railscasts: http://railscasts.com/episodes/164-cron-in-ruby
It easily integrates with capistrano with the following code (copied from README):
after "deploy:symlink", "deploy:update_crontab"
namespace :deploy do
desc "Update the crontab file"
task :update_crontab, :roles => :db do
run "cd #{release_path} && whenever --update-crontab #{application}"
end
end
As far as machine specific, you could use a local config file or even symlink the config/schedule.rb file on deploy. I think I would include a local file that would be symlinked on deploy local_schedule.rb and then put this at the top of the config/schedule.rb
if File.exists?(File.dirname(__FILE__) + '/config/local_schedule.rb')
require File.dirname(__FILE__) + '/local_schedule.rb'
end
Your schedule would run but then include anything local, just make sure it is symlinked before the cap task above is run and you should be good to go.
I hope this helps!
Related
I'm seeking help concerning the whenever gem. Here my case:
I have the task I generated and that works when I run it through command line as such rake dashboard_data_t:collect.
namespace :dashboard_data_t do
desc "TODO"
task collect: :environment do
#task...
end
end
I then followed the documentation provided here in such a way that my config/schedule.rb looks like so:
# config/schedule.rb
every :day, at: '10:43 am' do
rake "dashboard_data_t:collect"
end
Happily done with that, I thought it would go on and run itself alone, without me needing to do anything more. But I noticed it didn't. I thought it might come from my task so I created an other one, this time way more simple than the 1st. Its purpose was solely to experiment and find what was going wrong:
namespace :test_name do
desc "TODO"
task test_task: :environment do
sh('echo', 'test task runned successfully')
end
end
I then added the following to my config/schedule.rb:
# config/schedule.rb
every 1.minutes do
rake "test_name:test_task"
end
Once again, the task didn't execute (periodically), but was still working manually.
I noticed by running the crontab -e command that RAILS_ENV was set to production, I understood why my dashboard_data_t:collect task wasn't working, because it relied on the development db. So I did the following:
# config/schedule.rb
set :environment, 'development'
Unfortunately, this didn't change anything as both tasks still don't execute. Now I'm stuck here with no ideas whatsoever. Can anyone help me.
Cheers.
I am working on an event app, that showcases the events near my area.
A Boolean is_weekly_event is set for each entry.
If it is set to true I need to show that event in my app every week on the same data and time.
Therefore, I need to write a cron job, that re-creates this event every time it passes it's end_time, for the same time next week!
Thanks in advance!
you can use whenever gem to add cron job. I have used this in one of my projects and works very fine.
You can define rake tasks in lib/ and in your schedule.rb file which will be generated when you will do whenervize your project.
every 3.hours do # 1.minute 1.day 1.week 1.month 1.year is also supported
# your task here
end
every 1.day, :at => '4:30 am' do
# your task here
end
After finishing all you need is to update your crontab file for your machine using this command.
$ whenever --update-crontab
Note - just a take care to load proper environment(dev, prod) for the rake tasks, it took me hard to find out when i was trying to run the cron.
Thanks
I am using whenever gem. I am calling runner to run job. I called runner for the same job by 3 type. I am not sure which one working here.
I used command whenever --update-crontab project_name
then crontab -l
Schedule.rb
set :output, 'log/whenever.log'
every 1.days , :at => '03:51 pm' do
runner "SomeJob.perform_later", filename: '/app/jobs/some_job.rb'
end
every 1.days , :at => '03:51 pm' do
runner "SomeJob.perform_later", filename: './app/jobs/some_job.rb'
end
every 1.days , :at => '03:51 pm' do
runner "SomeJob.perform_later"
end
Also these ran only once. I am having difficult time to debug here.
Can anyone tell which one is correct? Also what is the correct way to debug in this scenario?
Running crontab -l gives me this -
51,51,51 16 * * * /bin/bash -l -c 'cd /home/rahul/orthoweb && bin/rails runner -e production '\''InvestigationStopJob.perform_later'\'' >> log/whenever.log 2>&1
To run in development environment I ran this command -
whenever --update-crontab --set environment='development'
But this gives me this message -
## [message] Above is your schedule file converted to cron syntax; your crontab file was not updated.
There are some issues with running the jobs in development mode.
What works is that you have to close all tabs before running your job and scheduler.
There is no need to change setup of your env in schedule file.
To run in development use this
whenever --update-crontab --set environment='development'
Without specifying filename jobs does not run properly. The following line works correctly.
every 1.days , :at => '03:51 pm' do
runner "SomeJob.perform_later", filename: '/app/jobs/some_job.rb'
end
I think, that the resulting crontab contains only one of those tasks, because they were overwritten.
To be sure, check out the resulting crontab by typing whenever. I think that there will be only one entrance.
Once upon a time, I had one app - Cashyy. It used sidekiq. I deployed it and used this upstart script to manage sidekiq (start/restart).
I decide to deploy another app to the same server. The app (let's call it Giimee) also uses sidekiq.
And here is the issue. Sometimes I need to restart sidekiq for Cashyy, but not for Giimee. Now, as I understand I will need to hack something using index thing (in upstart script and managing sidekiqs: sudo restart sidekiq index=1) (if I understood it correctly).
BUT!
I have zero desire to dabble with these indexes (nightmare to support? Like you need to know how many apps are using sidekiq and be sure to assign unique index to each sidekiq. And to know assigned index if you want to restart specific sidekiq).
So here is the question: how can I isolate each sidekiq (so I would not need to maintain the index) and still get the stability and usability of upstart (starting the process, restarting, etc.)?
Or maybe I don't understand something and the thing with index is state of art?
You create two services:
cp sidekiq.conf /etc/init/cashyy.conf
cp sidekiq.conf /etc/init/glimee.conf
Edit each as necessary. sudo start cashyy, sudo stop glimee, etc. Now you'll have two completely separate Sidekiq processes running.
As an alternative to an upstart script, you can use Capistrano and Capistrano-Sidekiq to manage those Sidekiqs.
We have Sidekiq running on 3 machines and have had a good experience with these two libraries/tools.
Note: we currently use an older version of Capistrano (2.15.5)
In our architecture, the three machines are customized slightly on deploy. This led us to break up our capistrano deploy scripts by machine so that we could customize some classes, manage Sidekiq, etc. Our capistrano files are structured something like this:
- config/
- deploy.rb
- deploy/
- gandalf.rb
- gollum.rb
- legolas.rb
With capistrano-sidekiq, we are able to control, well, Sidekiq :) at any time (during a deploy or otherwise). We set up the Sidekiq aspects of our deploy scripts in the following way:
# config/deploy.rb
# global sidekiq settings
set :sidekiq_default_hooks, false
set :sidekiq_cmd, "#{fetch(:bundle_cmd, 'bundle')} exec sidekiq"
set :sidekiqctl_cmd, "#{fetch(:bundle_cmd, 'bundle')} exec sidekiqctl"
set :sidekiq_role, :app
set :sidekiq_pid, "#{current_path}/tmp/pids/sidekiq.pid"
set :sidekiq_env, fetch(:rack_env, fetch(:rails_env, fetch(:default_stage)))
set :sidekiq_log, File.join(shared_path, 'log', 'sidekiq.log')
# config/deploy/gandalf.rb
# Custom Sidekiq settings
set :sidekiq_timeout, 30
set :sidekiq_processes, 1
namespace :sidekiq do
# .. code omitted from methods and tasks for brevity
def for_each_process(&block)
end
desc 'Quiet sidekiq (stop accepting new work)'
task :quiet, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
end
desc 'Stop sidekiq'
task :stop, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
end
desc 'Start sidekiq'
task :start, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
end
desc 'Restart sidekiq'
task :restart, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
end
end
When I need to restart one of my Sidekiq instances, I can just go to my terminal and execute the following:
$ bundle exec cap gandalf sidekiq:restart
$ bundle exec cap gollum sidekiq:stop
It's made Sidekiq management quite painless for our team and thought it would be worth sharing in the event something similar could help you out.
I am trying to set up a cron task for a model in a redmine plugin. The Whenever gem looked like the best option. Alas, it does not seem to do anything.
I have set up the gem, both in the config folder of the plugin and the primary one. I added it to the crontab (using the whenever -i command) have reset the server.
The action was not run, and no log was generated
here is the schedule.rb file.
set :output, "/path/to/my/cron_log.log"
every 1.day, :at => '9:00 am' do
runner "pluginMailMethod.send_emails"
end
every :thursday, :at => '4:20 pm' do
runner "pluginMailMethod.send_emails"
end