Is there a way to limit the number of instances of a rake task?
I have a rake task for reading emails that runs every 5 mins as a cron job.
Sometimes the rake tasks takes more than 5 mins to complete and another
rake task is launched before it finishes.
There are hacky workarounds to check ps -Af inside the rake file but I
am looking for cleaner way to limit launching multiple instances of the
rake tasks similar to how the daemon gem does.
Checking emails is just an example, I have several such rake tasks that involve
polling multiple servers.
You could also just use a PidFile.
First, install the 'pidfile' gem. Then make your task like this:
task :my_task => :environment do |task|
PidFile.new(:piddir => Rails.root.join('tmp', 'pids'), :pidfile => task.name)
# do some stuff
end
Still can't find a super elegant way, so I resorted to saving a unique file for
each rake task.
This is how the rake task looks now -
run_unique_rake(__FILE__) do
puts "\n is running\n"
sleep(40)
end
here is run_unique_rake
def self.run_unique_rake(file)
path = RAILS_ROOT + "/" + CONFIG['rake_log'] + "/" + File.basename(file)
unless File.exists?(path)
`touch #{path}`
yield if block_given?
`rm #{path}`
end
end
Still hoping for an elegant way within rake to limit to a single instance.
Related
I'm seeking help concerning the whenever gem. Here my case:
I have the task I generated and that works when I run it through command line as such rake dashboard_data_t:collect.
namespace :dashboard_data_t do
desc "TODO"
task collect: :environment do
#task...
end
end
I then followed the documentation provided here in such a way that my config/schedule.rb looks like so:
# config/schedule.rb
every :day, at: '10:43 am' do
rake "dashboard_data_t:collect"
end
Happily done with that, I thought it would go on and run itself alone, without me needing to do anything more. But I noticed it didn't. I thought it might come from my task so I created an other one, this time way more simple than the 1st. Its purpose was solely to experiment and find what was going wrong:
namespace :test_name do
desc "TODO"
task test_task: :environment do
sh('echo', 'test task runned successfully')
end
end
I then added the following to my config/schedule.rb:
# config/schedule.rb
every 1.minutes do
rake "test_name:test_task"
end
Once again, the task didn't execute (periodically), but was still working manually.
I noticed by running the crontab -e command that RAILS_ENV was set to production, I understood why my dashboard_data_t:collect task wasn't working, because it relied on the development db. So I did the following:
# config/schedule.rb
set :environment, 'development'
Unfortunately, this didn't change anything as both tasks still don't execute. Now I'm stuck here with no ideas whatsoever. Can anyone help me.
Cheers.
I am a beginner when it comes to Rails. I am trying to follow this example:
http://ryanselk.com/2014/09/25/using-background-jobs-in-rails-42-with-active-job/
It says:
"Jobs can be added to the job queue from anywhere. We can add a job to the queue by: ResizeImage.perform_later 'http://example.com/ex.png' "
[UPDATE] Sorry, I am stupid. I came up with this task:
namespace :simple do
# call from command line:
# rake simple:resize_images
desc "Resize images"
task resize_images: :environment do
Dir.foreach('storage') do |next_image|
puts next_image
next if next_image == '.' or next_image == '..'
ResizeImage.perform_later next_image
end
end
end
but now I do:
rake simple:resize_images
and I get:
zacek2_phpP9JGif.jpg
rake aborted!
NameError: uninitialized constant ResizeImage
I've tried:
require ResizeImage
but that did not fix the problem.
I am afraid I don't understand how loading works in Rails. How do I load ResizeImage?
Do I set it up as a cron job?
No.
From the rails guides:
Active Job is a framework for declaring jobs and making them run on a variety of queueing backends.
Active Job is an interface to queueing backends such as sidekiq, delayed_job or resque. It's simply a way for you to write background jobs where you don't have to care about which of the queueing backends will be used.
How do I start ActiveJob?
So ActiveJob doesn't run background jobs on it's own. You're still missing one of the backends. Say you have decided to use delayed_job: Get it installed and then start it via:
script/delayed_job start
I don't understand where "anywhere" is.
That means anywhere in your code, you could write something like:
user.rb
def send_registration_email
UserRegistraionMailJob.perform_later self
end
I have a rake task in my Rails application,and when I execute the order in my rails app path /home/hxh/Share/ruby/sport/:
rake get_sportdata
This will work fine.
Now,I want to use crontab to make this rake to be a timed task .so,I add a task:
* * * * * cd /home/hxh/Share/ruby/sport && /usr/local/bin/rake get_sportdata >/dev/null 2>&1
But this doesn't work.I get the log in cron.log file:
Job `cron.daily' terminated
I want to know where the error is.
Does the "cd /home/hxh/Share/ruby/sport && /usr/local/bin/rake get_sportdata >/dev/null 2>&1" can work in your terminal?
But use crontab in Rails normally is not a good idea. It will load Rails environment every time and slow down your performance.
I think whenever or rufus-scheduler are all good. For example, use rufus-scheduler is very easy. In config\initializers\schedule_task.rb
require 'rubygems'
require 'rufus/scheduler'
scheduler = Rufus::Scheduler.start_new(:thread_name => "Check Resources Health")
scheduler.every '1d', :first_at => Time.now do |job|
puts "###########RM Schedule Job - Check Resources Health: #{job.job_id}##########"
begin
HealthChecker.perform
rescue Exception => e
puts e.message
puts e.backtrace
raise "Error in RM Scheduler - Check Resources Health " + e.message
end
end
And implement "perform" or some other class method in your controller, now the controller is "HealthChecker". Very easy and no extra effort. Hope it help.
So that you can test better, and get a handle on whether it works I suggest:
Write a shell script in [app root]/script which sets up the right environment variables to point to Ruby (if necessary) and has the call to rake. E.g., something like script/get-sportdata.sh.
Test the script as root. E.g., first do sudo -s.
Call this script from cron. E.g., * cd [...] && script/get-sportdata.sh. If necessary, test that line as root too.
That's been my recipe for success, running rake tasks from cron on Ubuntu. This is because the cron environment is a bit different than the usual shell setup. And so limiting your actual cron jobs to simple commands to run a particular script are a good way to divide the configuration into smaller parts which can be individually tested.
I've got Resque workers that typically shouldn't take longer than about 1-5 minutes to run, but frequently those workers will get "stuck" and go idle, clogging up workers and doing nothing.
So I'd like to regularly check for workers that have been running longer than X time and purge them. But I need to do this automatically, so I don't have to personally go in and manually clear them (Resque.workers.each {|w| w.unregister_worker}) every few hours.
This needs to work on Heroku.
Put this into a rake task:
allocated_time = 60 * 60 # 1 hour
Resque::WorkerRegistry.working.each do |worker|
if (worker.started <=> Time.now - allocated_time) < 1
worker.unregister
end
end
Use heroku scheduler, you can set it to minimum of 10 minutes if that suites.
For Resque v1,
# lib/tasks/clear_stale_workers.rake
namespace :clear do
desc 'Clearing stuck workers ...'
task :stale_workers => :environment do
Resque.workers.each do |w|
w.unregister_worker unless w.started > 1.hour.ago
end
end
end
From the command line, rake clear:stale_workers
On Heroku, set the set the scheduler to run this Rake task.
This worked for me to remove the specific workers running stale jobs. You could add it to a rake task.
Resque::Worker.working.each{|w| w.done_working }
From what I remember, in the documentation is specified that in the test environment, the database is always cleared even when you run rake ( with no arguments ). I'd like to achieve such a thing, so that it doesn't matter if I run a task or not, when I run rake, there's always a Rake task being executed. Is this possible? Is this where the default task kicks in?
Create a file called rakefile in the directory you want to run the task from.
This code will make it so that if you just type "rake" my_default_task will run:
task :default => 'my_default_task'
task :my_default_task do
puts "Now I am doing the task that Tempus wants done when he/she types 'rake' in the console."
end
task :my_not_default_task do
puts "This isn't the default task."
end
However, if you typed rake my_not_default_task, then my_default_task would NOT run. If you want it to run regardless here is one thing you can do:
task :default => 'my_default_task'
task :my_default_task do
puts "This is the default task"
end
task :my_not_default_task do
puts "This isn't the default task."
end
Rake::Task['my_default_task'].invoke
The last line in this code ensures that my_default_task runs even when you call some other task, so if you typed rake my_not_default_task the my_default_task'would also run.
EDIT:
When you're working with rails you can put the tasks above in a file in the lib/tasks folder with an extension of .rake and rails will automagically run them when you do rake
Jason Seifer has a real nice tutorial on rake.