I am trying to run a background job on Heroku using Resque/Clockwork.
Locally, jobs are queued by Clockwork and executed by Resque as expected. On Heroku, sometimes it works... but most of the time Clockwork queues the job as expected, and when executed by Resque I get a failed job with the error 'No job defined for class'.
The exact error that shows up through the Resque admin panel is:
Worker 960f8a1b-cce9-497a-a7ab-9b40c166a600:2+1 on FEATURED_CATALOG_ITEMS_CACHE at 27 minutes ago Retry or Remove
Class nil
Arguments nil
Exception Error
Error No job defined for class 'Workers::FeaturedCatalogItemsCacheWorker'
My code looks like:
/app/models/workers/featured_catalog_items_cache_worker.rb
class Workers::FeaturedCatalogItemsCacheWorker
#queue = :featured_catalog_items_cache
def self.perform
p 'do work' # there is more code here but edited for brevity
end
end
/lib/clock.rb
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
module Clockwork
every(2.minutes, 'Queueing FeaturedCatalogItemsCacheWorker job') { Resque.enqueue(Workers::FeaturedCatalogItemsCacheWorker) }
end
/lib/tasks/resque.rake
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
Procfile
web: rackup -p $PORT
resque: env TERM_CHILD=1 bundle exec rake resque:work
clock: bundle exec clockwork lib/clock.rb
Am I missing something?
maybe in /lib/clock.rb
require 'Workers::FeaturedCatalogItemsCacheWorker'
Solution :
require './config/boot'
require './config/environment'
require 'clockwork'
require 'resque'
module Clockwork
handler do |job|
puts "Running job : #{job}"
Resque.enqueue(job)
end
every(15.minutes, NewsRssReader )
end
Related
Running Rails 4.2.0 so I am using the ActiveJob w/ Sidekiq backend. I need to invoke regularly scheduled background jobs so I am hoping to use Clockwork, but I haven't seen any examples of how to use it with ActiveJob.
Here is my lib/clock.rb based off the Clockwork Github example:
require 'activejob'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
ProductUpdateJob.perform_later
end
every(30.seconds, 'ProductUpdateJob.perform_later')
end
Update:
I was able to get it to work for my situation, but I'm not entirely satisfied with the solution since I am having to load the entire environment. I would only like to require the bare minimum to run the ActiveJobs.
Dir["./jobs/*.rb"].each {|file| require file }
require 'clockwork'
require './config/boot'
require './config/environment'
#require 'active_job'
#require 'active_support'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
#send("#{job}")
#puts "#{job}".constantize
"#{job}".constantize.perform_later
end
every(10.seconds, 'ProductUpdateJob')
end
Here is a working version
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job, time|
puts "Running #{job}, at #{time}"
"#{job}".constantize.perform_async
end
every(10.seconds, 'ProductUpdateJob')
end
It is possible, but it will have no context of your redis/job backend configuration. You could always require a file in config/initializers that configures sidekiq and redis. This is the minimum you will need:
require 'clockwork'
require 'active_job'
ActiveJob::Base.queue_adapter = :sidekiq
Dir.glob(File.join(File.expand_path('.'), 'app', 'jobs', '**')).each { |f| require f }
module Clockwork
every(1.minute, 'Do something') { SomeJob.perform_later }
end
I am trying to understand how to execute custom code with clockwork. This is the example lib/clock.rb file that Heroku uses in its devcenter document.
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
include Clockwork
every(4.minutes, 'Queueing interval job') { Delayed::Job.enqueue IntervalJob.new }
every(1.day, 'Queueing scheduled job', :at => '14:17') { Delayed::Job.enqueue ScheduledJob.new }
What is IntervalJob and ScheduledJob? Where are these files supposed to be located? I want to run my own custom job that has access to my database records.
EDIT
This is my /lib/clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(2.minutes, 'Filtering Streams') { Delayed::Job.enqueue FilterJob.new}
end
This is my /lib/filter_job.rb
class FilterJob
def perform
#streams = Stream.all
#streams.each do |stream|
# manipulating stream properties
end
end
end
I get the error:
uninitialized constant Clockwork::FilterJob (NameError)
/app/lib/clock.rb:11:in `block in <module:Clockwork>'
You need to do the following:
Firstly install the clockwork gem.
In your lib folder create a clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(1.day, 'Creating Cycle', :at => '22:00') { Delayed::Job.enqueue CyclePlannerJob.new}
end
In the example your provided IntervalJob and ScheduledJob, are delayed jobs. Clockwork triggers them on the time specified. I am calling the CyclePlannerJob, this is what my file looks like.
lib/cycle_planner_job.rb
class CyclePlannerJob
def perform
CyclePlanner.all.each do |planner|
if Time.now.in_time_zone("Eastern Time (US & Canada)").to_date.send("#{planner.start_day.downcase}?")
planner.create_cycle
end
end
end
end
In my example everyday at 10pm, I am running the CyclePlanner job, which runs the delayed job I have setup. Similar to the Heroku example.
Bare in mind to use this you need to setup the clock work and delayed jobs on your Heroku app in the dashboard.
Also your Procfile should look like this.
worker: bundle exec rake jobs:work
clock: bundle exec clockwork lib/clock.rb
Let me know if you have any questions, I can go into more detail if needed.
Looks like name space issue. Move your filter_job.rb to models directory and try.
I don't understand why my rake task is not running from within a resque worker. Running
rake :send_this_email
from the console works fine, I just want to run it as a cron job (as follows) but something is not working proplerly while invoking the rake task from within the worker.
My rescue_schedule.yml
send_this_email:
cron: "*/2 * * * *"
class: SendThisEmailWorker
args:
description: "Send email when condition defined in rake task is met"
My send_this_email_worker.rb in workers directory, where the problem must be if I can manually call the rake task myself from the console?
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
Rake::Task["send_this_email"].invoke
end
end
When I start my dev server this send_this_email rake task should run every 2 minutes correct? It's not and the resque admin panel shows it as a job in the queue. What am I missing here?
Thanks for your attention.
UPDATED from gerep comment
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
Only require 'rake' is not enough. For example if you do
Rake::Task.tasks #list down all task
You will get []
You need to tell your worker class to load tasks.
Try this
require 'rake'
Rake::Task.clear # necessary to avoid tasks being loaded several times in dev mode
YOUR_APP_NAME::Application.load_tasks
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
YOUR_APP_NAME is the name of your app and can be found at config/application.rb
I'm using Rails3 (Windows, Ruby 1.8.7) with rufus-scheduler gem. Gem works fine, but if I'm trying to run some standard rake task, error occurs:
Don't know how to build task 'db:version' # ofc, db:version is just example
Terminal command
rake -T
works
If I'm trying to define own simple rake commands, they works fine too:
# /lib/my_scheduler.rb
require 'rubygems'
require 'rake'
require 'rufus/scheduler'
load File.join( Rails.root, 'lib', 'tasks', 'my_own_tasks.rake')
scheduler = Rufus::Scheduler.start_new
scheduler.every '5s' do
Rake::Task["my_own_namespace:test"].invoke
end
end
# /lib/tasks/my_own_tasks.rb
namespace :my_own_namespace do
task :test do
puts "Some scheduler task"
end
end
... but using standard rake tasks *in my_own_tasks* throws the same error.
Some help would be appreciated
PS. I'm newbie, so sorry, if that was dumb question
Maybe someone will need solution:
system("rake namespace:task")
f.e:
system("rake db:version")
How do I make sure resque has access to all my ENV variables? I'm trying to send an email from within a resque job and it cannot send because the actionmailer smtp username/pass are set via ENV variables. It looks like even ENV['RAILS_ENV'] is not available from within the resque job.
Here's my resque.rake file:
# Run to start:
# rake resque:work QUEUE='*'
require 'resque/tasks'
require 'resque_scheduler/tasks'
task "resque:setup" => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
Here's my resque.rb initializer:
require 'resque_scheduler'
Resque.redis = 'localhost:6379'
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Here's my procfile that starts redis/resque/resque worker
mongo: mongod
redis-server: redis-server /usr/local/etc/redis.conf
scheduler: bundle exec rake resque:scheduler
worker: bundle exec rake resque:work QUEUE=images, notifications
Add them to resque.rake
task "resque:setup" => :environment do
# ... other stuff
ENV['RAILS_ENV'] = Rails.env
end
Well I can't tell for sure but if this is UNIX, open /proc/self/environ will work even if there's no language facility to read ENV.
This is, of course, last resort.