I'm using resque to do some (long time) job. And I have a few classes with the same mixed-in module for queuing. Class Service substitutes in tests, that's why it standalone and (maybe too much) complicated. So the story is when I call
Campaign.perform(user_id)
directly, everything works fine, but when I try to use queue:
Resque.enqueue(Campaign, user_id)
Job created, but seems like do nothing. At least, nothing saves into the database. Which is main task of Campaign class. I can see in resque-web-interface that jobs creates and finished, and finished (to fast, almost just after create) but no result.
I'm new in Resque and not really sure it calls it all (confused how to debug it).
Does anybody have similar problem? thanks for any help.
Module:
module Synchronisable
def self.included(base)
base.extend ClassMethods
end
module ClassMethods
def perform(user_id)
save_objects("#{self.name}::Service".constantize.get_objects(user_id))
end
protected
def save_objects(objects)
raise ArgumentError "should be implemented"
end
end
class Service
def self.get_objects(user)
raise ArgumentError "should be implemented"
end
end
end
One of the classes:
class Campaign < ActiveRecord::Base
include Synchronisable
#queue = :app
class << self
protected
def save_objects(objects)
#some stuff to save objects
end
end
class Service
def self.get_objects(user_id)
#some stuff to get objects
end
end
end
This is a very old question so not sure how rails folder structure was back then but I had the same problem and issue was with inheritance. Seems if you are using Resque your job classes shouldn't inherit from ApplicationJob.
so if your code was like this in (app/jobs/campaign_job.rb):
class Campaign < ApplicationJob
#queue = :a_job_queue
def self.perform
#some background job
end
end
then remove the inheritance i.e "< ApplicationJob"
These jobs are almost certainly failing, due to an Exception. What is resque-web showing you on the Failures tab? You can also get this from the Rails console with:
Resque.info
or
Resque::Failure.all(0)
You should run your worker like this:
nohup QUEUE=* rake resque:work & &> log/resque_worker_QUEUE.log
This will output everything you debug to "log/resque_worker_QUEUE.log" and you will be able to find out what's wrong with your Campaign class.
Try this:
env TERM_CHILD=1 COUNT=2 "QUEUE=*" bundle exec rake resque:workers
Related
I am using the most basic version of delayed_job in a Rails app. I have the max time allowed for a delayed_job set at 10 minutes. I would like to get the hooks/callbacks working so I can do something after a job stop executing at the 10 minute mark.
I have this set in my rails app:
config.active_job.queue_adapter = :delayed_job
This is how I normally queue a job:
object.delay.object_action
The hook/callback example is for a named job but the basic, getting started steps are not for a named job. So I don't think I have a named job. Here is the example given to get the callbacks working:
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def before(job)
record_stat 'newsletter_job/start'
end
def after(job)
record_stat 'newsletter_job/after'
end
def success(job)
record_stat 'newsletter_job/success'
end
def error(job, exception)
Airbrake.notify(exception)
end
def failure(job)
page_sysadmin_in_the_middle_of_the_night
end
end
I would love to get the after or error hooks/callbacks to fire.
Where do I put these callbacks in my Rails app to have them fire for the basic delayed_job setup? If I should be using ActiveJob callbacks where do you put those callbacks given delayed_job is being used?
You cannot use object.delay.object_action convenience syntax if you want more advanced features like callbacks. The #delay convenience method will generate a job object that works similar to this:
# something like this is already defined in delayed_job
class MethodCallerJob
def initialize(object, method, *args)
#object = object
#method = method
#args = args
end
def perform
#object.send(#method, *#args)
end
end
# `object.delay.object_action` does the below automatically for you
# instantiates a job with your object and method call
job = MethodCallerJob.new(object, :object_action, [])
Delayed::Job.enqueue(job) # enqueues it for running later
then later, in the job worker, something like the below happens:
job = Delayed::Job.find(job_id) # whatever the id turned out to be
job.invoke_job # does all the things, including calling #perform and run any hooks
job.delete # if it was successful
You have to create what the delayed_job README calls "Custom Jobs", which are just plain POROs that have #perform defined at a minimum. Then you can customize it and add all the extra methods that delayed_job uses for extra features like max_run_time, queue_name, and the ones you want to use: callbacks & hooks.
Sidenote: The above info is for using delayed_job directly. All of the above is possible using ActiveJob as well. You just have to do it the ActiveJob way by reading the documentation & guides on how, just as I've linked you to the delayed_job README, above.
You can create delayed_job hooks/callback by something like this
module Delayed
module Plugins
class TestHooks < Delayed::Plugin
callbacks do |lifecycle|
lifecycle.before(:perform) do |_worker, job|
.....
end
end
end
end
end
And need this plugin to initializer
config/initializers/delayed_job.rb
require_relative 'path_to_test_plugin'
Delayed::Worker.plugins << Delayed::Plugins::TestHooks
Similar to perform there are also hooks for success failure and error.
And similar to 'before' you can also capture the 'after' hooks.
I have installed the Ruby gem Delayed_Job to run tasks in a queue, but it shows some behavior that I don't understand. Delayed_Job is using my local active_record so a very standard installation.
I have the code for a job in a file called test_job.rb in my /lib folder
class TestJob
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def enqueue
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
When I try to call the code from a controller like this, the first time the job seems to get submitted (is listed in rake jobs:work) but it does not run:
require 'test_job'
class ExampleController < ApplicationController
def index
end
def job
# Create a new job instance
job = TestJob.new
# Enqueue the job into Delay_Job
job.enqueue
end
end
Then when I change the controller code to do what my lib class does, it works perfectly. The job does not only get submitted to the queue, but also runs and completes without failures.
require 'test_job'
class ExampleController < ApplicationController
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def index
end
def job
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
The strange thing is that when I switch back to calling the lib job class it works without a problem. Then it does not matter whether the struct is directly defined in the controller or in the class in the lib folder.
Defining the struct inside the controller and submitting the job to the queue this way always seems to work, but afterwards also the lib class starts working and sometimes the lib class works even after a restart of the rails server.
Any ideas? Thank you very much for the help.
Best,
Bastian
I'm attempting to use the whenever to execute a method every 30 minutes. However I'm having some trouble setting it up.
schedule.rb
every 1.minutes do
runner "Post.winner"
end
post.rb
class Post < ActiveRecord::Base
module Post
def winner
#do some stuff that saves a new value in database
I don't think the runner is working because the field that I save a value to still shows up as nil in the console.
You most probably do not want to run a controller method (eg you have no request to serve there). Either create a runner from a class or module.
# schedule.rb
every 30.minutes do
runner "Post.winner"
end
#lib/post.rb
module Post
def self.winner
...
end
end
or a rake task in lib/tasks/
# schedule.rb
every 30.minutes do
rake post:winner
end
#lib/tasks/post.rake
namespace :post do
desc 'calculate winner'
task :winner do
...
end
end
I'm almost sure you have a model Post. Create a class method:
class Post
...
def self.winner
...
end
end
But if you are using Post.winner only for the schedule, I prefer a rake task as #xlembouras suggested.
I cannot log messages from my delayed_job process. Here is the job that is being run.
class MyJob
def initialize(blahblah)
#blahblah = blahblah
#logger = Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
end
def perform
#logger.add Logger::INFO, "logging from delayed_job"
#do stuff
end
end
I've tried various logging levels, and I have config.log_level = :debug in my environment configuration. I run delayed_job from monit. I'm using delayed_job 3.0.1 with ruby 1.9.3 and rails 3.0.10.
An explation could be that the job gets initialized only once on producer side. Then it gets serialized, delivered through the queue (database for example) and unserialized in the worker. But the initialize method is not being called in the worker process again. Only the perform method is called via send.
However you can reuse the workers logger to write to the log file:
class MyJob
def perform
say "performing like hell"
end
def say(text)
Delayed::Worker.logger.add(Logger::INFO, text)
end
end
Don't forget to restart the workers.
In RAILS_ROOT/config/initializers have a file called delayed_job_config.rb with these lines:
Delayed::Worker.logger = Rails.logger
Delayed::Worker.logger.auto_flushing = true
Remember to re-start your workers after doing this.
Let me know if this helps
I don't see why you would set the logger in the job. When I've done this I set the worker to to use a specific file on start e.g. Logger.new("log/worker_#{worker_number}") which ensures that each worker outputs to it's own file and you don't have to worry about multiple workers writing to the same file at the same time (messy).
Also, in plain ol' ruby you can call #logger.info "logging from delayed_job".
Finally, i'm pretty sure that 'perform' is called directly by your worker and instantiated, so you can refactor to:
class MyJob
def perform(blahblah)
#logger.add Logger::INFO, "logging from delayed_job"
#blahblah = blahblah
#do stuff
end
end
This is working just fine for me in Rails 3.2:
class FiveMinuteAggregateJob < Struct.new(:link, :timestamp)
def perform
Rails.logger.info 'yup.'
end
end
I am trying to write a rspec for delayed job.
Currently I use delayed job as follows:
class IncomingMailsController < ApplicationController
...
MailingJob.new(#incoming_mail.id).perform
...
end
Then in /lib/mailing_job.rb:
class MailingJob < Struct.new(:mailing_id)
def perform
.......
How can I test that with rspec? Right now I have:
/spec/lib/mailing_job/mailingjob_spec.rb
require 'spec_helper'
describe MailingJob do
include DelayedJobSpecHelper
it "should have been worked on if I do something that queues jobs" do
#incoming_mail = IncomingMail.create(.........)
MailingJob.new(#incoming_mail.id).perform
#IncomingMail.method_that_queues_jobs
work_off
MailingJob.should be_worked_on
end
end
/spec/lib/delayed_job_spec_helper.rb
module DelayedJobSpecHelper
# http://erikonrails.snowedin.net/?p=230
def work_off
Delayed::Job.all.each do |job|
job.payload_object.perform
job.destroy
end
end
end
But this errors with:
1) MailingJob should have been worked on if I do something that queues jobs
Failure/Error: MailingJob.should be_worked_on
NoMethodError:
undefined method `worked_on?' for MailingJob:Class
# ./spec/lib/mailing_job/mailingjob_spec.rb:19
Ideas? Thanks
You're queueing your job incorrectly in the controller. perform is called by the worker, not by you.
Delayed::Job.enqueue MailingJob.new(#incoming_mail.id)
Similarly, don't call perform in your spec, that's what work_off does.
The structure of the spec should be this:
Access controller method that queues the job
Tell DelayedJob to work off the queue
Check to see that it did what it was supposed to do
I'm not sure where be_worked_on comes in, as worked_on? is not defined anywhere in DelayedJob (and indeed, is the cause of your error). I would instead check something was done, like a mail was sent, or whatever your job is supposed to do.