I want the execution of a method in a controller to run in background, so that the speed is not affected. I found out that this could be done using delayed jobs.
following is the method I want to delay:
private
def update_product_count(skus, qty)
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}> output"
end
I tried using:
def show
if defined? Delayed::Job
Delayed::Job.enqueue(update_product_count(#skus.to_s, #qty.to_s))
end
end
This runs the script within the delayed method, but gives error:
ArgumentError in OrdersController#show
Cannot enqueue items which do not respond to perform
and the view corresponding to the show does not get rendered.
then I tried:
def show
delay.update_product_count(#skus.to_s, #qty.to_s)
end
This doesn't run the method and also gives the following error:
ArgumentError in OrdersController#show
wrong number of arguments (1 for 0)
I also tried handle_asynchronously :update_product_count. But this too gives wrong number of arguments (1 for 0)
Could someone please help me figure this out. Thanks
EDIT: the following change does not give any error, but the script does seem to run
/lib/update_count.rb
class UpdateCount < Struct.new(:skus, :qty, :path)
def perform
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}"
end
end
/app/controller/order_controller.rb:
require 'update_count'
def show
Delayed::Job.enqueue(UpdateCount.new(#skus.to_s, #qty.to_s, path))
end
Place the code you want to execute in perform method, and enqueu the class in the delayed jobs, which when executed will call the perform method
Eg:
/lib/product_count.rb
class ProductCount < Struct.new(:skus, :qty)
def perform
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{skus.to_json} #{qty.to_json} #{path}> output"
end
end
Call the delayed job
Delayed::Job.enqueue(ProductCount.new(#skus.to_s, #qty.to_s), :queue => "product_count")
You need to create a new class with public method "perform" which will incapsulate all job that you want:
class MegaJob
def initialize(options=nil)
#skus = options[:skus]
#qty = options[:qty]
end
def perform
update_product_count
end
private
def update_product_count
path = File.expand_path('../../../voylla_scripts/eBay', __FILE__)
system "python2 "+path+"/ReviseOnOrder.py #{#skus.to_json} #{#qty.to_json} #{path}> output"
end
end
To start this job:
Delayed::Job.enqueue MegaJob.new(skus: your_skus, qty: your_qty)
PS Don`t copy and paste the example!
Kinda embarrassing:
script/delayed_job start.
phew!!
Related
Can I use the hooks of a delayed job to stop it before running?
If so how?
class Jobs::SomeJob < Struct.new
def before(job)
if should_not_run_job
# Do I raise an exception?
# Is there an official way to stop a Job from running?
end
end
def perform
# Working...
end
def queue_name
return "SomeJob";
end
end
Do I raise an error?
Which hook is recommended to use?
Raising an exception wouldn't work because then the retry mechanism would kick in and would retry running the job a couple of times.
Instead, I would just add a guard clause that in the first line of the perform method that returns without doing anything when the conditions is not true anymore. It depends on how your condition looks like but something like this might work for you:
def perform
return if job_should_not_run_anymore?
# Working
end
private
def job_should_not_run_anymore?
# Condition
end
When your condition is based on the job itself that is only available in a hook method like before but not in the perform method then I would store the result of the condition in an instance variable and check that variable in the perform method like this:
def before(job)
#outdated = job.run_at > 15.minutes.ago
end
def perform
return if #outdated
# Working
end
I am using the most basic version of delayed_job in a Rails app. I have the max time allowed for a delayed_job set at 10 minutes. I would like to get the hooks/callbacks working so I can do something after a job stop executing at the 10 minute mark.
I have this set in my rails app:
config.active_job.queue_adapter = :delayed_job
This is how I normally queue a job:
object.delay.object_action
The hook/callback example is for a named job but the basic, getting started steps are not for a named job. So I don't think I have a named job. Here is the example given to get the callbacks working:
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def before(job)
record_stat 'newsletter_job/start'
end
def after(job)
record_stat 'newsletter_job/after'
end
def success(job)
record_stat 'newsletter_job/success'
end
def error(job, exception)
Airbrake.notify(exception)
end
def failure(job)
page_sysadmin_in_the_middle_of_the_night
end
end
I would love to get the after or error hooks/callbacks to fire.
Where do I put these callbacks in my Rails app to have them fire for the basic delayed_job setup? If I should be using ActiveJob callbacks where do you put those callbacks given delayed_job is being used?
You cannot use object.delay.object_action convenience syntax if you want more advanced features like callbacks. The #delay convenience method will generate a job object that works similar to this:
# something like this is already defined in delayed_job
class MethodCallerJob
def initialize(object, method, *args)
#object = object
#method = method
#args = args
end
def perform
#object.send(#method, *#args)
end
end
# `object.delay.object_action` does the below automatically for you
# instantiates a job with your object and method call
job = MethodCallerJob.new(object, :object_action, [])
Delayed::Job.enqueue(job) # enqueues it for running later
then later, in the job worker, something like the below happens:
job = Delayed::Job.find(job_id) # whatever the id turned out to be
job.invoke_job # does all the things, including calling #perform and run any hooks
job.delete # if it was successful
You have to create what the delayed_job README calls "Custom Jobs", which are just plain POROs that have #perform defined at a minimum. Then you can customize it and add all the extra methods that delayed_job uses for extra features like max_run_time, queue_name, and the ones you want to use: callbacks & hooks.
Sidenote: The above info is for using delayed_job directly. All of the above is possible using ActiveJob as well. You just have to do it the ActiveJob way by reading the documentation & guides on how, just as I've linked you to the delayed_job README, above.
You can create delayed_job hooks/callback by something like this
module Delayed
module Plugins
class TestHooks < Delayed::Plugin
callbacks do |lifecycle|
lifecycle.before(:perform) do |_worker, job|
.....
end
end
end
end
end
And need this plugin to initializer
config/initializers/delayed_job.rb
require_relative 'path_to_test_plugin'
Delayed::Worker.plugins << Delayed::Plugins::TestHooks
Similar to perform there are also hooks for success failure and error.
And similar to 'before' you can also capture the 'after' hooks.
I have installed the Ruby gem Delayed_Job to run tasks in a queue, but it shows some behavior that I don't understand. Delayed_Job is using my local active_record so a very standard installation.
I have the code for a job in a file called test_job.rb in my /lib folder
class TestJob
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def enqueue
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
When I try to call the code from a controller like this, the first time the job seems to get submitted (is listed in rake jobs:work) but it does not run:
require 'test_job'
class ExampleController < ApplicationController
def index
end
def job
# Create a new job instance
job = TestJob.new
# Enqueue the job into Delay_Job
job.enqueue
end
end
Then when I change the controller code to do what my lib class does, it works perfectly. The job does not only get submitted to the queue, but also runs and completes without failures.
require 'test_job'
class ExampleController < ApplicationController
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def index
end
def job
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
The strange thing is that when I switch back to calling the lib job class it works without a problem. Then it does not matter whether the struct is directly defined in the controller or in the class in the lib folder.
Defining the struct inside the controller and submitting the job to the queue this way always seems to work, but afterwards also the lib class starts working and sometimes the lib class works even after a restart of the rails server.
Any ideas? Thank you very much for the help.
Best,
Bastian
I'm using resque to do some (long time) job. And I have a few classes with the same mixed-in module for queuing. Class Service substitutes in tests, that's why it standalone and (maybe too much) complicated. So the story is when I call
Campaign.perform(user_id)
directly, everything works fine, but when I try to use queue:
Resque.enqueue(Campaign, user_id)
Job created, but seems like do nothing. At least, nothing saves into the database. Which is main task of Campaign class. I can see in resque-web-interface that jobs creates and finished, and finished (to fast, almost just after create) but no result.
I'm new in Resque and not really sure it calls it all (confused how to debug it).
Does anybody have similar problem? thanks for any help.
Module:
module Synchronisable
def self.included(base)
base.extend ClassMethods
end
module ClassMethods
def perform(user_id)
save_objects("#{self.name}::Service".constantize.get_objects(user_id))
end
protected
def save_objects(objects)
raise ArgumentError "should be implemented"
end
end
class Service
def self.get_objects(user)
raise ArgumentError "should be implemented"
end
end
end
One of the classes:
class Campaign < ActiveRecord::Base
include Synchronisable
#queue = :app
class << self
protected
def save_objects(objects)
#some stuff to save objects
end
end
class Service
def self.get_objects(user_id)
#some stuff to get objects
end
end
end
This is a very old question so not sure how rails folder structure was back then but I had the same problem and issue was with inheritance. Seems if you are using Resque your job classes shouldn't inherit from ApplicationJob.
so if your code was like this in (app/jobs/campaign_job.rb):
class Campaign < ApplicationJob
#queue = :a_job_queue
def self.perform
#some background job
end
end
then remove the inheritance i.e "< ApplicationJob"
These jobs are almost certainly failing, due to an Exception. What is resque-web showing you on the Failures tab? You can also get this from the Rails console with:
Resque.info
or
Resque::Failure.all(0)
You should run your worker like this:
nohup QUEUE=* rake resque:work & &> log/resque_worker_QUEUE.log
This will output everything you debug to "log/resque_worker_QUEUE.log" and you will be able to find out what's wrong with your Campaign class.
Try this:
env TERM_CHILD=1 COUNT=2 "QUEUE=*" bundle exec rake resque:workers
I am trying to write a rspec for delayed job.
Currently I use delayed job as follows:
class IncomingMailsController < ApplicationController
...
MailingJob.new(#incoming_mail.id).perform
...
end
Then in /lib/mailing_job.rb:
class MailingJob < Struct.new(:mailing_id)
def perform
.......
How can I test that with rspec? Right now I have:
/spec/lib/mailing_job/mailingjob_spec.rb
require 'spec_helper'
describe MailingJob do
include DelayedJobSpecHelper
it "should have been worked on if I do something that queues jobs" do
#incoming_mail = IncomingMail.create(.........)
MailingJob.new(#incoming_mail.id).perform
#IncomingMail.method_that_queues_jobs
work_off
MailingJob.should be_worked_on
end
end
/spec/lib/delayed_job_spec_helper.rb
module DelayedJobSpecHelper
# http://erikonrails.snowedin.net/?p=230
def work_off
Delayed::Job.all.each do |job|
job.payload_object.perform
job.destroy
end
end
end
But this errors with:
1) MailingJob should have been worked on if I do something that queues jobs
Failure/Error: MailingJob.should be_worked_on
NoMethodError:
undefined method `worked_on?' for MailingJob:Class
# ./spec/lib/mailing_job/mailingjob_spec.rb:19
Ideas? Thanks
You're queueing your job incorrectly in the controller. perform is called by the worker, not by you.
Delayed::Job.enqueue MailingJob.new(#incoming_mail.id)
Similarly, don't call perform in your spec, that's what work_off does.
The structure of the spec should be this:
Access controller method that queues the job
Tell DelayedJob to work off the queue
Check to see that it did what it was supposed to do
I'm not sure where be_worked_on comes in, as worked_on? is not defined anywhere in DelayedJob (and indeed, is the cause of your error). I would instead check something was done, like a mail was sent, or whatever your job is supposed to do.