I'm trying to send options to a sidekiq worker. Sidekiq is using Activejob
class User
def do_background_task(object, options={})
MyJob.perform_later(id, object.id, options )
end
end
class MyJob < ActiveJob::Base
queue_as :default
def perform(user_id,object_id,options={})
user = User.find(user_id)
object = Object.find(user_id)
selector = options[:selector] if options[:type]
do some things....
if selector == 'true'
do some other things.....
end
end
end
This is not working, and seems to be because either ActiveJob or Sidekiq does not like to receive a keyed hash. So e.g., #user.do_background_task(#object, selector: true) causes an error NoMethodError: undefined method '[]' for nil:NilClass.
What is the accepted way to pass keys to a queue?
Related
I have an ActiveJob adapter I wrote that looks like this:
require 'aws-sdk-core'
module ActiveJob
module QueueAdapters
class SqsAdapter
class << self
def enqueue(job)
sqs = Aws::SQS::Client.new(region: 'us-east-1')
sqs.send_message(
queue_url: ENV['SQS_QUEUE_URL'],
message_body: MultiJson.dump(job.serialize)
)
end
end
class JobWrapper
class << self
def perform(job_data)
ActiveJob::Base.execute job_data
end
end
end
end
end
end
When I run a mailer job through it (e.g. UserMailer.registration_email(user).deliver_now), my log output looks like this:
Processing job e0f6afac-7b95-40c7-89d0-d9c63b808198 wrong number of
arguments (1 for 3+) {"job_class"=>"ActionMailer::DeliveryJob",
"job_id"=>"e0f6afac-7b95-40c7-89d0-d9c63b808198",
"queue_name"=>"mailers", "arguments"=>["UserMailer",
"registration_email", "deliver_now",
{"_aj_globalid"=>"gid://my-cms/RetailUser/863"}], "locale"=>"en"}
How come my adapter doesn't like mailer jobs, and how can I fix it?
I'm using ActiveJob with delayed_job (4.0.6) in the background and I want to find a scheduled job to deleted it.
For instance, if I have
class MyClass
def my_method
perform_stuff
MyJob.set(wait: 1.month.from_now).perform_later(current_user)
end
end
Then, if I edit MyClass instance and call my_method again, I want to cancel that job and schedule a new one.
As suggested in this post http://www.sitepoint.com/delayed-jobs-best-practices, I added two columns to the Delayed Job Table:
table.integer :delayed_reference_id
table.string :delayed_reference_type
add_index :delayed_jobs, [:delayed_reference_id], :name => 'delayed_jobs_delayed_reference_id'
add_index :delayed_jobs, [:delayed_reference_type], :name => 'delayed_jobs_delayed_reference_type'
So this way I may find a delayed Job and destroy it. But I wanted to do that inside a ActiveJob class, to maintain the pattern of jobs in my project.
I wanted to do something like:
class MyJob < ActiveJob::Base
after_enqueue do |job|
user = self.arguments.first
job.delayed_reference_id = user.id,
job.delayed_reference_type = "User"
end
def perform(user)
delete_previous_job_if_exists(user_id)
end
def delete_previous_job_if_exists(user_id)
Delayed::Job.find_by(delayed_reference_id: 1, delayed_reference_type: 'User').delete
end
end
But that doesn't work.
Anyone had this kind of issue?
Two changes:
1. updated the after_enqueue callback so that you can update the
delayed_jobs table directly
2. fixed a typo where delayed_reference_id was hard coded as 1
This should work:
class MyJob < ActiveJob::Base
after_enqueue do |job|
user = self.arguments.first
delayed_job = Delayed::Job.find(job.provider_job_id)
delayed_job.update(delayed_reference_id:user.id,delayed_reference_type:'User')
end
def perform(user)
delete_previous_job_if_exists(user.id)
end
def delete_previous_job_if_exists(user_id)
Delayed::Job.find_by(delayed_reference_id: user_id, delayed_reference_type: 'User').delete
end
end
If you want to access the Delayed::Job from your worker, in an initializer, you can monkey patch the JobWrapper class.
module ActiveJob
module QueueAdapters
class DelayedJobAdapter
class JobWrapper
def perform(job)
end
end
end
end
end
Any idea how to get the Delayed::Job id from the ActiveJob enqueuing? When I enqueue a job I get back an instance of ActiveJob::Base with a #job_id, but that job id seems to be internal to ActiveJob. My best guess so far is just to walk down the most recently created jobs:
active_job_id = GenerateReportJob.perform_later(self.id).job_id
delayed_job = Delayed::Job.order(id: :desc).limit(5).detect do |job|
YAML.load(job.handler).job_data['job_id'] == active_job_id
end
but that seems all kinds of hacky. Kind of surprised ActiveJob isn't returning the ID from Delayed::Job, especially since that is what is explicitly returned when the job gets enqueued.
== EDIT
Looks like I'm not the only one (https://github.com/rails/rails/issues/18821)
In case anyone finds this in the future: Rails just accepted a patch to allow you to get this id from provider_job_id in Rails 5. You can get it to work with a patch like
ActiveJob::QueueAdapters::DelayedJobAdapter.singleton_class.prepend(Module.new do
def enqueue(job)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
def enqueue_at(job, timestamp)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
end)
Inspired by the answer of Beguene and some reverse engineering of the Rails 5 ActiveJob code, I have made it work with Rails 4.2 by
1) adding following code in lib/active_job/queue_adapters/delayed_job_adapter.rb or config/initializers/delayed_job.rb (both locations have worked):
# file: lib/active_job/queue_adapters/delayed_job_adapter.rb
module ActiveJob
module Core
# ID optionally provided by adapter
attr_accessor :provider_job_id
end
module QueueAdapters
class DelayedJobAdapter
class << self
def enqueue(job) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name)
job.provider_job_id = delayed_job.id
delayed_job
end
def enqueue_at(job, timestamp) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name, run_at: Time.at(timestamp))
job.provider_job_id = delayed_job.id
delayed_job
end
end
class JobWrapper #:nodoc:
attr_accessor :job_data
def initialize(job_data)
#job_data = job_data
end
def perform
Base.execute(job_data)
end
end
end
end
end
The attr_accessor :provider_job_id statement is needed in Rails 4.2, since it is used in the enqueue method and is not yet defined in 4.2.
Then we can make use of it like follows:
2) define our own ActiveJob class:
# file: app/jobs/my_job.rb
class MyJob < ActiveJob::Base
queue_as :default
def perform(object, performmethod = method(:method))
# Do something later
returnvalue = object.send(performmethod)
returnvalue
end
end
end
3) Now we can create a new job anywhere in the code:
job = MyJob.perform_later(Myobject, "mymethod")
This will put the method Myobject.mymethod into the queue.
4) The code in 1) helps us to find the Delayed Job that is associated with our job:
delayed_job = Delayed::Job.find(job.provider_job_id)
5) finally, we can do, whatever we need to do with the delayed_job, e.g. delete it:
delayed_job.delete
Note: in Rails 5, step 1) will not be needed anymore, since the exact same code is integral part of Rails 5.
I made it work in Rails 4.2 using the new patch from Rails 5 like this:
create the file lib/active_job/queue_adapters/delayed_job_adapter.rb
module ActiveJob
module QueueAdapters
class DelayedJobAdapter
class << self
def enqueue(job) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name)
job.provider_job_id = delayed_job.id
delayed_job
end
def enqueue_at(job, timestamp) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name, run_at: Time.at(timestamp))
job.provider_job_id = delayed_job.id
delayed_job
end
end
class JobWrapper #:nodoc:
attr_accessor :job_data
def initialize(job_data)
#job_data = job_data
end
def perform
Base.execute(job_data)
end
end
end
end
end
Instead of removing the job from the queue if it is cancelled you could model a cancellation of the job itself.
Then, when you come to run the GenerateReportJob you can first check for a cancellation of the report. If there is one then you can destroy the cancellation record and drop out of the report generation. If there is no cancellation then you can carry on as normal.
I trying to use sidekiq in my app, but when I write this simple worker
class ParseWorker
include Sidekiq::Worker
def perform(instance)
instance.spideys << Spidey.create
end
end
and I use this worker here
def create
#user_link = UserLink.new(user_link_params)
if #user_link.save
binding.pry
ParseWorker.perform_async(#user_link)
redirect_to results_user_links_path
end
end
was returned the error
2015-04-09T13:14:56.757Z 11644 TID-ay1nc ERROR: Actor crashed!
NoMethodError: undefined method `spideys' for "#<UserLink:0x007f65f00d99b0>":String
/home/weare138/simple-parser/app/workers/parse_worker.rb:7:in `perform'
but why? #user_link is not a string
how fix?
upd
def perform(id)
user_link = UserLink.find(id)
user_link.spideys << Spidey.create
end
error
2015-04-09T14:15:21.889Z 11644 TID-ay1nc ERROR: Actor crashed!
NoMethodError: undefined method `spideys' for 39:Fixnum
upd2
class ParseWorker
include Sidekiq::Worker
require 'open-uri'
def perform(id)
user_link = UserLink.find(id)
user_link.spideys << Spidey.create
end
end
Sidekiq uses Redis, so when you pass to the worker an object, it serializes it to JSON.
From the Sidekiq documentation:
This means the arguments to your worker must be simple JSON datatypes
(numbers, strings, boolean, array, hash). Complex Ruby objects (e.g.
Date, Time, ActiveRecord instances) will not serialize properly.
Instead you should do
def create
...
ParseWorker.perform_async(#user_link.id)
end
def perform(id)
UserLink.find(id).spideys << Spidey.create
end
Here is what I'm calling:
UpdateRatingAndCountWorker.perform_async(133)
Here is my worker:
# app/workers/update_rating_and_count_worker.rb
class UpdateRatingAndCountWorker
include Sidekiq::Worker
def perform(review_id)
review = Review.find(review_id.to_i)
review.style.update_average_rating!
end
end
Here is the error:
"NoMethodError: undefined method `style' for \"#<Review:0x00000005bef438>\":String"
In the error message it looks like the variable review.style is of the type String.
Since you haven't posted the code for the model Review I can only guess but, should it be
# app/workers/update_rating_and_count_worker.rb
class UpdateRatingAndCountWorker
include Sidekiq::Worker
def perform(review_id)
review = Review.find(review_id.to_i)
review.update_average_rating!
end
end
Restart Sidekiq works for me, it picks up new method in my model