Can't figure out Delayed::DeserializationError - ruby-on-rails

I am running delayed_job 3.0.5 (delayed_job_active_record 0.4.1) with Rails 3.2.12. I am having problems with some of my jobs failing because of "Deserialization". Here is a really simple example of one of the failing handlers:
--- !ruby/struct:Delayed::PerformableMethod
object: LOAD;Project;924951
method: :send_project_open_close_without_delay
args: []
When I try to invoke this job:
Delayed::DeserializationError: Job failed to load: undefined method `members' for nil:NilClass.
Everyone seems to think this is caused by an AR object that no longer exists. In my case, I can run the handler just fine:
Project.find(924951).send_open_close_without_delay
What else could be causing the error?

I think I also experienced this issue when we upgrade to rails 3.2. The error I got was caused by the yaml handler used by delayed job. Try adding the following to config/boot.rb
require 'rubygems'
require 'yaml'
YAML::ENGINE.yamler = 'syck'

Just in case anyone ever runs into this issue. I figured out the cause was some of the jobs were injected with an older version of Delayed Job. So when the newer Delayed Job attempted to process them, it was unable to deserialize the handler.

This was caused by an upgrade to Delayed::Job that changed the serialization handler AND changed the type of object used for PerformableMethod. If you want to fix all old jobs and stick with the new defaults for Delayed::Job, you can fix failing ones with this query:
Delayed::Job.where('failed_at is not null').each do |dj|
dj.handler = dj.handler.gsub('struct:Delayed', 'object:Delayed')
Delayed::Worker.new.run(dj)
end
The change occurred around here: https://github.com/collectiveidea/delayed_job/commit/7b8a79a72c0ee5d8bac4bc0b183d1cce9cedff85
Making PerformableMethod a class instead of a Struct.

Related

Upgrading to Rails 4.2 breaks Delayed::Job: Job failed to load: undefined method 'fetch_value'

Upgrading to Rails 4.2, everything working fine locally, push to production 'kaboom'. Every Delayed::Job failed with the same error.
Job failed to load: undefined method 'fetch_value' for #. Handler: " --- YAMLYAMLYAMLYAML "
The error message we're getting is coming from the function below, which is calling into different libraries, catching the actual error, and returning their own which makes it really difficult to debug.
def payload_object
#payload_object ||= YAML.load_dj(handler)
rescue TypeError, LoadError, NameError, ArgumentError, SyntaxError, Psych::SyntaxError => e
raise DeserializationError, "Job failed to load: #{e.message}. Handler: #{handler.inspect}"
end
Other things that might be useful, we're only using Delayed::Job (not with Active::Job).
fetch_value is never called within delayed_job, it's an Active::Record method.
I came across a similar fetch_value error when I upgraded an app to Rails 4.2.1. To get a closer understanding of the error, grab any job with this error and run the following in the rails console:
YAML.load_dj(Delayed::Job.find(JOB_ID).handler)
It runs internal delayed job code, but should give you a better trace for debugging.
source: https://github.com/collectiveidea/delayed_job/blob/master/lib/delayed/backend/base.rb#L90
The error is likely the result of queueing/creating jobs pre-Rails 4.2 upgrade and before setting config.active_job.queue_adapter = :delayed_job in your application.rb. If the config was not properly set, then jobs would be serialized in a format that Rails 4.2 cannot handle. To run any jobs with the fetch_value error, you can switch to a branch of your code still on > Rails 4.2 and run the jobs from the rails console.
You can try to update delayed_job to newer version, there was a series of changes for rails 4/4.2/4.4 support, and see if it brakes things in your app.
And for rails 4.2 they expect you set config.active_job.queue_adapter = :delayed_job in config/application.rb

Rails Rspec warning: "This dynamic method is deprecated."

My Rails 4/Ruby 2 app is throwing the following warning every time my RSpec tests create a FactoryGirl object: "DEPRECATION WARNING: This dynamic method is deprecated. Please use e.g. Post.find_or_create_by(name: 'foo') instead."
This warning is not thrown when I run my app in development. Is FactoryGirl's code throwing this? I tried to find some information but it doesn't look like other people are getting this.
If you tell Rails to give you a full stack trace for the deprecation warning, you should be able to diagnose it pretty easily. The warnings are coming from a library called ActiveSupport::Deprecation - tell it to run in debug mode.
# config/environments/test.rb
ActiveSupport::Deprecation.debug = true
For me, the warnings were caused by an old version of the Stringex library.
FactoryGirl would make a new model, which would trigger a call to one of the Stringex methods, which would raise the warning, although there was no way to see that until I turned on full stack traces. bundle update stringex solved the issue with no problem.
It looks like it's coming from ActiveRecord.
module DeprecationWarning
def body
"#{deprecation_warning}\n#{super}"
end
def deprecation_warning
%{ActiveSupport::Deprecation.warn("This dynamic method is deprecated. Please use e.g. #{deprecation_alternative} instead.")}
end
end
I'm not sure why you're not getting the warnings in development. Is your environment suppressing the warnings?

Delayed_job won't make any changes to database

I am having many problems running DJ. Primarily, I cannot get delayed_job running any methods that change the database. I am testing locally with a sqlite3 database, DJ 3.0.0, and I even added the delayed_job_active_record gem.
I have, for example, tried to run the following method in the background:
#user = User.find(1)
#user.delay.recorder_method
Where this method is:
def recorder_method
self.relevant_field +=1
update
end
This creates a delayed job, and the handler has the appropriate info for the user and the appropriate method name. The script runs the job, thinks it has succeeded and thus deletes the record from the delayed_job table. BUT the user is unchanged (the database is unchanged).
What on earth is going wrong? Note that, when I run the same code ("#user.delay.recorder_method") from rails console, it works.. and the difference is the handler created by the rails console call is:
--- !ruby/object:Delayed::PerformableMethod
object: !ruby/ActiveRecord:User
attributes:
... (attribute info and rest of file)
Whereas the one created in by the call in a controller action is:
--- !ruby/struct:Delayed::PerformableMethod
object: !ruby/ActiveRecord:User
attributes:
Not the difference in line 1 of both things (ruby/object vs ruby/struct.. the former works and the latter doesn't). Maybe this is something that might signal what is going wrong. Any ideas, anybody?
Your Rails app is running an old version of the DJ gem and your console is running a new one. About six months ago, Delayed::PerformableMethod was refactored from a Struct into a regular class: https://github.com/collectiveidea/delayed_job/commit/7b8a79a72c0ee5d8bac4bc0b183d1cce9cedff85 (So your Rails app is running a gem at least six months old and your console is running one newer than that).
This is an easy fix. First update the appropriate line in your Rails Gemfile. You'll see a line like this:
gem 'delayed_job_active_record'
If it has a version specification, make sure it's updated to the newest. Then from the command line (in the Rails root), run:
bundle update delayed_job_active_record
For anyone interested, removing the following lines from boot.rb solved this issue for me:
require 'yaml'
YAML::ENGINE.yamler = 'syck'

in delayed_job, are hooks called when Delayed::Worker.delay_jobs is false?

I'm using collectiveidea/delayed_job. In my RSpec tests, the [:before, :after, :success] hooks aren't getting called as I would expect.
When Delayed::Worker.delay_jobs = true (the default), I see the enqueue() hook getting called and nothing more. This is the behavior I expect, since there's no separate task processing the jobs.
But when Delayed::Worker.delay_jobs = false, as recommended for testing by the documents, I see my process() method getting called, but none of of the hooks.
If this is not the expected behavior, any suggestions on what I'm doing wrong? (I can easily include code.) If this is the expected behavior, then what's a strategy for testing the hooks?
[Side note: The spec directory for delayed_job, notably the performable_method_spec tests, suggest that you can set Delayed::Worker.delay_jobs = false and still get callbacks to your hooks. But those tests are using the obj.delay.method construct rather than Delayed::Job.enqueue(object_with_a_perform_method) to enqueue the job. Would this make a difference?]
[Update: I've tried the obj.delay.method form as well as the Delayed::Job.enqueue(obj_with_a_perform_method) form -- I don't see the hooks getting called in either case.]
On the collectiveidea/delayed_job github page, I found this very same bug described, fixed, and pulled. Presumably an updated version of delayed_job will fix the problem.
Update: I've found a workaround other than pulling the latest version. You can explicitly call the Delayed::Job worker method. It will process items in the queue -- in the same thread as the tests of course -- but the callback hooks do get called:
[successes, failures] = Delayed::Worker.new.work_off

Rails 3: Delayed jobs, method 'delay' not known

I have delayed_job installed as a Gem and in my Gemfile. However, when I try to use delay in my controller as:
def send_warn_admin_email
UserMailer.delay.warn_admin_email(self).deliver
end
I get: undefined methoddelay' for UserMailer:Class`
What can cause this problem?
Thanks
From Documentation.
Due to how mailers are implemented in Rails 3, we had to do a little work
around to get delayed_job to work.
# without delayed_job
Notifier.signup(#user).deliver
# with delayed_job
Notifier.delay.signup(#user)
Remove the #.deliver# method to make it work. It's not ideal, but it's
the best we could do for now.
Bye

Resources