How to resolve deserialization error in delayed job? - ruby-on-rails

Question

I am trying to use DelayedJob and the job is failing, giving the following error in the database:
{Delayed::DeserializationError
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/serialization/active_record.rb:7:in `yaml_new'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/yaml.rb:133:in `transfer'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/yaml.rb:133:in `node_import'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/yaml.rb:133:in `load'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/yaml.rb:133:in `load'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/backend/base.rb:79:in `payload_object'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/backend/base.rb:87:in `invoke_job_without_newrelic_transaction_trace'
(eval):3:in `invoke_job'
/Library/Ruby/Gems/1.8/gems/newrelic_rpm-2.13.4/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:252:in `perform_action_with_newrelic_trace'
/Library/Ruby/Gems/1.8/gems/newrelic_rpm-2.13.4/lib/new_relic/agent/method_tracer.rb:141:in `trace_execution_scoped'
/Library/Ruby/Gems/1.8/gems/newrelic_rpm-2.13.4/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:247:in `perform_action_with_newrelic_trace'
(eval):2:in `invoke_job'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:120:in `run'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:120:in `run'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:119:in `run'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:177:in `reserve_and_run_one_job'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:104:in `work_off'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:103:in `times'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:103:in `work_off'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:78:in `start'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:77:in `start'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:74:in `loop'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/worker.rb:74:in `start'
/Library/Ruby/Gems/1.8/gems/delayed_job-2.1.3/lib/delayed/tasks.rb:9
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'
/Library/Ruby/Gems/1.8/gems/rake-0.8.7/bin/rake:31
/usr/bin/rake:19:in `load'
/usr/bin/rake:19
Not sure where to start in diagnosing this. This has never happened before and I have used delayed job before to serialize model objects without any issues. Why this time?
Thanks in advance!

Answer 1

It's not really a deserialization error, it's an ActiveRecord record-not-found error on a simple Model.find(id) query.
If you want to know the details, log them in the delayed_job-2.1.3/lib/delayed/serialization/active_record.rb file, in the rescue statement, just before delayed-job stupidly raises the DeserializationError and throws the useful information away.

Answer 2

Michiel is right. Look at your handler field for objects like "!ruby/ActiveRecord:YourClassName"
Then check if objects can be retrieved via the primary key
From the console you can also test this out by doing something like:
# first job in your delayed queue
YAML.load(Delayed::Backend::ActiveRecord::Job.first.handler)

Answer 3

I believe this occurs when you run a job against an unsaved or deleted AR object since the deserialization for AR loads the record by id. An exception should probably be thrown if you attempt to delay a method on an unsaved AR object.

Answer 4

There's also a documented bug with DJ when the params passed into the handler field in the DB are longer than a standard TEXT column:
https://github.com/collectiveidea/delayed_job/issues/491
If this happens to be your problem, changing the column to a MEDIUMINT should fix the issue.
I did this in a migration like so:
change_column :delayed_jobs, :handler, :text, :limit => 16777215
ActiveRecord::Base.connection.execute("DELETE FROM delayed_jobs WHERE LENGTH(handler) >= 65535")
You can check to see if it's an issue with a simple DB query:
SELECT * FROM delayed_jobs WHERE LENGTH(handler) >= 65535

Answer 5

If anyone wants to make delayed_job just finish the job as a no-op you can monkey patch with this code in the initializer:
https://gist.github.com/spilliton/8494752

Answer 6

Today, I also suffered through this error and after doing hectic analysis found:
delayed_job converts methods & parameters to YAML format and stores it into database
It can be found using select * from delayed_jobs;
deserialization error occurs when delayed_job is not able to deserialize it.
Probable causes can be:
args["xyz"] used before calling delayed_job & inside worker using it as args[:xyz]
Sometimes extra arguments passed along with object to delayed_job that time delayed_job fails to build object as it is indifferent access.
I Hope this will help!

Answer 7

Sometimes when we upgrade libs delayed jobs still keep old references.
Try to find the id of delayed_job in logs and play to parse its handler to ruby to find the wrong reference
j = DelayedJob.find(XXX)
data = YAML.load_dj(j.handler)
data.to_ruby
I made a pull request to help with this problem.
Meanwhile you can use this lines
# config/initializers/delayed_job.rb
# Monkey patch to use old class references
module Psych
class << self; attr_accessor :old_class_references end
#old_class_references = {}
class ClassLoader
private
def find klassname
klassname = ::Psych.old_class_references[klassname] || klassname
#cache[klassname] ||= resolve(klassname)
end
end
module Visitors
class ToRuby < Psych::Visitors::Visitor
def revive klass, node
if klass.is_a? String
klassname = ::Psych.old_class_references[klass] || klass
klass = Kernel.const_get(klassname) rescue klassname
end
s = register(node, klass.allocate)
init_with(s, revive_hash({}, node), node)
end
end
end
end
# Add all old dependencies (hash keys) pointing to new references (hash values)
Psych.old_class_references = {
'ActiveRecord::AttributeSet' => 'ActiveModel::AttributeSet'
# ...
}