I'm trying to make my delayed jobs hooks work, but they don't seem to be. Are they outdated? If they are not, can you show me an example of yours?
Here's mine:
class PhotoJob < Struct.new(:image_id)
def perform
Photo.find(self.image_id).regenerate_styles!
end
def error(job, exception)
if Photo.exists?(self.image_id)
Photo.find(self.image_id).regenerate_styles!
else
Delayed::Job.find(self).destroy
end
end
end
The reason I say they do not work is because if I upload a hundred images, half of them will fail with an error. If the error is raised, then the hook should be run, right?
Here's the catch, if I find the photo that's failing, and run Photo.find(n).regenerate_styles! , then the photo regenerates appropriately and works.
So I'm guessing that Delayed Jobs' hooks are not working.
They've been working ok for me:
class BaseProcessorJob
def perform
raise NotImplementedError
end
def error(job, exception)
Rails.logger.error "Job failed #{exception}"
end
end
I originally had a similar problem with hooks not getting called because of an outdated delayed_job gem (it was actually installed as a plugin, which caused my problems). Not sure if any of that helps you though.
Related
I need for testing purpose to reproduce an ActiveJob::DeserializationError on rails console, I need to test the behaviour manually on testing environment.
class WrapperJob < ApplicationJob
discard_on ActiveJob::DeserializationError
def perform(event)
# doing something
end
end
The WrapperJob aims to perform something on the event, but during the enqueue time it might happen to remove event from database. I need a way please to reproduce ActiveJob::DeserializationError. right now I tried to pass a missing event, or a missing GlobalID but actually fail on the # doing something part instead of raising DeserializationError.
Thanks.
I have some methods that works with API of third party app. To do it on button click is no problem, but it should be permanent process.
How to run them background? And how to pause the cycle for make some other works with same API and resume the cycle after the job is done.
Now I read about ActiveJob, but its has time dependences only...
UPDATE
I've tried to make it with whenever and sidekiq, task runs, but it do nothing. Where to look for logs I can't understand.
**schedule.rb**
every 1.minute do
runner "UpdateWorker.perform_async"
end
**update_worker.rb**
class UpdateWorker
include Sidekiq::Worker
include CommonMods
def perform
logger.info "Things are happening."
logger.debug "Here's some info: #{hash.inspect}"
myMethod
end
def myMethod
....
....
....
end
end
It's not exactly what I need, but better then nothing. Can somebody explain me with examples?
UPDATE 2 After manipulating with code it's absolutely necessary to restart sidekiq . With this problem is solved, but I'm not sure that this is the best way.
You can define a job which enqueues itself:
class MyJob < ActiveJob::Base
def perform(*args)
# Do something unless some flag is raised
ensure
self.class.set(wait: 1.hour).perform_later(*args)
end
end
There are several libraries to schedule jobs on a regular basis. For example you could use to sidekiq-cron to run a job every minute.
If you want to pause it for some time, you could set a flag somewhere (Redis/database/file) and skip execution as long it is detected.
On a somewhat related note: don't use sidetiq. It was really great but it's not maintained anymore and has incompatibilities to current Sidekiq versions.
Just enqueue next execution in ensure section after job completes after checking some flag that indicates that it should.
Also i recommend adding some delay there so that you don't end up with dead loop on some error inside job
I dont know ActiveJobs, but I can recommend the whenever gem to create cron (periodic background) jobs. Basically you end up writing a rake tasks. Like this:
desc 'send digest email'
task send_digest_email: :environment do
# ... set options if any
UserMailer.digest_email_update(options).deliver!
end
I never added a rake task to itself but for repeated processing you could do somehow like this (from answers to this specific question)
Rake::Task["send_digest_email"].execute
I'm having big issues trying to get delayed_job working with Amazon S3 and Paperclip. There are a few posts around about how to do it, but for whatever reason it's simply not working for me. I've removed a couple of things to how others are doing it - originally I had a save(validations => false) in regenerate_styles, but that seemed to cause an infinite loop (due to the after save catch), and didn't seem to be necessary (since the URLs have been saved, just the images not uploaded). Here's the relevant code from my model file, submission.rb:
class Submission < ActiveRecord::Base
has_attached_file :photo ...
...
before_photo_post_process do |submission|
if photo_changed?
false
end
end
after_save do |submission|
if submission.photo_changed?
Delayed::Job.enqueue ImageJob.new(submission.id)
end
end
def regenerate_styles!
puts "Processing photo"
self.photo.reprocess!
end
def photo_changed?
self.photo_file_size_changed? ||
self.photo_file_name_changed? ||
self.photo_content_type_changed? ||
self.photo_updated_at_changed?
end
end
And my little ImageJob class that sites at the bottom of the submission.rb file:
class ImageJob < Struct.new(:submission_id)
def perform
Submission.find(self.submission_id).regenerate_styles!
end
end
As far as I can tell, the job itself gets created correctly (as I'm able to pull it out of the database via a query).
The problem arises when:
$ rake jobs:work
WARNING: Nokogiri was built against LibXML version 2.7.8, but has dynamically loaded 2.7.3
[Worker(host:Jarrod-Robins-MacBook.local pid:21738)] New Relic Ruby Agent Monitoring DJ worker host:MacBook.local pid:21738
[Worker(host:MacBook.local pid:21738)] Starting job worker
Processing photo
[Worker(host:MacBook.local pid:21738)] ImageJob completed after 9.5223
[Worker(host:MacBook.local pid:21738)] 1 jobs processed at 0.1045 j/s, 0 failed ...
The rake task then gets stuck and never exits, and the images themselves don't appear to have been reprocessed.
Any ideas?
EDIT: just another point; the same thing happens on heroku, not just locally.
Delayed job is capturing a stack trace for all failed jobs. It’s saved in the last_error column of the delayed_jobs table. Use a database gui too see whats going on.
If you should be using Collective Ideas fork with ActiveRecord as backend you can query the model as usual. To fetch an array of all stack traces for example do
Delayed::Job.where('failed_at IS NOT NULL').map(&:last_error)
By default failed jobs are deleted after 25 failed attempts. It may be that there are no jobs anymore. Prevent deletion for debugging purposes by setting
Delayed::Worker.destroy_failed_jobs = false
in your config/initializers/delayed_job_config.rb
I am running a delayed job worker. When ever I invoke the foo method, worker prints hello.
class User
def foo
puts "Hello"
end
handle_asynchronously :foo
end
If I make some changes to the foo method, I have to restart the worker for the changes to reflect. In the development mode this can become quite tiresome.
I am trying to find a way to reload the payload class(in this case User class) for every request. I tried monkey patching the DelayedJob library to invoke require_dependency before the payload method invocation.
module Delayed::Backend::Base
def payload_object_with_reload
if Rails.env.development? and #payload_object_with_reload.nil?
require_dependency(File.join(Rails.root, "app", "models", "user.rb"))
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
This approach doesn't work as the classes registered using require_dependency needs to be reloaded before the invocation and I haven't figured out how to do it. I spent some time reading the dispatcher code to figure out how Rails reloads the classes for every request. I wasn't able to locate the reload code.
Has anybody tried this before? How would you advise me to proceed? Or do you have any pointers for locating the Rails class reload code?
I managed to find a solution. I used ActiveSupport::Dependencies.clear method to clear the loaded classes.
Add a file called config/initializers/delayed_job.rb
Delayed::Worker.backend = :active_record
if Rails.env.development?
module Delayed::Backend::Base
def payload_object_with_reload
if #payload_object_with_reload.nil?
ActiveSupport::Dependencies.clear
end
#payload_object_with_reload ||= payload_object_without_reload
end
alias_method_chain :payload_object, :reload
end
end
As of version 4.0.6, DelayedJob reloads automatically if Rails.application.config.cache_classes is set to false:
In development mode, if you are using Rails 3.1+, your application code will automatically reload every 100 jobs or when the queue finishes. You no longer need to restart Delayed Job every time you update your code in development.
This looks like it solves your problem without the alias_method hackery:
https://github.com/Viximo/delayed_job-rails_reloader
It seems like due to the threading issue logger.warn (thats what I tested) doesn't generate any output? my code is similar to this:
def deliver(args)
logger.info "delivery start"
thread_pool.defer(:deliver_deferred, args)
logger.info "delivery end"
end
def deliver_deferred(args)
logger.warn "whatsoever"
end
Any idea?
I'm not sure about the specifics, but assuming BackgrounDRb runs as a forked process, any open files would be closed for the forked child process. This would likely manifest itself as what you're seeing.
However, I would've assumed that BackgrounDRb would've been smart enough to handle that, since it ought to be a pretty obvious issue.