I want to use delayed_job to execute a function from controller. The function is stored in module lib/site_request.rb:
module SiteRequest
def get_data(query)
...
end
handle_asynchronously :get_data
end
query_controller.rb:
class QueryController < ApplicationController
include SiteRequest
def index
#query = Query.find_or_initialize_by_word(params[:query])
if #query.new_record?
#query.save
get_data(#query)
flash[:notice] = "Request for data is sent to server."
end
end
end
I also tried to remove handle_asynchronously clause from module and use delay.get_data(#query), both do not executed silently (without delayed_job code works)
I had trouble trying to use the built-in delay methods myself, too. The pattern I settled on in my own coding was to enqueue DelayedJobs myself, giving them a payload object from which to work. This should work for you too and would seem to make sense even. (This way, you may not even need your SiteRequest module, for example.)
class MyModuleName < Struct.new(:query)
def perform
# TODO
end
end
Then, instead of calling get_data(query) after saving, enqueue with:
Delayed::Job.enqueue(MyModuleName.new(query))
I found the same issue. My environment is:
Ruby 2.1.7
Rails 4.2.6
activejob (4.2.6)
delayed_job (4.1.2)
delayed_job_active_record (4.1.1)
MY solutions:
Turn the module into a class.
Instantiate a object from the class and apply the method to the instance.
It seems that ActiveJob can enqueue only instances.
In your case:
Class SiteRequest
def initialize
end
def get_data(query)
...
end
handle_asynchronously :get_data
end
def index
...
q= SiteRequest.new
q.get_data(#query)
flash[:notice] = "Request for data is sent to server."
end
end
Related
I am using the most basic version of delayed_job in a Rails app. I have the max time allowed for a delayed_job set at 10 minutes. I would like to get the hooks/callbacks working so I can do something after a job stop executing at the 10 minute mark.
I have this set in my rails app:
config.active_job.queue_adapter = :delayed_job
This is how I normally queue a job:
object.delay.object_action
The hook/callback example is for a named job but the basic, getting started steps are not for a named job. So I don't think I have a named job. Here is the example given to get the callbacks working:
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def before(job)
record_stat 'newsletter_job/start'
end
def after(job)
record_stat 'newsletter_job/after'
end
def success(job)
record_stat 'newsletter_job/success'
end
def error(job, exception)
Airbrake.notify(exception)
end
def failure(job)
page_sysadmin_in_the_middle_of_the_night
end
end
I would love to get the after or error hooks/callbacks to fire.
Where do I put these callbacks in my Rails app to have them fire for the basic delayed_job setup? If I should be using ActiveJob callbacks where do you put those callbacks given delayed_job is being used?
You cannot use object.delay.object_action convenience syntax if you want more advanced features like callbacks. The #delay convenience method will generate a job object that works similar to this:
# something like this is already defined in delayed_job
class MethodCallerJob
def initialize(object, method, *args)
#object = object
#method = method
#args = args
end
def perform
#object.send(#method, *#args)
end
end
# `object.delay.object_action` does the below automatically for you
# instantiates a job with your object and method call
job = MethodCallerJob.new(object, :object_action, [])
Delayed::Job.enqueue(job) # enqueues it for running later
then later, in the job worker, something like the below happens:
job = Delayed::Job.find(job_id) # whatever the id turned out to be
job.invoke_job # does all the things, including calling #perform and run any hooks
job.delete # if it was successful
You have to create what the delayed_job README calls "Custom Jobs", which are just plain POROs that have #perform defined at a minimum. Then you can customize it and add all the extra methods that delayed_job uses for extra features like max_run_time, queue_name, and the ones you want to use: callbacks & hooks.
Sidenote: The above info is for using delayed_job directly. All of the above is possible using ActiveJob as well. You just have to do it the ActiveJob way by reading the documentation & guides on how, just as I've linked you to the delayed_job README, above.
You can create delayed_job hooks/callback by something like this
module Delayed
module Plugins
class TestHooks < Delayed::Plugin
callbacks do |lifecycle|
lifecycle.before(:perform) do |_worker, job|
.....
end
end
end
end
end
And need this plugin to initializer
config/initializers/delayed_job.rb
require_relative 'path_to_test_plugin'
Delayed::Worker.plugins << Delayed::Plugins::TestHooks
Similar to perform there are also hooks for success failure and error.
And similar to 'before' you can also capture the 'after' hooks.
So I'm wanting to parse a table on about 10 websites, so I want to create a new thread for each site. However, I'm not exactly sure how to return the data from this type of request.
Here's one class:
class TestRequest
def initialize
end
def start
urls = ['site1','site2','site3']
existing_data = Data.pluck(:symbol, :page)
data = GetData.pool(size: 10)
urls.each do |url|
data.async.perform_requests(url, existing_data)
end
end
end
and then GetData class looks like this:
require 'celluloid/current'
class GetData
include Celluloid
def perform_requests(url, existing_data)
# perform HTTP request
# parse HTTP response
# return returned data ???
end
end
What I'd ultimately like to do is have an instance variable in TestRequest class and simply add the returned value from GetData into that instance variable from the TestRequest class. After the threads are finished, I want to perform another action using the data in the instance variable.
I tried playing around with attr_reader, but it doesn't seem to play in my favor.
I tried this:
class TestRequest
def initialize
end
def start
#returned_data = []
urls = ['site1','site2','site3']
existing_data = Data.pluck(:symbol, :page)
data = GetData.pool(size: 10)
urls.each do |url|
data.async.perform_requests(url, existing_data)
end
end
attr_reader :returned_data
end
and then
require 'celluloid/current'
class GetData
include Celluloid
def perform_requests(tr, existing_data)
# perform HTTP request
# parse HTTP response
t = TestData.new
t.returned_data << "value"
end
end
but this doesn't work either.
Multi-threading and Ruby on Rails don't mix very well.
However, you should consider using the ActiveJob documentation (http://guides.rubyonrails.org/active_job_basics.html).
With ActiveJob, you can enqueue jobs and have them executed in the background. There are hooks method defined as well to notify you when a job is about to start, is running or have finished.
Any idea how to get the Delayed::Job id from the ActiveJob enqueuing? When I enqueue a job I get back an instance of ActiveJob::Base with a #job_id, but that job id seems to be internal to ActiveJob. My best guess so far is just to walk down the most recently created jobs:
active_job_id = GenerateReportJob.perform_later(self.id).job_id
delayed_job = Delayed::Job.order(id: :desc).limit(5).detect do |job|
YAML.load(job.handler).job_data['job_id'] == active_job_id
end
but that seems all kinds of hacky. Kind of surprised ActiveJob isn't returning the ID from Delayed::Job, especially since that is what is explicitly returned when the job gets enqueued.
== EDIT
Looks like I'm not the only one (https://github.com/rails/rails/issues/18821)
In case anyone finds this in the future: Rails just accepted a patch to allow you to get this id from provider_job_id in Rails 5. You can get it to work with a patch like
ActiveJob::QueueAdapters::DelayedJobAdapter.singleton_class.prepend(Module.new do
def enqueue(job)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
def enqueue_at(job, timestamp)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
end)
Inspired by the answer of Beguene and some reverse engineering of the Rails 5 ActiveJob code, I have made it work with Rails 4.2 by
1) adding following code in lib/active_job/queue_adapters/delayed_job_adapter.rb or config/initializers/delayed_job.rb (both locations have worked):
# file: lib/active_job/queue_adapters/delayed_job_adapter.rb
module ActiveJob
module Core
# ID optionally provided by adapter
attr_accessor :provider_job_id
end
module QueueAdapters
class DelayedJobAdapter
class << self
def enqueue(job) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name)
job.provider_job_id = delayed_job.id
delayed_job
end
def enqueue_at(job, timestamp) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name, run_at: Time.at(timestamp))
job.provider_job_id = delayed_job.id
delayed_job
end
end
class JobWrapper #:nodoc:
attr_accessor :job_data
def initialize(job_data)
#job_data = job_data
end
def perform
Base.execute(job_data)
end
end
end
end
end
The attr_accessor :provider_job_id statement is needed in Rails 4.2, since it is used in the enqueue method and is not yet defined in 4.2.
Then we can make use of it like follows:
2) define our own ActiveJob class:
# file: app/jobs/my_job.rb
class MyJob < ActiveJob::Base
queue_as :default
def perform(object, performmethod = method(:method))
# Do something later
returnvalue = object.send(performmethod)
returnvalue
end
end
end
3) Now we can create a new job anywhere in the code:
job = MyJob.perform_later(Myobject, "mymethod")
This will put the method Myobject.mymethod into the queue.
4) The code in 1) helps us to find the Delayed Job that is associated with our job:
delayed_job = Delayed::Job.find(job.provider_job_id)
5) finally, we can do, whatever we need to do with the delayed_job, e.g. delete it:
delayed_job.delete
Note: in Rails 5, step 1) will not be needed anymore, since the exact same code is integral part of Rails 5.
I made it work in Rails 4.2 using the new patch from Rails 5 like this:
create the file lib/active_job/queue_adapters/delayed_job_adapter.rb
module ActiveJob
module QueueAdapters
class DelayedJobAdapter
class << self
def enqueue(job) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name)
job.provider_job_id = delayed_job.id
delayed_job
end
def enqueue_at(job, timestamp) #:nodoc:
delayed_job = Delayed::Job.enqueue(JobWrapper.new(job.serialize), queue: job.queue_name, run_at: Time.at(timestamp))
job.provider_job_id = delayed_job.id
delayed_job
end
end
class JobWrapper #:nodoc:
attr_accessor :job_data
def initialize(job_data)
#job_data = job_data
end
def perform
Base.execute(job_data)
end
end
end
end
end
Instead of removing the job from the queue if it is cancelled you could model a cancellation of the job itself.
Then, when you come to run the GenerateReportJob you can first check for a cancellation of the report. If there is one then you can destroy the cancellation record and drop out of the report generation. If there is no cancellation then you can carry on as normal.
I'm using delayed_job and I'm very happy with it (especially with the workless extension).
But I'd like to set that ALL mails from my app are sent asynchronously.
Indeed, the solution offered for mailers
# without delayed_job
Notifier.signup(#user).deliver
# with delayed_job
Notifier.delay.signup(#user)
doesn't suit me because:
it is not easily maintainable
mails sent from gems are not sent asynchronously (devise, mailboxer)
I could use this kind of extension https://github.com/mhfs/devise-async but I'd rather figure out a solution for the whole app at once.
Can't I extend ActionMailer to override the .deliver method (like here https://stackoverflow.com/a/4316543/1620081 but it is 4 years old, like pretty much all the doc I found on the topic)?
I'm using Ruby 1.9 and Rails 3.2 with activerecord.
Thanks for support
A simple solution would be to write a utility method on the Notifier object as follows:
class Notifier
def self.deliver(message_type, *args)
self.delay.send(message_type, *args)
end
end
Send the sign up email as follows:
Notifier.deliver(:signup, #user)
The utility method provides a single point where if needed you could replace delayed job with resque or sidekiq solutions.
If you have your ActiveJob and concurrency library set-up is done documentation here.The most simple solution is to override you device send_devise_notification instance methods involved with the transactions mails like shown here
class User < ApplicationRecord
# whatever association you have here
devise :database_authenticatable, :confirmable
after_commit :send_pending_devise_notifications
# whatever methods you have here
protected
def send_devise_notification(notification, *args)
if new_record? || changed?
pending_devise_notifications << [notification, args]
else
render_and_send_devise_message(notification, *args)
end
end
private
def send_pending_devise_notifications
pending_devise_notifications.each do |notification, args|
render_and_send_devise_message(notification, *args)
end
pending_devise_notifications.clear
end
def pending_devise_notifications
#pending_devise_notifications ||= []
end
def render_and_send_devise_message(notification, *args)
message = devise_mailer.send(notification, self, *args)
# Deliver later with Active Job's `deliver_later`
if message.respond_to?(:deliver_later)
message.deliver_later
# Remove once we move to Rails 4.2+ only, as `deliver` is deprecated.
elsif message.respond_to?(:deliver_now)
message.deliver_now
else
message.deliver
end
end
end
I'm using resque to do some (long time) job. And I have a few classes with the same mixed-in module for queuing. Class Service substitutes in tests, that's why it standalone and (maybe too much) complicated. So the story is when I call
Campaign.perform(user_id)
directly, everything works fine, but when I try to use queue:
Resque.enqueue(Campaign, user_id)
Job created, but seems like do nothing. At least, nothing saves into the database. Which is main task of Campaign class. I can see in resque-web-interface that jobs creates and finished, and finished (to fast, almost just after create) but no result.
I'm new in Resque and not really sure it calls it all (confused how to debug it).
Does anybody have similar problem? thanks for any help.
Module:
module Synchronisable
def self.included(base)
base.extend ClassMethods
end
module ClassMethods
def perform(user_id)
save_objects("#{self.name}::Service".constantize.get_objects(user_id))
end
protected
def save_objects(objects)
raise ArgumentError "should be implemented"
end
end
class Service
def self.get_objects(user)
raise ArgumentError "should be implemented"
end
end
end
One of the classes:
class Campaign < ActiveRecord::Base
include Synchronisable
#queue = :app
class << self
protected
def save_objects(objects)
#some stuff to save objects
end
end
class Service
def self.get_objects(user_id)
#some stuff to get objects
end
end
end
This is a very old question so not sure how rails folder structure was back then but I had the same problem and issue was with inheritance. Seems if you are using Resque your job classes shouldn't inherit from ApplicationJob.
so if your code was like this in (app/jobs/campaign_job.rb):
class Campaign < ApplicationJob
#queue = :a_job_queue
def self.perform
#some background job
end
end
then remove the inheritance i.e "< ApplicationJob"
These jobs are almost certainly failing, due to an Exception. What is resque-web showing you on the Failures tab? You can also get this from the Rails console with:
Resque.info
or
Resque::Failure.all(0)
You should run your worker like this:
nohup QUEUE=* rake resque:work & &> log/resque_worker_QUEUE.log
This will output everything you debug to "log/resque_worker_QUEUE.log" and you will be able to find out what's wrong with your Campaign class.
Try this:
env TERM_CHILD=1 COUNT=2 "QUEUE=*" bundle exec rake resque:workers