I cannot log messages from my delayed_job process. Here is the job that is being run.
class MyJob
def initialize(blahblah)
#blahblah = blahblah
#logger = Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
end
def perform
#logger.add Logger::INFO, "logging from delayed_job"
#do stuff
end
end
I've tried various logging levels, and I have config.log_level = :debug in my environment configuration. I run delayed_job from monit. I'm using delayed_job 3.0.1 with ruby 1.9.3 and rails 3.0.10.
An explation could be that the job gets initialized only once on producer side. Then it gets serialized, delivered through the queue (database for example) and unserialized in the worker. But the initialize method is not being called in the worker process again. Only the perform method is called via send.
However you can reuse the workers logger to write to the log file:
class MyJob
def perform
say "performing like hell"
end
def say(text)
Delayed::Worker.logger.add(Logger::INFO, text)
end
end
Don't forget to restart the workers.
In RAILS_ROOT/config/initializers have a file called delayed_job_config.rb with these lines:
Delayed::Worker.logger = Rails.logger
Delayed::Worker.logger.auto_flushing = true
Remember to re-start your workers after doing this.
Let me know if this helps
I don't see why you would set the logger in the job. When I've done this I set the worker to to use a specific file on start e.g. Logger.new("log/worker_#{worker_number}") which ensures that each worker outputs to it's own file and you don't have to worry about multiple workers writing to the same file at the same time (messy).
Also, in plain ol' ruby you can call #logger.info "logging from delayed_job".
Finally, i'm pretty sure that 'perform' is called directly by your worker and instantiated, so you can refactor to:
class MyJob
def perform(blahblah)
#logger.add Logger::INFO, "logging from delayed_job"
#blahblah = blahblah
#do stuff
end
end
This is working just fine for me in Rails 3.2:
class FiveMinuteAggregateJob < Struct.new(:link, :timestamp)
def perform
Rails.logger.info 'yup.'
end
end
Related
In my Rails app, I am using sidekiq for job scheduling to lift heavy tasks. So, I have this code in a lot of places:
if Rails.env.development? or Rails.env.test?
#Call the method directly
else #Rails.env.production?
#Call the job via sidekiq that calls the said method
end
Is there any way to clean this out? I am reading software design patterns these days. I am not able to apply that knowledge here. Can you suggest how can this be cleaned up or written in a way that is more manageable?
You can put
require 'sidekiq/testing'
Sidekiq::Testing.inline!
in your development.rb and test.rb config files to get the behaviour you are after. In your applications business logic you would remove the environment conditional and just call the worker (which will now run synchronously in test and development).
How about trying refactoring like following?
module Util
extend self
def execute_method_or_delay_its_execution(obj:, method_name:)
if Rails.env.development? or Rails.env.test?
obj.send(method_name)
else #Rails.env.production?
#Call the job via sidekiq that calls the said method
end
end
end
MyClass1
def my_method
Util.execute_method_or_delay_its_execution(obj: self, method_name: :my_method)
end
end
MyClass2
def my_method
Util.execute_method_or_delay_its_execution(obj: self, method_name: :my_method)
end
end
and then just invoke the methods on object as it should be and the internal delegation should take care of your desired direct execution or delayed execution
mc_1 = MyClass1.new
mc_1.my_method
mc_2 = MyClass2.new
mc_2.my_method
Hope that helps. Thanks.
I am using the most basic version of delayed_job in a Rails app. I have the max time allowed for a delayed_job set at 10 minutes. I would like to get the hooks/callbacks working so I can do something after a job stop executing at the 10 minute mark.
I have this set in my rails app:
config.active_job.queue_adapter = :delayed_job
This is how I normally queue a job:
object.delay.object_action
The hook/callback example is for a named job but the basic, getting started steps are not for a named job. So I don't think I have a named job. Here is the example given to get the callbacks working:
class ParanoidNewsletterJob < NewsletterJob
def enqueue(job)
record_stat 'newsletter_job/enqueue'
end
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
def before(job)
record_stat 'newsletter_job/start'
end
def after(job)
record_stat 'newsletter_job/after'
end
def success(job)
record_stat 'newsletter_job/success'
end
def error(job, exception)
Airbrake.notify(exception)
end
def failure(job)
page_sysadmin_in_the_middle_of_the_night
end
end
I would love to get the after or error hooks/callbacks to fire.
Where do I put these callbacks in my Rails app to have them fire for the basic delayed_job setup? If I should be using ActiveJob callbacks where do you put those callbacks given delayed_job is being used?
You cannot use object.delay.object_action convenience syntax if you want more advanced features like callbacks. The #delay convenience method will generate a job object that works similar to this:
# something like this is already defined in delayed_job
class MethodCallerJob
def initialize(object, method, *args)
#object = object
#method = method
#args = args
end
def perform
#object.send(#method, *#args)
end
end
# `object.delay.object_action` does the below automatically for you
# instantiates a job with your object and method call
job = MethodCallerJob.new(object, :object_action, [])
Delayed::Job.enqueue(job) # enqueues it for running later
then later, in the job worker, something like the below happens:
job = Delayed::Job.find(job_id) # whatever the id turned out to be
job.invoke_job # does all the things, including calling #perform and run any hooks
job.delete # if it was successful
You have to create what the delayed_job README calls "Custom Jobs", which are just plain POROs that have #perform defined at a minimum. Then you can customize it and add all the extra methods that delayed_job uses for extra features like max_run_time, queue_name, and the ones you want to use: callbacks & hooks.
Sidenote: The above info is for using delayed_job directly. All of the above is possible using ActiveJob as well. You just have to do it the ActiveJob way by reading the documentation & guides on how, just as I've linked you to the delayed_job README, above.
You can create delayed_job hooks/callback by something like this
module Delayed
module Plugins
class TestHooks < Delayed::Plugin
callbacks do |lifecycle|
lifecycle.before(:perform) do |_worker, job|
.....
end
end
end
end
end
And need this plugin to initializer
config/initializers/delayed_job.rb
require_relative 'path_to_test_plugin'
Delayed::Worker.plugins << Delayed::Plugins::TestHooks
Similar to perform there are also hooks for success failure and error.
And similar to 'before' you can also capture the 'after' hooks.
I have installed the Ruby gem Delayed_Job to run tasks in a queue, but it shows some behavior that I don't understand. Delayed_Job is using my local active_record so a very standard installation.
I have the code for a job in a file called test_job.rb in my /lib folder
class TestJob
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def enqueue
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
When I try to call the code from a controller like this, the first time the job seems to get submitted (is listed in rake jobs:work) but it does not run:
require 'test_job'
class ExampleController < ApplicationController
def index
end
def job
# Create a new job instance
job = TestJob.new
# Enqueue the job into Delay_Job
job.enqueue
end
end
Then when I change the controller code to do what my lib class does, it works perfectly. The job does not only get submitted to the queue, but also runs and completes without failures.
require 'test_job'
class ExampleController < ApplicationController
# Create a entry in the database to track the execution of jobs
DatabaseJob = Struct.new(:text, :emails) do
def perform
# Perform Test Code
end
end
def index
end
def job
#enqueue the job
Delayed::Job.enqueue DatabaseJob.new('lorem ipsum...', 'test email')
end
end
The strange thing is that when I switch back to calling the lib job class it works without a problem. Then it does not matter whether the struct is directly defined in the controller or in the class in the lib folder.
Defining the struct inside the controller and submitting the job to the queue this way always seems to work, but afterwards also the lib class starts working and sometimes the lib class works even after a restart of the rails server.
Any ideas? Thank you very much for the help.
Best,
Bastian
I have a very very large rails application and I've read every post there on how to reduce boot time, each suggesting to cut down on models or controllers or gems, but all of them are being used.
Problem i'm facing is that heroku pushes R10 error code since my app takes more than 60s to load.
I've been trying one thing, almost successfully. I'm trying to run Bundler.require and Application.inialize! in threads (latter thread waiting on the first one to finish). Advantage of this being that thin server boots almost instantly.
Problem is, when someone hits the app with a request, the initialization process stays incomplete. Any idea how i can achieve this?
You can take control of the initialization process by extending the initialize! method in config/application.rb.
Look at this sample application.rb file from a github project. You should be able to extend the initialize! method inside of the class Application < Rails::Application scope.
You can take the base methods from the Rails Initialization docs in section 2.3 railties/lib/rails/application.rb. So here is what you'll have if you extend the methods and leave them as they are implemented by default.
class Application < Rails::Application
def initialize!(group=:default) #:nodoc:
raise "Application has been already initialized." if #initialized
run_initializers(group, self)
#initialized = true
self
end
def run_initializers(group=:default, *args)
return if instance_variable_defined?(:#ran)
initializers.tsort_each do |initializer|
initializer.run(*args) if initializer.belongs_to?(group)
end
#ran = true
end
end
At this point you can add print statements and determine which piece you're getting stuck on. For example, is your #initialized variable never being set to true? If so, why? Perhaps you are getting stuck inside of run_intializers. If that is the case, then which initializer are you getting stuck inside of?
I've realized what i'm trying to do is not possible. So i'm ending the thread.
What i'm doing:
In my environment.rb,
def init_bundler
# If you want your assets lazily compiled in production, use this line
s = Time.now
_rails_groups = (ENV["RAILS_GROUPS"]||"").split(",")
if _rails_groups.length == 1 and _rails_groups.first.to_s == "assets"
Bundler.require(:assets) #load ONLY assets ..
elsif _rails_groups.length == 1 and _rails_groups.first.to_s == "clock"
Bundler.require(:clock) #load ONLY assets ..
else
Bundler.require(:default, *_rails_groups, Rails.env.to_s)
end
puts "bundler required in #{Time.now - s}s"
end
puts "starting t1"
$_t1 = Thread.new{ init_bundler }
In my environment.rb:
def init_app
# Initialize the rails application
puts "initialize started"
s = Time.now
Hedgepo::Application.initialize!
puts "initialize done in #{Time.now - s}s"
end
puts "starting t2"
$_t2 = Thread.new do
loop do
if defined?($_t1) and !$_t1.status
init_app
break
else
puts "waiting for bundler"
end
sleep(1)
end
end
the middleware i'm using above everyone else:
class MyInit
def initialize(app)
#app = app
end
def call(env)
loop do
if defined?($_t2) and !$_t2.status
break
else
puts "waiting for app init to finish...."
end
sleep(1)
end
#app.call(env)
end
end
But the problem is that Rack::Server allocates an 'app' variable and .start!s the server when the hit comes. So whatever I do, i cannot change the app that was initialized BEFORE this #app variable was populated.
I'm using resque to do some (long time) job. And I have a few classes with the same mixed-in module for queuing. Class Service substitutes in tests, that's why it standalone and (maybe too much) complicated. So the story is when I call
Campaign.perform(user_id)
directly, everything works fine, but when I try to use queue:
Resque.enqueue(Campaign, user_id)
Job created, but seems like do nothing. At least, nothing saves into the database. Which is main task of Campaign class. I can see in resque-web-interface that jobs creates and finished, and finished (to fast, almost just after create) but no result.
I'm new in Resque and not really sure it calls it all (confused how to debug it).
Does anybody have similar problem? thanks for any help.
Module:
module Synchronisable
def self.included(base)
base.extend ClassMethods
end
module ClassMethods
def perform(user_id)
save_objects("#{self.name}::Service".constantize.get_objects(user_id))
end
protected
def save_objects(objects)
raise ArgumentError "should be implemented"
end
end
class Service
def self.get_objects(user)
raise ArgumentError "should be implemented"
end
end
end
One of the classes:
class Campaign < ActiveRecord::Base
include Synchronisable
#queue = :app
class << self
protected
def save_objects(objects)
#some stuff to save objects
end
end
class Service
def self.get_objects(user_id)
#some stuff to get objects
end
end
end
This is a very old question so not sure how rails folder structure was back then but I had the same problem and issue was with inheritance. Seems if you are using Resque your job classes shouldn't inherit from ApplicationJob.
so if your code was like this in (app/jobs/campaign_job.rb):
class Campaign < ApplicationJob
#queue = :a_job_queue
def self.perform
#some background job
end
end
then remove the inheritance i.e "< ApplicationJob"
These jobs are almost certainly failing, due to an Exception. What is resque-web showing you on the Failures tab? You can also get this from the Rails console with:
Resque.info
or
Resque::Failure.all(0)
You should run your worker like this:
nohup QUEUE=* rake resque:work & &> log/resque_worker_QUEUE.log
This will output everything you debug to "log/resque_worker_QUEUE.log" and you will be able to find out what's wrong with your Campaign class.
Try this:
env TERM_CHILD=1 COUNT=2 "QUEUE=*" bundle exec rake resque:workers