Under Rails 4.2.3, I have setup ActiveJob to use delayed_job for its backend in all environments:
environment.rb
require File.expand_path('../boot', __FILE__)
require 'rails/all'
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)
module MyApp
class Application < Rails::Application
# ... snip
# Use delayed_job for Active Job queueing
config.active_job.queue_adapter = :delayed_job
end
end
(and queue_adapter is not otherwise set under config).
I have configured my DelayedJob instance as follows - including code from this question to back-port the Rails 5 feature of getting the database row for a job:
config/delayed_job.rb
Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 60
Delayed::Worker.max_attempts = 3
Delayed::Worker.max_run_time = 5.minutes
Delayed::Worker.read_ahead = 10
# Delayed::Worker.delay_jobs = true #!Rails.env.test?
Delayed::Worker.logger = Logger.new(Rails.root.join('log', 'delayed_job.log'))
class Delayed::Job < ActiveRecord::Base
belongs_to :owner, polymorphic: true
end
# Backport Rails 5 code to make the Delayed::Job database id avilable on ActiveJobs
module ActiveJob
module Core
# ID optionally provided by adapter
attr_accessor :provider_job_id
end
module QueueAdapters
class DelayedJobAdapter
class << self
send(:prepend, Module.new do
def enqueue(job)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
def enqueue_at(job, timestamp)
provider_job = super
job.provider_job_id = provider_job.id
provider_job
end
end)
end
end
end
end
I have model code to perform a job later, and hook the Delayed::Job instance onto the model instance:
app/models/my_model.rb
def schedule
job = MyJob.set(wait_until: scheduled_time).perform_later(self)
# Associate the delayed_job object with the item that has been scheduled
delayed_job = Delayed::Job.find(job.provider_job_id)
delayed_job.owner = self
delayed_job.save
end
(and I have set up appropriate associations for Delayed::Job).
This appears to work fine in development; but in test, the Delayed::Job.find(job.provider_job_id) fails and raises an exception, because provider_job_id is nil. Examining the logs shows no SQL INSERT for the delayed_jobs table.
Why is there no delayed job row being created in test?
I was having this same problem. Turns out that ActiveJob::TestCase hijacks the perform_later logic and doesn't actually queue them, but instead stores data about queued jobs in an array (see here).
My solution was to write a test that inherited from ActiveSupport::TestCase instead, which seems to queue delayed jobs like normal. I believe there are some test settings that will force jobs to be performed inline, but as far as I know, the default behavior is to queue them, so it should work as expected without much tinkering.
Related
I am trying to implement a Job in my Ruby on Rails application, but I keep getting this error:
NotImplementedError
Error from the server:
NotImplementedError (NotImplementedError):
app/controllers/cron_controller.rb:6:in `message_10_minutes'
Here's the ActiveJob:
class TestSmsJob < ActiveJob::Base
queue_as :default
#include Plivio
def perform(*args)
# do my stuff
end
end
This is the call to the function for executing the job:
class CronController < ApplicationController
def index
end
def message_10_minutes
TestSmsJob.set(wait: 10.minutes).perform_later()
render :layout => false
end
end
Do you guys know what am I missing?
Rails by default comes with own implementation of asynchronous queuing however you still needs to specify which adapter do you want to use. If you want to use built in adapter you simply don't have to bundle any other gem.
Here is the list of adapters - http://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html
Since sidekiq may be too hard for the start you can use sucker_punch - https://github.com/brandonhilkert/sucker_punch
After you choose your adapter you have to update config
module YourApp
class Application < Rails::Application
# Be sure to have the adapter's gem in your Gemfile
# and follow the adapter's specific installation
# and deployment instructions.
config.active_job.queue_adapter = :adapter_name
end
end
I have a problem with sidekiq/activejob integration. I have a controller that calls a perform_later method from a MyJob class. This works with the perform method, but when I change to perfom_later, the job is scheduled in my development log. However, when I see the sidekiq dashboard, at the retries section, I can see NameError: uninitialized constant (look below image)
These are my files:
# app/jobs/crime_job.rb
class CrimeJob < ActiveJob::Base
queue_as :default
def perform(crime)
puts "Perform #{crime}"
end
def self.job_name(crime)
"RadarCrime:#{crime.id}"
end
end
Crime Controller
# app/controllers/crime_controller.rb
def show
# [...]
CrimeJob.perform_later(#crime)
end
Sidekiq initializer
# config/initializers/active_job.rb
Rails.application.config.active_job.queue_adapter = :sidekiq
Well, I also open an issue in Sidekiq repository, and the solution is easier than I've think.
Just restart the sidekiq process and it's works fine.
Issue link: https://github.com/mperham/sidekiq/issues/2207
I'm creating a DripEmail campaign for my app and using resque scheduler to schedule the tasks.
I've sceduled a static job, which runs every day at a specific time and collect the user's list based on the drip settings and sends out an emailer to them.
This is my resque job user_follow_up.rb
class UserFollowUp
#queue = :user_follow_up
def self.perform
User.each do |u|
# Send the emailers to only those who are not converted
if !user.is_converted and Date.today <= user.next_email
stage(u)
end
end
end
end
This is my scheduler.yml
UserFollowUp:
cron: "0 16 * * *"
I have 2 resque workers, one has my default set of tasks and the other for scheduler.
rake environment resque:work QUEUE=publish_story,accept_story,image_queue,Mango_mailer
and
rake environment resque:scheduler QUEUE=user_follow_up
When I open the resque admin interface, I'm able to see my static job detected in the list. I clicked the Queue Now button to test it. It properly enqueues the task to the queue, but doesn't execute. It keeps these tasks in the pending queue forever.
This is my resque.rake, it's required
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque/scheduler/server'
require 'active_record'
require 'mongoid'
require 'action_controller/railtie'
require 'active_support/buffered_logger'
# load the Rails app all the time
namespace :resque do
puts 'Loading Rails environment for Resque'
task :setup => :environment do
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/scheduler.yml")
Resque::Scheduler.dynamic = true
Resque.logger.info 'Resque Scheduler Initialized!'
Resque.before_first_fork do
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info 'Resque Logger Initialized!'
puts 'Resque Logger Initialized!'
end
end
task 'resque:pool:setup' do
Resque::Pool.after_prefork do |job|
Resque.redis.client.reconnect
end
end
end
And this is my Gemfile
gem 'resque', github: 'resque/resque' , branch: '1-x-stable'
gem 'resque_mailer', github: 'zapnap/resque_mailer'
gem 'resque-scheduler'
I'm not sure what is the issue here. Please help me with this fix.
I am using Resque (and resque-scheduler) in my Rails app to run a recurring job. This was working fine for me, until today. I made some code changes, which I thought were unrelated, but now every worker fails before the perform method is even entered (checked with a debug statement). The same worker method works fine when I run it in the rails console. It only fails via resque on the development localhost (Postgres DB).
The error shown in the resque console for the failed worker is:
Exception
NoMethodError
Error
undefined method `write' for nil:NilClass
There is no additional stack trace for the error. Any idea why this is failing?
Additional info:
lib/tasks/resque.rake
# Resque tasks
require 'resque/tasks'
require 'resque_scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
# you probably already have this somewhere
Resque.redis = 'localhost:6379'
# If you want to be able to dynamically change the schedule,
# uncomment this line. A dynamic schedule can be updated via the
# Resque::Scheduler.set_schedule (and remove_schedule) methods.
# When dynamic is set to true, the scheduler process looks for
# schedule changes and applies them on the fly.
# Note: This feature is only available in >=2.0.0.
#Resque::Scheduler.dynamic = true
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
# If your schedule already has +queue+ set for each job, you don't
# need to require your jobs. This can be an advantage since it's
# less code that resque-scheduler needs to know about. But in a small
# project, it's usually easier to just include you job classes here.
# So, something like this:
# require 'jobs'
end
end
task "resque:setup" => :environment do
#ENV['QUEUE'] = '*'
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
config/resque.yml
development: localhost:6379
test: localhost:6379:1
staging: redis1.se.github.com:6379
fi: localhost:6379
production: redis1.ae.github.com:6379
initializers/resque.rb
rails_root = Rails.root || File.dirname(__FILE__) + '/../..'
rails_env = Rails.env || 'development'
resque_config = YAML.load_file(rails_root.to_s + '/config/resque.yml')
Resque.redis = resque_config[rails_env]
# This will make the tabs show up.
require 'resque_scheduler'
require 'resque_scheduler/server'
config/resque_schedule.yml
populate_game_data:
# you can use rufus-scheduler "every" syntax in place of cron if you prefer
every: 1m
# By default the job name (hash key) will be taken as worker class name.
# If you want to have a different job name and class name, provide the 'class' option
class: PopulateDataWorker
queue: high
args:
description: "This job populates the game and data"
Should note that the above files were not changed between working and non-working state.
We had the same issue this morning, and we pinned it down to a gem update by New Relic.
Version 3.5.6.46 of newrelic_rpm was yanked on rubygems, but it was somehow installed by bundle update.
They are still on the beta track for 3.5.6 and had some issues with Resque. See https://github.com/newrelic/rpm/commit/e81889c2bce97574ec682dafee12015e13ccb2e1
The fix was to add '~> 3.5.5.38' in our Gemfile for newrelic_rpm
I am processing my background jobs using Resque.
My model looks like this
class SomeClass
...
repo = Repo.find(params[:repo_id])
Resque.enqueue(ReopCleaner, repo.id)
...
end
class RepoCleaner
#queue = :repo_cleaner
def self.perform(repo_id)
puts "this must get printed in console"
repo = Repo.find(repo_id)
# some more action here
end
end
Now to test in synchronously i have added
Resque.inline = Rails.env.test?
in my config/initializers/resque.rb file
This was supposed to call #perform method inline without queuing it into Redis and without any Resque callbacks as Rails.env.test? returns true in test environment.
But
"this must get printed in console"
is never printed while testing. and my tests are also failing.
Is there any configurations that i have missed.
Currently i am using
resque (1.17.1)
resque_spec (0.7.0)
resque_unit (0.4.0)
I personally test my workers different. I use RSpec and for example in my user model I test something like this:
it "enqueue FooWorker#create_user" do
mock(Resque).enqueue(FooWorker, :create_user, user.id)
user.create_on_foo
end
Then I have a file called spec/workers/foo_worker_spec.rb with following content:
require 'spec_helper'
describe FooWorker do
describe "#perform" do
it "redirects to passed action" do
...
FooWorker.perform
...
end
end
end
Then your model/controller tests run faster and you don't have the dependency between model/controller and your worker in your tests. You also don't have to mock so much things in specs which don't have to do with the worker.
But if you wan't to do it like you mentioned, it worked for me some times ago. I put Resque.inline = true into my test environment config.
It looks like the question about logging never got answered. I ran into something similar to this and it was from not setting up the Resque logger. You can do something as simple as:
Resque.logger = Rails.logger
Or you can setup a separate log file by adding this to your /lib/tasks/resque.rake. When you run your worker it will write to /log/resque.log
Resque.before_fork = Proc.new {
ActiveRecord::Base.establish_connection
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::Logger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info "Resque Logger Initialized!"
}
Mocking like daniel-spangenberg mentioned above ought to write to STDOUT unless your methods are in the "private" section of your class. That's tripped me up a couple times when writing rspec tests. ActionMailer requires it's own log setup too. I guess I've been expecting more convention than configuration. :)