Resque scheduled jobs are stuck in pending queue - ruby-on-rails

I'm creating a DripEmail campaign for my app and using resque scheduler to schedule the tasks.
I've sceduled a static job, which runs every day at a specific time and collect the user's list based on the drip settings and sends out an emailer to them.
This is my resque job user_follow_up.rb
class UserFollowUp
#queue = :user_follow_up
def self.perform
User.each do |u|
# Send the emailers to only those who are not converted
if !user.is_converted and Date.today <= user.next_email
stage(u)
end
end
end
end
This is my scheduler.yml
UserFollowUp:
cron: "0 16 * * *"
I have 2 resque workers, one has my default set of tasks and the other for scheduler.
rake environment resque:work QUEUE=publish_story,accept_story,image_queue,Mango_mailer
and
rake environment resque:scheduler QUEUE=user_follow_up
When I open the resque admin interface, I'm able to see my static job detected in the list. I clicked the Queue Now button to test it. It properly enqueues the task to the queue, but doesn't execute. It keeps these tasks in the pending queue forever.
This is my resque.rake, it's required
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque/scheduler/server'
require 'active_record'
require 'mongoid'
require 'action_controller/railtie'
require 'active_support/buffered_logger'
# load the Rails app all the time
namespace :resque do
puts 'Loading Rails environment for Resque'
task :setup => :environment do
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/scheduler.yml")
Resque::Scheduler.dynamic = true
Resque.logger.info 'Resque Scheduler Initialized!'
Resque.before_first_fork do
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info 'Resque Logger Initialized!'
puts 'Resque Logger Initialized!'
end
end
task 'resque:pool:setup' do
Resque::Pool.after_prefork do |job|
Resque.redis.client.reconnect
end
end
end
And this is my Gemfile
gem 'resque', github: 'resque/resque' , branch: '1-x-stable'
gem 'resque_mailer', github: 'zapnap/resque_mailer'
gem 'resque-scheduler'
I'm not sure what is the issue here. Please help me with this fix.

Related

View Resque log output in Rails server logs

I've got a Rails 4 app on a Puma server with Resque/Resque-Scheduler running background jobs. What I'd like to know is how I merge the log output of my two Resque workers into my server log, or, of that is not possible, how I can view the log output of my Resque workers. Currently I have not been able to figure out how to view the log output for the workers, so I have no idea what's happening under the hood. I found this blogpost, which suggests adding the following likes to my resque.rake file:
task "resque:setup" => :environment do
Resque.before_fork = Proc.new {
ActiveRecord::Base.establish_connection
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info "Resque Logger Initialized!"
}
end
That didn't work. I also tried the suggestion in the comments, which was to replace Resque.logger = ActiveSupport::BufferedLogger.new(logfile) with Resque.logger = ActiveSupport::Logger.new(logfile), however that didn't work either. With the second option, I still get a NoMethodError: undefined method 'logger=' for Resque:Module error when I try to boot up a worker.
Here is my current resque.rake file:
require 'resque/tasks'
require 'resque_scheduler/tasks'
namespace :resque do
puts "Loading Rails environment for Resque"
task :setup => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
require 'postman'
end
end
I've looked at the Resque docs on logging, but am not sure how to use what's there as I admittedly don't know very much about logging in Rails. I haven't had any luck finding other useful resources on the subject.
How I fix it, it is not perfect but just works.
my environment: rails 5.0.1, resque: 1.26.0
at the first time, I set the Resque.logger and Resque.logger.level in config/initializers/resque.rb as most docs suggest:
# config/initializers/resque.rb
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
then in the job, I output log by Resque.logger.info:
# update_xxx_job.rb
class UpdateXxxJob
def self.perform
Resque.logger.info 'Job starts'
...
end
end
it doesn't work, I can see nothing in log/resque.log.
then someone said should set the logfile sync always, no buffer, so I update the config/initializers/resque.rb according a question from stackoverflow:
# config/initializers/resque.rb
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
logfile.sync = true
Resque.logger = ActiveSupport::Logger.new(logfile)
Resque.logger.level = Logger::INFO
still doesn't work.
I also tried config resque logger in lib/tasks/resque.rake:
# lib/tasks/resque.rake
require 'resque'
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque-scheduler'
require 'resque/scheduler/server'
namespace :resque do
task setup: :environment do
Resque.schedule = YAML.load_file(Rails.root + "config/resque_scheduler_#{Rails.env}.yml")
Resque.redis.namespace = "xxx_#{Rails.env}"
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::INFO
end
end
doesn't work.
finally, I decide to move the logger configuration from initializer to the job, so the job now looks like:
# update_xxx_job.rb
class UpdateXxxJob
def self.perform
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
Resque.logger.info 'Job starts'
...
end
end
then I can get what I want in the log/resque.log.
you can try it.
I've had the same problem while setting up mine. Here's what I did:
Resque.before_fork do
# Your code here
end
It seems before_fork accepts a block as an argument rather than assigning a block to it.
I faced the same issue so that I check the source of resque and finally I needed to do the followings at initialization process:
define log formatter.
then define logger with log-file path.
set any log level.
Here the example is at my config/initializers/resque.rb in rails case:
...
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
Resque.logger.formatter = ::Logger::Formatter.new # This is important
Resque default logger formatter is set here and its definitions is here. That apparently just ignores the output...

how setup active_job and rescue scheduler?

I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.

Scheduled Jobs and Custom Clock Processes with Clockwork

I am trying to understand how to execute custom code with clockwork. This is the example lib/clock.rb file that Heroku uses in its devcenter document.
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
include Clockwork
every(4.minutes, 'Queueing interval job') { Delayed::Job.enqueue IntervalJob.new }
every(1.day, 'Queueing scheduled job', :at => '14:17') { Delayed::Job.enqueue ScheduledJob.new }
What is IntervalJob and ScheduledJob? Where are these files supposed to be located? I want to run my own custom job that has access to my database records.
EDIT
This is my /lib/clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(2.minutes, 'Filtering Streams') { Delayed::Job.enqueue FilterJob.new}
end
This is my /lib/filter_job.rb
class FilterJob
def perform
#streams = Stream.all
#streams.each do |stream|
# manipulating stream properties
end
end
end
I get the error:
uninitialized constant Clockwork::FilterJob (NameError)
/app/lib/clock.rb:11:in `block in <module:Clockwork>'
You need to do the following:
Firstly install the clockwork gem.
In your lib folder create a clock.rb
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(1.day, 'Creating Cycle', :at => '22:00') { Delayed::Job.enqueue CyclePlannerJob.new}
end
In the example your provided IntervalJob and ScheduledJob, are delayed jobs. Clockwork triggers them on the time specified. I am calling the CyclePlannerJob, this is what my file looks like.
lib/cycle_planner_job.rb
class CyclePlannerJob
def perform
CyclePlanner.all.each do |planner|
if Time.now.in_time_zone("Eastern Time (US & Canada)").to_date.send("#{planner.start_day.downcase}?")
planner.create_cycle
end
end
end
end
In my example everyday at 10pm, I am running the CyclePlanner job, which runs the delayed job I have setup. Similar to the Heroku example.
Bare in mind to use this you need to setup the clock work and delayed jobs on your Heroku app in the dashboard.
Also your Procfile should look like this.
worker: bundle exec rake jobs:work
clock: bundle exec clockwork lib/clock.rb
Let me know if you have any questions, I can go into more detail if needed.
Looks like name space issue. Move your filter_job.rb to models directory and try.

Rake task not running from within worker

I don't understand why my rake task is not running from within a resque worker. Running
rake :send_this_email
from the console works fine, I just want to run it as a cron job (as follows) but something is not working proplerly while invoking the rake task from within the worker.
My rescue_schedule.yml
send_this_email:
cron: "*/2 * * * *"
class: SendThisEmailWorker
args:
description: "Send email when condition defined in rake task is met"
My send_this_email_worker.rb in workers directory, where the problem must be if I can manually call the rake task myself from the console?
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
Rake::Task["send_this_email"].invoke
end
end
When I start my dev server this send_this_email rake task should run every 2 minutes correct? It's not and the resque admin panel shows it as a job in the queue. What am I missing here?
Thanks for your attention.
UPDATED from gerep comment
require 'rake'
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
Only require 'rake' is not enough. For example if you do
Rake::Task.tasks #list down all task
You will get []
You need to tell your worker class to load tasks.
Try this
require 'rake'
Rake::Task.clear # necessary to avoid tasks being loaded several times in dev mode
YOUR_APP_NAME::Application.load_tasks
module SendThisEmailWorker
#queue = :send_this_email
def self.perform
puts "Hi from the console, I'm started"
Rake::Task["send_this_email"].invoke
end
end
YOUR_APP_NAME is the name of your app and can be found at config/application.rb

Resque workers fail immediately: undefined method `write' for nil:NilClass

I am using Resque (and resque-scheduler) in my Rails app to run a recurring job. This was working fine for me, until today. I made some code changes, which I thought were unrelated, but now every worker fails before the perform method is even entered (checked with a debug statement). The same worker method works fine when I run it in the rails console. It only fails via resque on the development localhost (Postgres DB).
The error shown in the resque console for the failed worker is:
Exception
NoMethodError
Error
undefined method `write' for nil:NilClass
There is no additional stack trace for the error. Any idea why this is failing?
Additional info:
lib/tasks/resque.rake
# Resque tasks
require 'resque/tasks'
require 'resque_scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
# you probably already have this somewhere
Resque.redis = 'localhost:6379'
# If you want to be able to dynamically change the schedule,
# uncomment this line. A dynamic schedule can be updated via the
# Resque::Scheduler.set_schedule (and remove_schedule) methods.
# When dynamic is set to true, the scheduler process looks for
# schedule changes and applies them on the fly.
# Note: This feature is only available in >=2.0.0.
#Resque::Scheduler.dynamic = true
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
# If your schedule already has +queue+ set for each job, you don't
# need to require your jobs. This can be an advantage since it's
# less code that resque-scheduler needs to know about. But in a small
# project, it's usually easier to just include you job classes here.
# So, something like this:
# require 'jobs'
end
end
task "resque:setup" => :environment do
#ENV['QUEUE'] = '*'
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
config/resque.yml
development: localhost:6379
test: localhost:6379:1
staging: redis1.se.github.com:6379
fi: localhost:6379
production: redis1.ae.github.com:6379
initializers/resque.rb
rails_root = Rails.root || File.dirname(__FILE__) + '/../..'
rails_env = Rails.env || 'development'
resque_config = YAML.load_file(rails_root.to_s + '/config/resque.yml')
Resque.redis = resque_config[rails_env]
# This will make the tabs show up.
require 'resque_scheduler'
require 'resque_scheduler/server'
config/resque_schedule.yml
populate_game_data:
# you can use rufus-scheduler "every" syntax in place of cron if you prefer
every: 1m
# By default the job name (hash key) will be taken as worker class name.
# If you want to have a different job name and class name, provide the 'class' option
class: PopulateDataWorker
queue: high
args:
description: "This job populates the game and data"
Should note that the above files were not changed between working and non-working state.
We had the same issue this morning, and we pinned it down to a gem update by New Relic.
Version 3.5.6.46 of newrelic_rpm was yanked on rubygems, but it was somehow installed by bundle update.
They are still on the beta track for 3.5.6 and had some issues with Resque. See https://github.com/newrelic/rpm/commit/e81889c2bce97574ec682dafee12015e13ccb2e1
The fix was to add '~> 3.5.5.38' in our Gemfile for newrelic_rpm

Resources