I have absolutely no idea how to run my resque scheduler. When i enqueue a single task and run it manually it works fine but when i try to implement resque scheduler using the command rake resque:scheduler --trace, i get ArgumentError: unsupported signal SIGUSR1. Below are the files needed for resque scheduler:
config/initializers/resque.rb
require 'resque/failure/multiple'
require 'resque/failure/redis'
Resque::Failure::Multiple.classes = [Resque::Failure::Redis]
Resque::Failure.backend = Resque::Failure::Multiple
Dir[File.join(Rails.root, 'app', 'jobs', '*.rb')].each { |file| require file }
config = YAML.load(File.open("#{Rails.root}/config/resque.yml"))[Rails.env]
Resque.redis = Redis.new(host: config['host'], port: config['port'], db: config['db'])
config/resque.yml
defaults: &defaults
host: localhost
port: 6379
db: 6
development:
<<: *defaults
test:
<<: *defaults
staging:
<<: *defaults
production:
<<: *defaults
lib/tasks/resque.rake
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'yaml'
task 'resque:setup' => :environment
namespace :resque do
task :setup_schedule => :setup do
require 'resque-scheduler'
# If you want to be able to dynamically change the schedule,
# uncomment this line. A dynamic schedule can be updated via the
# Resque::Scheduler.set_schedule (and remove_schedule) methods.
# When dynamic is set to true, the scheduler process looks for
# schedule changes and applies them on the fly.
# Note: This feature is only available in >=2.0.0.
# Resque::Scheduler.dynamic = true
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file(File.open("#{Rails.root}/config/resque_schedule.yml"))
end
task :scheduler => :setup_schedule
end
config/resque_schedule.yml
run_my_job:
cron: '30 6 * * 1'
class: 'MyJob'
queue: myjob
args:
description: "Runs MyJob"
Here's the error message for the rake resque:scheduler command:
error message
just found out that Windows doesn't support the SIGUSR1 signal. Here's a list of supported signals in Windows. The solution will be to use another OS such as Ubuntu to run the operation and it runs with no problems.
Related
Im attempting to run resque for a rails 7 app on my machine like this: QUEUE=* bundle exec rake resque:work
It fails with this:
rake aborted!
TypeError: Unsupported command argument type: Resque::Worker
Original Exception (TypeError): Unsupported command argument type: Resque::Worker
Original Exception (Resque::PruneDeadWorkerDirtyExit): Worker Davids-Mac-mini:38378:* did not gracefully exit while processing <Unknown Job>
I am sure redis-server is running correctly
This is my setup
application.rb
config.active_job.queue_adapter = :resque
config/initializers/resque.rb
require 'resque/server'
if Rails.env.development?
Resque.redis = Redis.new(host: 'localhost', port: '6379')
else
uri = URI.parse(ENV['REDIS_URL'])
REDIS = Redis.new(host: uri.host, port: uri.port, password: uri.password)
Resque.redis = REDIS
end
application_job.rb
class ApplicationJob < ActiveJob::Base
# Automatically retry jobs that encountered a deadlock
# retry_on ActiveRecord::Deadlocked
# Most jobs are safe to ignore if the underlying records are no longer available
# discard_on ActiveJob::DeserializationError
end
schedules_job.rb
class SchedulesJob < ApplicationJob
queue_as :default
def perform(id)
puts 'Hello world!'
end
end
Rakefile
# Add your own tasks in files placed in lib/tasks ending in .rake,
# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.
require_relative 'config/application'
require 'resque'
require 'resque/tasks'
task 'resque:setup' => :environment
Rails.application.load_tasks
So I just migrated to Rails 5.1.4 and I'm trying to make Active Job work, but the jobs are just stuck in the queue and never processed.
rails: 5.1.4
ruby: 2.4.3
sidekiq: 5.0.5
redis: 4.0.1
sidekiq.yml
---
:verbose: true
:concurrency: 5
:timeout: 60
development:
:concurrency: 25
staging:
:concurrency: 50
production:
:concurrency: 5
:queues:
- default
- [high_priority, 2]
sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = {url: ENV['ACTIVE_JOB_URL'], network_timeout: 5}
end
Sidekiq.configure_client do |config|
config.redis = {url: ENV['ACTIVE_JOB_URL'], network_timeout: 5}
end
Here the is how I perform the task from the rails console:
TestJob.perform_later
TestJob.rb content:
class TestJob < ApplicationJob
queue_as :default
def perform(*args)
Rails.logger.debug "#{self.class.name}: I'm performing my job with arguments: #{args.inspect}"
end
end
The jobs are just stuck in the queue and never processed:
Have you started the worker, eg in development it might be:
bundle exec sidekiq
If in production Heroku should do this for you, if you have configured your Procfile, eg:
web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq
You can also use your Procfile in development with foreman, eg:
foreman start -f Procfile
Hi I'm finding this hard to figure out as.
I'm deploying using Capistrano, and I have quite a few recurring scheduled tasks that I'm using resque-scheduler to run.
On my local environment I would run:
QUEUE=* rake environment resque:work
to start resque and to start resque-scheduler:
rake environment resque:scheduler
this is my resque.rake file:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
Resque.redis = ENV['REDIS_SERVER']
end
task :setup_schedule => :setup do
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
task :scheduler_setup => :setup_schedule
end
So what I'm trying to figure out is how I get this working in production? I assume I first need to install redis on my server even though I have a remote Redis DB? Then I need to make a capistrano task, or something that make it run?
To connect to remote redis replace Resque.redis value with
uri = URI.parse('redis://redistogo_url:9918/')
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
You don't need to install `redis' server just the normal gem
I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.
How do I make sure resque has access to all my ENV variables? I'm trying to send an email from within a resque job and it cannot send because the actionmailer smtp username/pass are set via ENV variables. It looks like even ENV['RAILS_ENV'] is not available from within the resque job.
Here's my resque.rake file:
# Run to start:
# rake resque:work QUEUE='*'
require 'resque/tasks'
require 'resque_scheduler/tasks'
task "resque:setup" => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
Here's my resque.rb initializer:
require 'resque_scheduler'
Resque.redis = 'localhost:6379'
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Here's my procfile that starts redis/resque/resque worker
mongo: mongod
redis-server: redis-server /usr/local/etc/redis.conf
scheduler: bundle exec rake resque:scheduler
worker: bundle exec rake resque:work QUEUE=images, notifications
Add them to resque.rake
task "resque:setup" => :environment do
# ... other stuff
ENV['RAILS_ENV'] = Rails.env
end
Well I can't tell for sure but if this is UNIX, open /proc/self/environ will work even if there's no language facility to read ENV.
This is, of course, last resort.