Hi I'm finding this hard to figure out as.
I'm deploying using Capistrano, and I have quite a few recurring scheduled tasks that I'm using resque-scheduler to run.
On my local environment I would run:
QUEUE=* rake environment resque:work
to start resque and to start resque-scheduler:
rake environment resque:scheduler
this is my resque.rake file:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
Resque.redis = ENV['REDIS_SERVER']
end
task :setup_schedule => :setup do
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
task :scheduler_setup => :setup_schedule
end
So what I'm trying to figure out is how I get this working in production? I assume I first need to install redis on my server even though I have a remote Redis DB? Then I need to make a capistrano task, or something that make it run?
To connect to remote redis replace Resque.redis value with
uri = URI.parse('redis://redistogo_url:9918/')
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
You don't need to install `redis' server just the normal gem
Related
I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.
I've been trying to figure what the issue is for couple days now but no luck, I'm running Rails 3.1.3 on Heroku Cedar with Unicorn and using Resque 1.20 for background jobs.
Redis add-on as been added and REDISTOGO_URL set, I have resque.rb initializer as
require 'resque'
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
uri = URI.parse(ENV["REDISTOGO_URL"])
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.redis = REDIS
I also tried
Resque.redis = Redis.connect(:url => ENV['REDISTOGO_URL'])
Also this, from the official RedisToGo site
ENV["REDISTOGO_URL"] ||= "redis://username:password#host:1234/"
uri = URI.parse(ENV["REDISTOGO_URL"])
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :thread_safe => true)
Heroku worker as:
bundle exec rake resque:work QUEUE='*'
Everything works fine locally and in Heroku console; but when I try to queue from Rails
Resque.enqueue(EmailQueue, id)
I get
NoMethodError (undefined method `sadd' for nil:NilClass):
Please any help will be appreciated, thank you.
Figured it out, since I was on Unicorn; I had to set config/unicorn.rb as below to connect or Redis.
after_fork do |server, worker|
# Replace with MongoDB or whatever
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
Rails.logger.info('Connected to ActiveRecord')
end
# If you are using Redis but not Resque, change this
if defined?(Resque)
Resque.redis = ENV['REDISTOGO_URL']
Rails.logger.info('Connected to Redis')
end
end
Hopes this helps someone else.
The resque jobs I have do not depend on anything in Rails, but I'm having a hard time starting workers without the rails env. I've seen this post, but it didn't help (ruby resque without loading rails environment)
Here is my current rake file:
require "resque/tasks"
task "resque:setup" do
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
#task "resque:setup" => :environment
The commented task would load the Rails env and everything works, but that's not what I want. When running rake resque:work I get this error:
rake aborted!
No such file to load -- application_controller
Tasks: TOP => resque:work => resque:preload
If you've only added a lib/tasks/resque.rake file and haven't modified your Rakefile, you'll still be loading your Rails environment when you call rake resque:work. Try this for Rakefile:
unless ENV['RESQUE_WORKER'] == 'true'
require File.expand_path('../config/application', __FILE__)
My::Application.load_tasks
else
ROOT_PATH = File.expand_path("..", __FILE__)
load File.join(ROOT_PATH, 'lib/tasks/resque.rake')
end
And then this for your resque.rake file:
require "resque/tasks"
task "resque:setup" do
raise "Please set your RESQUE_WORKER variable to true" unless ENV['RESQUE_WORKER'] == "true"
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
Then call rake resque:work RESQUE_WORKER=true
I referred the link here It worked perfectly for me:
This error was resolved, by running
$> QUEUE=* rake environment resque:work
a cleaner solution was to define a rake task:
task "resque:setup" => :environment do
ENV['QUEUE'] ||= '*'
#for redistogo on heroku http://stackoverflow.com/questions/2611747/rails-resque-workers-fail-with-pgerror-server-closed-the-connection-unexpectedl
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
and now
rake resque:work
Worked perfectly
Thanks.
I have an existing resque.rake file that lets me run heroku rake jobs:work to spawn a resque worker. Here is the file:
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"
Now I want to add a specified number of workers by adding the COUNT=X option, but I am unsure where and how to add it to my rake file. How should I do this?
I would thing you could just add another ENV[] key/value:
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
ENV['COUNT'] = 5
end
How do I make sure resque has access to all my ENV variables? I'm trying to send an email from within a resque job and it cannot send because the actionmailer smtp username/pass are set via ENV variables. It looks like even ENV['RAILS_ENV'] is not available from within the resque job.
Here's my resque.rake file:
# Run to start:
# rake resque:work QUEUE='*'
require 'resque/tasks'
require 'resque_scheduler/tasks'
task "resque:setup" => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
Here's my resque.rb initializer:
require 'resque_scheduler'
Resque.redis = 'localhost:6379'
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Here's my procfile that starts redis/resque/resque worker
mongo: mongod
redis-server: redis-server /usr/local/etc/redis.conf
scheduler: bundle exec rake resque:scheduler
worker: bundle exec rake resque:work QUEUE=images, notifications
Add them to resque.rake
task "resque:setup" => :environment do
# ... other stuff
ENV['RAILS_ENV'] = Rails.env
end
Well I can't tell for sure but if this is UNIX, open /proc/self/environ will work even if there's no language facility to read ENV.
This is, of course, last resort.