I have Resque set to use database 6 like this:
Resque.redis = "localhost:6779:6"
I then started my resque as follows:
$ RAILS_ENV=test PIDFILE=./resque.pid QUEUE=* rake resque:work
When I check workers, there are none:
pry(main)> Resque.redis = "localhost:6779:6"
=> "localhost:6779:6"
pry(main)> Resque.workers
=> []
But when I check on database 0, I can see it is there:
pry(main)> Resque.redis = "localhost:6779:0"
=> "localhost:6379:0"
pry(main)> Resque.workers
=> [#<Worker ub40:6352:*>]
How do I make my worker use a different database?
I was able to get it working. It turns out there was resque-scheduler rake task that was overriding Resque's redis setter.
Related
I'm using mongoid alongside postgresql in a rails 5 app.
My resque jobs work perfectly with the mongoid models, however, when I try to use one of my postgresql tables inside a job, I get the following error:
PG::UndefinedTable: ERROR: relation "admins" does not exist LINE 1: SELECT "admins".* FROM "admins" ^ : SELECT "admins".* FROM "admins"
This is my lib/tasks/resque.rake file
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
Resque.before_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
Resque.after_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
end
The mentioned postgres table does exist, and works perfectly with the rails app. It seems like, at least outside of the main rails app, ActiveRecord defaults to using mongoid, so none of my postgresql models are visible inside the worker. Or maybe not.
Am I missing something?
I ended up fixing it by tweaking my Procfile and task file a bit, the issue was due to resque not picking up the right environment (hence the undefined model relationships)
Procfile:
web: bundle exec puma -t 5:5 -p ${PORT:-3000} -e ${RACK_ENV:-development}
worker: bundle exec rake resque:pool
I ended up using lib/tasks/resque.rake just to declare tasks, and moved the initialisation code to lib/tasks/resque-pool.rake. Basically resque:setup gets called before resque:pool:setup, to it preloads the rails environment for the pool manager.
require 'resque/tasks'
require 'resque/pool/tasks'
task "resque:setup" => :environment do
Resque.before_fork = Proc.new do
ActiveRecord::Base.connection.disconnect!
end
Resque.after_fork = Proc.new do
ActiveRecord::Base.establish_connection
end
end
task "resque:pool:setup" do
end
I'm using whenever to generate the cronjobs for my Rails application.
Rails 4.2.1
In the schedule.rb I have something like that:
every 13.minutes do
rake "crons:dosomething"
end
This generates a cronjoblike this:
RAILS_ENV=production bundle exec rake crons:dosomething
Problem is that this one only runs cronjobs for production environment ... but I need the cronjob as well in test and development
I tried this:
every 13.minutes do
rake "crons:dosomething", :environment => :development
rake "crons:dosomething", :environment => :test
rake "crons:dosomething", :environment => :production
end
problem here is that the second rake runs before the first one is completely finished is there any solution for this problem?
Hi I'm finding this hard to figure out as.
I'm deploying using Capistrano, and I have quite a few recurring scheduled tasks that I'm using resque-scheduler to run.
On my local environment I would run:
QUEUE=* rake environment resque:work
to start resque and to start resque-scheduler:
rake environment resque:scheduler
this is my resque.rake file:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
Resque.redis = ENV['REDIS_SERVER']
end
task :setup_schedule => :setup do
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
task :scheduler_setup => :setup_schedule
end
So what I'm trying to figure out is how I get this working in production? I assume I first need to install redis on my server even though I have a remote Redis DB? Then I need to make a capistrano task, or something that make it run?
To connect to remote redis replace Resque.redis value with
uri = URI.parse('redis://redistogo_url:9918/')
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
You don't need to install `redis' server just the normal gem
I am using queue_classic for background jobs,
I need to monitor background jobs in production ie start, stop etc.
I found the similar question but it didn't help me
Also I found the god code:
but how would I stop, restart workers?
number_queues.times do |queue_num|
God.watch do |w|
w.name = "QC-#{queue_num}"
w.group = "QC"
w.interval = 5.minutes
w.start = "bundle exec rake queue:work" # This is your rake task to start QC listening
w.gid = 'nginx'
w.uid = 'nginx'
w.dir = rails_root
w.keepalive
w.env = {"RAILS_ENV" => rails_env}
w.log = "#{log_dir}/qc.stdout.log" # Or.... "#{log_dir}//qc-#{queue_num}.stdout.log"
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
end
end
UPDATE
This code seems doesn't work
namespace :queue_classic do
desc "Start QC worker"
task :start, roles: :web do
run "cd #{release_path} && RAILS_ENV=production bundle exec rake qc:work"
end
after "deploy:restart", "queue_classic:restart"
end
As said in the documentation you can restart your worker by issuing
god restart QC-<worker_number>
where QC-<worker_number> is the name you assign to your worker
Depending on what kind of monitoring you need, you might also look at Toro, which provides a great deal of monitoring, both in a web interface and through the fact that jobs store a great deal of data and can easily be queried using ActiveRecord queries. Toro also supports middleware, which may be useful for your needs.
The resque jobs I have do not depend on anything in Rails, but I'm having a hard time starting workers without the rails env. I've seen this post, but it didn't help (ruby resque without loading rails environment)
Here is my current rake file:
require "resque/tasks"
task "resque:setup" do
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
#task "resque:setup" => :environment
The commented task would load the Rails env and everything works, but that's not what I want. When running rake resque:work I get this error:
rake aborted!
No such file to load -- application_controller
Tasks: TOP => resque:work => resque:preload
If you've only added a lib/tasks/resque.rake file and haven't modified your Rakefile, you'll still be loading your Rails environment when you call rake resque:work. Try this for Rakefile:
unless ENV['RESQUE_WORKER'] == 'true'
require File.expand_path('../config/application', __FILE__)
My::Application.load_tasks
else
ROOT_PATH = File.expand_path("..", __FILE__)
load File.join(ROOT_PATH, 'lib/tasks/resque.rake')
end
And then this for your resque.rake file:
require "resque/tasks"
task "resque:setup" do
raise "Please set your RESQUE_WORKER variable to true" unless ENV['RESQUE_WORKER'] == "true"
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
Then call rake resque:work RESQUE_WORKER=true
I referred the link here It worked perfectly for me:
This error was resolved, by running
$> QUEUE=* rake environment resque:work
a cleaner solution was to define a rake task:
task "resque:setup" => :environment do
ENV['QUEUE'] ||= '*'
#for redistogo on heroku http://stackoverflow.com/questions/2611747/rails-resque-workers-fail-with-pgerror-server-closed-the-connection-unexpectedl
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
and now
rake resque:work
Worked perfectly
Thanks.