How do I make sure resque has access to all my ENV variables? I'm trying to send an email from within a resque job and it cannot send because the actionmailer smtp username/pass are set via ENV variables. It looks like even ENV['RAILS_ENV'] is not available from within the resque job.
Here's my resque.rake file:
# Run to start:
# rake resque:work QUEUE='*'
require 'resque/tasks'
require 'resque_scheduler/tasks'
task "resque:setup" => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
Here's my resque.rb initializer:
require 'resque_scheduler'
Resque.redis = 'localhost:6379'
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Here's my procfile that starts redis/resque/resque worker
mongo: mongod
redis-server: redis-server /usr/local/etc/redis.conf
scheduler: bundle exec rake resque:scheduler
worker: bundle exec rake resque:work QUEUE=images, notifications
Add them to resque.rake
task "resque:setup" => :environment do
# ... other stuff
ENV['RAILS_ENV'] = Rails.env
end
Well I can't tell for sure but if this is UNIX, open /proc/self/environ will work even if there's no language facility to read ENV.
This is, of course, last resort.
Related
Hi I'm finding this hard to figure out as.
I'm deploying using Capistrano, and I have quite a few recurring scheduled tasks that I'm using resque-scheduler to run.
On my local environment I would run:
QUEUE=* rake environment resque:work
to start resque and to start resque-scheduler:
rake environment resque:scheduler
this is my resque.rake file:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
Resque.redis = ENV['REDIS_SERVER']
end
task :setup_schedule => :setup do
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
end
task :scheduler_setup => :setup_schedule
end
So what I'm trying to figure out is how I get this working in production? I assume I first need to install redis on my server even though I have a remote Redis DB? Then I need to make a capistrano task, or something that make it run?
To connect to remote redis replace Resque.redis value with
uri = URI.parse('redis://redistogo_url:9918/')
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
You don't need to install `redis' server just the normal gem
I am trying to run a background job on Heroku using Resque/Clockwork.
Locally, jobs are queued by Clockwork and executed by Resque as expected. On Heroku, sometimes it works... but most of the time Clockwork queues the job as expected, and when executed by Resque I get a failed job with the error 'No job defined for class'.
The exact error that shows up through the Resque admin panel is:
Worker 960f8a1b-cce9-497a-a7ab-9b40c166a600:2+1 on FEATURED_CATALOG_ITEMS_CACHE at 27 minutes ago Retry or Remove
Class nil
Arguments nil
Exception Error
Error No job defined for class 'Workers::FeaturedCatalogItemsCacheWorker'
My code looks like:
/app/models/workers/featured_catalog_items_cache_worker.rb
class Workers::FeaturedCatalogItemsCacheWorker
#queue = :featured_catalog_items_cache
def self.perform
p 'do work' # there is more code here but edited for brevity
end
end
/lib/clock.rb
require File.expand_path('../../config/boot', __FILE__)
require File.expand_path('../../config/environment', __FILE__)
require 'clockwork'
module Clockwork
every(2.minutes, 'Queueing FeaturedCatalogItemsCacheWorker job') { Resque.enqueue(Workers::FeaturedCatalogItemsCacheWorker) }
end
/lib/tasks/resque.rake
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
Procfile
web: rackup -p $PORT
resque: env TERM_CHILD=1 bundle exec rake resque:work
clock: bundle exec clockwork lib/clock.rb
Am I missing something?
maybe in /lib/clock.rb
require 'Workers::FeaturedCatalogItemsCacheWorker'
Solution :
require './config/boot'
require './config/environment'
require 'clockwork'
require 'resque'
module Clockwork
handler do |job|
puts "Running job : #{job}"
Resque.enqueue(job)
end
every(15.minutes, NewsRssReader )
end
I'am using Capistrano for deployment and Sidekiq for queues. I need to load user ids to set :sidekiq_queues variable in deploy.rb file to perform sidekiq queues. Now I use following code
set :sidekiq_queues, ::User.all.map {|u| "-q parsing_user_#{u.id}"}.join(" ") + " -q parsing_user_0"
but it throws following error
./config/deploy.rb:29:in `load': uninitialized constant User (NameError)
I tried to require 'rubygems' and 'active_record' into deploy.rb, but it didn't help.
In result I should have
sidekiq_queues == "-q parsing_user_1 -q parsing_user_2 -q parsing_user_3 -q parsing_user_4 -q parsing_user_5 -q parsing_user_0".
Hardcoding queue names isn't a solution.
Capistrano executes deploy.rb locally, and it doesn't load your Rails environment in deploy.rb.
It might be more trouble than it is worth. Especially if you want to execute this on your remote server, you might consider doing a Rake task instead. The => :environment in the rake task ensures that your Rails environment is loaded.
# in deploy.rb
namespace :sidekiq do
desc "Do something with queues"
task :queues, :roles => :web do
run "cd #{current_path} ; RAILS_ENV=#{rails_env} bundle exec rake sidekiq:queues"
end
end
# you'll need to decide when to execute this in your deployment process,
# something like this:
after "deploy:update_code", "sidekiq:queues"
# in lib/tasks/sidekiq.rake
namespace :sidekiq do
desc "Do something with queues"
task :queues => :environment do
queues = (User.scoped.pluck(:id) + [0]).map{|id| "-q parsing_user_#{id}"}.join(" ")
# do what you need to do
end
end
The resque jobs I have do not depend on anything in Rails, but I'm having a hard time starting workers without the rails env. I've seen this post, but it didn't help (ruby resque without loading rails environment)
Here is my current rake file:
require "resque/tasks"
task "resque:setup" do
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
#task "resque:setup" => :environment
The commented task would load the Rails env and everything works, but that's not what I want. When running rake resque:work I get this error:
rake aborted!
No such file to load -- application_controller
Tasks: TOP => resque:work => resque:preload
If you've only added a lib/tasks/resque.rake file and haven't modified your Rakefile, you'll still be loading your Rails environment when you call rake resque:work. Try this for Rakefile:
unless ENV['RESQUE_WORKER'] == 'true'
require File.expand_path('../config/application', __FILE__)
My::Application.load_tasks
else
ROOT_PATH = File.expand_path("..", __FILE__)
load File.join(ROOT_PATH, 'lib/tasks/resque.rake')
end
And then this for your resque.rake file:
require "resque/tasks"
task "resque:setup" do
raise "Please set your RESQUE_WORKER variable to true" unless ENV['RESQUE_WORKER'] == "true"
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
Then call rake resque:work RESQUE_WORKER=true
I referred the link here It worked perfectly for me:
This error was resolved, by running
$> QUEUE=* rake environment resque:work
a cleaner solution was to define a rake task:
task "resque:setup" => :environment do
ENV['QUEUE'] ||= '*'
#for redistogo on heroku http://stackoverflow.com/questions/2611747/rails-resque-workers-fail-with-pgerror-server-closed-the-connection-unexpectedl
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
and now
rake resque:work
Worked perfectly
Thanks.
I'm using Sinatra, and I wanted to set up some of the convenience rake tasks that Rails has, specifically rake db:seed.
My first pass was this:
namespace :db do
desc 'Load the seed data from db/seeds.rb'
task :seed do
seed_file = File.join(File.dirname(__FILE__), 'db', 'seeds.rb')
system("racksh < #{seed_file}")
end
end
racksh is a gem that mimics Rails' console. So I was just feeding the code in the seed file directly into it. It works, but it's obviously not ideal. What I'd like to do is create an environment task that allows commands to be run under the Sinanta app/environment, like so:
task :environment do
# what goes here?
end
task :seed => :environment do
seed_file = File.join(File.dirname(__FILE__), 'db', 'seeds.rb')
load(seed_file) if File.exist?(seed_file)
end
But what I can't figure out is how to set up the environment so the rake tasks can run under it. Any help would be much appreciated.
I've set up a Rakefile for Sinatra using a kind of Rails-like environment:
task :environment do
require File.expand_path(File.join(*%w[ config environment ]), File.dirname(__FILE__))
end
You then have something in config/environment.rb that contains what you need to start up your app properly. It might be something like:
require "rubygems"
require "bundler"
Bundler.setup
require 'sinatra'
Putting this set-up in a separate file avoids cluttering your Rakefile and can be used to launch your Sinatra app through config.ru if you use that:
require File.expand_path(File.join(*%w[ config environment ]), File.dirname(__FILE__))
run Sinatra::Application