We're currently in transition between a Ruby/RoR 1.8/3.2 and a rewritten 2.0/4.0 app on the same server. Since I don't want to mess with the current resque configuration and risk updating any gems that might break production, I elected to setup a separate Redis server on a new port and use the newest version of Resque with it. Resque appears to be running fine; if I manually launch a worker rake resque:work QUEUE='*' and watch the process it runs a queued job from resque-scheduler. However, none of my workers, launched manually or via script, show up in the resque-web instance running. The stats page of resque-web shows it's looking at the second redis instance. Does anyone have experience with this or any ideas?
/config/initializers/resque.rb:
require 'resque_scheduler'
rails_root = ENV["RAILS_ROOT"] || File.dirname(__FILE__) + "/../.."
rails_env = ENV["RAILS_ENV"] || "demo"
resque_config = YAML.load_file(rails_root + "/config/resque.yml")
Resque.redis = resque_config[rails_env]
Resque.redis.namespace = "resque-reos:#{rails_env}"
/config/resque.yml
demo: localhost:6380
Related
So far I had a simple application that only required the classic rails server to boot.
I have recently added the react_on_rails gem and it requires to boot a nodejs server to handle webpack and javascript stuff.
So I understand I need this foreman gem that is capable of managing several processes. So far so good, but then I'm still having a few problems understanding and deploying this enhanced app to my production environment (Phusion Passenger on Apache/Nginx)
So several questions :
Does passenger handle the transition from rails s to foreman start -f Procfile.dev automatically ?
If no then where do I setup things so passenger works ?
Side question : almost all google results refer to puppet when looking for foreman on passenger. Anyone could explain what puppet does in 1 line and if I really need it in production ? So far everythings runs smoothly on localhost with the foreman start -f Procfile.dev command so I don't know where this is coming from...
I am deploying my application to the Amazon Cloud using Capistrano, and I was expecting to have the rails + nodejs setup on every autoscaled instance (and Passenger would graciously handle all that). Am I thinking wrong ?
In our production environment we use eye to manage other processes related to the rails app. (Passenger will run from mod_passenger while the workers are controlled by eye)
And here is an example of how to start 4 concurrent queue_classic workers:
APP_ROOT = File.expand_path(File.dirname(__FILE__))
APP_NAME = File.basename(APP_ROOT)
Eye.config do
logger File.join(APP_ROOT, "log/eye.log")
end
Eye.application APP_NAME do
working_dir File.expand_path(File.dirname(__FILE__))
stdall 'log/trash.log' # stdout,err logs for processes by default
env 'RAILS_ENV' => 'production' # global env for each processes
trigger :flapping, times: 10, within: 1.minute, retry_in: 10.minutes
group 'qc' do
1.upto(4) do |i|
process "worker-#{i}" do
stdall "log/worker-#{i}.log"
pid_file "tmp/pids/worker-#{i}.pid"
start_command 'bin/rake qc:work'
daemonize true
stop_on_delete true
end
end
end
end
I have a Rails app running on Nginx/Passenger. It has a rufus-scheduler cron job that runs in the background and sends out notifications via email.
When I start up the app in production on Nginx/Passenger, the emails don't get sent. In the production logs it doesn't show any logs for rufus-scheduler.
I'm stuck in this problem. Not able to debug the scheduler.
But after reading some issues & articles, I am sure that scheduler thread is kill on production by passenger.
Code snippet:
Example:
require 'rufus-scheduler'
scheduler = Rufus::Scheduler::singleton
scheduler.cron '0 12 * * *' do
puts "alert scheduler called at"
puts Time.now
system("rake trigger:email RAILS_ENV=production")
end
Kindly help me on the conf for the passenger/nginx that allow the scheduler thread to run & trigger the mails.
You need to set this in passanger config block
passenger_spawn_method direct;
I am using Rufus Scheduler to call a function once every 2 minutes.
In development mode when running rufus scheduler with WEBrick the system works as expected and my function is called every 2 minutes.
When I deploy to our production server which runs passenger and NGINX the scheduler does not call the function anymore.
I am assuming this is to do with threads, with the scheduler running on a separate thread that might be getting destroyed on our production server.
I have seen answers for similar issues when using Passenger and Apache together, but I am unable to come up with a solution for running rufus scheduler with passenger and NGINX in production mode.
Any help would be appreciated.
Code below setting up rufus
# :nocov:
require 'rufus-scheduler'
include TwitterParser
unless ENV['NODAEMON'] || Rails.env != 'production'
scheduler = Rufus::Scheduler.new
TwitterParser.set_up
scheduler.every '2m' do
begin
TwitterParser.parse_tweets
rescue Twitter::Error::TooManyRequests => error
puts 'Too Many Twitter requests, resets in: ' + error.rate_limit.reset_in.to_s
end
end
end
# :nocov:
Please read the rufus-scheduler FAQ.
https://github.com/jmettraux/rufus-scheduler#faq
It contains links to (old) articles about Passenger + rufus-scheduler
Amog them, this SO answer by one of the authors of Passenger might help:
rufus cron job not working in Apache/Passenger
Have a good time.
I'm trying moving my Reddis server off to an external box. Have followed the Resque readme on github through.
In development mode, it loads the config just fine and connects to localhost on 6379:
resque.rb initialiser
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'
resque_config = YAML.load_file(rails_root + '/config/resque.yml')
Resque.redis = resque_config[rails_env]
resque.yaml
development: localhost:6379
playground: redis1.play.xxx.com:6379
production: redis1.pro.xxx.com:6379
However, in playground / production modes, it falls back to development server and doesn't connect. I'm assuming this is because unicorn's not declaring the environment correctly?
If I replace 'development' with 'playground' in the initialiser, it works.
I'm starting unicorn with:
unicorn -c config/unicorn.rb -E playground -l 8000 -D
How can I get it to pick up the correct conf??
Finally sorted although I don't really understand why... Won't accept my own answer for a couple of days if someone wants to interject.
By getting God to manage the service instead of starting / stopping manually, it picked up the correct environment.
Now I'm connecting to a remote redis service with no issues.
I'm using Redis To Go on Heroku, and I'd like to use it to store some data alongside Resque jobs, much like I can store DelayedJobs in the same Postgres database as the rest of my model data.
Is this possible? Care to explain briefly how Resque stores jobs in Redis? I'm new to Redis, and though I understand on a higher level how it works, I'm unfamiliar with the implementation details.
Yes, you can. According to the Resque documentation:
Resque has a redis setter which can be given a string or a Redis object. This means if you're already using Redis in your app, Resque can re-use the existing connection.
Also, the documentation for Resque.redis= states that a namespace is accepted, to prevent clashes with your other apps running on the same instance:
A 'hostname:port/namespace' String (to set the Redis namespace)
Check out the "Configuration" section of the Resque readme for more information: https://github.com/resque/resque/blob/master/README.markdown
Redis is an advanced key value store which is used by Resque to store data about jobs. Exact details depend on your application however the two should be fine working together. As long as your code stays away from using the lists Resque uses then you will be golden.
yes, completely agree with other answers, the Redis can be used for other purposes, not only for background processing. By the way, Rescue provides easy interface to setup/config/use/browse Redis.
Setup/Run
$ git clone --depth=1 git://github.com/defunkt/resque.git
$ cd resque
$ rake redis:install dtach:install
$ vim config/resque.yml
"development: localhost:6379"
$ vim config/initializers/rescue.rb
"rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'
resque_config = YAML.load_file(rails_root + '/config/resque.yml')
Resque.redis = resque_config[rails_env]"
$ rake redis:start
Use
# model
def user_activity
a = Resque.redis.lrange "log_" + self.id.to_s, 0, -1
a.map{|i| JSON.parse(i)}
end
Browse
$ resque-web
It allows you to browse Redis content, not only background processing