I am monitoring my redis server using God (a Ruby gem). However, my existing server may already has an instance of redis up. How can I be sure it monitors the existing Redis server process that is already up?
This is my God file for redis:
rails_root = ENV['RAILS_ROOT']
redis_root = "/usr/local/bin"
# Redis
%w{6379}.each do |port|
God.watch do |w|
w.name = "redis"
w.interval = 30.seconds
w.start = "#{redis_root}/redis-server /etc/redis/redis.conf"
w.stop = "#{redis_root}/redis-cli shutdown"
w.restart = "#{w.stop} && #{w.start}"
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.log = File.join(rails_root, 'log', 'redis.log')
w.keepalive(:memory_max => 5000.megabytes)
w.start_if do |start|
start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end
end
To answer this question:
I put a
w.pid_file = "SOMETHING" in my God file, and made sure this PID file was also set in the configuration file for Redis.
You should also add:
w.pid_file = "Your_pid_file_name"
and then clean the pid file with
w.behaviour(:clean_pid_file)
Related
in my rails 5 app i use resque and resque-scheduler for sending message to my customer.for that i created different different queue for messages and created 3 worker for sending message using queue.
so, here my question is how can i use one specific worker for one specific queue.
i.e i have four queue like birthday_checker, reminder_message, appointment_checker, confirmation_message. For appointment_checker i set cron it will run every 56s. for that my all 3 worker busy to run appointment_checker queue. and other queue job going in pending. but here can i reserve one worker for this appointment_checker queue. so is it possible?
i tried find some related questions on stack but i can't find specific solu for that.
Here is my resque.god file code
rails_env = ENV['RAILS_ENV']
rails_root = File.dirname(__FILE__) + '/..'
num_workers = rails_env == 'production' ? 3 : 2
num_workers.times do |num|
God.watch do |w|
w.dir = "#{rails_root}"
w.name = "resque-#{num}"
w.group = 'resque'
w.interval = 30.seconds
w.env = {"QUEUE"=>"*", "RAILS_ENV"=>rails_env}
w.start = "bundle exec rake -f #{rails_root}/Rakefile environment resque:work"
w.log = "#{rails_root}/log/resque.log"
w.err_log = "#{rails_root}/log/resque_error.log"
# w.uid = 'git'
# w.gid = 'git'
# restart if memory gets too high
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 350.megabytes
c.times = 2
end
end
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
c.interval = 5.seconds
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
c.interval = 5.seconds
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_running) do |c|
c.running = false
end
end
end
end
And this one is lib/tasks/resque_scheduler.rake file code
namespace :resque do
task :setup => :environment do
require 'resque'
ENV['QUEUE'] = '*'
end
task :setup_schedule => :setup do
require 'resque-scheduler'
Resque.schedule = YAML.load_file('config/resque_schedule.yml')
end
task :scheduler => :setup_schedule
end
I tried my best to ask my first question on stackoverflow so please ignore small mistake and give me some solution for above question.
Thanks!
The ENV['QUEUE'] = '*' is a wildcard for all queues. Each Queue will have the name of the class you are queuing with. Just replace the * with the name of the queue you want to run.
Here is the god configuration I have done.
God.watch do |w|
pid_file = "#{Rails.root}/tmp/pids/delayed_job.pid"
w.pid_file = pid_file
pid = File.read(pid_file)
w.log = "#{Rails.root}/log/god.log"
w.name = "delayed_job"
w.dir = Rails.root
w.interval = 15.seconds
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.start = "RAILS_ENV=production ruby #{Rails.root}/bin/delayed_job start"
w.stop = "RAILS_ENV=production ruby #{Rails.root}/bin/delayed_job stop"
w.restart = "RAILS_ENV=production ruby #{Rails.root}/bin/delayed_job restart"
w.env = { 'RAILS_ENV' => "production" }
w.behavior(:clean_pid_file)
w.start_if do |start|
start.condition(:process_running) do |c|
c.interval = 60.seconds
c.running = false
end
end
# When to restart a running deamon?
w.restart_if do |restart|
restart.condition(:memory_usage) do |c|
c.above = 200.megabytes
c.times = [2,5]
end
restart.condition(:cpu_usage) do |c|
c.above = 90.percent
c.times = 2
end
end
end
It is entering into the :cpu_usage condition when the CPU load is high but does not do so when the memory load goes higher than 200 MB. Is there anything in the above script that is obviously wrong, but that I'm unable to spot?
Or is there something more that I need to add?
Excerpt from God documentation
The number of times the test needs to fail in order to trigger a restart is set with times. This can be either an integer or an array. An integer means it must fail that many times in a row while an array [x, y] means it must fail x times out of the last y tests.
The :cpu_usage conditions gets resolved first as its times value is 2, meaning that after 2 consecutive fails God will trigger this condition. While the :memory_usage is set to [2,5], meaning that it should fail 2 times in 5 checks.
I'm trying to monitor a redis server with god (ruby gem). I changed the PID file path in /etc/redis/redis.conf to a rails app temp folder of the user deploying my app (using capistrano), and in the redis.god file I added the line "w.pid_file= ..." which points to the same PID path as the one I changed in the redis.conf file. So the redis.god file looks like this at the moment:
rails_env = ENV['RAILS_ENV'] || 'production'
raise "Please specify RAILS_ENV." unless rails_env
rails_root = ENV['RAILS_ROOT'] || File.expand_path(File.join(File.dirname(__FILE__), '..', '..'))
# Redis
%w{6379}.each do |port|
God.watch do |w|
w.dir = "#{rails_root}"
w.name = "redis"
w.interval = 30.seconds
w.start = "/etc/init.d/redis-server start /etc/redis/redis.conf"
w.stop = "/etc/init.d/redis-server stop"
w.restart = "/etc/init.d/redis-server restart"
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.log = "#{rails_root}/log/redis.log"
w.pid_file = "/home/deployer/myapp/current/tmp/pids/redis-server.pid"
w.behavior(:clean_pid_file)
w.start_if do |start|
start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end
end
So the problem that I'm having is that god can get redis started. I looked at its log for the god file and it says the following:
Starting redis-server: touch: cannot touch `/var/run/redis/redis-server.pid': Permission denied
Why is it still trying to look in /var/run/redis/redis-server.pid? I changed the PID path in the redis.conf file to the new one shown above because I was getting Permission denied, but it is still insisting in looking in /var/run/redis/redis-server.pid. FYI, this where I got the idea to changed the PID path: God configuration file to monitor existing processes?
Please make sure you have disabled SElinux
you can disable selinux with this command:
setenforce 0
I have an app where users will be uploading files directly to S3. This is working. Now, I need a background worker (presently delayed_job) to retrieve the file and stash it in 'tmp/files' for processing.
How can this be done?
Edits: The app is presently running in EC2.
Background workers will run independently from your web app.
Try Resque for a commonly used Rails background worker solution. The idea is that you start Resque up independently from your web app, and it does its jobs independently of the application.
Have this worker sent basic HTTP requests to S3. Here's an API reference card to get you started. The idea is that you use some sort of Ruby REST client to send these requests, and parse the response you get from S3. Rest-client is a gem you can use to do this.
Optionally, you can also have the worker use the S3 gem which could be a bit easier.
With this approach, you'd have your worker run a script that does something like
picture = S3Object.find 'headshot.jpg', 'photos'
Use Resque.
add
gem 'resque'
gem 'resque-status'
With Resque you need Redis (to store information about workers) either use Redis-to-go or install Redis locally on your EC2 machine.
So after Resque is installed edit config/initializers/resque.rb
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'production'
resque_config = YAML.load_file(rails_root + '/config/resque.yml')
Resque.redis = resque_config[rails_env]
# This is if you are using Redis to go:
# ENV["REDISTOGO_URL"] ||= "redis://REDISTOGOSTUFFGOESHERE"
# uri = URI.parse(ENV["REDISTOGO_URL"])
# Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :thread_safe => true)
Resque::Plugins::Status::Hash.expire_in = (24 * 60 * 60) # 24hrs in seconds
Dir["#{Rails.root}/app/workers/*.rb"].each { |file| require file }
Here we are using local Redis, so resque.yml looks like this:
development: localhost:6379
test: localhost:6379
fi: localhost:6379
production: localhost:6379
You will need something like God to start/manage workers
So install it then add "resque-production.god" to config/ folder of your app
You will be able to start your workers via this: god -c config/resque-production.god
the config/resque-production.god file willl have something like:
rails_env = ENV['RAILS_ENV'] || "production"
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/..'
num_workers = 1
num_workers.times do |num|
God.watch do |w|
w.dir = "#{rails_root}"
w.name = "resque-#{num}"
w.group = 'resque'
w.interval = 30.seconds
w.env = {"QUEUE"=>"*", "RAILS_ENV"=>"production"}
w.start = "rake -f #{rails_root}/Rakefile environment resque:work --trace"
w.log = "#{rails_root}/log/resque.log"
w.err_log = "#{rails_root}/log/resque_error.log"
# restart if memory gets too high
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 350.megabytes
c.times = 2
end
end
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
c.interval = 5.seconds
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
c.interval = 5.seconds
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_running) do |c|
c.running = false
end
end
end
end
Finally workers. They go into app/workers/ folder (here is app/workers/processor.rb)
class Processor
include Resque::Plugins::Status
#queue = :collect_queue
def perform
article_id = options["article_id"]
article = Article.find(article_id)
article.download_remote_file(article.file_url)
end
end
It is triggered by the callback in the Article model (app/models/article.rb)
class Article < ActiveRecord::Base
after_create :process
def download_remote_file(url)
# OpenURI extends Kernel.open to handle URLs as files
io = open(url)
# overrides Paperclip::Upfile#original_filename;
# we are creating a singleton method on specific object ('io')
def io.original_filename
base_uri.path.split('/').last
end
io.original_filename.blank? ? nil : io
end
def process
Processor.create(:article_id => self.id)
end
end
I'm struggling with using Sidekiq alongside the God Gem. I would like to be able to start the sidekiq process manually, using $god start sidekiq, however this fails to start the process. I can only get it to start the sidekiq process if I set w.keepalive (commented out in the below code).
I'm launching God using: $ god -c "./config.god" -D --log-level info, launching god in the foreground which gets me the following output:
I [2013-01-22 17:46:00] INFO: Started on drbunix:///tmp/god.17165.sock
I [2013-01-22 17:46:00] INFO: sidekiq move 'unmonitored' to 'up'
I [2013-01-22 17:46:00] INFO: sidekiq moved 'unmonitored' to 'up'
Using $god start sidekiq I get:
Sending 'start' command
The following watches were affected:
sidekiq
But I get no output from God, nothing is written to sidekiq's log and I can clearly see no sidekiq process has started by using $ ps auxwww | grep sidekiq.
# config.god
PROJECT_ROOT = ENV['PROJECT_ROOT'] || File.dirname(__FILE__) # Dir containing this file
# Sidekiq Process
God.watch do |w|
w.name = "sidekiq"
w.group = "conversion"
w.dir = PROJECT_ROOT
w.interval = 20.seconds
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.behavior(:clean_pid_file)
# w.keepalive
w.start = "bundle exec sidekiq -v -C #{File.join(PROJECT_ROOT, 'config.yml')}"
w.stop = "bundle exec sidekiqctl stop '/Users/me/.god/pids/sidekiq.pid' 5"
w.log = File.join(PROJECT_ROOT, 'log/god_sidekiq.log')
end
Try this:
# config.god
PROJECT_ROOT = ENV['PROJECT_ROOT'] || File.dirname(__FILE__) # Dir containing this file
# Sidekiq Process
pid_file = '/Users/me/.god/pids/sidekiq.pid'
God.watch do |w|
w.name = "sidekiq"
w.group = "conversion"
w.dir = PROJECT_ROOT
w.interval = 20.seconds
w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.pid_file = pid_file
w.behavior(:clean_pid_file)
w.start = "cd #{PROJECT_ROOT}; bundle exec sidekiq -v -C #{File.join(PROJECT_ROOT, 'config.yml')} -P #{pid_file}&"
w.stop = "cd #{PROJECT_ROOT}; bundle exec sidekiqctl stop #{pid_file} 5"
w.log = File.join(PROJECT_ROOT, 'log/god_sidekiq.log')
# determine the state on startup
w.transition(:init, {true => :up, false => :start}) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_exits)
end
# lifecycle
w.lifecycle do |on|
on.condition(:flapping) do |c|
c.to_state = [:start, :restart]
c.times = 5
c.within = 5.minute
c.transition = :unmonitored
c.retry_in = 10.minutes
c.retry_times = 5
c.retry_within = 2.hours
end
end
end