I have the following:
/config/delayed_job_config.rb
# Enable DelayedJob Logging
Delayed::Worker.logger = Rails.logger
Delayed::Worker.logger.auto_flushing = 1
# Do not delete failed jobs (default is true)
Delayed::Worker.destroy_failed_jobs = false
# Dont wait so long between jobs
Delayed::Worker.read_ahead = 100 # default was 5
Delayed::Worker.sleep_delay = 1 #default was 5
Problem here is the settings are not being used:
$ rails c
Loading development environment (Rails 3.2.5)
1.9.3-p125 :001 > Delayed::Worker.read_ahead
=> 5
1.9.3-p125 :002 > Delayed::Worker.sleep_delay
=> 5
Any ideas what's wrong here? Thanks
The configuration goes in:
config/initializers/delayed_job_config.rb
Documentation is here.
Related
ActiveJob assigns don't seem to show up in the sidekiq queues.. But the jobs still get executed.
This is my active_job setup in config/environments/production.rb
config.active_job.queue_adapter = :sidekiq
config.active_job.queue_name_prefix = "faces_#{Rails.env}"
My sidekiq.yml that gets loaded via Capistrano.
---
:concurrency: 5
:queues:
- default
production:
:concurrency: 20
:queues:
- faces_production_default
Sample output when I start to queue a job and check if it's there.
2.6.3 :001 > require 'sidekiq/api'
=> true
2.6.3 :002 > Sidekiq::Queue.new('faces_production_default').count
=> 0
2.6.3 :003 > stats = Sidekiq::Stats.new
=> #<Sidekiq::Stats:0x0000000004dd80e8 #stats={:processed=>48, :failed=>0, :scheduled_size=>1, :retry_size=>0, :dead_size=>0, :processes_size=>2, :default_queue_latency=>0, :workers_size=>0, :enqueued=>0}>
2.6.3 :004 > stats.queues
=> {"default"=>0, "faces_production_default"=>0, "mailers"=>0}
2.6.3 :005 > FaceParserJob.set(wait: 5.minutes).perform_later(200)
Enqueued FaceParserJob (Job ID: 9848e560-dbf5-4fc0-a3c3-6b8f00164a8a) to Sidekiq(faces_production_default) at 2019-04-28 15:16:25 UTC with arguments: 200
=> #<FaceParserJob:0x0000000004f19b00 #arguments=[200], #job_id="9848e560-dbf5-4fc0-a3c3-6b8f00164a8a", #queue_name="faces_production_default", #priority=nil, #executions=0, #scheduled_at=1556464585.1313956, #provider_job_id="43d2c83a305411bfda6626b1">
2.6.3 :006 > Sidekiq::Queue.new('faces_production_default').count
=> 0
Thing is if I monitor the sidekiq.log in 5 minutes it does execute the job. What am I doing wrong?
Job is put into a queue only when it's being "launched", until then - it waits in scheduled queue, you can check with
Sidekiq::ScheduledSet.new.size
I'm having issues with the workers that I have set up for a website. Here's Some background of the app:
It runs on rails 3.0.2, using mongodb, redis gem 2.2.2. In order to keep the workers up and running I have set up the god gem and specify that 6 workers should run in the production environment. I'll be pasting the resque.god.rb file bellow. Also, there's a unique Ubuntu server that is set up for the resque-workers and elasticsearch services only so it doesn't share any other service.
My problem is that for any reason, the workers keep dying and just log "***Exiting..." in my log/resque-worker.log file which is extremely annoying because I don't know what is going on. It doesn't log anything in the syslog file nor the dmesg
This is a piece of what I get in the log (not helpful for me)
*** Starting worker workers:19166:*
*** Starting worker workers:19133:*
*** Running before_first_fork hook
*** Exiting...
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /data/www/tap-production/releases/20170904162514)
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/activesupport-3.0.20/lib/active_support/dependencies.rb:242:in `block in require': iconv will be deprecated in the future, use String#encode instead.
*** Running before_first_fork hook
*** Exiting...
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /data/www/tap-production/releases/20170904162514)
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/activesupport-3.0.20/lib/active_support/dependencies.rb:242:in `block in require': iconv will be deprecated in the future, use String#encode instead.
*** Starting worker workers:19251:*
*** Starting worker workers:19217:*
*** Running before_first_fork hook
*** Exiting...
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /data/www/tap-production/releases/20170904162514)
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/activesupport-3.0.20/lib/active_support/dependencies.rb:242:in `block in require': iconv will be deprecated in the future, use String#encode instead.
*** Running before_first_fork hook
*** Exiting...
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /data/www/tap-production/releases/20170904162514)
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/activesupport-3.0.20/lib/active_support/dependencies.rb:242:in `block in require': iconv will be deprecated in the future, use String#encode instead.
*** Starting worker workers:19330:*
*** Starting worker workers:19297:*
*** Running before_first_fork hook
*** Exiting...
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /data/www/tap-production/releases/20170904162514)
/usr/local/www/tap-production/shared/bundle/ruby/1.9.1/gems/activesupport-3.0.20/lib/active_support/dependencies.rb:242:in `block in require': iconv will be deprecated in the future, use String#encode instead.
Here's my resque.god.rb code:
require 'tlsmail'
rails_env = ENV['RAILS_ENV']
rails_root = ENV['RAILS_ROOT']
rake_root = ENV['RAKE_ROOT']
num_workers = rails_env == 'production' ? 6 : 1
# Change cache to my_killer_worker_job if you are testing in development. remember to enable it on config/resque_schedule.yml - Fabian
queue = rails_env == 'production' ? '*' : 'my_killer_worker_job'
God::Contacts::Email.defaults do |d|
Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)
if rails_env == "production"
#Change this settings for your own purposes
d.from_name = "#{rails_env.upcase}: Process monitoring"
d.delivery_method = :smtp
d.server_host = 'smtp.gmail.com'
d.server_port = 587
d.server_auth = :login
d.server_domain = 'gmail.com'
d.server_user = 'XXXX#gmail.com'
d.server_password = 'XXXX'
end
end
God.contact(:email) do |c|
c.name = 'engineering'
c.group = 'developers'
c.to_email = 'engineering#something.com'
end
num_workers.times do |num|
God.watch do |w|
w.name = "resque-#{num}"
w.group = 'resque'
w.interval = 30.seconds
w.env = { 'RAILS_ENV' => rails_env, 'QUEUE' => queue, 'VERBOSE' => '1' }
w.dir = rails_root
w.start = "bundle exec #{rake_root}/rake resque:work"
w.start_grace = 10.seconds
w.log = File.join(rails_root, 'log', 'resque-worker.log')
# restart if memory gets too high
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 200.megabytes
c.times = 2
# c.notify = 'engineering'
end
end
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
# c.notify = 'engineering'
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
c.interval = 5.seconds
# c.notify = 'engineering'
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
c.interval = 5.seconds
# c.notify = 'engineering'
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_running) do |c|
c.running = false
c.notify = {:contacts => ['engineering'], :priority => 1, :category => "workers"}
end
end
end
end
Please let me know your thoughts.
Looks like i got to the bottom of the problem. On my routes file I had added a call to a method in order to load some dynamic routes and looks like resque didn't like it and just kept killing the processes without saying anything (still very weird). however, after removing that line (DynamicRouter.load) the workers stopped the crazy behavior. I hope this helps someone else and I'll be glad to give more details on what I could find.
I am trying to set this option to true as described in the documentation. I used rails g kaminari:config which generated the following kaminari_config.rb :
Kaminari.configure do |config|
# config.default_per_page = 25
# config.max_per_page = nil
# config.window = 4
# config.outer_window = 0
# config.left = 0
# config.right = 0
# config.page_method_name = :page
# config.param_name = :page
end
The params_on_first_page option not being present, I tried to add it before the end, this way : config.params_on_first_page = true
However, this doesn't seem to work, and I can't launch my rails server anymore. It gives me the following error :
Exiting
/home/vincent/workspace/bam-rails/config/initializers/kaminari_config.rb:10:in block in <top (required)>': undefined methodparam_on_first_page=' for #Kaminari::Configuration:0x0055c09deaddb0 (NoMethodError)
Did you mean? param_name=
I am using the version 0.17.0 of Kaminari. Am I missing something ?
Thank you in advance for your help.
Solved here : https://github.com/amatsuda/kaminari/issues/831
I missed to add github: "amatsuda/kaminari" in my Gemfile, whereas the option was only available on branch master.
Im trying delayed job to work on production(Using Windows, Rails
4.2.5, ActiveRecord as my ORM) In my gemfile I add the following:
gem 'delayed_job'
gem 'delayed_job_active_record'
gem 'daemons'
In development to use delayed job I type this in cmd.
rake jobs:work
And its working fine. To use in production, I add bin/delayed_job.rb,
and in my initialiazer I add delayed_job.rb. Inside my
initialiazers/delayed_job.rb
Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 60
Delayed::Worker.max_attempts = 3
Delayed::Worker.max_run_time = 5.minutes
Delayed::Worker.read_ahead = 10
Delayed::Worker.default_queue_name = 'default'
Delayed::Worker.delay_jobs = !Rails.env.test?
Delayed::Worker.raise_signal_exceptions = :term
Delayed::Worker.logger = Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
if Rails.env.production?
system 'RAILS_ENV=production bin/delayed_job start'
system 'RAILS_ENV=production bin/delayed_job stop'
end
When I run the program in production DelayedJob just creating record in the database, but it is not running. There is no logs recorded in my delayed_job.log
Here is updated code.
Delayed::Worker.destroy_failed_jobs = false
Delayed::Worker.sleep_delay = 60
Delayed::Worker.max_attempts = 3
Delayed::Worker.max_run_time = 5.minutes
Delayed::Worker.read_ahead = 10
Delayed::Worker.default_queue_name = 'default'
Delayed::Worker.delay_jobs = !Rails.env.test?
Delayed::Worker.raise_signal_exceptions = :term
Delayed::Worker.logger = Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
if Rails.env.production?
system "RAILS_ENV=production #{Rails.root}/bin/delayed_job start"
system "RAILS_ENV=production #{Rails.root}/bin/delayed_job start"
end
I just change the Rails_env=development to production and added the Rails.root, Iforgot to say that Im using Jruby 9.0.4.0 (2.2.2)
I think you are missing the absolute path.
You should also only have one delayed_job start call.
If you want two instances of delayed_job you should instead pass on the number of workers (updated my example to show that)
Try to change it to:
if Rails.env.production?
system "RAILS_ENV=development #{Rails.root}/bin/delayed_job start --number_of_workers=2"
end
After that change, and a restart in production, run:
$ ps -ef | grep delayed_job
...should show:
$ your_user 4242 1 0 10:12 ? 00:00:00 delayed_job
ENV
Rails 3.2.13
ruby 1.9.3p392
airbrake 3.1.11
rake, version 10.0.4
my config file and rake task file
#lib/tasks/raise.rake
namespace :raise do
desc "raise sth"
task :sth => :environment do
num = 1 / 0 # should raise error
end
end
end
# config/initializers/airbrake.rb
Airbrake.configure do |config|
config.rescue_rake_exceptions = true
config.api_key = Setting.airbrake_api
end
I hope run RAILS_ENV=production rake raise:sth send message to my airbrake, but nothing happen, just output divided by 0, not anything call stack info.
please help me, thank you ~