I've been getting a whole bunch of uninitialized constant errors lately and I can't figure out why. Below is a specific example. In this example, I am calling a job from within a job. But I am getting a similar uninitialized constant error will many of my other jobs. All jobs are in app/jobs. Am I missing something? Sidekiq has been working just fine until recently.
I've purged my Heroku cache and killed all retries in Sidekiq and I'm still getting these issues. There's something really strange here. With another error I'm getting related to a sidekiq job, I'm getting "wrong number of arguments (given 2, expected 1)". I updated the function in question to receive two arguments weeks ago. Is it possible that Sidekiq is somehow stuck on a cached version of the codebase?
Ruby version: ruby 2.5.3p105
Sidekiq version: 6.0.7
app/jobs/process_email_notifications_job.rb
class ProcessEmailNotificationsJob < ApplicationJob
queue_as :default
def perform
user_ids = UserNotification.where(is_read: false).pluck(:user_id).uniq
user_ids.each do |user_id|
ProcessIndividualEmailNotificationsJob.perform_later user_id
end
end
end
app/jobs/process_individual_email_notifications_job.rb
class ProcessIndividualEmailNotificationsJob < ApplicationJob
queue_as :default
def perform(user_id)
...
end
end
Error message:
2020-05-06T20:07:45.720Z pid=56028 tid=owp0sdcm8 DEBUG: enqueued retry: {"retry":true,"queue":"production_default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"ProcessIndividualEmailNotificationsJob","args":[{"job_class":"ProcessIndividualEmailNotificationsJob","job_id":"4b31bc4f-d034-4190-b24f-d0464cf81df0","provider_job_id":null,"queue_name":"production_default","priority":null,"arguments":[988],"executions":0,"locale":"en"}],"jid":"0ecc861f5870a7b9a70f176f","creat
4:07:45 PM sidekiq.1 | > ed_at":1588794273.2726498,"enqueued_at":1588795006.4009435,"error_message":"uninitialized constant ProcessIndividualEmailNotificationsJob\nDid you mean? ProcessEmailNotificationsJob","error_class":"NameError","failed_at":1588794279.9911764,"retry_count":5,"retried_at":1588795006.763224}
Initializer:
require 'sidekiq'
require 'sidekiq/web'
Sidekiq.configure_client do |config|
config.redis = { :size => 5 }
end
Sidekiq.configure_server do |config|
config.redis = { :size => 25 }
end
Sidekiq::Web.set :sessions, false
sidekiq.yml
:concurrency: 18
development:
:verbose: true
:queues:
- [development_priority, 2]
- development_default
- development_mailers
staging:
:queues:
- [staging_priority, 2]
- staging_default
- staging_mailers
production:
:queues:
- [production_priority, 2]
- production_default
- production_mailers
OK. I seem to have resolved the issue by adding a namespace to sidekiq on redis per what was suggested here: https://github.com/mperham/sidekiq/issues/2834
Related
In my Rails 6 app I've got ActiveRecord callback (after_create) which should call SyncProductsWorker (Sidekiq worker) each time when record ProductsBatch is created:
ProductsBatch model with after_create:
module Imports
class ProductsBatch < ImportsRecord
attr_accessor :product_codes
after_create :enqueue
def enqueue
::Imports::SyncProductsWorker.perform_async(product_codes, self.id)
end
end
end
# base class for the above model
class ImportsRecord < ApplicationRecord
self.abstract_class = true
connects_to database: { writing: :imports }
end
SyncProductsWorker class:
module Imports
class SyncProductsWorker
include Sidekiq::Worker
sidekiq_options queue: 'imports_sync'
def perform(list, id)
# do some things
end
end
end
config/sidekiq.yml
:max_retries: 16
:queues:
- default
- imports_sync
- imports_fetch_all
:dynamic: true
config/initializers/sidekiq.rb
redis = { url: ENV['REDIS_URL'], ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE } }
Sidekiq.configure_server do |config|
config.redis = redis
end
Sidekiq.configure_client do |config|
config.redis = redis
end
Everything works well locally but when I deploy code into Heroku worker doesn't seem to call. The strange thing is that, based on Heroku logs, it doesn't even work when I try to call it directly from Heroku rails console:
heroku run rails console --app test-app
› Warning: heroku update available from 7.47.7 to 7.60.2.
Running rails console on ⬢ test-app... up, run.8489 (Hobby)
Loading production environment (Rails 6.1.4.1)
irb(main):013:0> ::Imports::SyncProductsWorker.perform_async(['11'], 10)
=> "5edf93e27fa2f41245587d49"
But nothing happens inside Heroku logs:
2022-06-06T22:02:00.240650+00:00 app[worker.1]: [ActiveJob] [ProductAvailabilityAdjusterJob] [25c15f9d-e032-438e-bda8-16ffd557cc32] Performed ProductAvailabilityAdjusterJob (Job ID: 25c15f9d-e032-438e-bda8-16ffd557cc32) from Sidekiq(default) in 5.44ms
2022-06-06T22:02:00.240789+00:00 app[worker.1]: pid=4 tid=2xbk class=ProductAvailabilityAdjusterJob jid=91ad7e69e061df9f2f681ef3 elapsed=0.006 INFO: done
Is there anything special I should do to make this worker work on Heroku?
I followed the sideqik gem tutorial docs and heroku Redistogo addon docs
initializers/sidekiq.rb:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redistogo:xxx.redistogo.com:10076/' }
end
Sidekiq.configure_client do |config|
config.redis = { url: 'redis://redistogo:xxx.redistogo.com:10076/' }
end
app/workers/hard_woker.rb:
class HardWorker
include Sidekiq::Worker
def perform(shop_domain, webhook)
#performing stuffs
end
Webhook I am putting into background job (trying to at least):
class OrdersCreateJob < ActiveJob::Base
def perform(shop_domain:, webhook:)
shop = Shop.find_by(shopify_domain: shop_domain)
shop.with_shopify_session do
HardWorker.perform_async(shop_domain, webhook)
end
end
end
Procfile:
hardworker: bundle exec sidekiq -t 25
There are no errors in console.
Is something wrong here, did I miss something?
My queue:
irb(main):003:0> Sidekiq::Queue.all
=> [#<Sidekiq::Queue:0x000055b53a2d0920 #name="default", #rname="queue:default">]
I assume this means nothing is in the queue?
My goal is to take all my my CreateOrderWebhook code (which is almost 500 lines) into a background job to put less strain on the app and allow webhooks /prevent webhooks from being blocked
So I just migrated to Rails 5.1.4 and I'm trying to make Active Job work, but the jobs are just stuck in the queue and never processed.
rails: 5.1.4
ruby: 2.4.3
sidekiq: 5.0.5
redis: 4.0.1
sidekiq.yml
---
:verbose: true
:concurrency: 5
:timeout: 60
development:
:concurrency: 25
staging:
:concurrency: 50
production:
:concurrency: 5
:queues:
- default
- [high_priority, 2]
sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = {url: ENV['ACTIVE_JOB_URL'], network_timeout: 5}
end
Sidekiq.configure_client do |config|
config.redis = {url: ENV['ACTIVE_JOB_URL'], network_timeout: 5}
end
Here the is how I perform the task from the rails console:
TestJob.perform_later
TestJob.rb content:
class TestJob < ApplicationJob
queue_as :default
def perform(*args)
Rails.logger.debug "#{self.class.name}: I'm performing my job with arguments: #{args.inspect}"
end
end
The jobs are just stuck in the queue and never processed:
Have you started the worker, eg in development it might be:
bundle exec sidekiq
If in production Heroku should do this for you, if you have configured your Procfile, eg:
web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq
You can also use your Procfile in development with foreman, eg:
foreman start -f Procfile
I'm creating Elasticsearch callbacks using Sidekiq. I started Redis and Sidekiq, and created an object in rails console but it seems like perform_sync is not firing. If I add require 'sidekiq/testing';Sidekiq::Testing.inline! to sidekiq.rb, it logs and raises an error as I expect though. Am I missing something?
Ruby 2.3.0
Rails 5.0.2
Sidekiq 4.2.9
Redis 3.2.0
sidekiq.rb
REDIS_URL = 'localhost'
REDIS_PORT = '6379'
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{REDIS_URL}:#{REDIS_PORT}" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{REDIS_URL}:#{REDIS_PORT}" }
end
indexable.rb
module Indexable
extend ActiveSupport::Concern
included do
def index_elasticsearch
Rails.logger.debug 'call'
Rails.logger.debug Indexer::Logger
Rails.logger.debug Indexer::Client
Rails.logger.debug self.id.to_s
Indexer.perform_async(:index, self.id.to_s) # nothing happens here
Rails.logger.debug 'after'
self
end
end
class Indexer
include Sidekiq::Worker
sidekiq_options queue: :elasticsearch, retry: false, backtrace: true
Logger = Sidekiq.logger.level = Logger::DEBUG ? Sidekiq.logger : nil
raise 'No config/elasticsearch.yml' unless File.exists? "config/elasticsearch.yml"
erb = ERB.new( File.read('config/elasticsearch.yml') ).result
config = YAML.load(erb)[Rails.env].symbolize_keys
config.merge! logger: Logger
Client = Elasticsearch::Client.new(config)
def perform(operation, record_id)
Rails.logger.debug [ operation, "ID: #{record_id}"]
raise
end
end
end
some_mongoid_class.rb
class SomeMongoidClass
...
include ::Indexable
...
after_save :index_elasticsearch
end
console
bundle exec sidekiq -e development --queue elasticsearch --verbose
...Booting Sidekiq 4.2.9 with redis options {:url=>"redis://localhost:6379"}...
2017-11-10T ... DEBUG: {:queues=>["elasticsearch"], :labels=>[], :concurrency=>25, :require=>".", :environment=>"development", :timeout=>8, ...
rails console
SomeMongoidClass.create(...)
before
#<Logger:0x...
#<Elasticsearch::Transport::Client:0x...
BSON::ObjectId('...')
after
This problem was solved as soon as I posted the last comment.
It was just a silly mistake. I didn't put the worker file into the workers folder of the app!
I have a Sidekiq worker that syncs the data of a Rails 3.2 application with a remote database. When I just execute it from the rails console, everything works as expected. I can verify that it connects to the remote DB and pulls in the data I am expecting.
The moment I try to set up the following:
recurrence {daily.hour_of_day(20).minute_of_hour(02)}
I see the following in Sidekiq but no signs of completion/execution:
2014-05-14T00:02:18Z 6747 TID-dvc2c INFO: [Sidetiq] Enqueue: BackendWorker (at: 1400112120.0) (last: -1.0)
I've got retries set to false (which I would like to change) currently, and I have attempted to clear out anything in redis before setting up another test run again.
Is there a way to squeeze some more information about what is going on? Is there a Sidekiq flag I can use to get some additional debug information?
Thanks in advance
Environment:
Rails 3.2
Ruby 2.1
Sidetiq 0.5.0
Sidekiq 2.17.7 (due to an issue with using 3.0)
UPDATE (5/14/2014):
I decided to test a simple example with Sidetiq and just have a worker that creates a DB entry with ActiveRecord:
class BackendTest
include Sidekiq::Worker
include Sidetiq::Schedulable
sidekiq_options :retry => false, :backtrace => true
recurrence {
daily.hour_of_day(11).minute_of_hour(05)
}
def perform
Book.create(title: "test", author: "testing", price: 9.99)
end
end
Another experiment I did was just trying to get Sidetiq and Sidekiq to just show something with logger.info:
class BackendTest
include Sidekiq::Worker
include Sidetiq::Schedulable
sidekiq_options :retry => false, :backtrace => true
recurrence {
daily.hour_of_day(16).minute_of_hour(00)
}
def perform
logger.info "test"
logger.info "test"
end
end
The jobs just still enters the default queue without being started or completed. Everything still works if I just fire it off from the rails console with perform_async, Sidekiq::Client.push, and Sidekiq::Client.enqueue still work from console.
I am a thoroughly confused newbie at this point.
Output:
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: start
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: test
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: test
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: done: 0.002 sec
After poking around other people's projects on github, I saw something interesting and tried it out and it seemed to solve my problem with Sidetiq.
I created /config/initializers/sidekiq.rb with the following content inside of it:
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://localhost:6379/3', :namespace => 'sidekiq' }
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://localhost:6379/3', :namespace => 'sidekiq' }
end
Job tasks enqueue and execute as expected.