Say I have a worker class that looks like this:
class BuilderWorker
include Sidekiq::Worker
sidekiq_options retry: false
def perform(order_id)
if(order_id == 5)
# How can I finish the job here? Say I want to finish it with a status of FAIL or COMPLETE.
end
end
end
I am looking for a way to finish a job from the Worker class, and when finished give it the status of FAILED. The finish should be quiet, (not raising an exception)
With Sidekiq there are only two job results:
success - a job returns from the perform method without error
failure - a job raises an error, going onto the retry queue to be retried in the future
Your complicated scenario is called application logic and Sidekiq cannot provide it. It is up to you to write that logic.
Related
I am trying to implement an API endpoint that would queue a request and return immediately.
I am using the gem https://rubygems.org/gems/activejob/versions/5.2.0 (I am on an old version for historical reasons).
I have defined a job that looks something like:
class Service::ExportBooks::Job < ActiveJob::Base
def perform
## ... Do the job
rescue StandardError
binding.pry
raise
end
end
In the controller, I am calling:
Service::ExportBooks::Job.perform_later
The job gets called synchronously and the controller gets even any errors raised by the job.
I've also tried other options such as:
job = Service::ExportBooks::Job.new
job.enqueue(wait: 5.seconds)
but it does the same, the job is not enqueued, is immediately executed.
UPDATE:
It looks like the method Resque.inline? returns true and so the execution is inline and not async. How can I make sure that it's async? I tried to set Resque.inline = false manually and the job was queued but it wasn't executed...
I have started a worker using the command:
QUEUE=* PIDFILE=./tmp/resque.pid bundle exec rake environment resque:work
Two things to do here.
Make sure Resque.inline = false.
Start up the resque workers in another process. See here.
This will get the job enqueued and run on the worker process.
I have a SchedulerJob:
class SchedulerJob < ActiveJob::Base
queue_as :scheduler
def perform
logger.debug("Start")
DelayedJob.set(wait: 10.seconds).perform_later
logger.debug("After Job 1")
DelayedJob.set(wait: 20.seconds).perform_later
logger.debug("After Job 2")
DelayedJob.set(wait: 30.seconds).perform_later
logger.debug("End")
end
end
and a DelayedJob:
class DelayedJob < ActiveJob::Base
queue_as :delayed_jobs
def perform
puts "I'm done"
end
end
If I call SchedulerJob.new.perform the job runs in just a few milliseconds. If I call SchedulerJob.perform_later to run the job in Sidekiq it takes about 90 seconds to finish, and by looking at the logs I can tell that each of those .perform_later calls takes about 30 seconds each.
Why would this happen?
By definition,
perform_later will be performed as soon as queuing system is free
perform will be performed regardless of your queue status
My guess is that when you call perform_later, there are some jobs running in the queue.
The problem was the initializer. This was the solution:
My guess is you’ve destroyed all concurrency by hacking in one global $REDIS connection into the pool. Don’t do that. Let Sidekiq create and manage the pool of connections.
I encountered an issue with sidekiq: I want to set timeout for jobs, meaning when a job has process time greater than timeout then that job will stop.
I have searched how to set global timeout config in file sidekiq.yml. But I want to set separate timeout for difference separate jobs meaning one of classes to define worker will have particular timeout config.
Can you help me. Thanks so much.
There's no approved way to do this. You cannot stop a thread safely while it is executing. You need to change your job to check periodically if it should stop.
You can set network timeouts on any 3rd party calls you are making so that they time out.
You can wrap your job code inside a timeout block like the below:
Timeout::timeout(2.hours) do
***. do possibly long-running task *****
end
The job will fail automatically after 2 hours.
This is the same method as yassen suggested, but more concrete.
class MyCustomWorker
include Sidekiq::Worker
def perform
begin
Timeout::timeout(30.minutes) do # set timeout to 30 minutes
perform_job()
end
rescue Timeout::Error
Rails.logger.error "timeout reached for worker"
end
end
def perform_job
# worker logic here
end
end
I've set up Sidekiq to monitor some asynchronous and scheduled tasks.
When I queue a job I can see it on the web monitoring tool.
Here is an example of a job
class HardWorker
include Sidekiq::Worker
def perform(name, count)
raise "Error"
end
end
If I then run HardWorker.perform_in(10.second,'bob', 5) and the job fails (which it always does intentionally), it seems to disappear from the web GUI. 'Failed', 'retries', 'processed' etc don't go up. None of the graphs change.
Here is what the log spits out:
2015-04-19T11:03:40.013Z 1438 TID-3fk WARN: uninitialized constant HardWorker
This makes sense as I created the class through console rather than in my project but shouldn't sidekiq show this as a failed job?
I've also tried force setting the following:
sidekiq_options :retry => false
sidekiq_options :failures => true
Anyone got any suggestions how to get the web app to show those failed jobs?
Turns out because I wrote the class in rails c the subsequent threads didn't know about the class. This was intentional but I didn't think about the fact that within the class it's involving Sidekiq meaning in my scenarios when it failed, it never spoke to Sidekiq reporting the failure.
I have a sidekiq worker that shouldn't take more than 30 seconds, but after a few days I'll find that the entire worker queue stops executing because all of the workers are locked up.
Here is my worker:
class MyWorker
include Sidekiq::Worker
include Sidekiq::Status::Worker
sidekiq_options queue: :my_queue, retry: 5, timeout: 4.minutes
sidekiq_retry_in do |count|
5
end
sidekiq_retries_exhausted do |msg|
store({message: "Gave up."})
end
def perform(id)
begin
Timeout::timeout(3.minutes) do
got_lock = with_semaphore("lock_#{id}") do
# DO WORK
end
end
rescue ActiveRecord::RecordNotFound => e
# Handle
rescue Timeout::Error => e
# Handle
raise e
end
end
def with_semaphore(name, &block)
Semaphore.get(name, {stale_client_timeout: 1.minute}).lock(1, &block)
end
end
And the semaphore class we use. (redis-semaphore gem)
class Semaphore
def self.get(name, options = {})
Redis::Semaphore.new(name.to_sym,
:redis => Application.redis,
stale_client_timeout: options[:stale_client_timeout] || 1.hour,
)
end
end
Basically I'll stop the worker and it will state done: 10000 seconds, which the worker should NEVER be running for.
Anyone have any ideas on how to fix this or what is causing it? The workers are running on EngineYard.
Edit: One additional comment. The # DO WORK has a chance to fire off a PostgresSQL function. I have noticed in logs some mention of PG::TRDeadlockDetected: ERROR: deadlock detected. Would this cause the worker to never complete even with a timeout set?
Given you want to ensure unique job execution, i would attempt removing all locks and delegate job uniqueness control to a plugin like Sidekiq Unique Jobs
In this case, even if sidetiq enqueue the same job id twice, this plugin ensures it will be enqueued/processed a single time.
You might also try the ActiveRecord with_lock mechanism: http://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html
I have had a similar problem before. To solve this problem, you should stop using Timeout.
As explained in this article, you should never use Timeout in a Sidekiq job. If you use Timeout, Sidekiq processes and threads can easily break.
Not only Ruby, but also Java has a similar problem. Stopping a thread from the outside is inherently dangerous, regardless of the language.
If you continue to have the same problem after deleting Timeout, check that if you are using threads carelessly in your code.
As Sidekiq's architecture is so sophisticated, in almost all cases, the source of the bug is outside of Sidekiq.