Currently I'm restarting my sidekiq jobs using a cron job that triggers the method below
class RestartSidekiqJobs
require 'sidekiq/api'
class << self
def restart_jobs
Sidekiq::ScheduledSet.new.clear
channels = Channel.all
channels.each do |channel|
FetchMoreDataJob.perform_later(channel: channel)
end
end
end
end
I'm not convinced however that this is the best way to handle this so I figured I'd see if there were better solutions out there.
Related
In my Rails 6 API only app I've got FetchAllProductsWorker background job which takes around 1h30m.
module Imports
class FetchAllProductsWorker
include Sidekiq::Worker
sidekiq_options queue: 'imports_fetch_all'
def perform
# do some things
end
end
end
During this time the frontend app sends requests to the endpoint on BE which checks if the job is still running. I need to send true/false of that process. According to the docs there is a scan method - https://github.com/mperham/sidekiq/wiki/API#scan but none of these works for me even when worker is up and running:
# endpoint method to check sidekiq status
def status
ss = Sidekiq::ScheduledSet.new
render json: ss.scan('FetchAllProductsWorker') { |job| job.present? }
end
The console shows me:
> ss.scan("\"class\":\"FetchAllProductsWorker\"") {|job| job }
=> nil
> ss.scan("FetchAllProductsWorker") { |job| job }
=> nil
How to check if particular sidekiq process is not finished?
Maybe this will be useful for someone. Sidekiq provides programmatic access to the current active worker using Sidekiq::Workers https://github.com/mperham/sidekiq/wiki/API#workers
So based on that we could do something like:
active_workers = Sidekiq::Workers.new.map do |_process_id, _thread_id, work|
work
end
active_workers.select do |worker|
worker['queue'] == 'imports_fetch_all'
end.present?
Something simple, I'm sure but I've been searching and can't find an answer.
in brief: I want to set up a daily mailer to email a lists of tasks daily.
I have a worker (scheduled every minute, and only puts'ing for dev):
class DailyReminderWorker
include Sidekiq::Worker
include Sidetiq::Schedulable
recurrence do
hourly.minute_of_hour(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59)
end
def perform
User.find_each do |user|
#user = user.name
puts "user name is #{#user}"
#reminder = Remindarrr.where(user_id: user.id)
#reminder.each do |r|
puts r.title
end
end
end
end
I know I can call this with the following in a controller:
DailyReminderWorker.perform_async
This works and outputs every minute but every time the page refreshes it also fires. (less useful for a daily mailer.)
How do you call the worker to queue the job without it firing immediately?
Where would you put the perform.async?
Thanks a lot!
If you are on a Linux platform, just use cron. very easy to setup.
I need help developing a worker with sidekiq for this situation:
I have a helper that looks like this:
module UploadsHelper
def save_image
response = HTTParty.get(ENV['IMAGE_URI'])
image_data = JSON.parse(response.body)
images = image_data["rows"].map do |line|
u = Image.new
u.description = line[5]
u.image_url = line[6]
u.save
u
end
images.select(&:persisted?)
end
end
In my app/views/uploads/index.html.erb I just do this
<% save_image %>
Now, when a user visits the uploads/index page the images are saved to the database.
The problem is that the get request to the API is really slow. I want to prevent request timeouts by moving this to a background job with sidekiq.
This is my workers/api_worker.rb
class ApiWorker
include Sidekiq::Worker
def perform
end
end
I just don't know the best way to proceed from here.
Performing this task using a Sidekiq worker implies that the task will run in async, and thus, it will not be able to return the response immediately, which is being sent by images.select(&:persisted?).
First of all, instead of calling save_image, you need to call the worker's perform_async method.
<% ApiWorker.perform_async %>
This will enqueue a job in Sidekiq's queue (your_queue in this example). Then in worker's perform method, call the save_image method of UploadsHelper.
class ApiWorker
include Sidekiq::Worker
sidekiq_options queue: 'your_queue'
include UploadsHelper
def perform
save_image
end
end
You may want to save the response of save_image somewhere. To get Sidekiq start processing the jobs, you can run bundle exec sidekiq from your app directory.
I have a question about how can you run sidekiq job only once f.e. just on the start of rails web-app. One thing I tried is to initialize redis semaphore in the config/initializer.rb and then used it in job, but it kinda didn't work. Here's the code I'm trying to get working:
config/initializer.rb
SEMAPHORE = Redis::Semaphore.new(:semaphore_name, :host => "localhost")
queue_worker.rb
class QueueWorker
include Sidekiq::Worker
def perform
logger.info 'Start polling'
unless SEMAPHORE.locked?
logger.info 'Im here only once'
SEMAPHORE.lock
end
end
end
root_controller_action
QueueWorker.perform_async
Well another variant I don't know if it's possible to run sidekiq job in the initializer. If you can do that, there's no need in semaphore at all.
Thx for answering.
I have problem with sending recurring mails with Sidekiq and Sidetiq. I'v tried almost everything and I didn't find the solution.
I have Sidekiq worker which looks like this:
class InvoiceEmailSender
include Sidekiq::Worker
include Sidetiq::Schedulable
recurrence {minutely(2)}
def perform(invoice_id, action)
#invoice = Invoice.find(invoice_id.to_i)
if action == "invoice"
send_invoice
else
send_reminder
end
end
private
def send_invoice
if #invoice.delivery_date == Date.today
InvoiceMailer.delay.send_invoice(#invoice)
else
InvoiceMailer.delay_for(#invoice.delivery_date.to_time).send_invoice(#invoice)
end
end
def send_reminder
InvoiceMailer.delay.send_invoice_reminder(#invoice) unless #invoice.paid?
end
end
End in controller I use it in this way:
InvoiceEmailSender.perform_async(#invoice.id, "invoice")
And when I try to send this emails I have the following error in sidekiq console:
2014-08-26T05:36:01.107Z 4664 TID-otcc5idts WARN: {"retry"=>true, "queue"=>"default", "class"=>"InvoiceEmailSender", "args"=>[1409031120.0, 1409031240.0], "jid"=>"06dc732831c24e1a6f78d929", "enqueued_at"=>1409031120.7438812, "error_message"=>"Couldn't find Invoice with 'id'=1409031120", "error_class"=>"ActiveRecord::RecordNotFound", "failed_at"=>1409031249.1003482, "retry_count"=>2, "retried_at"=>1409031361.1066737}
2014-08-26T05:36:01.107Z 4664 TID-otcc5idts WARN: Couldn't find Invoice with 'id'=1409031120
2014-08-26T05:36:01.107Z 4664 TID-otcc5idts WARN: /home/mateusz/.rvm/gems/ruby-2.0.0-p0#rails4/gems/activerecord-4.1.2/lib/active_record/relation/finder_methods.rb:320:in `raise_record_not_found_exception!'
In sideiq web monitor in scheduled tab it looks like this:
Please help because I have not idea what is going on...
The data passed in looks like epoch timestamps, turns out Sidetiq passes the last and current times as the 2 parameters to your worker, according to the documentation.
I'm not sure how you go about having custom parameters with a scheduled worker, you'll probably need a different strategy like instead of trying to create more scheduled workers, just have 1 (or two, since it looks like you made this class do 2 jobs) scheduled worker(s) that processes a list of work to do every so often.