I want the worker to run on a specific date. I am able to schedule jobs in sidekiq. And sidekiq UI also shows scheduled jobs perfectly. But due to unknown reason my data on sidekiq (processed count,scheduled jobs etc.) gets deleted and everything is reset to 0 in sidekiq UI. Can someone please help me understand this issue.
I suspect you are calling flush on Redis, clearing all your data.
Related
I have a worker class that finds out who to send notifications and then creates notifications at which point push will send out the pushes.
But i have a problem with Sidekiq, that is when i do SomeWorker.perform_in(5.seconds,someID, otherID) it does show up in scheduled on the sidekiq dashboard. but after 5 seconds the job disappears. the count for processed does not update nor it shows up in failed. Infact it just won't show up in any counter. Although sometimes the job is processed and i see the counter and logs and the push. but most of the times it just disappears to be found nowhere. Where is the problem?
You have an old Sidekiq process consuming the job. Make sure all old processes are dead.
I would like to know how to kill many Sidekiq jobs from the same worker at once.
I deployed a bug on a production environment and there are queued jobs that are bugging out. I can simply fix the bug and deploy again, but the jobs are time-sensitive (they send out SMS alert to people).
When the bug is gone, the jobs will be executed and many people will get outdated SMS alerts. So I would like to kill all the jobs from that worker before deploying my fix.
Any suggestions? The buggy jobs are enqueued with many other jobs and I can't just remove all jobs from one queue.
Ideally you should enqueue those messages to a different queue so you can clear that queue on its own. There's no other efficient way to remove a set of jobs.
If I forget to start delayed_job workers on server, the delay jobs will always pending and seems I can't get any errors from Delayed::Job API. is there any easy way to debug with this mistake? I have a dashboard to list the failed background jobs for admins, it could be great to have an alert if there is no worker running. Thanks!
Hmmm, I can find a empty job with Delayed::Job in this case, and it will continue if the worker working again, so I think it should be fine.
I am running a rails app and using redis for jbulder's cache and sidekiq queue. I use sidekiq to send emails asyncly, everytime when I try to send mass emails, say 20k emails in background using sidekiq, after a while, all the background jobs in sidekiq queue are cleared, left 0 jobs in queue.
I filed an issue on sidekiq github page(link), the author said it could be something or someone flushing my redis. There's no one flush redis manually and I wonder how can I find when and how redis gets flushed.
I've checked redis log file with nothing strange.
Here is the documentation on changing certain commands. Perhaps consider changing flushAll and flushDB to something abnormal.
I'm using the sidekiq gem to process background jobs in Rails. For some reason, the job just hang after a while -- the process either becomes unresponsive, showing up on top but not much else, or mysteriously vanishes, without errors (nothing is reported to airbrake.io).
Has anyone had experience with this?
Use the TTIN signal to get a backtrace of all threads in the process so you can figure out where the workers are stuck.
https://github.com/mperham/sidekiq/wiki/Signals
I've experienced this, and haven't found a solution/root cause.
I couldn't resolve this cleanly, but came up with a hack.
I configured God to monitor my Sidekiq processes, and to restart them if a file changed.
I then setup a Cron Job that ran every 5 minutes that checked all the current Sidekiq workers for a queue. If a certain % of the workers had a start time of <= 5 minutes in the past, it meant those workers hung for some reason. If that happened, I touched a file, which made God restart Sidekiq. For me, 5 minutes was ideal but it depends on how long your jobs typically run.
This is the only way I could resolve hanging Sidekiq jobs without manually checking on them every hour, and restarting it myself.