Delayed Job Queue, Priority, Worker - ruby-on-rails

Question

I have around 10 queues in my code, each having separate workers. All have default priorities - 1.
Currently, the requests coming from user (UI) & rake tasks are going in same queue. Suppose, request coming for fetching data from user & rake task are going in same queue with priority 1 & both are executing one by one depends on request time.
But I want to run the request from user first & then the rake task. How can I manage this ?
I also want to understand if I create a separate queue for rake task & give it low priority, then how it will work. Will it run along with another queues or wait for them to execute first?
Delayed::Worker.queue_attributes = {
high_priority: { priority: -10 },
low_priority: { priority: 10 }
}

Answer 1

This article could help you further - it also has coding examples that you can use to try with your own code:
https://phil.tech/2016/tips-on-sidekiq-queues/
Basically, if you want to run the request from users first, you should make a queue for them and assign it a higher priority. No need to assign -10, just start from 0 and work your way up.
But also keep in mind that you should not have more than a handful queues, so it might make sense to bundle different priorities together:
I don't recommend having more than a handful of queues. Lots of queues makes for a more complex system and Sidekiq Pro cannot reliably handle multiple queues without polling. M Sidekiq Pro processes polling N queues means O(M*N) operations per second slamming Redis.
https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues
Finally, if one part of your code has to run before another, you could also enqueue the rake task from the job that handles the user requests.

Answer 2

But I want to run the request from user first & then the rake task.
How can I manage this ?
Recommend change Delayed::Worker.default_priority to a positive number, for example: 5, then define a priority method in the user workers return a lower number
class UserRequestJob < ApplicationJob
def priority
1 # lower numbers run first
end
...
end
I also want to understand if I create a separate queue for rake task &
give it low priority, then how it will work. Will it run along with
another queues or wait for them to execute first?
It will run higher priority jobs first, and then lower priority jobs, in fact queue name doesn't matter after a job is enqueued, unless you have dedicated workers, when you define queues with priority:
Delayed::Worker.queue_attributes = {
high_priority: { priority: -10 },
low_priority: { priority: 10 }
}
and then you enqueue a job to a queue, delayed_job will get the priority by queue name, as you can see in job_preparer.rb
References:
https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters.html
https://github.com/collectiveidea/delayed_job#named-queues