This question already has answers here:
how to remove a specific class of jobs from a sidekiq queue?
(3 answers)
how to delete a job in sidekiq
(8 answers)
Closed 6 months ago.
Hi friends I am using Sidekiq for background work, in this case to send an email at a certain time if the user has not completed some steps, but I want to stop or delete that Worker for a case where the user disconnects their email account.
Is there any way to eliminate all of a worker’s jobs for a specific queue in Sidekiq ?
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am developing a system in which the API should handle simultaneous, continuous by rails 4.0
In system, each user has 3 scripts to be run in background. The scripts grab the user's information from DB to call API repeatedly and process transaction. Currently I am using cronjob (whenever gem) to run scripts in the background for each individual user
So my problem is when the system has 1,000 people, I need to run 3000 cronjobs.
I think this system will have problems. Can anyone help me solve this problem?
At this point you have a system that performs some tasks periodically, and the amount of work your system has to handle (let's say, per hour) is less than the amount of work it could handle.
However, the amount of work increases with the number of users in your system so, as you have already guessed, there will be a moment when the situation will be critical. Your system will not be able to handle all the tasks it has to do.
One way to solve this problem is adding more machines to your system, that is, if you are currently using a single machine to run all your tasks, consider adding another one and split the job. You can split the job between the machines in a number of ways, but I would use a consumer-producer approach.
You will need to use a queue manager where your producer periodically sends a batch of tasks to be done (you can still use whenever gem for that) and a number of consumers (1 makes no sense, 2 would be OK by now but you could increase this number) get the tasks done one by one until there is none left.
The manager I like the most is Sidekiq but you can find some others that might match your needs better.
This question already has answers here:
Cannot play cached audio from service worker
(2 answers)
Closed 1 year ago.
Is there any restriction on what kind of files could be cached by a service worker?
Check out https://samdutton.github.io/samples/service-worker/prefetch-video/ which works around this issue by manually created ranged responses.
Fixing this is gated on figuring out what browsers should be doing here, and updating the service worker spec if needed.
Original answer: https://stackoverflow.com/a/37614302/6773912
There are no restrictions on the kind of files you can cache.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I have a Rails application that displays thousands of products.
The products are loaded from product feeds, so the source may be a large XML file or web service API calls.
I want to be able to re-use my models in my existing rails application in my import process.
What are my options in importing data into my Rails application?
I could use sidekiq to fire off rake tasks, but not sure if sidekiq is suitable for tasks that take 10+ minutes to run? Most use cases that I have seen is for sending of emails and other similiar light tasks
I could create maybe a stand-alone ruby script, but not sure how I could re-use my Rails models if I go this route.
Update
My total product could is around 30-50K items.
Sidekiq would be a great option for this as others have mentioned. 10+ minutes isn't unreasonable as long as you understand that if you restart your sidekiq process mid run that job will be stopped as well.
The concern I have is if you are importing 50K items and you have a failure near the beginning you'll never get to the last ones. I would suggest looking at your import routine and seeing if you can break it up into smaller components. Something like this:
Start sidekiq import job.
First thing job does is reschedule itself N hours later.
Fetch data from API/XML.
For each record in that result schedule a "import this specific data" job with the data as an argument.
Done.
The key is the second to last step. By doing it this way your primary job has a much better chance of succeeding as all it is doing is reading API/XML and scheduling 50K more jobs. Each of those can run individually and if a single one fails it won't affect the others.
The other thing to remember is that unless you configure it not to Sidekiq will rerun failed jobs. So make sure that "import specific data" job can be run multiple times and still do the right thing.
I have a very similar setup that has worked well for me for two years.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I am trying to put a function in delay. The delay will start in a loop:
class Test
def check_function1
while a < b do
self.delay.check_function2(t1, t2)
end
end
def check_function(t1, t2)
#SAVING DATA
end
end
If the loop runs twice, will that mix up the data here? Is there any way to rectify? Also can we do any sleep after one loop?
Delayed Job runs a polling daemon in the background that polls the jobs table at regular intervals for new jobs.
If you only run one DJ worker, then all delayed tasks will be processed in sequence and the potential of a race condition isn't there.
If you run multiple workers then you need to take into account race conditions and guard against "mixing up data" (I assume this is what you meant with "mix up the data").
So yes, running multiple delayed jobs has the potential to "mix data" IF you run multiple DJ workers.
See here for a more detailed explanation:
http://ternarylabs.com/2012/04/16/handle-job-queue-workers-concurrency-in-rails/
Sleeping when launching jobs has practically no effect on what happens to the job queue. Generally you would not want to sleep as it doesn't really do anything other than delay the user interface of your Rails app.
This question already has answers here:
Scheduling tasks with rails
(2 answers)
Closed 9 years ago.
We have an application that needs to perform lots of short background tasks (each might schedule more). In the future we might need to run the tasks on multiple servers.
We also need the tasks (and their parameters) to be persistent (stored in a DB) and be able to monitor tasks (status / logs /etc)
Is there a ready made solution that works with Ruby ?
You can try delayed_job or resque.