I'm using Quartz.NET and I'm having trouble finding documentation regarding the lifetime of a job.
Is a new instance of the job class created each time the job is run, or will an existing instance be reused?
What about if the job is currently running when the trigger fires again?
Each firing of a trigger creates a new instance of a job. If two triggers overlap, two instances will be created.
These instances aren't pooled, once the job is done, they become eligible for garbage collection.
Related
I am using Sidekiq to schedule some tasks based on a schedule that the user provides. However, if the user changes the schedule, I want to be able to simply update the old schedule with the new one.
Suggestion one
I saw a suggestion to just find the old job with Sidekiq::ScheduledSet.new.find_job(job_id), but I am trying to avoid having to create a new model just to simply store the job ID and the task.
Suggestion two
Another suggestion I saw was to just have the worker check if the time of the task matches the current time, but that won't work because if the server is offline, it won't process jobs when it returns back online because the time of those delayed jobs won't match the current time.
If I could assign my own job ID, like a hex version of the job name or a padded version of the task ID, then I could easily avoid having to create a new model to store the job IDs. So when the user reschedules a task, then it would be a lot easier.
Other thoughts
Maybe if I could check the job's at attribute and match that with the task, that may work, but I'm not sure how to access that attribute from within the worker without knowing the job ID.
Edit
I just tried to pull the current job's at attribute, but it looks like once the job kicks off, it doesn't exist anymore in Sidekiq::ScheduledSet, so there's no matching this job's time with Task's time from what it seems like.
I am using Sidekiq to schedule some tasks based on a schedule that the user provides...
There's an extension for that. Sidekiq-Scheduler gives you a cron-like schedule configuration file. Then you can alter the schedule as you see fit. This seems like the best option as it avoids having to write your own scheduler interface.
Can I assign my own randomized Job ID to Sidekiq?
Yes, though it's undocumented. You can give Sidekiq::Client.push a jid attribute.
Sidekiq::Client.push('class' => MyWorker, 'args' => [1, 2, 3], 'jid' => ... )
This is not a good way to solve your problem. It's relying on an undocumented feature. And it invites collisions with normal Sidekiq IDs.
Maybe if I could check the job's at attribute and match that with the task, that may work, but I'm not sure how to access that attribute from within the worker without knowing the job ID.
This sounds very error prone. You'd have to store the timestamp in a model anyway. Better to store the job ID in the first place.
I am trying to avoid having to create a new model just to simply store the job ID and the task.
Storing things in models is what Rails does really well. This would seem to be the way to go. It will take a trivial amount of coding, database storage, and processing. You should have a model, view, and controller for your scheduled jobs anyway else how will you create scheduled jobs and view your schedule?
However, the Sidekiq docs notes that find_job is "a slow, inefficient operation. Do not use under normal conditions. Sidekiq Pro contains a faster version." This is because it has to iterate through all jobs.
I had a case where I had to reschedule jobs based on updates from the User. It is actually pretty slow and complicated.
It's simpler to not reschedule, but instead make the old queued tasks no-ops (no operations) and then queue up the new tasks.
This is basically defined by the logic within the task. You'd have to know that the user updated their schedule somehow and check that within the old jobs and based on some if-check, not go through with the job.
I am looking for a way to be able run Active Job serially. Ideally, a long running Job 1 is scheduled to run at a certain time. A similarly running Job 2 is slated to run only after Job 1 completes. Job 3 then waits for Job 2 to run to completion before it starts and so on.
I have to admit that I am rather new to background jobs in Rails but I am already using Active Job with Sidekiq as the job runner for simple fire-and-forget tasks.
I like Active Job because it provides a simple enough interface to dive almost immediately into background jobs processing. I can use Sidekiq without having to define workers, for example.
For reference, I have achieved something similar but it was on .NET using the excellent Hangfire library which has continuations where you pass the ID of a parent job ensuring that the job will run only after the parent job has successfully completed.
It would be nice to have something as clean and simple as that using Sidekiq and Active Job but really any alternative ways to achieve the same thing are welcome. It doesn't have to be Sidekiq and Active Job.
The most straightforward way to to this is to call a third job from within a second job, and the second one from within a first job
I have several quartz scheduled jobs. With the help of attribute DisallowConcurrentExecution they do not execute simultaneously. But if they fire at one time, one job is missed. How to fix it (to put in queue)? I tried to use Mutex, but in result jobs were started and executed simultaneously, interrupting each other. I want one is done, the second will start after/ What to do?
You could use a JobChainingJobListener for this. It listens for your job to finish and then triggers the next job on the list and so on.
Take a look at the code to see how it works if you want to do this on your own.
We have designed our test jobs to some sort of "abstract" test jobs, that run according to a set of parameters. These jobs are triggered use "runner" jobs that simply trigger them with the correct parameters (mostly generated by matrix jobs).
When we run multiple "runners", that all they do is simply trigger the abstract jobs, they occupy much needed workers (especially when it is a matrix job, that creates multiple temporary "runner" jobs).
Is there a way to tell jenkins to not spend a worker on a job that only trigger other jobs, or trigger jobs within the same worker?
It depends on what you use to trigger the jobs
If you use Trigger/call builds on other projects action, it has an option to Block until the triggered projects finish their builds. If that is checked, the triggering parent job will remain running and waiting for the triggered job to finish (thus occupying at least 2 executors). However, if you keep that unchecked, it will launch the triggered job, and the triggering job will end soon after.
I want my builds to wait until triggered jobs are completed, for reporting purposes and such (I don't want that logic in the triggered jobs due to their abstract nature).
What I decided to do, since the triggering jobs are very lightweight, I restricted them all to the master. I allocated a large number of workers to the master, since they won't do much work and they will simply manage the triggering of other jobs.
Configuration
Quartz.net 2.0.1
JobStore: SqlServer
Statefull jobs are running inside a windows service.
There is also a console application that allow firing the same jobs inside service.
I want to initialize correctly quartz scheduler so that it will respect non-concurrent job execution while allowing to fire a certain job immediately from console.
As long as you mark your job with the DisallowConcurrentExecutionAttribute attribute, you can schedule your jobs from any source you want and the scheduler will make sure only one instance runs at a time.