I have an application using Grails Quartz plugin. I need to have two jobs to have multiple instances running, but have separate limitation on number of threads to be used for each job. As far as I understand, I need separate Thread Pools, which is possible by having separate schedulers. However, I cannot figure out how to create multiple schedulers with Quartz plugin.
Assuming you want to use different triggers to start the job multiple times. this works for me.
class MyJob {
static triggers = {
cron name: 'trigger1', cronExpression: "0 30 12 ? * WED"
cron name: 'trigger2', cronExpression: "0 30 12 ? * SAT"
}
def execute() {
// execute task, do your thing here
println "Job executed"
}
}
Finally, about concurrent tasks.
This is from the plug-in page:
By default Jobs are executed in concurrent fashion, so new Job
execution can start even if previous execution of the same Job is
still running.
Quartz plugin 2.0.13
According to the official documentation :
Multiple triggers per job are allowed.
For instance,
class MyJob {
static triggers = {
simple name:'simpleTrigger', startDelay:10000, repeatInterval: 30000, repeatCount: 10
cron name:'cronTrigger', startDelay:10000, cronExpression: '0/6 * 15 * * ?'
custom name:'customTrigger', triggerClass:MyTriggerClass, myParam:myValue, myAnotherParam:myAnotherValue
}
Related
Below is a simplified case.
I have one node named comp01. And I have a Jenkins job named Compatibility.
Compatibility is scheduled as follows:
0 12 * * 1 %IntegrationNode=Software_1
0 17 * * 1 %IntegrationNode=Software_2
0 22 * * 1 %IntegrationNode=Software_3
0 2 * * 2 %IntegrationNode=Software_4
0 7 * * 2 %IntegrationNode=Software_5
The jobs start as scheduled. But sometimes, because of some verification failure, the previous job takes more than expected time. So, the next job starts before the completion of the previous job.
Is there a way available in Jenkins, in which the next scheduled job stays in a queue until previous job is complete? Or can we schedule based on previous job status?
We have tried limiting executors for this job, but when more than a couple of jobs are queued, then the expected behavior is not observed.
We have also tried by creating resource-groups and adding multiple nodes to it, but still, expected behavior is not observed when multiple jobs are in queue.
EDIT-1:
We can't use options { disableConcurrentBuilds() } since we start the job concurrently on different nodes. Here we are struggling to ensure that when a job is started on a node, then the other scheduled jobs for the same node should wait till the current job completes.
Have you tried setting the below option?
options { disableConcurrentBuilds() }
Update
AFAIK there is no OOB solution for your problem. But you can definitely implement something. Without seeing your actual Pipelines I can't give a concrete answer. But here ae some options.
Option 01
Use Lockable Resources and create a resource per Jenkins IntegrationNode and acquire it when running the Job, the next build will wait until the lock is released.
lock(resource: 'IntegrationNode1', skipIfLocked: false) {
echo "Run your logic"
}
Option 02
You can implement a waiting logic to check the status of the previous Build. Here is an sample Pipeline and possible Groovy code you can leverage.
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
echo "Waiting"
def jobName = "JobA"
def buildNum = "92"
waitUntil { !isPending(jobName, buildNum) }
echo "Actual Run"
}
}
}
}
}
def isPending(def JobName, def buildNumber) {
def buildA = Jenkins.instance.getItemByFullName(JobName).getBuild(buildNumber)
return buildA.isInProgress()
}
I have a grails application and a quartz job running on it. The job contains the below code similar to below .
class MyJob{
static triggers = {}
def printLog(msg){
String threadId = Thread.currentThread().getId()
String threadName = Thread.currentThread().getName()
log.info(threadId+" - "+threadName+" : "+msg)
}
def execute(context)
{
printLog("Before Sync");
synchronized(MyJob){
printLog("Inside Sync");
try{
printLog("Before sleep 20 minutes")
Thread.sleep(1200000)
printLog("After sleep")
}catch (Exception e){
log.error("Error while sleeping")
}
}
printLog("After Sync")
}
}
I have scheduled it to trigger a job every minute
I can see in the logs that one thread is getting the synchronized block and then the other jobs start piling up, waiting for the thread to finish, this is working as expected.
The issue here is the jobs stop after 10 minutes by that time it have created 10 Threads. Out of that one is sleeping for 20 minutes and other 9 are waiting for the 1st thread to release the lock. Why is no new jobs created ?
I saw in some answers I can fix the issue by modifying my triggers section like below
static triggers = {
simple repeatInterval: 100
}
I tried the above option and its still showing only 10 jobs.
From where its taking the default configuration of 10 ?
How can i modify the value to do infinitely ?
I am new to grails and quartz, so I have no idea what is happening.
I think the Grails plugin sets the threadCount to 10 in the bundled quartz.properties file, assuming you're using Grails 3 you can override in application.yml like this:
quartz:
threadPool:
threadCount: 25
Grails 2 - application.groovy
quartz {
props {
threadPool.threadCount = 100
}
}
In general, it's not a a good idea to lock the Job thread with sleeps
If you have a job running a long process you must to split it in several jobs in order to release the Thread as soon as posible
I'm actually working in Grails 3.x and I need to use Quartz Cron expressions. I've implemented successfully regular cron expression and works correctly, but now what I need to do is User (from a GUI) change that cron expression for any other he/she wants.
This is my Job
class ScheduleJob
{
static triggers =
{
cron name: 'myTrigger', cronExpression: "*/5 * * * * ?"
}
def execute()
{
println "------ Every 5 seconds"
}
}
I would appreciate so much if someone could help me to know how to do that, thanks for your time. :D
You have a number of options. One thing you can do is call ScheduleJob.schedule(' your cron expression goes here ').
You could also go get the job manager bean from the application context and reconfigure job beans there.
Some of the info at http://www.tothenew.com/blog/removing-triggers-and-rescheduling-a-quartz-2-job-programatically/ might be helpful.
Hi i am new to grails and I am using quartz plug-in for scheduling jobs. I scheduled job for every 60 sec but it is actually taking more than 60 sec some times So in that case one more threads is started and the first thread is still running So can any one tell me how to execute threads sequentially one by one.
When using the Grails Quartz plugin you can simply set the concurrent property to false to avoid concurrent executions of a Job:
class MyJob {
static triggers = {
...
}
def concurrent = false
def execute(context) {
...
}
}
If you are using Quartz as a plain dependency (not as a Grails plugin) you need to extend StatefulJob (Quartz < 2.0) or set the #StatefulJob and #DisallowConcurrentExecution annotations (Quartz >= 2.0).
I want to execute a cron job at a certain time (10pm) every day. Currently I am using following code but it is not working:
class ReviewBatchJob {
private static final logger = LogFactory.getLog(this)
def batchReviewScheduler
static triggers = {
cron cronExpression: "0 0 22 1/1 * ? *"
}
def execute() {
batchReviewScheduler.reviewBatch()
}
}
You should just be able to use an expression of 0 0 22 * * ?.
I'm assuming you're using the quartz plugin; if you don't have it installed, you'll need to. Here's a good reference with quartz cron expression examples.