I need custom behavior for the timeout function. For example, when I use:
timeout(time: 10, unit: 'MINUTES') {
doSomeStuff()
}
it terminates the doSomeStuff() function.
What I want to achieve is not to terminate the execution of the function, but to call another function every 10 minutes until doSomeStuff() is done with executing.
I can't use the Build-timeout Plugin from Jenkins since I need to apply this behavior to pipelines.
Any help would be appreciated.
In case anyone else has the same issue: After some research, the only way that came to my mind to solve my problem was to modify the notification plugin for the jenkins pipeline, in a way to add a new field that would contain value of time (in minutes) to delay the invoking of the url. In the code itself, where the url was invoked, i put those lines in a new thread and let that thread sleep for the needed amount of time before executing the remaining code. Something like this:
#Override
public void onStarted(final Run r, final TaskListener listener) {
HudsonNotificationProperty property = (HudsonNotificationProperty) r.getParent().getProperty(HudsonNotificationProperty.class);
int invokeUrlTimeout = 0;
if (property != null && !property.getEndpoints().isEmpty()){
invokeUrlTimeout = property.getEndpoints().get(0).getInvokeUrlTimeout();
}
int finalInvokeUrlTimeout = invokeUrlTimeout;
new Thread(() -> {
sleep(finalInvokeUrlTimeout * 60 * 1000);
Executor e = r.getExecutor();
Phase.QUEUED.handle(r, TaskListener.NULL, e != null ? System.currentTimeMillis() - e.getTimeSpentInQueue() : 0L);
Phase.STARTED.handle(r, listener, r.getTimeInMillis());
}).start();
}
Maybe not the best solution but it works for me, and I hope it helps other people too.
Related
I have Jenkins jobs that trigger twice a day and I would like to know if the current build is the first cron trigger of the day or not and do some action.
My cron job is as below
triggers {
// regression --> 3:00GMT, 14:00GMT
cron("00 3 * * 1-5 \n 00 14 * * 1-5")
}
Can I set some boolean param in my Jenkins file to check if it's the first trigger of the day?
The simplest option would be to check the build history. If the previous build was executed on the previous day, then the current build is the first build of the day. The logic must be defined in the executed job configurations.
The currentBuild object is an instance of the org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper class which provides all necessary information.
steps {
echo "The first build of the day started by trigger: ${isFirstBuildOfDayStartedByTrigger(currentBuild)}"
}
// ...
boolean isFirstBuildOfDayStartedByTrigger(currentBuild) {
if (isStartedByTrigger(currentBuild)) {
return false
}
def today = toLocalDate(currentBuild.startTimeInMillis)
def build = currentBuild.previousBuild
while(build != null) {
if (toLocalDate(build.startTimeInMillis).isBefore(today)) {
return true
}
if (isStartedByTrigger(build)) {
return false
}
build = build.previousBuild
}
return true
}
LocalDate toLocalDate(long millis) {
return Instant.ofEpochMilli(millis).atZone(ZoneId.systemDefault()).toLocalDate()
}
boolean isStartedByTrigger(build) {
// TODO: use build.buildCauses or build.getBuildCauses('cause.class.Name')
// to analyze if the job was started by trigger
return true // or false
}
You have to figure out which build cause is added when the job is started by trigger.
If you just want to find the first build of the day executed by anything or anyone, then the code is much simpler:
steps {
echo "The first build of the day: ${isFirstBuildOfDay(currentBuild)}"
}
boolean isFirstBuildOfDay(currentBuild) {
def today = toLocalDate(currentBuild.startTimeInMillis)
def previousBuild = currentBuild.previousBuild
return previousBuild == null || toLocalDate(previousBuild.startTimeInMillis).isBefore(today)
}
LocalDate toLocalDate(long millis) {
return Instant.ofEpochMilli(millis).atZone(ZoneId.systemDefault()).toLocalDate()
}
I used the new date API which I think is not whitelisted, so you have to put that code to the Jenkins library or approve the used method signatures.
Found the answer, it's simple but working fine for me.
First I am checking if this is a scheduled job or not and the current hour is less than 5 (scheduled job runs before 5)
def isItFirstScheduledJob = (params.JOB_IS_SCHEDULED && new Date().getHours() < 5) ? true : false
I need to persist a counter value between executions of a Grails Quartz plugin job. This runs at the correctly timed intervals and I can set the jobDataMap and read the value back correctly (during the same execution run), but it refuses to remember it between between executions.
I've set concurrent = false as the docs advised. Any ideas? I just need to persist and increment a counter. I want to avoid using a DB if at all possible, I think this should just use memory? Or other work arounds?
My TestJob.groovy, in /server/grails-app/jobs:
package myPackage
class MyJob {
static triggers = {
simple repeatInterval: 5000l // execute job every 5 seconds
}
def concurrent = false // Don't run multiple simultaneous instances of this job
def execute(context) {
if(context.jobDetail.jobDataMap['recCounter'] == null) { context.jobDetail.jobDataMap['recCounter'] = 1 }
else { context.jobDetail.jobDataMap['recCounter'] = context.jobDetail.jobDataMap['recCounter'] + 1 }
println(context.jobDetail.jobDataMap['recCounter'])
}
The output when run is a new line with '1' every 5 seconds. It should be incrementing the counter each time.
1
1
1
1
etc..
I'm running Grails 3.3.9 and build.gradle has compile "org.grails.plugins:grails-spring-websocket:2.4.1" in dependencies
Thanks
I have never used context object in my apps, but a counter can be implemented in a straight-forward way:
class MyJob {
//some static stuff
AtomicInteger counter = new AtomicInteger()
def execute(context) {
counter.incrementAndGet()
println counter.intValue()
}
}
We have a Jenkins pipeline script that requests approval from the user after all the preparatory steps are complete, before it actually applies the changes.
We want to add a timeout to this step, so that if there is no input from the user then the build is aborted, and are currently working on using this kind of method:
try {
timeout(time: 30, unit: 'SECONDS') {
userInput = input("Apply changes?")
}
} catch(err) {
def user = err.getCauses()[0].getUser()
if (user.toString == 'SYSTEM') { // if it's system it's a timeout
didTimeout = true
echo "Build timed out at approval step"
} else if (userInput == false) { // if not and input is false it's the user
echo "Build aborted by: [${user}]"
}
}
This code is based on examples found here: https://support.cloudbees.com/hc/en-us/articles/226554067-Pipeline-How-to-add-an-input-step-with-timeout-that-continues-if-timeout-is-reached-using-a-default-value and other places online, but I really dislike catching all errors then working out what's caused the exception using err.getCauses()[0].getUser(). I'd rather explicitly catch(TimeoutException) or something like that.
So my question is, what are the actual exceptions that would be thrown by either the approval step timing out or the userInput being false? I haven't been able to find anything in the docs or Jenkins codebase so far about this.
The exception class they are referring to is org.jenkinsci.plugins.workflow.steps.FlowInterruptedException.
Cannot believe that this is an example provided by CloudBeeds.
Most (or probably all?) other exceptions won't even have the getCauses() method which of course would throw another exception then from within the catch block.
Furthermore as you already mentioned it is not a good idea to just catch all exceptions.
Edit:
By the way: Scrolling further down that post - in the comments - there you'll find an example catching a FlowInterruptedException.
Rather old topic, but it helped me, and I've done some more research on it.
As I figured out, FlowInterruptedException's getCauses()[0] has .getUser() only when class of getCauses()[0] is org.jenkinsci.plugins.workflow.support.steps.input.Rejection. It is so only when timeout occured while input was active. But, if timeout occured not in input, getCause()[0] will contain object of another class: org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution$ExceededTimeout (directly mentioning timeout).
So, I end up with this:
def is_interrupted_by_timeout(org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e, Boolean throw_again=true) {
// if cause is not determined, re-throw exception
try {
def cause = e.getCauses()[0]
def cause_class = cause.getClass()
//echo("cause ${cause} class: ${cause_class}")
if( cause_class == org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution$ExceededTimeout ) {
// strong detection
return true
} else if( cause_class == org.jenkinsci.plugins.workflow.support.steps.input.Rejection ) {
// indirect detection
def user = cause.getUser()
if( user.toString().equals('SYSTEM') ) {
return true
} else {
return false
}
}
} catch(org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException e_access) {
// here, we may deal with situation when restricted methods are not approved:
// show message and Jengins' admin will copy/paste and execute them only once per Jenkins installation.
error('''
To run this job, Jenkins admin needs to approve some Java methods.
There are two possible ways to do this:
1. (better) run this code in Jenkins Console (URL: /script):
import org.jenkinsci.plugins.scriptsecurity.scripts.ScriptApproval;
def scriptApproval = ScriptApproval.get()
scriptApproval.approveSignature('method org.jenkinsci.plugins.workflow.steps.FlowInterruptedException getCauses')
scriptApproval.approveSignature('method org.jenkinsci.plugins.workflow.support.steps.input.Rejection getUser')
scriptApproval.save()
'''.stripIndent())
return null
}
if( throw_again ) {
throw e
} else {
return null
}
}
And now, you may catch it with something like this:
try {
...
} catch (org.jenkinsci.plugins.workflow.steps.FlowInterruptedException err) {
if( is_interrupted_by_timeout(err) ) {
echo('It is timeout!')
}
}
P.S. I agree, this is bad Jenkins design.
All,
Here is a unit test for checking the size of a collection
main() {
test("Resource Manager Image Load", () {
ResourceManager rm = new ResourceManager();
int WRONG_SIZE = 1000000;
rm.loadImageManifest("data/rm/test_images.yaml").then((_){
print("Length="+ rm.images.length.toString()); // PRINTS '6' - WHICH IS CORRECT
expect(rm.images, hasLength(WRONG_SIZE));
});
});
}
I am running this from a browser (client-side Dart libraries are in use) and it ALWAYS passes, no matter what the value of WRONG_SIZE.
Help appreciated.
In such simple cases you can just return the future. The unit test framework recognizes it and waits for the future to complete. This also works for setUp/tearDown.
main() {
test("Resource Manager Image Load", () {
ResourceManager rm = new ResourceManager();
int WRONG_SIZE = 1000000;
return rm.loadImageManifest("data/rm/test_images.yaml").then((_) {
//^^^^
print("Length="+ rm.images.length.toString()); // PRINTS '6' - WHICH IS CORRECT
expect(rm.images, hasLength(WRONG_SIZE));
});
});
}
The problem is that your code returns a Future, and your test completes before the code in the Future has finished, so there's nothing to cause it to fail.
Check out the Asynchronous Tests section on the Dart site. There are methods like expectAsync that allow the future to be passed to the test framework so that it can wait for them to complete and handle the result correctly.
Here's an example (note the expect call is now inside the function passed to expectAsync)
test('callback is executed once', () {
// wrap the callback of an asynchronous call with [expectAsync] if
// the callback takes 0 arguments...
var timer = Timer.run(expectAsync(() {
int x = 2 + 3;
expect(x, equals(5));
}));
});
i use executor service to launch multiple thread to sent request to api and get data back. sometimes i see some threads haven't finished their job yet, the service kill that thread already, how can i force the service to wait until the thread finish their job?
here is my code:
ExecutorService pool = Executors.newFixedThreadPool(10);
List<Future<List<Book>>> futures = Lists.newArrayList();
final ObjectMapper mapper1 = new ObjectMapper();
for (final Author a : authors) {
futures.add(pool.submit(new Callable<List<Book>>() {
#Override
public List<Book> call() throws Exception {
String urlStr = "http://localhost/api/book?limit=5000&authorId=" + a.getId();
List<JsonBook> Jsbooks = mapper1.readValue(
new URL(urlStr), BOOK_LIST_TYPE_REFERENCE);
List<Book> books = Lists.newArrayList();
for (JsonBook jsonBook : Jsbooks) {
books.add(jsonBook.toAvro());
}
return books;
}
}));
}
pool.shutdown();
pool.awaitTermination(3, TimeUnit.MINUTES);
List<Book> bookList = Lists.newArrayList();
for (Future<List<Book>> future : futures) {
if (!future.isDone()) {
LogUtil.info("future " + future.toString()); <-- future not finished yet
throw new RuntimeException("Future to retrieve books: " + future + " did not complete");
}
bookList.addAll(future.get());
}
and i saw some excepitons at the (!future.isDone()) block. how can i make sure every future is done when executor service shutdown?
I like to use the countdown latch.
Set the latch to the size that you're iterating and pass that latch into your callables, then in your run / call method have a try/finally block that decrements the countdown latch.
After everything has been enqueued to your executor service, just call your latch's await method, which will block until it's all done. At that time all your callables will be finished, and you can properly shut down your executor service.
This link has an example of how to set it up.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html