We run unit tests on Jenkins and one of our tests freezes sometimes.
We have timeouts defined in the Jenkins pipeline and the freeze triggers the timeout and that kills the testing process.
Is there a way (via Jenkins pipelines, maybe via Groovy) to execute a command (e.g. create a process dump of the testing process) as soon as we run into a timeout, but (of course) before the timeout kills the testing process?
You can wrap our test execution with a try-catch and do whatever you need after catching the timeout exception. Here is a sample Pipeline.
pipeline {
agent any
stages {
stage('Hello') {
steps {
script{
try {
timeout(unit: 'SECONDS', time: 5) {
echo "Running your Tests here!!!!"
sleep 10
}
} catch (e){
echo "The tests erroredout!!!" + e.getCauses()
if(e.getCauses()[0] instanceof org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution$ExceededTimeout) {
echo "This is a timeout, do whatever you want..."
}
}
}
}
}
}
}
Related
I have a site in production. And I have a simple playwright test that browses to the site and does some basic checks to make sure that it's up.
I'd like to have this job running in Jenkins every 5 minutes, and if the tests fail I want to run a script that will restart the production server. If the tests pass, I don't want to do anything.
What's the easiest way of doing this?
I have the MultiJob plugin that I thought I could use, and have the restart triggered on the failed test step, but it doesn't seem to have the ability to trigger specifically on fail.
Something like the following will do the Job for you. I'm assuming you have a second Job that will take care of the restart.
pipeline {
agent any
triggers{
cron('*/5 * * * *')
}
stages {
stage("Run the Test") {
steps{
echo "Running the Test"
// I'm returning exit code 1 so jenkins will think this failed
sh '''
echo "RUN SOMETHING"
exit 1
'''
}
}
}
post {
success {
echo "Success: Do nothing"
}
failure {
echo 'I failed :(, Execute restart Job'
// Executing the restart Job.
build job: 'RestartJob'
}
}
}
I've a Jenkins pipeline with multiple stages but because of some issue, in one of the stages, it is likely to run longer unnecessarily.
Instead of aborting entire pipeline build and skip next stages, I want to kill that specific stage on which other stages are not dependent.
Is there a way to kill specific stage of Jenkins pipeline?
There are ways to skip a stage. But I'm not sure if there are options to kill a long running stage. I'd simply add a conditional expression to run the stage or not OR maybe you could put a timeout condition wrapped in a try..catch block for the long running unnecessary stage to skip and proceed to other stages you want like as below.
pipeline {
agent any
stages {
stage('stage1') {
steps {
script {
try {
timeout(time: 2, unit: 'NANOSECONDS')
echo "do your stuff"
} catch (Exception e) {
echo "Ended the never ending stage and proceeding to the next stage"
}
}
}
}
stage('stage2') {
steps {
script {
echo "Hi Stage2"
}
}
}
}
}
OR Check this page for conditional step/stage.
You can try using "try/catch" block in the scripted pipeline. Even if there is error in a particular stage, Jenkins will continue to execute the next stage.
node {
stage('Example') {
try {
sh 'exit 1'
}
catch (exc) {
echo 'Something failed, I should sound the klaxons!'
throw
}
}
}
You can refer documentation here: https://jenkins.io/doc/book/pipeline/syntax/
I'm trying to replicate the equivalent of a conditional stage in Jenkins pipeline using a try / catch around a preceding stage, which then sets a success variable, which is used to trigger the conditional stage.
It appears that a try catch block is the way to go, setting a success var to SUCCESS or FAILED, which is used as part of a when statement later (as part of the conditional stage).
The code I am using is as follows:
pipeline {
agent any
stages {
try{
stage("Run unit tests"){
steps{
sh '''
# Run unit tests without capturing stdout or logs, generates cobetura reports
cd ./python
nosetests3 --with-xcoverage --nocapture --with-xunit --nologcapture --cover-package=application
cd ..
'''
currentBuild.result = 'SUCCESS'
}
}
} catch(Exception e) {
// Do something with the exception
currentBuild.result = 'SUCCESS'
}
stage ('Speak') {
when {
expression { currentBuild.result == 'SUCCESS' }
}
steps{
echo "Hello, CONDITIONAL"
}
}
}
}
The latest syntax error I am receiving is as follows:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
failed:
WorkflowScript: 4: Expected a stage # line 4, column 9.
try{
I've also tried lots of variations.
Am I taking the wrong approach here? This seems like a fairly common requirement.
Thanks.
This might solve your problem depending on what you are going for. Stages are only run when the preceding stages succeed, so if you actually have two stages like in your example, and if you want the second to only run when the first succeeds, you want to ensure that the first stage fails appropriately when tests fail. Catching will prevent the (desirable) failure. Finally will preserve the failure, and can also still be used to grab your test results.
So here, the second stage will only run when the tests pass, and the test results will be recorded regardless:
pipeline {
agent any
stages {
stage("Run unit tests"){
steps {
script {
try {
sh '''
# Run unit tests without capturing stdout or logs, generates cobetura reports
cd ./python
nosetests3 --with-xcoverage --nocapture --with-xunit --nologcapture --cover-package=application
cd ..
'''
} finally {
junit 'nosetests.xml'
}
}
}
}
stage ('Speak') {
steps{
echo "Hello, CONDITIONAL"
}
}
}
}
Note that I'm actually using try in a declarative pipeline, but like StephenKing says, you can't just use try directly (you have to wrap arbitrary groovy code in the script step).
I would like to have a Jenkins build flow that looks like this.
After the build is triggered all slaves run the same job in parallel (a setup job).
If any slaves fail this job they should not continue on.
For the all the slaves that to pass that job, they should grab a job out of a pool of jobs that need to be completed. And once a slave completes a job they should go back to complete another job in the pool.
I have only started working with Jenkins a few weeks ago and they way I have it setup now is as each job is picked up by a slave they have to run the setup job first. This really slows down build times because I have about 30 jobs and the setup takes ~2 minutes.
I am using Jenkins as an automated testing platform and all the jobs in the job pool can run independently of each other. I have 5 slaves currently and ~30 jobs.
The following should do the trick:
def jobPool = new ArrayDeque()
jobPool.add({
echo "Doing stuff on ${env.NODE_NAME}"
});
jobPool.add({
echo "Doing other stuff on ${env.NODE_NAME}, a little slower"
sleep 4
});
jobPool.add({
echo "Doing more stuff on ${env.NODE_NAME}, even slower"
sleep 10
});
jobPool.add({
echo "Doing stuff quick on ${env.NODE_NAME}"
});
jobPool.add({
echo "Doing stuff quicker on ${env.NODE_NAME}"
});
def par = [:]
for (x in ["master", "urban"]) {
def nodeName = x; // needed due to variable scoping
par[nodeName] = {
node (nodeName) {
try {
echo "Doing setup on ${env.NODE_NAME}!"
// Do you're setup
echo "Done with setup"
} catch (Exception e) {
echo "Will not use this node as it failed setup!"
return;
}
while (true) {
// echo "${jobPool.size()}"
def subTask = jobPool.poll()
//echo "${jobPool.size()} ${subTask}"
if (subTask == null) {
break;
}
// Might wan't try catch around the next line if you wan't to continue if a job fails
subTask()
}
}
}
}
parallel par
if (!jobPool.isEmpty()) {
error "Not all tasks was done!"
}
Simply add your "job pool jobs" to the jobPool variable and modify the setup part.
It seems like you want separate stages in the same job. This is made much easier in jenkins 2's pipelines. There are some pictures here:
https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin
the [groovy] code ends up looking like this:
node {
stage 'Checkout'
svn 'https://svn.mycorp/trunk/'
stage 'Build'
sh 'make all'
stage 'Test'
sh 'make test'
}
Is there a way to continue execution of the scripted pipeline even if the previous stage failed? I need to run specific commands (cleanup) when the build fails before the whole job fails.
The accepted answer wouldn't fail the stage or even mark it as unstable. It is now possible to fail a stage, continue the execution of the pipeline and choose the result of the build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this is a fairly new feature.
The usual approach is to wrap your steps within a try block.
try {
sh "..."
} catch (err) {
echo "something failed"
}
// cleanup
sh "rm -rf *"
To ease the pain and make the pipeline code more readable, I've encapsulated this in another method here in my global library code.
Another approach, esp. created because of this very issue, are the declarative pipelines (blog, presentation).
post {
always {
cleanWs()
}
}
}
Will always cleanup the job even if the rest fails