I have created a test workflow as below:
node("master") {
ws("/opt/mount1/jenkins/jobs/GoogleFlow/workspace/${env.BUILD_NUMBER}") {
try {
stage name: 'sync', concurrency: 3
echo "before sync"
sh '''touch buildFile
echo "This is from ${BUILD_NUMBER}" >> buildFile
cat buildFile'''
sh "sleep 5"
echo "after sync"
sh "date"
stage name: 'build', concurrency: 1
echo "before build"
sh "date"
sh '''sleep 10
cat buildFile'''
echo "build 1/3"
sh "sleep 5"
echo "build 2/3"
sh '''sleep 5
cat buildFile'''
echo "build 3/3"
sh "date"
stage name: 'test', concurrency: 3
echo "before test"
sh "date"
sh '''sleep 10
cat buildFile'''
sh "date"
stage name: 'delete', concurrency: 1
sh '''pwd
ls -al'''
//deleteDir()
//sh '''pwd
//ls -al'''
}
catch (err){
stage 'Send Notification'
mail (to: 'XXXXXXX#gmail.com',
subject: "test",
body: "test");
}
}
}
I am trying to get an email notification using the try-catch. I have referred this blog post, but when it comes to stage to 'send notification' it just sits there hung.
But if i use the old Jenkins job way, i can receive emails. This shows that the SMTP setup is working. Below is the console output, when it hungs
[Pipeline] Allocate node : Start
Running on master in /opt/mount1/jenkins/jobs/GoogleFlow/workspace#4
[Pipeline] node {
[Pipeline] Allocate workspace : Start
Running in /opt/mount1/jenkins/jobs/GoogleFlow/workspace/205
[Pipeline] ws {
[Pipeline] stage: sync
Entering stage sync
Proceeding
[Pipeline] echo
before sync
[Pipeline] sh
[205] Running shell script
+ touch buildFile
+ echo 'This is from 205'
+ cat buildFile
This is from 205
[Pipeline] sh
[205] Running shell script
+ sleep 5
[Pipeline] echo
after sync
[Pipeline] sh
[205] Running shell script
+ date
Tue Feb 2 22:54:52 UTC 2016
[Pipeline] stage: build
Entering stage build
Waiting for builds [204]
Canceled since #206 got here
[Pipeline] stage: Send Notification
Entering stage Send Notification
Proceeding
[Pipeline] mail
Any idea on how to fix?
Jenkins - 1.643
Mailer plugin - 1.11
I had the same problem and fixed it by upgrading mailer to v 1.16.
Related
I got a little problem with Jenkins Pipeline... To be more specific it seems like it always fails, but without any reasons, (probably there is one, but cannot find it)
Snippet of the pipeline.... (Whole Pipeline)
pipeline {
environment {
DOCKERHUB_CREDENTIALS=credentials("Dockerhub")
APPLICATION_HOST="0.0.0.0:8000"
APPLICATION_PORT="8000"
}
stage("build"){
steps{
script {
def inspectDockerNetwork = sh script: "docker network create global_store_network", returnStatus: true
if (inspectDockerNetwork == 0) {
sh "echo 'Creating Docker Network...'"
sh "docker network create global_store_network"
sh "echo 'Network Has been Created..'"
}
}
dir("test_env"){
sh "docker-compose up -d"
sleep 10
sh "echo 'Docker Built Image Successfully! Running Container....'"
}
}
}
stage("test"){
steps{
load "./test_env/version_env.groovy"
sh "echo 'Running Test Pipeline'"
sh "echo 'Running Healtcheck Test...'"
sh "echo 'Sleeping until the Application will be fully ready...'"
sleep 10
script {
command = """curl -s -X GET -H 'accept: */*' 'http://${env.APPLICATION_HOST}:${env.APPLICATION_PORT}/healthcheck/'"""
responseStatus = sh(script: command, returnStdout: true).trim()
if (responseStatus != "200") {
sh "echo 'Application Responded with Failure, Not Ready for Production...'"
error "Health Check Stage Failure."
}
}
}
post {
always {
dir("test_env"){
sh "echo 'Removing Testing Environment'"
sh "docker-compose down"
}
}
}
}. /////////// It has not start an Execution of the Deployment, So the Error Is Somewhere above ////////////////////////////////////
stage("deployment"){
steps {
load "./test_env/version_env.groovy"
sh "echo 'Running Deployment Pipeline Stage...'"
sh "echo 'Tagging new Image Version'"
withCredentials([usernamePassword(
credentialsId: "DockerHub", // Credential Id that should be created at Jenkins Server...
usernameVariable: env.DOCKERHUB_CREDENTIALS_USR, // Credential Username that should be created at jenkins Server.
passwordVariable: env.DOCKERHUB_CREDENTIALS_PSW, // Credential Password that shoud be created at Jenkins Server..
)]){
sh "docker login -u ${env.DOCKERHUB_CREDENTIALS_USR} -p ${env.DOCKERHUB_CREDENTIALS_PSW}"
sh "echo 'Logged In.. Into Docker.'"
sh "echo 'Tagging An Image'"
sh "docker tag new_versioned_image ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged... Pushing onto docker repo.'"
sh "docker push ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged Successfully.. Pushing Image On Docker Hub..'"
sh "echo 'Image has been Pushed Successfully! Pipeline Finished.'"
}
}
}
}
So the Output of that snippet is following...
( I've Separated Stage Logs In order to make it easier to read )
/////////// Build Stage ///////////////
First time build. Skipping changelog.
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKERHUB_CREDENTIALS or $DOCKERHUB_CREDENTIALS_PSW
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ docker-compose up -d
Container test_postgres_store_database Creating
Container test_postgres_store_database Created
Container test_store_application_server Creating
Container test_store_application_server Created
Container test_postgres_store_database Starting
Container test_postgres_store_database Started
Container test_store_application_server Starting
Container test_store_application_server Started
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] sh
+ echo Docker Built Image Successfully! Running Container....
Docker Built Image Successfully! Running Container....
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] load
[Pipeline] { (./test_env/version_env.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] sh
////////////// Testing Stage goes there /////////////////////
+ echo Running Test Pipeline
Running Test Pipeline
[Pipeline] sh
+ echo Running Healtcheck Test...
Running Healtcheck Test...
[Pipeline] sh
+ echo Sleeping until the Application will be fully ready...
Sleeping until the Application will be fully ready...
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ curl -s -X GET -H accept: */* http://0.0.0.0:8000/healthcheck/
[Pipeline] }
[Pipeline] // script
Post stage
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ echo Removing Testing Environment
Removing Testing Environment
[Pipeline] sh
+ docker-compose down
Container test_store_application_server Stopping
Container test_store_application_server Stopping
Container test_store_application_server Stopped
Container test_store_application_server Removing
Container test_store_application_server Removed
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopped
Container test_postgres_store_database Removing
Container test_postgres_store_database Removed
///// Deployment Stage Goes there.... ////////////
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deployment)
Stage "deployment" skipped due to earlier failure(s) ///// The Error Message Goes There....
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
//// ERROR /////
[Pipeline] End of Pipeline
ERROR: script returned exit code 7
Finished: FAILURE
The Problem is that it does not respond what's exactly the problem is, (just simply skipped due to earlier failure(s), above logs does not shows any of the errors.
So would really appreciate any help or any suggestions how to solve this Issue.
Thanks.
I need to filter a file in jenkins. Filtering works as long as the result is not empty. But if the resulting output is empty, the pipeline fails with ERROR: script returned exit code 1 Finished: FAILURE
Example:
#!groovy
pipeline {
agent any
stages {
stage ('mystage') {
steps {
script {
sh "echo '' > myfile"
sh "echo 'foo 0' >> myfile"
sh "echo 'foo 1' >> myfile"
sh "grep foo myfile"
sh "grep ba myfile"
}
}
}
}
}
output:
+ echo ''
[Pipeline] sh
+ echo 'foo 0'
[Pipeline] sh
+ echo 'foo 1'
[Pipeline] sh
+ grep foo myfile
foo 0
foo 1
[Pipeline] sh
+ grep ba myfile
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
routing the output to a file with grep ba myfile > catchoutput does not work.
How can I output the grep result, without the pipeline failing in this edge case?
Adding a dummy line like sh "echo 'dummyline that won't match' >> myfile" seems to work but is a hack. Is there a clean solution?
We can take return value in a variable:
def ret = sh(script: 'grep ba myfile', returnStdout: true)
More info : https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#code-sh-code-shell-script.
Note you can also add returnStatus: true, so that jenkins step does not fail even if there is failure in the command.
I don't find a way using the BUILD_NUMBER provided by jenkins in a sh script. I read some answers to similar questions but nothing seem to help.
node {
echo "Build number 1 $BUILD_NUMBER"
// output ok
stage('stage1') {
echo "Build number 2 $BUILD_NUMBER"
// output ok
def BUILD_NUMBER = "$BUILD_NUMBER"
withCredentials([sshUserPrivateKey(credentialsId: 'github-rsa-key', variable: 'RSAKEY')]) {
echo "Build number 3 " + BUILD_NUMBER
// output ok
echo "Build number 4 $BUILD_NUMBER"
// output ok
// -----------------
sh 'echo $BUILD_NUMBER' // NullPointer
sh "echo $BUILD_NUMBER" // NullPointer
sh "echo \$BUILD_NUMBER" // NullPointer
sh "echo BUILD_NUMBER" // NullPointer
withEnv(["BUILD_NUMBER=BUILD_NUMBER"]) {
sh "echo $BUILD_NUMBER" // NullPointer!!
}
env.BUILD_NUMER = "$BUILD_NUMBER"
sh "echo $BUILD_NUMBER" // NullPointer
sh "echo ${env.BUILD_NUMBER}" // NullPointer
}
}
}
Basic solution: wrap shell script in """ block
node {
echo "Build number 1: $BUILD_NUMBER"
// output ok
stage('stage1') {
echo "Build number 2: $BUILD_NUMBER"
// output ok
def BUILD_NUMBER = "$BUILD_NUMBER"
echo "Build number 3: " + BUILD_NUMBER
// output ok
echo "Build number 4: $BUILD_NUMBER"
// output ok
// -----------------
sh 'printenv'
sh """
echo "Build number in sh script: ${env.BUILD_NUMBER}"
echo "Job base name: ${env.JOB_BASE_NAME}"
"""
// output ok
}
}
Console Output:
Running on Jenkins in /var/lib/jenkins/workspace/test-infra-env
[Pipeline] {
[Pipeline] echo
Build number 1: 5
[Pipeline] stage
[Pipeline] { (stage1)
[Pipeline] echo
Build number 2: 5
[Pipeline] echo
Build number 3: 5
[Pipeline] echo
Build number 4: 5
[Pipeline] sh
+ printenv
JENKINS_HOME=/var/lib/jenkins
MAIL=/var/mail/jenkins
USER=jenkins
...
...
JOB_BASE_NAME=test-infra-env
BUILD_NUMBER=5
...
...
[Pipeline] sh
+ echo Build number in sh script: 5
Build number in sh script: 5
+ echo Job base name: test-infra-env
Job base name: test-infra-env
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
There may be a more idiomatic approach (please share if you know) but it works if you define it in an environment block first. Something like:
stage('Show Build Number') {
environment {
BUILD_NUMBER = "${env.BUILD_NUMBER}"
}
steps {
sh '''
echo "This is build $BUILD_NUMBER"
'''
}
}
There is a good post on code maven with useful examples.
Here's a simple example that works for me. Jenkins 2.164.2
Edit to add a physical script as well: /tmp/script.sh contains..
#!/bin/bash
echo "Script: - Build number: $BUILD_NUMBER"
And the Jenkins job
node {
echo "Node: Build number: $BUILD_NUMBER"
stage('stage1') {
echo "Stage: Build number: $BUILD_NUMBER"
sh ("echo Shell: Build number: $BUILD_NUMBER")
sh ("/tmp/script.sh")
}
}
This example uses a "withCredentials" block. Note the single quotes, which is referenced here - https://jenkins.io/doc/pipeline/steps/credentials-binding/
node {
echo "Build number 1 $BUILD_NUMBER"
// output ok
stage('stage1') {
withCredentials([string(credentialsId: 'my_password', variable: 'TOKEN')]) {
sh '''
echo "Shell: Build number: $BUILD_NUMBER"
'''
sh ('/tmp/script.sh')
}
}
}
I have a building-server where I have Jenkins 2.73.3 and another servers where I deploy my apps.
I have also set up a credential to connect from building-server to the other servers.
But everytime I add another server it is difficult to add it because I set up the authorized key in the new server and in the command line works, but not in Jenkins.
Here is a little recipe that fails:
pipeline {
agent any
stages {
stage('Set conditions') {
steps {
sshagent(['xxxx-xxxx-xxxx-xxxx-xxxx']) {
sh "ssh user#product.company.com 'echo $HOME'"
}
}
}
}
}
And here is the Log failure:
[ssh-agent] Started.
[Pipeline] {
[Pipeline] sh
[check] Running shell script
+ ssh user#product.company.com echo /var/lib/jenkins
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 12567 killed;
[ssh-agent] Stopped.
Host key verification failed.
[Pipeline] }
[Pipeline] // sshagent
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 255
Finished: FAILURE
It seems that the solution was to add the parameter StrictHostKeyChecking to the shell script line
sh "ssh -o StrictHostKeyChecking=no user#product.company.com 'echo $HOME'"
I want to configure a Jenkins pipeline job so that it should be able to run multiple shell script jobs. Even if one shell script fails the job should run the other two before failing the job.
You need to tweak your shell script, not Jenkins pipeline to achieve what you want!
Try this in your shell script
shell script command > /dev/null 2>&1 || true
so fail/pass it will execute and go to next shell script
You can always try catch the potentially failing sh execution
node {
sh "echo test"
try {
sh "/dev/null 2>&1"
} catch (error) {
echo "$error"
}
sh "echo test1"
}
Above runs successfully and produces
Started by user Blazej Checinski
[Pipeline] node
Running on agent2 in /home/build/workspace/test
[Pipeline] {
[Pipeline] sh
[test] Running shell script
+ echo test
test
[Pipeline] sh
[test] Running shell script
+ /dev/null
/home/build/workspace/test#tmp/durable-b4fc2854/script.sh: line 2: /dev/null: Permission denied
[Pipeline] echo
hudson.AbortException: script returned exit code 1
[Pipeline] sh
[test] Running shell script
+ echo test1
test1
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS