unable to run sh script inside jenkins pipeline - jenkins

Team,
I tried several online links but can't figure out why my shell script is not running inside jenkins pipeline.
any hint?
my whole pipeline is
pipeline {
options {
buildDiscarder( logRotator( artifactDaysToKeepStr: '15', artifactNumToKeepStr: '15', daysToKeepStr: '15', numToKeepStr: '15'))
disableConcurrentBuilds()
skipDefaultCheckout true
timeout(time: 30, unit: 'MINUTES')
ansiColor('xterm')
}
parameters {
booleanParam( name: 'Refresh', defaultValue: false, description: 'Reload job from the Jenkinsfile and then exit')
}
agent {
label 'built-in'
}
triggers {
cron '0 8 * * *'
}
stages {
stage('Build') {
when { expression { !params.Refresh } }
steps {
script {
sh ''' set +x
apt-get update && apt install dnsutils -y
nslookup git.github-private.com | grep "Non-authoritative answer" -A 3
git_av_url=$(nslookup git.github-private.com | grep "Non-authoritative answer" -A 3 | grep git | awk \'{print $2}\')
git_av_ip=$(nslookup git.github-private.com | grep "Non-authoritative answer" -A 3 | grep Address | awk \'{print $2}\')
echo $git_av_ip
echo $git_av_ip $git_av_url >> /etc/hosts
cat /etc/hosts'''
}
}
}
}
}
output
09:28:53 [Pipeline] stage
09:28:53 [Pipeline] { (Build)
09:28:53 [Pipeline] script
09:28:53 [Pipeline] {
09:28:53 [Pipeline] sh
09:28:53 + set -ex
09:28:53 + apt-get update
09:28:53 Hit:1 http://security.debian.org/debian-security bullseye-security InRelease
09:28:53 Hit:2 http://deb.debian.org/debian bullseye InRelease
09:28:53 Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
09:28:54 Reading package lists...
09:28:54 + apt install dnsutils -y
09:28:54
09:28:54 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
09:28:54
09:28:55 Reading package lists...
09:28:55 Building dependency tree...
09:28:55 Reading state information...
09:28:55 dnsutils is already the newest version (1:9.16.33-1~deb11u1).
09:28:55 0 upgraded, 0 newly installed, 0 to remove and 43 not upgraded.
09:28:55 + nslookup git.github-private.com
09:28:55 + grep Non-authoritative answer -A 3
09:28:55 [Pipeline] }
09:28:55 [Pipeline] // script
09:28:55 [Pipeline] }
09:28:55 [Pipeline] // stage
09:28:55 [Pipeline] }
09:28:55
09:28:55 [Pipeline] // ansiColor
09:28:55 [Pipeline] }
09:28:55 [Pipeline] // timeout
09:28:55 [Pipeline] }
09:28:56 [Pipeline] // node
09:28:56 [Pipeline] End of Pipeline
09:28:56 ERROR: script returned exit code 1
09:28:56 Finished: FAILURE

I flipped logic of my variable value evaluation and it worked and i also placed #!/bin/bash. got help from here ssh script format
below is sample pipeline.
steps {
sh '''#!/bin/bash
git_av_url=$(awk '{print $2}' <<< $(nslookup git.github-private.com | grep -e "git" -A 3))
git_av_ip=$(awk '{print $2}' <<< $(nslookup git.github-private.com | grep -e "Non-authoritative answer" -A 3))
echo $git_av_ip $git_av_url >> /etc/hosts
grep $git_av_url /etc/hosts
}
output
10.1.0.1 git.github-private.com

Related

Jenkins throws error when running pipeline

I got a little problem with Jenkins Pipeline... To be more specific it seems like it always fails, but without any reasons, (probably there is one, but cannot find it)
Snippet of the pipeline.... (Whole Pipeline)
pipeline {
environment {
DOCKERHUB_CREDENTIALS=credentials("Dockerhub")
APPLICATION_HOST="0.0.0.0:8000"
APPLICATION_PORT="8000"
}
stage("build"){
steps{
script {
def inspectDockerNetwork = sh script: "docker network create global_store_network", returnStatus: true
if (inspectDockerNetwork == 0) {
sh "echo 'Creating Docker Network...'"
sh "docker network create global_store_network"
sh "echo 'Network Has been Created..'"
}
}
dir("test_env"){
sh "docker-compose up -d"
sleep 10
sh "echo 'Docker Built Image Successfully! Running Container....'"
}
}
}
stage("test"){
steps{
load "./test_env/version_env.groovy"
sh "echo 'Running Test Pipeline'"
sh "echo 'Running Healtcheck Test...'"
sh "echo 'Sleeping until the Application will be fully ready...'"
sleep 10
script {
command = """curl -s -X GET -H 'accept: */*' 'http://${env.APPLICATION_HOST}:${env.APPLICATION_PORT}/healthcheck/'"""
responseStatus = sh(script: command, returnStdout: true).trim()
if (responseStatus != "200") {
sh "echo 'Application Responded with Failure, Not Ready for Production...'"
error "Health Check Stage Failure."
}
}
}
post {
always {
dir("test_env"){
sh "echo 'Removing Testing Environment'"
sh "docker-compose down"
}
}
}
}. /////////// It has not start an Execution of the Deployment, So the Error Is Somewhere above ////////////////////////////////////
stage("deployment"){
steps {
load "./test_env/version_env.groovy"
sh "echo 'Running Deployment Pipeline Stage...'"
sh "echo 'Tagging new Image Version'"
withCredentials([usernamePassword(
credentialsId: "DockerHub", // Credential Id that should be created at Jenkins Server...
usernameVariable: env.DOCKERHUB_CREDENTIALS_USR, // Credential Username that should be created at jenkins Server.
passwordVariable: env.DOCKERHUB_CREDENTIALS_PSW, // Credential Password that shoud be created at Jenkins Server..
)]){
sh "docker login -u ${env.DOCKERHUB_CREDENTIALS_USR} -p ${env.DOCKERHUB_CREDENTIALS_PSW}"
sh "echo 'Logged In.. Into Docker.'"
sh "echo 'Tagging An Image'"
sh "docker tag new_versioned_image ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged... Pushing onto docker repo.'"
sh "docker push ${env.DOCKERHUB_REPOSITORY_LINK}:latest"
sh "echo 'Tagged Successfully.. Pushing Image On Docker Hub..'"
sh "echo 'Image has been Pushed Successfully! Pipeline Finished.'"
}
}
}
}
So the Output of that snippet is following...
( I've Separated Stage Logs In order to make it easier to read )
/////////// Build Stage ///////////////
First time build. Skipping changelog.
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $DOCKERHUB_CREDENTIALS or $DOCKERHUB_CREDENTIALS_PSW
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ docker-compose up -d
Container test_postgres_store_database Creating
Container test_postgres_store_database Created
Container test_store_application_server Creating
Container test_store_application_server Created
Container test_postgres_store_database Starting
Container test_postgres_store_database Started
Container test_store_application_server Starting
Container test_store_application_server Started
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] sh
+ echo Docker Built Image Successfully! Running Container....
Docker Built Image Successfully! Running Container....
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] load
[Pipeline] { (./test_env/version_env.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] sh
////////////// Testing Stage goes there /////////////////////
+ echo Running Test Pipeline
Running Test Pipeline
[Pipeline] sh
+ echo Running Healtcheck Test...
Running Healtcheck Test...
[Pipeline] sh
+ echo Sleeping until the Application will be fully ready...
Sleeping until the Application will be fully ready...
[Pipeline] sleep
Sleeping for 10 sec
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ curl -s -X GET -H accept: */* http://0.0.0.0:8000/healthcheck/
[Pipeline] }
[Pipeline] // script
Post stage
[Pipeline] dir
Running in /var/jenkins_home/workspace/Store Pipeline/test_env
[Pipeline] {
[Pipeline] sh
+ echo Removing Testing Environment
Removing Testing Environment
[Pipeline] sh
+ docker-compose down
Container test_store_application_server Stopping
Container test_store_application_server Stopping
Container test_store_application_server Stopped
Container test_store_application_server Removing
Container test_store_application_server Removed
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopping
Container test_postgres_store_database Stopped
Container test_postgres_store_database Removing
Container test_postgres_store_database Removed
///// Deployment Stage Goes there.... ////////////
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deployment)
Stage "deployment" skipped due to earlier failure(s) ///// The Error Message Goes There....
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
//// ERROR /////
[Pipeline] End of Pipeline
ERROR: script returned exit code 7
Finished: FAILURE
The Problem is that it does not respond what's exactly the problem is, (just simply skipped due to earlier failure(s), above logs does not shows any of the errors.
So would really appreciate any help or any suggestions how to solve this Issue.
Thanks.

Jenkins Pipeline failed

I am running a scripted version of the Jenkins pipeline. It ran through all the stages except the last stage. It failed every time.
Here is the Jenkins code that keep on failing:
stage('Deploy Production') {
echo "Deploy Prod on: ${env.BRANCH_NAME}"
try {
if (env.BRANCH_NAME == 'master'){
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'XX',
usernameVariable: 'XXXXX', passwordVariable: 'XXXXXX']]) {
sh 'npm run build-prod-ci'
sh 'cf login -u ${XXXXX} -p ${XXXXXX} -a website.com -o XXX -s XXXX'
sh 'cf blue-green-deploy Dashboard'
sh 'cf delete Dashboard-old -f'
}
}
} finally {
deleteDir()
sh 'cf logout'
}
}
This is the printout and the error message. Any help in resolving this issue is appreciated.
Plugin blue-green-deploy 1.3.0 successfully installed.
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy Development)
[Pipeline] echo
Deploy Dev on: master
[Pipeline] deleteDir
[Pipeline] sh
+ cf logout
Logging out ...
OK
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deploy Production)
[Pipeline] echo
Deploy Prod on: master
[Pipeline] withCredentials
Masking supported pattern matches of $XXXXXX
[Pipeline] {
[Pipeline] sh
+ npm run build-prod-ci
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /v/wl/ws/Dashboard_master/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/v/wl/ws/Dashboard_master/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /v/wl/wsp/Dashboard_master/.npm/_logs/2022-07-06T22_43_57_239Z-debug.log
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] deleteDir
[Pipeline] sh
+ cf logout
Logging out ...
OK
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 254
Previous stages with npm commands worked fine. Here is an example of a stage that worked:
stage('Run project tests and build') {
echo "Run project tests and build on: ${env.BRANCH_NAME}"
sh "npm run lint"
//sh "npm run compodoc-ci"
//sh "npm run test-coverage"
sh "npm run build-dev-ci"
}
This issue was self inflicted, and the comments that was given above gave me some hints to arrive at the solution.
The problem was that I was moving the original Jenkins code (I think it's called Declarative Pipeline) over to this form (scripted pipeline?), and I converted the code below without thinking because Jenkins is not something I work with everyday...
steps {
when {
branch 'dev'
}
}
post {
always {
deleteDir()
to
try {
if (env.BRANCH_NAME == 'dev'){
} finally {
deleteDir()
Anyway, the command deleteDir() delete the directory, and therefore it can't find that file anymore.
Thanks for the help guys. Much appreciated.
Lesson: check your code to understand what's it's doing. Copy and paste will only help to a certain extent.

jenkins ssh agent fail to copy war to remote server

I am new to jenkin, I have created a jenkinFile to build a war and copy it to a remote machine on tomcat server based on the following tutorial:
https://thenucleargeeks.com/2020/05/31/declarative-jenkins-pipeline-to-deploy-java-web-application/
Please find below jenkinFile which i have created:
#!/usr/bin/env groovy
pipeline {
environment {
NAME = readMavenPom().getArtifactId()
}
agent any
options {
timeout(time: 1, unit: 'HOURS')
buildDiscarder(logRotator(daysToKeepStr: '10', numToKeepStr: '5', artifactNumToKeepStr: '2'))
}
stages {
stage("Git Checkout"){
steps{
git branch: 'develop',
credentialsId: 'jenkins', url: 'https://gitlab.gov/ih/ih-por.git'
}
}
stage('Maven build') {
steps {
sh "mvn clean package"
sh "mv target/*.war target/UI.war"
}
}
stage("deploy-dev"){
steps{
sshagent(['user-id-tomcat-deployment']) {
sh """
scp -o StrictHostKeyChecking=no target/UI.war
root#192.168.1.000:/opt/tomcat/webapps/
ssh root#192.168.1.000 /opt/tomcat/bin/shutdown.sh
ssh root#192.168.1.000 /opt/tomcat/bin/startup.sh
"""
}
}
}
}
}
I have also created added the private key on my jenkin server.
However when I build the project on jenkins the following error is displayed by ssh agent:
[Pipeline] { (deploy-dev)
[Pipeline] sshagent (hide)
[ssh-agent] Using credentials root (This defines the credential to login remote server where tomcat is installed for deployment purpose)
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-r2GnAq9cX2F8/agent.37244
SSH_AGENT_PID=37247
Running ssh-add (command line suppressed)
Identity added: /var/lib/jenkins/workspace/InfoHighway/portal-ui-deploy#tmp/private_key_6145932096571627059.key (root#localhost.localdomain)
[ssh-agent] Started.
[Pipeline] {
[Pipeline] sh
+ scp -o StrictHostKeyChecking=no target/portalUI.war
usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
[-l limit] [-o ssh_option] [-P port] [-S program]
[[user#]host1:]file1 ... [[user#]host2:]file2
[Pipeline] }
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 37247 killed;
[ssh-agent] Stopped.
[Pipeline] // sshagent
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Any idea what I am doing wrong please?
You have a new line after the target/UI.war so the scp command is missing the target parameter.
Try to run it with the full scp command in a single line:
stage("deploy-dev"){
steps{
sshagent(['user-id-tomcat-deployment']) {
sh """
scp -o StrictHostKeyChecking=no target/UI.war root#192.168.1.000:/opt/tomcat/webapps/
ssh root#192.168.1.000 /opt/tomcat/bin/shutdown.sh
ssh root#192.168.1.000 /opt/tomcat/bin/startup.sh
"""
}
}
}

Jenkins fails on empty grep result

I need to filter a file in jenkins. Filtering works as long as the result is not empty. But if the resulting output is empty, the pipeline fails with ERROR: script returned exit code 1 Finished: FAILURE
Example:
#!groovy
pipeline {
agent any
stages {
stage ('mystage') {
steps {
script {
sh "echo '' > myfile"
sh "echo 'foo 0' >> myfile"
sh "echo 'foo 1' >> myfile"
sh "grep foo myfile"
sh "grep ba myfile"
}
}
}
}
}
output:
+ echo ''
[Pipeline] sh
+ echo 'foo 0'
[Pipeline] sh
+ echo 'foo 1'
[Pipeline] sh
+ grep foo myfile
foo 0
foo 1
[Pipeline] sh
+ grep ba myfile
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
routing the output to a file with grep ba myfile > catchoutput does not work.
How can I output the grep result, without the pipeline failing in this edge case?
Adding a dummy line like sh "echo 'dummyline that won't match' >> myfile" seems to work but is a hack. Is there a clean solution?
We can take return value in a variable:
def ret = sh(script: 'grep ba myfile', returnStdout: true)
More info : https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#code-sh-code-shell-script.
Note you can also add returnStatus: true, so that jenkins step does not fail even if there is failure in the command.

Jenkins Pipeline "yarn install" command not found

This is my first Jenkins script, it currently operates well on Linux but I migrate to MacOS (High Sierra) with the result of getting shell script error.
Node and yarn packages are installed on local Jenkins user. I can't figure out why this error just happens, could anyone give me a hand on this?
Here is my Jenkins file:
node {
stage('Check out') {
checkout scm
}
stage('Prepare') {
sh "yarn install"
}
stage('Test') {
sh "yarn test"
}
stage('Sonar') {
if (env.BRANCH_NAME == 'dev') {
def scannerHome = tool 'sonar scanner';
withSonarQubeEnv('sonar') {
sh "${scannerHome}/bin/sonar-scanner"
}
}
}
}
And full log:
14:43:11 Connecting to https://api.github.com using hariklee/******
Obtained Jenkinsfile from 6c639bd70ac86cbe6a49ac0b58bcc10e3c64a375
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in
/Users/Shared/Jenkins/Home/workspace/wingman_423_ci_cd-7PSSGRAMBTXUQRESYCNVODXU7IZJLJLPHQOE3KYEPCSAAYAFFD4A
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Check out)
[Pipeline] checkout
git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url https://github.com/wingman-xyz/app.git # timeout=10
Fetching without tags
Fetching upstream changes from https://github.com/wingman-xyz/app.git
git --version # timeout=10
using GIT_ASKPASS to set credentials
git fetch --no-tags --progress https://github.com/wingman-xyz/app.git +refs/heads/423_ci_cd:refs/remotes/origin/423_ci_cd
Checking out Revision 6c639bd70ac86cbe6a49ac0b58bcc10e3c64a375 (423_ci_cd)
git config core.sparsecheckout # timeout=10
git checkout -f 6c639bd70ac86cbe6a49ac0b58bcc10e3c64a375
Commit message: "jenkins test"
First time build. Skipping changelog.
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Prepare)
[Pipeline] sh
[wingman_423_ci_cd-7PSSGRAMBTXUQRESYCNVODXU7IZJLJLPHQOE3KYEPCSAAYAFFD4A] Running shell script
yarn install
/Users/Shared/Jenkins/Home/workspace/wingman_423_ci_cd-7PSSGRAMBTXUQRESYCNVODXU7IZJLJLPHQOE3KYEPCSAAYAFFD4A#tmp/durable-cf573520/script.sh: line 2: yarn: command not found
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
GitHub has been notified of this commit’s build result
ERROR: script returned exit code 127
Finished: FAILURE
There is no yarn command in your PATH variable.
Do npm install -g yarn before
stage('Prepare') {
sh "npm install -g yarn"
sh "yarn install"
}
If you get an error about not found npm command then you will have to add npm explicitly to your PATH using withEnv() {}
withEnv(['PATH+NODE=/something=/path/to/node/bin']) {
stage('Prepare') {
sh "npm install -g yarn"
sh "yarn install"
}
}

Resources