Pipeline run on new branch - fails on first build, but succeeds if the build is replayed - jenkins

I have a build job which is showing strange behavior. When I create a new branch, or tag, the first deploy will fail in my "Build config" stage. If I replay that same build with no alterations, it works beautifully. I assume it has something to do with the workspace, but the logs are truly unhelpful.
The log is just as unhelpful:
15:57:34 [Pipeline] stage
15:57:34 [Pipeline] { (Building config)
15:57:34 [Pipeline] withEnv
15:57:34 [Pipeline] {
15:57:34 [Pipeline] echo
15:57:34 Building config file /home/jenkins/workspace/MyProject_1.11.111/src/globals.inc.php
15:57:34 [Pipeline] script
15:57:34 [Pipeline] {
15:57:35 [Pipeline] readFile
15:57:35 [Pipeline] }
15:57:35 [Pipeline] // script
15:57:35 [Pipeline] }
15:57:35 [Pipeline] // withEnv
15:57:35 [Pipeline] }
15:57:35 [Pipeline] // stage
stage ('Building config'){
when {
anyOf{
buildingTag()
expression{
return params.DEPLOY == true
}
}
}
environment {
TEMPLATE_FILE="globals.template.inc.php"
CONFIG_FILE="globals.inc.php"
}
steps {
echo "Building config file ${SOURCE_DIR}/${CONFIG_FILE}"
script {
def inptext = readFile file: "${SOURCE_DIR}/${TEMPLATE_FILE}"
//save deploydate
def deployDate = (new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm")).format((new Date()))
inptext = inptext.replaceAll(~/¤SITE_TITLE¤/, "${SITE_TITLE}")
inptext = inptext.replaceAll(~/¤SITE_NAME¤/, "${SITE_NAME}")
inptext = inptext.replaceAll(~/¤SITE_FQDN¤/, "${SITE_FQDN}")
inptext = inptext.replaceAll(~/¤DEFAULT_FROM_EMAIL¤/, "${DEFAULT_FROM_EMAIL}")
inptext = inptext.replaceAll(~/¤SESSION_NAME¤/, "${SESSION_NAME}")
inptext = inptext.replaceAll(~/¤CERT_LOCATION¤/, "${CERT_LOCATION}")
inptext = inptext.replaceAll(~/¤MYSQL_HOSTNAME¤/, "${MYSQL_HOSTNAME}")
inptext = inptext.replaceAll(~/¤MYSQL_PORT¤/, "${MYSQL_PORT}")
inptext = inptext.replaceAll(~/¤MYSQL_DB_NAME¤/, "${MYSQL_DB_NAME}")
inptext = inptext.replaceAll(~/¤MYSQL_USERNAME¤/, "${MYSQL_USERNAME}")
inptext = inptext.replaceAll(~/¤MYSQL_PASSWORD¤/, "${MYSQL_PASSWORD}")
inptext = inptext.replaceAll(~/¤DEPLOY_DATE¤/, "${deployDate}")
inptext = inptext.replaceAll(~/¤GIT_BRANCH¤/, "${env.GIT_BRANCH}")
inptext = inptext.replaceAll(~/¤GIT_COMMIT¤/, "${env.GIT_COMMIT}")
inptext = inptext.replaceAll(~/¤GIT_TAG¤/, "${env.TAG_NAME}")
writeFile file: "${SOURCE_DIR}/${CONFIG_FILE}", text: inptext
}
}
}
I am aware that solutions like dotenv exist, but I can't implement that at the moment
If it has any relevance, here is the preceding stage as well..
stage ('Staging'){
steps {
script {
properties([
parameters([
booleanParam(
defaultValue: false,
description: 'whether to trigger a deploy',
name: 'DEPLOY'
),
string(
defaultValue: "${env.BRANCH_NAME}",
name: 'BUILD_NAME'
),
string(
defaultValue: '',
description: 'path to deploy to',
name: 'DEPLOY_DIR'
),
string(
defaultValue: '',
description: 'SSH server name to deploy to',
name: 'SSH_SERVER_NAME'
),
string(
defaultValue: '',
description: 'Username for SSH connection',
name: 'SSH_USERNAME'
),
string(
defaultValue: '',
description: 'Title of the site',
name: 'SITE_TITLE'
),
string(
defaultValue: '',
name: 'SITE_NAME'
),
string(
defaultValue: '',
description: 'Fully Qualified Domain Name',
name: 'SITE_FQDN',
trim: true
),
string(
defaultValue: '',
description: 'email to send from',
name: 'DEFAULT_FROM_EMAIL',
trim: true
),
string(
defaultValue: '',
description: 'Sessionname (this should be unique foer every environment)',
name: 'SESSION_NAME',
trim: true
),
string(
defaultValue: '',
description: 'full path to certificate location',
name: 'CERT_LOCATION',
trim: true
),
string(
defaultValue: 'localhost',
description: 'MySQL Servername',
name: 'MYSQL_HOSTNAME',
trim: true
),
string(
defaultValue: '3306',
description: 'MySQL port number',
name: 'MYSQL_PORT',
trim: true
),
string(
defaultValue: '',
description: 'MySQL database name',
name: 'MYSQL_DB_NAME',
trim: true
),
string(
defaultValue: '',
description: 'MySQL username ',
name: 'MYSQL_USERNAME',
trim: true
),
string(
defaultValue: '',
description: 'Password for MYSQL_USERNAME',
name: 'MYSQL_PASSWORD',
trim: true
)
])
])
currentBuild.displayName = "${params.BUILD_NAME}-${env.BUILD_NUMBER}"
}
echo "Cleanup build artifacts"
dir("${WORKSPACE}/build"){
deleteDir()
}
dir("${WORKSPACE}/src/!devHelpers"){
deleteDir()
}
echo "Prepare for build"
//re-create folders
sh "mkdir ${WORKSPACE}/build ${WORKSPACE}/build/api ${WORKSPACE}/build/coverage ${WORKSPACE}/build/logs ${WORKSPACE}/build/pdepend ${WORKSPACE}/build/phpdox"
echo "Running composer"
sh "composer install -o -d ${SOURCE_DIR}"
}
}

Related

groovy: MissingPropertyException: No such property:

I am facing an issue when shell command is returning non existent value because output produces no value as env.version == '1.0.0.232'-->false, does not exist in pypy server.
but when env.version == '1.0.0.23'--> true, does exist in pypy server, code proceed as normal.
Jenkins code:
pipeline {
agent { label 'master' }
parameters {
string(defaultValue: 'DEV', description: '', name: 'ENV', trim: true)
string(defaultValue: 'sys', description: '', name: 'platform_type', trim: true)
string(defaultValue: 'server2', description: '', name: 'dev_app_host', trim: true)
string(defaultValue: 'server1', description: '', name: 'dev_xbar_host', trim: true)
string(defaultValue: '1.0.0.23', description: '', name: 'VERSION', trim: true)
booleanParam(defaultValue: false, description: 'force build if possible', name: 'force_build')
}
environment {
}
stages {
stage('build') {
steps {
script {
try{
try{
def version_exists = sh(script: "ssh -o StrictHostKeyChecking=no ansible#pip_server ls /var/pypi/packages/dev/ | grep ${env.app_module_name} | grep ${env.VERSION}" , returnStdout: true) ?: 'no_files_found'
echo version_exists
echo version_exists.inspect()
echo version_exists.dump()
} catch(e){
echo "inner exception: ${e}"
}
} catch (e) {
echo "outer exception: ${e}"
currentBuild.result = 'FAILURE'
}
}
}
}
}
}
Jenkins relevant long:
+ grep 1.0.0.232
+ grep dvmt_event_processor
+ ssh -o StrictHostKeyChecking=no ansible#pip_server ls /var/pypi/packages/dev/
[Pipeline] echo
inner exception: hudson.AbortException: script returned exit code 1
[Pipeline] echo
outer exception: groovy.lang.MissingPropertyException: No such property: version_exists for class: groovy.lang.Binding
PS: can the shell command be improved upon?
grep returns status code 1 when it finds no matching lines. Jenkins interprets a non-0 status as the script failing, so it throws a hudson.AbortException rather than assigning the output to version_exists.
Try something like this:
def version_exists = sh( " ... | grep ${env.VERSION} || echo not_found", returnStdout: true)

Unable to pass a parameter from one pipeline job to another

Editing the question: I am trying to run a simple pipeline job which triggers another pipeline job and sends parameter values.
I tried a simplified usecase in the example below
Piepeline - Parent
pipeline{
agent any
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('Invoke sample_pipleline') {
steps {
CommitID = 'e2b6cdf1e8018560b3ba51cbf253de4f33647b5a'
Branch = "master"
input id: 'Environment', message: 'Approval', parameters: [choice(choices: ['FRST', 'QA', 'PROD'], description: '', name: 'depServer')], submitter: 'FCMIS-SFTP-LBAAS', submitterParameter: 'user'
build job: 'simple_child',
parameters: [string(name: 'CommitID', value: 'e2b6cdf1e8018560b3ba51cbf253de4f33647b5a'),string(name: 'Environment', value: depServer), string(name: 'Branch', value: 'master')],
quietPeriod: 1
}
}
}
}
Pipeline - child
pipeline{
agent any
parameters {
string defaultValue: '', description: 'K', name: 'depServer'
}
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('CodePull') {
steps {
echo "Testing"
echo "${depServer}"
}
}
}
}
When I run the parent pipeline it did not trigger child pipeline but gave error.
Started by user ARAV
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Windows_aubale in C:\Users\arav\Documents\Proj\Automation\Jenkins\Jenkins_slave_root_directory\workspace\sample_parent2
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Invoke sample_pipleline)
[Pipeline] input
Input requested
Approved by ARAV
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
groovy.lang.MissingPropertyException: No such property: depServer for class: groovy.lang.Binding
After implementing changes suggested and with some tweaks The parent job triggers the child job but the child log shows that it doesn't receive the parameter passed.
Started by upstream project "sample_parent" build number 46
originally caused by:
Started by user ARAV
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/simple_child
[Pipeline] {
[Pipeline] stage
[Pipeline] { (CodePull)
[Pipeline] echo
Testing
[Pipeline] echo
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Please help me understand what am I doing wrong here.
Appreciate your help!!
If your child pipeline has parameter named "depServer":
parameters {
string name: 'depServer', defaultValue: '', description: ''
}
You should provide a value for it:
build job: 'simple_child',
parameters: [string(name: 'depServer', value: 'SOMETHING']
Finally, you should address it:
steps {
echo "Testing"
echo "${params.depServer}"
}
groovy.lang.MissingPropertyException: No such property: depServer for
class: groovy.lang.Binding
This means that you don't have defined a variable depServer.
Fix it by assigning the result of the input step to variable depServer:
steps {
script {
def input_env = input id: 'Environment', message: 'Approval', parameters: [choice(choices: ['FRST', 'QA', 'PROD'], description: '', name: 'depServer')], submitter: 'FCMIS-SFTP-LBAAS', submitterParameter: 'user'
build job: 'simple_child',
parameters: [string(name: 'CommitID', value: 'aa21a592d1039cbce043e5cefea421efeb5446a5'),string(name: 'Environment', value: input_env.depServer), string(name: 'Branch', value: "master")],
quietPeriod: 1
}
}
I've added a script block, to be able to create and assign a variable.
The input actually returns a HashMap that looks like this:
[depServer:QA, user:someUser]
That's why we have to write input_env.depServer as argument for the build job.
Thank you so much zett42 and MaratC! So finally the code that worked is as follows(combining both the answers):
Parent script:
pipeline{
agent any
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('Invoke sample_pipleline') {
steps {
script{
def CommitID
def depServer
def Branch
CommitID = 'e2b6cdf1e8018560b3ba51cbf253de4f33647b5a'
Branch = "master"
userip = input id: 'Environment', message: 'Approval', parameters: [choice(choices: ['FRST', 'QA', 'PROD'], description: '', name: 'input_env')], submitter: 'FCMIS-SFTP-LBAAS', submitterParameter: 'user'
depServer = userip.input_env
echo "${depServer}"
build job: 'simple_child',
parameters: [string(name: 'CommitID', value: "${CommitID}"),
string(name: 'Environment', value: "${depServer}"),
string(name: 'Branch', value: "${Branch}")],
quietPeriod: 1
}
}
}
}
}
Child script:
pipeline{
agent any
parameters {
string defaultValue: '', description: 'K', name: 'Environment'
}
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('CodePull') {
steps {
echo "Testing"
echo "${params.Environment}"
}
}
}
}

Jenkins pipeline input script with one prompt only

Don't need 2 prompt in Jenkins pipeline with input script. In snippet, there is highlighted 1 and 2 where pipeline prompt 2 times for user input.
If execute this pipeline, prompt 1 will provide user to "Input requested" only, so no option to "Abort". Prompt 2 is okay as it ask to input the value and also "Abort" option.
Prompt 1 is almost useless, can we skip promt this and directly reach to prompt 2?
pipeline {
agent any
parameters {
string(defaultValue: '', description: 'Enter the Numerator', name: 'Num')
string(defaultValue: '', description: 'Enter the Denominator', name: 'Den')
}
stages {
stage("foo") {
steps {
script {
if ( "$Den".toInteger() == 0) {
echo "Denominator can't be '0', please re-enter it."
env.Den = input message: 'User input required', ok: 'Re-enter', // 1-> It will ask whether to input or not
parameters: [string(defaultValue: "", description: 'Enter the Denominator', name: 'Den')] // 2-> It will ask to input the value
}
}
sh'''#!/bin/bash +x
echo "Numerator: $Num\nDenominator: $Den"
echo "Output: $((Num/Den))"
'''
}
}
}
}
Yes, you can simplify the input by using $class: 'TextParameterDefinition'.
e.g.
pipeline {
agent any
parameters {
string(defaultValue: '', description: 'Enter the Numerator', name: 'Num')
string(defaultValue: '', description: 'Enter the Denominator', name: 'Den')
}
stages {
stage("foo") {
steps {
script {
if ( "$Den".toInteger() == 0) {
env.Den = input(
id: 'userInput', message: 'Wrong denominator, give it another try?', parameters: [
[$class: 'TextParameterDefinition', defaultValue: '', description: 'Den', name: 'den']
]
)
}
}
sh'''#!/bin/bash +x
echo "Numerator: $Num\nDenominator: $Den"
echo "Output: $((Num/Den))"
'''
}
}
}
}
If you want to go the extra mile and ask for both values:
def userInput = input(
id: 'userInput', message: 'Let\'s re-enter everything?', parameters: [
[$class: 'TextParameterDefinition', defaultValue: '', description: 'Denominator', name: 'den'],
[$class: 'TextParameterDefinition', defaultValue: '', description: 'Numerator', name: 'num']
]
)
print userInput
env.Den = userInput.den
env.Num = userInput.num

Version Number Plugin in Jenkins declarative pipeline

I'm trying to use version number plugin to format a version number for our
packages.
From some reason the placement for the version variable doesn't work
and when I echo the following I only get the build number, for instance: "...54"
def Version_Major = '1'
def Version_Minor = '0'
def Version_Patch = '0'
pipeline {
environment {
VERSION = VersionNumber([
versionNumberString: '${Version_Major}.${Version_Minor}.${Version_Patch}.${BUILD_NUMBER}',
worstResultForIncrement: 'SUCCESS'
]);
}
stage ('Restore packages'){
steps {
script{
echo "${VERSION}"
}
}
}
}
Edit: It does look like an issue with the plugin usage since this works:
properties([
parameters([
string(name: 'Version_Major', defaultValue: '1', description: 'Version Major'),
string(name: 'Version_Minor', defaultValue: '0', description: 'Version Minor'),
string(name: 'Version_Patch', defaultValue: '0', description: 'Version Patch')
])
])
pipeline {
agent any
environment {
VERSION = "${params.Version_Major}.${params.Version_Minor}.${params.Version_Patch}.${BUILD_NUMBER}"
}
stages{
stage ('Test'){
steps {
echo "${VERSION}"
}
}
}
}
You must define the variables inside the pipeline.
Try this:
pipeline {
environment {
Version_Major = '1'
Version_Minor = '0'
Version_Patch = '0'
VERSION = VersionNumber([
versionNumberString: '${Version_Major}.${Version_Minor}.${Version_Patch}.${BUILD_NUMBER}',
worstResultForIncrement: 'SUCCESS'
]);
}
stage ('Restore packages'){
steps {
script{
echo "${VERSION}"
}
}
}
}
If you need to use a parameter instead that's also possible via:
parameters {
string(name: 'PERSON', defaultValue: 'Mr Jenkins', description: 'Who should I say hello to?')
}
usage:
"Hello ${params.PERSON}"

How to get user input data in shell (Jenkinsfile)

I am trying to take user input through Jenkinsfile but I can't use that in the shell. Here is the code:
def userInput = timeout(time:60, unit:'SECONDS') {input(
id: 'userInput', message: 'URL Required', parameters: [
[$class: 'TextParameterDefinition', defaultValue: '', description: 'URL', name: 'url'],
])
}
node{
echo "Env jsfsjaffwef:"+userInput) //this works
echo "${userInput}" //but this does not
sh '''
python test.py ${userInput}
'''
}
Be careful about string interpolation: Variables will be replaced inside double quotes ("..", or the multi-line variant """), but not inside single quotes ('..', resp. '''..'''). So the sh step shouldn't have it replaced, the echo above should have it correctly.
def userInput = timeout(time:60, unit:'SECONDS') {input(
id: 'userInput', message: 'URL Required', parameters: [
[$class: 'TextParameterDefinition', defaultValue: '', description: 'URL', name: 'url'],
])
}
node {
echo "Env jsfsjaffwef:"+userInput) //this works
echo "${userInput}" // this should have also worked before
sh """
python test.py ${userInput}
"""
}
So make sure that you exactly apply the right quotes and don't just replace them compared to what people suggest here.
The following works for me on Jenkins 2.62:
def userInput = input(
id: 'userInput', message: 'URL Required', parameters: [
[$class: 'TextParameterDefinition', defaultValue: '', description: 'URL', name: 'url'],
])
node('master') {
sh "echo userInput is: ${userInput}"
}

Resources