I'm trying to reference a secret file to run with newman under a sub directory of the workspace like so:
String integrationFile = "src/test/postman/integration.json"
String environmentFile = "src/test/postman/environment-dev.json"
String reportFile = "integrationReport.xml"
String reportArgs = "--reporters cli,junit --reporter-junit-export ${reportFile}"
node {
withCredentials([file(credentialsId: "${env.FILE_KEY}", variable: "FILE_PATH")]) {
String dataFile = "${env.FILE_PATH}"
dir('sub-dir') {
git branch: 'master',
credentialsId: "${config.GitHubKeyId}",
url: 'https://github.com/xxx/repo.git'
withEnv(["PATH=${tool 'nodejs-12.8.0'}/bin:${env.PATH}"]) {
try {
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d ${dataFile} ${reportArgs}")
} catch (error) {
throw error
} finally {
junit "${reportFile}"
}
}
}
}
}
}
But when I run the code above, Jenkins throws an error:
error: iteration data could not be loaded
ENOENT: no such file or directory, open '/var/jenkins_home/workspace/Platform'
The path looks to be truncated because when I run a pwd command before the node closure runs, the workspace should be:
/var/jenkins_home/workspace/Platform Management/JJob#2
My question is, why is Jenkins doing this? Do I need to format the variable of the secret another way? Or should I reference it differently?
I know the file exists because in another Jenkins pipeline that does not have a sub directory (dir("")), it works fine.
I see from the Jenkins docs about withCredentials shows that how the file is reference gets tricky when you move between directories, see here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Here are the things that I've tried:
${env.FILE_PATH}
${FILE_PATH}
$FILE_PATH
(all of the above with double and single quotes around the sh command)
Any help is appreciated,
Thanks!
Ok - after playing around alot more with it, I eventually added a double quote around the variable to keep the spaces. This SO helped me out: Jenkins coping with spaces in batch file arguments
In the newman command line script, I just had add double quotes within the param args, like so:
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d \"${dataFile}\" ${reportArgs}")
Related
I'm attempting to run the following command in a shell block in my Jenkins pipeline:
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'"'
This command works perfectly fine when I run it directly on the Jenkins node in shell.
When I insert it into the Pipeline like this:
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${env.AWS_ACCOUNT_NUMBER}:task-definition/\${env.TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I get a Bad substitution error without any more information. As far as I know, I'm escaping variables and quotation correctly. I can bypass the error if I remove the double quotes like this:
jq '.Resources[].TargetService.Properties.TaskDefinition = "arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'
But that ends up processing the variables literally.
Notes: I'm aware of the security issue by not passing jq --arg and prepared to modify my command after I can get the simpler format working. revision.txt contains a numeric value. The env.* variables are declared earlier as part of the pipeline environment.
env is a Jenkins Object and you seem to be escaping env.* variables as well. If you have already exported these variables as Environment variables they should be available to you in the shell environment. So simply drop the env part from the variables or remove the escape characters from such variables and let Jenkins interpolate them.
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${AWS_ACCOUNT_NUMBER}:task-definition/\${TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I want to get a line from a file in my workspace. I am using this script :
stage('Test') {
steps {
script {
outputJenkins = 'output-jenkins.log'
sh "cd invoker && mvn clean install && mvn exec:java -Dexec.mainClass=\"com.JenkinsRunner\" -Dexec.args=\"qal ${GIT_COMMIT_HASH}\" > ../${outputJenkins}"
logFile = readFile(outputJenkins)
echo logFile
adminRepoLogLine = sh "echo logFile | grep \"Admin repo url is :::\""
echo adminRepoLogLine
}
}
}
But I am getting this error:
+ echo logFile
+ grep Admin repo url is :::
script returned exit code 1
The script works fine in my shell when I try it locally. Are there any contains around doing it in a JenkinsFile?
If we apply various fixes and improvements to the code in the question to achieve the desired functionality, then it will succeed:
stage('Test') {
steps {
script {
dir('invoker') {
sh(label: 'Maven Clean Install', script: 'mvn clean install')
// assign maven output to variable
String output = sh(label: 'Maven Git Log', script: "mvn exec:java -Dexec.mainClass=\"com.JenkinsRunner\" -Dexec.args=\"qal ${GIT_COMMIT_HASH}\"", returnStdout: true)
}
// assign regex return to variable
def adminRepoLogLine = output =~ /(.*Admin repo url is :::.*)/
// print extracted string from return
print adminRepoLogLine[0][1]
}
}
}
Note that GIT_COMMIT_HASH is neither an intrinsic Jenkins environment variable, nor defined in the pipeline code in the question, so it will need to be defined at Pipeline scope elsewhere in your code.
This is because the string literal logFile does not contain the string Admin repo url is :::. If there's no such match, then grep will exit with status 1.
You probably want to use
cat logFile | grep \"Admin repo url is :::\"
instead, or, even simpler:
grep \"Admin repo url is :::\" logFile
Append || true (or ||:) to the command if you want to avoid the errors when the log line does not appear.
I am having similar issue as mentioned here
I am trying to deploy an application via Jenkinsfile. For which I have to run this command on the deploy stage in Jenkins (if I hardcode the value then it works fine):
xldDeploy serverCredentials: 'usernam', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: 'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'
"testapp_1.0.4.5.Build39_TAG-test" is getting generated at running time. Which can be created by concating "${TagVersion}.Build${env.BUILD_NUMBER}_${ComponentTagName}"
I tried below code in my Jenkins pipeline:
stage('Deploy') {
node('noibuild01') {
if ("${env.Build_WildflyCPECommon}" == 'true') {
echo "${TagVersion}"
echo "${ComponentTagName}"
echo "${env.BUILD_NUMBER}"
script {
env.buildNumber = "${TagVersion}.Build${env.BUILD_NUMBER}_${ComponentTagName}"
env.packageid = "'Applications/Testapp/${env.buildNumber}'"
}
echo "${env.buildNumber}"
echo "${env.packageid}"
xldDeploy serverCredentials: 'nex8voo', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: "${env.packageid}"
}
}
}
I checked the output, it is showing correctly:
echo "${env.buildNumber}" giving
testapp_1.0.4.5.Build39_TAG-test
echo "${env.packageid}" giving
'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'
But xldDeploy serverCredentials: 'username', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: "${env.packageid}"
is taking as:
[/repository/ci/'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test']
Repository entity: ['Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'] not found
I think I can't use packageId: "${env.packageid}".
Is there anything I could try? Maybe Groovy or Python code?
Your packageid environment variable is not being assigned a concatenated string correctly. You have literal quotes inside the string interpolation quotes. You should change it to:
env.packageid = "Applications/Testapp/${env.buildNumber}"
to only interpolate the string, which is the functionality you want here.
Additionally, you do not need to interpolate the environment variable inside an empty string for your method parameter, so your method invocation can be cleaned up as:
xldDeploy serverCredentials: 'nex8voo', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: env.packageid
I have just started looking into a shared libarary with jenkins in order to combine a load of scripts and pipelines across multiple repos that are pretty much identical.
I have the shared lib loaded and working but when tryign to execute the scripts i the resources folder i keep geting not found errors:
../releaseTagging-EO2DMYOPJ6JGB6JT5Q2RSFJWJWWPALA7F25H7CQNYBEV4ITTEB6Q#tmp/build.sh: not found
I am creating a copy of the file using the following:
createTempLocation(String path) {
String tmpDir = pwd tmp: true
return tmpDir + File.separator + new File(path).getName()
}
and
copyGlobalLibraryScriptcall(String srcPath, String destPath = null) {
destPath = destPath ?: createTempLocation(srcPath)
writeFile file: destPath, text: libraryResource(srcPath)
echo "copyGlobalLibraryScript: copied ${srcPath} to ${destPath}"
sh "chmod +x ${destPath}"
echo "added executable permissions to ${destPath}"
return destPath
}
I am then calling the last function thusly:
runBuild(Map config) {
def script = copyGlobalLibraryScript('build.sh')
sh script
}
(i realise i can collapse the above function in to one line)
This in turn then gets called via (trimed the whole file to relevent part):
pipeline {
agent any
stages {
stage('Build') {
steps {
timestamps {
checkout scm
bbNotify( key: buildKey, name: BuildName) {
runBuild()
}
stash includes: '**', name: 'RelToSTAN'
}
}
}
}
This all fails with the error at the top of the question, however when sshing on to the build server i can find that file int he location specified.
I dont understand why Jenkins cannot find it and execute it.
The issue will be the following:
When using a java File object it‘ll always refer to some location on the Jenkins master. And of course it usually cannot run inside the sandbox.
On the other hand the readFile and writeFile methods always refer to some path on the build agent reserved by the node block where the call is encapsulated.
Long story short: Do not use the File class. Unfortunately you’ll need to create the temp path manually. But that shouldn’t be too hard.
I try to make simple pipeline on Jenkins to remove files from few directories time to time. I decided not to create python script with Jenkinsfile as new project, instead of it I try to define new pipeline script in Jenkins job.
pipeline {
agent any
stages {
stage('Check virtualenv') {
steps {
sh """
rm -r /mnt/x/some/directory/Problem\ 1.0/path
"""
}
}
}
}
And I got an error WorkflowScript: 4: unexpected char: '\'. How can I use path with whitespace on it without using backslash? Any other ideas how define path?
The '\' character is a special character in Groovy. If you tried to compile this kind of code with the normal Groovy compiler, it would give you a better error message. The easiest way to handle it would be to escape it:
"""
rm -r /mnt/x/some/directory/Problem\\ 1.0/path
"""
You can modify the shell command as follows:
sh """
rm -r /mnt/x/some/directory/Problem""" + """ 1.0/path"""
Provide space before 1.0 as required. Hope this helps.