Hi I am trying to remove substring "-unsigned" from filename in jenkins pipeline script.
where filePattern app/build/outputs/**/-release.apk".
I wrote below groovy script
findFiles(glob: filePattern).each { file ->
sh """
mv ${file.path} "${file.path//-unsigned/}"
"""
}
getting error unexpected char : 0XFFFF.
Can suggest where exactly I am missing. or suggest how to remove substring from file name in groovy.
not sure it's the best way to rename files however:
findFiles(glob: filePattern).each { file ->
sh """
mv ${file.path} "${file.path - '-unsigned'}"
"""
}
issue in your code that you have // in this expression ${file.path // ...}
and compiler could take it as a single line comment
try to run this in groovy console:
"""
${'abc' //no matter what here}
"""
//comment here
^^^ compilation error: unexpected char: 0xFFFF
See bash(1) - Linux man page:
EXPANSION
[...]
Parameter Expansion
[...]
${parameter/pattern/string}
Pattern substitution. [...] Parameter is expanded and the longest match of pattern against its value is replaced with string.
So it should be "${file.path/-unsigned//}".
Related
I'm attempting to run the following command in a shell block in my Jenkins pipeline:
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'"'
This command works perfectly fine when I run it directly on the Jenkins node in shell.
When I insert it into the Pipeline like this:
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${env.AWS_ACCOUNT_NUMBER}:task-definition/\${env.TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I get a Bad substitution error without any more information. As far as I know, I'm escaping variables and quotation correctly. I can bypass the error if I remove the double quotes like this:
jq '.Resources[].TargetService.Properties.TaskDefinition = "arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'
But that ends up processing the variables literally.
Notes: I'm aware of the security issue by not passing jq --arg and prepared to modify my command after I can get the simpler format working. revision.txt contains a numeric value. The env.* variables are declared earlier as part of the pipeline environment.
env is a Jenkins Object and you seem to be escaping env.* variables as well. If you have already exported these variables as Environment variables they should be available to you in the shell environment. So simply drop the env part from the variables or remove the escape characters from such variables and let Jenkins interpolate them.
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${AWS_ACCOUNT_NUMBER}:task-definition/\${TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I'm trying to reference a secret file to run with newman under a sub directory of the workspace like so:
String integrationFile = "src/test/postman/integration.json"
String environmentFile = "src/test/postman/environment-dev.json"
String reportFile = "integrationReport.xml"
String reportArgs = "--reporters cli,junit --reporter-junit-export ${reportFile}"
node {
withCredentials([file(credentialsId: "${env.FILE_KEY}", variable: "FILE_PATH")]) {
String dataFile = "${env.FILE_PATH}"
dir('sub-dir') {
git branch: 'master',
credentialsId: "${config.GitHubKeyId}",
url: 'https://github.com/xxx/repo.git'
withEnv(["PATH=${tool 'nodejs-12.8.0'}/bin:${env.PATH}"]) {
try {
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d ${dataFile} ${reportArgs}")
} catch (error) {
throw error
} finally {
junit "${reportFile}"
}
}
}
}
}
}
But when I run the code above, Jenkins throws an error:
error: iteration data could not be loaded
ENOENT: no such file or directory, open '/var/jenkins_home/workspace/Platform'
The path looks to be truncated because when I run a pwd command before the node closure runs, the workspace should be:
/var/jenkins_home/workspace/Platform Management/JJob#2
My question is, why is Jenkins doing this? Do I need to format the variable of the secret another way? Or should I reference it differently?
I know the file exists because in another Jenkins pipeline that does not have a sub directory (dir("")), it works fine.
I see from the Jenkins docs about withCredentials shows that how the file is reference gets tricky when you move between directories, see here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Here are the things that I've tried:
${env.FILE_PATH}
${FILE_PATH}
$FILE_PATH
(all of the above with double and single quotes around the sh command)
Any help is appreciated,
Thanks!
Ok - after playing around alot more with it, I eventually added a double quote around the variable to keep the spaces. This SO helped me out: Jenkins coping with spaces in batch file arguments
In the newman command line script, I just had add double quotes within the param args, like so:
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d \"${dataFile}\" ${reportArgs}")
I have a Jenkinsfile which uses three tick marks surrounding a command to execute as in:
sh ''' command '''
We have no idea why three tick marks are required or what role they perform.
This syntax is seen in the Jenkinsfile doc set.
This has nothing at all to do with bash (in which triple-quotes have no special meaning at all), and everything to do with Groovy (the separate, non-bash interpreter that parses Jenkinsfiles).
In Groovy, but not in bash, strings must use triple-quotes to span multiple lines.
In the context of a sh directive in a Jenkinsfile, the content of your triple-quoted string is passed to a shell as a script to execute; however, the syntax is parsed by Groovy, so it's only Groovy that cares about the quotes themselves (as opposed to the quoted content).
Can you give more idea about what kind of command is it, is it a unix command or some script ?
The single quote and its variation like '''(3 ticks) as mentioned in question skip the variable expansion, and it could used to show what is being executed.
echo '''Updating JAVA_HOME variable :
export $JAVA_HOME="$NEW_JAVA_HOME" '''
However in your question, a command (some string) is enclosed between 3 ticks marks and sh tries to execute this command or script. One such example below
$ echo "echo hello" > /tmp/tesh.sh
$ sh '''/tmp/test.sh'''
hello
I try to make simple pipeline on Jenkins to remove files from few directories time to time. I decided not to create python script with Jenkinsfile as new project, instead of it I try to define new pipeline script in Jenkins job.
pipeline {
agent any
stages {
stage('Check virtualenv') {
steps {
sh """
rm -r /mnt/x/some/directory/Problem\ 1.0/path
"""
}
}
}
}
And I got an error WorkflowScript: 4: unexpected char: '\'. How can I use path with whitespace on it without using backslash? Any other ideas how define path?
The '\' character is a special character in Groovy. If you tried to compile this kind of code with the normal Groovy compiler, it would give you a better error message. The easiest way to handle it would be to escape it:
"""
rm -r /mnt/x/some/directory/Problem\\ 1.0/path
"""
You can modify the shell command as follows:
sh """
rm -r /mnt/x/some/directory/Problem""" + """ 1.0/path"""
Provide space before 1.0 as required. Hope this helps.
How to append text to a file in a Jenkinsfile injecting the Jenkins BUILD_ID
I wish to see:
version := "1.0.25"
where 25 is the BUILD_ID
Here is my attempt:
import hudson.EnvVars
node {
stage('versioning'){
echo 'retrieve build version'
sh 'echo version := 1.0.${env.BUILD_ID} >> build.sbt'
}
}
Error:
version:=1.0.${env.BUILD_ID}: bad substitution
Note the file is in the current directory
The pipeline built in writeFile is also very useful here but requires a read+write process to append to a file.
def readContent = readFile 'build.sbt'
writeFile file: 'build.sbt', text: readContent+"\r\nversion := 1.0.${env.BUILD_ID}"
env.BUILD_ID is a groovy variable, not a shell variable. Since you used single-quotes (') groovy will not substitute the variables in your string and the shell doesn't know about ${env.BUILD_ID}. You need to either use double-quotes " and let groovy do the substitution
sh "echo version := 1.0.${env.BUILD_ID} >> build.sbt"
or use the variable the shell knows
sh 'echo version := 1.0.$BUILD_ID >> build.sbt'
and since you need the version surrounded with doublequotes, you'd need something like this:
sh "echo version := \\\"1.0.${env.BUILD_ID}\\\" >> build.sbt"
I've used dirty little wrapper function to implement Stefan Crain's answer above:
def appendFile(String fileName, String line) {
def current = ""
if (fileExists(fileName)) {
current = readFile fileName
}
writeFile file: fileName, text: current + "\n" + line
}
I really don't like it, but it does the trick and it gets round escaping quotes via slashy strings,e.g.:
def tempFile = '/tmp/temp.txt'
writeFile file: tempFile, text: "worthless line 1\n"
// now append the string 'version="1.2.3" # added by appendFile\n' to tempFile
appendFile(tempFile,/version="1.2.3" # added by appendFile/ + "\n")