I would like the \r\n characters will not be escaped.
Here is my script example in jenkins pipeline:
String value = "test\r\n"
def searchInfo = /${value}/
def searchInfo1 = /test\r\n/
println value
println searchInfo
println searchInfo1
and the output is :
[Pipeline] echo
test
[Pipeline] echo
test
[Pipeline] echo
test\r\n
which means only /test\r\n/ works, but I would like to use a string variable in / /, because the value is not fixed. and what I can think out is using ${} in / /, but it does not work well.
I have no idea why /${test}/ does not work properly.
please help, thanks!
I'm coding this function in Jenkins to query Artifactory:
def curlDockerArtifact(URL, registryName, moduleName, tag, token) {
def controlURI = "${URL}/artifactory/api/storage/${registryName}/${moduleName}/${tag}"
def result = sh(script: """
curl -I -H \'Authorization: Bearer $token\' \
https://$controlURI -o /dev/null -w \'%{http_code}\' -s
""", returnStdout: true)
}
But I get this warning which I'm tying to avoid.
Warning: A secret was passed to "sh" using Groovy string interpolation, which is insecure.
I tried using single quotes but the variables don't get correctly interpreted from Groovy. Any idea how to fix/refactor the code?
You have to keep double quotes like you do, but you need to escape the $ sign for the token. Like this :
curl -I -H \'Authorization: Bearer \$token\'
The groovy will not interpolate the variable and the correct value will be passed on the shell level.
More informations : https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#interpolation-of-sensitive-environment-variables
I'm running a curl command in my Jenkinsfile.
post {
success {
script {
sh '''
|SCORE=+1
|GERRIT_COMMENT="$(cat <<-EOL
|Sonar result was: SUCCESS
|Report: ${Jenkins_URL}/job/${JOB_NAME}/${BUILD_NUMBER}/artifact/report1.txt
|EOL
|)"
|curl -s -u ${apiToken}: ${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children | json_pp -json_opt pretty,canonical > report1.txt
|echo "Voting unsuccessful"
'''.stripMargin().stripIndent()
archiveArtifacts artifacts: 'report1.txt', fingerprint: true
echo 'I Succeeded'
}
}
But I get the error
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/bin/json_pp
I can't use jq as it's not installed and installing it isn't an option.
The curl command works fine on my terminal but is failing in my Jenkins pipeline.
Also, when I do this instead, it works.
post {
success {
script {
sh '''
|SCORE=+1
|GERRIT_COMMENT="$(cat <<-EOL
|Sonar result was: SUCCESS
|Report: ${Jenkins_URL}/job/${JOB_NAME}/${BUILD_NUMBER}/artifact/report1.txt
|EOL
|)"
|echo "Voting unsuccessful"
'''.stripMargin().stripIndent()
sh """
curl -s -u ${apiToken}: '${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children' | json_pp -json_opt pretty,canonical > report1.txt
"""
archiveArtifacts artifacts: 'report1.txt', fingerprint: true
echo 'I Succeeded'
}
}
But it throws a warning in the console output.
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure. Affected argument(s) used the following variable(s): [apiToken]
What am I doing wrong, please?
In a Jenkins pipeline, how would you properly pass a JSON response using curl into a file?
I recommend to not use shell scripts whenever it is possible. Shell scripts are not cross platform and require installing additional tools (e.g. curl).
In your case the curl call could be replaced by the httpRequest step.
First let's replace the curl call and saves the result in a componentTree.json file:
httpRequest(
url: "${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children",
authorization: 'id-of-credentials-which-was-used-to-create-the-apiToken-variable',
outputFile: 'componentTree.json'
)
You want to format the JSON data in a human-readable format, so let's use the readJSON and writeJSON steps:
def json = readJSON(file: 'componentTree.json')
writeJSON(json: json, file: 'report1.txt', pretty: 4)
Now the report1.txt file contains JSON formatted with indent 4.
The componentTree.json file is written and read only once, so let's decrease the number of the IO operations:
def response = httpRequest(
url: "${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children",
authorization: 'id-of-credentials-which-was-used-to-create-the-apiToken-variable'
)
def json = readJSON(text: response.content)
writeJSON(json: json, file: 'report1.txt', pretty: 4)
About the warning:
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure. Affected argument(s) used the following variable(s): [apiToken]
Secrets never should be interpolated because they may contain special characters which could be interpreted. Example:
my secret: My' https://example.org; cat /etc/passwd; echo \
command: curl -u '${password}' https://private.server/path/file.txt
After the interpolation the following command is called:
curl -u 'My' https://example.org; cat /etc/passwd; echo \' https://private.server/path/file.txt
There are two options to fix it:
if apiToken is an environment variable:
sh "curl -s -u \$apiToken: '${Sonar_URL}/api/measures/component..."
if apiToken is a Groovy variable:
withEnv(["CREDENTIALS=${apiToken}"]) {
sh "curl -s -u \$CREDENTIALS: '${Sonar_URL}/api/measures/component..."
}
In both cases the dollar sign ($) is escaped before the credentials which means that shell script will resolve it (it will be taken from environment variables).
I have a variable containing an HTML file string (basically an E2E test result) in my jenkinspipeline:
OUT = sh(script: "kubectl --kubeconfig ./kubeconfig-${scenario} logs \$(kubectl --kubeconfig ./kubeconfig-${scenario} get pods --selector=job-name=greyboxtest -o jsonpath='{.items[0].metadata.name}')", returnStdout:true)
HTML = OUT.substring(OUT.indexOf('HTMLSTART')+9, OUT.indexOf('HTMLEND')-1)
echo(HTML)
sh "echo ${HTML} >> ./reports/test.html"
However, the echo line gives me the following error: Syntax error: newline unexpected. I suspect it's because of the special characters like <, > or something like that. since the JSON version of the testresult works using the same method.
Is there any other way or trick to write the HTML string to the filesystem?
Here's an example of doing it a little Groovier, so to speak.
node() {
stage ('WriteHTMLOutput') {
OUT = sh(script: "kubectl --kubeconfig ./kubeconfig-${scenario} logs \$(kubectl --kubeconfig ./kubeconfig-${scenario} get pods --selector=job-name=greyboxtest -o jsonpath='{.items[0].metadata.name}')", returnStdout:true)
HTML = OUT.substring(OUT.indexOf('HTMLSTART')+9, OUT.indexOf('HTMLEND')-1)
sh "rm -f /tmp/test.html"
f = new File("/tmp/test.html")
f.append("${HTML}")
}
}
I am trying to print a variable in Jenkins. But I am getting an error saying "bad substitution". I am using Jenkinsfile to achieve that. This is what I am doing.
static def printbn() {
sh '''
#!/usr/bin/env bash
echo \"${env.BUILD_NUMBER}\"
'''
}
pipeline {
agent any
stages {
stage('Print Build Number') {
steps {
printbn()
}
}
}
}
Error that I am getting
/var/lib/jenkins/workspace/groovymethod#tmp/durable-7d9ef0b0/script.sh: line 4: ${steps.env.BUILD_NUMBER}: bad substitution
NOTE: I am using Jenkins version Jenkins ver. 2.163
In Shell, variable name is not allow use ., that's why you get following error: bad substitution
In Groovy, there are 4 ways to represent a string:
single quote: ' a string '
tripe single quote: ''' a string '''
double quote: " a string "
tripe double quote: """ a string """
And Groovy only execute string interpolation on double and triple double quote string.
For example:
def name = 'Tom'
print "Hello ${name}"
print """Hello ${name}"""
// do interpolation before print, thus get Hello Tom printed out
print 'Hello ${name}'
print '''Hello ${name}'''
//no interpolation thus, print Hello ${name} out directly.
BUILD_NUMBER is Jenkins job's build-in environment variable. You can directly access it in shell/bat.
static def printbn() {
sh '''
#!/usr/bin/env bash
echo ${BUILD_NUMBER}
// directly access any Jenkins build-in environment variable,
// no need to use pattern `env.xxxx` which only works in groovy not in shell/bat
'''
}
If you want use env.xxxx pattern, you can archive that via groovy string interpolation.
static def printbn() {
// use pipeline step: echo
echo "${env.BUILD_NUMBER}" // env.BUILD_NUMBER is groovy variable
// or use pipeline step: sh
sh """#!/usr/bin/env bash
echo ${env.BUILD_NUMBER}
"""
// will do interpolation firstly, to replace ${env.BUILD_NUMBER} with real value
// then execute the whole shell script.
}