Handle string interpolation in shell command in jenkins - jenkins

We have a pipeline where we need to invoke external API with Authorization header, whose value comes from Jenkins secret, which has been pre-configured.
When implemented as below, Jenkins complains for string interpolation.
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh """curl --location --request POST 'https://abc.example.com/api/endpoint' \
--header 'Authorization: Bearer ${Token}' \
--header 'Content-Type: application/json' \
--data-raw ${payload}""
We have tried will single quotes for sh and double quotes but nothing works out.
How it can be handled here?

Jenkins doesn't want you to interpolate passwords in the code but rather pass them as environment variables to the shell and let the shell command extract them, that is possible only for parameters that are loaded into the shell execution environment.
In declarative pipelines loading parameters and secrets into shell environment can be done using the environment directive and for scripted pipelines loading secrets can be done via the withCredentials keyword and loading regular parameters can be done via the 'withEnv` keyword.
In your case you have the Token parameter which is loaded into environment by the withCredentials step and the payload parameter which is probably not, so you are mixing two type of parameter contexts, more information on this is available in the Answer for this question.
To solve it you have two options.
The first option is to load the payload into the environment of the shell and use a single quoted groovy string:
withEnv(["PAYLOAD=${payload}"]) {
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw $PAYLOAD'''
}
}
Second option is to separate the construction of the string into two types, and handle each section with the relevant method:
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw ''' + payload
}

Related

Jenkins : Pass mixed parameters to Jenkins API

Current Jenkins parameter looks like below and <input_paramter> are the actual values to be substituted over there.
I have an idea on how to pass string parameters or json parameters separately but unsure how to handle them both simultaneously.
Referred Jenkins documentation as well as multiple questions posted on this forum, but could not figure that out.
curl -X POST https://myjenkins.instance/build \
-H 'Content-Type: application/json' \
--user user:token \
FILE_PATH="/home/mac/results" \
PACKAGE_VERSION=64 \
<how to handle the json parameters>

JRXML upload from Jenkins give HTTP-500 but works elsewhere

I am using Jasper REST API to publish reports on Jasper Server.
Below are the scenarios:
When I upload the .JRXML file from POSTMAN and then create a report from POSTMAN using the JRXML. It works.
When I upload the .JRXML file from POSTMAN and then create a report from Jenkins using the JRXML. It works.
When I upload the .JRXML file from Jenkins and then create a report from Jenkins using the JRXML. It gives me HTTP 500 Error
When I upload the .JRXML file from Jenkins and then create a report from Jenkins using the JRXML. It gives me HTTP 500 Error.
I have even tried with cURL and it works fine. Its just the Jenkins where it gives the error. I have tried a lot of things but it has no outcome.
cURL from POSTMAN:
#To upload JRXML
curl --location --request POST 'https://server-url/jasperserver-pro/rest_v2/resources/Reports/' \
--header 'Content-Type: application/jrxml' \
--header 'Content-Disposition: attachment; filename=form.jrxml' \
--header 'Content-Description: Uploaded jrxml file POSTMAN' \
--header 'Authorization: Basic %Base64 Creds%' \
--data-binary 'report-templates/form.jrxml'
#To create report
curl --location --request POST 'https://server-url/jasperserver-pro/rest_v2/resources/Reports' \
--header 'Content-Type: application/repository.reportUnit+xml' \
--header 'Authorization: Basic Base64 Creds' \
--data-raw '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<reportUnit>
<description/>
<label>new_report1</label>
<permissionMask>1</permissionMask>
<uri>/Reports</uri>
<version>0</version>
<dataSourceReference>
<uri>/Data_Sources/mysql</uri>
</dataSourceReference>
<alwaysPromptControls>true</alwaysPromptControls>
<controlsLayout>popupScreen</controlsLayout>
<inputControlRenderingView/>
<jrxmlFileReference>
<uri>/Reports/form.jrxml</uri>
</jrxmlFileReference>
</reportUnit>'
JenkinsFile:
pipeline {
agent any
parameters{
string(defaultValue: 'form5_report', description: 'Provide the name of JRXML file, without the extension that should be used to create report. Defaults to form5_report.', name: 'JRXML_FILE', trim: false)
string(defaultValue: "${params.JRXML_FILE}.${BUILD_NUMBER}", description: 'Provide a name for the generated report. By default, it would be same as JRXML with BUILD_NUMBER.', name: 'TESTUNIT', trim: false)
}
environment{
JRXML_FILE = "${JRXML_FILE}.jrxml"
def XML = """<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<reportUnit>
<description/>
<label>${TESTUNIT}</label>
<permissionMask>1</permissionMask>
<uri>/Reports</uri>
<version>0</version>
<dataSourceReference>
<uri>/Data_Sources/mysql</uri>
</dataSourceReference>
<alwaysPromptControls>true</alwaysPromptControls>
<controlsLayout>popupScreen</controlsLayout>
<inputControlRenderingView/>
<jrxmlFileReference>
<uri>/Reports/${JRXML_FILE}</uri>
</jrxmlFileReference>
</reportUnit>"""
}
stages {
stage('File Check') {
steps {
script {
String url = "https://server-url/jasperserver-pro/rest_v2/resources?q=${JRXML_FILE}"
def (String code) =
sh(script: "curl -s -o /dev/null -w '%{http_code}' -H 'Authorization: Basic base64 creds' $url", returnStdout: true).trim().tokenize("\n")
if(code == '200'){
stage ('Create Report'){
sh '''
curl --location --request POST 'https://server-url/jasperserver-pro/rest_v2/resources/Reports' \
--header 'Content-Type: application/repository.reportUnit+xml' \
--header 'Authorization: Basic base64 creds' \
-d "$XML"
'''
}
}else{
stage('SCM Checkout'){
git branch: 'branchname',
credentialsId: 'git',
url: 'repo_URL'
}
stage('Upload JRXML File to JasperServer'){
sh '''
curl --location --request POST 'https://server-url/jasperserver-pro/rest_v2/resources/Reports/' \
--header 'Content-Type: application/jrxml' \
--header "Content-Disposition: attachment; filename=${JRXML_FILE}" \
--header 'Content-Description: Uploaded jrxml file' \
--header 'Authorization: Basic base64 creds' \
--data-binary "$WORKSPACE/${JRXML_FILE}"
'''
}
stage('Create Report'){
sh '''
curl --location --request POST 'https://server-url/jasperserver-pro/rest_v2/resources/Reports' \
--header 'Content-Type: application/repository.reportUnit+xml' \
--header 'Authorization: Basic base64 creds' \
-d "$XML"
'''
}
}
}
}
}
}
}
What am I doing wrong? Any help is greatly appreciated.
Alright, so I got it working.
Apparently, I was unaware of the fact that when referencing/passing a file with --data-binary option, you need to prepend it with the '#' symbol.
Therefore changing line no.61 from --data-binary "$WORKSPACE/report-templates/${JRXML_FILE}" to --data-binary "#$WORKSPACE/report-templates/${JRXML_FILE}" did the trick. Otherwise, it was just saving the absolute path as the content of the file and not the actual XML data.

Quote in curl command in Jenkinsfile

I try to launch a curl with json data in a jenkinsfile.
But I have strange issue with quote.
CUB_CONFIG = "1614874915062/v0001a0x_test.conf"
url = "https://..."
sh "curl $url -d {'iro': '${CUB_CONFIG}', 'src': '/app/data/'} -H 'Content-Type: application/json'"
It is interpreted like (and so do not work...) :
curl https://... -d '{iro:' 1614874915062/v0001a0x_test.conf, src: '/app/data/}' -H 'Content-Type: application/json'
How I should write these damn quote ??
{ and } have special meaning in shell scripts, so when a literal { or } character is desired they must be quoted or escaped.
Here is your correct sh call (I use """ so I don't need to use \" escapes):
sh """curl '$url' -d '{"iro": "${CUB_CONFIG}", "src": "/app/data/"}' -H 'Content-Type: application/json'"""

Use credentials to make API calls from a Jenkinsfile

What is a secure way to use credentials to make REST API calls from a Jenkinsfile?
For example, the following would not be secure because the username and password are exposed in code:
sh "curl -D- -u username:password -X GET -H 'Content-Type: application/json' http://<ip-of-server-with-rest-api>/path/to/api/endpoint"
You should use Credentials Binding Plugin
Here is an example:
withCredentials([usernamePassword(credentialsId: 'amazon',
usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
// available as an env variable, but will be masked if you try to print it
out any which way
// note: single quotes prevent Groovy interpolation; expansion is by
Bourne Shell, which is what you want
sh 'echo $PASSWORD'
// also available as a Groovy variable
echo USERNAME
// or inside double quotes for string interpolation
echo "username is $USERNAME"
sh "curl -D- -u $USERNAME:$PASSWORD -X GET -H 'Content-Type: application/json' http://<ip-of-server-with-rest-api>/path/to/api/endpoint"
}
Your credentials will be hidden in Console Output.

Artifactory API AQL with Jenkins environment variables

I try to perform the following code in a postbuild Jenkins task:
curl -H "X-JFrog-Art-Api:***********" -X POST https://artifactory_url/artifactory/api/search/aql -H "Content-Type: text/plain" -d 'items.find({"repo":{"$eq":"REPO"},"name":{"$match":"*${env.SUBSYSTEM}*"},"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'
(Here it is broken up over several lines for readability):
curl -X POST https://artifactory_url/artifactory/api/search/aql \
-H "X-JFrog-Art-Api:***********" \
-H "Content-Type: text/plain" \
-d 'items.find( \
{"repo":{"$eq":"REPO"}, \
"name":{"$match":"*${env.SUBSYSTEM}*"}, \
"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'
This is not working because the environment variable ${env.SUBSYSTEM} is not solved. Is there anyway for use variables in curl with the aql?
Thanks and Regards
It's probably not resolving the environment variable because you're wrapping that piece of string in single quotes ('), which preserve the literal value of each character within the quotes (meaning variables aren't resolved). You could use double quotes (") with escaping, which would look like:
... -d "items.find({\"repo\":{\"$eq\":\"REPO\"},\"name\":{\"$match\":\"*${env.SUBSYSTEM}*\"},\"name\":{\"$nmatch\":\"*pdf*\"}}).include(\"repo\",\"name\",\"path\")"
Or possibly just break the environment variable out of the quoting:
... -d 'items.find({"repo":{"$eq":"REPO"},"name":{"$match":"*'${env.SUBSYSTEM}'*"},"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'

Resources