I try to launch a curl with json data in a jenkinsfile.
But I have strange issue with quote.
CUB_CONFIG = "1614874915062/v0001a0x_test.conf"
url = "https://..."
sh "curl $url -d {'iro': '${CUB_CONFIG}', 'src': '/app/data/'} -H 'Content-Type: application/json'"
It is interpreted like (and so do not work...) :
curl https://... -d '{iro:' 1614874915062/v0001a0x_test.conf, src: '/app/data/}' -H 'Content-Type: application/json'
How I should write these damn quote ??
{ and } have special meaning in shell scripts, so when a literal { or } character is desired they must be quoted or escaped.
Here is your correct sh call (I use """ so I don't need to use \" escapes):
sh """curl '$url' -d '{"iro": "${CUB_CONFIG}", "src": "/app/data/"}' -H 'Content-Type: application/json'"""
Related
I have a Jenkins pipeline where this command works and send me a notification through google chat :
script {
sh 'curl -k "https://chat.googleapis.com/v1/spaces/AAAABHT3HT0/messages?key=*****&token=******" -d "#chat_notification.json" -XPOST -H "Content-Type: application/json; charset=UTF-8"'
}
But if I enter the url in a variable, that does not work any more :
script {
url = "https://chat.googleapis.com/v1/spaces/AAAAfF9CGEQ/messages?key=******&token=******"
sh 'curl -k ${url} -d "#chat_notification.json" -XPOST -H "Content-Type: application/json; charset=UTF-8"'
}
With the error :
curl -k -d #chat_notification.json -XPOST -H 'Content-Type: application/json; charset=UTF-8'
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
It's probably a quote issue ?
Yes, if you are using single quotes in Groovy, that means you cannot interpolate the string with variables using the $ syntax.
See https://groovy-lang.org/syntax.html#_string_interpolation .
Same thing applies to shell code, by the way, or sh in this case.
But since you are not using any shell variables in your code, simply swapping the quotes, i.e. quoting your Groovy string with ", and using ' within the curl command would probably work.
Yes it is a quote issue try to use "" instead of '' to use the value of url variable in curl command. You should also seperate -X and POST (you have -XPOST in your script)
script {
url = "https://chat.googleapis.com/v1/spaces/AAAAfF9CGEQ/messages?key=******&token=******"
sh "curl -k ${url} -d #chat_notification.json -X POST -H Content-Type: application/json; charset=UTF-8"
}
We have a pipeline where we need to invoke external API with Authorization header, whose value comes from Jenkins secret, which has been pre-configured.
When implemented as below, Jenkins complains for string interpolation.
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh """curl --location --request POST 'https://abc.example.com/api/endpoint' \
--header 'Authorization: Bearer ${Token}' \
--header 'Content-Type: application/json' \
--data-raw ${payload}""
We have tried will single quotes for sh and double quotes but nothing works out.
How it can be handled here?
Jenkins doesn't want you to interpolate passwords in the code but rather pass them as environment variables to the shell and let the shell command extract them, that is possible only for parameters that are loaded into the shell execution environment.
In declarative pipelines loading parameters and secrets into shell environment can be done using the environment directive and for scripted pipelines loading secrets can be done via the withCredentials keyword and loading regular parameters can be done via the 'withEnv` keyword.
In your case you have the Token parameter which is loaded into environment by the withCredentials step and the payload parameter which is probably not, so you are mixing two type of parameter contexts, more information on this is available in the Answer for this question.
To solve it you have two options.
The first option is to load the payload into the environment of the shell and use a single quoted groovy string:
withEnv(["PAYLOAD=${payload}"]) {
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw $PAYLOAD'''
}
}
Second option is to separate the construction of the string into two types, and handle each section with the relevant method:
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw ''' + payload
}
I have Jenkinsfile that creates mt object and passes vaultToken to psl library:
Jenkinsfile:
#Library('shared-library#psl')
Maintenance mt = new Maintenance()
mt.setVaultToken(config.vaultToken)
mt.setApiUrl(config.apiUrl)
pslService.createMaintenance(mt)
pslService:
String createMaintenance(Maintenance mt){
dockerBuildHelper.getDockerImage(dockerBuildHelper.getWdBuildDockerImageName()).inside('-u root'){
String cmd = "curl -X POST '${mt.getApiUrl()}'" +
" -H 'Content-Type: application/json'" +
" -H 'Authorization: Api-Token ${mt.getVaultToken()}'" +
" -d ${mt.getPayload()} | jq -r '.id'"
return sh(script: cmd, returnStdout: true).trim()
}
}
But this prints curl command and exposes vault token in the pipeline.
Does anyone know how I can hide the sensitive info and/or entire curl command?
I do not want to store this in credentials store, unless I have no choice.
I heard I can use set +x. But I am not sure how to use it and if it helps. Any thoughts?
Try to use mask password plugin. Or create secrets in jenkins and call them in environment block of pipeline.
I went with set +x
String cmd = """set +x
curl -X POST '${mt.getApiUrl()}' -H 'Content-Type: application/json' -H 'Authorization: Api-Token ${mt.getVaultToken()}' -d ${mt.getPayload()} | jq -r '.id'
"""
return sh(script: cmd, returnStdout: true).trim()
Hey folks I want to execute a sh command in a jenkinsfile which is pretty long, into multiple lines. thing is I´m unable to do so. command gets executed but only first line. I tried with "\" with "+" but i´m unable to execute the hole command. this is how I have it right now:
node {
stage('purge URL content in cloudflare') {
sh """\
curl -X GET "https://api.cloudflare.com/client/v4/zones/cd7d030xxxxxxx420df9514dad0" +
-H "X-Auth-Email: mail.user#domain.com" +
-H "X-Auth-Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" +
-H "Content-Type: application/json" +
--data '{"files":["${params.URL1}",{"url":"${params.URL2}","headers":{"Origin":"cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}' """
}
}
But I´m getting error
How is it in groovy that you can add one command in multiple lines?
There's no need to use + to concatenate strings in a triple-double-quoted string:
node {
stage('purge URL content in cloudflare') {
sh """
curl -X GET "https://api.cloudflare.com/client/v4/zones/cd7d030xxxxxxx420df9514dad0"
-H "X-Auth-Email: mail.user#domain.com"
-H "X-Auth-Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
-H "Content-Type: application/json"
--data '{"files":["${params.URL1}",{"url":"${params.URL2}","headers":{"Origin":"cloudflare.com","CF-IPCountry":"US","CF-Device-Type":"desktop"}}]}'
"""
}
}
Source: http://groovy-lang.org/syntax.html#_triple_double_quoted_string
I try to perform the following code in a postbuild Jenkins task:
curl -H "X-JFrog-Art-Api:***********" -X POST https://artifactory_url/artifactory/api/search/aql -H "Content-Type: text/plain" -d 'items.find({"repo":{"$eq":"REPO"},"name":{"$match":"*${env.SUBSYSTEM}*"},"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'
(Here it is broken up over several lines for readability):
curl -X POST https://artifactory_url/artifactory/api/search/aql \
-H "X-JFrog-Art-Api:***********" \
-H "Content-Type: text/plain" \
-d 'items.find( \
{"repo":{"$eq":"REPO"}, \
"name":{"$match":"*${env.SUBSYSTEM}*"}, \
"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'
This is not working because the environment variable ${env.SUBSYSTEM} is not solved. Is there anyway for use variables in curl with the aql?
Thanks and Regards
It's probably not resolving the environment variable because you're wrapping that piece of string in single quotes ('), which preserve the literal value of each character within the quotes (meaning variables aren't resolved). You could use double quotes (") with escaping, which would look like:
... -d "items.find({\"repo\":{\"$eq\":\"REPO\"},\"name\":{\"$match\":\"*${env.SUBSYSTEM}*\"},\"name\":{\"$nmatch\":\"*pdf*\"}}).include(\"repo\",\"name\",\"path\")"
Or possibly just break the environment variable out of the quoting:
... -d 'items.find({"repo":{"$eq":"REPO"},"name":{"$match":"*'${env.SUBSYSTEM}'*"},"name":{"$nmatch":"*pdf*"}}).include("repo","name","path")'