What is a secure way to use credentials to make REST API calls from a Jenkinsfile?
For example, the following would not be secure because the username and password are exposed in code:
sh "curl -D- -u username:password -X GET -H 'Content-Type: application/json' http://<ip-of-server-with-rest-api>/path/to/api/endpoint"
You should use Credentials Binding Plugin
Here is an example:
withCredentials([usernamePassword(credentialsId: 'amazon',
usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
// available as an env variable, but will be masked if you try to print it
out any which way
// note: single quotes prevent Groovy interpolation; expansion is by
Bourne Shell, which is what you want
sh 'echo $PASSWORD'
// also available as a Groovy variable
echo USERNAME
// or inside double quotes for string interpolation
echo "username is $USERNAME"
sh "curl -D- -u $USERNAME:$PASSWORD -X GET -H 'Content-Type: application/json' http://<ip-of-server-with-rest-api>/path/to/api/endpoint"
}
Your credentials will be hidden in Console Output.
Related
I have a Jenkins pipeline where this command works and send me a notification through google chat :
script {
sh 'curl -k "https://chat.googleapis.com/v1/spaces/AAAABHT3HT0/messages?key=*****&token=******" -d "#chat_notification.json" -XPOST -H "Content-Type: application/json; charset=UTF-8"'
}
But if I enter the url in a variable, that does not work any more :
script {
url = "https://chat.googleapis.com/v1/spaces/AAAAfF9CGEQ/messages?key=******&token=******"
sh 'curl -k ${url} -d "#chat_notification.json" -XPOST -H "Content-Type: application/json; charset=UTF-8"'
}
With the error :
curl -k -d #chat_notification.json -XPOST -H 'Content-Type: application/json; charset=UTF-8'
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
It's probably a quote issue ?
Yes, if you are using single quotes in Groovy, that means you cannot interpolate the string with variables using the $ syntax.
See https://groovy-lang.org/syntax.html#_string_interpolation .
Same thing applies to shell code, by the way, or sh in this case.
But since you are not using any shell variables in your code, simply swapping the quotes, i.e. quoting your Groovy string with ", and using ' within the curl command would probably work.
Yes it is a quote issue try to use "" instead of '' to use the value of url variable in curl command. You should also seperate -X and POST (you have -XPOST in your script)
script {
url = "https://chat.googleapis.com/v1/spaces/AAAAfF9CGEQ/messages?key=******&token=******"
sh "curl -k ${url} -d #chat_notification.json -X POST -H Content-Type: application/json; charset=UTF-8"
}
We have a pipeline where we need to invoke external API with Authorization header, whose value comes from Jenkins secret, which has been pre-configured.
When implemented as below, Jenkins complains for string interpolation.
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh """curl --location --request POST 'https://abc.example.com/api/endpoint' \
--header 'Authorization: Bearer ${Token}' \
--header 'Content-Type: application/json' \
--data-raw ${payload}""
We have tried will single quotes for sh and double quotes but nothing works out.
How it can be handled here?
Jenkins doesn't want you to interpolate passwords in the code but rather pass them as environment variables to the shell and let the shell command extract them, that is possible only for parameters that are loaded into the shell execution environment.
In declarative pipelines loading parameters and secrets into shell environment can be done using the environment directive and for scripted pipelines loading secrets can be done via the withCredentials keyword and loading regular parameters can be done via the 'withEnv` keyword.
In your case you have the Token parameter which is loaded into environment by the withCredentials step and the payload parameter which is probably not, so you are mixing two type of parameter contexts, more information on this is available in the Answer for this question.
To solve it you have two options.
The first option is to load the payload into the environment of the shell and use a single quoted groovy string:
withEnv(["PAYLOAD=${payload}"]) {
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw $PAYLOAD'''
}
}
Second option is to separate the construction of the string into two types, and handle each section with the relevant method:
withCredentials([string(credentialsId: '<SECRETNAME>', variable: 'Token')]) {
sh '''curl --location --request POST "https://abc.example.com/api/endpoint" \
--header "Authorization: Bearer $Token" \
--header "Content-Type: application/json" \
--data-raw ''' + payload
}
I added credentials in Credentials Binding Plugin in Jenkins.
Then I send this credentials as text to company's chat in Execute shell section using curl with data parameter:
curl --silent $url -d "{ \"text\": \"$credentials\" }" -H "X-Auth-Token: $auth_token" -H "X-User-Id: $user_id" -H "Content-type:application/json"
And I receive unmasked credentials in chat. How can I mask credentials or modify Binding plugin which will mask secret text same as echo output
I have Jenkinsfile that creates mt object and passes vaultToken to psl library:
Jenkinsfile:
#Library('shared-library#psl')
Maintenance mt = new Maintenance()
mt.setVaultToken(config.vaultToken)
mt.setApiUrl(config.apiUrl)
pslService.createMaintenance(mt)
pslService:
String createMaintenance(Maintenance mt){
dockerBuildHelper.getDockerImage(dockerBuildHelper.getWdBuildDockerImageName()).inside('-u root'){
String cmd = "curl -X POST '${mt.getApiUrl()}'" +
" -H 'Content-Type: application/json'" +
" -H 'Authorization: Api-Token ${mt.getVaultToken()}'" +
" -d ${mt.getPayload()} | jq -r '.id'"
return sh(script: cmd, returnStdout: true).trim()
}
}
But this prints curl command and exposes vault token in the pipeline.
Does anyone know how I can hide the sensitive info and/or entire curl command?
I do not want to store this in credentials store, unless I have no choice.
I heard I can use set +x. But I am not sure how to use it and if it helps. Any thoughts?
Try to use mask password plugin. Or create secrets in jenkins and call them in environment block of pipeline.
I went with set +x
String cmd = """set +x
curl -X POST '${mt.getApiUrl()}' -H 'Content-Type: application/json' -H 'Authorization: Api-Token ${mt.getVaultToken()}' -d ${mt.getPayload()} | jq -r '.id'
"""
return sh(script: cmd, returnStdout: true).trim()
I have the following Jenkinsfile
def ARTIFACTS_JOB_ID;
def ARTIFACTS_URL;
node('CentOs') {
checkout scm
stage('Example') {
echo 'Download artifacts'
sh 'curljson --header "PRIVATE-TOKEN: tdzis3" "https://gitlab-sample.com/api/v4/projects/63/jobs" > jobs.json'
ARTIFACTS_JOB_ID = sh(returnStdout: true, script: 'python GetID.py').trim()
ARTIFACTS_URL = "https://gitlab-sample.com/api/v4/projects/63/jobs/${ARTIFACTS_JOB_ID}/artifacts"
echo "======> $ARTIFACTS_URL"
sh 'echo $ARTIFACTS_URL'
sh 'curl --output artifacts.zip --header "PRIVATE-TOKEN: tdzis3" "$ARTIFACTS_URL"'
}
}
while trying to get the artifact, it's calling
'curl --output artifacts.zip --header "PRIVATE-TOKEN: tdzis3" ""' with empty url
Similarly
echo "======> $ARTIFACTS_URL" //works fine
sh 'echo $ARTIFACTS_URL' // shows empty string, tried with ${ARTIFACTS_URL} aswell.
is there a way I can run it as below (so I don't have to use sh)
def ARTIFACTS_JOB_ID;
def ARTIFACTS_URL;
node('CentOs') {
checkout scm
stage('Example') {
script {
echo 'Download artifacts'
curljson --header "PRIVATE-TOKEN: tdzis3" "https://gitlab-sample.com/api/v4/projects/63/jobs" > jobs.json
}
}
}
without sh I am getting invalid syntax error while doing
curljson --header
The variable declaration ARTIFACTS_URL = "https://..." is a Groovy global variable. Hence, it is not naturally available to the sh step as a shell variable.
You need to wrap the commands inside the sh step with double quotes instead of single quotes as sh "echo $ARTIFACTS_URL" for Groovy to interpolate it as a Groovy variable.
Have you tried
sh "curl --output artifacts.zip --header 'PRIVATE-TOKEN: tdzis3' '{$ARTIFACTS_URL}'"
?
for multiple sh commands you could also use
sh """
curl command here
cd command next
etc
"""