I have written a function to record http code after the application is up to verify if response code is appropriate. This is on windows agent hosted on Azure VM.
def result = bat (returnStdout: true,
script: "curl --output /dev/null --silent --write-out '\\n%{http_code} http://localhost:8080")
echo "HTTP Code: ${result}"
So as I run the pipeline I get the following response in console log
**F:\jenkin_workspace\workspace\curl-test-pipeline>curl --write-out '\n//localhost:8080**
I tried various methods like using double quotes, single quotes and even without quotes but none helps.
Please suggest where am I wrong here?
I'm coding this function in Jenkins to query Artifactory:
def curlDockerArtifact(URL, registryName, moduleName, tag, token) {
def controlURI = "${URL}/artifactory/api/storage/${registryName}/${moduleName}/${tag}"
def result = sh(script: """
curl -I -H \'Authorization: Bearer $token\' \
https://$controlURI -o /dev/null -w \'%{http_code}\' -s
""", returnStdout: true)
}
But I get this warning which I'm tying to avoid.
Warning: A secret was passed to "sh" using Groovy string interpolation, which is insecure.
I tried using single quotes but the variables don't get correctly interpreted from Groovy. Any idea how to fix/refactor the code?
You have to keep double quotes like you do, but you need to escape the $ sign for the token. Like this :
curl -I -H \'Authorization: Bearer \$token\'
The groovy will not interpolate the variable and the correct value will be passed on the shell level.
More informations : https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#interpolation-of-sensitive-environment-variables
I'm running a curl command in my Jenkinsfile.
post {
success {
script {
sh '''
|SCORE=+1
|GERRIT_COMMENT="$(cat <<-EOL
|Sonar result was: SUCCESS
|Report: ${Jenkins_URL}/job/${JOB_NAME}/${BUILD_NUMBER}/artifact/report1.txt
|EOL
|)"
|curl -s -u ${apiToken}: ${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children | json_pp -json_opt pretty,canonical > report1.txt
|echo "Voting unsuccessful"
'''.stripMargin().stripIndent()
archiveArtifacts artifacts: 'report1.txt', fingerprint: true
echo 'I Succeeded'
}
}
But I get the error
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/bin/json_pp
I can't use jq as it's not installed and installing it isn't an option.
The curl command works fine on my terminal but is failing in my Jenkins pipeline.
Also, when I do this instead, it works.
post {
success {
script {
sh '''
|SCORE=+1
|GERRIT_COMMENT="$(cat <<-EOL
|Sonar result was: SUCCESS
|Report: ${Jenkins_URL}/job/${JOB_NAME}/${BUILD_NUMBER}/artifact/report1.txt
|EOL
|)"
|echo "Voting unsuccessful"
'''.stripMargin().stripIndent()
sh """
curl -s -u ${apiToken}: '${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children' | json_pp -json_opt pretty,canonical > report1.txt
"""
archiveArtifacts artifacts: 'report1.txt', fingerprint: true
echo 'I Succeeded'
}
}
But it throws a warning in the console output.
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure. Affected argument(s) used the following variable(s): [apiToken]
What am I doing wrong, please?
In a Jenkins pipeline, how would you properly pass a JSON response using curl into a file?
I recommend to not use shell scripts whenever it is possible. Shell scripts are not cross platform and require installing additional tools (e.g. curl).
In your case the curl call could be replaced by the httpRequest step.
First let's replace the curl call and saves the result in a componentTree.json file:
httpRequest(
url: "${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children",
authorization: 'id-of-credentials-which-was-used-to-create-the-apiToken-variable',
outputFile: 'componentTree.json'
)
You want to format the JSON data in a human-readable format, so let's use the readJSON and writeJSON steps:
def json = readJSON(file: 'componentTree.json')
writeJSON(json: json, file: 'report1.txt', pretty: 4)
Now the report1.txt file contains JSON formatted with indent 4.
The componentTree.json file is written and read only once, so let's decrease the number of the IO operations:
def response = httpRequest(
url: "${Sonar_URL}/api/measures/component_tree?ps=100&s=qualifier,name&component=sonarqube&metricKeys=ncloc,bugs,vulnerabilities,code_smells,security_hotspots,coverage,duplicated_lines_density&strategy=children",
authorization: 'id-of-credentials-which-was-used-to-create-the-apiToken-variable'
)
def json = readJSON(text: response.content)
writeJSON(json: json, file: 'report1.txt', pretty: 4)
About the warning:
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure. Affected argument(s) used the following variable(s): [apiToken]
Secrets never should be interpolated because they may contain special characters which could be interpreted. Example:
my secret: My' https://example.org; cat /etc/passwd; echo \
command: curl -u '${password}' https://private.server/path/file.txt
After the interpolation the following command is called:
curl -u 'My' https://example.org; cat /etc/passwd; echo \' https://private.server/path/file.txt
There are two options to fix it:
if apiToken is an environment variable:
sh "curl -s -u \$apiToken: '${Sonar_URL}/api/measures/component..."
if apiToken is a Groovy variable:
withEnv(["CREDENTIALS=${apiToken}"]) {
sh "curl -s -u \$CREDENTIALS: '${Sonar_URL}/api/measures/component..."
}
In both cases the dollar sign ($) is escaped before the credentials which means that shell script will resolve it (it will be taken from environment variables).
I'm looking to get a get response on different jenkins pipelines to recreate them using job dsl plugin, but I'm facing issues with the credentials, so far i have been using the logic below but if trying to use the jenkins credentials in credentialsBinding, it fails to recognize them, if I use my own user and password it works fine
this is the logic im looking to implement
job('seed'){
wrappers {
credentialsBinding {
usernamePassword('USER','PASSWORD', 'credentials')
}
}
label('centos')
def confXml = "curl -s -XGET ${url} -u \$USER:\$PASSWORD".execute().text.replace("\n", "")
//do something with the respose
//recreate dsl after checking an attribute in the response
pipelineJob("Sandbox_pipelines/pipelineName") {
definition {
cpsScm {
scm {
git(repo_git, "master")
}
scriptPath("somepath")
}
}
}
}
when i run this job this should be creating the other pipelines, please let me know if you can help me on this.
Thanks in advance
The issue is that credentialsBinding loads the credentials during the build of the job being created. You want to use the credential to decide what to create and that's just not how it works.
You can use withCredentials though:
def confXml
withCredentials([usernameColonPassword(credentialsId: 'credentials', variable: 'USERPASS')]) {
confXml = "curl -s -XGET ${url} -u \$USERPASS".execute().text.replace("\n", "")
}
Need some help on fetching the GitHub payload into the Jenkins file without installing any plugin.
If anyone can provide the Jenkins file code snippet to access the GitHub payload from the webhook. it would be of great help.
I am able to call the Jenkins job from GitHub webhook. but need the payload as well to process further.
Any help would be appreciated. Thanks.
Please find the below groovy script:
stage('Pull Request Info') {
agent {
docker {
image 'maven'
args '-v $HOME/.m2:/home/jenkins/.m2 -ti -u 496 -e MAVEN_CONFIG=/home/jenkins/.m2 -e MAVEN_OPTS=-Xmx2048m'
}
}
steps {
script {
withCredentials([usernameColonPassword(credentialsId: "${env.STASH_CREDENTIAL_ID}",
variable: 'USERPASS')]) {
def hasChanges = maven.updateDependencies()
if (hasChanges != '') {
def pr1 = sh(
script: "curl -s -u ${"$USERPASS" } -H 'Content-Type: application/json' https://xx.example/rest/some/endpoint",
returnStdout: true
)
def pr = readJSON(text: pr1)
println pr.fromRef
}
}
}
}
}
Above script uses, curl to fetch the details about Pull request. I have stored the credentials in the jenkins and created an environment variable for the credentialId.
You can replace the url with your endpoint. You can also modify the script to use jq if you have jq installed in the machine.
In this case, I'm using readJSON to parse the JSON which is part of pipeline utilities plugin. My question would be, why not use the plugin as it provides the needed functionalities?
If you still dont want to use plugin, take a look at json parsing with groovy.