I wrote the following function which is loaded to my pipeline:
def userTrigger() {
[$class: 'UsernamePasswordMultiBinding', credentialsId: jenkins_creds, usernameVariable: 'J_USER', passwordVariable: 'J_PASS'],
]){
cmd = "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}api/json | python -mjson.tool | grep userId | awk '{print \$2}' | tr -d '"|,' "
def ut = sh(returnStdout: true, script: cmd)
return ut
}
}
The result should be the username of the one who triggered the build, before cleaning the output it looks like so:
"userId": "itaig",
Basically, this:
awk -F'"' '{print $4}'
would also give me the required output but i'm not sure how to escape chars in this statement too.
When I run the job, I get the following error:
java.io.NotSerializableException: org.codehaus.groovy.control.ErrorCollector
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
I'm pretty much sure it has something to do with my tries of escaping the relevant characters but I have a feeling that I haven't done it right, I've tried escaping the (") in the end of the cmd line but to no avail.
Can you try and find my issue?
What is that right parenthesis and right bracket closing on line #3?
]){
It might be more of the double-quote vs single-quote then the escaping of characters - make sure they are are pairing up nicely.
Related
I am trying to execute below script inside jenkinsfile
awk 'NR==FNR{new=new $0 ORS; next} /^}]$/{printf "}],\n%s", new} 1' output2 ${WORKSPACE}/${REPO}/file1 >> output2
This script works fine in linux command promt but it gives error when tried to run inside jenkins file.I tried below
sh """
awk 'NR==FNR{new=new $0 ORS; next} /^}]$/{printf "}],\n%s", new} 1' output2 ${WORKSPACE}/${REPO}/file1 >> output2
"""
But it gives syntax error
You need to watch out for escaping single or double quotes, backslashes and dollar signs. Here I've escaped everything:
sh """
awk \'NR==FNR{new=new \$0 ORS; next} /^}]\$/{printf \"}],\\n%s\", new} 1\' output2 \${WORKSPACE}/\${REPO}/file1 >> output2
"""
Maybe you actually don't want to escape ${WORKSPACE} or ${REPO} if those are pipeline environment variables that you want substituted by groovy as opposed to by sh. And maybe you don't need to escape the single quotes inside a triple """ multistring. But you definitely need to escape the double quotes and the backslash in \\n.
In my pipeline script I want to execute the following sh command including a password (e.g. 123$ABC) with special characters:
withCredentials([sshUserPrivateKey(credentialsId: 'id', keyFileVariable: 'keyFile', passphraseVariable: '', usernameVariable: 'user')]) {
sh "ssh -i $keyFile ${user}#${virtualMachine} -C \"CONTAINER_NAME=${dockerContainerName} DOCKER_TAG=${dockerImageTag} JASYPT_MASTER_PASSWORD='${JASYPT_MASTER_PW}' docker-compose -f /tmp/docker-compose-jenkins.yml up\""
}
How must the command look like to obtain the special characters in my password?
I have the same issue, the possible solution is to decode the password variable, something like:
def encodedPassword = URLEncoder.encode("$JASYPT_MASTER_PW",'UTF-8')
and then use the encodedPassword variable instead of JASYPT_MASTER_PW
Let's try
We have a requirement to execute below in Jenkinsfile and it is working fine directly on the Server:
Server: echo $myval | sed 's/,/\n/g' | grep analysisId | cut -d":" -f2
However, when I am trying to execute it in Jenkinsfile, it is treating newline as spaces, even though when I tried using '\' to supress '\':
Jenkinsfile: echo $myval | sed 's/,/\\n/g' | grep analysisId | cut -d":" -f2
Any idea what I may be doing wrong ? Intent over here is that I am trying to parse JSON, and it's the only option I am left with
Value of myval
{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}
Need to get Value of analysisId, which is AW1eTP3AX12345.
Section of Jenkinsfile:
def analysisId = sh script: "echo $sonarUrlContent | sed 's/,/\\n/g' | grep analysisId | cut -d':' -f2",returnStdout:true
echo "analysisId: ${analysisId}"
Since there is no way for you to install the Pipeline Utility Plugin, you can improvise some parsing of your own.
This is a hacky hacky solution! It only works since you only need one value:
def json = '{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}'
def data = json.split(",")
data.each{ item ->
if(item.startsWith('"analysisId"')){
result = item.split(":")[1].replace('"', '')
}
}
echo "$result"
Using readJSON would be preferable in any case, but this should work.
I don't know why the sed command fails, but if you want to parse JSON like #daggett implied you should use the jenkins native readJSON method instead of the groovy jsonSlurper
You have to install the Pipeline Utility Plugin and can use it like this:
def json = '{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}'
def data = readJSON text: json
echo "$data.task.analysisId"
readJSON returns a groovy map which can be used like one would expect.
I'd like to populate the the groovy variable "committer" with the output of the command:
def committer = utils.sh("curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json | python -mjson.tool | grep authorEmail | awk '{print \$2}' | tr -d '"|,' ")
Because of the known issue in Jenkins (JENKINS-26133) it is not possible to do that but only to populate the variable with the exit status of the command.
So I've go these 2 functions:
def gen_uuid(){
randomUUID() as String
}
def sh_out(cmd){ // As required by bug JENKINS-26133
String uuid = gen_uuid()
sh """( ${cmd} )> ${uuid}"""
String out = readFile(uuid).trim()
sh "set +x ; rm ${uuid}"
return out
}
These functions allow me to wrap my shell commands in sh_out(COMMAND) and in the background I'm using the workaround which is suggested in the mentioned above known issue link which means running the command while redirecting it's output to a file (in the case of my function it's a random filename) and then reading it into a variable.
So, In the beginning of my pipeline I load my functions file which ends with return this; like so:
fileLoader.withGit('git#bitbucket.org:company/pipeline_utils.git', 'master', git_creds, ''){
utils = fileLoader.load('functions.groovy');
}
And that's why the "utils.sh_out" that you see in the command, but when I use the shown above command in my Jenkins pipeline, I get the following error:
/home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ#tmp/durable-006d5c7e/script.sh: 2: /home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ#tmp/durable-006d5c7e/script.sh: Bad substitution
Running the command in a shell works properly:
$ curl -s -u user:password http://IPADDR:8080/job/COMPANY_BitBucket_Integration/job/research/job/COMPANY-6870-bitbucket-integration/3/api/json/api/json | python -mjson.tool | grep authorEmail | awk '{print $2}' | tr -d '"|,'
user#email.com
I suspect it has something to do with the tr command in the end and with the character escaping I did there but whatever I try fails, anyone got an idea?
according to the documentation now sh supports std output.
and i know i'm not answering your question directly, but i suggest to use groovy to parse json.
you are trying to get the value of authorEmail from json
if the response from /api/json looks like this (just an example):
{
"a":{
"b":{
"c":"ccc",
"authorEmail":"user#email.com"
}
}
}
then the groovy to take athorEmail:
def cmd = "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
def json = sh(returnStdout: true, script: cmd).trim()
//parse json and access it as an object (Map/Array)
json = new groovy.json.JsonSlurper().parseText(json)
def mail = json.a.b.athorEmail
you could receive java.io.NotSerializableException explained here
so i changed the code like this:
node {
def json = sh(
returnStdout: true,
script: "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
).trim()
def mail = evaluateJson(json, '${json.a.b.authorEmail}')
echo mail
}
#NonCPS
def evaluateJson(String json, String gpath){
//parse json
def ojson = new groovy.json.JsonSlurper().parseText(json)
//evaluate gpath as a gstring template where $json is a parsed json parameter
return new groovy.text.GStringTemplateEngine().createTemplate(gpath).make(json:ojson).toString()
}
I wanted to show the user who triggered a Jenkins job in the post job email. This is possible by using the plugin Build User Vars Plugin and the env variable BUILD_USER.
But this variable do not get initialized when the job is triggered by a scheduler.
How can we achieve this? I know we have a plugin called - EnvInject Plugin, and that can be used...
But I just want to know how we can use this and achieve the solution...
Build user vars plugin wasn't working for me so I did a quick-and-dirty hack:
BUILD_CAUSE_JSON=$(curl --silent ${BUILD_URL}/api/json | tr "{}" "\n" | grep "Started by")
BUILD_USER_ID=$(echo $BUILD_CAUSE_JSON | tr "," "\n" | grep "userId" | awk -F\" '{print $4}')
BUILD_USER_NAME=$(echo $BUILD_CAUSE_JSON | tr "," "\n" | grep "userName" | awk -F\" '{print $4}')
SIMPLE SOLUTIONS (NO PLUGINS) !!
METHOD 1: Via Shell
BUILD_TRIGGER_BY=$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | egrep '^userId>|^userName>' | sed 's/.*>//g' | sed -e '1s/$/ \//g' | tr '\n' ' ')
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
METHOD 2: Via Groovy
node('master') {
BUILD_TRIGGER_BY = sh ( script: "BUILD_BY=\$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | egrep '^userId>|^userName>' | sed 's/.*>//g' | sed -e '1s/\$/ \\/ /g'); if [[ -z \${BUILD_BY} ]]; then BUILD_BY=\$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | grep '^shortDescription>' | sed 's/.*user //g;s/.*by //g'); fi; echo \${BUILD_BY}", returnStdout: true ).trim()
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
}
METHOD 3: Via Groovy
BUILD_TRIGGER_BY = "${currentBuild.getBuildCauses()[0].shortDescription} / ${currentBuild.getBuildCauses()[0].userId}"
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
OUTPUT:
Started by user Admin / user#example.com
Note: Output will be both User ID and User Name
This can be done using the Jenkins Build User Vars Plugin which exposes a set of environment variables, including the user who started the build.
It gives environment variables like BUILD_USER_ID, EMAIL, etc.
When the build is triggered manually by a logged-in user, that user's userid is available in the BUILD_USER_ID environment variable.
However, this environment variable won't be replaced / initialized when the build is automatically triggered by a Jenkins timer / scheduler.
Attached a screenshot for details
This can be resolved by injecting a condition to the Job by using Conditional Build Step Plugin / Run Condition Plugin,where in to each job we can add a condition to initialize the variable BUILD_USER_ID only when the build is caused or triggered by the Timer or scheduler, by setting a condition using the regular expression..
Without Plugin ->
def cause = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
echo "userName: ${cause.userName}"
Install 'Build User Vars Plugin' and use like below:- [ See https://plugins.jenkins.io/build-user-vars-plugin ]
Be sure to check mark the Set jenkins user build variables checkbox under Build Environment for your Jenkins job's configuration.
I found similar but really working on Jenkins 2.1.x and easy for my understanding way.
And it works without any plugins.
if (currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')['userId']){
// Will be run only if someone user triggers build
// Because in other cases this contructions returns null
}
You can use in this construction any classes described here.
They will be returns maps with usable values.
This gets the username who clicked "Build Now" in a Jenkins pipeline job.
#NonCPS
def getBuildUser() {
return currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
}
I'm using a combination of the 'Execute Shell' and 'Env Inject' plugin as follows:
Create an 'Execute Shell' build step that uses shell parameter substitution to write default the value and echo that value into a file. Example highlighted in screen shot below.
Use the 'Env Inject' file to read that file as properties to set.
The token $BUILD_CAUSE from the email-ext plugin is what you are looking for.
You can see the full content token reference when you click the ? just after the Attach build log combobox at the email content configuration.
Some tokens get added by plugins, but this one should be aviable by default.
Edit: As pointed out by bishop in the comments, when using the EnvInject plugin, the $BUILD_CAUSE token gets changed to behave differently.
I have written a groovy script to extract the started by which would correctly get the source, regardless if user, scm or timer (could add more). It would recursively navigate the build tree to get the "original" 'started by' cause https://github.com/Me-ion/jenkins_build_trigger_cause_extractor
I wanted to trigger build initiator info to one of my slack/flock group so I used following way to get build initiator email and name by writing in Declarative fashion .
I am just printing here, you can use to store in some environment variable or write in one file giving file path according to your own convenience..
pipeline {
environment {
BRANCH_NAME = "${env.BRANCH_NAME}"
}
agent any
stages{
stage('Build-Initiator-Info'){
sh 'echo $(git show -s --pretty=%ae)'
sh 'echo $(git show -s --pretty=%an)'
}
}
}
Just to elaborate on Musaffir Lp's answer. The Conditional Build Step plugin now supports the Build Cause directly - it requires the Run Condition Plugin also.
If you wanted to detect when the build was started by a timer you can select a Run? value of Build Cause, with Build Cause of: TimerTrigger
This is a little simpler and more robust than using a regex. There are also other triggers you can detect, for example when the build was a result of Source Control Management commit, you can select: SCMTrigger.
This below is working for me.
Install "user build vars plugin"
Build Name = ${BUILD_NUMBER}_${TICKET}_${ENV,var="BUILD_USER_ID"}
I created a function that return the Triggered Job Name:
String getTriggeredJob(CURRENT_BUILD) {
if (CURRENT_BUILD.upstreamBuilds.size() > 0) {
TRIGGERED_JOB = CURRENT_BUILD.upstreamBuilds[0].projectName
if (!TRIGGERED_JOB.isEmpty()) {
return TRIGGERED_JOB
}
}
return "Self"
}
CURRENT_BUILD is env var currentBuild
How to return Username & UserId:
UserName: currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserName()
UserId: currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
There is other way to get user_id, where you don't need to install anything.
BUILD_USER_ID = sh (
script: 'id -u',
returnStdout: true
).trim()
echo "bUILD USER: ${BUILD_USER_ID }"
For declarative pipeline syntax, here is a quick hack, base on #Kevin answer.
For declarative pipeline you need to enclose them in a node, else you will get an error/ build failure
node {
def BUILD_FULL = sh (
script: 'curl --silent '+buildURL+' | tr "{}" "\\n" | grep -Po \'"shortDescription":.*?[^\\\\]"\' | cut -d ":" -f2',
returnStdout: true
)
slackSend channel: '#ci-cd',
color: '#000000',
message: "The pipeline was ${BUILD_FULL} ${GIT_COMMIT_MSG} "
}
The output will be slack notification sent to your slack channel with the git short description