I have a config file config.txt with following key:values
a=1,2,3
b=5,6,7
I want to read the keys a nd b using groovy script but its giving following error message:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods withInputStream java.io.File groovy.lang.Closure
The code is as under:
Properties properties = new Properties()
File propertiesFile = new File('config.txt')
propertiesFile.withInputStream {
properties.load(it)
}
def runtimeString = 'a'
assert properties."$runtimeString" == '1'
assert properties.b == '2'
What am I missing?
Pipeline DSL context runs on master node even that your write node('someAgentName') in your pipeline. new File will work only on master.
But you can read data from file via sh(). Something like:
def a = sh(returnStdout: true, script: "cat config.txt | grep a | cut -f2 -d'='").trim()
def b = sh(returnStdout: true, script: "cat config.txt | grep b | cut -f2 -d'='").trim()
I tested the following in the Groovy console and the assertions pass
new File('config.txt').withReader {
def props = new Properties()
props.load(it)
assert props.getProperty('a') == '1,2,3'
assert props.getProperty('b') == '5,6,7'
}
Related
I'm absolutely new to Groovy and Jenskins, please ignore if question sounds noob. Following is a code snippet from a jenkins file containing groovy code.
def boolean hasChanged(String searchText) {
return sh(
returnStatus: true,
script: "git diff --name-only ${GIT_PREVIOUS_COMMIT} ${GIT_COMMIT} | grep \"${searchText}\""
) == 0
}
Questions:
Is the above snippet is function/method written in groovy?
what does return sh do?
Per my understanding script: "git diff --name-only ${GIT_PREVIOUS_COMMIT} ${GIT_COMMIT} | grep \"${searchText}\"" the output of grep \"${searchText}\"" is fed into it diff --name-only ${GIT_PREVIOUS_COMMIT} ${GIT_COMMIT}, is the understanding correct?
Please assist.
It is looks like a Groovy with Jenkins plugins
(sh)
Here I Added comments to explain this code.
// hasChanged method return boolean value
def boolean hasChanged(String searchText) {
// Insted of
// def shResult = sh(...); return shResult
// the sh results is returned
return sh(
// Preform the sh script and return the script exist code
returnStatus: true,
script: "git diff --name-only ${GIT_PREVIOUS_COMMIT} ${GIT_COMMIT} | grep \"${searchText}\""
) == 0 // check script exist code status
}
The output of git diff is piped to grep command that searches for given text in the git diff output
Yes
In this case, the entire Groovy function returns True if grep finds ${searchText} in the output of command git diff --name-only ${GIT_PREVIOUS_COMMIT} ${GIT_COMMIT}, or else False.
Hi Guys I'm trying to list only files names in a specific bucket in my s3 , and I got suceeded with the following command in aws cli,
aws s3 ls s3://inc-eb-deployments/my-bucket/ | awk '{ print $4 }'
But I want to add this AWS cli command in to my groovy script in Jenkins "Active Choice Parameter" Plugin, I've added it like below,
def sout = new StringBuilder(), serr = new StringBuilder()
def proc = 'aws s3 ls s3://inc-eb-deployments/my-bucket/ | awk '{ print $4 }' '.execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(2000)
def values = "$sout".split('.zip')
def trimmedValues
def parameters=[]
values.each { println "${it}" }
def cleanValues = "$sout".split('inc')
def last = cleanValues.last().split('.zip')[0]
cleanValues.each { "${it}".toString();
trimmedValues = "${it}".trim();
parameters<<trimmedValues
}
parameters.remove(parameters.size() - 1);
parameters.add(last)
parameters
But When I run the Job I don't see any filename outputs on the build page.
Anyone can help me where's the issue in this Groovy script ?
This can not work, as String.execute() does "stupid things" (it splits on whitespace) and execute at all does not run a shell at all - it just execs. If you want to use shellism in the groovy world use:
def proc = ["/bin/sh", "-c", "aws s3 ls s3://inc-eb-deployments/my-bucket/ | awk '{ print \$4 }' "].execute()
def proc = ["/bin/sh", "-c", "aws s3 ls s3://inc-eb-deployments/my-bucket/ | awk '{ print \$4 }' "].execute()
It works for me
I use the following code:
def s3_path = "s3://bucket/path/to/files/"
def aws_s3_ls_output = "aws s3 ls ${s3_path} --region us-east-1".execute() \
| ['awk', '{ print $NF }'].execute()
//aws_s3_ls_output.waitFor() // might be needed if execution is too long
def files = aws_s3_ls_output.text.tokenize().reverse()
return files
We have a requirement to execute below in Jenkinsfile and it is working fine directly on the Server:
Server: echo $myval | sed 's/,/\n/g' | grep analysisId | cut -d":" -f2
However, when I am trying to execute it in Jenkinsfile, it is treating newline as spaces, even though when I tried using '\' to supress '\':
Jenkinsfile: echo $myval | sed 's/,/\\n/g' | grep analysisId | cut -d":" -f2
Any idea what I may be doing wrong ? Intent over here is that I am trying to parse JSON, and it's the only option I am left with
Value of myval
{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}
Need to get Value of analysisId, which is AW1eTP3AX12345.
Section of Jenkinsfile:
def analysisId = sh script: "echo $sonarUrlContent | sed 's/,/\\n/g' | grep analysisId | cut -d':' -f2",returnStdout:true
echo "analysisId: ${analysisId}"
Since there is no way for you to install the Pipeline Utility Plugin, you can improvise some parsing of your own.
This is a hacky hacky solution! It only works since you only need one value:
def json = '{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}'
def data = json.split(",")
data.each{ item ->
if(item.startsWith('"analysisId"')){
result = item.split(":")[1].replace('"', '')
}
}
echo "$result"
Using readJSON would be preferable in any case, but this should work.
I don't know why the sed command fails, but if you want to parse JSON like #daggett implied you should use the jenkins native readJSON method instead of the groovy jsonSlurper
You have to install the Pipeline Utility Plugin and can use it like this:
def json = '{"task":{"id":"AW1eTPbXXXXXXXX","type":"REPORT","componentId":"AWz2VsZM-CVpcXXXXXX","componentKey":"Sonar-Scanner-SFDX_CI-ProjectKey","componentName":"BV GitLAB Test Project","componentQualifier":"TRK","analysisId":"AW1eTP3AX12345","status":"SUCCESS","submittedAt":"2019-09-23T13:26:05+0000","submitterLogin":"vgulati","startedAt":"2019-09-23T13:26:07+0000","executedAt":"2019-09-23T13:26:08+0000","executionTimeMs":1216,"logs":false,"hasScannerContext":true,"organization":"default-organization"}}'
def data = readJSON text: json
echo "$data.task.analysisId"
readJSON returns a groovy map which can be used like one would expect.
I'd like to populate the the groovy variable "committer" with the output of the command:
def committer = utils.sh("curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json | python -mjson.tool | grep authorEmail | awk '{print \$2}' | tr -d '"|,' ")
Because of the known issue in Jenkins (JENKINS-26133) it is not possible to do that but only to populate the variable with the exit status of the command.
So I've go these 2 functions:
def gen_uuid(){
randomUUID() as String
}
def sh_out(cmd){ // As required by bug JENKINS-26133
String uuid = gen_uuid()
sh """( ${cmd} )> ${uuid}"""
String out = readFile(uuid).trim()
sh "set +x ; rm ${uuid}"
return out
}
These functions allow me to wrap my shell commands in sh_out(COMMAND) and in the background I'm using the workaround which is suggested in the mentioned above known issue link which means running the command while redirecting it's output to a file (in the case of my function it's a random filename) and then reading it into a variable.
So, In the beginning of my pipeline I load my functions file which ends with return this; like so:
fileLoader.withGit('git#bitbucket.org:company/pipeline_utils.git', 'master', git_creds, ''){
utils = fileLoader.load('functions.groovy');
}
And that's why the "utils.sh_out" that you see in the command, but when I use the shown above command in my Jenkins pipeline, I get the following error:
/home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ#tmp/durable-006d5c7e/script.sh: 2: /home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ#tmp/durable-006d5c7e/script.sh: Bad substitution
Running the command in a shell works properly:
$ curl -s -u user:password http://IPADDR:8080/job/COMPANY_BitBucket_Integration/job/research/job/COMPANY-6870-bitbucket-integration/3/api/json/api/json | python -mjson.tool | grep authorEmail | awk '{print $2}' | tr -d '"|,'
user#email.com
I suspect it has something to do with the tr command in the end and with the character escaping I did there but whatever I try fails, anyone got an idea?
according to the documentation now sh supports std output.
and i know i'm not answering your question directly, but i suggest to use groovy to parse json.
you are trying to get the value of authorEmail from json
if the response from /api/json looks like this (just an example):
{
"a":{
"b":{
"c":"ccc",
"authorEmail":"user#email.com"
}
}
}
then the groovy to take athorEmail:
def cmd = "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
def json = sh(returnStdout: true, script: cmd).trim()
//parse json and access it as an object (Map/Array)
json = new groovy.json.JsonSlurper().parseText(json)
def mail = json.a.b.athorEmail
you could receive java.io.NotSerializableException explained here
so i changed the code like this:
node {
def json = sh(
returnStdout: true,
script: "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
).trim()
def mail = evaluateJson(json, '${json.a.b.authorEmail}')
echo mail
}
#NonCPS
def evaluateJson(String json, String gpath){
//parse json
def ojson = new groovy.json.JsonSlurper().parseText(json)
//evaluate gpath as a gstring template where $json is a parsed json parameter
return new groovy.text.GStringTemplateEngine().createTemplate(gpath).make(json:ojson).toString()
}
I have created a very simple script and would like to pass arguments to the script.
like:
grails> helloworld -n Howdy
grails> helloworld -name Howdy
with the script:
target(main: 'Hello World') {
def cli = new CliBuilder()
cli.with
{
h(longOpt: 'help', 'Help - Usage Information')
n(longOpt: 'name', 'Name to say hello to', args: 1, required: true)
}
def opt = cli.parse(args)
if (!opt) return
if (opt.h) cli.usage()
println "Hello ${opt.n}"
}
I seem to fail in every attempt that i do. The script keeps complain about the -n option being not present.
When i debug the script the value op the args parameters looks like the values are rearranged.
When calling the script with :
grails> helloworld -n Howdy
the value of args inside the script is: Howdy -n
What am i missing here of doing wrong? Any suggestions?
Your problem is that you're running your code through grails shell. I've converted your code to CLI.groovy like this:
class CLI{
public static void main(String [] args){
def cli = new CliBuilder()
cli.with
{
h(longOpt: 'help', 'Help - Usage Information')
n(longOpt: 'name', 'Name to say hello to', args: 1, required: true)
}
def opt = cli.parse(args)
if (!opt) return
if (opt.h) cli.usage()
println "Hello ${opt.n}"
}
}
After that I'm using groovy command to run it from linux shell like that:
archer#capitan $ groovy CLI -n Daddy
It outputs:
archer#capitan $ groovy CLI -n Daddy
Hello Daddy
So it works like a charm.
I did a Google search for site:github.com grailsScript CliBuilder and came across:
https://github.com/Grails-Plugin-Consortium/grails-cxf/blob/master/scripts/WsdlToJava.groovy
That gave me the hint that the args variable needs to be formatted. Unfortunately it mutates -n Howdy into Howdy\n-n (not sure why the order is rearranged or the newline character is added).
The github page above has a doSplit() method to handle some of this, but it keeps the rearranged order. The best thing I've found is to remove the space between -n and Howdy, which will work with CliBuilder.
The following is what I have working:
target(main: 'Hello World') {
def cli = new CliBuilder()
cli.with
{
h(longOpt: 'help', 'Help - Usage Information')
n(longOpt: 'name', 'Name to say hello to', args: 1, required: true)
}
def ops = doSplit(args)
def opt = cli.parse(ops)
if (!opt) return
if (opt.h) cli.usage()
println "Hello ${opt.n}"
}
private doSplit(String string){
string.split(/(\n|[ ]|=)/).collect{ it.trim() }.findResults { it && it != '' ? it : null }
}
Run this with: helloworld -nHowdy