I have trouble accessing nested JSON with readJSON
oldJson string:
{"branch":{"type-0.2":{"version":"0.2","rc":"1","rel":"1","extras":"1"}}}
I try to access it as in example
https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace
assert oldJson["rc"] == '1'
but it fails. I think it because "rc" is nested in "type-02". How could I access it?
You can always get the value of a nested element by its nested key using bracket notation or dot notation.
stage('Read-JSON') {
steps {
script {
def oldJson = '{"branch":{"type-0.2":{"version":"0.2","rc":"1","rel":"1","extras":"1"}}}'
def props = readJSON text: oldJson
println(props['branch']['type-0.2']['rc'])
\\ or println(props.'branch'.'type-0.2'.'rc')
}
}
}
Output:
[Pipeline] stage
[Pipeline] { (Read-JSON)
[Pipeline] script
[Pipeline] {
[Pipeline] readJSON
[Pipeline] echo
1
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
Related
(edited/updated from original post to attempt to address confusion about what the problem is)
The problem is: Values that are set in a Jenkinsfile environment section are not added to the object returned by env.getEnvironment()
The question is: How do I get a map of the complete environment, including values that were assigned in the environment section? Because env.getEnvironment() doesn't do that.
Example Jenkinsfile:
pipeline {
agent any
environment {
// this is not included in env.getEnvironment()
ONE = '1'
}
stages {
stage('Init') {
steps {
script {
// this is included in env.getEnvironment()
env['TWO'] = '2'
}
}
}
stage('Test') {
steps {
script {
// get env values as a map (for passing to groovy methods)
def envObject = env.getEnvironment()
// see what env.getEnvironment() looks like
// notice ONE is not present in the output, but TWO is
// ONE is set using ONE = '1' in the environment section above
// TWO is set using env['TWO'] = '2' in the Init stage above
println envObject.toString()
// for good measure loop through the env.getEnvironment() map
// and print any value(s) named ONE or TWO
// only TWO: 2 is output
envObject.each { k,v ->
if (k == 'ONE' || k == 'TWO') {
println "${k}: ${v}"
}
}
// now show that both ONE and TWO are indeed in the environment
// by shelling out and using the env linux command
// this outputs ONE=1 and TWO=2
sh 'env | grep -E "ONE|TWO"'
}
}
}
}
}
Output (output of envObject.toString() shortened to ... except relevant part):
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Init)
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
[..., TWO:2]
[Pipeline] echo
TWO: 2
[Pipeline] sh
+ env
+ grep -E ONE|TWO
ONE=1
TWO=2
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Notice ONE is missing from the env.getEnvironment() object, but TWO is present.
Also notice that both ONE and TWO are set in the actual environment and I am not asking how to access the environment or how to iterate through the values returned by env.getEnvironment(). The issue is that env.getEnvironment() does not return all the values in the environment, it excludes any values that were set inside the environment section of the Jenkinsfile.
I don't have a "why" answer for you, but you can cheat and get a map by parsing the output from env via the readProperties step.
def envMap = readProperties(text: sh(script: 'env', returnStdout: true))
println(envMap.getClass())
println("${envMap}")
I would get the env and convert it to map with the help of properties
pipeline {
agent any
environment {
// this is not included in env.getEnvironment()
ONE = '1'
}
stages {
stage('Init') {
steps {
script {
// this is included in env.getEnvironment()
env['TWO'] = '2'
}
}
}
stage('Test') {
steps {
script {
def envProp = readProperties text: sh (script: "env", returnStdout: true).trim()
Map envMapFromProp = envProp as Map
echo "ONE=${envMapFromProp.ONE}\nTWO=${envMapFromProp.TWO}"
// now show that both ONE and TWO are indeed in the environment
// by shelling out and using the env linux command
// this outputs ONE=1 and TWO=2
sh 'env | grep -E "ONE|TWO"'
}
}
}
}
}
Output of env.getEnvironment() method will not return a list or Map, Hence it's difficult to iterate with each but there are some workaround you can do to make this work.
import groovy.json.JsonSlurper
pipeline {
agent any;
environment {
ONE = 1
TWO = 2
}
stages {
stage('debug') {
steps {
script {
def jsonSlurper = new JsonSlurper()
def object = jsonSlurper.parseText(env.getEnvironment().toString())
assert object instanceof Map
object.each { k,v ->
echo "Key: ${k}, Value: ${v}"
}
}
}
}
}
}
Note - env.getEnvironment().toString() will give you a JSON String . While parsing the JOSN string if groovy jsonSlurper.parseText found any special character it will through an error
You can also explore a little bit around env Jenkins API and find an appropriate method that will either return a Map or List so that you can use each
I want to use some common value across different conditions in post section of pipeline hence I tried following -
1.
post {
script {
def variable = "<some dynamic value here>"
}
failure{
script{
"<use variable here>"
}
}
success{
script{
"<use variable here>"
}
}
}
2
post {
def variable = "<some dynamic value here>"
failure{
script{
"<use variable here>"
}
}
success{
script{
"<use variable here>"
}
}
}
But it results into compilation error.
Can you please suggest how I can declare a variable in post section which can be used across conditions?
You could use always condition which is guaranteed to be executed before any other conditions like success or failure. If you want to store String value, you can use use env variable to store it (environment variable always casts given value to a string). Alternatively, you can define a global variable outside the pipeline and then initialize it with the expected dynamic value inside the always condition. Consider the following example:
def someGlobalVar
pipeline {
agent any
stages {
stage("Test") {
steps {
echo "Test"
}
}
}
post {
always {
script {
env.FOO = "bar"
someGlobalVar = 23
}
}
success {
echo "FOO is ${env.FOO}"
echo "someGlobalVar = ${someGlobalVar}"
}
}
}
Output:
Running on Jenkins in /home/wololock/.jenkins/workspace/pipeline-post-sections
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] echo
Test
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] echo
FOO is bar
[Pipeline] echo
someGlobalVar = 23
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
I have trouble reading all keys "type-X.X" from JSON with readJSON
oldJson string:
{
"branch":{
"type-0.2":{"version":"0.2","rc":"1","rel":"1","extras":"1"}}
"type-0.3":{"version":"0.3","rc":"1","rel":"1","extras":"1"}}
}
I try to access it as in example
https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace
def branchList = new JsonSlurper().parseText(oldJson['branch'])
echo (branchList.keySet().toString())
but it fails:
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No
signature of method: groovy.json.JsonSlurper.parseText() is applicable
for argument types: (net.sf.json.JSONObject) values:
I would like to get a list ["type-0.2", "type-0.3"]
The JSON string you have provided is not valid. There is an extra } and a missing , after the first child element. It has to be:
{
"branch":{
"type-0.2":{"version":"0.2","rc":"1","rel":"1","extras":"1"},
"type-0.3":{"version":"0.3","rc":"1","rel":"1","extras":"1"}
}
}
Now, you can parse this using the readJSON step in your pipeline to get a list of keys.
stage('Read-JSON') {
steps {
script {
def oldJson = '''{
"branch":{
"type-0.2":{"version":"0.2","rc":"1","rel":"1","extras":"1"},
"type-0.3":{"version":"0.3","rc":"1","rel":"1","extras":"1"}
}
}'''
def props = readJSON text: oldJson
def keyList = props['branch'].keySet()
echo "${keyList}"
// println(props['branch'].keySet())
}
}
}
Output:
[Pipeline] stage
[Pipeline] { (Read-JSON)
[Pipeline] script
[Pipeline] {
[Pipeline] readJSON
[Pipeline] echo
[type-0.2, type-0.3]
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
I am trying to split a URL http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip in groovy DSL of Jenkins. It is a single line string. But following code does not work
String[] arr= string_var.split('/');
String[] arr=string_var.split('\\/');
It does not split it and returns itself in arr[0].
I am not sure if it is a bug. Please let me know if any other way is there in groovy to get "sub1" from URL string.
Are you sure that you doing DSL script correctly? As the groovy code looks to be OK.
Try to skip declaring types
def url_str = 'http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip'
def sub = url_str.split('/')[-2]
println(sub)
in one line:
println('http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip'.split('/')[-2])
no split, indexes:
def url_str = 'http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip'
int[] indexes = url_str.findIndexValues {it == "/"}
println url_str.substring(indexes[-2] + 1, indexes[-1])
Try enclosing your code inside a 'script' tag of DSL language like the following piece of code:
pipeline {
agent any
stages {
stage('Test') {
steps {
script {
def string_var = "http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip"
String[] arr= string_var.split('/');
println "${arr[0]}"
}
}
}
}
}
Executing code above I get this result on the console:
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
http:
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Thus, giving the expected 'http:' String
Another Groovy way to get the string 'sub1' (regex):
String s = "http://localhost:8081/artifactory/api/storage/myrepo/sub1/file.zip"
def match = s =~ /(sub1)/
if (match.size() > 0) { // If url 's' contains the expected string 'sub1'
println match[0][1]
// Or do something with the match
}
My question is similar to this one about how to load an external groovy script, and then calling a method from it in a different groovy script. So far I have been able to get methods that don't return a value to work but I am having trouble getting a returned value into a variable that is called.
For example, the following pipeline code works but gives a value of null for $build_user when I run the Jenkins pipeline. It doesn't actually return what I expect it to and I don't know why.
node {
stage('test') {
def tools = load "/var/lib/jenkins/workflow-libs/vars/tools.groovy"
build_user = tools.get_user()
echo "build_user: $build_user"
}
}
Here is what the relevant tools.groovy looks like.
def exampleMethod() {
// Do stuff
}
// Try to get a build username
def get_user() {
try {
wrap([$class: 'BuildUser']) {
// Set up our variables
fallback_user = 'GitHub'
github_user = BUILD_USER
commit_author = 'Test1'
// Try to use Jenkins build user first
if (github_user) {
echo "using github_user: $github_user"
return github_user
}
// Otherwise try to use commit author
else if (commit_author) {
echo "using commit_author: $commit_author"
return commit_author
}
// Otherwise username is blank so we use the default fallback
else {
echo "using fallback: $fallback_user"
return fallback_user
}
}
}
catch (err) {
// Ignore errors
}
echo "Done."
}
return this
Here is the full Jenkins output for the above code.
Started by user XXX
[Pipeline] node
Running on master in /var/lib/jenkins/workspace/test
[Pipeline] {
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] load
[Pipeline] { (/var/lib/jenkins/workflow-libs/vars/tools.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] wrap
[Pipeline] {
[Pipeline] echo
using github_user: XXX
[Pipeline] }
[Pipeline] // wrap
[Pipeline] echo
Done.
[Pipeline] echo
build_user: null
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
The above method doesn't work at all if I remove return this at the end and throws the following error in Jenkins.
java.lang.NullPointerException: Cannot invoke method get_user() on
null object
...
What am I doing wrong? I suspect that I'm missing something easy but I'm not great with Groovy, so I'm not sure what it could be.
You have to end your tools.groovywith return this.
See the answer on this question How do you load a groovy file and execute it
your function get_user() returns nothing.
the return(s) inside wrap([$class: 'BuildUser']) {...} do return from wrap class and not from your function.