I have a string in groovy that I want to convert into a map. When I run the code on my local computer through a groovy script for testing, I have no issues and a lazy map is returned. I can then convert that to a regular map and life goes on. When I try the same code through my Jenkins DSL pipeline, I run into the exception
groovy.json.internal.Exceptions$JsonInternalException: Not that kind of map
Here is the code chunk in question:
import groovy.json.*
String string1 = "{value1={blue green=true, red pink=true, gold silver=true}, value2={red gold=false}, value3={silver brown=false}}"
def stringToMapConverter(String stringToBeConverted){
formattedString = stringToBeConverted.replace("=", ":")
def jsonSlurper = new JsonSlurper().setType(JsonParserType.LAX)
def mapOfString = jsonSlurper.parseText(formattedString)
return mapOfString
}
def returnedValue = stringToMapConverter(string1)
println(returnedValue)
returned value:
[value2:[red gold:false], value1:[red pink:true, gold silver:true, blue green:true], value3:[silver brown:false]]
I know that Jenkins and Groovy differ in various ways, but from searches online others suggest that I should be able to use the LAX JsonSlurper library within my groovy pipeline. I am trying to avoid hand rolling my own string to map converter and would prefer to use a library if it's out there. What could be the difference here that would cause this behavior?
Try to use
import groovy.json.*
//#NonCPS
def parseJson(jsonString) {
// Would like to use readJSON step, but it requires a context, even for parsing just text.
def lazyMap = new JsonSlurper().setType(JsonParserType.LAX).parseText(jsonString.replace("=", ":").normalize())
// JsonSlurper returns a non-serializable LazyMap, so copy it into a regular map before returning
def m = [:]
m.putAll(lazyMap)
return m
}
String string1 = "{value1={blue green=true, red pink=true, gold silver=true}, value2={red gold=false}, value3={silver brown=false}}"
def returnedValue = parseJson(string1)
println(returnedValue)
println(JsonOutput.toJson(returnedValue))
You can find information about normalize here.
Related
After reviewing the docs, a number of questions here on SO, and trying a dozen or so different script configurations, I cannot figure out how to reference a shared Groovy library. I've added the library like so:
This appears to be working. I'm referencing the script like so:
You can see the error message therein:
Script1: 1: unable to resolve class Library , unable to find class
for annotation # line 1, column 1. #Library('sonarQubeAPI')_
The script code, not I think it matters, looks like this:
import groovy.json.JsonSlurper
class SonarQubeAPI{
static string getVersion(){
return "1.0";
}
static void getSonarStatus(projectKey){
def sonarQubeUserToken = "USERTOKEN";
def projectStatusUrl = "pathtosonarqube/api/qualitygates/project_status?projectKey=" + projectKey;
println("Retrieving project status for " + projectKey);
def json = getJson(sonarQubeUserToken, projectStatusUrl);
def jsonSlurper = new JsonSlurper();
def object = jsonSlurper.parseText(json);
println(object.projectStatus.status);
}
static string getJson(userToken, url){
def authString = "${userToken}:".getBytes().encodeBase64().toString();
def conn = url.toURL().openConnection();
conn.setRequestProperty( "Authorization", "Basic ${authString}" );
return conn.content.text;
}
}
I'm probably just a magic character off, but I can't seem to lock it down.
Shared libraries are a feature of Jenkins Pipelines, not of Jenkins (core) itself. You can use them only in Pipeline jobs (and child types like Multibranch Pipeline).
I am retrieving JSON object from a URL using httpRequest in a groovy script.
pipeline {
agent any
stages {
stage ('Extract Data') {
steps {
script {
def response = httpRequest \
authentication: 'user', \
httpMode: 'GET', \
url: "https://example.com/data"
writeFile file: 'output.json', text: response.content
def data = readFile(file: 'output.json')
def details = new groovy.json.JsonSlurperClassic().parseText(data)
echo "Data: ${details.fields.customfield}"
}
}
}
}
}
I am interested in the customfieldstring. The format of the string is:
Application!01.01.01 TestSuite1,TestSuite2,TestSuite3,TestSuite4 Product!01.01.01,Product2!01.01.02
I would like to parse the string into 3 data sets:
Map of Applications [Application: version] (there will always be one Appliction)
List of TestSuites [TestSuite1,...,TestSuite]
Map of Prodcts [Product1: version,..., ProductN: version].
However, I am not sure how to do this.
Are there any Jenkins Groovy libraries that I can use to do this in a declarative pipeline?
EDIT
Based on the answer below I can see that I can make a map in the following way:
def applications = groups[0].split(',').collect { it.split('!') }.collectEntries { [(it):it] }
In the example I have:
application = [Application: Application]
How do I get:
application = [Application: 01.01.01]
EDIT2
Note the following output:
def applications = groups[0].split(',').collect { it.split('!') }
[[Application, 01.01.01]]
There're no libraries I'm aware of that will have functionality to parse the data but, since you know the format of the data it's easy to parse them manually.
There are 3 groups in the input (applications, suites, products) separated by a character. To get the groups you need:
def input = "Application!01.01.01 TestSuite1,TestSuite2,TestSuite3,TestSuite4 Product!01.01.01,Product2!01.01.02"
def groups = input.split(' ')
To process the applications you need to split group 0 with , character (just in case there are many applications). You got a list of pairs in format: name!version. Every pair must be splitted with !, so you get a list of lists in format: [[name, version]]. From the last structure it's easy to create a map. All steps together:
def applications = groups[0].split(',').collect { it.split('!') }.collectEntries { [(it[0]):it[1]] }
Getting the list of suites is easy, just split group 1 with , character:
def suites = groups[1].split(',')
Finally, products are analogical to the list of applications but this time group 2 should be used:
def products = groups[2].split(',').collect { it.split('!') }.collectEntries { [(it[0]):it[1]] }
You can simplifier your issue by using pipeline utility step: readJSON
def data = readJSON(file: 'output.json')
echo data.fields.customfield
I found a method. Groovy can convert the values of an Object array and convert them into a map with the toSpreadMap(). However, the array must have an even number of elements.
def appList = ['DevOpsApplication', '01.01.01']
def appMap = appList.toSpreadMap()
For some better answers please refer to this
I am facing some issues reading a JSON file.
I am using Jenkins Active Choice Parameter to read value from a JSON file via groovy script. This is how my JSON file look.
{
"smoke": "Test1.js",
"default": "Test2.js"
}
I want my groovy script to print out smoke and default. Below is what my groovy code look like.
import groovy.json.JsonSlurper
def inputFile = new File(".\TestSuitesJ.json")
def InputJSON = new JsonSlurper().parseText(inputFile)
InputJson.each
{
return[
key
]
}
Above code is not working for me. Can someone please suggest a better groovy way?
Anyone in similar situation as me trying to import a JSON file at runtime. I used Active Choice parameter to solve my problem. There is an option to write groovy script in Active Choice Parameter plugin of Jenkins. There i have written below code to import a JSON file to achieve desired results.
import groovy.json.JsonSlurper
def inputFile = new File('.//TestSuitesJ.json')
def inputJSON = new JsonSlurper().parse(inputFile)
def keys = inputJSON.keySet() as List
Thanks #sensei to help me learn groovy.
You really should read the groovy dev kit page, and this in particular.
Because in your case parseText() returns a LazyMap instance, the it variable you're getting in your each closure represents a Map.Entry instance. So you could println it.key to get what you want.
A more groovy way would be:
inputJson.each { k, v ->
println k
}
In which case groovy passes your closure the key (k) and the value (v) for every element in the map.
I'm trying to create a job that will run a certain flow multiple times, each time with different parameters on multiple nodes in parallel.
I have a csv file, on which each line contains the requested parameters for a run.
I tried using multi configuration job, and I read about the dynamic axis, but I don't quite understand how to use it with the data from my csv file.
I also saw build flow and workflow plugins, but again, I couldn't understand how to use it with my csv file.
I'd appreciate if anyone can give me ideas how to solve this.
Thanks in advance,
Sivan
Beneath a solution without eachLine closure (works in Jenkins ver. 2.89.3).
Some closures like eachLine still seem to be broken in Jenkins.
def nodes = [:]
readFile("input.csv").split('\n').eachWithIndex { line, index ->
def params = line.split(',')
nodes[name] = {
// ...
}
If you don't need the counter, you can use 'each' instead
readFile("input.csv").split('\n').each { line -> ... }
Using the workflow plugin you could read the file, parse the contents with standard groovy then set up your nodes - something like
def nodes = [:]
readFile("myfile.csv").eachLine { line, count ->
def params = line.split(',')
nodes["line${count}"] = {
node {
// do stuff
}
}
}
parallel nodes
if you dont need the count variable you could use splitEachLine instead
def nodes = [:]
readFile("abc.csv").splitEachLine(/,/) { runName, param2, p3 ->
nodes[runName] = {
// dostuff with param2, p3
}
}
When promoting a build, I want to expose some choices of how/what to promote. To do this properly, I need some info about the build which I am promoting, and I'd really like to use the Extended Choice Parameter to do this.
How can I access a build's parameters in an Extended Choice Parameter block in a build promotion step? For example, say the build I want to promote was built with a parameter myvar="1,2,5", and I want to let the build promoter select 1, 2, 5, or some combination. How can I access that particular build's myvar value, in an extended choice parameter, in a promotion process?
I can auto-populate these with available Git branches for a certain repo, with global environment variables, but I haven't seen the groovy magic I need to access info about the build I'm (about to) promote.
This was not obvious to me, but I found that environment variables such as PROMOTED_URL, PROMOTED_JOB_NAME, etc, were not available until the promotion steps were in progress. I need this information exposed when the promotion parameters are being populated via Extended Choice Parameter's Groovy Script fields. I accomplished it by peeking into the thread name:
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
def jenkins = Jenkins.instance
def thread = Thread.currentThread().toString()
def (jobName,buildNumber) = thread.replaceAll(/.*GET \/job\/(.*)\/promotion\/.*/, '$1').split('/')
def job = jenkins.getJob(jobName)
def build = job.getBuild(buildNumber)
return build.getBuildVariables()["SOME_BUILD_PARAM"]
What do I use this for? Say I can select certain targets to build but not others, when building. I want to be able to promote only some subset of those that were actually built, so the parameter can't just hold a default value. Pretty narrow edge-case, I'll grant, but sometimes it seems like my entire life, as far as Jenkins is concerned, is dealing with some edge-case or another. If there is a more elegant way, I am all ears!
building on the #courtlandj solution (with the latest hudson API):
import hudson.model.*
def paramToLookup = "SOME_BUILD_VARIABLE_NAME"
// you can of course parse the jobName from the currentThread
// but that is a case-by-case basis depending on how you name them
// build numbers are strictly numeric - and more predictable to parse with regex
def jobName = "some-jenkins-full-job-name"
int buildNumber = 0
// Similar approach to courtlandj to get THIS build number (of page being viewed):
def thisPromotionInfo = Thread.currentThread().toString()
// groovy regex that gets the promotion digits (\d)
def buildRegex = /.*\/(\d+)\/promotion\/.*/
def matcher = ( thisPromotionInfo =~ buildRegex )
if (matcher.matches()) {
buildNumber = matcher[0][1].toInteger()
} else {
return "unable to identify buildNumber in Extended Choice Parameter (see Jenkins Config)"
}
// borrowed slightly from here: https://wiki.jenkins-ci.org/display/JENKINS/Display+job+parameters
try {
def job = Hudson.instance.getItemByFullName(jobName)
def build = job.getBuildByNumber(buildNumber)
return build.getProperties()["buildVariableResolver"].resolve(paramToLookup).toString()
} catch(Exception ex) {
return ex.toString()
}
relevant links I borrowed from:
https://groups.google.com/forum/#!topic/jenkinsci-users/cHClVbjuSN4
http://groovy.jmiguel.eu/groovy.codehaus.org/Tutorial+5+-+Capturing+regex+groups.html
http://hudson-ci.org/javadoc/hudson/model/Hudson.html