Merge two based yaml with indentation in groovy - jenkins

I am trying to merge two yamls in jenkins file (.Jenkinsfile)
def runConf = """
fail_fast: false
tests:
- ABC:
"""
print(runConf)
def defaultRunConfig = """
test:
- ABC.json
- DEF.json
Seconds: 60
Type: dynamic
"""
def runConfigObj
runConfigObj = readYaml([text:defaultRunConfig])
String time = defaultRunConfig.Seconds.toString()
echo ("Duration for run: " +time)
def runConfigBase
runConfigBase = readYaml([text:runConf])
Map<Object, Object> runConfiguration = [:]
runConfiguration.putAll(runConfigBase)
print(runConfiguration)
for (tests in runConfigObj.test) {
print(runConfigObj.test)
runConfiguration.put(runConfigObj.test)
}
runConfiguration['Seconds'] = runConfigObj.Seconds.toString()
runConfiguration['Type'] = runConfigObj.Type
Expectation is to get something like:
fail_fast: false
tests:
- ABC:
test:
- ABC.json
- DEF.json
Seconds: 60
Type: dynamic
But I am getting below Error:
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.util.LinkedHashMap.put() is applicable for argument types: (java.util.ArrayList) values:
Tried looking into options like MAP in groovy but not able to resolve the issue
How to avoid such an error? I am new to groovy, any help is much appreciated

So you want to add an array of "tests" as values to the key test into a map object of runConfiguration, then it should be something like this:
runConfiguration['test']=[]
for (test in runConfigObj) {
runConfiguration['test'].add(test)
}

Related

print pass or failed job in a parallel Jenkins code

I'm running the following code where the idea is to run as much "TestRunner" as possible during night. I've removed some unnecessary code if some variables aren't there the for easier reading.
I want to know each job if it was successful, aborted or failed and when I'm using parallel I'm not able to see it.
How can I modify my code so i can print each job state when its done? adding a variable is being wiped out to the last element in the for loop.
Thanks a lot
def numberOfRuns = 0
def availableExecutors = 5
def parallelRuns = [:]
// building executers for later use in parallel
for (int i = 0; i < availableExecutors; i++) {
parallelRuns[i] = {
waitUntil {
build job: 'TestRunner', parameters: [
string(name: 'randomBit', value: "${randomBit}"),
], propagate: false
numberOfRuns++
def now = new Date()
return (now > workDayMorning)
}
}
}
// Parallel stage - running al executers
parallel parallelRuns
I've tried to enter a variable to track job process, i tried to use the parallelRuns as object but didnt manage to get the result of each test passed or not.
A solution i've found is:
def Job = build job: 'TestRunner', parameters: [
string(name: 'randomBit', value: ${randomBit}")
], propagate: false
echo "Runner: ${Job.getDisplayName()} has ${Job.getResult()}
with duration: ${Job.getDuration()}"
This prints data of the job.

why is my for-each loop not working properly in parallel stages in jenkins scripted syntax? [duplicate]

In the context of Jenkins pipelines, I have some Groovy code that's enumerating a list, creating closures, and then using that value in the closure as a key to lookup another value in a map. This appears to be rife with some sort of anomaly or race condition almost every time.
This is a simplification of the code:
def tasks = [:]
for (platformName in platforms) {
// ...
tasks[platformName] = {
def componentUploadPath = componentUploadPaths[platformName]
echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
// ...
}
tasks.failFast = true
parallel(tasks)
platforms has two values. I will usually see two iterations and two tasks registered and the keys in tasks will be correct, but the echo statement inside the closure indicates that we're just running one of the platforms twice:
14:20:02 [platform2] Uploading for platform [platform1] to [some_path/platform1].
14:20:02 [platform1] Uploading for platform [platform1] to [some_path/platform1].
It's ridiculous.
What do I need to add or do differently?
It's the same issue as you'd see in Javascript.
When you generate the closures in a for loop, they are bound to a variable, not the value of the variable.
When the loop exits, and the closures are run, they will all be using the same value...that is -- the last value in the for loop before it exited
For example, you'd expect the following to print 1 2 3 4, but it doesn't
def closures = []
for (i in 1..4) {
closures << { -> println i }
}
closures.each { it() }
It prints 4 4 4 4
To fix this, you need to do one of two things... First, you could capture the value in a locally scoped variable, then close over this variable:
for (i in 1..4) {
def n = i
closures << { -> println n }
}
The second thing you could do is use groovy's each or collect as each time they are called, the variable is a different instance, so it works again:
(1..4).each { i ->
closures << { -> println i }
}
For your case, you can loop over platforms and collect into a map at the same time by using collectEntries:
def tasks = platforms.collectEntries { platformName ->
[
platformName,
{ ->
def componentUploadPath = componentUploadPaths[platformName]
echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
}
]
}
Hope this helps!

Test if there is a match when using regular expression in jenkins pipeline

I am using regex to grab a number from a string in my pipeline
it works ok as long that I have a match, but when there is no match I get an error
java.lang.IndexOutOfBoundsException: index is out of range 0..-1 (index = 0)
There error happens when i try to capture the group on following line
env.ChangeNr = chngnr[0][1]
How can i test if there isn't a match from my capture group ?
This is the pipeline
pipeline {
agent {
node {
label 'myApplicationNode'
}
}
environment {
GIT_MESSAGE = "${bat(script: "git log --no-walk --format=format:%%s ${GIT_COMMIT}", returnStdout: true)}".readLines().drop(2).join(" ")
}
stages {
stage('get_commit_msg'){
steps {
script {
def gitmsg=env.GIT_MESSAGE
def chngnr = gitmsg =~/([0-9]{1,8})/
env.ChangeNr = chngnr[0][1] /* put test if nothing is extracted */
}
}
}
}
}
In groovy when you use the =~ (find operator) it actually creates a java.util.regex.Matcher and therefore you can use any of its standard methods like find() or size(), so in your case you can jest use the size function to test if there are any matched patterns before you attempt to extract any groups:
def chngnr = gitmsg =~/([0-9]{1,8})/
assert chngnr.size() > 0
env.ChangeNr = chngnr[0][1]
Another nice option is to use the =~ operator in context of boolean, in this case, Groovy implicitly invokes the matcher.find() method, which means that the expression evaluates to true if any part of the string matches the pattern:
def chngnr = gitmsg =~/([0-9]{1,8})/
if(chngnr){
env.ChangeNr = chngnr[0][1]
}
else {
...
}
You can read more info on Groovy Regular Expressions Here

How to fix java.io.NotSerializable exception when pushing each stage data from jenkins pipeline to influx db?

I’m trying to push stage data from Jenkins pipeline to influx db and I get issue as below
Issue: In jenkin builds console output, after each stage, I see : java.io.NotSerializableException: sun.net.www.protocol.http.HttpURLConnection
Jenkins pipeline 2.150.2
InfluxDB 1.6.2
Any suggestions would be helpful. I'm new to this topic.
Note: I have commented #NonCPS annotation. If I uncomment then it sends only first stage data and exits and fails to iterate for loop which has 20 stages data.
//Maps for Field type columns
myDataField1 = [:]
myDataField2 = [:]
myDataField3 = [:]
//Maps for Custom Field measurements
myCustomDataFields1 = [:]
myCustomDataFields2 = [:]
myCustomDataFields3 = [:]
//Maps for Tag type columns
myDataTag1 = [:]
myDataTag2 = [:]
myDataTag3 = [:]
//Maps for Custom Tag measurements
myCustomDataTags1 = [:]
myCustomDataTags2 = [:]
myCustomDataTags3 = [:]
//#NonCPS
def pushStageData() {
def url_string = "${JENKINS_URL}job/ENO_ENG_TP/job/R421/13/wfapi/describe"
def get = new URL(url_string).openConnection();
get.addRequestProperty ("User-Agent","Mozilla/4.0");
get.addRequestProperty("Authorization", "Basic dZXZvceDIwMTk=");
//fetching the contents of the endpoint URL
def jsonText = get.getInputStream().getText();
//converting the text into JSON object using
JsonSlurperClassic
def jsonObject = new JsonSlurperClassic().parseText(jsonText)
// Extracting the details of all the stages present in that particular build number
for (int i=0; i<jsonObject.stages.size()-1; i++){ //size-1 to ignore the post stage
//populating the field type columns of InfluxDB measurements and pushing them to the map called myDataField1
def size = jsonObject.stages.size()-1
myDataField1['result'] = jsonObject.stages[i].status
myDataField1['duration'] =
jsonObject.stages[i].durationMillis
myDataField1['stage_name'] = jsonObject.stages[i].name
//populating the tag type columns of InfluxDB
measurements and pushing them to the map called
myDataTag1
myDataTag1['result_tag'] = jsonObject.stages[i].status
myDataTag1['stage_name_tag'] = jsonObject.stages[i].name
//assigning field type columns to the measurement called CustomData
myCustomDataFields1['CustomData'] = myDataField1
//assigning tag type columns to the measurement called CustomData
myCustomDataTags1['CustomData'] = myDataTag1
//Push the data into influx instance
try
{ step([$class: 'InfluxDbPublisher', target: 'jenkins_data', customPrefix: null, customDataMapTags: myCustomDataTags1]) }
catch (err)
{ println ("pushStagData exception: " + err) }
}
}
Expected: I want to push each stages of jenkins pipeline data to influx db.
Try to use jenkins' standard methods instead of creating objects manually. You can use httpRequest instead of URL. This should fix exception you are getting. You can also use readJson instead of JsonSlurperClassic (latter is optional since classic slurper is actually serializable).
you just need to explicitly set get to null to make sure there's none non serializable object being instantiated.
def jsonText = get.getInputStream().getText();
get = null;

How to autogenerate parameters in an input step from Jenkins pipeline?

I would like to dynamically generate my parameters on an input step from a loop inside my Jenkins pipeline.
This is my code:
var = input message: 'Tags a saisir', ok: 'Build!',
parameters: [
choice(name: 'name1', choices: file1),
choice(name: 'name2', choices: file2),
choice(name: 'name3', choices: file3),
choice(name: 'name4', choices: file4)
]
I would like to know if it is possible to generate each parameter from a loop like :
for (int i = 0; i < myList.size(); i++) {
theChoice = "choice(name : "+myList.get(i)+'choices: file"+i+")
}
and generate the input step from those lines.
Is that kind of approach possible ?
The main goal is to generate an input step with modular variables dependending on the jenkins file clone from the Git SCM
Regards,
Guillaume
You can do it like this
def parameterNames = fileMap.keySet().toList()
def choices = []
for (int i = 0; i < parameterNames.size(); i++) {
choices += choice(name : parameterNames[i], choices: fileMap[parameterNames[i]])
}
def var = input message: 'Tags a saisir', ok: 'Build!', parameters: choices
Where fileMap is a Map which contains possible choices fore each parameter name (key is a parameter name, value is a String of possible choices)

Resources