I'm running the following code where the idea is to run as much "TestRunner" as possible during night. I've removed some unnecessary code if some variables aren't there the for easier reading.
I want to know each job if it was successful, aborted or failed and when I'm using parallel I'm not able to see it.
How can I modify my code so i can print each job state when its done? adding a variable is being wiped out to the last element in the for loop.
Thanks a lot
def numberOfRuns = 0
def availableExecutors = 5
def parallelRuns = [:]
// building executers for later use in parallel
for (int i = 0; i < availableExecutors; i++) {
parallelRuns[i] = {
waitUntil {
build job: 'TestRunner', parameters: [
string(name: 'randomBit', value: "${randomBit}"),
], propagate: false
numberOfRuns++
def now = new Date()
return (now > workDayMorning)
}
}
}
// Parallel stage - running al executers
parallel parallelRuns
I've tried to enter a variable to track job process, i tried to use the parallelRuns as object but didnt manage to get the result of each test passed or not.
A solution i've found is:
def Job = build job: 'TestRunner', parameters: [
string(name: 'randomBit', value: ${randomBit}")
], propagate: false
echo "Runner: ${Job.getDisplayName()} has ${Job.getResult()}
with duration: ${Job.getDuration()}"
This prints data of the job.
Related
In the context of Jenkins pipelines, I have some Groovy code that's enumerating a list, creating closures, and then using that value in the closure as a key to lookup another value in a map. This appears to be rife with some sort of anomaly or race condition almost every time.
This is a simplification of the code:
def tasks = [:]
for (platformName in platforms) {
// ...
tasks[platformName] = {
def componentUploadPath = componentUploadPaths[platformName]
echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
// ...
}
tasks.failFast = true
parallel(tasks)
platforms has two values. I will usually see two iterations and two tasks registered and the keys in tasks will be correct, but the echo statement inside the closure indicates that we're just running one of the platforms twice:
14:20:02 [platform2] Uploading for platform [platform1] to [some_path/platform1].
14:20:02 [platform1] Uploading for platform [platform1] to [some_path/platform1].
It's ridiculous.
What do I need to add or do differently?
It's the same issue as you'd see in Javascript.
When you generate the closures in a for loop, they are bound to a variable, not the value of the variable.
When the loop exits, and the closures are run, they will all be using the same value...that is -- the last value in the for loop before it exited
For example, you'd expect the following to print 1 2 3 4, but it doesn't
def closures = []
for (i in 1..4) {
closures << { -> println i }
}
closures.each { it() }
It prints 4 4 4 4
To fix this, you need to do one of two things... First, you could capture the value in a locally scoped variable, then close over this variable:
for (i in 1..4) {
def n = i
closures << { -> println n }
}
The second thing you could do is use groovy's each or collect as each time they are called, the variable is a different instance, so it works again:
(1..4).each { i ->
closures << { -> println i }
}
For your case, you can loop over platforms and collect into a map at the same time by using collectEntries:
def tasks = platforms.collectEntries { platformName ->
[
platformName,
{ ->
def componentUploadPath = componentUploadPaths[platformName]
echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
}
]
}
Hope this helps!
I have the following code as part of my declarative pipeline:
String CRON_SETTINGS = BRANCH_NAME ==~ /(master|.*release.*)/ ? '''30 23 * * * % param1=value1''' : ""
pipeline {
parameters {
choice(name: 'param1', choices: ['value1', 'value2'], description: 'param')
}
triggers {
parameterizedCron(CRON_SETTINGS)
}
}
Currently the Cron behaves in the following way:
every night at 23:30 PM a build of the job is built if my branch name is master or if it contains the string 'release', always with the value of param1 set to value1.
What I would like to achieve is this:
In case the barnch name is master, run the cron with value1 set to param1 parameter,
However, if the branch name contains 'release', then run the cron with value2 set to param1 parameter.
Would apprreciate your help to achieve this,
Thanks.
Late answer and I haven't tested this, but how about using a switch statement?
switch (BRANCH_NAME) {
case 'master':
cronSettings = '30 23 * * * % param1=value1'
break
case '.*release.*':
cronSettings = '30 23 * * * % param1=value2'
break
default:
cronSettings = ""
break
}
pipeline {
parameters {
choice(name: 'param1', choices: ['value1', 'value2'], description: 'param')
}
triggers {
parameterizedCron(cronSettings)
}
}
I have a Shared Library containing a declarative pipeline which numerous jobs are using to build with.
However I want to be able to pass the trigger block in from the Jenkinsfile to the Shared Library as some jobs I want to trigger via Cron, others I want to trigger via SNS and others with an upstream job trigger.
Is there a way I can do this? Everything I have tried so fails
I have tried
#Jenkinsfile
#Library('build') _
buildAmi{
OS = "amazonlinux"
owners = "amazon"
filters = "\"Name=name,Values=amzn-ami-hvm-*-x86_64-gp2\""
template = "linux_build_template.json"
trigger = triggers {
cron('0 H(06-07) * * *')
}
#Shared Lib
pipeline {
$buildArgs.trigger
which fails with
Not a valid section definition
Have also tried passing just the cron schedule into the Shared lib e.g.
triggers {
cron("${buildArgs.cron}")
}
but that gives the error
Method calls on objects not allowed outside "script" blocks
Have tried various other thing but it seems the declarative style requires a trigger block with just triggers inside.
Does anyone know of a way to achieve what I am trying to do?
Too much code for a comment:
We wanted to merge some preset properties with jenkinsfile properties.
This is the way to get this to work since the properties method can only be called once.
We used an additional pipeline parameter with the trigger info:
Jenkinsfile:
mypipeline (
pipelineTriggers: [pollSCM(scmpoll_spec: '0 5,11,15,22 * * 1-5')],
)
then in the shared library
#Field def pipelineProperties = [
disableConcurrentBuilds(abortPrevious: true),
buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '0'))
]
def pipelineTriggersArgs = args.pipelineTriggers
def setJobProperties() {
def props = pipelineProperties
def str = ""
pipelineProperties.each { str += (blue + it.getClass().toString() + it.toString() + "\n") }
if (pipelineTriggersArgs) {
if (debugLevel > 29) {
def plTrig = pipelineTriggers(pipelineTriggersArgs)
str += (mag + "args: " + pipelineTriggersArgs.getClass().toString() + "\t\t" + pipelineTriggersArgs.toString() + "\n" + off)
str += (red + "expr: " + plTrig.getClass().toString() + "\t\t" + plTrig.toString() + "\n" + off)
echo str
}
props?.add(pipelineTriggers(pipelineTriggersArgs)) // pass param with list as the arg
}
properties(props)
}
That was the only way to merge preset and parameters since the properties cannot be overwritten. not exactly the answer but an approach to solve it I think.
I have a pipeline job which run some sequence of action (Eg; Build >> Run >> Report). I have put this sequence in a for loop as I can get a parameter which has list of value for which I should repeat the same sequence. Please find the sample code I have written.
param_val = params.param_1
param_list = param_val.split()
for (int i = 0; i < size(param_list); ++i){
item_value = param_list[i].trim()
builds[i] ={
stage('Build') {
build 'Build', parameters: [string(name: 'item', value: item_value)]
}
stage('Run') {
build 'Run', parameters: [string(name: 'item', value: item_value)]
}
stage('Reporting') {
build 'Reporting', parameters: [string(name: 'item', value: item_value)]
}
}parallel builds
}
I can get even more than 100 values for the iteration. So it is difficult to track it in the pipeline stage view. it just grows to right side.
Do we have any other plugin which can be used for the status view ? Some tree like view would be good.
I’m trying to push stage data from Jenkins pipeline to influx db and I get issue as below
Issue: In jenkin builds console output, after each stage, I see : java.io.NotSerializableException: sun.net.www.protocol.http.HttpURLConnection
Jenkins pipeline 2.150.2
InfluxDB 1.6.2
Any suggestions would be helpful. I'm new to this topic.
Note: I have commented #NonCPS annotation. If I uncomment then it sends only first stage data and exits and fails to iterate for loop which has 20 stages data.
//Maps for Field type columns
myDataField1 = [:]
myDataField2 = [:]
myDataField3 = [:]
//Maps for Custom Field measurements
myCustomDataFields1 = [:]
myCustomDataFields2 = [:]
myCustomDataFields3 = [:]
//Maps for Tag type columns
myDataTag1 = [:]
myDataTag2 = [:]
myDataTag3 = [:]
//Maps for Custom Tag measurements
myCustomDataTags1 = [:]
myCustomDataTags2 = [:]
myCustomDataTags3 = [:]
//#NonCPS
def pushStageData() {
def url_string = "${JENKINS_URL}job/ENO_ENG_TP/job/R421/13/wfapi/describe"
def get = new URL(url_string).openConnection();
get.addRequestProperty ("User-Agent","Mozilla/4.0");
get.addRequestProperty("Authorization", "Basic dZXZvceDIwMTk=");
//fetching the contents of the endpoint URL
def jsonText = get.getInputStream().getText();
//converting the text into JSON object using
JsonSlurperClassic
def jsonObject = new JsonSlurperClassic().parseText(jsonText)
// Extracting the details of all the stages present in that particular build number
for (int i=0; i<jsonObject.stages.size()-1; i++){ //size-1 to ignore the post stage
//populating the field type columns of InfluxDB measurements and pushing them to the map called myDataField1
def size = jsonObject.stages.size()-1
myDataField1['result'] = jsonObject.stages[i].status
myDataField1['duration'] =
jsonObject.stages[i].durationMillis
myDataField1['stage_name'] = jsonObject.stages[i].name
//populating the tag type columns of InfluxDB
measurements and pushing them to the map called
myDataTag1
myDataTag1['result_tag'] = jsonObject.stages[i].status
myDataTag1['stage_name_tag'] = jsonObject.stages[i].name
//assigning field type columns to the measurement called CustomData
myCustomDataFields1['CustomData'] = myDataField1
//assigning tag type columns to the measurement called CustomData
myCustomDataTags1['CustomData'] = myDataTag1
//Push the data into influx instance
try
{ step([$class: 'InfluxDbPublisher', target: 'jenkins_data', customPrefix: null, customDataMapTags: myCustomDataTags1]) }
catch (err)
{ println ("pushStagData exception: " + err) }
}
}
Expected: I want to push each stages of jenkins pipeline data to influx db.
Try to use jenkins' standard methods instead of creating objects manually. You can use httpRequest instead of URL. This should fix exception you are getting. You can also use readJson instead of JsonSlurperClassic (latter is optional since classic slurper is actually serializable).
you just need to explicitly set get to null to make sure there's none non serializable object being instantiated.
def jsonText = get.getInputStream().getText();
get = null;