Original situation:
I have a job in Jenkins that is running an ant script. I easily managed to test this ant script on more then one software version using a "Multi-configuration project".
This type of project is really cool because it allows me to specify all the versions of the two software that I need (in my case Java and Matlab) an it will run my ant script with all the combinations of my parameters.
Those parameters are then used as string to be concatenated in the definition of the location of the executable to be used by my ant.
example: env.MATLAB_EXE=/usr/local/MATLAB/${MATLAB_VERSION}/bin/matlab
This is working perfectly but now I am migrating this scripts to a pipline version of it.
Pipeline migration:
I managed to implement the same script in a pipeline fashion using the Parametrized pipelines pluin. With this I achieve the point in which I can manually select which version of my software is going to be used if I trigger the build manually and I also found a way to execute this periodically selecting the parameter I want at each run.
This solution seems fairly working however is not really satisfying.
My multi-config project had some feature that this does not:
With more then one parameter I can set to interpolate them and execute each combination
The executions are clearly separated and in build history/build details is easy to recognize which settings hads been used
Just adding a new "possible" value to the parameter is going to spawn the desired executions
Request
So I wonder if there is a better solution to my problem that can satisfy also the point above.
Long story short: is there a way to implement a multi-configuration project in jenkins but using the pipeline technology?
I've seen this and similar questions asked a lot lately, so it seemed that it would be a fun exercise to work this out...
A matrix/multi-config job, visualized in code, would really just be a few nested for loops, one for each axis of parameters.
You could build something fairly simple with some hard coded for loops to loop over a few lists. Or you can get more complicated and do some recursive looping so you don't have to hard code the specific loops.
DISCLAIMER: I do ops much more than I write code. I am also very new to groovy, so this can probably be done more cleanly, and there are probably a lot of groovier things that could be done, but this gets the job done, anyway.
With a little work, this matrixBuilder could be wrapped up in a class so you could pass in a task closure and the axis list and get the task map back. Stick it in a shared library and use it anywhere. It should be pretty easy to add some of the other features from the multiconfiguration jobs, such as filters.
This attempt uses a recursive matrixBuilder function to work through any number of parameter axes and build all the combinations. Then it executes them in parallel (obviously depending on node availability).
/*
All the config axes are defined here
Add as many lists of axes in the axisList as you need.
All combinations will be built
*/
def axisList = [
["ubuntu","rhel","windows","osx"], //agents
["jdk6","jdk7","jdk8"], //tools
["banana","apple","orange","pineapple"] //fruit
]
def tasks = [:]
def comboBuilder
def comboEntry = []
def task = {
// builds and returns the task for each combination
/* Map the entries back to a more readable format
the index will correspond to the position of this axis in axisList[] */
def myAgent = it[0]
def myJdk = it[1]
def myFruit = it[2]
return {
// This is where the important work happens for each combination
node(myAgent) {
println "Executing combination ${it.join('-')}"
def javaHome = tool myJdk
println "Node=${env.NODE_NAME}"
println "Java=${javaHome}"
}
//We won't declare a specific agent this part
node {
println "fruit=${myFruit}"
}
}
}
/*
This is where the magic happens
recursively work through the axisList and build all combinations
*/
comboBuilder = { def axes, int level ->
for ( entry in axes[0] ) {
comboEntry[level] = entry
if (axes.size() > 1 ) {
comboBuilder(axes[1..-1], level + 1)
}
else {
tasks[comboEntry.join("-")] = task(comboEntry.collect())
}
}
}
stage ("Setup") {
node {
println "Initial Setup"
}
}
stage ("Setup Combinations") {
node {
comboBuilder(axisList, 0)
}
}
stage ("Multiconfiguration Parallel Tasks") {
//Run the tasks in parallel
parallel tasks
}
stage("The End") {
node {
echo "That's all folks"
}
}
You can see a more detailed flow of the job at http://localhost:8080/job/multi-configPipeline/[build]/flowGraphTable/ (available under the Pipeline Steps link on the build page.
EDIT:
You can move the stage down into the "task" creation and then see the details of each stage more clearly, but not in a neat matrix like the multi-config job.
...
return {
// This is where the important work happens for each combination
stage ("${it.join('-')}--build") {
node(myAgent) {
println "Executing combination ${it.join('-')}"
def javaHome = tool myJdk
println "Node=${env.NODE_NAME}"
println "Java=${javaHome}"
}
//Node irrelevant for this part
node {
println "fruit=${myFruit}"
}
}
}
...
Or you could wrap each node with their own stage for even more detail.
As I did this, I noticed a bug in my previous code (fixed above now). I was passing the comboEntry reference to the task. I should have sent a copy, because, while the names of the stages were correct, when it actually executed them, the values were, of course, all the last entry encountered. So I changed it to tasks[comboEntry.join("-")] = task(comboEntry.collect()).
I noticed that you can leave the original stage ("Multiconfiguration Parallel Tasks") {} around the execution of the parallel tasks. Technically now you have nested stages. I'm not sure how Jenkins is supposed to handle that, but it doesn't complain. However, the 'parent' stage timing is not inclusive of the parallel stages timing.
I also noticed is that when a new build starts to run, on the "Stage View" of the job, all the previous builds disappear, presumably because the stage names don't all match up. But after the build finishes running, they all match again and the old builds show up again.
And finally, Blue Ocean doesn't seem to vizualize this the same way. It doesn't recognize the "stages" in the parallel processes, only the enclosing stage (if it is present), or "Parallel" if it isn't. And then only shows the individual parallel processes, not the stages within.
Points 1 and 3 are not completely clear to me, but I suspect you just want to use “scripted” rather than “Declarative” Pipeline syntax, in which case you can make your job do whatever you like—anything permitted by matrix project axes and axis filters and much more, including parallel execution. Declarative syntax trades off syntactic simplicity (and friendliness to “round-trip” editing tools and “linters”) for flexibility.
Point 2 is about visualization of the result, rather than execution per se. While this is a complex topic, the usual concrete request which is not already supported by existing visualizations like Blue Ocean is to be able to see test results distinguished by axis combination. This is tracked by JENKINS-27395 and some related issues, with design in progress.
Related
I want to trigger Jenkins job on "hyp-z" and "hyp-x" build nodes. I tried to write it this way but getting "There are no nodes with the label ‘hyp-x&&hyp-z’"
node ('hyp-z&&hyp-x') {
// write something here
}
What is the mistake I am doing and what is the exact working format?
This isn't possible in this form.
The && expression is for narrowing down your pool of nodes for certain features.
e.g. I want to run on a node which has the label UBUNTU and DOCKER.
As opposed to running on two different nodes with those labels.
You can use parallel block to do what you are wanting.
If you are using the Declarative syntax then see this article
https://jenkins.io/blog/2017/09/25/declarative-1/
or here for scripted
https://jenkins.io/doc/book/pipeline/jenkinsfile/#parallel-execution
I have tried this in two ways
def labels = ["hyp-x", "hyp-z"]
def builders = [:]
for (x in labels) {
def label = x
builders[label] = {
node(label) {
// build script
}
}
}
parallel builders
Above code is working as expected from Jenkinsfile but I see both builds triggered in one common job and log looks clumsy. So I have tried 2nd approach like below
Created Jenkinsfile.x and Jenkinsfile.z and each file represents x and z platform build.
I am looking for a generic way to determine the name of the failed stage at the end of a Jenkins scripted Pipeline.
Please note that this is different than Determine Failed Stage in Jenkins Declaritive Pipeline which is about declarative pipeline.
Also note that use of try/catch inside each stage is out of question because it would make the pipeline script impossible to read. This is because we have like 10-15 stages which are stored in multiple files and they are compiled using JJB to create the final pipeline script. They are already complex so I need a clean approach on finding which stage failed.
U could also create a custom step in a shared library, a super_stage
Quick example:
// vars/super_stage.groovy
def call(name, body) {
try {
stage(name) {
body()
}
} catch(e) {
register_failed_stage(name, e)
throw e
}
}
In that way you can 'reuse' the same exception handler.
In your scripted pipeline you would then use it like:
super_stage("mystage01") {
//do stuff
}
Source
Use a GraphListener:
def listener = new GraphListener.Synchronous(){
#NonCPS
void onNewHead(FlowNode node){
if (node instanceof StepStartNode){
// before steps execution
}else if (node instanceof StepEndNode){
// after steps execution
}
}
def execution = (FlowExecution) currentBuild.getRawBuild().getExecution()
execution.addListener(listener)
You are going to need a few helper functions in order to make it work, for example StepStartNode and StepEndNode gets called twice so you have to filter the one with the label. Also variables like env are available inside the listener so you can store anything in there to be picked up later.
This answer is pretty generic but I've found that is useful in many of the stackoverflow questions regarding doing something before/after some stage (or in all).
You cannot try/catch exceptions inside the pipeline as this approach is not a wrapper for the stage but just a listener that gets executed once per each line instruction but you can just record the stage at the begining and at the end check currentBuild.result to see if the stage failed. You can do pretty much anything at this point.
At some point with the FlowExecution you have access to the pipeline script, I don't know if it's writtable at that point but it would be awesome to rewrite the pipeline to actually try/catch the stages. If you do something in this line please let me know ;)
Through experimentation I have determined that I can mask the built-in pipeline steps such as build by defining a global function of the same name in shared library.
Example:
(root)
+- vars
+- build.groovy
where build.groovy is:
def call(Map args) {
echo "BUILD: ${args}"
}
If I load this library, then none of my calls to build actually do anything. They just echo that build was called and with what args. This is very useful for testing out pipeline scripts to ensure the script logic itself is correct while avoiding actually doing long running tasks.
But testing is only one use of this. What I really want to do is decorate build, node, stage and a few other steps to capture usage metrics. For example to record for every node that is ever allocated, what time of day it was allocated, and how long it was allocated for. This could be really useful for capacity analysis and planning.
Another application would be to enforce certain policies, such that nodes always be allocated by label and never by explicit node name.
To make any of this work though, the node.groovy decorator needs some way to invoke the real node step that it is masking. Any ideas how to do this?
Figured this out this evening. All the dsl steps are available as members of the steps variable. Which allows me to write something like:
#Library('pipeline-utils')
import mycompany.analytics.AnalyticsClient
import mycompany.analytics.Utils
node('linux') { sh 'echo test' }
def node(String label, Closure nodeAction) {
def executionTime
def actualNode
def allocationTime = Utils.startMeasureDuration()
steps.node(label){
allocationTime.stop()
actualNode = env.NODE_NAME
executionTime = Utils.measureDuration(nodeAction)
}
def fact = [
type: 'node_usage',
job_name: currentBuild.getProjectName(),
node_label: label,
node_name: actualNode,
ts: allocationTime.startTS,
time_in_queue: allocationTime.durationMillis,
execution_time: executionTime.durationMillis
]
AnalyticsClient.recordFact(fact)
if(!executionTime.success) throw executionTime.exception
}
I have a requirement to run a set of tasks for a build in parallel, The tasks for the build are dynamic it may change. I need some help in the implementation of that below are the details of it.
I tasks details for a build will be generated dynamically in an xml which will have information of which tasks has to be executed in parallel/serial
example:
say there is a build A.
Which had below task and the order of execution , first task 1 has to be executed next task2 and task3 will be executed in parallel and next is task 4
task1
task2,task3
task4
These details will be in an xml dynamically generated , how can i parse that xml and schedule task accordingly using pipeline plugin. I need some idea to start of with.
You can use Groovy to read the file from the workspace (readFile) and then generate the map containing the different closures, similar to the following:
parallel(
task2: {
node {
unstash('my-workspace')
sh('...')
}
},
task3: {
node {
unstash('my-workspace')
sh('...')
}
}
}
In order to generate such data structure, you simply iterate over the task data read using XML parsing in Groovy over the file contents you read previously.
By occasion, I gave a talk about pipelines yesterday and included very similar example (presentation, slide 34ff.). In contrast, I read the list of "tasks" from another command output. The complete code can be found here (I avoid pasting all of this here and instead refer to this off-site resource).
The kind of magic bit is the following:
def parallelConverge(ArrayList<String> instanceNames) {
def parallelNodes = [:]
for (int i = 0; i < instanceNames.size(); i++) {
def instanceName = instanceNames.get(i)
parallelNodes[instanceName] = this.getNodeForInstance(instanceName)
}
parallel parallelNodes
}
def Closure getNodeForInstance(String instanceName) {
return {
// this node (one per instance) is later executed in parallel
node {
// restore workspace
unstash('my-workspace')
sh('kitchen test --destroy always ' + instanceName)
}
}
}
We're considering using the Jenkins Pipeline plugin for a rather complex project consisting of several deliveries that need to be build using different tools (on different machines) before being merged. Still, it seems to be easy enough to do a complete build with a single Jenkinsfile, and I like the automatic discovery of git branches that comes with Pipeline.
However, at this point, we have jobs for each of the deliveries and use a build-flow based "meta" job to orchestrate the individual jobs. The nice thing about this is that it also allows starting just one individual job if only small changes were made, just to see whether this delivery still compiles.
To emulate this, some ideas came to mind:
Use different Jenkinsfiles for the deliveries and load them in the top-level Jenkinsfile; it seems that the Multibranch Pipeline job does not allow configuring the Jenkinsfile to use yet (https://issues.jenkins-ci.org/browse/JENKINS-35415), however, so creating the jobs for the individual deliveries is still open.
Provide a configuration option for the "top-level" job and have ifs for all deliveries in the Jenkinsfile to be able to select which should be build. This would mix different build types in one pipeline, though, and, at the very least, mess up the estimation of the build time.
Are those viable options, or is there a better one?
What you could do is to write a pipelining script that has has "if"-guards around the single stages, like this:
stage "s1"
if (theStage in ["s1","all"]) {
sleep 2
}
stage "s2"
if (theStage in ["s2", "all"]) {
sleep 2
}
stage "s3"
if (theStage in ["s3", "all"]) {
sleep 2
}
Then you can make a "main" job that uses this script and runs all stages at once by setting the parameter "theStage" to "all". This job will collect the statistics when all stages are run at once and give you useful estimation times.
Furthermore, you can make a "partial run" job that uses this script and that is parametrized with the stage that you want to run. The estimation will not be very useful, though.
Note that I put the stage itself to the main script and put only the execution code into the conditional, as suggested by Martin Ba. This makes sure that the visualization of the job is more reliable
As an expansion of the previous answer, I would propose something like that:
def stageIf(String name, Closure body) {
if (params.firstStage <= name && params.lastStage >= name) {
stage(name, body)
} else {
stage(name) {
echo "Stage skipped: $name"
}
}
}
node('linux') {
properties([
parameters([
choiceParam(
name: 'firstStage',
choices: '1.Build\n' +
'2.Docker\n' +
'3.Deploy',
description: 'First stage to start',
defaultValue: '1.Build',
),
choiceParam(
name: 'lastStage',
choices: '3.Deploy\n' +
'2.Docker\n' +
'1.Build',
description: 'Last stage to start',
defaultValue: '3.Deploy',
),
])
])
stageIf('1.Build') {
// ...
}
stageIf('3.Deploy') {
// ...
}
}
Not as perfect as I wish but at least its working.