Jenkins matrix job not triggering multiple downstream jobs - jenkins

I've configured a Jenkins matrix job that has a total of 4 axis points. For each axis there is a single build step (parameterized-trigger) triggering another project adding the current build parameters. All four jobs are triggering the same job with different parameters.
The problem I am seeing is only one drown stream job gets executed, I would expect 4.
This is NOT a post-build task, it is setup as a build step. This is the description in the Parameterized Trigger Plugin wiki page:
Build step
When using the trigger parameterized build as a buildstep
it will be called for every different configuration, so if triggering
another project with no parameters it will be done the same number of
times as you have configurations, possible causing the triggered job
to run more than once.
However this also allows you to trigger other jobs with parameters
relating to the current configuration, i.e. triggering a build on the
same node with the same JDK.
This is the log entry for the downstream job, it shows all for triggers, but they all seem to be launching the same exact job:
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x/ProductType=video,SecurityType=standard" build number 12
originally caused by:
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x" build number 12
originally caused by:
Started by user anonymous
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x/ProductType=video,SecurityType=secure" build number 12
originally caused by:
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x" build number 12
originally caused by:
Started by user anonymous
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x/ProductType=voice,SecurityType=standard" build number 12
originally caused by:
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x" build number 12
originally caused by:
Started by user anonymous
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x/ProductType=voice,SecurityType=secure" build number 12
originally caused by:
Started by upstream project "AndroidLibraries_Mx_Branch_5_1_x" build number 12
originally caused by:
Started by user anonymous
[Pipeline] node
Running on master in /var/lib/jenkins/workspace/AndroidLibrary_pipeline
<... job details ... >
If it makes any difference, the downstream job is a pipeline job as noted by the name.
Jenkins 2.19.2
Matrix Plugin 1.7.1
Parameterized Trigger Plugin 2.32
Pipeline 2.4
I've done a good bit of searching, and while I found a similar issue, the accepted answer does not help.

I figured it out. The parameters were not being passed correctly to the child job. I had the "Current build parameters" option set in the trigger parameterized job set, but I had to add "Predefined parameters" also, adding the parameters that the child job was expecting.
SecurityType=${SecurityType}
ProductType=${ProuductType}

Related

JMeter & Jenkins - passing jmeter parameters to downstream build

The Setup - A jenkins job using jenkins parameters testApp and testEnv. The Execution Batch looks like this:
C:\jmeter\apache-jmeter-3.2\bin\jmeter.bat -n -t
C:\JMeter\Scripts\API_scripts\%testApp%.jmx -Jtestenv=%testEnv% -JtestApp=%testApp% -JtestBrowser=NA -l
C:\AUTO_Results\jtl\%testApp%_%testEnv%.jtl
Post-build Actions
Console output (build lob) parsing with a global rule so that the Failures that are logged in the Jenkins Console window will consider the JMeter script failing. (discussed Jenkins shows JMeter script failure even though the script actually passed)
Triggered parameterized build - this is a separate jmeter script that updates a wiki page with either PASS/FAIL and uploads the JMeter report.
The Issue - How do I get the downstream Triggered build to use the parameters from the upstream script? I set the Parameter = Current build parameters but it's not applying those. Also, I wont know the value of the testResult parameter until the upstream build finishes. I tried adding %testResult%=PASS to the 'Predefined parameters' box
As per Parameterized Trigger Plugin page:
The parameters section can contain a combination of one or more of the following:
a set of predefined properties
properties from a properties file read from the workspace of the triggering build
the parameters of the current build
Subversion revision: makes sure the triggered projects are built with the same revision(s) of the triggering build. You still have to make sure those projects are actually configured to checkout the right Subversion URLs.
Restrict matrix execution to a subset: allows you to specify the same combination filter expression as you use in the matrix project configuration and further restricts the subset of the downstream matrix builds to be run.
So you basically need to copy over the parameters you would like to have in the "downstream" job from the current one.
As a workaround to current performance plugin limitations you can consider running JMeter using Taurus tool as a wrapper, it has flexible and powerful pass/fail criteria subsystem which will basically return to Jenkins non-zero exit code triggering build failure in case of issue in the test. If everything goes well Taurus exit code will be 0 which is considered successful by Jenkins. Check out How to Run Taurus with the Jenkins Performance Plugin article for more details.

How to use a different set of parameters for release builds in jobs triggered via parameterized trigger plugin

I have a set of jobs that run shell scripts. Parameters for those scripts can be chosen via some choices defined in the build.
Now i want to use the release plugin to prevent people from accidentally choosing from a set of "release only" parameters.
So what i basically need is the ability to have one build with two distinct sets of parameter choices.
To achieve this i have configured the jobs as follows:
Master Job
normal build
- choiceParameter name:TEST values:"normal"
release build
- choiceParameter name:TEST values:"release"
Child Job
normal build
- choiceParameter name:TEST values:"normal"
release build
- choiceParameter name:TEST values:"release"
MasterJob triggers ChildJob via "Parameterized Build" plugin
When i execute a normal build everything works fine.
But when i trigger a Release Build on the MasterJob i get the following exception:
ERROR: Build step failed with exception
java.lang.IllegalArgumentException: Illegal choice for parameter TEST: release
at hudson.model.ChoiceParameterDefinition.checkValue(ChoiceParameterDefinition.java:75)
at hudson.model.ChoiceParameterDefinition.createValue(ChoiceParameterDefinition.java:87)
at hudson.model.ChoiceParameterDefinition.createValue(ChoiceParameterDefinition.java:19)
at hudson.plugins.parameterizedtrigger.ProjectSpecificParameterValuesActionTransform.convertToDefinedType(ProjectSpecificParameterValuesActionTransform.java:83)
at hudson.plugins.parameterizedtrigger.ProjectSpecificParameterValuesActionTransform.transformParametersAction(ProjectSpecificParameterValuesActionTransform.java:34)
at hudson.plugins.parameterizedtrigger.ProjectSpecificParametersActionFactory.getProjectSpecificBuildActions(ProjectSpecificParametersActionFactory.java:32)
at hudson.plugins.parameterizedtrigger.BuildTriggerConfig.getBuildActions(BuildTriggerConfig.java:290)
at hudson.plugins.parameterizedtrigger.BuildTriggerConfig.perform2(BuildTriggerConfig.java:336)
at hudson.plugins.parameterizedtrigger.BlockableBuildTriggerConfig.perform2(BlockableBuildTriggerConfig.java:57)
at hudson.plugins.parameterizedtrigger.TriggerBuilder.perform(TriggerBuilder.java:85)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:761)
at hudson.model.Build$BuildExecution.build(Build.java:203)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:536)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:374)
Build step 'Trigger/call builds on other projects' marked build as failure
Finished: FAILURE
Fixing this error is easy. I just have to add the value "release" to the choices in the normal build. But this destroys the whole intention of this setup.
Is there a way to get this kind of setup to work?
If you want people to restrict running arbitrary script on production boxes, You can use Node label plugin.
You can configure the job to select which node(box/machine) user can run the job, This way you can restrict user running jobs on prod env.

Unifying Jenkins console logs

We have a nightly build of our projects and each of them has several downstream jobs, and I configured Jenkins to send the build logs when the main job ends. The problem is that the main console only shows the details of its own job, and from the downstream jobs it shows this:
Waiting for the completion of "Project BBB"
"Project BBB" #41 completed. Result was SUCCESS
Is there any way I could make it show the details of the downstream jobs?
Started by upstream project "Project AAA" build number 38
originally caused by:
Started by user "User Name"
Building in workspace C:\WORKSPACE\Project BBB
Reverting C:\WORKSPACE\Project BBB\. to depth infinity with ignoreExternals: true
Updating https://111.111.111.111/svn/Project BBB at revision '2014-10-10T11:00:11.785 +0100'
At revision 43143
no change for https://111.111.111.111/svn/Project BBB since the previous build
Notifying upstream projects of job completion
Finished: SUCCESS
Or at least send the downstream jobs build.log also?
The build log of each completed job is available in plain text:
http://<jenkins_server>/job/<job_name>/<job_number>/consoleText
In your downstream job, you can pull this log into a file, and simply cat it out in the downstream's console log.

promotion after completion of non-downstream job

My question is somewhat continuation of Cloudbees jenkins does not trigger jobs on promotion. To recap: I am using jenking on cloudbees. I have a job named "package" with a couple of promotion processes. The first promotion process "promote to e2etesting" is configured as: promote after job named "testing" is successfully run. "Testing" is configured as downstream job of "package" job (build other project of post build action section). This promotion works nicely and as soon as "promote to e2etesting" is performed a new job named "e2e-testing" is fired.
My "package" job has other promotion processes, one of them being named "promotion-to-staging" with criteria being : "when the following downstream projects build successfully: e2e-testing". Unfortunately after successful completion of "e2e-testing" (triggered by "promote to e2etesting"), "promotion-to-staging" does not happen.
On "e2e-testing" job console output i can see:
WARNING: package appears to use this job as a promotion criteria, but no fingerprint is
recorded. Fingerprint needs to be enabled on both this job and package.
See http://hudson.gotdns.com/wiki/display/HUDSON/Fingerprint for more details
I have no idea if the warning is related to my problem but on the other side it is weird since i have "record fingerprint of files ..." enabled for "e2e-testing job", and "package" job (being maven job) seems to have it enabled out-of-box - as i click on "passport icon" on artifact built by "package" job i can see it utilization among other jobs.
In jenkins logs i can see:
SEVERE: I/O error in channel s-74638b6e
java.io.IOException: Unexpected termination of the channel at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)
May 24, 2013 3:43:31 AM hudson.model.Run execute
INFO: e2e-testing #10 main build action completed: SUCCESS
May 24, 2013 3:43:19 AM hudson.slaves.WorkspaceList log
FINE: Executor #0 for s-74638b6e : executing e2e-testing #10 acquired /scratch/jenkins/workspace/e2e-testing
May 24, 2013 3:43:14 AM hudson.model.Run execute
INFO: package » promotion » promotion-to-e2e-testing #10 main build action completed: SUCCESS
Any idea?
e2e-testing need not be a configured downstream job of package (i.e. kicked off directly as a post-build action), but there need to be a downstream relationship between the builds as identified by fingerprints. In other words, some (successful) build of e2e-testing must record a fingerprint of an input artifact which matches the fingerprint of an artifact produced by some build of package. When that happens, promotion-to-staging should be triggered on that build of package.
Given the warning message you saw, I suspect that e2e-testing either does not record fingerprints, or is recording fingerprints but none matching an artifact produced by package. You should be able to check this by browsing the Fingerprints link for a successful e2e-testing build, verifying that the expected artifact is listed, then clicking on details for that artifact and verifying that it is also listed as being “produced” by a package build.

Cloudbees jenkins does not trigger jobs on promotion

I am trying to follow one of Koshuke's apporaches to organize my builds by using job-promotion on cludbees dev#cloud. In his presentation everything worked as a charm (beside running his examples from his own Jenkins instance deployed on localhost and me running jenkins jobs in dev#cloud).
Basically i have a couple of jobs and my main job called package within folder joy defines promotion process named "promotion-to-e2e-testing" (with criteria being completion of one of downstream jobs and no extra action defined for promotion). From the jenkins dashboard i can see that this promotion was successful. Nevertheless a job (called e2e-testing within folder joy) being configured with trigger "Build when another project is promoted : Job name: "joy/package", Promotion: "promotion-to-e2e-testing" is not fired !
I have looked at jenkins systems logs (via cloudbees Manage Jenkins link) and i can see:
May 20, 2013 6:04:33 AM hudson.remoting.SynchronousCommandTransport$ReaderThread run SEVERE: I/O error in channel s-8770fc61
java.io.IOException: Unexpected termination of the channel
athudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50) Caused by: java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at hudson.remoting.Command.readFrom(Command.java:92)
at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)May 20, 2013 6:04:26 AM hudson.model.Run execute
INFO: joy » package » promotion » promotion-to-e2e-testing #5 main build action completed: SUCCESS
May 20, 2013 6:04:26 AM hudson.slaves.WorkspaceList log
FINE: Executor #0 for s-8770fc61 : executing joy » package » promotion » promotion-to- e2e-testing #5 acquired /scratch/jenkins/workspace/joy/package
May 20, 2013 6:04:25 AM hudson.slaves.ChannelPinger setUpPingForChannel
By reading from the bottom to the top it seems that SynchronousCommandTransport is thrown after successful promotion. To make sure that throwing the exception is not volatile i have run my jobs a couple of times but still see the same exceptions in logs and my e2e-testing job is not fired.
Anyone can help me with that ? Maybe triggering a jobs by promotion is not available on cloudbees? Or maybe it is because my jobs are stored in a jenkins folder (all jobs being in a single folder) ?
configured with trigger Build when another project is promoted: Job name: joy/package
I suspect you have hit one of the symptoms of JENKINS-17955. If my hypothesis is correct, joy » package will work as the upstream name even though joy/package is really what the plugin ought to be expecting. But I have not yet dug further and tried to reproduce and fix it.

Resources