I'm pretty new to Jenkins and trying to run or be able to run multiple jenkins jobs in parallel with different goals (inside my goals will have a mvn command, also cucumber #tags). Basically I want to run multiple cucumber tags on multiple jenkins jobs at the same time. From the research I've done so far looks like I have a few options - multijob or pipeline plugins..please advice. Thanks!
If you don't worry about any resource consumption, you can use separated jobs as below and also you can hook them each other or trigger them with one job via Jenkins features:
Example job configurations :
1. job :
mvn clean test -Dcucumber.options="--tags #Smoke"
2. job :
mvn clean test -Dcucumber.options="--tags #Regression"
Related
Is there a way to run Jenkins build step in parallel.
I know I can run multiple jobs in parallel, but I need to collate the unit test result and code coverage and use it generate a report.
So jobs in parallel might not be achieve the purpose, so I need to run build steps in parallel.
Any ideas?
Sounds like a job for the Jenkins Multijob plugin. You can create 2 phases:
Phase 1 : Builds jobs which run in parallel
Phase 2 : Collate results and generate report. You can specify this job to run only after the first phase is complete, and only if its successful etc.
I'm new to Jenkins. I have a requirement where I need to run part of a job on the Master node and the rest on a slave node.
I tried searching on forums but couldn't find anything related to that. Is it possible to do this?
If not, I'll have to break it into two separate jobs.
EDIT
Basically I have a job that checks out source code from svn, then compiles and builds jar files. After that it's building a wise installer for this application. I'd like to do source code checkout and compilation on the master(Linux) and delegate Wise Installer setup to a Windows slave.
It's definitely easier to do this with two separate jobs; you can make the master job trigger the slave job (or vice versa).
If you publish the files that need to be bundled into the installer as build artifacts from the master build, you can pull them onto the slave via a Jenkins URL and create the installer. Use the "Archive artifacts" post build step in the master build to do this.
The Pipeline Plugin allows you to write jobs that run on multiple slave nodes. You don't even have to go create other separate jobs in Jenkins -- just write another node statement in the Pipeline script and that block will just run on an assigned node. You can specify labels if you want to restrict the type of node it runs on.
For example, this Pipeline script will execute parts of it on two different nodes:
node('linux') {
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
sh "make"
step([$class: 'ArtifactArchiver', artifacts: 'build/program', fingerprint: true])
}
node('windows && amd64') {
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
sh "mytest.exe"
}
Some more information at the Pipeline plugin tutorial. (Note that it was previously called the Workflow Plugin.)
You can use the Multijob plugin which adds an the idea of a build phase which runs other jobs in parallel as a build step. You can still continue to use the regular freestyle job build and post build options as well
I started using Jenkins in my project and I am trying to parallelize my test suite (Rspec test cases) written in 4 files in Jenkins
spec/features/
|-- test1.rb
|-- test2.rb
|-- test3.rb
|-- test4.rb
We can run all test cases with below command, it will run all tests written in test1.rb ..test4.rb sequentially which will take around 1 hour.
script spec/features/
If you want to excute test cases from each test file we can run like
script spec/features/test1.rb
Now I want to parallelize these test cases which can reduce the run from 1hr to 15 mins, All these test cases can run in one machine in parallel
I followed below approach in Jenkins
1) Set a new job "Main_Test_job"
2)
Selected "Trigger/Call builds on other projects"
projects to build " Child_test_job"
Build on same node
Predefined Parameters TEST_UNIT=test1.rb
Block until the triggered projects finish their builds ( Not selected this)
Add trigger --->
Selected "Trigger/Call builds on other projects"
projects to build " Child_test_job"
Build on same node
Predefined Parameters TEST_UNIT=test2.rb
Block until the triggered projects finish their builds ( Not selected this)
Add trigger --->
Selected "Trigger/Call builds on other projects"
projects to build " Child_test_job"
Build on same node
Predefined Parameters TEST_UNIT=test3.rb
Block until the triggered projects finish their builds ( Not selected this)
Add trigger --->
Selected "Trigger/Call builds on other projects"
projects to build " Child_test_job"
Build on same node
Predefined Parameters TEST_UNIT=test4.rb
Block until the triggered projects finish their builds ( Not selected this)
3)
Created job "Child_test_job" as which was included in main_test_job like below
Select Build step "Execute Shell" with below command
script spec/$TEST_UNIT
When I start "Main_Test_job", it will automatically start 4 Child_Test_Jobs in same machine, which will reduce my total run time to 15 mins.
But in this case "Main_test_job" has no way to monitor statuses of 4
child_test_jobs, It always succeeds immediately after starting 4
child jobs
"Block until the triggered projects finish their builds" this option
monitors child jobs but if we select this option for all child jobs,
they are running sequentially instead of parallel.
I can't use join plugin as I am not running different jobs instead triggering same job multiple times.
My Ideas:
have separate jobs for each test.rb and use join trigger to monitor
statuses of all jobs
have some shell script as the post-build task of "Main_Test_job"
which will aggregate/monitor statuses/results of each jobs.
I think this must be a common scenario in many organizations and there must be a easy way in Jenkins to achieve this.
Please let me know your approaches/ideas. May be I am missing some thing here.
If your jobs can run in parallel on same machine then Multijob plugin might be of interest to you. It starts the jobs in parallel but waits till all of them finish.
You can also use Build Flow Plugin
You can run any type of jobs by using this plugin.
I'm trying to set up Jenkins as follows:
Test Job --> (Test Job 1 & Test Job 2 in parallel) --> Test Job 3 --> Test Job 4
I have this working at present using the Join Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin) and Build Pipeline Plugin.
However the display on the Build Pipeline unnecessarily 2 x Test Job 3s and 2 x Test Job 4s after the join, see below:
Set up for each job is as follows:
Test Job:
Test Job 1 & 2:
Test Job 3:
Test Job 4:
I would like to remove the "blue" versions of Test Job 3 and Test Job 4 from my Build Pipeline after the two parallel processes finish.
Anybody able to help me to remove these?
Cheers
Try with Build Flow plugin
It will do both parallel and sequential jobs.
I recommend using the Multijob Plugin alone without the Build Pipeline Plugin.
The Multijob Plugin gives you the functionality of the Join Plugin, and its configuration is more straightforward. I actually prefer how it displays my running build.
You can put a multijob into a build pipeline, but the placement of the jobs within the pipeline is wrong The jobs within the multijob are displayed in vertically in alphabetical order (not build order). On the positive side, everything else seems to work, so this should be easy to fix. I reported this problem as Jenkins bug 22074.
the 'Jenkins - Build Pipeline' plugin support customize css , maybe you could make it unvisable by css
You can use build pipeline plugin together with Multijob plugin. Just use Multijob plugin as a substitute for Join plugin. Basically, Multijob plugin will only be used to make certain jobs to be executed simultaneously.
If you do this way, then build Pipeline view won't get screwed up.
This is how it looks in Pipeline Build view
build-bv-docker-images is a Multijob plugin Job.
build-(activemq|postgres|tomcat|wildfly)-bv_image are simple jobs used for building docker images
deploy-staging is a job, which is triggered after build-bv-docker-images job. Logically speaking it suppose to appear right after stack of build-*-bv-images jobs, but it appears as a part of this stack. Nevertheless, it's waiting until all jobs of this stack are completed. I had to prefix deploy-staging job with + sign in order to make it appear on the top of the stack. It looks awkward, but it's still better than to see deploy-staging job in the bottom of the stack.
This is how build-bv-docker-images multijob is configured
I am using Jenkins to run a bunch of my scripts on regular basis. And the Execute shell section of my job looks like:
runner.py my_script A.config run
The problem is I have a bunch of configs A.config, B.config .... S.config. I am currently creating separate jobs for each config. And have a bunch of jobs. Is there any plugin you would recommend so that I can just have "runner.py my_script" and pass in A.config or B.config... using a simpler option than having a bunch of job like I have right now?
You can a achieve this with the multi configuration job in Jenkins. Select it as the radio button entry when naming the job
Here is more information Jenkins and multi-configuration (matrix) jobs
It is a standard Jenkins feature so doesn't need a plugin and you can add your own labels in.