I am using Jenkins to run a bunch of my scripts on regular basis. And the Execute shell section of my job looks like:
runner.py my_script A.config run
The problem is I have a bunch of configs A.config, B.config .... S.config. I am currently creating separate jobs for each config. And have a bunch of jobs. Is there any plugin you would recommend so that I can just have "runner.py my_script" and pass in A.config or B.config... using a simpler option than having a bunch of job like I have right now?
You can a achieve this with the multi configuration job in Jenkins. Select it as the radio button entry when naming the job
Here is more information Jenkins and multi-configuration (matrix) jobs
It is a standard Jenkins feature so doesn't need a plugin and you can add your own labels in.
Related
I'm pretty new to Jenkins and trying to run or be able to run multiple jenkins jobs in parallel with different goals (inside my goals will have a mvn command, also cucumber #tags). Basically I want to run multiple cucumber tags on multiple jenkins jobs at the same time. From the research I've done so far looks like I have a few options - multijob or pipeline plugins..please advice. Thanks!
If you don't worry about any resource consumption, you can use separated jobs as below and also you can hook them each other or trigger them with one job via Jenkins features:
Example job configurations :
1. job :
mvn clean test -Dcucumber.options="--tags #Smoke"
2. job :
mvn clean test -Dcucumber.options="--tags #Regression"
What are seed jobs in Jenkins and how does it work ?
Can we create a new job from seed job without using github ?
That depends on context. Jenkins itself does not provide "seed jobs".
There's plugins that allow creating jobs from other jobs, like the excellent Job-DSL plugin. With that, you can create jobs where a groovy script creates a larger number of jobs for you.
The Job-DSL plugin refers to those jobs as "seed jobs" (but they're regular freestyle or pipeline jobs). The Job-DSL plugin does not require a github connection.
The seed job is a normal Jenkins job that runs the Job DSL script; in turn, the script contains instructions that create additional jobs. In short, the seed job is a job that creates more jobs. In this step, you will construct a Job DSL script and incorporate it into a seed job. The Job DSL script that you’ll define will create a single freestyle job that prints a 'Hello World!' message in the job’s console output.
A Job DSL script consists of API methods provided by the Job DSL plugin; you can use these API methods to configure different aspects of a job, such as its type (freestyle versus pipeline jobs), build triggers, build parameters, post-build actions, and so on. You can find all supported methods on the API reference site.
The jobs we used for creating new jobs are called Seed Jobs and this seed job generates new jobs using Jenkins files (using JobDSL plugin).
Here, we disabling this feature (Enable script security for Job DSL scripts)
Jenkins Dashboard→ Manage Jenkins → Configure Global Security
Way to create seed job :
JobDSL scripts for generating new jobs.
Job1.groovy
job("Job1"){
description("First job")
authenticationToken('secret')
label('dynamic')
scm {
github('Asad/jenkins_jobDSL1', 'master')
}
triggers {
gitHubPushTrigger()
}
steps {
shell ('''
echo "test"
''')
}
}
buildPipelineView('project-A') {
title('Project A CI Pipeline')
displayedBuilds(5)
selectedJob('Job1')
showPipelineParameters()
refreshFrequency(60)
}
and create same way others Job2.groovy and so on.
For Jenkins Job DSL documentation:-
Follow https://jenkinsci.github.io/job-dsl-plugin/
Think about a job - what is it actually ?
It is actually just a java/jre object that represents like this
How you generates such job/build ?
Configure Jenkins UI -> rest api to Jenkins url -> Jenkins service receive your call on the relevant endpoint -> calling to the relevant code/method and generate this new job
How Seed job will make it ?
Configure seed job on Jenkins UI only once -> run this seed job - > this code run against the internal Jenkins methods and skip all the manual process describes above
Now, when your code can talk directly to Jenkins code , things are much easier.just update your code on the relevant repo - and you are done
I want to know how can we parameterize build in jenkins. I need to create a jenkins job which will contain 5 sub jobs in it. i need to create a drop down , selelct any of the module and build it. But the script used is different for every sub build? can any1 guide on the same is it possible.
string parameters in Jenkins result in environment variables of the same name.
So, you could write a wrapper script in bash which would look for the environment variables that could be set as a result of the parameterized build (i.e. your 5 sub-jobs) in a series of if-elif statements, and within each one, you would invoke the necessary build script from there.
The build script that you would have Jenkins run would be the wrapper script.
I'm new to Jenkins. I have a requirement where I need to run part of a job on the Master node and the rest on a slave node.
I tried searching on forums but couldn't find anything related to that. Is it possible to do this?
If not, I'll have to break it into two separate jobs.
EDIT
Basically I have a job that checks out source code from svn, then compiles and builds jar files. After that it's building a wise installer for this application. I'd like to do source code checkout and compilation on the master(Linux) and delegate Wise Installer setup to a Windows slave.
It's definitely easier to do this with two separate jobs; you can make the master job trigger the slave job (or vice versa).
If you publish the files that need to be bundled into the installer as build artifacts from the master build, you can pull them onto the slave via a Jenkins URL and create the installer. Use the "Archive artifacts" post build step in the master build to do this.
The Pipeline Plugin allows you to write jobs that run on multiple slave nodes. You don't even have to go create other separate jobs in Jenkins -- just write another node statement in the Pipeline script and that block will just run on an assigned node. You can specify labels if you want to restrict the type of node it runs on.
For example, this Pipeline script will execute parts of it on two different nodes:
node('linux') {
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
sh "make"
step([$class: 'ArtifactArchiver', artifacts: 'build/program', fingerprint: true])
}
node('windows && amd64') {
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
sh "mytest.exe"
}
Some more information at the Pipeline plugin tutorial. (Note that it was previously called the Workflow Plugin.)
You can use the Multijob plugin which adds an the idea of a build phase which runs other jobs in parallel as a build step. You can still continue to use the regular freestyle job build and post build options as well
I'm trying to set up Jenkins as follows:
Test Job --> (Test Job 1 & Test Job 2 in parallel) --> Test Job 3 --> Test Job 4
I have this working at present using the Join Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin) and Build Pipeline Plugin.
However the display on the Build Pipeline unnecessarily 2 x Test Job 3s and 2 x Test Job 4s after the join, see below:
Set up for each job is as follows:
Test Job:
Test Job 1 & 2:
Test Job 3:
Test Job 4:
I would like to remove the "blue" versions of Test Job 3 and Test Job 4 from my Build Pipeline after the two parallel processes finish.
Anybody able to help me to remove these?
Cheers
Try with Build Flow plugin
It will do both parallel and sequential jobs.
I recommend using the Multijob Plugin alone without the Build Pipeline Plugin.
The Multijob Plugin gives you the functionality of the Join Plugin, and its configuration is more straightforward. I actually prefer how it displays my running build.
You can put a multijob into a build pipeline, but the placement of the jobs within the pipeline is wrong The jobs within the multijob are displayed in vertically in alphabetical order (not build order). On the positive side, everything else seems to work, so this should be easy to fix. I reported this problem as Jenkins bug 22074.
the 'Jenkins - Build Pipeline' plugin support customize css , maybe you could make it unvisable by css
You can use build pipeline plugin together with Multijob plugin. Just use Multijob plugin as a substitute for Join plugin. Basically, Multijob plugin will only be used to make certain jobs to be executed simultaneously.
If you do this way, then build Pipeline view won't get screwed up.
This is how it looks in Pipeline Build view
build-bv-docker-images is a Multijob plugin Job.
build-(activemq|postgres|tomcat|wildfly)-bv_image are simple jobs used for building docker images
deploy-staging is a job, which is triggered after build-bv-docker-images job. Logically speaking it suppose to appear right after stack of build-*-bv-images jobs, but it appears as a part of this stack. Nevertheless, it's waiting until all jobs of this stack are completed. I had to prefix deploy-staging job with + sign in order to make it appear on the top of the stack. It looks awkward, but it's still better than to see deploy-staging job in the bottom of the stack.
This is how build-bv-docker-images multijob is configured