I am building a test system with Jenkins with multiple slave nodes. I have multiple test cases where each of them take at more than 15 minutes to run.
I want to make the system in a way that when I start tests Jenkins running each test case in a node which is free and at the end collects and summarizes the test results.
I have opened a Jenkins job which is general test-case job and it is parametrized where the parameter is the "test name". But I see that Jenkins is executing the jobs sequentially.
How can I configure Jenkins to run builds for the same job (with different parameters) in parallel?
simple syntax for parallel:
pipeline {
stages {
stage('Run Tests In Parallel') {
parallel {
stage('Projects Test 1') {
agent {
node { label "your jenkins label" }
}
steps{
script {
your test 1
}
}
post{
always {
script {
echo ' always'
}
}
}
}
stage('Projects Test 2') {
agent {
node { label "your jenkins label" }
}
steps{
script {
your test 2
}
}
post{
always {
script {
echo ' always'
}
}
}
}
}
}
}
}
Hi you can use parallel stages in jenkins which runs parallely. also use agent any at each stage so it will use any free node.
Check Parallel Stages document for more info
Related
Please, can you advise as I am planning to invoke a job that is on different server. For example:
Slave1 server1: deploy job
Slave2 server2: build job
I want build job trigger the deploy job. Any suggestion, please
If you are trying to run two Jobs in the same Jenkins server. Your Pipeline should look something like below. Here from build Job you can call the Deploy Job.
Build Job
pipeline {
agent { label 'server2' }
stages {
stage('Build') {
steps {
build job: 'DeployJobName'
}
}
}
}
Deploy Job
pipeline {
agent { label 'server1' }
stages {
stage('Deploy') {
steps {
// Deploy something
}
}
}
}
Update
If you want to trigger a Job in a different Jenkins server, you can use a Plugin like RemoteTriggerPlugin or simply using the Jenkins API.
curl http://serer1:8080/job/DeployJobName/build?token=YOUR_REMOTE_TRIGGER_TOKEN
pipeline {
agent { label 'server2' }
stages {
stage('Build') {
steps {
sh 'curl http://server1:8080/job/DeployJobName/build?token=YOUR_REMOTE_TRIGGER_TOKEN'
}
}
}
}
In the Jenkins pipeline master-slave environment having multiple slaves type, how to force a post-build task to run in the same slave & workspace where a previous stage was executed.
For example, in the following pipeline code snippet, 3 different slaves are used. The "Dynamic Server Creation" stage is executed on "miscslave" type agent where we run a "vagrant up" command. The next 2 stages are executed in different slaves, 'performance_slave and 'seleniumslave'. Now when the tests are executed against the dynamic servers, we need to destroy it by running "vagrant destroy" as a post stage task. However, it needs to be run from the same slave "miscslave" workspace where the "vagrant up" executed, as it created a ".vagrant" directory there with the dynamic server machine info, which is required to run the "destroy".
How can we force the pipeline here to execute this post build task in the same workspace where "Dynamic Server Creation" was executed?
pipeline {
agent { label 'miscslave' }
stages {
stage('Stage 1') { }
stage("DynamicServer Creation") {
agent {
label 'miscslave'
}
stages {
stage('DynamicServer Creation') {
//Create Dynamic server using vagrant up, create a .vagrant dir to save the create machine info
}
stage('DynamicServer Test') {
parallel {
stage("Execute Performance Tests") {
agent { label 'performance_slave' }
}
stage("Execute UI Tests") {
agent { label 'seleniumslave' }
}
}
}
}
post {
always {
//Delete the dynamic server using vagrant destroy. It has to be run in the same workspace where "vagrant up" was executed in "Dynamic Server creation stage"
}
}
}
}
}```
I want to run Jenkins scripted pipeline job at the specified slave at the moment when no other job is running on it.
After my job will be started, no other jobs should be performed on this slave, they will have to wait for my job ending running
All the tutorials that I found allowed me to run job on the node when it is free but did not protect me from launching other jobs on this node
Could you tell me how can I do this?
Because Pipeline has two Syntax, there are two ways to achieve that. For scripted pipeline please check the second one.
Declarative
pipeline {
agent none
stages {
stage('Build') {
agent { label 'slave-node' }
steps {
echo 'Building..'
sh '''
'''
}
}
}
post {
success {
echo 'This will run only if successful'
}
}
}
Scripted
node('your-node') {
try {
stage 'Build'
node('build-run-on-this-node') {
sh ""
}
} catch(Exception e) {
throw e
}
}
I have a pipeline with multiple stages, some of them are in parallel. Up until now I had a single code block indicating where the job should run.
pipeline {
triggers { pollSCM '0 0 * * 0' }
agent { dockerfile { label 'jenkins-slave'
filename 'Dockerfile'
}
}
stages{
stage('1'){
steps{ sh "blah" }
} // stage
} // stages
} // pipeline
What I need to do now is run a new stage on a different slave, NOT in docker.
I tried by adding an agent statement for that stage but it seems like it tries to run that stage withing a docker container on the second slave.
stage('test new slave') {
agent { node { label 'e2e-aws' } }
steps {
sh "ifconfig"
} // steps
} // stage
I get the following error message
13:14:23 unknown flag: --workdir
13:14:23 See 'docker exec --help'.
I tried setting the agent to none for the pipeline and using an agent for every step and have run into 2 issues
1. My post actions show an error
2. The stages that have parallel stages also had an error.
I can't find any examples that are similar to what I am doing.
You can use the node block to select a node to run a particular stage.
pipeline {
agent any
stages {
stage('Init') {
steps {
node('master'){
echo "Run inside a MASTER"
}
}
}
}
}
I'm building jobs that will be using Jenkins piplines extensively. Our nodes are designated per project by their tags, but unlike regular jobs the pipeline build does not seem to have the "Restrict where this project can be run" checkbox. How can I specify on which node the pipeline will run the way I do for regular jobs?
You specify the desired node or tag when you do the node step:
node('specialSlave') {
// Will run on the slave with name or tag specialSlave
}
See https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#node-allocate-node for an extended explanation of the arguments to node.
Edit 2019: This answer (and question) was made back in 2017, back then there only was one flavor of Jenkins pipeline, scripted pipeline, since then declarative pipeline has been added. So above answer is true for scripted pipeline, for answers regarding declarative pipeline please see other answers below.
Choosing a node which has the label X in a declarative pipeline with json format:
pipeline {
agent { label 'X' }
...
...
}
You also can apply multiple labels with or (||) or with and (&&) operator.
Running the job on any of the nodes which has label X or label Y:
agent { label 'X || Y' }
Running the job only on nodes which have both label:
agent { label 'X && Y' }
More in the Jenkins Pipeline reference guide.
ps: if you are reading this you probably have just started using Jenkins pipeline and you are not sure if you should use declarative or scripted pipeline. Short answer: it's better to start with declarative. From jenkins.io:
Declarative and Scripted Pipelines are constructed fundamentally
differently. Declarative Pipeline is a more recent feature of Jenkins
Pipeline which:
provides richer syntactical features over Scripted Pipeline syntax, and
is designed to make writing and reading Pipeline code easier.
To be clear, because Pipeline has two Syntax, there are two ways to achieve that.
Declarative
pipeline {
agent none
stages {
stage('Build') {
agent { label 'slave-node' }
steps {
echo 'Building..'
sh '''
'''
}
}
}
post {
success {
echo 'This will run only if successful'
}
}
}
Scripted
node('your-node') {
try {
stage 'Build'
node('build-run-on-this-node') {
sh ""
}
} catch(Exception e) {
throw e
}
}
Agent or Node where we should not execute the jenkins job :
This is the negation of the problem statment i.e. node where not to run
It was most weird solution to me but issue has been already raised to jenkins community
agent { label '!build-agent-name' }
If you need to run entire jenkins pipeline to run on single node, use the following format
pipeline {
agent {
label 'test1'
}
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
if you need to execute each stage in different nodes, use the below format,
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
node("test1"){
echo 'Testing..'
}
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}