I'm trying to get the following features to work in Jenkins' Declarative Pipeline syntax:
Conditional execution of certain stages only on the master branch
input to ask for user confirmation to deploy to a staging environment
While waiting for confirmation, it doesn't block an executor
Here's what I've ended up with:
pipeline {
agent none
stages {
stage('1. Compile') {
agent any
steps {
echo 'compile'
}
}
stage('2. Build & push Docker image') {
agent any
when {
branch 'master'
}
steps {
echo "build & push docker image"
}
}
stage('3. Deploy to stage') {
when {
branch 'master'
}
input {
message "Deploy to stage?"
ok "Deploy"
}
agent any
steps {
echo 'Deploy to stage'
}
}
}
}
The problem is that stage 2 needs the output from 1, but this is not available when it runs. If I replace the various agent directives with a global agent any, then the output is available, but the executor is blocked waiting for user input at stage 3. And if I try and combine 1 & 2 into a single stage, then I lose the ability to conditionally run some steps only on master.
Is there any way to achieve all the behaviour I'm looking for?
You need to use the stash command at the end of your first step and then unstash when you need the files
I think these are available in the snippet generator
As per the documentation
Saves a set of files for use later in the same build, generally on
another node/workspace. Stashed files are not otherwise available and
are generally discarded at the end of the build. Note that the stash
and unstash steps are designed for use with small files. For large
data transfers, use the External Workspace Manager plugin, or use an
external repository manager such as Nexus or Artifactory
Related
I have parallel stages setup in my Jenkins pipeline script which execute tests on separate nodes (AWS EC2 instances)
e.g.
stage('E2E-PR') {
parallel {
stage('Cypress Tests 1') {
agent { label 'aws_slave_cypress' }
steps {
runE2eTests()
}
}
stage('Cypress Tests 2') {
agent { label 'aws_slave_cypress' }
steps {
runE2eTests()
}
}
}
}
I would like to re-use the checked out repo\generated workspace generated from the parent node used at the start of my pipeline rather than each parallel stage checking out it's own copy.
I came across an approach using sequential stages, and nesting stages within stages to share workspaces across multiple stages, which I tried like below
parallel {
stage('Cypress Tests') {
agent { label 'aws_slave_cypress' }
stages {
stage('Cypress 1') {
steps {
runE2eTests()
}
}
stage('Cypress 2') {
steps {
runE2eTests()
}
}
}
}
}
But I can see from my Jenkins build output that only one aws instance gets spun up and used for both of my nested stages which doesnt give me the benefit of parallelism.
I have also come across the stash\unstash commands, but I have read these should be used for small files and not for large directories\entire repositories?
What's the right approach here to allow me to have parallel stages across multiple nodes use the same originally generated workspace? Is this possible?
Thanks
I can share a bit information on a similar situation we have in our company:
We have an automation regression suite for UI testing that needs to be executed on several different machines (Win32,Win64, Mac and more). The suite configuration is controlled by a config file that contains all relevant environment parameters and URLs.
The Job allows a user to select the suites that will be executed (and the git branch), select labels (agents) that the suites will be executed on, and select the environment configuration that those suites will use.
How does the flow look like:
Generate the config file according to the user input and save (stash) it.
In parallel for all given labels: clone the suites repository using shallow clone -> copy the configuration file (unsatsh) -> run the tests -> save (stash) the output of the tests.
Collect the outputs of all tests (unstash), combine them into a single report, publish the report and notify users.
The pipeline itself (simplified) will look like:
pipeline {
agent any
parameters {
extendedChoice(name: 'Agents', ...)
}
stages {
stage('Create Config') {
steps {
// create the config file called config.yaml
...
stash name: 'config' , includes: 'config.yaml'
}
}
stage('Run Tests') {
steps {
script {
parallel params.Agents.collectEntries { agent ->
["Running on ${agent}": {
stage(agent) {
node(agent) {
println "Executing regression tests on ${agent}"
stage('Clone Tests') {
// Git - shallow clone the automation tests
}
stage('Update Config') {
unstash 'config'
}
stage('Run Suite') {
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE') {
// Run the tests
}
}
stage('Stash Results'){
stash name: agent , includes: 'results.json'
}
}
}
}]
}
}
}
}
stage('Combine and Publish Report') {
steps {
// Unstash all results from each agent to the same folder
params.Agents.each { agent ->
dir(agent){
unstash agent
}
}
// Run command to combine the results and generated the HTML report
// Use publishHTML to publish the HTML report to Jenkins
}
}
}
post{
// Notify users
}
}
So as you can see for handling relatively small files like the config files and the test result files (which are less the 1M) stash and unstash are quick and very easy to use and offer great flexibility, but when you use them for bigger files, then they start to overload the system.
Regarding the tests themself, eventually you must copy the files to each workspace in the different agents, and shallow cloning them from the repository is probably the most efficient and easy to use method if your tests are not packaged.
If your tests can easily be packaged, like a python package for example then you can create a different CI pipeline that will archive them as a package for each code change, and then your automation pipeline can use them with tools like pip or so and avoid the need to clone the repository.
That is also true when you run your tests in a container, as you can create a CI that will prepare an image that already contains the tests and then just use that image in the pipeline without need for cloning it agin.
If your Git sever is somewhat very slow you can always package the tests in your CI using zip or a similar archiver, upload them to S3 (using aws-steps) or equivalent and download them during the automation pipeline execution - but in most cases this will not offer any performance benefits.
I suppose you need to send generated files to s3 on post step for example. Download these files on the first step of your pipeline.
Think about storage, which you need to share between jenkins agents.
We have "stage (build)" on all our branches. Temporarily how can we skip this stage to run on all our branches in multibranch pipeline. I know one solution is use when condition on stage and ask all developers to pull that branch into their branch. But thats lot of work and coordination. Instead I am looking for a global configuration where we can simply skip the stage by name on any branch.
It sounds like you keep the Jenkinsfile alongside the code, but want to change how the Jenkinsfile runs from an administrative point of view.
You could store this stage in a separate (version control flavor of preference) repository. Before you execute the stage, load in the repository then load in the script file and within the stage execute a method defined in that file.
Alternatively, add a parameter to the build job (this assumes you aren't setting parameters via options in the Jenkinsfile) that is a boolean to use or skip the stage and you can modify the project configuration when you want to turn it on or off
What about this approach ?
stage('build') {
if(buildFlag=="disable"){
//jump to next stage
return;
}
//do something
}
buildFlag could be
a simple var in the same pipeline
a global var configured in jenkins. You could set disabled temporarily to skip build stage and change to enabled to restore to normalcy
Also you could set the status to failure, instead of return :
currentBuild.result = 'FAILURE'
Or throw an exception and exit the entire job or pipeline.
One option is to skip the body of the stage if the condition is met. For example:
stage('Unit Tests') {
steps {
script{
if (env.branch.startsWith(releaseBranch)} {
echo 'Running unit tests with coverage...'
} else {
// run actual unit tests
}
}
}
}
The only downside is that the UI will show the "green box" for this step - even though it effectively did nothing.
If you want to remove the stage completely for the branch, use a when directive.
stage('deploy') {
when {
branch 'production'
}
steps {
echo 'Deploying'
}
}
Bonus: You can also specify a "when not" directive as well. See How to specify when branch NOT (branch name) in jenkinsfile?
I was reading about the best practices of a Jenkins pipeline.
I have created a declarative pipeline which is not executing parallel jobs and I want to run everything on the same slave.
I use:
agent {
label 'xxx'
}
The rest of my pipeline looks like:
pipeline {
agent {
label 'xxx'
}
triggers {
pollSCM pipelineParams.polling
}
options {
buildDiscarder(logRotator(numToKeepStr: '3'))
}
stages {
stage('stage1') {
steps {
xxx
}
}
stage('stage2') {
steps {
xxx
}
}
}
post {
always {
cleanWs()
}
failure {
xxx"
}
success {
xxx
}
}
}
Now I read the best practices here.
Point 4 is telling:
Do: All Material Work Within a Node
Any material work within a pipeline should occur within a node block.
Why? By default, the Jenkinsfile script itself runs on the Jenkins
master, using a lightweight executor expected to use very few
resources. Any material work, like cloning code from a Git server or
compiling a Java application, should leverage Jenkins distributed
builds capability and run an agent node.
I suspect this is for scripted pipelines.
Now my questions are:
Do I ever have to create a node inside a stage in a declarative pipeline (it is possible) or do I have to use agent inside the stage when I want to run my stage on another specific agent?
My current pipeline has defined a label which is on 4 agents. But my whole pipeline is always executed on one agent (what I want) but I would suspect it's executing stage1 on slaveX and maybe stage2 on slaveY. Why is this not happening?
The documentation is quite misleading.
What the documentation is suggesting is to take advantage of distributed builds. Distributed builds activated either by using the agent or node block.
The agent should be used when you want to run the pipeline almost exclusively on one node. The node block allows for more flexibilty as it allows you to specify where a granular task should be executed.
If you running the pipeline on some agent and you encapsulate a step with node with the same agent, there won't be any effect execpt that a new executor will be allocated to the step encapsulated with node.
There is no obvious benefit in doing so. You will simply be consuming executors that you don't need.
In conclusion, you are already using distributed builds when using agent and this is what the documentation is vaguely recommending.
I have a CD pipeline that requires user confirmation at some stages, so I would like to free up server resources while the pipeline is waiting for the user input.
pipeline {
agent any
stages {
stage ('Build Stage') {
steps {
...
}
}
stage ('User validation stage') {
agent none
steps {
input message: 'Are you sure you want to deploy?'
}
}
stage ('Deploy Stage') {
steps {
...
}
}
}
}
You can see above that I have a global agent any but in the User Validation Stage I added agent none.
Can someone confirm that this doing what I want (no agent/node is waiting for the user input)? I don't see how to verify it, nothing different in the execution log...
If not, how could I do it?
This will not work as you expect. You cannot specify agent any on the entire pipeline and then expect agent none to not occupy the executor.
To prove this, you can run this code as you have it, and while it is waiting at the input stage, go to your main jenkins page and look at the Build Executor Status. You will see there is an executor still running your job.
Next, switch your pipeline to agent none and add agent any to all the other steps (besides your input step) and do the same test. You can see that while waiting at the input stage, none of the executors are occupied.
As to your question about different workspaces on different nodes... Assuming you are using code from an SCM, it will be checked out on every new node, so that isn't a concern. The only thing you need to worry about is artifacts you created in each stage.
It is not safe to "hope" that you will stay on the same node, though Jenkins will "try" to keep you there. But even then, there is not a guarantee that you will get the same workspace directory.
The correct way to handle this is to stash all the files that you may have created or modified that you will need in later stages. Then in the following stages, unstash the required files. Never assume files will make it between stages that have their own node declaration.
I am investigating the use of Jenkins Pipeline (specifically using Jenkinsfile). The context of my implementation is that I'm deploying a Jenkins instance using Chef. Part of this deployment may include some seed jobs, which will pull job configurations from source control (Jenkinsfile), to automate creation of our build jobs via Chef.
I've investigated the Jenkins documentation for both Pipeline as well as Jenkinsfile, and it seems to me that in order to use Jenkins Pipeline agents are required to be configured and set up in addition to Jenkins Master.
Am I understanding this correctly? Must Jenkins agents exist in order to use Jenkins Pipeline's Jenkinsfile? This specific line in the Jenkinsfile documentation leads me to believe this to be true:
Jenkinsfile (Declarative Pipeline)
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
The Declarative Pipeline example above contains the minimum necessary
structure to implement a continuous delivery pipeline. The agent
directive, which is required, instructs Jenkins to allocate an
executor and workspace for the Pipeline.
Thanks in advance for any Jenkins guidance!
The 'agent' part of the pipeline is required however this does not mean that you are required to have an external agent in addition to your master. If all you have is the master this pipeline will execute on the master. If you have additional agents available the pipeline would execute on whichever agent happens to be available when you run the pipeline.
If you go into
Manage Jenkins -> Manage Nodes and Clouds, you can see 'Master' itself is treated as one of the Default Nodes. With declarative format agent anyindicates any available agent which (including 'Master' as well from node configuration see below).
In case if you configure any New node, this can then be treated as New Agent in the pipeline agent any can be replaced by agent 'Node_Name'
You may can refer this LINK which give hint on Agent, Node and Slave briefly.