How can the Jenkins Continuous Integration Game plugin (ci-game) be used in a Jenkins pipeline as code (Jenkinsfile) job?
Unfortunately, the ci-game plugin does not (yet) support pipelines.
The plugin does not appear in the Plugin Compatibility with Pipeline list.
There's already an open ticket on this issue (JENKINS-42683).
It seems that the latest update 1.26 includes the DSL for ci-game (see https://github.com/jenkinsci/ci-game-plugin/pull/19/commits/89e6c3e6ff11294418c2e741ebade5cfaa53ba1d
)
I tested it out and it seems to work when you put ciGame() :
post {
always {
ciGame()
}
}
However, this writer complained that it doesn't work:
https://github.com/jenkinsci/ci-game-plugin/commit/89e6c3e6ff11294418c2e741ebade5cfaa53ba1d
Simple Jenkins declarative pipeline with single Stage
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Hello world!'
}
}
}
}
Simple Jenkins declarative pipeline with multiple Stage
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Inside Stage 1'
}
}
stage('Stage 2') {
steps {
echo 'Inside Stage 2'
}
}
}
}
Simple Jenkins declarative pipeline with Post Actions
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Inside Stage 1'
}
post {
failure {
script { echo 'failure Inside Stage 1' }
}
success {
script { echo 'failure Inside Stage 1' }
}
}
}
stage('Stage 2') {
steps {
echo 'Inside Stage 2'
}
post {
failure {
script { echo 'failure Inside Stage 2' }
}
success {
script { echo 'failure Inside Stage 1' }
}
}
}
}
https://devopsdiagnosis.wixsite.com/tech/forum/jenkins/jenkins-pipeline
Related
I running our maven project on a jenkins server with multiple stages inside the pipeline.
Every time I decide that the branch test does not need to continue and click on abort in the jenkins ui, I need to repeat this many times until the jenkins pipeline really stops.
I guess that our jenkinsfile does not really pick up that the job was aborted and I need to abort every stage to come to the end.
Is there a way to help jenkins to get out of the pipeline?
For example a variable I can check?
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
if (!currentBuild.isAborted) {
stage('Unit Tests') {
steps {
echo 'Unit Testing'
}
}
}
if (!currentBuild.isAborted) {
stage('Deploy') {
steps {
echo 'Deploying'
}
}
}
if (!currentBuild.isAborted) {
stage('Backend Integration Tests') {
steps {
echo 'Backend Int Tests'
}
}
}
if (!currentBuild.isAborted) {
stage('Frontend Integration Tests') {
steps {
echo 'Deploying....'
}
}
}
// done
}
}
In my declarative pipeline I have several stages where Xvfb is not required and several testing stages, where it is.
Is it possible to define a Jenkins wrapper ones for several stages? Something like this:
pipeline {
agent any
stages {
stage('Build the context') {
steps {
echo 'No Xvfb here'
}
}
wrap([$class: 'Xvfb', screen: '1920x1080x24']) {
stage('Test Suite 1') {
steps {
echo 'Use Xvfb here'
}
}
stage('Test Suite 2') {
steps {
echo 'Use Xvfb here'
}
}
}
stage('cleanup') {
steps {
echo 'No Xvfb here'
}
}
}
I'm getting compilation errors wherever I put the wrap block for several stages:
WorkflowScript: 10: Expected a stage # line 10, column 17.
wrap([$class: 'Xvfb', screen: '1920x1080x24'])
As wrap is a step, we have to call it from a steps or script context. Only the latter allows us to create nested stages inside of a wrap block. Try this:
pipeline {
agent any
stages {
stage('Build the context') {
steps {
echo 'No Xvfb here'
}
}
stage('Test Start') {
steps {
script {
wrap([$class: 'Xvfb', screen: '1920x1080x24']) {
stage('Test Suite 1') {
echo 'Use Xvfb here'
}
stage('Test Suite 2') {
echo 'Use Xvfb here'
}
}
}
}
}
//...
}
}
The extra stage "Test Start" may look a bit ugly, but it works.
Note: There are no steps blocks required in the nested test stages, because we are already inside a script block, so the same rules as in scripted pipeline apply.
In my scripted pipeline I would like to set global timestamps and ansicolor option.
Below scripted pipeline not working. How can we add these two options in scripted pipeline?
Declarative Pipeline
pipeline {
agent none
options {
timestamps()
ansiColor('xterm')
}
stages {
stage('Checkout') {
agent { label 'linux' }
steps{
echo "test"
}
}
}
}
Scripted Pipeline
node('linux') {
options {
timestamps()
ansiColor('xterm')
}
stage('Pre Build Setup') {
task('Display env') {
echo "test"
}
}
}
In case of a scripted pipeline, all you have to do is to wrap your script with timestamps and ansiColor('xterm') steps as shown in the example down below:
node {
timestamps {
ansiColor("xterm") {
stage("A") {
echo 'This is stage A'
sh 'printf "\\e[31mHello World\\e[0m\\n"'
sh "sleep 3s"
}
stage("B") {
echo "This is stage B"
}
}
}
}
I want to run multiple stages inside a lock within a declarative Jenkins pipeline:
pipeline {
agent any
stages {
lock(resource: 'myResource') {
stage('Stage 1') {
steps {
echo "my first step"
}
}
stage('Stage 2') {
steps {
echo "my second step"
}
}
}
}
}
I get the following error:
Started by user anonymous
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 10: Expected a stage # line 10, column 9.
lock(resource: 'myResource') {
^
WorkflowScript: 10: Stage does not have a name # line 10, column 9.
lock(resource: 'myResource') {
^
WorkflowScript: 10: Nothing to execute within stage "null" # line 10, column 9.
lock(resource: 'myResource') {
^
3 errors
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:116)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:430)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:393)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:257)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Finished: FAILURE
What's the problem here? The documentation explicitly states:
lock can be also used to wrap multiple stages into a single
concurrency unit
It should be noted that you can lock all stages in a pipeline by using the lock option:
pipeline {
agent any
options {
lock resource: 'shared_resource_lock'
}
stages {
stage('will_already_be_locked') {
steps {
echo "I am locked before I enter the stage!"
}
}
stage('will_also_be_locked') {
steps {
echo "I am still locked!"
}
}
}
}
This has since been addressed.
You can lock multiples stages by grouping them in a parent stage, like this :
stage('Parent') {
options {
lock('something')
}
stages {
stage('one') {
...
}
stage('two') {
...
}
}
}
(Don't forget you need the Lockable Resources Plugin)
The problem is that, despite the fact that declarative pipelines were technically available in beta in September, 2016, the blog post you reference (from October) is documenting scripted pipelines, not declarative (it doesn't say as much, so I feel your pain). Lockable resources hasn't been baked in as a declarative pipeline step in a way that would enable the feature you're looking for yet.
You can do:
pipeline {
agent { label 'docker' }
stages {
stage('one') {
steps {
lock('something') {
echo 'stage one'
}
}
}
}
}
But you can't do:
pipeline {
agent { label 'docker' }
stages {
lock('something') {
stage('one') {
steps {
echo 'stage one'
}
}
stage('two') {
steps {
echo 'stage two'
}
}
}
}
}
And you can't do:
pipeline {
agent { label 'docker' }
stages {
stage('one') {
lock('something') {
steps {
echo 'stage one'
}
}
}
}
}
You could use a scripted pipeline for this use case.
If the resource is only used by this pipeline you could also disable concurrent builds:
pipeline {
agent any
options {
disableConcurrentBuilds()
}
stages {
stage('will_already_be_locked') {
steps {
echo "I am locked before I enter the stage!"
}
}
stage('will_also_be_locked') {
steps {
echo "I am still locked!"
}
}
}
}
Altho the options{} block offers this functionality it is not posible to use it in some use cases.
Lets say that you have to name your lock() with a specific name depending on a branch or an environment. You have a pipeline which you dont want to be block by disableConcurrentBuilds() and lock resources depending on a discriminator. You can not name your lock() inside the options{} block by using a environment variable or any other variable from the pipeline because the block is evaluated outside the agent.
The best solution in my opinion is the following:
pipeline {
agent { label 'docker' }
stages {
stage('Wrapper') {
steps {
script {
lock(env.BRANCH_NAME) {
stage('Stage 1') {
sh('echo "stage1"')
}
stage('Stage 2') {
sh('echo "stage2"')
}
}
}
}
}
}
}
Keep in mind that the script {} block takes a block of Scripted Pipeline and executes that in the Declarative Pipeline so no steps{} are allowed inside.
I run multiple build and test containers on the same build nodes. The test containers must lock up the node name as db username for the tests.
lock(resource: "${env.NODE_NAME}" as String, variable: 'DBUSER')
Locks in options are computed at load time, but NODE_NAME is unknown that early. In order to lock multiple stages for visual effect, we can create stages inside script block, i.e. 'run test' stage in the snippet. The stage visualization is just as good as other stage blocks.
pipeline {
agent any
stages {
stage('refresh') {
steps {
echo "freshing on $NODE_NAME"
lock(resource: "${env.NODE_NAME}" as String, variable: 'DBUSER') {
sh '''
printenv | sort
'''
script {
stage('run test')
sh '''
printenv | sort
'''
}
}
}
}
}
}
I'm using Jenkins Pipeline with the declarative syntax, currently with the following stages:
Prepare
Build (two parallel sets of steps)
Test (also two parallel sets of steps)
Ask if/where to deploy
Deploy
For steps 1, 2, 3, and 5 I need and agent (an executor) because they do actual work on the workspace. For step 4, I don't need one, and I would like to not block my available executors while waiting for user input. This seem to be referred to as either a "flyweight" or "lightweight" executor for the classic, scripted syntax, but I cannot find any information on how to achieve this with the declarative syntax.
So far I've tried:
Setting an agent directly in the pipeline options, and then setting agent none on the stage. This has no effect, and the pipeline runs as normalt, blocking the executor while waiting for input. It is also mentioned in the documentation that it will have no effect, but I thought I'd give it a shot anyway.
Setting agent none in the pipeline options, and then setting an agent for each stage except #4. Unfortunately, but expectedly, this allocates a new workspace for every stage, which in turn requires me to stash and unstash. This is both messy and gives me further problems in the parallel stages (2 and 3) because I cannot have code outside the parallel construct. I assume the parallel steps run in the same workspace, so stashing/unstashing in both would have unfortunate results.
Here is an outline of my Jenkinsfile:
pipeline {
agent {
label 'build-slave'
}
stages {
stage("Prepare build") {
steps {
// ...
}
}
stage("Build") {
steps {
parallel(
frontend: {
// ...
},
backend: {
// ...
}
)
}
}
stage("Test") {
steps {
parallel(
jslint: {
// ...
},
phpcs: {
// ...
},
)
}
post {
// ...
}
}
stage("Select deploy target") {
steps {
script {
// ... code that determines choiceParameterDefinition based on branch name ...
try {
timeout(time: 5, unit: 'MINUTES') {
deployEnvironment = input message: 'Deploy target', parameters: [choiceParameterDefinition]
}
} catch(ex) {
deployEnvironment = null
}
}
}
}
stage("Deploy") {
when {
expression {
return binding.variables.get("deployEnvironment")
}
}
steps {
// ...
}
}
}
post {
// ...
}
}
Am I missing something here, or is it just not possible in the current version?
Setting agent none at the top level, then agent { label 'foo' } on every stage, with agent none again on the input stage seems to work as expected for me.
i.e. Every stage that does some work runs on the same agent, while the input stage does not consume an executor on any agent.
pipeline {
agent none
stages {
stage("Prepare build") {
agent { label 'some-agent' }
steps {
echo "prepare: ${pwd()}"
}
}
stage("Build") {
agent { label 'some-agent' }
steps {
parallel(
frontend: {
echo "frontend: ${pwd()}"
},
backend: {
echo "backend: ${pwd()}"
}
)
}
}
stage("Test") {
agent { label 'some-agent' }
steps {
parallel(
jslint: {
echo "jslint: ${pwd()}"
},
phpcs: {
echo "phpcs: ${pwd()}"
},
)
}
}
stage("Select deploy target") {
agent none
steps {
input message: 'Deploy?'
}
}
stage("Deploy") {
agent { label 'some-agent' }
steps {
echo "deploy: ${pwd()}"
}
}
}
}
However, there are no guarantee that using the same agent label within a Pipeline will always end up using the same workspace, e.g. as another build of the same job while the first build is waiting on the input.
You would have to use stash after the build steps. As you note, this cannot be done normally with parallel at the moment, so you'd have to additionally use a script block, in order to write a snippet of Scripted Pipeline for the stashing/unstashing after/before the parallel steps.
There is a workaround to use the same build slave in the other stages.
You can set a variable with the node name and use it in the others.
ie:
pipeline {
agent none
stages {
stage('First Stage Gets Agent Dynamically') {
agent {
node {
label "some-agent"
}
}
steps {
echo "first stage running on ${NODE_NAME}"
script {
BUILD_AGENT = NODE_NAME
}
}
}
stage('Second Stage Setting Node by Name') {
agent {
node {
label "${BUILD_AGENT}"
}
}
steps {
echo "Second stage using ${NODE_NAME}"
}
}
}
}
As of today (2021), you can use nested stages (https://www.jenkins.io/doc/book/pipeline/syntax/#sequential-stages) to group all the stages that must run in the same workspace before the input step, and all the stages that must be run in the same workspace after the input step. Of course, you need to stash or to store artifacts in some external repository before the input step, because the second workspace may not be the same than the first one:
pipeline {
agent none
stages {
stage('Deployment to Preproduction') {
agent any
stages {
stage('Stage PRE.1') {
steps {
echo "StagePRE.1"
sleep(10)
}
}
stage('Stage PRE.2') {
steps {
echo "Stage PRE.2"
sleep(10)
}
}
}
}
stage('Stage Ask Deploy') {
steps {
input message: 'Deploy to production?'
}
}
stage('Deployment to Production') {
agent any
stages {
stage('Stage PRO.1') {
steps {
echo "Stage PRO.1"
sleep(10)
}
}
stage('Stage PRO.2') {
steps {
echo "Stage PRO.2"
sleep(10)
}
}
}
}
}
}