this is very tough to me to understand how Jenkins works. In general when you read documentation and define pipeline, things go smooth. I understand pipelines, stages, steps, scripts. What I don't understand is declaration vs runtime. Especially when it comes to WHEN declaration and evaluating expression. For example:
What is the form of expression? Should it return something like: return true; or maybe it should be statement like: true
When it gets executed? If I access params.MY_PARAMETER_FROM_INPUT, do WHEN has access to its value picked by user?
Is it possible to switch execution between runtime vs pipeline declaration time?
Can I ask for stage (input with message box) only if given condition within WHEN is meet and if not, then don't ask for it but run stage anyway?
When you use IF from script and when WHEN from stage. Can WHEN be defined else where? Within steps, scripts, pipeline?
For example in a stage I've put when { expression { params.ENV == 'prod' } } input { message "Really?" ok "Yeah!" } but the expression was ignored and the question was always asked (current understanding is that it should skip stage/abort whole pipeline when ENV input param is different than "prod" value)
Any thoughts?
OK. Using existing semantic of Jenkins I have manage to achieve what I wanted with following snippet:
stage('Confirm if production related') {
when {
beforeInput true
expression { params.ENV == 'production'; }
}
input {
message "Should I deploy to PRODUCTION?"
ok "Yes, do it!"
}
steps {
script { _ }
}
}
Not bad but not good either.
Related
I have the following declarative pipeline where I write a global build variable
during a parallel matrix, the write in stage Build Detection is probably (wasn't clear to me) a race condition but I am not sure. I have 3 questions regarding the below simple pipeline:
Is it correct that since Build-Detection uses the same agent (note only Build uses a different agent), it is definitely a race condition ?
If I would have one agent for each parallel line, it would not be a
race condition as the global build is different in each agent?
Is there a way to make a variable copy of build inside the stage such that its not global anymore?
How should we deal with global variable communicating stuff (for when steps etc)
and parallel matrix feature?
Map<String,Boolean> build
pipeline {
stages {
stage('Test') {
failFast false
matrix {
axes {
axis {
name 'CONTAINER'
values 'A', 'B'
}
}
stages {
stage('Build Detection') {
steps {
script {
build[CONTAINER] = CONATAINER == 'A'
echo "Should Build: ${build[CONTAINER]}"
}
}
}
stage('Build') {
agent {
kubernetes {
yamlFile '.jenkins/pods/build-kaniko.yaml'
}
}
when {
beforeAgent true
expression { return build[CONTAINER] }
}
steps {
echo "BUILDING....."
}
}
}
}
}
}
}
No, it has nothing to do with build agents. The JVM that's executing the compiled groovy code is running on the Jenkins master, not a build agent. Therefore, using a global variable is shared by each thread running in the Jenkins master JVM. Whether there's a possible race condition is not related to stages using the same or different build agents.
Same answer as 1.
Yes, simply define a variable using "def" or a specific type in the stage's script block. Just be sure to not reference a new variable without a type because in Groovy that causes it to be declared globally.
Using a map with a key that is specific to each thread like you're doing seems like a good way to me. If you really want to make sure there is no possibility of two unsafe thread operations modifying the map at the same time, then make sure that a threadsafe map is used. You could print out the class of the map to find out what implementation is getting instantiated. I would hope it's something threadsafe like ConcurrentHashMap.
I have a jenkins declarative pipeline which I am interested to be able to perform a stage only if a specific environment variable contains a specific substring(not fully equals to it, just contains it).
Does anyone got any idea on how can I implement it(maybe using the when condition if possible).
Thanks in advance,
Alon
As you mentioned, in declarative pipeline you can use the when directive to establish a condition in which the stage will be executed.
Among the built in condition options like triggeredBy,branch and tag there is the generic expression option, which allows you to run any groovy code and calculate the relevant Boolean value according to your needs.
So for your case for example you can just use the groovy contains methods to achieve what you want, something like:
pipeline {
agent any
stages {
stage('Conditional Stage') {
when {
expression { return env.MyParamter.contains('MySubstring') }
}
steps {
echo "Running the conditional stage"
}
}
}
}
In my Jenkinsfile, I have 2 stages: Pre Live and Live. I ask the user for input on stage Pre Live to know whether a deploy should be done to a pre live environment, and then on stage Live, I ask the user again for input to know whether to do a deploy to a live environment or not.
I managed to implement this. This is how the code looks:
stage("Pre Live") {
input {
message 'Deploy to Pre Live?'
parameters {
booleanParam(name: 'RELEASE_PRE_LIVE', defaultValue: false)
}
}
when {
beforeInput false
expression {
return RELEASE_PRE_LIVE.toBoolean()
}
}
steps {
// ...
}
}
stage("Live") {
input {
message 'Deploy to Live?'
parameters {
booleanParam(name: 'RELEASE_LIVE', defaultValue: false)
}
}
when {
beforeInput false
expression {
return RELEASE_LIVE.toBoolean()
}
}
steps {
// ...
}
}
What I am not able to do, however, is too keep all of this logic, but also only ask for input on the stage Live if the the previous stage (Pre Live) was executed. Normally, this could be done through the when directive in the Live stage, but the problem is that I need my when directive in that stage to evaluate after the input, because I need the input value to know if the user wants to deploy to live or not, but I also don't want to unnecessarily wait for input on this stage if Pre Live was never ran, because it doesn't make sense.
Have you considered using build step that would trigger another pipeline and splitting your pipeline into two pipelines instead of one?
Here is an example article for declarative pipeline syntax:
https://support.cloudbees.com/hc/en-us/articles/360019828412-Pipeline-How-to-write-a-declarative-pipeline-to-invoke-another-job
I believe you can implement an additional check by setting then passing a variable from the Pre-Live stage to the Live stage. Then acting on that variable in the Live stage.
You already have this next part down, but I thought it would be good here for context.
Evaluating when before the input directive
By default, the when condition for a stage will not be evaluated before the input, if one is defined. However, this can be changed by specifying the beforeInput option within the when block. If beforeInput is set to true, the when condition will be evaluated first, and the input will only be entered if the when condition evaluates to true.
beforeInput true takes precedence over beforeAgent true.
From: https://jenkins.io/doc/book/pipeline/syntax/#when
I am creating a Jenkins pipeline, I want certain stage to be triggered only when a particular log file's(log file is located in the server node where all the stages are going to run) last modified date is updated after the initiation of pipeline job, I understand we need to use "When" condition but not really sure how to implement it.
Tried referring some of the pipeline related portals but could not able to find an answer
Can some please help me through this?
Thanks in advance!
To get data about file is quite tricky in a Jenkins pipeline when using the Groovy sandbox since you're not allowed to do new File(...).lastModified. However there is the findFiles step, which basically returns a list of wrapped File objects with a getter for last modified time in millis, so we can use findFiles(glob: "...")[0].lastModified.
The returned array may be empty, so we should rather check on that (see full example below).
The current build start time in millis is accessible via currentBuild.currentBuild.startTimeInMillis.
Now that we git both, we can use them in an expression:
pipeline {
agent any
stages {
stage("create file") {
steps {
touch "testfile.log"
}
}
stage("when file") {
when {
expression {
def files = findFiles(glob: "testfile.log")
files && files[0].lastModified < currentBuild.startTimeInMillis
}
}
steps {
echo "i ran"
}
}
}
}
Is it possible to create a Jenkins pipeline with an optional input stage?
The below snippet doesn't achieve this goal.
Expected behaviour
The stage (and therefore the input prompt) should only run for specific branches.
Actual behaviour
This stage runs for all branches. The when filter is ignored when an input step is used.
stage('Approve') {
when {
expression { BRANCH_NAME ==~ /^qa[\w-_]*$/ }
}
input {
message "Approve release?"
ok "y"
submitter "admin"
parameters {
string(name: 'IS_APPROVED', defaultValue: 'y', description: 'Deploy to master?')
}
}
steps {
script {
if (IS_APPROVED != 'y') {
currentBuild.result = "ABORTED"
error "User cancelled"
}
}
}
}
The filter is not ignored, it is just evaluated after the input step. In your example, you would always be asked whether to deploy, and in the case that you are not on a QA branch, nothing would happen.
Now you could ask why Jenkins isn't evaluating the 'when' directive first. In that case, you could not use the input parameter in your when condition.
And having multiple when directives would be like scripting within the declarative pipeline.
However, there is an expression that allows you controlling when the 'when' directive is evaluated. This is beforeAgent. It allows you to evaluate the when statement before the agent is allocated. Similar to that, you would need something like beforeInput. You could create a feature request for that.
I stepped away from using the input directive and I use input within a scripting block now, because that provides much more flexibility, e.g. I am sending Slack notifications when somebody has to approve something, which is impossible with the declarative approach. You would need a notify directive for that. And if there was one, is that going to be evaluated before or after the input step?
You see, doing everything declarative is not always the best way. So my recommended approach is the following (disclaimer: this is untested!):
pipeline {
// We want to use agents per stage to avoid blocking our build agents
// while we are waiting for user input.
agent none
...
// The question mark naming convention is helpful to show you which
// approval stage belongs to which work stage.
stage('Release?') {
// Don't allocate an agent because we don't want to block our
// slaves while waiting for user input.
agent none
when {
// You forgot the 'env.' in your example above ;)
expression { env.BRANCH_NAME ==~ /^qa[\w-_]*$/ }
}
options {
// Optionally, let's add a timeout that we don't allow ancient
// builds to be released.
timeout time: 14, unit: 'DAYS'
}
steps {
// Optionally, send some notifications to the approver before
// asking for input. You can't do that with the input directive
// without using an extra stage.
slackSend ...
// The input statement has to go to a script block because we
// want to assign the result to an environment variable. As we
// want to stay as declarative as possible, we put noting but
// this into the script block.
script {
// Assign the 'DO_RELEASE' environment variable that is going
// to be used in the next stage.
env.DO_RELEASE = input ...
}
// In case you approved multiple pipeline runs in parallel, this
// milestone would kill the older runs and prevent deploying
// older releases over newer ones.
milestone 1
}
}
stage('Release') {
// We need a real agent, because we want to do some real work.
agent any
when {
// Evaluate the 'when' directive before allocating the agent.
beforeAgent true
// Only execute the step when the release has been approved.
environment name: 'DO_RELEASE', value: 'yes'
}
steps {
// Make sure that only one release can happen at a time.
lock('release') {
// As using the first milestone only would introduce a race
// condition (assume that the older build would enter the
// milestone first, but the lock second) and Jenkins does
// not support inter-stage locks yet, we need a second
// milestone to make sure that older builds don't overwrite
// newer ones.
milestone 2
// Now do the actual work here.
...
}
}
}
The correct syntax would be more like (completely untested):
stage('Approve') {
when {
expression { BRANCH_NAME ==~ /^qa[\w-_]*$/ }
}
steps {
script {
def IS_APPROVED = input(
message: "Approve release?"
ok: "y"
submitter: "admin"
parameters: [
string(name: 'IS_APPROVED', defaultValue: 'y', description: 'Deploy to master?')
]
)
if (IS_APPROVED != 'y') {
currentBuild.result = "ABORTED"
error "User cancelled"
}
}
}
}
So essentially, you're hitting the limits of declarative pipelines have to fall back to groovy scripting / scripted pipelines.