I need to run jenkins pipeline every 5 minutes and trigger stages only if /tmp/*.json exists. How to do that ?
I have the skeleton :
pipeline {
agent any
environment {
def JSON_PATH = '/tmp/*.json'
}
triggers {
cron('*/5 * * * *')
}
stages {
[...]
}
}
You're on the right track. To check file existence, you can use ls and read the exit status.
stage('Check file existence') {
steps {
script {
fileExists = sh(returnStatus: true, script: 'ls /tmp/*.json') == 0
}
}
}
stage('Do actions') {
when {
expression { fileExists }
}
steps {
<insert desired steps here>
}
}
Related
I looked at Call stage from function in Jenkinsfile which did some of what I wanted, but I had issues adding the code from the answer to my pipeline. I want to be able to scan for files in a folder and generate a stage for each file:
def foo = sh (
script: 'find ./collections/*.json -printf "%f\n"',
returnStdout: true
).trim().split("\n")
def parallelStagesFromMap = foo.collectEntries {
["Build ${it}" : generateStage(it)]
}
def generateStage(bar) {
return {
stage("Build ${bar}") {
echo "Building for ${bar}"
}
}
}
pipeline {
agent { label 'myExecutor' }
triggers {
pollSCM('') // empty cron expression string
cron('H 6 * * 1-5') // run every weekday at 06:00 https://crontab.guru/#0_6_*_*_1-5
}
options {
timeout(time: 20, unit: "MINUTES")
buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '10'))
disableConcurrentBuilds()
}
stages {
stage('Setup') {
steps {
script {
foo = ["1", "2", "3", "4", "5"]
}
}
}
stage('parallel') {
steps {
script {
parallel parallelStagesFromMap
generateStage("skipped") // no invocation, stage is skipped
generateStage("nonparallel").call()
}
}
}
}
post {
always {
echo ' * * * Doing POST actions'
//...
}
}
}
node {
parallel parallelStagesFromMap
generateStage("skipped") // no invocation, stage is skipped
generateStage("nonparallel").call()
}
This is essentially a hybrid between declarative and scripted pipelines, but it can be done like this:
Essentially you need to define the variables and execute the lookup in the setup stage and then run the stage generating code in a subsequent stage.
The node block is executed after the pipeline as you have put it, and the variable definitions are called before the code is checked out.
def foo
def parallelStagesFromMap
def generateStage(bar) {
return {
stage("Build ${bar}") {
echo "Building based on file ${bar}"
}
}
}
pipeline {
agent { label 'myExecutor' }
triggers {
pollSCM('') // empty cron expression string
cron('H 6 * * 1-5') // run every weekday at 06:00 https://crontab.guru/#0_6_*_*_1-5
}
options {
timeout(time: 20, unit: "MINUTES")
buildDiscarder(logRotator(numToKeepStr: '10', artifactNumToKeepStr: '10'))
disableConcurrentBuilds()
}
stages {
stage('Setup') {
steps {
script {
//scan collections folder for .json files
foo = sh (
script: 'find ./collections/*.json -printf "%f\n"',
returnStdout: true
).trim().split("\n")
parallelStagesFromMap = foo.collectEntries {
["Build ${it}" : generateStage(it)]
}
}
// do other setup-y stuff...
}
}
stage('Parallel Dynamic') {
steps {
script {
// https://stackoverflow.com/questions/55340071/call-stage-from-function-in-jenkinsfile
parallel parallelStagesFromMap
//generateStage("skipped") // no invocation, stage is skipped
//generateStage("nonparallel").call()
}
}
}
stage('static stage') {
steps {
// step to the mic ...
}
}
// more stages ...
}
post {
always {
echo ' * * * Doing POST actions'
//...
}
}
}
I've set a cron job to run every night, however I only want it to run stage B within the Jenkinsfile not all of them.
pipeline {
agent any
triggers {
cron('#midnight')
}
}
stages {
stage('A') {
...
}
stage('B'){
when {
allOf {
expression { env.CHANGE_TARGET == "master" }
branch "PR-*"
}
}
steps {
sh """
echo 'running script'
make run-script
"""
}
}
stage('C') {
...
}
Without removing the conditionals in Stage B, I can't seem to figure out how to specify the cron to explicitly only run Stage B of the Jenkinsfile - I need to run that makefile script only when those conditionals are met OR during the daily midnight cron job
You can achieve what you want with the Parameterized Scheduler Plugin which enables you to define cron triggers that trigger the job with a specific environment variable, you can then use this variable as a condition to determine which step to execute.
in your case you can use that environment variable in the when directive of each stage to determine if it should run or not according to the variable.
Something like:
pipeline {
agent any
parameters {
booleanParam(name: 'MIDNIGHT_BUILD', defaultValue: 'true', description: 'Midnight build')
}
triggers {
parameterizedCron('''
0 * * * * %MIDNIGHT_BUILD=true
''')
}
stages {
stage('A') {
when {
expression { !env.MIDNIGHT_BUILD }
}
steps {
...
}
}
stage('B') {
when {
expression { env.MIDNIGHT_BUILD || env.CHANGE_TARGET == "master" }
}
steps {
sh """
echo 'running script'
make run-script
"""
}
}
stage('C') {
when {
expression { !env.MIDNIGHT_BUILD }
}
steps {
...
}
}
}
}
My pipeline looks like that:
pipeline{
...
post {
always {
archiveArtifacts artifacts: 'artifacts/**/*'
script {
...
}
rtp stableText: '${FILE:artifacts/summary.html}', parserName: 'HTML'
}
success {
script {
...
}
}
}
}
I'd like that the script which is executed on success, was executed also on unstable, how can I achieve that?
Is there a way to specify success or unstable {?
Or is there a way to declare the action to take somewhere else and "invoke" it in a success and in an unstable tags?
you can also do like below
def commonPostSteps() {
echo "Hello World"
script {
def x =10
print x + 20
}
}
pipeline {
agent any;
stages {
stage('one') {
steps {
echo "${env.STAGE_NAME}"
}
}
}
post {
always {
echo "post always"
}
success {
commonPostSteps()
}
unstable {
commonPostSteps()
}
}
}
I'm using Jenkins declarative pipeline and I want to make a conditional step depending on an environment variable, which is set according the existence of a file.
So I just want to make something like that : if Dockerfile exist, perform next stage, else don't.
To perform this I tried :
pipeline {
// ...
stage {
stage('Docker') {
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
when {
environment name: 'IS_DOCKERFILE', value: true
}
stage('Build') {
// ...
}
}
}
}
Or :
pipeline {
// ...
stage {
stage('Docker') {
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
when {
expression {
env.IS_DOCKERFILE == true
}
}
stage('Build') {
// ...
}
}
}
}
In both cases, the Dockerfile exist and it is in the workspace. I also tried with strings ("true") but everytime, the pipeline continue without executing the stage 'Build'.
Any suggestions ?
This is because the exprsssion:
IS_DOCKERFILE = fileExists 'Dockerfile'
Creates the environment variable with boolean value as string:
$ set
IS_DOCKERFILE='false'
So the solution would be to use .toBoolean() like this:
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
stages {
stage("build docker image") {
when {
expression {
env.IS_DOCKERFILE.toBoolean()
}
}
steps {
echo 'fileExists'
}
}
stage("build libraries") {
when {
expression {
!env.IS_DOCKERFILE.toBoolean()
}
}
steps {
echo 'fileNotExists'
}
}
}
As #Sergey already posted, the problem is that you're comparing a string to a boolean. See fileExists: Verify if file exists in workspace.
Besides his answer, you can compare directly to a string:
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
stages {
stage("build docker image") {
when {
expression {IS_DOCKERFILE == 'true'}
}
steps {
echo 'fileExists'
}
}
stage("build libraries") {
when {
expression {IS_DOCKERFILE == 'false'}
}
steps {
echo 'fileNotExists'
}
}
}
i have a jenkins pipeline witch triggers another pipeline. is it possible to save the result from this triggerd job pipeline in the first pipeline?
Example pipelie 1:
stage('do something') {
when { expression { params.true == true } }
steps {
script {
env.var = build job: 'jenkinsjob2',
parameters: [
]
print(env.var)
}
}
}
jenkinsjob2 pipeline:
stage('do something else') {
when { expression { params.false == false } }
steps{
script {
my_new_result = sh( returnStdout: true, script: 'python script.py')
println my_new_result // echo 12345
I now want to use the result 12345 from the second job in env.var from the first pipeline.
is this possible?
i got for env.var = org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper#6c4ddc63 and not 12345
KR
Marvin
You can check this pipeline to get an idea of what to do.
stage('do something') {
steps
{
script {
def job = build job: "jenkinsjob2",parameters: []
env.my_new_result = job.buildVariables.my_new_result
}sh "echo ${env.my_new_result}"
}
}