Can I block the run of my current job if the job B is currently running?
There is a plugin for Jenkins job (Blocking Job) to do this but I'm not sure how to do this in Pipeline using Jenkinsfile.
Yes you use lockable resources https://plugins.jenkins.io/lockable-resources/
Job B grabs the lock, and Job A cannot run untill the lock is freed and it can grab it
stage("Run Post-Deployment Test") {
options {
lock(resource: "deploy-env")
}
This will take the "deploy-env" lock untill the end of the stage. Meaning if another deploy tried to happen it couldnt grab the lock untill the tests stage had finished.
Note if a lock doesnt exist Jenkins will create it for you.
Related
I want to use the Jenkins "PRQA" plugin, which seems not to have the option to use it from a pipeline. The plugin would run static code analysis and publish the results.
In my case, it requires some preparations that are already done in a pipelinejob. Because of that, I want to include the job into that pipeline, but on the same executor with the data prepared by the pipeline as some kind of inlined job-step.
I have tried to create a job for the PRQA-Plugin-Step and execute this with the build step from the pipeline. But this tries to start the job on a new executor (and stalls because I have only one executor).
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Prepare'
}
}
stage('SCA') {
steps {
//Run this without using a new executor with the Environment that exists now
build 'PRQA_Job'
}
}
}
}
What is the correct way to run the job on the same executor with the current working directory.
With specified build 'PRQA_Job' it's not possible to run second job on the same executor (1 job = 1 executor), since main job just waiting for a triggered job to be finished. But you can run another job on the same agent with more than 1 executor to reach workspace from main job.
For a test porpose specify agent name in both jobs: agent 'agent_name_here'
If you want to use plugin functionality for a plugin, which has no native pipeline support, you could try using "step: General Build step" feature for Jenkins Pipelines. You can use the Pipeline Syntax wizzard linked in the Job configuration windows to generate the needed Pipeline description.
If the plugin does not show up in the "step: General Build step" part of Jenkins you can use a separate Job. To copy all the needed files/Data into this second Job you will require to use Archive Artifact/Copy Artifact functionality of Jenkins to save files from your Pipeline build.
For more information on how to sue Archive Artifact/Copy Artifact see https://plugins.jenkins.io/copyartifact/ and
https://www.jenkins.io/doc/pipeline/tour/tests-and-artifacts/
I can call another jenkins job using the build command. Is there a way I can tell another job to do a branch scan?
A multibranch pipeline job has a UI button "Scan Repository Now". When you press this button, it will do a checkout of the configured SCM repository and detect all the branches and create subjobs for each branch.
I have a multibranch pipeline job for which I have selected the "Suppress automatic SCM triggering" option because I only want it to run when I call it from another job. Because this option is selected, the multibranch pipeline doesn't automatically detect when new branches are added to the repository. (If I click "Scan Repository Now" in the UI it will detect them.)
Essentially I have a multibranch pipeline job and I want to call it from another multibranch pipeline job that uses the same git repository.
node {
if(env.BRANCH_NAME == "the-branch-I-want" && other_criteria) {
//scanScm "../my-other-multibranch-job" <--- scanScm is a fake command I made up
build "../my-other-multibranch-job/${env.BRANCH_NAME}"
I get an error on that build line, because the target multibranch pipeline job does not yet know that BRANCH_NAME exists. I need a way to trigger an SCM re-scan in the target job from this current job.
Similar to what you figured out yourself, I can contribute my optimization that actually waits until the scan has finished (but is subject to Script Security):
// Helper functions to trigger branch indexing for a certain multibranch project.
// The permissions that this needs are pretty evil.. but there's currently no other choice
//
// Required permissions:
// - method jenkins.model.Jenkins getItemByFullName java.lang.String
// - staticMethod jenkins.model.Jenkins getInstance
//
// See:
// https://github.com/jenkinsci/pipeline-build-step-plugin/blob/3ff14391fe27c8ee9ccea9ba1977131fe3b26dbe/src/main/java/org/jenkinsci/plugins/workflow/support/steps/build/BuildTriggerStepExecution.java#L66
// https://stackoverflow.com/questions/41579229/triggering-branch-indexing-on-multibranch-pipelines-jenkins-git
void scanMultiBranchAndWaitForJob(String multibranchProject, String branch) {
String job = "${multibranchProject}/${branch}"
// the `build` step does not support waiting for branch indexing (ComputedFolder job type),
// so we need some black magic to poll and wait until the expected job appears
build job: multibranchProject, wait: false
echo "Waiting for job '${job}' to appear..."
while (Jenkins.instance.getItemByFullName(job) == null || Jenkins.instance.getItemByFullName(job).isDisabled()) {
sleep 3
}
}
Ended up figuring this out shortly after posting the question. Calling build against the base multibranch pipeline job as opposed to a branch causes it to re-scan. The solution to my above snippet would have ended up looking something like...
node {
if(env.BRANCH_NAME == "the-branch-I-want" && other_criteria) {
build job: "../my-other-multibranch-job", wait: false, propagate: false // scan for branches
sleep 2 // scanning takes time
build "../my-other-multibranch-job/${env.BRANCH_NAME}"
The wait: false is important because otherwise you get "ERROR: Waiting for non-job items is not supported". The multibranch "parent" job is closer to a folder than a job, but it's a folder that supports the build command, and it does so by scanning the SCM.
But solving this just led to another problem, which is that with wait: false we have no way of knowing when the SCM Scan finished. If you have a large repository (or you're short on jenkins agents), the branch won't get discovered until after the second build command has already failed due to the branch not existing. You could bump the sleep time even higher, but that doesn't scale.
Fortunately, it turns out manually initiating the SCM scan isn't even needed if you have github webhooks set up for your jenkins. The branch will be discovered more-or-less instantly, so for my purposes this is solved another way. The reason I was running into it is we don't have webhooks set up in our dev jenkins, but once I move this code to prod it will work fine.
If you're trying to use JobDSL to set up multibranches calling multibranches and you don't have webhooks or something equivalent, the better path is probably to abandon multibranch for your second tier of jobs and use JobDSL to create folders and manage the branch jobs yourself.
Thanks for looking into my concern.
I have 3 jenkins jobs. JOb A, B & C.
Job A starts at 10PM at night.
JOB B is a down stream of Job A and runs only if job A is success.
Job C is a downstream job of job B
Now I want job C to be triggered after successful completion of job B or at at a scheduled time. Problem is if I schedule job C as down stream as well as with a schedule. It runs twice.
But, it should run only once.
Please help me to achieve this.
Did you try "Conditional BuildStep" plug-in? You can execute a downstream job (or a script) based on "Build cause"
You can add more than 1 "single" conditions for each build cause.
Now you'll need to decide when to run a job, as a timer or as a downstream
You can use jenkins pipeline plugin. You can create a pipeline job with stages. A pipeline will proceed only to next stage if previous stage is successful. Refer documentation for more details on pipeline.
Pipeline comes with a lot of flexibilities in which you can define the flow. You can either use a declarative pipeline or a scripted pipeline. Good number of examples can be found in here
In the old configuration we had 2 jobs, test and build.
The build ran after test had run successfully, but we could manually trigger build if we want to skip the tests.
After we switched to multiple pipeline using Jenkinsfile, we had to put those 2 build jobs in to the same file:
stage('Running tests'){
...
}
stage('Build'){
...
}
So now the build step is only triggered after running tests successfully, and we cannot manually trigger build, without commenting out the test steps and commit to the repository.
I am wondering if there is a better approach/practise to utilise the Jenkinsfile to overcome this limitation?
Using pipeline and Jenkinsfile is becoming the standard and preferred way of running jobs on Jenkins now a days. So using a Jenkinsfile is certainly the way to go.
One way to solve the problem is to make the job parameterized:
// Set the parameter properties, this will be done at the first run so that we can trigger with parameters manually
properties([parameters([booleanParam(defaultValue: true, description: 'Testing will be done if this is checked', name: 'DO_TEST')])])
stage('Running tests'){
// Putting the check inside of the stage step so that we don't confuse the stage view
if (params['DO_TEST']) {
...
}
}
stage('Build'){
...
}
The first time the job runs, it will add a parameter to the job. After that we can trigger manually and select whether tests should run. The default value will be used when it's triggered by SCM.
I have one pipeline build called, for example UAT. This build is scheduled every 3 minutes. And another build called DEV. DEV is scheduled every minute. The task is: to run UAT only if the last DEV execution was SUCCESS. If not - skip the execution. And run it after other 3 minutes with the same condition.
How can I achieve that ?
Don't schedule your UAT job as a separate job but instead trigger the launch once your first DEV pipeline finishes with success.
As you are using pipelines you actually have 2 solutions :
1)
Don't call another job but just call a Groovy function to integrate the DEV part, such as :
node() {
stage "UAT"
// Your existing UAT pipeline content here
stage "DEV"
git 'http://urlToYourGit/projectContainingYourDevScript'
pipeline = load 'functions.groovy'
pipeline.dev()
}
2) Just call a second Jenkins job with this kind of line :
node() {
stage "UAT"
// Your existing UAT pipeline content here
build job: "dev-job"
}
With these 2 solutions you can configure your first job to run every minute and it will trigger the second part/job only if the first one finishes with success (otherwise Jenkins will just fail the build as it would normally do).