I am building a declarative pipeline Jenkinsfile for semantic branching. It has the format:
pipeline {
stages {
stage('develop branch build') {
when {
branch 'develop'
}
// build and deploy to QA environment
}
stage('release branch build') {
when {
branch 'release'
}
// build and deploy to live/preproduction environment
}
}
}
I would like an additional stage to run upon a Bitbucket pull request. It would do a particular PR test and deploy task, and pass or fail the pipeline accordingly.
How might I enhance this script to do that?
I use the generic webhook plugin. This work pretty nice with bitbucket.
Related
I coded a generic pipeline which accepts several parameters in order to deploy releases from a pre-defined GitHub repository to specific nodes. I wanted to host this pipeline on a Jenkinsfile on GitHub, so I configured the job to work with a "Pipeline script from SCM". The fact is - when I try and build the job - the Jenkinsfile gets checked out on every node. Is it possible to checkout and execute the Jenkinsfile only on, say, the master node and run the pipeline as intended?
EDIT: As I stated before, the pipeline works just fine and as intended setting the job to work with a pipeline script. The thing is when I try and change it to be a "Pipeline script from SCM", the Jenkinsfile gets checked out on every agent, which is a problem since I don't have git installed on any agent other than master. I want the Jenkinsfile to be checked out only on master agent and be executed as intended. FYI the pipeline below:
def agents = "$AGENTS".toString()
def agentLabel = "${ println 'Agents: ' + agents; return agents; }"
pipeline {
agent none
stages {
stage('Prep') {
steps {
script {
if (agents == null || agents == "") {
println "Skipping build"
skipBuild = true
}
if (!skipBuild) {
println "Agents set for this build: " + agents
}
}
}
}
stage('Powershell deploy script checkout') {
agent { label 'master' }
when {
expression {
!skipBuild
}
}
steps {
git url: 'https://github.com/owner/repo.git', credentialsId: 'git-credentials', branch: 'main'
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
}
stage('Deploy') {
agent { label agentLabel }
when {
expression {
!skipBuild
}
}
steps {
unstash 'deploy-script'
script {
println "Execute powershell deploy script on agents set for deploy"
}
}
}
}
}
I think that skipDefaultCheckout is what are you looking for:
pipeline {
options {
skipDefaultCheckout true
}
stages {
stage('Prep') {
steps {
script {
........................
}
}
}
}
}
Take a look to the documentation:
skipDefaultCheckout
Skip checking out code from source control by default in the agent directive.
https://www.jenkins.io/doc/book/pipeline/syntax/
I think you are requesting the impossible.
Now:
your Jenkinsfile is inside your jenkins configuration and is sent as such to each of your agents. No need for git on your agents.
Pipeline script for SCM:
Since you use git, SCM = git. So you are saying: my Pipeline needs to be fetched from a git repository. You are declaring the Deploy step to run on agent { label agentLabel }, so that step is supposed to run on another agent than master.
How would you imagine that agent could get the content of the Jenkinsfile to know what to do, but not use git ?
What happens in Jenkins?
Your master agent gets triggered that it needs to build
the master agent checkouts the Jenkinsfile using git (since it is a Pipeline script from SCM)
jenkins reads the Jenkinsfile and sees what has to be done.
for the Prep stage, I'm not quite sure what happens without agent, I guess that runs on master agent.
the Powershell deploy script checkout is marked to run on master agent, so it runs on master agent (note that the Jenkinsfile will get checked out with git two more times:
before starting the stage, because jenkins needs to know what to execute
one more checkout because you specify git url: 'https://github.com/owner/repo.git'...
the Deploy stage is marked to run on agentLabel, so jenkins tries to checkout your Jenkinsfile on that agent (using git)...
You can use Scripted Pipeline to do this, it should basically look like this
node('master') {
checkout scm
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
def stepsForParallel = [:]
env.AGENTS.split(' ').each { agent ->
stepsForParallel["deploy ${agent}"] = { ->
node(agent) {
unstash 'deploy-script'
}
}
parallel stepsForParallel
you can find all info about jenkins agent section here.
Shortly: you can call any agent by name or label.
pipeline {
agent {
label 'master'
}
}
If it will not work for you, then you will need to set any label on master node and call it by label
pipeline {
agent {
label 'master_label_here'
}
}
Currently we are developing centralized control system for our CI/CD projects. There are many projects with many branches so we are using multibranch pipeline ( This forces us to use Jenkinsfile from project branches so we can't provide custom Jenkinsfile like Pipeline projects ). We want to control everything under 1 git repo where for every project there should be kubernetes YAMLS's, Dockerfile and Jenkinsfile. When developer presses build button, Jenkinsfile from their project repo suppose to run our jenkinsfile. Is it possible to do this?
E.g. :
pipeline {
agent any
stages {
stage('Retrieve Jenkinsfile From Repo') { // RETRIEVE JENKINSFILE FROM REPO
steps {
git branch: "master",
credentialsId: 'gitlab_credentials',
url: "jenkinsfile_repo"
scripts {
// RUN JENKINSFILE FROM THE REPO
}
}
}
}
}
Main reason we are doing this, there are sensetive context in jenkinsfile like production database connections. We don't want to store jenkinsfile under developers' repo. Also you can suggest correct way to achieve that beside using only 1 repo.
EDIT: https://plugins.jenkins.io/remote-file/
This plugin solved all my problems. I could'not try comments below
As an option you can use pipeline build step.
pipeline {
agent any
stages {
stage ('build another job') {
steps {
build 'second_job_name_here'
}
}
}
}
Try load step
scripts {
// rename Jenkinsfile to .groovy
sh 'mv Jenkinsfile Jenkins.groovy'
// RUN JENKINSFILE FROM THE REPO
load 'Jenkinsfile.groovy'
}
I'm trying to configure job in Jenkins Multibranch pipeline. There are a lot of branches in SVN and I want the job to checkout only the latest one and ignores the rest of them. This job triggers a pipeline that does multiple checks on the whole build... so I always need to trigger this on the latest branch because there I will have the latest revision of the build.
The SVN structure is like this: V01_01_01 till the latest one V01_08_03. Currently I have it set up like the below and in the Jenkins pipeline I have "checkout scm", but if a new branch appears e.g. V01_08_04 I need V01_08_03 to be replaced by V01_08_04. Is there any way to do this ?
My set-up in Jenkins Multibranch pipeline
I found a hack to this. I created a python script that checks the whole repository for the latest folder that was updated.
pipeline
{
agent any
parameters
{
string(name: 'latest_folder', defaultValue: '')
}
stages
{
stage ('find latest folder')
{
steps
{
withPythonEnv('System-CPython-3.8')
{
sh 'pip3 install svn'
script {
def folder_name = sh(script: 'python3 latest_folder_svn.py', returnStdout: true)
env.latest_folder = folder_name
}
}
}
}
stage ('Checkout Step')
{
steps
{
echo "${env.latest_folder}"
}
}
}
}
This variable I will add it in the checkout step in order to have always the latest branch.
The python script is pretty straightforward. I use svn library to parse the repository and extract what I need.
With Jenkins using the Declarative Pipeline Syntax how do i get the Dockerfile (Dockerfile.ci in this example) from the SCM (Git) since the agent block is executed before all the stages?
pipeline {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
}
stage ('Checkout') {
steps {
git(
url: 'https://www.github.com/...',
credentialsId: 'CREDENTIALS',
branch: "develop"
)
}
}
[...]
}
In all the examples i've seen, the Dockerfile seems to be already present in the workspace.
You could try to declare agent for each stage separately, for checkout stage you could use some default agent and docker agent for others.
pipeline {
agent none
stage ('Checkout') {
agent any
steps {
git(
url: 'https://www.github.com/...',
credentialsId: 'CREDENTIALS',
branch: "develop"
)
}
}
stage ('Build') {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
steps {
[...]
}
}
}
[...]
}
If you're using a multi-branch pipeline it automatically checks out your SCM before evaluating the agent. So in that case you can specify the agent from a file in the SCM.
The answer is in the Jenkins documentation on the Dockerfile parameter:
In order to use this option, the Jenkinsfile must be loaded from
either a Multibranch Pipeline or a Pipeline from SCM.
Just scroll down to the Dockerfile section, and it's documented there.
The obvious problem with this approach is that it impairs pipeline development. Now instead of testing code in a pipeline field on the server, it must be committed to the source repository for each testable change. NOTE also that the Jenkinsfile checkout cannot be sparse or lightweight as that will only pick up the script -- and not any accompanying Dockerfile to be built.
I can think of a couple ways to work around this.
Develop against agents in nodes with the reuseNode true directive. Then when code is stable, the separate agent blocks can be combined together at the top of the Jenkinsfile which must then be loaded from the SCM.
Develop using the dir() solution that specs the exact workspace directory, or alternately use one of the other examples in this solution.
I have a Multibranch Pipeline project which configures Jenkins Jobs based on a Jenkinsfile per branch. The sourcecode is hosted on a Github Enterprise Server.
When I view the configuration of a branch which is created by the Jenkinsfile, I noticed that there is a option GitHub project. This option allows to define the URL of the corresponding GitHub project.
I want to define this property via my Jenkinsfile in Pipeline syntax, but I don't know command to use and how.
Relevant parts of my Jenkinsfile:
pipeline {
agent {
docker {
image 'plinzen/android:latest'
label 'android'
}
}
triggers {
githubPush()
}
stages {
stage('build') {
steps {
checkout scm
sh './gradlew clean assembleDebug'
}
}
}
}
How can I define the GitHub project properties via my Jenkinsfile? I use the Jenkins GitHub Plugin in my project.
You can add a new agent node and add this code snippet to do your things. For more info you can refer to this url also. For Additional Info. Hope this helps.
git(
url: 'git#github.com<repo_name>.git',
credentialsId: 'xpc',
branch: '${branch}'
)