I need to clear workspace before build starts. I tried using cleanDir() in stages, but in the declarative pipeline, check out happens first and when stage with cleadDir runs, checked out code also gets cleared which is not desired. How can we clear the workspace before check out in the declarative pipeline?
Actually, I have to revise my answer based on recent changes to the pipeline plugins, e.g. GitHub Branch Source Plugin 2.2.0 with JENKINS-43507.
Besides the different branch discovery behaviours, which can be configured, one can now define additional steps to take, including Clean before checkout (and Clean after checkout):
The resulting output in the pipeline execution will then be
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
so, pretty close to the calling git clean yourself.
stage('Git') {
steps {
step([$class: 'WsCleanup'])
checkout scm
}
}
the WsCleanup does the trick
Use the means of your VCS, with Git run
git clean -fdx
Add Wipe out repository & force clone in Additional Behaviours of SCM configuration as on picture below:
Something like this should work:
env.WORKSPACE = pwd()
sh "rm ${env.WORKSPACE}/* -fr"
A slightly different approach would be to use the stash step to stash your code first:
stash includes: 'src/**', name: 'source-code'
After that you can delete everything in your current Workspace. In a later stage you can simply unstash the source code again:
unstash 'source-code'
Another advantage of stash/unstash is that you can use it to share files between multiple jenkins nodes.
Related
I've created a Jenkins Multibranch Pipeline with the GitHub Branch Source plugin. The Jenkinsfile essentially just calls a Cake Build script (build.ps1, build.cake) that contains all the build/deploy logic. This allows me to move to another CI service easily.
Unfortunately, I cannot seem to figure out how to add my Cake Build scripts as a trusted file so that PR's from forks will pull the files from the source repo instead. The Trust setting of the Discover pull requests from forks behavior seems to indicate that there can be other trusted files besides Jenkinsfile:
Nobody
Pull requests from forks will all be treated as untrusted. This means that where Jenkins requires a trusted file (e.g. Jenkinsfile) the contents of that file will be retrieved from the target branch on the origin repository and not from the pull request branch on the fork repository.
However, I cannot seem to find any documentation on adding other trusted files. The primary reason for this is to prevent a PR from a fork from accessing credentials from the Cake script. They wouldn't be able to change Jenkinsfile, but they could still change the Cake script to expose the credentials.
Is it actually possible to add other trusted files?
It seems like Jenkins does not support this. My solution is checking out the untrusted files manually from the base version instead. First getting the commit 's hash of the base version with:
def commit = sh(
script: 'git rev-parse HEAD',
returnStdout: true
).trim()
def base = sh(
script: "git rev-list --parents -n 1 ${commit}",
returnStdout: true
).trim().split('\\s+')[2]
git rev-list --parents -n 1 ${commit} will return the hash of current commit, which is a merge commit that was created by Jenkins; the latest commit of the PR and the latest commit of the target branch, separated by a space (e.g. 05e9322574ea03003f87dcbb44f172e6fa62581f b3f6ef892af9c645f490106757d7d05df3a26060 069ffd55ae36414a51b4de166aef86966f9447a8). Hence, we grab the hash of the latest commit of the target branch by trim().split('\\s+')[2].
Now we can do sh "git checkout ${base} FILE" on any file that we don't trust from the PR.
This does not works if the PR is already merged with the latest version of target branch. So what I did is something like this:
// revert untrusted files to the base version and backup it before we execute any untrusted code so the attacker
// don't have a chance to put a malicious content
def latest = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
sh "git checkout origin/${env.CHANGE_TARGET}"
def baseCompose = readFile('docker-compose.yml')
// switch back to latest commit
sh "git checkout ${latest}"
sh 'git clean -d -f -f -q -x'
Jenkins declarative pipelines offer a post directive to execute code after the stages have finished. Is there a similar thing to run code before the stages are running, and most importantly, before the SCM checkout?
For example something along the lines of:
pre {
always {
rm -rf ./*
}
}
This would then clean the workspace of my build before the source code is checked out.
pre is a cool feature idea, but doesn't exist yet. skipDefaultCheckout and checkout scm (which is the same as the default checkout) are the keys:
pipeline {
agent { label 'docker' }
options {
skipDefaultCheckout true
}
stages {
stage('clean_workspace_and_checkout_source') {
steps {
deleteDir()
checkout scm
}
}
stage('build') {
steps {
echo 'i build therefore i am'
}
}
}
}
For the moment there are no pre-build steps but for the purpose you are looking for, it can be done in the pipeline job configurarion and also multibranch pipeline jobs, when you define where is your jenkinsfile, choose Additional Behaviours -> Wipe out repository & force clone.
Delete the contents of the workspace before building, ensuring a fully fresh workspace.
If you do not really want to delete everything and save some network usage, you can just use this other option: Additional Behaviours -> Clean before checkout.
Clean up the workspace before every checkout by deleting all untracked files and directories, including those which are specified in .gitignore. It also resets all tracked files to their versioned state. This ensures that the workspace is in the same state as if you cloned and checked out in a brand-new empty directory, and ensures that your build is not affected by the files generated by the previous build.
This one will not delete the workspace but just reset the repository to the original state and pull new changes if there are some.
I use "Prepare an environment for the run / Script Content"
There is an open bug on jenkins 2.0 pipeline scripts relating to included regions in git, so it means for a large mono-repo as in my case each checkin to master will cause multiple pipelines to be kicked off which is not the desired behavior.
So to visualize:
top-level:
->application folder 1
->application folder 2
What I want to do is to do a git fetch first so then I can do a git diff to see if anything in the particular folder has changed and if it has then run the pipeline for that particular folder and not do anything if nothing changed
Code I have is below:
node{
git credentialsId: 'cred id', url: 'ssh://git#git-repo:1234/app/mono-repo.git'
ret = sh(script: 'git fetch; git diff origin/master remotes/origin/master | grep "folder-name"', returnStatus: true)
if(ret == 0){
doSomething()
}else{
doNothing()
}
}
The issue I have that the git fetch fails due a permissions error, I can use a the checkout but then I cannot get the diff before hand which is not what. Is there a way of using the u tiling the git fetch using the credentias?
It might help to simply get the references to tags. Note, I believe this is not equivalent to git fetch --tags. See Does "git fetch --tags" include "git fetch"? for example.
git([branches: [
[name: '*/master'],
[name: 'refs/tags/*:refs/tags/*']],
credentialsId: CREDENTIALS_ID_GIT,
url: REPO])
I noticed that on the Jenkin's console the Git plugin is performing a git fetch --tags, so by default the Git Plugin may already provide this functionality. Please check on this.
I'd like to also add this solution:
withCredentials(
[usernamePassword(
credentialsId: CREDENTIALS_ID_GIT,
passwordVariable: 'GIT_PASSWORD',
usernameVariable: 'GIT_USERNAME')]) {
sh("git fetch --tags https://${GIT_USERNAME}:${GIT_PASSWORD}#${REPO}")
}
Make sure you are using SSH credentials with the correct user. You can check this answer, which is summarized by the capture below :
In particular, make sure that in the url ssh://git#git-repo:1234/app/mono-repo.git the git# part is your actual SSH user. In my case it is the jenkins user, so I would use ssh://jenkins#git-repo:1234/app/mono-repo.git instead.
I'm doing sonar integration and would like to pass git branch as a parameter. It will be run on Jenkins server.
Before I was using next line of code to get current git branch:
def workingBranch = """git rev-parse --abbrev-ref HEAD""".execute().text.trim()
After I replaced it with:
grgit.branch.current.fullName
But this always gives me "HEAD". How to achieve same functionality?
I am doing something very similar. As it turns out, the Git plugin in Jenkins is tuned in several ways to minimize the git clone and checkout. There are 2 ways that I've found to deal with this.
The Easy Way
Use Jenkins' built-in environment variables, as you already suggested in the comments.
def workingBranch = System.getenv("GIT_BRANCH") ?: grgit.branch.current.fullName
The Job Configuration Way
You can also set up the job to checkout the branch as a local branch, rather than a detached HEAD. This is under "Additional Behaviors", named "Check out to specific local branch". There are many other questions that detail the settings for that, and/or the Declarative Pipeline approach, depending on what your needs.
Declarative Pipeline custom checkout
Configuring local branch option
I need to know which branch is being built in my Jenkins multibranch pipeline in order for it to run steps correctly.
We are using a gitflow pattern with dev, release, and master branches that all are used to create artifacts. The dev branch auto deploys, the other two do not. Also there are feature, bugfix and hotfix branches. These branches should be built, but not produce an artifact. They should just be used to inform the developer if there is a problem with their code.
In a standard build, I have access to the $GIT_BRANCH variable to know which branch is being built, but that variable isn't set in my multibranch pipeline. I have tried env.GIT_BRANCH too, and I tried to pass $GIT_BRANCH as a parameter to the build. Nothing seems to work. I assumed that since the build knows about the branch being built (I can see the branch name at the top of the console output) that there is something that I can use - I just can't find any reference to it.
The env.BRANCH_NAME variable contains the branch name.
As of Pipeline Groovy Plugin 2.18, you can also just use BRANCH_NAME
(env isn't required but still accepted.)
There is not a dedicated variable for this purpose yet (JENKINS-30252). In the meantime you can take advantage of the fact that the subproject name is taken from the branch name, and use
env.JOB_NAME.replaceFirst('.+/', '')
This has now been resolved, see Krzysztof KrasoĊ's answer.
There are 2 branches to consider in a Jenkins multibranch pipeline job:
The Jenkins job branch - env.BRANCH_NAME. This may have the same name as a git branch, but might also be called PR-123 or similar
The git branch - env.GIT_BRANCH. This is the actual branch name in git.
So a job might have BRANCH_NAME=PR-123 and GIT_BRANCH=my-scm-branch-name
Jenkins documentation has a list of all the env variable for your perusal here
Another way is using the git command to obtain the branch name on the current jenkins pipeline. For example, you can add the following snippet to print the branch name in your Jenkinsfile.
...
script {
def BRANCH = sh(returnStdout: true, script: 'git rev-parse --abbrev-ref HEAD').trim()
echo ${BRANCH}
}
...
I found this stackoverflow post example useful: Git Variables in Jenkins Workflow plugin
sh '//...
git rev-parse --abbrev-ref HEAD > GIT_BRANCH'
git_branch = readFile('GIT_BRANCH').trim()
echo git_branch
//...
'