Jenkins Pipeline - Git checkout collision - jenkins

Question

We have a multi-stage pipeline in our CI, and some of the stages have their own nested stages that are parallelized and may run on the same or different agents (we request a certain agent label).
As with most CI pipelines, we build our artifacts and deploy and run our tests later.
As the pipeline may take some time to complete, we had an issue where new commits that are merged to our master branch may be picked up in the later stages, and it creates an incompatibility between the pre-packaged code and the new checkout-out one.
I'm currently using the skipDefaultCheckout directive and added my own function to checkout the commit SHA1 that is set in the GIT_COMMIT in every one of the parallel stages
void gitCheckoutByCommitHash(credentialsId, gitCommit=GIT_COMMIT) {
script {
println("Explicitly checking out git commit: ${gitCommit}")
}
checkout changelog: false, poll: false,
scm: [
$class: 'GitSCM',
branches: [[name: gitCommit]],
doGenerateSubmoduleConfigurations: false,
extensions: [
[
$class: 'CloneOption',
noTags: true,
shallow: true
],
[
$class: 'SubmoduleOption',
disableSubmodules: false,
parentCredentials: true,
recursiveSubmodules: true,
reference: '',
trackingSubmodules: false
],
],
submoduleCfg: [],
userRemoteConfigs: [[
credentialsId: credentialsId,
url: GIT_URL
]]
]
}
The problem I'm facing is that sometimes, two or more of the parallel stages are trying to run on the same agent and perform the checkout, and I get an error that a process has already retained .git/index.lock, and the stage that is locked out fails.
Is there any way to work around that?
This is a sample pipeline
pipeline {
agent {
label 'docker_v2'
}
options {
timestamps()
timeout(time: 1, unit: 'HOURS')
}
stages {
stage('Prepare test environment') {
options {
skipDefaultCheckout()
}
steps {
gitCheckoutByCommitHash('some-creds-id')
}
}
stage('Parallel stuff'){
parallel {
stage('Checkout 1') {
agent {
label 'docker_v2'
}
options {
skipDefaultCheckout()
}
steps {
gitCheckoutByCommitHash('some-creds-id')
}
}
stage('Checkout 2') {
agent {
label 'docker_v2'
}
options {
skipDefaultCheckout()
}
steps {
gitCheckoutByCommitHash('some-creds-id')
}
}
}
}
}
}

Answer 1

The best way to solve this issue would be to only perform the checkout once, and then use stash, with unstash in later stages:
pipeline {
agent {
label 'docker_v2'
}
options {
timestamps()
timeout(time: 1, unit: 'HOURS')
skipDefaultCheckout()
}
stages {
stage('Prepare test environment') {
steps {
// you can use this:
gitCheckoutByCommitHash('some-creds-id')
// or the usual "checkoutScm"
stash name: 'sources', includes: '**/*', allowEmpty: true , useDefaultExcludes: false
}
}
stage('Parallel stuff'){
parallel {
stage('Checkout 1') {
agent {
label 'docker_v2'
}
steps {
unstash 'sources'
}
}
stage('Checkout 2') {
agent {
label 'docker_v2'
}
steps {
unstash 'sources'
}
}
}
}
}
}
This would also speed up your pipeline.
To prevent collisions in the workspace, you can also use one of the following:
workspace directive so that different parallel stages would use different workspaces; or
Define your agents so that there is only one executor per agent; or
Both of the above.