How to push code back to git in a Jenkins Pipeline script? - jenkins

How can I check out code from git like this
stage('Checkout code') {
checkout scm
}
and modify the repository in the subsequent phase? Things like a git tag or a version commit.

Best I could come up with.
stage('Stage Checkout') {
checkout([
$class : 'GitSCM',
branches : scm.branches,
doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
extensions : [] + [
$class : 'LocalBranch'
],
userRemoteConfigs : scm.userRemoteConfigs,
])
}
//do git stuff
stage('Push Version Back to Git') {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'jenkins', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh 'echo ${GIT_AUTHOR_NAME} pushing '
sh 'git config --global user.email "user#test.com"'
sh 'git config --global user.name "Jenkins"'
sh 'git config --global push.default simple'
sh('git push https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#github.com/a/a.git')
}
}

Having it as follows made it working to me. And another thing to remember is, if your password or username contains any special characters, make sure that they have been saved in the credentials with the encoded characters. Other it's never gonna work.
This happened to me too and I wasted hours... Crazzzy...
pipeline {
// ..
stages {
stage('Clone') {
steps {
sh "git config --global user.email 'root#my-domain.com'"
sh "git config --global user.name 'My Name'"
sh 'git config --global push.default simple'
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '123123ghghjg13123', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh('git clone https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#my-repo.git')
}
}
}
stage('Build') {
steps {
// ...
}
}
stage('Publish') {
steps {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '123123ghghjg13123', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh('git push https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#my-repo.git --tags -f --no-verify')
}
}
}
}
}

Related

Cannot scm git pull jenkinsfile from private commit on gerrit

Team,
I want to pull the Jenkinsfile from my private commit which is on private branch on gerrit server. When i set it to main pull works but somehow pull of commit is not working.
steps
I cloned main
created a private branch
pushed Jenkinsfile as a commit to this private branch
added scm config as shown in jenkins gui.
but i am observing that it is not finding my commit. any hint what could be missing in my jenkins gui config? or is this not supported and jenkinsfile needs to be on main branch of the repo?
Couldn't find any revision to build. Verify the repository and branch configuration for this job.
error i get on jenkins log is
Checking out git ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test into /var/jenkins/workspace/repo_test/code-coverage-stage#script/7d000137d1c23eb647ed5a846ade258380196b11b7142f8fc6bebd5f9c212d93 to read src/jenkins/ci/Jenkinsfile
The recommended git tool is: git
using credential git-av-repo_test-ci-ssh
> git rev-parse --resolve-git-dir /var/jenkins/workspace/repo_test/code-coverage-stage#script/7d000137d1c23eb647ed5a846ade258380196b11b7142f8fc6bebd5f9c212d93/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test # timeout=10
Fetching upstream changes from ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test
> git --version # timeout=10
> git --version # 'git version 2.30.2'
using GIT_SSH to set credentials git-av-repo_test-ci-ssh
> git fetch --tags --force --progress -- ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
> git rev-parse origin/b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
> git rev-parse b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
config on jenkins gui is
SCM: git
Branches to build: commit_sha_id << here if I put MAIN it checksout repo but does not work be
Script Path: src/jenkins/ci/Jenkinsfile
sample Jenkinsfile
def checkoutRepo(containerName = 'main') {
container(containerName) {
sh 'rm -rf archive; rm -rf tmp'
checkout(
[
$class: 'GitSCM',
branches: [[name: '$GERRIT_REFSPEC']],
// branches: [[name: params.COMMIT_SHA_ID]],
extensions: [[
$class: 'BuildChooserSetting',
buildChooser: [$class: 'GerritTriggerBuildChooser'],
],
[$class: 'CloneOption', shallow: true, noTags: true, depth: 2, honorRefspec: true]
],
// userRemoteConfigs: scm.userRemoteConfigs
userRemoteConfigs: [[
credentialsId: 'git-product-ci-ssh',
name: 'origin',
refspec: '+$GERRIT_REFSPEC:$GERRIT_REFSPEC',
url: 'ssh://svc-product-org-sa#git.company.com:120/product']
]
]
)
}
}
def bazelInit(containerName = 'main') {
container(containerName) {
withCredentials([
string(credentialsId: 'product-ci-build-cache-jwt', variable: 'CACHE_TOKEN'),
usernamePassword(credentialsId: 'artifactory-build-suborg-ai-bazel', usernameVariable: 'ARTIFACTORY_USER', passwordVariable: 'ARTIFACTORY_TOKEN')
]) {
sh '''
BUILDAUTH_SILENT=1 ./buildauth static
set +e
BUILD_ID=nokill BUILD_TAG=nokill BUILD_VERSION=nokill BUILD_NUMBER= ./bazel version
if [ $? -ne 0 ]; then
./bazel shutdown || true
fi
'''.stripIndent()
}
}
}
pipeline {
agent {
label 'product-verify'
}
options {
// ansiColor('xterm')
parallelsAlwaysFailFast()
buildDiscarder( logRotator( artifactDaysToKeepStr: '30', artifactNumToKeepStr: '100', daysToKeepStr: '30', numToKeepStr: '100'))
throttleJobProperty(
throttleEnabled: true,
throttleOption: 'project',
maxConcurrentPerNode: 1,
maxConcurrentTotal: 20,
)
}
parameters {
booleanParam( name: 'Refresh', defaultValue: false, description: 'Reload job from the Jenkinsfile and then exit')
string( name: 'GERRIT_CREDENTIALS_ID', defaultValue: 'git-product-ci-http', description: 'Gerrit credentials')
string( name: 'GERRIT_API_URL', defaultValue: 'https://git.company.com/r', description: 'Gerrit API URL')
string(name: 'COMMIT_SHA_ID', defaultValue: '', description: 'commit ID')
}
stages {
stage('Checkout') {
steps {
preBuild(bazel_init)
script {
bazel_init = false
}
}
}
stage('Code Scan') {
agent {
label 'product-coverage-cli'
}
when { expression { !params.Refresh } }
steps {
checkoutRepo()
bazelInit()
container('main') {
sh '''
echo "Running on STAGE git"
sonar-scanner -v
echo "installed sonar scanner!"
echo "Start scan"
src/jenkins/ci/code-scan-all.sh
'''.stripIndent()
}
}
}
}
}
You can specify */main branch in job configured, and try the following workaround:
parameters { string(name: 'COMMIT_SHA_ID', defaultValue: '', description: 'commit ID') }
...
stages {
stage('Checkout') {
steps {
script {
if (params.COMMIT_SHA_ID?.trim()) {
echo "Build from ${params.COMMIT_SHA_ID}"
checkout([
$class: 'GitSCM',
branches: [[name: params.COMMIT_SHA_ID]],
userRemoteConfigs: scm.userRemoteConfigs])
}
} else {
echo "Build from main"
}
}
}
}
Might require a whitelisting/permission by an administrator to accessing scm fields.

restarting one stage in jenkins pipeline wiping out existing directory

I am using jenkins declarative pipeline jenkinsfile for our project. we want to try the option restart at stage.
pipeline {
agent { label 'worker' }
stages {
stage('clean directory') {
steps {
cleanWs()
}
}
stage('checkout') {
steps {
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: devops], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/devops.git]]])
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: harness], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/harness.git]]])
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: automation], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/automation.git]]])
}
}
stage('build initial commit to release train') {
steps {
sh '''#!/bin/bash
export TASK="build_initial_commit"
cd automation
sh main.sh
'''
}
}
stage('deploy application') {
steps {
sh '''#!/bin/bashexport TASK="deploy"
cd automation
sh main.sh
'''
}
}
}
}
and in jenkins I am using 'Pipeline script from SCM'. Jenkinsfile is present in automation.git repo (which is also defined in checkout stage)
Whenever I am restarting stage from GUI from 3rd one .. the workspace directory automatically gets cleaned up and it checksout automation.git ..
and the run fails as the other cloned repos were got cleaned...
how to handle this.. I want to restart the stage without wiping out the workspace dir..
if we just want to run the 3rd step 'deploy application' ..
I am not able to do , as the step depends on all 3 repos.. and
while restarting only 3rd stage the workspace is getting wiped out.. and as checkout is done in 1st stage(skipped) ... job is failing
how do I run only 3rd stage with retaining the old workspace ..
How about this:
SHOULD_CLEAN = true
pipeline {
agent { label 'worker' }
stages {
stage('clean directory') {
steps {
script {
if (SHOULD_CLEAN) {
cleanWs()
SHOULD_CLEAN = false
} else {
echo 'Skipping workspace clean'
}
}
}
}

Jenkins MultiBranch Pipeline: Select to build only 2 specific branches

I have a Jenkins MultiBranch project and I want the test circle to run only on two specific branches on master and on dev. I tried to add on all stages the following
when { anyOf { branch 'master'; branch 'dev' } }
but the only thing I managed to achieve was to deactivate all branch runs
Here is my full pipeline Jenkinsfile
pipeline {
agent any
triggers {
cron('H 0 * * *')
}
options {
disableConcurrentBuilds()
}
stages {
stage('Prepare env') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
sh 'rm -rf venv'
sh 'rm -rf "${WORKSPACE}/uploads"'
sh 'rm -rf "${WORKSPACE}/downloads"'
sh 'mkdir "${WORKSPACE}/uploads"'
sh 'mkdir "${WORKSPACE}/downloads"'
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
sh 'docker kill $(docker ps -q)'
sh 'docker rm $(docker ps -a -q)'
sh 'docker volume rm $(docker volume ls -q)'
}
}
}
stage('Start Services') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
}
}
stage('Test Common') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
}
}
stage('Test Validations') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
}
}
stage('Test CSV Issuance') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
}
}
stage('Test XLS Issuance') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
}
}
stage('Clean env') {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
sh 'rm -rf venv'
sh 'rm -rf "${WORKSPACE}/uploads"'
sh 'rm -rf "${WORKSPACE}/downloads"'
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
sh 'docker kill $(docker ps -q)'
sh 'docker rm $(docker ps -a -q)'
sh 'docker volume rm $(docker volume ls -q)'
}
}
}
}
Can you post the full pipeline you have?
You would use the when block on the stage you want run only on the two branches e.g.
pipeline {
agent any
stages
{
stage ("Testing") {
when {
anyOf {
branch 'master'
branch 'dev'
}
}
steps {
echo "run testing"
}
}
stage ("everything") {
steps{
echo "run on all branches"
}
}
}
}
tested pipeline
pipeline {
agent any
stages {
stage("stage") {
when { anyOf { branch 'master'; branch 'dev' } }
steps {
echo "Hello"
}
}
}
}
on master
[Pipeline] stage
[Pipeline] { (stage)
[Pipeline] echo
Hello
[Pipeline] }
[Pipeline] // stage
On Fish
[Pipeline] stage
[Pipeline] { (stage)
Stage "stage" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage

Jenkinsfile Credentials as parameter is not read

I am trying to use the credentials parameter for a git clone. But i am getting the error that the variables are not found
Param definition
credentials (credentialType: 'Username with password', defaultValue: 'fcb2d7c3-4b35-4ef2-bdf0-24fc4ff1137c', description: 'Git credential', name: 'git_credential', required: true)
Usage in stage
withCredentials([usernamePassword(credentialsId: params.git_credential, passwordVariable: 'DOCKER_PASSWORD', usernameVariable: 'DOCKER_USERNAME')]) {
sh 'git reset --hard; git clone https://${DOCKER_USERNAME}:${DOCKER_PASSWORD}#repo_url/scm/${params.repository}.git --branch master'
Error: Wrong variable used
This works for me:
pipeline {
agent any
parameters {
// credentials : defaultValue is Credentials ID
credentials(name: 'git_credential', description: 'Git credential', defaultValue: 'fcb2d7c3-4b35-4ef2-bdf0-24fc4ff1137c', credentialType: "Username with password", required: true )
}
stages {
stage('Print') {
steps {
withCredentials([usernamePassword(
credentialsId: '${git_credential}',
usernameVariable: 'DOCKER_USERNAME',
passwordVariable: 'DOCKER_PASSWORD',
)]) {
sh """
echo id:${DOCKER_USERNAME} pw:${DOCKER_PASSWORD}
"""
}
}
}
}
}

Best way to clone or pull gitlab code using Jenkins to avoid merge issues

What is the best way to clone or pull gitlab code using Jenkins, I have this pipeline. However i am seeing merge issues popping up and then it ignored other builds. What is the best approach to do this. Below is my pipeline and errors:
pipeline {
agent any
environment {
APPIUM_PORT_ONE= 4723
APPIUM_PORT_TWO= 4724
}
tools {nodejs "node"}
stages {
stage('Checkout App 1') {
steps {
dir("/Users/Desktop/app1") {
sh 'git pull ###'
}
echo "Building.."
}
}
stage('Checkout App 2') {
steps {
dir("/Users//Desktop/app2") {
echo "Building.."
sh 'git pull ###'
}
}
}
stage('Checkout Mirror') {
steps {
echo "Building.."
}
}
stage('Checkout End to End Tests') {
steps {
dir("/Users/Desktop/qa-end-to-end/") {
sh 'git pull ###'
}
}
}
stage('Starting Appium Servers') {
steps {
parallel(
ServerOne: {
echo "Starting Appium Server 1"
dir("/Users/Desktop/qa-end-to-end/") {
}
},
ServerTwo: {
echo "Starting Appium Server 2"
})
}
}
stage('Starting End to End Tests') {
steps {
echo "Starting End to End Tests"
dir("/Users/Desktop/qa-end-to-end/") {
sh './tests.sh'
echo "Shutting Down Appium Servers"
}
}
}
stage('Publish Report') {
steps {
echo "Publishing Report"
}
}
}
}
Should i clone from scratch instead of doing pull?. Any documentation would be helpful.
Unless the repos are large and time consuming to clone from scratch then I would do that.
Then you are certain that you have clean correct code to run with
checkout([$class: 'GitSCM',
branches: [[name: '*/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'CleanCheckout']],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'GIT', url: 'git#git.com:repo.git']]])
Can either run this in you DIR block or add the extension to checkout to a subdirectory
extensions: [[$class: 'RelativeTargetDirectory',
relativeTargetDir: 'checkout-directory']]
Dont forget to delete the old checkouts if you are persisting workspaces across builds.

Resources