Team,
I want to pull the Jenkinsfile from my private commit which is on private branch on gerrit server. When i set it to main pull works but somehow pull of commit is not working.
steps
I cloned main
created a private branch
pushed Jenkinsfile as a commit to this private branch
added scm config as shown in jenkins gui.
but i am observing that it is not finding my commit. any hint what could be missing in my jenkins gui config? or is this not supported and jenkinsfile needs to be on main branch of the repo?
Couldn't find any revision to build. Verify the repository and branch configuration for this job.
error i get on jenkins log is
Checking out git ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test into /var/jenkins/workspace/repo_test/code-coverage-stage#script/7d000137d1c23eb647ed5a846ade258380196b11b7142f8fc6bebd5f9c212d93 to read src/jenkins/ci/Jenkinsfile
The recommended git tool is: git
using credential git-av-repo_test-ci-ssh
> git rev-parse --resolve-git-dir /var/jenkins/workspace/repo_test/code-coverage-stage#script/7d000137d1c23eb647ed5a846ade258380196b11b7142f8fc6bebd5f9c212d93/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test # timeout=10
Fetching upstream changes from ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test
> git --version # timeout=10
> git --version # 'git version 2.30.2'
using GIT_SSH to set credentials git-av-repo_test-ci-ssh
> git fetch --tags --force --progress -- ssh://svc-repo_test-team-sa#git-av.company.com:12013/repo_test +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
> git rev-parse origin/b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
> git rev-parse b2c27446fdccb414fb1e92984d528179babf5e78^{commit} # timeout=10
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
config on jenkins gui is
SCM: git
Branches to build: commit_sha_id << here if I put MAIN it checksout repo but does not work be
Script Path: src/jenkins/ci/Jenkinsfile
sample Jenkinsfile
def checkoutRepo(containerName = 'main') {
container(containerName) {
sh 'rm -rf archive; rm -rf tmp'
checkout(
[
$class: 'GitSCM',
branches: [[name: '$GERRIT_REFSPEC']],
// branches: [[name: params.COMMIT_SHA_ID]],
extensions: [[
$class: 'BuildChooserSetting',
buildChooser: [$class: 'GerritTriggerBuildChooser'],
],
[$class: 'CloneOption', shallow: true, noTags: true, depth: 2, honorRefspec: true]
],
// userRemoteConfigs: scm.userRemoteConfigs
userRemoteConfigs: [[
credentialsId: 'git-product-ci-ssh',
name: 'origin',
refspec: '+$GERRIT_REFSPEC:$GERRIT_REFSPEC',
url: 'ssh://svc-product-org-sa#git.company.com:120/product']
]
]
)
}
}
def bazelInit(containerName = 'main') {
container(containerName) {
withCredentials([
string(credentialsId: 'product-ci-build-cache-jwt', variable: 'CACHE_TOKEN'),
usernamePassword(credentialsId: 'artifactory-build-suborg-ai-bazel', usernameVariable: 'ARTIFACTORY_USER', passwordVariable: 'ARTIFACTORY_TOKEN')
]) {
sh '''
BUILDAUTH_SILENT=1 ./buildauth static
set +e
BUILD_ID=nokill BUILD_TAG=nokill BUILD_VERSION=nokill BUILD_NUMBER= ./bazel version
if [ $? -ne 0 ]; then
./bazel shutdown || true
fi
'''.stripIndent()
}
}
}
pipeline {
agent {
label 'product-verify'
}
options {
// ansiColor('xterm')
parallelsAlwaysFailFast()
buildDiscarder( logRotator( artifactDaysToKeepStr: '30', artifactNumToKeepStr: '100', daysToKeepStr: '30', numToKeepStr: '100'))
throttleJobProperty(
throttleEnabled: true,
throttleOption: 'project',
maxConcurrentPerNode: 1,
maxConcurrentTotal: 20,
)
}
parameters {
booleanParam( name: 'Refresh', defaultValue: false, description: 'Reload job from the Jenkinsfile and then exit')
string( name: 'GERRIT_CREDENTIALS_ID', defaultValue: 'git-product-ci-http', description: 'Gerrit credentials')
string( name: 'GERRIT_API_URL', defaultValue: 'https://git.company.com/r', description: 'Gerrit API URL')
string(name: 'COMMIT_SHA_ID', defaultValue: '', description: 'commit ID')
}
stages {
stage('Checkout') {
steps {
preBuild(bazel_init)
script {
bazel_init = false
}
}
}
stage('Code Scan') {
agent {
label 'product-coverage-cli'
}
when { expression { !params.Refresh } }
steps {
checkoutRepo()
bazelInit()
container('main') {
sh '''
echo "Running on STAGE git"
sonar-scanner -v
echo "installed sonar scanner!"
echo "Start scan"
src/jenkins/ci/code-scan-all.sh
'''.stripIndent()
}
}
}
}
}
You can specify */main branch in job configured, and try the following workaround:
parameters { string(name: 'COMMIT_SHA_ID', defaultValue: '', description: 'commit ID') }
...
stages {
stage('Checkout') {
steps {
script {
if (params.COMMIT_SHA_ID?.trim()) {
echo "Build from ${params.COMMIT_SHA_ID}"
checkout([
$class: 'GitSCM',
branches: [[name: params.COMMIT_SHA_ID]],
userRemoteConfigs: scm.userRemoteConfigs])
}
} else {
echo "Build from main"
}
}
}
}
Might require a whitelisting/permission by an administrator to accessing scm fields.
Related
I am using jenkins declarative pipeline jenkinsfile for our project. we want to try the option restart at stage.
pipeline {
agent { label 'worker' }
stages {
stage('clean directory') {
steps {
cleanWs()
}
}
stage('checkout') {
steps {
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: devops], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/devops.git]]])
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: harness], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/harness.git]]])
checkout([$class: 'GitSCM', branches: [[name: 'develop']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: automation], [$class: 'LocalBranch', localBranch: "**"]], userRemoteConfigs: [[credentialsId: 'xxxxxx', url: git#github.com/test/automation.git]]])
}
}
stage('build initial commit to release train') {
steps {
sh '''#!/bin/bash
export TASK="build_initial_commit"
cd automation
sh main.sh
'''
}
}
stage('deploy application') {
steps {
sh '''#!/bin/bashexport TASK="deploy"
cd automation
sh main.sh
'''
}
}
}
}
and in jenkins I am using 'Pipeline script from SCM'. Jenkinsfile is present in automation.git repo (which is also defined in checkout stage)
Whenever I am restarting stage from GUI from 3rd one .. the workspace directory automatically gets cleaned up and it checksout automation.git ..
and the run fails as the other cloned repos were got cleaned...
how to handle this.. I want to restart the stage without wiping out the workspace dir..
if we just want to run the 3rd step 'deploy application' ..
I am not able to do , as the step depends on all 3 repos.. and
while restarting only 3rd stage the workspace is getting wiped out.. and as checkout is done in 1st stage(skipped) ... job is failing
how do I run only 3rd stage with retaining the old workspace ..
How about this:
SHOULD_CLEAN = true
pipeline {
agent { label 'worker' }
stages {
stage('clean directory') {
steps {
script {
if (SHOULD_CLEAN) {
cleanWs()
SHOULD_CLEAN = false
} else {
echo 'Skipping workspace clean'
}
}
}
}
I have a stage in Jenkins as follows, How do I mark the build to fail or unstable if there is a test case failure? I generated the script pipeline for textfinder plugin but it is not working. "findText alsoCheckConsoleOutput: true, regexp: 'There are test failures.', unstableIfFound: true" not sure where to place the textFinder regex.
pipeline {
agent none
tools {
maven 'maven_3_6_0'
}
options {
timestamps ()
buildDiscarder(logRotator(numToKeepStr:'5'))
}
environment {
JAVA_HOME = "/Users/jenkins/jdk-11.0.2.jdk/Contents/Home/"
imageTag = ""
}
parameters {
choice(name: 'buildEnv', choices: ['dev', 'test', 'preprod', 'production', 'prodg'], description: 'Environment for Image build')
choice(name: 'ENVIRONMENT', choices: ['dev', 'test', 'preprod', 'production', 'prodg'], description: 'Environment for Deploy')
}
stages {
stage("Tests") {
agent { label "xxxx_Slave"}
steps {
checkout([$class: 'GitSCM', branches: [[name: 'yyyyyyyyyyz']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'zzzzzzzzzzz', url: 'abcdefgh.git']]])
sh'''
cd dashboard
mvn -f pom.xml surefire-report:report -X -Dsurefire.suiteXmlFiles=src/test/resources/smoke_test.xml site -DgenerateReports=false
'''
}
}
}
}
All I did to make this request possible is as below:
added a post block of code below the steps block code.
post {
Success {
findText alsoCheckConsoleOutput: true, refexp: 'There are test failures.', unstableIfFound: true
}
}
What is the best way to clone or pull gitlab code using Jenkins, I have this pipeline. However i am seeing merge issues popping up and then it ignored other builds. What is the best approach to do this. Below is my pipeline and errors:
pipeline {
agent any
environment {
APPIUM_PORT_ONE= 4723
APPIUM_PORT_TWO= 4724
}
tools {nodejs "node"}
stages {
stage('Checkout App 1') {
steps {
dir("/Users/Desktop/app1") {
sh 'git pull ###'
}
echo "Building.."
}
}
stage('Checkout App 2') {
steps {
dir("/Users//Desktop/app2") {
echo "Building.."
sh 'git pull ###'
}
}
}
stage('Checkout Mirror') {
steps {
echo "Building.."
}
}
stage('Checkout End to End Tests') {
steps {
dir("/Users/Desktop/qa-end-to-end/") {
sh 'git pull ###'
}
}
}
stage('Starting Appium Servers') {
steps {
parallel(
ServerOne: {
echo "Starting Appium Server 1"
dir("/Users/Desktop/qa-end-to-end/") {
}
},
ServerTwo: {
echo "Starting Appium Server 2"
})
}
}
stage('Starting End to End Tests') {
steps {
echo "Starting End to End Tests"
dir("/Users/Desktop/qa-end-to-end/") {
sh './tests.sh'
echo "Shutting Down Appium Servers"
}
}
}
stage('Publish Report') {
steps {
echo "Publishing Report"
}
}
}
}
Should i clone from scratch instead of doing pull?. Any documentation would be helpful.
Unless the repos are large and time consuming to clone from scratch then I would do that.
Then you are certain that you have clean correct code to run with
checkout([$class: 'GitSCM',
branches: [[name: '*/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'CleanCheckout']],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'GIT', url: 'git#git.com:repo.git']]])
Can either run this in you DIR block or add the extension to checkout to a subdirectory
extensions: [[$class: 'RelativeTargetDirectory',
relativeTargetDir: 'checkout-directory']]
Dont forget to delete the old checkouts if you are persisting workspaces across builds.
I am using jenkins and having scripted syntax in jenkinsfile
In the main job after source checkout I need to run other job n times (parallel) with different inputs .
Any tips to start this?
def checkout(repo, branch) {
checkout(changelog: false,
poll: false,
scm: [$class : 'GitSCM',
branches : [[name: "*/${branch}"]],
doGenerateSubmoduleConfigurations: false,
recursiveSubmodules : true,
extensions : [[$class: 'LocalBranch', localBranch: "${branch}"]],
submoduleCfg : [], userRemoteConfigs: [[credentialsId: '', url: "${repo}"]]])
withCredentials([[$class : '',
credentialsId : '',
passwordVariable: '',
usernameVariable: '']]) {
sh "git clean -f && git reset --hard origin/${branch}"
}
}
node("jenkins02") {
stage('Checkout') {
checkout gitHubRepo, gitBranch
}
}
We do this by storing all the jobs we want to run in a Map and then pass it into the parallel step for execution. So you just setup the different params and add each definition into the map, then execute.
Map jobs = [:]
jobs.put('job-1', {
stage('job-1') {
node {
build(job: "myorg/job-1/master", parameters: [new StringParameterValue('PARAM_NAME','VAL1')], propagate: false)
}
}
})
jobs.put('job-2', {
stage('job-2') {
node {
build(job: "myorg/job-2/master", parameters: [new StringParameterValue('PARAM_NAME','VAL2')], propagate: false)
}
}
})
parallel(jobs)
How can I check out code from git like this
stage('Checkout code') {
checkout scm
}
and modify the repository in the subsequent phase? Things like a git tag or a version commit.
Best I could come up with.
stage('Stage Checkout') {
checkout([
$class : 'GitSCM',
branches : scm.branches,
doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations,
extensions : [] + [
$class : 'LocalBranch'
],
userRemoteConfigs : scm.userRemoteConfigs,
])
}
//do git stuff
stage('Push Version Back to Git') {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'jenkins', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh 'echo ${GIT_AUTHOR_NAME} pushing '
sh 'git config --global user.email "user#test.com"'
sh 'git config --global user.name "Jenkins"'
sh 'git config --global push.default simple'
sh('git push https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#github.com/a/a.git')
}
}
Having it as follows made it working to me. And another thing to remember is, if your password or username contains any special characters, make sure that they have been saved in the credentials with the encoded characters. Other it's never gonna work.
This happened to me too and I wasted hours... Crazzzy...
pipeline {
// ..
stages {
stage('Clone') {
steps {
sh "git config --global user.email 'root#my-domain.com'"
sh "git config --global user.name 'My Name'"
sh 'git config --global push.default simple'
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '123123ghghjg13123', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh('git clone https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#my-repo.git')
}
}
}
stage('Build') {
steps {
// ...
}
}
stage('Publish') {
steps {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '123123ghghjg13123', usernameVariable: 'GIT_AUTHOR_NAME', passwordVariable: 'GIT_PASSWORD']]) {
sh('git push https://${GIT_AUTHOR_NAME}:${GIT_PASSWORD}#my-repo.git --tags -f --no-verify')
}
}
}
}
}