Jenkins git checkout on agent not working - jenkins

The Jenkins file in my github repository is used in a Jenkins Master/Slave environment.
I need to execute a testing command on a remote Jenkins Slave Server.
In my declarative pipeline the agent is called like this:
stage("Testautomation") {
agent { label 'test-device' }
steps {
bat '''
#ECHO ON
ECHO %WORKSPACE%
... '''
}
}
Before Jenkins can even execute a remote command, it starts checking out from version control. The checkout on Jenkins Master is no problem and working fine. But on this Jenkins Slave I always receive this error message.
using credential github-enterprise:...
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://...git # timeout=10
Fetching upstream changes from https://...git
> git --version # timeout=10
using GIT_ASKPASS to set credentials GitHub Enterprise Access Token
> git fetch --tags --force --progress --depth=1 -- https://...git +refs/heads/development:refs/remotes/origin/development # timeout=120
Checking out Revision ... (development)
> git config core.sparsecheckout # timeout=10
> git checkout -f ...
Could not checkout ...

The Declarative pipeline performs a SCM checkout on every agent by default. Check if Git is installed on the Jenkins slave.
Conversely, if you want the code to be checked out on master but not on the agent, disable the default checkout in the options directive and use the scm checkout step inside a stage.
pipeline {
agent { label 'master' }
options {
skipDefaultCheckout(true)
}
stages {
stage('Build') {
steps {
checkout scm
// do other stuff on master
}
}
stage("Testautomation") {
agent { label 'test-device' }
steps {
bat '''
#ECHO ON
ECHO %WORKSPACE%
'''
}
}
}
}
You can further customize the checkout behavior as described in this answer https://stackoverflow.com/a/42293620/8895640.

Related

Jenkins multibranch pipeline triggers pipeline on unrelated branches

I have a problem with Jenkins multibranch pipleline using JenkinsFile and the GIT plugin.
The problem is that every push to staging branch triggers the pipeline of master as well.
The desired behavior is that push to staging branch only triggers the pipleine for staging, and push to master branch only triggers the pipeline for master
This is my JenkinsFile
#!/usr/bin/env bash
pipeline {
agent any
triggers {
pollSCM('*/1 * * * *')
}
environment {
GCLOUD_PATH="/var/jenkins_home/GoogleCloudSDK/google-cloud-sdk/bin"
}
stages {
stage('Git Checkout'){
steps{
// Clean Workspace
cleanWs()
// Get source from Git
git branch: 'staging',
credentialsId: ****',
url: 'git#github.com:***/****.git'
}
}
stage('Update Staging') {
when {
branch 'staging'
}
environment{
INSTANCE="***"
}
steps {
sshagent(credentials : ['****']) {
sh 'ssh -tt -o StrictHostKeyChecking=no jenkins#"${INSTANCE}" sudo /opt/webapps/****/deploy.sh firstinstance'
}
}
}
stage('Update Production') {
when {
branch 'master'
}
environment{
gzone="us-central1-a"
}
steps {
sh '''
#!/bin/bash
echo "${BRANCH_NAME}"
export instances=$("${GCLOUD_PATH}"/gcloud compute instances list --filter="status:(running) AND tags.items=web" --format="value(name)")
FIRST=1
for instance in ${instances}
do
echo "### Running Instance: ${instance} ###"
if [[ $FIRST == 1 ]]; then
echo "first instance"
${GCLOUD_PATH}/gcloud compute ssh jenkins#${instance} --zone ${gzone} '--ssh-flag=-tt -i /root/.ssh/id_rsa -o StrictHostKeyChecking=no' --command="echo first"
else
${GCLOUD_PATH}/gcloud compute ssh jenkins#${instance} --zone ${gzone} '--ssh-flag=-tt -i /root/.ssh/id_rsa -o StrictHostKeyChecking=no' --command="sudo uptime"
fi
FIRST=0
done
'''
}
}
}
post {
success {
cleanWs()
}
}
}
I'll share some logs:
The is a log for master branch
http://34.69.57.212:8080/job/tinytap-server/job/master/2/pollingLog/ returns
Started on Dec 10, 2019 1:42:00 PM
Using strategy: Specific revision
[poll] Last Built Revision: Revision 12ecdbc8d2f7e7ff1f578b135ea0b23a28d7672d (master)
using credential ccb9a735-04d9-4aab-8bab-5c86fe0f363c
> git --version # timeout=10
using GIT_ASKPASS to set credentials
> git ls-remote -h -- https://github.com/tinytap/tinytap-web.git # timeout=10
Found 222 remote heads on https://github.com/tinytap/tinytap-web.git
[poll] Latest remote head revision on refs/heads/master is: 12ecdbc8d2f7e7ff1f578b135ea0b23a28d7672d - already built by 1
Using strategy: Default
[poll] Last Built Revision: Revision f693e358ce14bc5dfc6111e62ed88e6dd1d0dfc9 (refs/remotes/origin/staging)
using credential 17f45a89-da78-4969-b18f-cb270a526347
> git --version # timeout=10
using GIT_SSH to set credentials jenkins key
> git ls-remote -h -- git#github.com:tinytap/tinytap-web.git # timeout=10
Found 222 remote heads on git#github.com:tinytap/tinytap-web.git
[poll] Latest remote head revision on refs/heads/staging is: 907899a0e7e131e9416ee65aad041c8da111e2fe
Done. Took 1 sec
Changes found
The is a log for master branch, but only staging had a new commit :
http://34.69.57.212:8080/job/tt-server/job/master/3/pollingLog/ returns
Started on Dec 10, 2019 1:55:00 PM
Using strategy: Specific revision
[poll] Last Built Revision: Revision 12ecdbc8d2f7e7ff1f578b135ea0b23a28d7672d (master)
using credential ****-****-****-****-5c86fe0f363c
> git --version # timeout=10
using GIT_ASKPASS to set credentials
> git ls-remote -h -- https://github.com/tt/tt-web.git # timeout=10
Found 222 remote heads on https://github.com/tt/tt-web.git
[poll] Latest remote head revision on refs/heads/master is: 12ecdbc8d2f7e7ff1f578b135ea0b23a28d7672d - already built by 2
Using strategy: Default
[poll] Last Built Revision: Revision 907899a0e7e131e9416ee65aad041c8da111e2fe (refs/remotes/origin/staging)
using credential ****-****-****-****-cb270a526347
> git --version # timeout=10
using GIT_SSH to set credentials jenkins key
> git ls-remote -h -- git#github.com:tt/tt-web.git # timeout=10
Found 222 remote heads on git#github.com:tt/tt-web.git
[poll] Latest remote head revision on refs/heads/staging is: eab6e8bc6d8586084e9fe9856dec7fd8b31dd098
Done. Took 0.98 sec
Changes found
Notice "changes found" even though head did not change on master branch
Jenkins ver. 2.190.1
Git plugin ver 4.0.0
Git client plugin ver 2.9.0
I use this plugin - https://github.com/lachie83/jenkins-pipeline and it works fine for me. You need to have separate if blocks for each branch and then the stage block inside it. Example below:
#!/usr/bin/groovy
#Library('https://github.com/lachie83/jenkins-pipeline#master')
def pipeline = new io.estrado.Pipeline()
def cloud = pipeline.getCloud(env.BRANCH_NAME)
def label = pipeline.getPodLabel(cloud)
// deploy only the staging branch
if (env.BRANCH_NAME == 'staging') {
stage ('deploy to k8s staging') {
//Deploy to staging
}
}
// deploy only the master branch
if (env.BRANCH_NAME == 'master') {
stage ('deploy to k8s production') {
//Deploy to production
}
}
I think you have some logical omissions in your Jenkinsfile. As it currently stands, you poll SCM for changes. If any change is detected, first stage 'Git Checkout' will checkout staging branch (always). Then you have another stage which does something if the branch is 'staging' (which it is, because it's hardcoded to checkout that branch above) etc. This will be the first thing to fix - if SCM changes are detected, checkout the right branch. How - there are a few options. I usually use 'skipDefaultCheckout()' in 'options' together with explicit checkout in my first pipeline stage:
steps {
sshagent(['github-creds']) {
git branch: "${env.BRANCH_NAME}", credentialsId: 'github-creds', url: 'git#github.com:x/y.git'
}
}
The second thing is that you try to squeeze handling two different branches into a single Jenkinsfile. This is not how it should be done. Jenkins wil use Jenkinsfile from a given branch - just make sure Jenkinsfile on staging contains what you want it to contain, same with Jenkinsfile on master.
Hope it helps.

Jenkins file diffs with declarative pipeline and multibranch

I have a scenario where I manage two pipelines within the same repo, with two different Jenkinsfiles.
I have set up two Jenkins multibranch pipelines to handle the two different Jenkinsfiles, by path discovery and set up github webhooks to trigger a build when a PR on particular branches is created.
I have not found a way to get changes in files for a particular PR so I thought of leveraging git by doing git diff --name-status origin/master...HEAD but it fails, since Jenkins only checks out the target branch.
The logstash:
using credential github-user-token-uname-pwd
Fetching changes from the remote Git repository
Fetching without tags
> git rev-parse --is-inside-work-tree # timeout=10
> git config remote.origin.url https://github.com/myreponame # timeout=10
Fetching upstream changes from https://github.com/myreponame
> git --version # timeout=10
using GIT_ASKPASS to set credentials Github token in uname-pwd form used by jenkins to register and manage webhooks
> git fetch --no-tags --force --progress https://github.com/myreponame +refs/heads/BRANCH-X:refs/remotes/origin/BRANCH-X
Checking out Revision 440df9b08667458fa4ade4be57ecbf59a4 (BRANCH-X)
Commit message: "move post build where it belongs"
> git config core.sparsecheckout # timeout=10
> git checkout -f 440df9b08667458fa4ade4be57ecbf59a4
> git rev-list --no-walk ab28e843c0fc7807c4cbd2d6f612e5d76b # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $SECRET_ACCESS_KEY or $ACCESS_KEY_ID
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] sh
+ git diff --name-status origin/master...HEAD
and the error I get:
fatal: ambiguous argument 'origin/master...HEAD': unknown revision or path not in the working tree.
Is there a way to retrieve the files' changes of a PR within multibranch pipelines or get Jenkins to be able to discover the source branch of said PR?
You could try to fetch origin and then diff with FETCH_HEAD
{ git fetch origin master
git diff —name-status FETCH_HEAD...HEAD
}
So here is what I had to do.
The error I was facing was due to Jenkins default checkout behaviour with multibranch pipelines: it checks out and track a branch and that branch only, it does not fetch other branches.
I eventually added
options {
skipDefaultCheckout true
}
and a stage that checks out all the origin remote branches
stage('Checking out repo'){
steps {
script {
checkout(
[
$class: 'GitSCM',
branches: [[name: 'origin/FEATUREBRANCH*']],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'LocalBranch'],
[$class: 'CleanBeforeCheckout']],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'github-deploy-key',
url: 'git#github.com:myawesomerepo.git']]
]
)
}
}
}

Simple Jenkins application to show gcc version

I am after a simple Jenkin application to show gcc version after each push to bitbucket.
Here is the Jenkinsfile:
pipeline {
agent { docker { image 'kernelci/build-gcc-7_arm:3.3.3' } }
stages {
stage('build') {
steps {
sh 'gcc --version'
}
}
}
}
Bitbucket
Repository Settings/Post Webhooks:
http://jenkins.SOMEADDRESS:8080/bitbucket-scmsource-hook/notify
After each push, the build does not start automatically but not a problem. I starts the builds manually.
However, The Jenkin build output result looks like this
Started by user MY_USERNAME
[Office365connector] No webhooks to notify
Building on master in workspace c:\w\PROJECTNAME
> git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git.exe config remote.origin.url http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git # timeout=10
Fetching upstream changes from http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git
> git.exe --version # timeout=10
using GIT_ASKPASS to set credentials Credentials for Jenkins in Bitbucket
> git.exe fetch --tags --progress http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git +refs/heads/*:refs/remotes/origin/*
Seen branch in repository origin/feature/LAST_BRANCH
Seen branch in repository origin/feature/BRANCH_A
Seen branch in repository origin/feature/BRANCH_B
Seen branch in repository origin/feature/BRANCH_C
Seen branch in repository origin/feature/BRANCH_D
Seen branch in repository origin/feature/BRANCH_E
Seen branch in repository origin/feature/BRANCH_F
Seen branch in repository origin/feature/BRANCH_G
Seen 8 remote branches
> git.exe show-ref --tags -d # timeout=10
Checking out Revision e3152f566543dfb813c6e33ac0d1099da7ec26a6 (origin/LAST_BRANCH)
> git.exe config core.sparsecheckout # timeout=10
> git.exe checkout -f e3152f566543dfb813c6e33ac0d1099da7ec26a6
Commit message: "Merge branch 'feature/BRANCH_A' into LAST_BRANCH"
First time build. Skipping changelog.
[Office365connector] No webhooks to notify
Finished: SUCCESS
It does not look like loading a dock in 4 seconds. I do not see any gcc version on the screen either. Where is the problem from?
Is it from Jenkinsfile?

How do you print out the value of a class property in a Jenkinsfile?

I'm creating a Jenkinsfile for use with GitHub Enterprise. I used the GUI settings in the pipeline job to specify a Jenkinsfile from a GitHub repo.
I'm using the scripted syntax instead of the declarative syntax.
I am able to checkout the repo in my Jenkinsfile using checkout scm. I want to use some information about the checkout in my script, such as the branch name and commit hash. However, I can't figure out how to access variables of the scm class.
When I run the job, it fails in the Checkout stage. The checkout from git seems to work properly, but it fails without printing any errors. If I delete the echo scm.GIT_BRANCH line it works fine.
node {
stage('Checkout') {
checkout scm
echo scm.GIT_BRANCH
}
}
Here's the output:
Started by user spark
Obtained nightly/Jenkinsfile from git https://github.enterprise.instance.com/spark/ci_flow_test
[Pipeline] node
Running on jenkins-server in /home/spark/ci_flow_test/pipeline_test
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Checkout)
[Pipeline] checkout
> /apps/git/git18/bin/git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> /apps/git/git18/bin/git config remote.origin.url https://github.enterprise.instance.com/spark/ci_flow_test # timeout=10
Fetching upstream changes from https://github.enterprise.instance.com/spark/ci_flow_test
> /apps/git/git18/bin/git --version # timeout=10
using GIT_ASKPASS to set credentials
> /apps/git/git18/bin/git fetch --tags --progress https://github.enterprise.instance.com/spark/ci_flow_test +refs/heads/*:refs/remotes/origin/*
> /apps/git/git18/bin/git rev-parse refs/remotes/origin/working^{commit} # timeout=10
> /apps/git/git18/bin/git rev-parse refs/remotes/origin/origin/working^{commit} # timeout=10
Checking out Revision 396f172c6061ba2760a71cba817df24836ec7e3b (refs/remotes/origin/working)
Commit message: "try echo"
> /apps/git/git18/bin/git config core.sparsecheckout # timeout=10
> /apps/git/git18/bin/git checkout -f 396f172c6061ba2760a71cba817df24836ec7e3b
> /apps/git/git18/bin/git rev-list 778c36171927020bd1afbd7206d86bf94abd1ed8 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] mail
[Pipeline] echo
Post script
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE
You can use the checkout scm return value to get git infomation
node {
stage('Checkout') {
def d = checkout scm
echo "branch: " + d.GIT_BRANCH
echo "commit: " + d.GIT_COMMIT
}
}
// supported fields
GIT_AUTHOR_EMAIL
GIT_AUTHOR_NAME
GIT_BRANCH
GIT_COMMIT
GIT_COMMITTER_EMAIL
GIT_COMMITTER_NAME
GIT_LOCAL_BRANCH
GIT_PREVIOUS_COMMIT
GIT_PREVIOUS_SUCCESSFUL_COMMIT
GIT_URL

access workspace in jenkins pipeline

I have a jenkins pipeline job with the job config pulling Jenkinsfile from a repo . Once the job runs and pulls the jenkinsfile, it clones the repo and I can see it in workspace icon for the job.
Now when in Jenkinsfile I do cd ${workspace} and ls , it doesnt display anything. How do I access the workspace of the repo of the Jenkinsfile ? Or does it just store the Jenkinsfile itself ?
This is my Jenkinsfile :
node ("master"){
// Mark the code checkout 'Checkout'....
stage 'Checkout'
sh "pwd ; ls"
}
As I run it, I get the following log:
> GitHub pull request #282 of commit
> 0045a729838aae0738966423ff19c250151ed636, no merge conflicts. Setting
> status of 0045a729838aae0738966423ff19c250151ed636 to PENDING with url
> https://10.146.84.103/job/test1/ and message: 'Pull request builder
> for this security group pr started' Using context: SG Terraform
> Validate/Plan
> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository
> > git config remote.origin.url https://github.xxx.net/Terraform/djin-sg/ # timeout=10 Fetching
> upstream changes from https://github.xxx.net/Terraform/djin-sg/
> > git --version # timeout=10 using GIT_ASKPASS to set credentials wsjbuild
> > git fetch --tags --progress https://github.xxx.net/Terraform/djin-sg/
> +refs/heads/*:refs/remotes/origin/*
> > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9dd8491b7b18c47eac09cec5a4bff4f16df979bf (origin/master)
> > git config core.sparsecheckout # timeout=10
> > git checkout -f 9dd8491b7b18c47eac09cec5a4bff4f16df979bf First time build. Skipping changelog. [Pipeline] node Running on master in
> /var/lib/jenkins/workspace/test1 [Pipeline] { [Pipeline] stage
> (Checkout) Using the ‘stage’ step without a block argument is
> deprecated Entering stage Checkout Proceeding [Pipeline] wrap
> [Pipeline] { [Pipeline] sh [test1] Running shell script
> + cd /var/lib/jenkins/workspace/test1
> + ls
My question specifically is that to get the Jenkinsfile it clones the djin-sg repo. Its in the workspace as well. So When I do ls why does it show no files ?
As I go to the Jenkins job pipeline steps and open workspace in console I can see the full repo in workspace but I cant seem to access it in the job.
Try instead a Jenkins pipeline syntax, like:
pipeline {
agent { node { label 'master' } }
stages {
stage('After Checkout') {
steps {
sh 'pwd; ls'
}
}
}
}
You can do checkout scm to actually checkout a repository to the workspace or you can find it under ../${env.JOB_NAME}#script only on master.
It's better to always checkout checkout scm manually because slaves do not have ../$env.JOB_NAME#script folder.

Resources