How to override an agent in a jenkins pipeline - jenkins

My goal is to create a pipeline, where every stage is having the same agent, except of 1 stage (Stage E in the example below)
pipeline {
agent {
dockerfile {
filename 'Dockerfile.tester'
args '-v $HOME/.docker:/root/.docker'
}
}
stages {
stage('A') { ... }
stage('B') { ... }
stage('C') { ... }
stage('D') { ... }
stage('E') {
agent {
dockerfile {
filename 'Dockerfile.deploy'
args '-v $HOME/.docker:/root/.docker'
}
}
}
stage('F') { ... }
}
}
What I can do so far is to set globally the agent to none and then set the agent Dockerfile.tester for each stage and for the 1 other stage I set it to Dockerfile.deploy. Any idea how I can set it globally once and then just override it?
Here the new error from if I do it like above:
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://bitbucket.org/*********/*********.git # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
Fetching without tags
Fetching upstream changes from https://bitbucket.org/*********/*********.git
> git --version # timeout=10
using GIT_ASKPASS to set credentials BitBucket Login for checkout / pushing
> git fetch --no-tags --progress https://bitbucket.org/*********/********.git +refs/heads/dev:refs/remotes/origin/dev
Checking out Revision ab0d4a522d872a129f53a74f6ecadafb8fd82f11 (dev)
> git config core.sparsecheckout # timeout=10
> git checkout -f ab0d4a522d872a129f53a74f6ecadafb8fd82f11
Commit message: "chore: test"
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[Bitbucket] Notifying commit build result
docker login failed

Related

Jenkins - restarting a pipeline on a different node fails on 'git checkout' (git plugin)

There is a job that gets triggered on pull request creation or update. I do not have the permissions to edit the job itself. Sometimes it will fail due to network connection problems, timeouts, etc. In this case I'd like to be able to restart the build with the same parameters. However in case of a restart, the build fails when checking out. It also skips the merge step for some reason
Is it possible to force the git-plugin to 'start fresh' on every build through Jenkinsfile? I was hoping that 'CleanBeforeCheckout' extension would solve my problem, but unfortunately this did not help.
Now for the details. Here's what my checkout step looks like in Jenkinsfile:
gitBranches = [[name: 'origin/pull-requests/**'], [name:"origin/$fromBranch"]]
gitExtensions = [
[
$class: 'UserIdentity',
email: email,
name: name
],
[
$class: 'PreBuildMerge',
options:
[
fastForwardMode: 'FF',
mergeRemote : 'origin',
mergeTarget : toBranch
]
],
[
$class: 'CleanBeforeCheckout',
deleteUntrackedNestedRepositories: false
]
]
println "Cloning repo with url '$gitUrl'"
checkout changelog: true, poll: true, scm: [
$class : 'GitSCM',
branches : gitBranches,
doGenerateSubmoduleConfigurations: false,
extensions : gitExtensions,
gitTool : 'Default',
submoduleCfg : [],
userRemoteConfigs : [[credentialsId: credentialsId, url: gitUrl]]
]
On the first run, the git checkout never fails. Here's what it looks like:
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential <credential_id>
Cloning the remote Git repository
Cloning repository <repo_url>
> git init /home/jenkins/agent/workspace/<workspace_id> # timeout=10
Fetching upstream changes from <repo_url>
> git --version # timeout=10
> git --version # 'git version 2.7.1'
using GIT_ASKPASS to set credentials
> git fetch --tags --progress <repo_url> +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Merging Revision <my_feature_hash> (origin/feature/my_feature) to origin/develop, UserMergeOptions{mergeRemote='origin', mergeTarget='develop', mergeStrategy='DEFAULT', fastForwardMode='FF'}
Checking out Revision <target_hash> (origin/develop)
Commit message: "Merge commit '<my_feature_hash>' into HEAD"
First time build. Skipping changelog.
> git config remote.origin.url <repo_url> # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
<...>
Seen 27 remote branches
> git show-ref --tags -d # timeout=10
> git rev-parse origin/develop^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f origin/develop # timeout=10
> git remote # timeout=10
> git config --get remote.origin.url # timeout=10
using GIT_ASKPASS to set credentials
> git merge --ff <my_feature_hash> # timeout=10
> git rev-parse HEAD^{commit} # timeout=10
<...>
Seen 27 remote branches
> git show-ref --tags -d # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f <target_hash> # timeout=10
However, when restarting the build from the Jenkins UI, the job fails with such output:
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential <credential id>
Cloning the remote Git repository
Cloning repository <repository_url>
> git init /home/jenkins/agent/workspace/<workspace_id> # timeout=10
Fetching upstream changes <repository_url>
> git --version # timeout=10
> git --version # 'git version 2.7.1'
using GIT_ASKPASS to set credentials
> git fetch --tags --progress <repository_url> +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision <target_hash> (origin/develop)
> git config remote.origin.url <repository_url> # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
<...>
Seen 27 remote branches
> git show-ref --tags -d # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f <target_hash> # timeout=10
Could not checkout <target_hash>
The problem supposedly arises because the git-plugin saves the previous build state somehow. This is bad because my builds will run on different nodes every time.

Why Bitbucket souce code repository not getting cloned using Jenkins?

I am using latest Jenkins in my Linux box. I am trying to clone a Bitbucket repository and checkout master branch using Jenkins pipeline. What I am trying to achieve is, after getting my source code cloned, I can make use of it and proceed with build. Below is my code;
node {
stage('Build') {
git branch: 'master',
credentialsId: '12345678-1234-1234-1234-123456789123',
url: 'https://username#bitbucket.org/username/test.git'
}
stage('Test') {
sh """pwd"""
}
}
and below is my console output;
Started by user User
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/test_pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] git
The recommended git tool is: NONE
using credential 12345678-1234-1234-1234-123456789123
> git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/test_pipeline/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://username#bitbucket.org/username/test.git # timeout=10
Fetching upstream changes from https://username#bitbucket.org/username/test.git
> git --version # timeout=10
> git --version # 'git version 2.32.0'
using GIT_ASKPASS to set credentials bitbucket_token_1
> git fetch --tags --force --progress -- https://username#bitbucket.org/username/test.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision ae8839d74d91d4ae28ba37a4ec91fa2265058b75 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ae8839d74d91d4ae28ba37a4ec91fa2265058b75 # timeout=10
> git branch -a -v --no-abbrev # timeout=10
> git branch -D master # timeout=10
> git checkout -b master ae8839d74d91d4ae28ba37a4ec91fa2265058b75 # timeout=10
Commit message: "Initial commit"
> git rev-list --no-walk ae8839d74d91d4ae28ba37a4ec91fa2265058b75 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ pwd
/var/lib/jenkins/workspace/test_pipeline
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
When I cd to /var/lib/jenkins/workspace/test_pipeline I can see nothing. Why is this happening?

Jenkins Pipeline - Jenkins SCM step. Checkout returns invalid data not referring to parameters provided

I am setting up CI/CD for my project and decided to use Jenkins Pipeline. What I need basically:
Build from chosen branch(chosen from dropdown)
Execute some stages conditionally(depending on branch)
I configured pipeline to fetch Jenkinsfile from scm:
Then I decided to write cutom scm checkout as following:
stage('Checkout specific branch') {
steps {
echo "Checking out ${params.branchToBuild}..."
script {
def actualBranchCheckout = checkout([$class: 'GitSCM',
branches: [
[name: "${params.branchToBuild}"]
],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'LocalBranch', localBranch: "**"]],
submoduleCfg: [],
userRemoteConfigs: [
[credentialsId: "${env.REPO_SSH_CREDS_ID}", url: "${env.REPO_SSH_URL}"]
]
])
}
}
}
And I have sveral concerns. GIT_ environment variables are set when checking out branch for Jenkinsfile.
GIT_BRANCH=origin/master
GIT_COMMIT=11a5800b6ca31bc81545fa4874d73fa275c820c2, GIT_LOCAL_BRANCH=master, GIT_PREVIOUS_COMMIT=11a5800b6ca31bc81545fa4874d73fa275c820c2
That is fine, but when my checkout is performed the are not getting updated with actual values and remain the same.
Okay, the method itself returns map which may contain what I need so I can manually reset them, right?
But they turned out to be the same when checking out develop branch:
Here are the logs:
Obtained Jenkinsfile from git git#bitbucket.org:team/repo.git
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/Repo/repo_pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
using credential jenkins-bitbucket
Cloning the remote Git repository
Cloning repository git#bitbucket.org:team/repo.git
> git init /var/lib/jenkins/workspace/Repo/repo_pipeline # timeout=10
Fetching upstream changes from git#bitbucket.org:team/repo.git
> git --version # timeout=10
using GIT_SSH to set credentials
> git fetch --tags --progress git#bitbucket.org:team/repo.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url git#bitbucket.org:team/repo.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url git#bitbucket.org:team/repo.git # timeout=10
Fetching upstream changes from git#bitbucket.org:team/repo.git
using GIT_SSH to set credentials
> git fetch --tags --progress git#bitbucket.org:team/repo.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 31e727bf81b3379c9aa80224b8f941bc867acd77 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 31e727bf81b3379c9aa80224b8f941bc867acd77
Commit message: "print out variables"
> git rev-list --no-walk 11a5800b6ca31bc81545fa4874d73fa275c820c2 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Checkout specific branch)
[Pipeline] echo
Checking out develop...
[Pipeline] script
[Pipeline] {
[Pipeline] checkout
using credential jenkins-bitbucket
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git#bitbucket.org:team/repo.git # timeout=10
Fetching upstream changes from git#bitbucket.org:team/repo.git
> git --version # timeout=10
using GIT_SSH to set credentials
> git fetch --tags --progress git#bitbucket.org:team/repo.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse origin/develop^{commit} # timeout=10
Checking out Revision 11a5800b6ca31bc81545fa4874d73fa275c820c2 (origin/develop)
> git config core.sparsecheckout # timeout=10
> git checkout -f 11a5800b6ca31bc81545fa4874d73fa275c820c2
> git branch -a -v --no-abbrev # timeout=10
> git checkout -b develop 11a5800b6ca31bc81545fa4874d73fa275c820c2
Commit message: "another try"
First time build. Skipping changelog.
[Pipeline] echo
[GIT_BRANCH:origin/master, GIT_COMMIT:31e727bf81b3379c9aa80224b8f941bc867acd77, GIT_LOCAL_BRANCH:master, GIT_PREVIOUS_COMMIT:11a5800b6ca31bc81545fa4874d73fa275c820c2, GIT_PREVIOUS_SUCCESSFUL_COMMIT:11a5800b6ca31bc81545fa4874d73fa275c820c2, GIT_URL:git#bitbucket.org:team/repo.git]
I have another option to do everything manually, like git rev-parse --abbrev-ref HEAD for branch and so on. Do anyone know how to solve this/ is it a known issue or I am doing something wrong way?

Simple Jenkins application to show gcc version

I am after a simple Jenkin application to show gcc version after each push to bitbucket.
Here is the Jenkinsfile:
pipeline {
agent { docker { image 'kernelci/build-gcc-7_arm:3.3.3' } }
stages {
stage('build') {
steps {
sh 'gcc --version'
}
}
}
}
Bitbucket
Repository Settings/Post Webhooks:
http://jenkins.SOMEADDRESS:8080/bitbucket-scmsource-hook/notify
After each push, the build does not start automatically but not a problem. I starts the builds manually.
However, The Jenkin build output result looks like this
Started by user MY_USERNAME
[Office365connector] No webhooks to notify
Building on master in workspace c:\w\PROJECTNAME
> git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git.exe config remote.origin.url http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git # timeout=10
Fetching upstream changes from http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git
> git.exe --version # timeout=10
using GIT_ASKPASS to set credentials Credentials for Jenkins in Bitbucket
> git.exe fetch --tags --progress http://bb.MYWEBSITE.COM/scm/bbb/MYREPO.git +refs/heads/*:refs/remotes/origin/*
Seen branch in repository origin/feature/LAST_BRANCH
Seen branch in repository origin/feature/BRANCH_A
Seen branch in repository origin/feature/BRANCH_B
Seen branch in repository origin/feature/BRANCH_C
Seen branch in repository origin/feature/BRANCH_D
Seen branch in repository origin/feature/BRANCH_E
Seen branch in repository origin/feature/BRANCH_F
Seen branch in repository origin/feature/BRANCH_G
Seen 8 remote branches
> git.exe show-ref --tags -d # timeout=10
Checking out Revision e3152f566543dfb813c6e33ac0d1099da7ec26a6 (origin/LAST_BRANCH)
> git.exe config core.sparsecheckout # timeout=10
> git.exe checkout -f e3152f566543dfb813c6e33ac0d1099da7ec26a6
Commit message: "Merge branch 'feature/BRANCH_A' into LAST_BRANCH"
First time build. Skipping changelog.
[Office365connector] No webhooks to notify
Finished: SUCCESS
It does not look like loading a dock in 4 seconds. I do not see any gcc version on the screen either. Where is the problem from?
Is it from Jenkinsfile?

Jenkins Does Not Execute Steps in Jenkinsfile

Why does jenkins doesn't execute my jenkinsfile? The console ouput of my build is this:
20:06:19 Started by GitLab push by
20:06:19 Building in workspace /var/jenkins_home/workspace/repo
20:06:19 [WS-CLEANUP] Deleting project workspace...
20:06:19 [WS-CLEANUP] Done
20:06:19 Cloning the remote Git repository
20:06:19 Cloning repository
20:06:19 > git init /var/jenkins_home/workspace/ # timeout=10
20:06:19 Fetching upstream changes from https://gitlab.com/
20:06:19 > git --version # timeout=10
20:06:19 using GIT_ASKPASS to set credentials
20:06:19 > git fetch --tags --progress https://gitlab.com/t+refs/heads/*:refs/remotes/origin/*
20:06:21 > git config remote.origin.url https://gitlab.com/ # timeout=10
20:06:21 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
20:06:21 > git config remote.origin.url https://gitlab.com/ # timeout=10
20:06:21 Fetching upstream changes from https://gitlab.com/
20:06:21 using GIT_ASKPASS to set credentials
20:06:21 > git fetch --tags --progress https://gitlab.com/ +refs/heads/*:refs/remotes/origin/*
20:06:22 skipping resolution of commit remotes/origin/jenkins, since it originates from another repository
20:06:22 > git rev-parse refs/remotes/origin/jenkins^{commit} # timeout=10
20:06:22 > git rev-parse refs/remotes/origin/origin/jenkins^{commit} # timeout=10
20:06:22 Checking out Revision 44f2afa95f961b5b28a482516abf05a17180ea48 (refs/remotes/origin/jenkins)
20:06:22 > git config core.sparsecheckout # timeout=10
20:06:22 > git checkout -f 44f2afa95f961b5b28a482516abf05a17180ea48
20:06:22 Commit message: "chore: test jenkins"
20:06:22 > git rev-list --no-walk dfff4c2214daa01de600086d6d28cf8f9d53cdc5 # timeout=10
20:06:22 Finished: SUCCESS
In my jenkins file I have this sample pipeline
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
Is there something I missed?
You need to choose pipeline type after click new item on jenkins page, rather than choose freestyle/maven type. Input a name and click 'OK', then choose pipeline from SCM

Resources