Why doesn't the customWorkspace doesn't work for a new directory for example /tmp/checkout-directory?
If I choose /var/jenkins_home/workspace/44. pipeline-agent-dockerfile-customWorkspace/checkout-directory then it works.
pipeline {
agent none
stages {
stage('Checkout') {
agent any
steps {
checkout([$class: 'GitSCM',
branches: [[name: "origin/master"]],
userRemoteConfigs: [[
url: 'https://github.com/test/pipeline-agent-dockerfile.git' ]],
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'checkout-directory']]
])
}
}
stage('dockerfile') {
agent {
dockerfile {
customWorkspace '/tmp/checkout-directory'
}
}
steps {
sh 'cat /etc/lsb-release'
}
}
}
}
Error:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] stage
[Pipeline] { (Checkout)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/44. pipeline-agent-dockerfile-customWorkspace
[Pipeline] {
[Pipeline] checkout
The recommended git tool is: NONE
No credentials specified
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/test/pipeline-agent-dockerfile.git # timeout=10
Fetching upstream changes from https://github.com/test/pipeline-agent-dockerfile.git
> git --version # timeout=10
> git --version # 'git version 2.20.1'
> git fetch --tags --force --progress -- https://github.com/test/pipeline-agent-dockerfile.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 49b4a985b835c6bc63ed4d0a548c733c516444a0 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 49b4a985b835c6bc63ed4d0a548c733c516444a0 # timeout=10
Commit message: "Dockerfile"
> git rev-list --no-walk 49b4a985b835c6bc63ed4d0a548c733c516444a0 # timeout=10
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (dockerfile)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/44. pipeline-agent-dockerfile-customWorkspace
[Pipeline] {
[Pipeline] ws
Running in /tmp/checkout-directory
[Pipeline] {
[Pipeline] isUnix
[Pipeline] readFile
[Pipeline] }
[Pipeline] // ws
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
java.nio.file.NoSuchFileException: /tmp/checkout-directory/Dockerfile
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at hudson.FilePath.newInputStreamDenyingSymlinkAsNeeded(FilePath.java:2113)
at hudson.FilePath.read(FilePath.java:2098)
at hudson.FilePath.read(FilePath.java:2090)
at org.jenkinsci.plugins.workflow.steps.ReadFileStep$Execution.run(ReadFileStep.java:104)
at org.jenkinsci.plugins.workflow.steps.ReadFileStep$Execution.run(ReadFileStep.java:94)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Sadly I think you encountered a bug: https://issues.jenkins.io/browse/JENKINS-48319
A fix was implemented it seems but as you showed it still might not work: https://issues.jenkins.io/browse/JENKINS-53711
Related
Observations:
Jenkinsfile
pipeline {
agent { node 'master' }
stages {
stage('Hello') {
steps {
echo env.GIT_COMMIT
}
}
}
}
Output:
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Hello)
[Pipeline] echo
e5b4b2deed1c8db486cef5641ef14c61fd48f9ed
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Jenkinsfile
pipeline {
agent { node 'master' }
options {
skipDefaultCheckout(true)
}
stages {
stage('Hello') {
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[url: '/tmp/r']]])
echo env.GIT_COMMIT
}
}
}
}
Output:
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/HelloWorld_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Hello)
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
No credentials specified
> git rev-parse --resolve-git-dir /var/jenkins_home/workspace/HelloWorld_master/.git # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url /tmp/r # timeout=10
Fetching upstream changes from /tmp/r
> git --version # timeout=10
> git --version # 'git version 2.30.2'
> git fetch --tags --force --progress -- /tmp/r +refs/heads/*:refs/remotes/origin/* # timeout=10
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision e5b4b2deed1c8db486cef5641ef14c61fd48f9ed (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e5b4b2deed1c8db486cef5641ef14c61fd48f9ed # timeout=10
Commit message: "init"
First time build. Skipping changelog.
[Pipeline] echo
null
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Question:
I am using jenkins version 2.349 and the project is of type Multi-branch pipeline.
As I read on the Jenkins git plugin's page, it is supposed to inject the environment variables but it seems it only does when the default checkout isn't skipped but not when we explicitly checkout. I could only find ways to access those SCM values (in my case GIT_COMMIT) by using a script { } block. Is there a way to get those variables without jumping into a script block?
My Project is to launch instances on AWS using terraform script, the script is running well in local, i want to automate the process through jenkins and i put the code in git, below is the pipeline script i used in jenkins
pipeline{
agent any
tools {
terraform 'terraform'
}
stages{
stage('Git Checkout'){
steps{
git credentialsId: 'git credentials id', url: 'https://gitlab.com/ndey1/kafka-infra'
}
}
stage('Terraform init'){
steps{
sh 'terraform init'
}
}
stage('Terraform apply'){
steps{
sh 'terraform apply --auto approve'
}
}
}
}
but getting the error in jenkins
Started by user Company
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/infra-kaka
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Tool Install)
[Pipeline] tool
[Pipeline] envVarsForTool
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Git Checkout)
[Pipeline] tool
[Pipeline] envVarsForTool
[Pipeline] withEnv
[Pipeline] {
[Pipeline] git
The recommended git tool is: NONE
using credential "credential id"
Cloning the remote Git repository
Cloning repository https://gitlab.com/ndey1/kafka-infra
> git init /var/lib/jenkins/workspace/infra-kaka # timeout=10
Fetching upstream changes from https://gitlab.com/ndey1/kafka-infra
> git --version # timeout=10
> git --version # 'git version 2.25.1'
using GIT_ASKPASS to set credentials
> git fetch --tags --force --progress -- https://gitlab.com/ndey1/kafka-infra +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://gitlab.com/ndey1/kafka-infra # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Terraform init)
Stage "Terraform init" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Terraform apply)
Stage "Terraform apply" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
Finished: FAILURE
Terraform already installed in system, also all plugin updated
I found the fix, I used following lines in stage
stages{
stage('Git Checkout'){
steps{
git branch: 'main', credentialsId: 'gitlab id', url: 'https://gitlab.com/ndey1/kafka-infra'
}
}
It is able to fetch now
I have a Jenkins pipeline code that has 4 stages viz
1. CHECKOUT <--- git checkout only happens here
2. DEV
3. DEV2
3. DEV3
I do git checkout only in the first stage i.e CHECKOUT
Below is my pipeline code
stages {
stage('checkout') {
steps{
script {
def git_params = checkout([$class: 'GitSCM'])
println "*/${GIT_BRANCH}"
// Loading environment variables
checkout([
$class: 'GitSCM',
branches: [
[name: "*/${GIT_BRANCH}"]
],
extensions: [
[$class: 'WipeWorkspace'],
],
userRemoteConfigs: [[credentialsId: MYAPP_GIT_CREDENTIAL_ID, url: MYAPP_GIT_REPOSITORY]]
])
PIPELINE_NODE = NODE_NAME
dir('temp1'){
checkout([
$class: 'GitSCM',
branches: [
[name: '*/master']
],
userRemoteConfigs: [[credentialsId: MY_GIT_CREDENTIAL_ID, url: MY_GIT_REPOSITORY]]
])
}
stash name: "temp1", includes: "temp1/*"
//Load variable file from GitSCM inorder to get Maven variable
load "temp1/${APP_NAME}_${STARTENV}_EnvFile"
//load "temp1/${APP_NAME}_*_EnvFile"
sh 'chmod 755 temp1/git_version.sh'
env.APP_VERSION = sh(script: "temp1/git_version.sh", returnStdout: true).toString().trim()
sh 'echo REL#=$APP_VERSION'
}
}
}
}
stage('DEV'){
agent any
when {
branch 'develop'
}
steps{
script {
env.APP_VERSION = sh(script: "temp1/git_version.sh", returnStdout: true).toString().trim()
}
}
}
stage('DEV2'){
agent any
when {
branch 'develop2'
}
steps{
script {
env.APP_VERSION = sh(script: "temp1/git_version.sh", returnStdout: true).toString().trim()
}
}
}
stage('DEV3'){
agent any
when {
branch 'develop3'
}
steps{
script {
env.APP_VERSION = sh(script: "temp1/git_version.sh", returnStdout: true).toString().trim()
}
}
}
However, in the output you can see that the checkout can be seen multiple times in the logs when i expected it to show only one time. Although DEV & DEV2 stages are skipped the git checkout gets logged after the skip as you can see in the below output.
......
Stage "DEV" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (DEV2)
[Pipeline] node
Running on JENKINS_SLAVE5-SLAVE-01 in /apps/jenkins/workspace/ASERVE_release_2.0.4#2
[Pipeline] {
[Pipeline] checkout
The recommended git tool is: /bin/git
using credential JENKINS_CREDEN
Fetching changes from the remote Git repository
Fetching without tags
Checking out Revision a547d4a01537e9869ae3fccc8debf66a9343f723 (release/2.0.4)
Commit message: "VelocityAdapter_Jenkinsfile edited online with Bitbucket"
> /bin/git config remote.origin.url https://bitbucket.mycomp.com/scm/mhmwp/adapter.git # timeout=10
> /bin/git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> /bin/git config core.sparsecheckout # timeout=10
> /bin/git checkout -f a547d4a01537e9869ae3fccc8debf66a9343f723 # timeout=10
[Pipeline] withEnv
[Pipeline] {
> /bin/git rev-parse --is-inside-work-tree # timeout=10
> /bin/git config remote.origin.url https://bitbucket.mycomp.com/scm/mhmwp/adapter.git # timeout=10
Fetching upstream changes from https://bitbucket.mycomp.com/scm/mhmwp/adapter.git
> /bin/git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_ASKPASS to set credentials JENKINS_CREDEN
> /bin/git fetch --no-tags --progress https://bitbucket.mycomp.com/scm/mhmwp/adapter.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> /bin/git config core.sparsecheckout # timeout=10
> /bin/git checkout -f a547d4a01537e9869ae3fccc8debf66a9343f723 # timeout=10
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
Stage "DEV2" skipped due to when conditional
........
......
Can you please suggest ?
I have the folowing Pipeline Script that seems to be failing at some point. The CredentialId is correct, the github URL is correct, the path to dotnet is correct, and I'm like 99% sure the syntax is correct because the output (below) is reading my script correctly. So, I'm just wondering if I'm doing something incorrect.
I've never used Jenkins or any CI/CD tools before so this is all new to me.
My hopeful goal is to be able to get Jenkins to build and run a .NET app every-time something is merged with master, but I'm just trying to get this to build successfully every hour, or just to get it to build successfully when I click Build Now in Jenkins itself.
(Also, if there is an easy way to modify my script to run with every master merge, instead of every hour, that would be super helpful!)
pipeline{
agent any
environment {
dotnet ='C:\\Program Files (x86)\\dotnet\\'
}
triggers {
pollSCM 'H * * * *'
}
stages{
stage('Checkout') {
steps {
git credentialsId: '0b02e39b-17f0-4628-ba3d-24146ec7a469', url: 'https://github.com/bellingboe/OktaKenkinsCI.git', branch: 'master'
}
}
stage('Restore packages'){
steps{
bat "dotnet restore C:\\Users\\Brenden\\Documents\\GitHub\\OktaKenkinsCI\\OktaKenkinsCI.csproj"
}
}
stage('Clean'){
steps{
bat "dotnet clean C:\\Users\\Brenden\\Documents\\GitHub\\OktaKenkinsCI\\OktaKenkinsCI.csproj"
}
}
stage('Build'){
steps{
bat "dotnet build C:\\Users\\Brenden\\Documents\\GitHub\\OktaKenkinsCI\\OktaKenkinsCI.csproj --configuration Releas'"
}
}
}
}
which gives me the following output:
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in C:\WINDOWS\system32\config\systemprofile\AppData\Local\Jenkins\.jenkins\workspace\JenkinsPipeline
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Checkout)
[Pipeline] git
The recommended git tool is: NONE
using credential 0b02e39b-17f0-4628-ba3d-24146ec7a469
> git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git.exe config remote.origin.url https://github.com/bellingboe/OktaKenkinsCI.git # timeout=10
Fetching upstream changes from https://github.com/bellingboe/OktaKenkinsCI.git
> git.exe --version # timeout=10
> git --version # 'git version 2.29.2.windows.3'
using GIT_ASKPASS to set credentials
> git.exe fetch --tags --force --progress -- https://github.com/bellingboe/OktaKenkinsCI.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
> git.exe rev-parse "origin/master^{commit}" # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Restore packages)
Stage "Restore packages" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Clean)
Stage "Clean" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
Stage "Build" skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
Finished: FAILURE
Is there something in my script that's wrong? I can't really tell what would be failing as I'm not seeing any errors at all.
I figured it out. My master branch was named "main" in github.
and I had a typo!
I have a Jenkinsfile with the following :
node('buildprod_iamsg'){
echo "\u001B[34mSending status to dynamodb pib-deployments ...\u001B[0m"
checkout scm
def dynamo = load "aws/dynamo.groovy"
dynamo.add_metadata(substring, stripped_env, "down", lz, slackChannelName, slackToken, awsAccount, stack, ldapCommaSepList, githuburl, approver_name, time)
echo "\u001B[34mJob status sent to dynamodb pib-deployments\u001B[0m"
}
When I execute this It seems to checkout scm but then doesn't print anything. after and exits out. Inside my dynamo.groovy I have :
public add_metadata(substring, stripped_env, statuss, lz, slackChannelName, awsAccount, stack, ldapCommaSepList, githuburl, approver_name, time){
echo "reached here"
//--projection-expression lz
def expression_attribute_values = readFile("expression_attribute_values.json").trim()
echo expression_attribute_values
parseralgo(expression_attribute_values)
}
heres the result of execution :
Running on build_iamsg in /home/jenkins/workspace/djin_aweb_sample
[Pipeline] {
[Pipeline] echo
Sending status to dynamodb pib-deployments ...
[Pipeline] checkout
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://githubnet/djin-productivity/Deptwo.0.git/ # timeout=10
Fetching upstream changes from https://githubnet/djin-productivity/Deptwo.0.git/
> git --version # timeout=10
> git fetch --tags --progress https://githubnet/djin-productivity/Deptwo.0.git/ +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/dbint^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/dbint^{commit} # timeout=10
Checking out Revision 7daa4d196eafe4698f941f01631f5ccb13947793 (refs/remotes/origin/dbint)
Commit message: "adding dynamo integration with groovy load"
> git config core.sparsecheckout # timeout=10
> git checkout -f 7daa4d196eafe4698f941f01631f5ccb13947793
[Pipeline] load
[Pipeline] { (aws/dynamo.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // timeout
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: UNSTABLE
doesn't print reached here or the statement after in my jenkinsfile which is Job status sent to dynamodb pib-deployments
By the looks of it I think you forgot to add the slackToken parameter in your groovy script.
public add_metadata(substring, stripped_env, statuss, lz, slackChannelName, awsAccount, stack, ldapCommaSepList, githuburl, approver_name, time){