pass variables between stages jenkins pipeline - jenkins

I'm creating a Jenkins pipeline for simple deployment into kubernetes cluster, I have my private Docker registry, in here I simply clone my repo and build my docker image and update build docker image id into kubernetes deployment manifest and deploy the pod. but I'm having trouble passing my build image id to next stage, I did some research and try to solve it so I managed to pass the id to next stage but when I try to add the new id to deployment manifests its empty.
here is my pipeline
pipeline {
environment {
BUILD_IMAGE_ID = ''
}
agent any
stages {
stage('Cloning Git') {
steps {
git( url: 'https://xxxxxx.git',
credentialsId: 'id',
branch: 'master')
}
}
stage('Login Docker Registry') {
steps{
script {
sh 'docker login --username=xxxx --password=xxxx registry.xxxx.com'
}
}
}
stage('Building Image') {
steps{
script {
def IMAGE_ID = sh script:'docker run -e REPO_APP_BRANCH=xxxx -e REPO_APP_NAME=xxx --volume /var/run/docker.sock:/var/run/docker.sock registry.xxxx/image-build', returnStdout: true
println "Build image id: ${IMAGE_ID} "
BUILD_IMAGE_ID = IMAGE_ID.replace("/n","")
env.BUILD_IMAGE_ID = BUILD_IMAGE_ID
}
}
}
stage('Integration'){
steps{
script{
echo "passed: ${BUILD_IMAGE_ID} "
//update deployment manifests with latest docker tag
sh 'sed -i s,BUILD_ID,${BUILD_IMAGE_ID},g deployment-manifests/development/Service-deployments.yaml'
}
}
}
}
}
I don't want to save that value into a file and read and do the operation
output
[Pipeline] echo
Build image id:
registry.xxxx.com/service:3426d51-baeffc2
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Integration)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
passed:
registry.xxxx.com/service:3426d51-baeffc2
[Pipeline] sh
[orderservice] Running shell script
+ sed -i s,BUILD_ID,,g deployment-manifests/development/service-deployments.yaml

Related

Dockerfile in Declarative pipeline job fails

Jenkins Version:-
Jenkins - 2.277.1 LTS.
My Dockerfile:-
FROM maven:3.6.0-jdk-13
RUN useradd -m -u 1000 -s /bin/bash jenkins
My Declarative Pipeline:-
pipeline {
agent {
label "VM-Linux-Agent"
}
environment {
DOCKERFILE = "Dockerfile"
}
stages {
stage("Checkout") {
steps {
git(
url: 'git#gitlab.company.com:maven-prj-group/mavenapp.git',
branch: "master"
)
}
}
stage("Build") {
agent {
dockerfile {
filename DOCKERFILE
args "-v $WORKSPACE:/var/maven"
}
}
steps {
sh "mvn clean install"
}
}
}
}
From Jenkins master i have configured Linux server as node VM-Linux-Agent and using this node pipeline job code checkout is happening and further using Dockerfile building a docker container then to run build and others steps on docker itself steps are not working. it shows below errors.
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/Dockerfile-Pipeline
[Pipeline] {
[Pipeline] isUnix
[Pipeline] readFile
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.nio.file.NoSuchFileException: /var/jenkins_home/workspace/Dockerfile-Pipeline/Dockerfile
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at hudson.FilePath.newInputStreamDenyingSymlinkAsNeeded(FilePath.java:2112)
at hudson.FilePath.read(FilePath.java:2097)
at hudson.FilePath.read(FilePath.java:2089)
at org.jenkinsci.plugins.workflow.steps.ReadFileStep$Execution.run(ReadFileStep.java:104)
at org.jenkinsci.plugins.workflow.steps.ReadFileStep$Execution.run(ReadFileStep.java:94)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Build step instead of running on docker container it is running on master then it fails. Since i want to run only build + test steps on docker container from my node system(This is Docker Host). So how do i fix this in my declarative pipeline? please let me know the way to do this. Thanks in advance.
In the pipeline code section global agent replaced with none and to checkout the code on specific slave(In my case VM-Linux-Agent) added as agent label.
The below code working for me.
pipeline {
agent none
stages {
stage("Checkout") {
agent {
label "VM-Linux-Agent"
}
steps {
git(
url: 'git#gitlab.company.com:maven-prj-group/mavenapp.git',
branch: "master"
)
}
}
stage("Build") {
agent {
dockerfile {
filename 'Dockerfile'
label 'VM-Linux-Agent'
args "-v /home/user/maven:/var/maven"
}
}
steps {
sh "mvn clean install"
}
}
}
}

How to run the build inside docker container in Jenkins?

In my application I have a build script in package.json.
The build makes dist folder and inside I have my application.
I set Jenkins master and Jenkins agent as say in boxboat setup jenkins with docker and watch the video in youtube.
But now after I did this, I don't think my bash commands running inside a container.
I want to clone the repo and run npm i and npm run build - inside the docker container.
How I modify this configuration to able to do that?
throttle(['throttleDocker']) {
node('docker') {
wrap([$class: 'AnsiColorBuildWrapper']) {
try{
stage('Build') {
checkout scm
sh '''
echo "in Setup"
docker ps -a
echo "after docker"
# ./ci/docker-down.sh
# ./ci/docker-up.sh
'''
}
stage('Test'){
parallel (
"unit": {
sh '''
echo "in unit"
# ./ci/test/unit.sh
'''
},
"functional": {
sh '''
echo "in functional"
# ./ci/test/functional.sh
'''
}
)
}
stage('Capacity Test') {
sh '''
echo "in Capacity Test"
# ./ci/test/stress.sh
'''
}
}
finally {
stage('Cleanup') {
sh '''
echo "in Cleanup"
# ./ci/docker-down.sh
'''
}
}
}
}
}
I tried to this codes but they don't work. I also add agent after try.
stage('Build') {
agent {
docker {
label 'docker'
image 'node:latest'
}
}
steps {
checkout scm
sh 'node -v'
}
...
You can try below scripted pipeline
node {
docker.image('yourimage').inside {
stage('Build'){
sh 'echo "Build stage inside container"'
}
stage('Test'){
sh 'echo "Test Stage inside container"'
}
}
}

Jenkins pipeline fails to login to docker hub

I am trying to use pipeline to run a few things and when i started running my pipeline it failed to login to docker.
the weird thing is that i am able to login on the machine itself but when i run the pipeline is fails with this weird error:
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] stage
[Pipeline] { (Front-end)
[Pipeline] node
Running on test-env in /var/www/test-env/workspace/client-e2e
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withDockerRegistry
Using the existing docker config file.Removing blacklisted property: auths$ docker login -u ***** -p ******** https://hub.docker.com/?namespace=******
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: login attempt to https://hub.docker.com/v2/ failed with status: 404 Not Found
[Pipeline] // withDockerRegistry
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
ERROR: docker login failed
Finished: FAILURE
I don't know why it performs a login when this image is publicly available for everyone.
can someone help me ?
this is the pipeline itself:
pipeline {
agent none
stages {
stage('Front-end') {
agent {
docker {
image 'node:8-alpine'
label "test-env"
}
}
steps {
sh 'node --version'
}
}
}
}
You can try with Credentials Binding Plugin in:
steps {
sh 'node --version'
}
You can do:
withCredentials([string(credentialsId: 'mytoken', variable: 'TOKEN')]) {
sh '''
docker login -u '<your_user>' -p '<$TOKEN>'
node --version
'''
}
There is an example here:
https://issues.jenkins-ci.org/browse/JENKINS-41051
Ok, so after a while i found out that it was as simple as doing this
pipeline {
agent none
stages {
stage('Front-end') {
agent {
docker {
image 'node:8-alpine'
registryUrl 'https://index.docker.io/v1/'
label "test-env"
}
}
steps {
sh 'node --version'
}
}
}
}
this was aded: registryUrl 'https://index.docker.io/v1/'

Jenkins is adding private regisrty URL tag while pulling Image

I am getting this issue in Jenkins pipeline where I want to pull 'node' image but jenkins is adding the private docker registry url tag to it so the image is not found (artifactory.x.com/node:7-alpine)
Here is the pipeline
pipeline {
agent {
docker
{
image 'node:7-alpine'
registryUrl 'https://artifactory.x.com/'
registryCredentialsId 'jenkins-artifactory'
}
}
stages {
stage('Test') {
steps {
sh 'node --version'
}
}
}
}
This is the error I am getting
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/jobs/enterprise-master/workspace
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withDockerRegistry
$ docker login -u jenkins -p ******** https://artifactory.x.com/
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /var/jenkins_home/jobs/enterprise-master/workspace#tmp/f54c8b21-837b-4652-b12c-d489fb7e4c4c/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . node:7-alpine
Error: No such object: node:7-alpine
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . artifactory.x.com/node:7-alpine
Error: No such object: artifactory.x.com/node:7-alpine
[Pipeline] isUnix
[Pipeline] sh
+ docker pull artifactory.x.com/node:7-alpine
Error response from daemon: unknown: Not Found
[Pipeline] }
[Pipeline] // withDockerRegistry
Now the problem is that there is no image artifactory.x.com/node:7-alpine so it cant be found.
How do I tell jenkins not to add the private repo URL while pulling.
Fixed this by removing Docker Registry URL and setting Registry credentials to none
Jenkins -> Manage Jenkins->Configure > Pipeline Model Definition
Also the pipeline definition is still same
pipeline {
agent {
docker
{
image 'node:7-alpine'
registryUrl 'https://artifactory.X.com/'
registryCredentialsId 'jenkins-artifactory'
}

Docker Container Jenkins Pipeline Script: Permission denied while executing the script

I am running jenkins inside a docker container. I have created a simple pipleline to checkout,build and run docker image, but I am getting the following error.
Below is my pipleline script:
node {
def mvnHome = tool name: 'Maven Path', type: 'maven'
stage('Git CheckOut') {
git branch: '2019_DOCKER_SERVICES', credentialsId: 'git-creds', url: 'http://10.10.10.84:8111/scm/git/JEE_M_SERVICES'
}
stage('Maven Build') {
// Run the maven build
withEnv(["MVN_HOME=$mvnHome"]) {
if (isUnix()) {
sh '"$MVN_HOME/bin/mvn" -f Services/user-service/pom.xml clean install'
} else {
// bat(/"%MVN_HOME%\bin\mvn" -f Services\\user-service\\pom.xml clean install/)
}
}
}
stage('Docker Image Build') {
sh '"Services/user-service/" docker build -t user-service'
}
}
But I am getting the follow error in last stage, the first two stages ran successfully.
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Docker Image Build)
[Pipeline] sh
+ Services/user-service/ docker build -t user-service
/var/jenkins_home/jobs/docker-demo/workspace#tmp/durable-a5c035cf/script.sh: 1: /var/jenkins_home/jobs/docker-demo/workspace#tmp/durable-a5c035cf/script.sh: Services/user-service/: Permission denied
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
You have to set up new Jenkins slaves using Docker
It's weird to run Docker inside the Docker container
To access low-level operations you have to run your Docker container privileged

Resources