I want to edit the Helm deployment file in a step of a jenkins pipeline by using Sed.
stage('Updating Kubernetes deployment file'){
steps {
sh "cat deployment.yml"
sh "sed -i 's/${APP_NAME}.*/${APP_NAME}:${IMAGE_TAG}/g' deployment.yml"
sh "cat deployment.yml"
}
}
And then push the changes on the repo where I have all my charts like this:
stage('Push the changed deployment file to Git'){
steps {
script{
sh """
git config --global user.name "vikram"
git config --global user.email "username#gmail.com"
git add deployment.yml
git commit -m 'Updated the deployment file' """
withCredentials([usernamePassword(credentialsId: 'github', passwordVariable: 'pass', usernameVariable: 'user')]) {
sh "git push http://$user:$pass#github.com/papealioune/gitops-demo.git dev"
}
}
}
Thank you in advance
Related
So I have a Jenkins instance that I need to automatically tag a Docker Image with using a Jenkinsfile Pipeline that automatically tags the image with the commit hash and then pushes it to the Docker Repository. Jenkins is configured correctly, but my pipeline is still failing. At first, I tried using the following command, which returns the current commit hash of my repository.
git rev-parse --short=10 HEAD
Then I noticed this was returning more than one line, so I started using this:
git rev-parse --short=10 HEAD | tail -n +2
Which returns a commit hash similar to this:
338fcaa318b17c40dacf81dcf7a5826e3e3f0160
My goal is to tag my docker images with this hash using a Jenkinsfile:
pipeline {
agent any
environment {
tag = sh(returnStdout: true, script: "git rev-parse -short=10 HEAD | tail -n +2")
}
stages {
stage('Build core lodestone image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.lodestone -t mirantiseng/lodestone:${var.tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone:${var.tag}"
}
}
}
stage('Build core lodestone-comment image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile -t mirantiseng/lodestone-comment:${var.tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-comment:${var.tag}"
}
}
}
stage('Build core lodestone-mover image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.mover -t mirantiseng/lodestone-mover:${var.tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-mover:${var.tag}"
}
}
}
}
}
The build works if I take out the :${var.tag} and the following block, but it just pushes latest. This leaves the working file looking like this:
pipeline {
agent any
stages {
stage('Build core lodestone image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.lodestone -t mirantiseng/lodestone ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone"
}
}
}
stage('Build core lodestone-comment image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile -t mirantiseng/lodestone-comment ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-comment"
}
}
}
stage('Build core lodestone-mover image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.mover -t mirantiseng/lodestone-mover ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-mover"
}
}
}
}
}
But I need the docker images to be tagged. I read up on the Jenkinsfile and it said I could make an environmental variable using environment {<something>} to set global variables. I have a variable called tag that I would like to implement that tags the docker images with the commit hash. How can I accomplish this?
You could have used is just $GIT_COMMIT like this
sh "docker build -f Dockerfile.mover -t mirantiseng/lodestone-mover:${GIT_COMMIT} ."
So all I had to do was change the way tag was defined:
pipeline {
agent any
environment {
tag = sh(returnStdout: true, script: "git rev-parse --short=10 HEAD").trim()
}
stages {
stage('Build core lodestone image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.lodestone -t mirantiseng/lodestone:${tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone:${tag}"
}
}
}
stage('Build core lodestone-comment image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile -t mirantiseng/lodestone-comment:${tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-comment:${tag}"
}
}
}
stage('Build core lodestone-mover image') {
steps {
// TODO: proper tagging
sh "docker build -f Dockerfile.mover -t mirantiseng/lodestone-mover:${tag} ."
withCredentials([usernamePassword(credentialsId: 'common-dockerhub-up', usernameVariable: 'HUB_USER', passwordVariable: 'HUB_PASS')]) {
sh "docker login -u ${HUB_USER} -p ${HUB_PASS} && docker push mirantiseng/lodestone-mover:${tag}"
}
}
}
}
}
The main change was this
tag = sh(returnStdout: true, script: "git rev-parse --short=10 HEAD").trim()`
I created the following jenkinsfile with the help of Blue Ocean:
pipeline {
agent any
stages {
stage('git') {
steps {
git(url: 'git-url-to-code', branch: 'master', credentialsId: 'gitjenkins')
}
}
stage('Deploy') {
steps {
sshagent(credentials: ['cert']) {
sh 'ssh -tt -o StrictHostKeyChecking=no user#machine ls -trl'
sh 'ssh -o StrictHostKeyChecking=no user#machine rm -rf /apps/shiny/spielMA/ '
sh 'scp -rp ./. user#machine:/apps/shiny/spielMA/'
}
}
}
}
}
But it dose not work. to be more precise, my jenkins code is in repository J und der Code in the repo C and it can not trigger the pipeline if I push newe code in master in C!
Is there any possibility to trigger the pipeline, with a jenkinscode which is not in the repository of the code?
Yes. You can use the Remote Jenkinsfile Provider plugin.
I am trying to deploy k8s cluster using Helm 3 and jenkins. Jenkins and k8s running on different servers.I merged the kubeconfig files and I had all information in one config file ./kube directory. I would like to deploy my app to the related environment and namespace according to the GIT_BRANCH value. I have two question for below script.
1.What is the best way should I store k8s cluster credentials and will use in pipeline. I saw some plugins such as Kubernetes CLI but I can not be sure whether it will cover my requirement. If I use this plugin, should I store k8s file in to Jenkins machine manually or this plugin already handle this with uploading config file.
2.Should I change anything in below script to follow best practices?
stage('Deploy to dev'){
script{
steps{
if(env.GIT_BRANCH.contains("dev")){
def namespace="dev"
def ENV="development"
withCredentials([file(credentialsId: ...)]) {
// change context with related namespace
sh "kubectl config set-context $(kubectl config current-context) --namespace=${namespace}"
//Deploy with Helm
echo "Deploying"
sh "helm upgrade --install road-dashboard -f values.${ENV}.yaml --set tag=$TAG --namespace ${namespace}"
}
}
}
}
stage('Deploy to Test'){
script{
steps{
if(env.GIT_BRANCH.contains("test")){
def namespace="test"
def ENV="test"
withCredentials([file(credentialsId: ...)]) {
// change context with related namespace
sh "kubectl config set-context $(kubectl config current-context) --namespace=${namespace}"
//Deploy with Helm
echo "Deploying"
sh "helm upgrade --install road-dashboard -f values.${ENV}.yaml --set tag=$TAG --namespace ${namespace}"
}
}
}
}
}
stage ('Deploy to Production'){
when {
anyOf{
environment name: 'DEPLOY_TO_PROD' , value: 'true'
}
}
steps{
script{
DEPLOY_PROD = false
def namespace = "production"
withCredentials([file(credentialsId: 'kube-config', variable: 'kubecfg')]){
//Change context with related namespace
sh "kubectl config set-context $(kubectl config current-context) --namespace=${namespace}"
//Deploy with Helm
echo "Deploying to production"
sh "helm upgrade --install road-dashboard -f values.${ENV}.yaml --set tag=$TAG --namespace ${namespace}"
}
}
}
}
I have never tried this, but in theory the credentials variable is available as environment variable. Try to use KUBECONFIG as a variable name
withCredentials([file(credentialsId: 'secret', variable: 'KUBECONFIG')]) {
// change context with related namespace
sh "kubectl config set-context $(kubectl config current-context) --namespace=${namespace}"
//Deploy with Helm
echo "Deploying"
sh "helm upgrade --install road-dashboard -f values.${ENV}.yaml --set tag=$TAG --namespace ${namespace}"
}
A workaround that worked for me:
withCredentials([file(credentialsId: 'k8s-dk-staging', variable: 'KUBECRED')]) {
sh 'cat $KUBECRED > ~/.kube/config'
sh './deploy-app.sh'
}
I don't like to do that, ideally I would like use KUBECONFIG but for now this is that works for me.
I have the following pipelines file:
node('git') {
stage('Set Git Config') {
sh 'git config --global user.email "jenkins#test.com"'
sh 'git config --global user.name "jenkins"'
sh 'git config --global credential.helper cache'
sh "git config --global credential.helper 'cache --timeout=3600'"
}
stage('Set Git Credentials') {
git credentialsId: 'gitlab1', url: '${GITLAB1_REPO}'
git credentialsId: 'gitlab2', url: '${GITLAB2_REPO}'
}
stage('Synchronize with Gitlab2'){
sh 'git clone --bare ${GITLAB1_REPO} tfs'
dir("tfs") {
//add a remote repository
sh 'git remote add --mirror=fetch second ${GITLAB2_REPO}'
// update the local copy from the first repository
sh 'git fetch origin --tags'
// update the local copy with the second repository
sh 'git fetch second --tags'
// sync back the second repository
sh 'git push second --all'
sh 'git push second --tags'
}
}
}
Stage 1 and Stage 2 work perfectly. Stage 3 fails with permission denied.
I find this strange because on Stage 2, I can already see what the last commit was so it indicates that the credentials do work. Why aren't they working on stage 3?
This is the error I am seeing:
git clone --bare git#bitbucket.test/test.git tfs Cloning
into bare repository 'tfs'... Permission denied (publickey). fatal:
Could not read from remote repository.
While in stage 2, I see:
git config core.sparsecheckout # timeout=10
git checkout -f 30f1a7d1b77ef64e1cd44eab11a6ef4541c23b43
git branch -a -v --no-abbrev # timeout=10
git branch -D master # timeout=10
git checkout -b master 30f1a7d1b77ef64e1cd44eab11a6ef4541c23b43 Commit message: "test commit"
Stage 1 - you add some settings in shell to local git
Stage 2 - you point to actual credentials to be used and use a Jenkins plugin - which would just work
Satge 3 - back to shell, no credentials provided from jenkins, so the context is slave/local jenkins user.
Solution would be to use withCredentials for username and password or sshagent(credentials...) for private key
// credentialsId here is the credentials you have set up in Jenkins for pushing
// to that repository using username and password.
withCredentials([usernamePassword(credentialsId: 'git-pass-credentials-ID', passwordVariable: 'GIT_PASSWORD', usernameVariable: 'GIT_USERNAME')]) {
sh("git tag -a some_tag -m 'Jenkins'")
sh('git push https://${GIT_USERNAME}:${GIT_PASSWORD}#<REPO> --tags')
}
// For SSH private key authentication, try the sshagent step from the SSH Agent plugin.
sshagent (credentials: ['git-ssh-credentials-ID']) {
sh("git tag -a some_tag -m 'Jenkins'")
sh('git push <REPO> --tags')
}
I am unable to login via Jenkins pipeline on my docker my code as follow.
// Build Docker image
stage 'Build and Push code'
withCredentials(
[[
$class: 'UsernamePasswordMultiBinding',
credentialsId: 'rkstar007',
passwordVariable: 'DOCKERHUB_PASSWORD',
usernameVariable: 'DOCKERHUB_USERNAME'
]]
) {
sh "docker login -u '${env.DOCKERHUB_USERNAME}' -p '${env.DOCKERHUB_PASSWORD}'"
sh "docker build -t rkstar007/mesosphere:${gitCommit()} ."
sh "docker push rkstar007/mesosphere:${gitCommit()}"
}