Jenkins pipeline store SSH credentials as parameter - jenkins

I have the following pipeline code:
pipeline {
agent any
environment {
SSH_KEY_PARAMETER = credentials('SSH_KEY')
}
stages {
stage('Checkout') {
steps{
script{
git credentialsId: 'SSH_KEY_PARAMETER', url: 'git#github.com:SomeUser/PrivateRepo.git', branch: 'main'
}
}
}
}
}
Expected output is cloning the repo.
Actual output:
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git#github.com:SomeUser/PrivateRepo.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1001)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1242)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1302)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- git#github.com:SomeUser/PrivateRepo.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2681)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2102)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:86)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:624)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:999)
If I change the pipeline code to use the credential ID instead of the variable it works:
pipeline {
agent any
environment {
SSH_KEY_PARAMETER = credentials('SSH_KEY')
}
stages {
stage('Checkout') {
steps{
script{
git credentialsId: 'SSH_KEY', url: 'git#github.com:SomeUser/PrivateRepo.git', branch: 'main'
}
}
}
}
}
In my pipeline I need to manage a bunch of SSH keys depending on the environment, so I need to be able to parameterize the SSH key, am I missing something here or is it not possible to assign an SSH key as variable?

Your environment directive with use of the credentials helper method is syntactically correct. Your syntax for accessing the variable is incorrect. You are assigning a literal string value 'SSH_KEY_PARAMETER' to the credentialsId argument. The actual value also is stored in the env object, so you would access it with the appropriate key env.SSH_KEY_PARAMETER. The usage would then look like:
credentialsId: env.SSH_KEY_PARAMETER
However, the git step is expecting the credentials ID for that argument and not the actual credentials. Therefore, you need to also assign the value of the credentialsId in the environment directive instead:
SSH_KEY_PARAMETER = 'SSH_KEY'
Putting it all together, we have:
environment {
SSH_KEY_PARAMETER = 'SSH_KEY'
}
stages {
...
git credentialsId: env.SSH_KEY_PARAMETER, url: 'git#github.com:SomeUser/PrivateRepo.git', branch: 'main'
}

Related

Authentication problem of my pipeline with my gitlab project

I am in multi-branch option with my jenkins and I have a problem of authentication to Gitlab. Here is my jenkins file :
pipeline {
agent any
environment {
registry = "*****#gmail.com/test"
registryCredential = 'test'
dockerImage = ''
}
stages {
stage('Cloning our Git') {
steps{
git 'https://gitlab.com/**********/*************/************.git'
}
}
stage('Build docker image') {
steps {
script {
dockerImage = docker.build registry + ":$BUILD_NUMBER"
}
}
}
stage('Deploy our image') {
steps{
script {
docker.withRegistry( '', registryCredential ){
dockerImage.push()
}
}
}
}
stage('Cleaning up') {
steps{
sh "docker rmi $registry:$BUILD_NUMBER"
}
}
}
}
This is the error I got:
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://gitlab.com/************/*******/***************.git +refs/heads/:refs/remotes/origin/" returned status code 128:
stdout:
stderr: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://gitlab.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
I would like to know how to authenticate with the jenkinsfile to gitlab or if you have a better solution for me I am interested. Thanks
If you follow the link provided in the error message, you end up here:
https://docs.gitlab.com/ee/user/profile/account/two_factor_authentication.html#troubleshooting
You need to create a Personal Access Token which is kind of a special ID to delegate access to parts of your account rights.
The documentation for PAT is here:
https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
In the Gitlab repository interface, it is under Settings > Access Tokens.
As you try to read an HTTPS repository, it seems you need to create a token with rights read_repository.
Then you should be able to access the repository with:
https://<my-user-id>:<my-pat>#gitlab.com/<my-account>/<my-project-name>.git

How to pass down variables to credential parameters in JenkinsFiles?

I'm trying to write a JenkinsFile that automatically will reach to a git repo via ssh and perform some actions but I want to make the repo and ssh key use variables with the ssh id stored in Jenkins but I seem to be missing the Jenkins documentation for how to pass down variables to Jenkins Files as I'm not able to pass values down into the credentials key. The variables being passed down to the sh commands resolve perfectly fine though...
Example Pipeline Below:
pipeline {
parameters {
string(name: 'SSH_priv', defaultValue: 'd4f19e34-7828-4215-8304-a2d1f87a2fba', description: 'SSH Credential with the private key added to Jenkins and the public key to the username stored in Git Server, this id can be found in the credential section of Jenkins post its creation.')
string(name: 'REPO', defaultValue: 'git#--------------------')
}
stages {
stage ('Output Variables'){
// checks I can get these variables
steps{
sh("echo ${params.SSH_priv}")
sh("echo ${params.REPO}")
}
}
stage('Do Something') {
steps {
// this below commented line, does not work.
// sshagent (credentials: ['${params.SSH_priv}']){
// this line does work
sshagent (credentials: ['d4f19e34-7828-4215-8304-a2d1f87a2fba']){
sh("git clone --mirror ${params.REPO} temp")
dir("temp"){
// start doing fancy stuff ...
....
....
}
}
}
}
The aim is a Pipeline that my fellow developers could call and will work with their own repos and own ssh id's that I'm not using. When I try to run this with the SSH_priv parameter passing down the value I get the below failure in Jenkins.
The JenkinsFile works perfectly fine with the credential id hard-coded- as shown below:
So after testing different things a friend solved this in sub 5 minutes. Quotation mark types matter in Groovy Script
Changing
sshagent (credentials: ['${params.SSH_lower}']){
To
sshagent (credentials: ["${params.SSH_lower}"]){
Solved the issue.
Better to use environment step in pipeline.
pipeline {
agent any
environment {
AN_ACCESS_KEY = credentials('an_access_key_id')
}
stages {
stage('Example') {
steps {
sh 'printenv'
}
}
}
}
And credentials should exist in jenkins with id an_access_key_id
Take a look at official documentation here

$Variable is not expanding in Library definition in Jenkinsfile

This Groovy Jenkinsfile script works
#Library('jenkins-library#master')_
pipeline {
agent { label 'my-host-01' }
stages {
...
...
But it don't work if I change the first line to use a variable $MY_BRANCH
#Library('jenkins-library#$MY_BRANCH')_
Got below error:
ERROR: Could not resolve $MY_BRANCH
hudson.plugins.git.GitException: Command "git rev-parse $MY_BRANCH^{commit}" returned status code 128:
stdout: $MY_BRANCH^{commit}
MY_BRANCH is a parameter to the job, so this variable is available, but its not getting expanded in Jeenkinsfile script, is there a different syntax to use?
My pipeline job is setup like below, the above code is in Jenkinsfile.

Terraform cannot pull modules as part of jenkins pipeline

I have a jenkinsfile that was working and able to deploy some infrastructure automatically with terraform. Unfortunately after adding a terraform module with a git source it stopped working with the following error:
+ terraform init -input=false -upgrade
Upgrading modules...
- module.logstash
Updating source "git::https://bitbucket.org/*****"
Error downloading modules: Error loading modules: error downloading 'https://bitbucket.org/*****': /usr/bin/git exited with 128: Cloning into '.terraform/modules/34024e811e7ce0e58ceae615c545a1f8'...
fatal: could not read Username for 'https://bitbucket.org': No such device or address
script returned exit code 1
The urls above were obfuscated after the fact. Below is the cut down module syntax:
module "logstash" {
source = "git::https://bitbucket.org/******"
...
}
Below is the Jenkinsfile:
pipeline {
agent {
label 'linux'
}
triggers {
pollSCM('*/5 * * * *')
}
stages {
stage ('init') {
steps {
sh 'terraform init -input=false -upgrade'
}
}
stage('validate') {
steps {
sh 'terraform validate -var-file="production.tfvars"'
}
}
stage('deploy') {
when {
branch 'master'
}
steps {
sh 'terraform apply -auto-approve -input=false -var-file=production.tfvars'
}
}
}
}
I believe this to be a problem with terraform internally using git to checkout the module but Jenkins has not configured the git client within the pipeline job itself. Preferably I would be able to somehow pass the credentials used by the multibranch pipeline job into the job itself and configure git but I am at a loss of how to do that. Any help would be appreciated.
So I found a non-ideal solution that requires you to specify the credentials inside your Jenkinsfile rather than automatically using the credentials used by the job for checkout.
withCredentials([usernamePassword(credentialsId: 'bitbucketcreds', passwordVariable: 'GIT_PASS', usernameVariable: 'GIT_USER')]) {
sh "git config --global credential.helper '!f() { sleep 1; echo \"username=${env.GIT_USER}\\npassword=${env.GIT_PASS}\"; }; f'"
sh 'terraform init -input=false -upgrade'
sh 'git config --global --remove-section credential'
}
The trick is to load the credentials into environment variables using the withCredentials block and then I used the answer from this question to set the credential helper for git to read in those creds. You can then run terraform init and it will pull down your modules. Finally it clears the modified git settings to hopefully avoid contaminating other builds. Note that the --global configuration here is probably not a good idea for most people but was required for me due to a quirk in our Jenkins agents.
If anyone has a smoother way of doing this I would be very interested in hearing it.

Clone from bitbucket private repository using jenkins Pipeline as code

im using jenikins pipeline as code to clone a git project which is in private bitbucket repository(stash repository). i used this code block to clone the project in my pipeline script.
node {
//checkout from master
stage 'checkout'
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'MyID', usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD']]) {
git url: 'https://paulrda#devMyCompany.org/stash/scm/test_automation.git' , branch: 'development'
}
}
'MyID' is the credential ID and my username and password is correct.i save my credentials in global credentials feature in jenkins. but i get this error when i build the jenkins task.
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://paulrda#devMyCompany.org/stash/scm/test_automation.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:803)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1063)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1094)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:83)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:73)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:221)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://paulrda#devMyCompany.org/stash/scm/test_automation.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: fatal: Authentication failed for 'https://paulrda#devMyCompany.org/stash/scm/test_automation.git/'
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1745)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1489)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:315)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:801)
In my mac machine under my paulrda account i can successfully clone my project using jenkins pipeline script but when i change to another account and run jenkins i get this error. still i cant understand why i get this error. please provide a solution to this problem.
my configurations.
Jenkins version : 2.19.2
Credentials Plugin : 2.1.8
Git plugin : 3.0.0
Git client plugin : 2.1.0
It's failing to authenticate because you are not passing the credentials to the git call correctly.
Since you are using the Git plugin and not a shell command, there's really no need to use withCredentials at all. You can pass the credentialsId directly to git call, like that:
stage('checkout') {
git credentialsId: 'MyID', url: 'https://devMyCompany.org/stash/scm/test_automation.git', branch: 'development'
}

Resources