I have a Jenkins pipeline with multiple stages that all require the same environment variables, I run this like so:
script {
withCredentials([usernamePassword(credentialsId: 'COMPOSER_REPO_MAGENTO', passwordVariable: 'MAGE_REPO_PASS', usernameVariable: 'MAGE_REPO_USER')]) {
def composerAuth = """{
"http-basic": {
"repo.magento.com": {
"username": "${MAGE_REPO_USER}",
"password": "${MAGE_REPO_PASS}"
}
}
}""";
// do some stuff here that uses composerAuth
}
}
I don't want to have to re-declare composerAuth every time, so I want to store the credentials in a global variable, so I can do something like:
script {
// do some stuff here that uses global set composerAuth
}
I've tried putting it in the environment section:
environment {
DOCKER_IMAGE_NAME = "magento2_website_sibo"
withCredentials([usernamePassword(credentialsId: 'COMPOSER_REPO_MAGENTO', passwordVariable: 'MAGE_REPO_PASS', usernameVariable: 'MAGE_REPO_USER')]) {
COMPOSER_AUTH = """{
"http-basic": {
"repo.magento.com": {
"username": "${MAGE_REPO_USER}",
"password": "${MAGE_REPO_PASS}"
}
}
}""";
}
}
But (groovy noob as I am) that doesn't work. So what's the best approach on setting a globally accessible variable with credentials but only have to declare it once?
You can use credentials helper method of the environment section. For "Username and passwrd" type of credentials it assigns 2 additional environment variables. Example:
environment {
MAGE_REPO_CREDENTIALS = credentials('COMPOSER_REPO_MAGENTO')
COMPOSER_AUTH = """{
"http-basic": {
"repo.magento.com": {
"username": "${env.MAGE_REPO_CREDENTIALS_USR}",
"password": "${env.MAGE_REPO_CREDENTIALS_PSW}"
}
}
}"""
}
Read more
After a lot of search (and struggle), i came up with an easy workaround:
As better explained in the jenkins docs for Handling Credentials, when injecting a usernamePassword type credential into an environment variable named VAR_NAME, jenkins automatically generates two other variables ending with _USR and _PSW respectively for usernameVariable and passwordVariable parameters.
What i did was to inject my variables with the values from both USR and PSW new variables.
In #Giel Berkers case, it should be something like this:
environment {
DOCKER_IMAGE_NAME = "magento2_website_sibo"
COMPOSER_REPO_MAGENTO_CREDENTIAL = credentials('COMPOSER_REPO_MAGENTO')
COMPOSER_AUTH = """{
"http-basic": {
"repo.magento.com": {
"username": "${COMPOSER_REPO_MAGENTO_CREDENTIAL_USR}",
"password": "${COMPOSER_REPO_MAGENTO_CREDENTIAL_PSW}"
}
}
}""";
}
Here is how you can accomplish that
pipeline {
agent any
stages {
stage('first') {
steps {
script {
withCredentials([usernamePassword(credentialsId: 'COMPOSER_REPO_MAGENTO', passwordVariable: 'MAGE_REPO_PASS', usernameVariable: 'MAGE_REPO_USER')]) {
def user = env.MAGE_REPO_USER
def password = env.MAGE_REPO_PASS
//Initializing a global variable. Notice there is no def here
composerAuth = """{
"http-basic": {
"repo.magento.com": {
"username": "${user}",
"password": "${password}"
}
}
}"""
}
}
}
}
stage('second') {
steps {
script {
println composerAuth
}
}
}
}
}
I found this and it is helpful:
Source: https://wiki.jenkins.io/display/JENKINS/Credentials+Binding+Plugin
// Basic example
withCredentials([usernamePassword(credentialsId: 'amazon',
usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
//available as an env variable, but will be masked if you try to print it out any which way
sh 'echo $PASSWORD'
echo "${env.USERNAME}"
}
// You can also request multiple credentials in a single call
withCredentials([usernamePassword(credentialsId: 'amazon',
usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD'),
string(credentialsId: 'slack-url',
variable: 'SLACK_URL'),]) {
sh 'echo $PASSWORD'
echo "${env.SLACK_URL}"
}
// Older code might not use the new syntax (usernamePassword, string, ...) yet, and directly call the class:
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'amazon',
usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
//available as an env variable, but will be masked if you try to print it out any which way
sh 'echo $PASSWORD'
echo "${env.USERNAME}"
}
You may need to deal with multi-field credentials or vendor-specific credential types that the plugin does not (yet) support.
In this situation you have a couple of choices:
Use the closest standard multi-field credential (e.g. Username With Password) that fits your requirements.
Use a string credential, serialize all the fields into the secret value (e.g. as JSON or as a delimited string), and parse them in the job script. (This is a last resort when other methods don't work, e.g. when secret rotation would cause multiple fields to change.)
Example: Jenkins authenticates to Secrets Manager using the primary AWS credential (from the environment). You have a job that performs a particular AWS operation in a different account, which uses a secondary AWS credential. You choose to encode the secondary AWS credential as JSON in the string credential foo:
node {
withCredentials([string(credentialsId: 'foo', variable: 'secret')]) {
script {
def creds = readJSON text: secret
env.AWS_ACCESS_KEY_ID = creds['accessKeyId']
env.AWS_SECRET_ACCESS_KEY = creds['secretAccessKey']
env.AWS_REGION = 'us-east-1' // or whatever
}
sh "aws sts get-caller-identity" // or whatever
}
}
A typical example of a username password type credential (example from here) would look like:
withCredentials([usernamePassword(credentialsId: 'amazon', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
// available as an env variable, but will be masked if you try to print it out any which way
// note: single quotes prevent Groovy interpolation; expansion is by Bourne Shell, which is what you want
sh 'echo $PASSWORD'
// also available as a Groovy variable
echo USERNAME
// or inside double quotes for string interpolation
echo "username is $USERNAME"
}
ReadMore1
ReadMore2
Related
I have a jenkins stage which requires different credentials based on some parameters. The contents stays the same for each stage, the only change is the credentials, so I could achieve this by just having multiple stages with single credentials then just using those, but obviously not ideal.
I'm trying it essentially like this but jenkins doesn't like it
stage('test execute') {
steps {
withCredentials([
if (params.Env == 'env1') {
usernamePassword(credentialsId: 'creds1', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')
} else if (params.Env == 'env2') {
usernamePassword(credentialsId: 'creds2', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')
}
])
}
}
I get the below error though
WorkflowScript: 25: unexpected token: if # line 25, column 21.
if (params.Env == 'env1') {
^
so I'm not sure if there's any way to add conditional logic to withcredentials, or if I just have to have multiple stages with essentially the same code
I got around this by storing the credentialsId in an environment variable, and then using that env variable in the usernamePassword() statement.
def getDeploymentCreds(envName)
{
switch(envName) {
case ~/^Dev/: return 'MY_DEV_CREDS'
case ~/^QA/: return 'MY_QA_CREDS'
case ~/^Beta/: return 'MY_BETA_CREDS'
}
}
// ...
script {
env.DEPLOY_CREDS_ID = getDeploymentCreds(env.TARGET_ENVIRONMENT)
}
// ...
withCredentials([
usernamePassword(credentialsId: env.DEPLOY_CREDS_ID, passwordVariable: 'DEPLOY_PASS', usernameVariable: 'DEPLOY_USER')
])
I want to have this code with exactly this syntax in my pipeline script:
withXCredentials(id: 'some-cred-id', usernameVar: 'USER', passwordVar: 'PASS') {
//do some stuff with $USER and $PASS
echo "${env.USER} - ${env.PASS}"
}
Note that you can put any code within withXCredenitals to be executed. withXCredentials.groovy resides in my Jenkins shared library under vars folder and it will use
Jenkins original withCredentials:
//withXCredentials.groovy
def userVar = params.usernameVar
def passwordVar = params.passwordVar
def credentialsId = params.credentialsId
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: credentialsId, usernameVariable: usernameVar, passwordVariable: passwordVar]]) {
body()
}
I am still learning advanced groovy stuff but I can't work out how to do this.
Please note:
My question is more about the syntax in groovy and using Closure and the answer here is not what I am after. With that solution, I need to instantiate the class first and then call the method. So I'm trying to avoid doing something like this:
new WithXCredentials(this).doSomthing(credentialsId, userVar, passwordVar)
In Jenkins documentation it has an example of using closure:
// vars/windows.groovy
def call(Closure body) {
node('windows') {
body()
}
}
//the above can be called like this:
windows {
bat "cmd /?"
}
But it doesn't explain how to pass parameters like this
windows(param1, param2) {
bat "cmd /?"
}
See here
So after digging internet I finally found the answer. In case anyone needs the same thing. The following code will work:
// filename in shared lib: /vars/withXCredentials.groovy
def call(map, Closure body) {
def credentialsId = map.credentialsId
def passwordVariable = map.passwordVariable
def usernameVariable = map.usernameVariable
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: credentialsId, usernameVariable: usernameVariable, passwordVariable: passwordVariable]]) {
echo 'INSIDE withXCredentials'
echo env."${passwordVariable}"
echo env."${usernameVariable}"
body()
}
}
With this you can have the following in your pipeline:
node('name') {
withXCredentials([credentialsId: 'some-credential', passwordVariable: 'my_password',
usernameVariable: 'my_username']) {
echo 'Outside withXCredenitals'
checkout_some_code username: "$env.my_username", password: "$env.my_password"
}
}
I have a jenkins pipeline where I am executing different scripts at different stages. However in one stage I want to get the output of the stage to a variable where I want to pass that variable as an input to next stage . Here is my code in Jenkinsfile
timestamps
{
node('cf_slave')
{
checkout scm
stage('Download HA image from GSA')
{
withCredentials(usernamePassword(credentialsId: 'ssc4icp_GSA', usernameVariable: 'GSA_USERNAME', passwordVariable: 'GSA_PASSWORD')
{
environment {
script {
OUTPUT = """${sh(
returnStdout: true,
script: 'bash jenkins/try_install.sh $VAR_ABC'
)}"""
echo $OUTPUT
}
}
}
}
}
}
Here i am getting syntax error. I want to get the OUTPUT in OUTPUT variable and pass that to next stage. Please help me how to do that in a correct way
When referencing variable outside of a string you should not us a dollar sign ($). The code should be (including changes suggested by Matt):
timestamps
{
node('cf_slave')
{
checkout scm
stage('Download HA image from GSA')
{
withCredentials(usernamePassword(credentialsId: 'ssc4icp_GSA', usernameVariable: 'GSA_USERNAME', passwordVariable: 'GSA_PASSWORD'))
{
environment {
script {
OUTPUT = sh returnStdout: true,
script: "bash jenkins/try_install.sh $VAR_ABC"
echo OUTPUT
}
}
}
}
}
}
I'd like to use a withCredentials() block in a shared-variable ("vars/") script rather than directly in the Jenkins pipeline because this is a lower-level semantic of a particular library, and also may or may not be required depending on the situation. However, withCredentials (or, at least, that signature of it) doesn't appear to be in scope.
script:
def credentials = [
[$class: 'UsernamePasswordMultiBinding', credentialsId: '6a55c310-aaf9-4822-bf41-5500cd82af4e', passwordVariable: 'GERRIT_PASSWORD', usernameVariable: 'GERRIT_USERNAME'],
[$class: 'StringBinding', credentialsId: 'SVC_SWREGISTRY_PASSWORD', variable: 'SVC_SWREGISTRY_PASSWORD']
]
withCredentials(credentials) {
// ...
}
Console:
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: BuildagentInstallAndRun.withCredentials() is applicable for argument types: (java.util.ArrayList, org.jenkinsci.plugins.workflow.cps.CpsClosure2) values: [[[$class:UsernamePasswordMultiBinding, credentialsId:6a55c310-aaf9-4822-bf41-5500cd82af4e, ...], ...], ...]
Has anyone had any success with this?
I'm using a shared library rather than a shared variable, but I guess it is a similar situation.
I'm not using the $class parameter, but i'm calling directly one of the functions suggested by the pipeline snippet generator. You can have a list here. In the example below, I use the usernameColonPassword binding.
In the pipeline, I instantiate the class utilities and I pass this to the constructor. Then, in the library, I use the step object to access the pipeline steps (such as withCredentials or usernameColonPassword).
class Utilities implements Serializable {
def steps
Utilities(steps) {
this.steps = steps
}
def doArchiveToNexus(String credentials, String artifact, String artifact_registry_path){
try {
this.steps.withCredentials([steps.usernameColonPassword(credentialsId: credentials, variable: 'JENKINS_USER')]) {
this.steps.sh "curl --user " + '${JENKINS_USER}' + " --upload-file ${artifact} ${artifact_registry_path}"
}
} catch (error){
this.steps.echo error.getMessage()
throw error
}
}
}
You can try following:
import jenkins.model.*
credentialsId = '6a55c310-aaf9-4822-bf41-5500cd82af4e'
def creds = com.cloudbees.plugins.credentials.CredentialsProvider.lookupCredentials(
com.cloudbees.plugins.credentials.common.StandardUsernameCredentials.class, Jenkins.instance, null, null ).find{
it.id == credentialsId}
println creds.username
println creds.password
But it is not secure, everything will be in console log
I was able to obtain credentials inside the shared library with proper passwords masking with such code:
class Utilities implements Serializable {
def steps
Utilities(steps) {
this.steps = steps
}
def execute() {
this.steps.withCredentials(
bindings: [
this.steps.usernameColonPassword(
credentialsId: this.credentialsId,
variable: "unameColonPwd")
]) {
this.steps.sh "echo {this.steps.env.unameColonPwd}"
}
}
pipeline {
agent any
stages {
stage("foo") {
steps {
script {
env.RELEASE_SCOPE = input message: 'User input required', ok: 'Release!',
parameters: [choice(name: 'RELEASE_SCOPE', choices: 'patch\nminor\nmajor',
description: 'What is the release scope?')]
}
echo "${env.RELEASE_SCOPE}"
}
}
}
}
In this above code, The choice are hardcoded (patch\nminor\nmajor) -- My requirement is to dynamically give choice values in the dropdown.
I get the values from calling api - Artifacts list (.zip) file names from artifactory
In the above example, It request input when we do the build, But i want to do a "Build with parameters"
Please suggest/help on this.
Depends how you get data from API there will be different options for it, for example let's imagine that you get data as a List of Strings (let's call it releaseScope), in that case your code be following:
...
script {
def releaseScopeChoices = ''
releaseScope.each {
releaseScopeChoices += it + '\n'
}
parameters: [choice(name: 'RELEASE_SCOPE', choices: ${releaseScopeChoices}, description: 'What is the release scope?')]
}
...
hope it will help.
This is a cutdown version of what we use. We separate stuff into shared libraries but I have consolidated a bit to make it easier.
Jenkinsfile looks something like this:
#!groovy
#Library('shared') _
def imageList = pipelineChoices.artifactoryArtifactSearchList(repoName, env.BRANCH_NAME)
imageList.add(0, 'build')
properties([
buildDiscarder(logRotator(numToKeepStr: '20')),
parameters([
choice(name: 'ARTIFACT_NAME', choices: imageList.join('\n'), description: '')
])
])
Shared library that looks at artifactory, its pretty simple.
Essentially make GET Request (And provide auth creds on it) then filter/split result to whittle down to desired values and return list to Jenkinsfile.
import com.cloudbees.groovy.cps.NonCPS
import groovy.json.JsonSlurper
import java.util.regex.Pattern
import java.util.regex.Matcher
List artifactoryArtifactSearchList(String repoKey, String artifact_name, String artifact_archive, String branchName) {
// URL components
String baseUrl = "https://org.jfrog.io/org/api/search/artifact"
String url = baseUrl + "?name=${artifact_name}&repos=${repoKey}"
Object responseJson = getRequest(url)
String regexPattern = "(.+)${artifact_name}-(\\d+).(\\d+).(\\d+).${artifact_archive}\$"
Pattern regex = ~ regexPattern
List<String> outlist = responseJson.results.findAll({ it['uri'].matches(regex) })
List<String> artifactlist=[]
for (i in outlist) {
artifactlist.add(i['uri'].tokenize('/')[-1])
}
return artifactlist.reverse()
}
// Artifactory Get Request - Consume in other methods
Object getRequest(url_string){
URL url = url_string.toURL()
// Open connection
URLConnection connection = url.openConnection()
connection.setRequestProperty ("Authorization", basicAuthString())
// Open input stream
InputStream inputStream = connection.getInputStream()
#NonCPS
json_data = new groovy.json.JsonSlurper().parseText(inputStream.text)
// Close the stream
inputStream.close()
return json_data
}
// Artifactory Get Request - Consume in other methods
Object basicAuthString() {
// Retrieve password
String username = "artifactoryMachineUsername"
String credid = "artifactoryApiKey"
#NonCPS
credentials_store = jenkins.model.Jenkins.instance.getExtensionList(
'com.cloudbees.plugins.credentials.SystemCredentialsProvider'
)
credentials_store[0].credentials.each { it ->
if (it instanceof org.jenkinsci.plugins.plaincredentials.StringCredentials) {
if (it.getId() == credid) {
apiKey = it.getSecret()
}
}
}
// Create authorization header format using Base64 encoding
String userpass = username + ":" + apiKey;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
return basicAuth
}
I could achieve it without any plugin:
With Jenkins 2.249.2 using a declarative pipeline,
the following pattern prompt the user with a dynamic dropdown menu
(for him to choose a branch):
(the surrounding withCredentials bloc is optional, required only if your script and jenkins configuration do use credentials)
node {
withCredentials([[$class: 'UsernamePasswordMultiBinding',
credentialsId: 'user-credential-in-gitlab',
usernameVariable: 'GIT_USERNAME',
passwordVariable: 'GITLAB_ACCESS_TOKEN']]) {
BRANCH_NAMES = sh (script: 'git ls-remote -h https://${GIT_USERNAME}:${GITLAB_ACCESS_TOKEN}#dns.name/gitlab/PROJS/PROJ.git | sed \'s/\\(.*\\)\\/\\(.*\\)/\\2/\' ', returnStdout:true).trim()
}
}
pipeline {
agent any
parameters {
choice(
name: 'BranchName',
choices: "${BRANCH_NAMES}",
description: 'to refresh the list, go to configure, disable "this build has parameters", launch build (without parameters)to reload the list and stop it, then launch it again (with parameters)'
)
}
stages {
stage("Run Tests") {
steps {
sh "echo SUCCESS on ${BranchName}"
}
}
}
}
The drawback is that one should refresh the jenkins configration and use a blank run for the list be refreshed using the script ...
Solution (not from me): This limitation can be made less anoying using an aditional parameters used to specifically refresh the values:
parameters {
booleanParam(name: 'REFRESH_BRANCHES', defaultValue: false, description: 'refresh BRANCH_NAMES branch list and launch no step')
}
then wihtin stage:
stage('a stage') {
when {
expression {
return ! params.REFRESH_BRANCHES.toBoolean()
}
}
...
}
this is my solution.
def envList
def dockerId
node {
envList = "defaultValue\n" + sh (script: 'kubectl get namespaces --no-headers -o custom-columns=":metadata.name"', returnStdout: true).trim()
}
pipeline {
agent any
parameters {
choice(choices: "${envList}", name: 'DEPLOYMENT_ENVIRONMENT', description: 'please choose the environment you want to deploy?')
booleanParam(name: 'SECURITY_SCAN',defaultValue: false, description: 'container vulnerability scan')
}
The example of Jenkinsfile below contains AWS CLI command to get the list of Docker images from AWS ECR dynamically, but it can be replaced with your own command. Active Choices Plug-in is required.
Note! You need to approve the script specified in parameters after first run in "Manage Jenkins" -> "In-process Script Approval", or open job configuration and save it to approve
automatically (might require administrator permissions).
properties([
parameters([[
$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
name: 'image',
description: 'Docker image',
filterLength: 1,
filterable: false,
script: [
$class: 'GroovyScript',
fallbackScript: [classpath: [], sandbox: false, script: 'return ["none"]'],
script: [
classpath: [],
sandbox: false,
script: '''\
def repository = "frontend"
def aws_ecr_cmd = "aws ecr list-images" +
" --repository-name ${repository}" +
" --filter tagStatus=TAGGED" +
" --query imageIds[*].[imageTag]" +
" --region us-east-1 --output text"
def aws_ecr_out = aws_ecr_cmd.execute() | "sort -V".execute()
def images = aws_ecr_out.text.tokenize().reverse()
return images
'''.stripIndent()
]
]
]])
])
pipeline {
agent any
stages {
stage('First stage') {
steps {
sh 'echo "${image}"'
}
}
}
}
choiceArray = [ "patch" , "minor" , "major" ]
properties([
parameters([
choice(choices: choiceArray.collect { "$it\n" }.join(' ') ,
description: '',
name: 'SOME_CHOICE')
])
])