In a Jenkinsfile, I am trying to set an environment variable by using withCredentials to call a shell script to return a list of data. I want to use this environment variable to populate a drop-down list.
environment {
withCredentials([
usernamePassword(
credentialsId: "aws-creds",
usernameVariable: "AWS_ACCESS_KEY_ID",
passwordVariable: "AWS_SECRET_ACCESS_KEY"
)
]) {
AWS_DATA = sh 'aws ec2 describe-security-groups --region my-region | grep "xxxx" | sort -u | cut -d\" -f4'
}
}
Any help appreciated.
Thanks,
B
Related
I have upstream and downstream pipelines and in downstream pipeline, I want to store user, who started upstream build, in downstream build variable.
I used following in downstream pipeline but it results into error - java.lang.NullPointerException: Cannot invoke method getUserId() on null object
Owner = currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
One way is to store it in a variable in upstream pipeline and pass it as param to downstream pipeline but that is additional handling and considering my parameters list, it is little tricky.
Is there any way I can get user who started upstream build in downstream build without passing parameter?
Similarly, JOB_NAME and BUILD_NUMBER of upstream to downstream.
Your current build would print the below as the build cause when triggered by a user.
env.current_BUILD_CAUSE = currentBuild.getBuildCauses().toString()
env.current_BUILD_CAUSE:
[
{
"_class": "hudson.model.Cause$UpstreamCause",
"shortDescription": "Started by upstream project \"ABC/XYZ/JobA\" build number 621",
"upstreamBuild": 621,
"upstreamProject": "ABC/XYZ/JobA",
"upstreamUrl": "job/ABC/job/XYZ/job/JobA/"
}
]
So now I'd take the necessary parameters from the above values. I'm not much familiar with the groovy way of doing it. So have used the sh directive.
env.upstream_BUILD_NUMBER = sh(script: 'echo \"${current_BUILD_CAUSE}\" | jq .[].upstreamBuild | tr -d \'"\'', returnStdout: true).trim()
env.upstream_JOB_NAME = sh(script: 'echo \"${current_BUILD_CAUSE}\" | jq .[].upstreamProject | tr -d \'"\'', returnStdout: true).trim()
env.upstream_JOB_URL = sh(script: 'echo \"${current_BUILD_CAUSE}\" | jq .[].upstreamUrl | tr -d \'"\'', returnStdout: true).trim()
Since, you don't want to pass parameters from one to another, I'd just curl the JSON api of the build and then use the same to get the values as required from upstream build. I'm using the credentials here since you may or may not need to authenticate to jenkins as per your requirement. You can remove that if not required along with -u arguments in curl.
withCredentials([usernamePassword(credentialsId: 'your-CredentialsID-for-Jenkins', usernameVariable: 'user', passwordVariable: 'password')]) {
env.upstream_OWNER_ID = sh(script: "curl -s -u $user:$password https://${JENKINS_DASHBOARD}/${upstream_JOB_URL}/{upstream_BUILD_NUMBER}/api/json | jq -r \'.actions[].causes[]?.userId | select(length>0)\'", returnStdout: true).trim()
}
Now I'll have everything that is needed from both current and upstream build causes without passing any parameters from or to.
Note: The code samples above also contains the escape sequences which I'm currently using in my jenkins job.
I am writing a sensitive data to a .yml file as a string in my Jenkins groovy file. I have attached the .groovy file which would show what I am trying to achieve but it reveals the data in Pipeline steps which is also attached. Nothing shows in the console log. Value from AWS Parameter store is retrieved and passed as a string to a .yml file and this string value is shown in the pipeline steps. I am looking for a way to hide this from being shown in the Pipeline steps. I have tried using set +x but it is not hiding the string value in the pipeline steps. Any thoughts on how this can be achieved would be of great help.Screenshot
#!/usr/bin/env groovy
properties([
parameters([
string(name:'NODE_NAME', defaultValue: 'node', description: 'Node Name')
])
])
nodeName = params.NODE_NAME
node("${nodeName}") {
def AWS_REGION = "eu-west-2"
def paramPath = "path"
def paramName = "PASSWORD"
stage("Data from SSM") {
param = sh (script: "aws ssm get-parameter --name ${paramPath} --with-decryption --region ${AWS_REGION} | /var/jenkins_home/tools/jq .Parameter.Value | tr -d '\"'", returnStdout: true).trim()
sh('#!/bin/sh -e\n' + "echo '${paramName}: ${param}' >> vars.yml")
}
}
You can make use of credentials in jenkins. Go to Jenkins -> credentials -> global -> add credentials
Select either 'Username with password'/'secret text' depending on your need
This will generate a secret in jenkins
Then in your groovy file you can use the credentials as env variables which will not be printed by jenkins
withCredentials([usernamePassword(credentialsId: 'Your-Credential-ID', passwordVariable: 'pwdToken', usernameVariable: 'userToken')]){
//Use pwdToken as a variable here which will not be printed
}
I am not getting google credentials when i am using the below code.
def GOOGLE_CREDENTIALS = credentials('XYZ Credentials')
[Edited]
Please check this official guide to see if you have defined it correctly.
Also check if you're passing credentials ID, not a description to credentials() method.
If you're using Jenkins pipelines, you also can try Credentials Binding Plugin.
From plugin wiki, a typical example of a username password type credential would look like:
withCredentials([usernamePassword(credentialsId: 'amazon', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
// available as an env variable, but will be masked if you try to print it out any which way
// note: single quotes prevent Groovy interpolation; expansion is by Bourne Shell, which is what you want
sh 'echo $PASSWORD'
// also available as a Groovy variable
echo USERNAME
// or inside double quotes for string interpolation
echo "username is $USERNAME"
}
For scripted pipeline you can use withCredentials() as well (see this):
node {
withCredentials([usernameColonPassword(credentialsId: 'mylogin', variable: 'USERPASS')]) {
sh '''
set +x
curl -u "$USERPASS" https://private.server/ > output
'''
}
}
Or you can use withEnv() section:
node {
withEnv(["CREDS=credentials('jenkins-creds')"]) {
stage('Build') {
sh 'printenv'
}
}
}
Here is my working example
node {
stage('Preparing env') {
withCredentials([usernamePassword(credentialsId: 'XYZ Credentials', passwordVariable: 'GOOGLE_CREDENTIALS_PSW', usernameVariable: 'GOOGLE_CREDENTIALS_USR')]) {
// Do something here with your username and password variables
}
}
}
i am trying to use jenkins scripted pipeline to invoke config file provider plugin along with fetching credentials from jenkins for the username and password, but the below doesn't seem to work.
node {
def mvnHome
def mvnSettings
stage('Prepare') {
mvnHome = tool 'maven-3.5.4'
}
stage('Checkout') {
checkout scm
}
stage('Deploy'){
def usernameLocal, passwordLocal, usr, psw
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'xyz', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME']]) {
usernameLocal = env.USERNAME
passwordLocal = env.PASSWORD
}
configFileProvider(
[configFile(fileId: '*********', variable: 'MAVEN_SETTINGS', replaceTokens: true)])
{
usr="${usernameLocal}"
psw="${passwordLocal}"
sh "echo $usr"
sh "'${mvnHome}/bin/mvn' -s $MAVEN_SETTINGS deploy -Dserver.username="${usernameLocal}" -Dserver.password="${passwordLocal}""
}
}
}
where server.username and server.password are defined as properties under settings.xml server section for username and password.
Looks like i found out the issue and its nothing to do with withCredentials used here rather to do with the config file provider plugin. So i am able to print the credentials username correctly but somehow the config file provider is unable to substitute the variable value in the settings.xml.
so i don't get any error anymore, its just that the deployment doesn't go through with 401 unauthorized since the below in my settings.xml never gets the correct values :-
<server>
<id>snapshot</id>
<username>${server.username}</username>
<password>${server.password}</password>
</server>
Could you please advise how to resolve this?
The variables created by withCredentials are Groovy variables not environment variables. Try the following:
stage('Deploy'){
withCredentials([usernamePassword(credentialsId:'xyz', passwordVariable: 'Password', usernameVariable: 'Username')]) {
configFileProvider([configFile(fileId: 'abcde', variable:'MAVEN_SETTINGS')]) {
sh "'${mvnHome}/bin/mvn' -s $MAVEN_SETTINGS deploy -Dserver.username=${Username} -Dserver.password=${Password}"
}
}
}
Ok I figured out the solution, declare the configFileProvider entire section under the block of withCredentials and pass:
-Dserver.username='${usernameLocal}' -Dserver.password='${passwordLocal}'
(Please note single quotes). This way the values also get substituted and are outputted in the logs as masked.
I am trying to pass variables extracted in a stage in Jenkinsfile between stages. For example:
stage('Dummy Stage') {
sh '''#!/bin/bash -l
export abc=`output of some command`
.....
.....
'''
Now, how can I pass the variable abc to a subsequent stage? I have tried setting the variable by adding a def section at the top of the file but looks like it doesnt work. In the absence of a neater way, I am having to retype the commands
Here is what I do to get the number of commits on master as a global environment variable:
pipeline {
agent any
environment {
COMMITS_ON_MASTER = sh(script: "git rev-list HEAD --count", returnStdout: true).trim()
}
stages {
stage("Print commits") {
steps {
echo "There are ${env.COMMITS_ON_MASTER} commits on master"
}
}
}
}
You can use the longer form of the sh step and return the output (see Pipeline document). Your variable should be defined outside the stages.
You can use an imperatively created environment variable inside a script block in you stage steps, for example:
stage("Stage 1") {
steps {
script {
env.RESULT_ON_STAGE_1 = sh (
script: 'echo "Output of some command"',
returnStdout: true
)
}
echo "In stage 1: ${env.RESULT_ON_STAGE_1}"
}
}
stage("Stage 2") {
steps {
echo "In stage 2: ${env.RESULT_ON_STAGE_1}"
}
}
This guide explains use of environment variables in pipelines with examples.
My issue concerned having two 'sh' commands where one uses single quotes (where I set a variable) and the other uses double quotes (where I access 'env' variables set in the jenkinsfile such as BUILD_ID).
Here's how I solved it.
script {
env.TEST = sh(
script:
'''
echo "TEST"
''',
returnStdout: true
)
sh """
echo ${env.BUILD_ID}
echo ${env.TEST}
"""
}