In my pipeline script I want to execute the following sh command including a password (e.g. 123$ABC) with special characters:
withCredentials([sshUserPrivateKey(credentialsId: 'id', keyFileVariable: 'keyFile', passphraseVariable: '', usernameVariable: 'user')]) {
sh "ssh -i $keyFile ${user}#${virtualMachine} -C \"CONTAINER_NAME=${dockerContainerName} DOCKER_TAG=${dockerImageTag} JASYPT_MASTER_PASSWORD='${JASYPT_MASTER_PW}' docker-compose -f /tmp/docker-compose-jenkins.yml up\""
}
How must the command look like to obtain the special characters in my password?
I have the same issue, the possible solution is to decode the password variable, something like:
def encodedPassword = URLEncoder.encode("$JASYPT_MASTER_PW",'UTF-8')
and then use the encodedPassword variable instead of JASYPT_MASTER_PW
Let's try
Related
We have Jenkins stage which is calling a "GetVMPassword" function from library. The function returns credential and it will be used to login remote server. We dont want to print the "ssh command" and "calling a funtion command" and its reponse on stage logs. So we used ‘#!/bin/sh -e \n’ before every command. Because if we print, this could reveal the remote server credentials in the stage log. This was working when we don't use "parallel execution" block.
When we include "ssh command" and "calling a function command" inside "parallel execution" block, passwords are printed in stage logs.
How can we avoid printing in stage logs the library command and its response when we use "parallel execution" block ?
This is snippet of my stage and parallel execution block.
Jenkins Version: 2.235.3
#Library ('MyLib_API') _
pipeline{
agent {
label 'master'
}
stages{
stage('BuildAll'){
steps{
script{
def executions = APPSERVERS.split(',').collectEntries {APPS ->
["Execution ${APPS}": {
stage(APPS) {
APP_USERNAME = "ubuntu"
response = getPassword("${APPS}","${APP_USERNAME}")
sh '#!/bin/sh -e \n' + "sshpass -p '${response}' ssh -o StrictHostKeyChecking=no ${APP_USERNAME}#${APPS} 'ls'"
sleep 2
}
}]
}
parallel executions
}
}
}
}
}
"getPassword" is the function in library used to get the vm password dynamically.
"APPSERVERS" values we are getting from Active choice parameters option.This has list of IP's of servers.
Please help me to hide those library commands and responses from stage logs.
We have tried below options.
Used set +x and it is not worked for us.
Password masking plugin will not work. Since response from the command will get print for our case.
We tried routing all the execution of commands to file and tried fetching it from there. In this option, also while parsing the file logs are printed in stage logs.
Try starting your script with set +x, if not use password masking plugins
as mentioned here - https://issues.jenkins.io/browse/JENKINS-36007
You can use input to pass the credential and mask it in log.
Here is a detailed answer stackoverflow credentials masking
you can use this as well it works for me.
node('Node Name'){
println('Please enter the username')
def userName = input(
id: 'userName', message: 'VPN Username', parameters: [
[$class: 'hudson.model.TextParameterDefinition', defaultValue :'', name: 'Username', description: 'Please enter your username']
])
println('Please enter the password')
def userPassword = input(
id: 'userPassword', message: 'VPN Password', parameters: [
[$class: 'hudson.model.PasswordParameterDefinition', defaultValue :'', name: 'Password', description: 'Please enter your password']
])
connectToClient = bat(returnStdout: true, script: 'start Forticlient connect -h v3 -u ' + userName+ ':' + userPassword)
stage('Deploy (Test)'){
withCredentials([usernamePassword(credentialsId: 'IH_IIS_JENKINS', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
bat"msdeploy command"
}
}
}
I have checked many suggestions regarding the issue, but didn't find a suitable solution.
Initially my pipeline code was working fine but with a Jenkins security warning The following steps that have been detected may have insecure interpolation of sensitive variables. sh: [pass, user]
My code is as below:
withCredentials([usernamePassword(credentialsId: 'Jenkins-Nexus-tech', passwordVariable: 'pass', usernameVariable: 'user')]){
sh "wget --user=$user --password=$pass 'https://nexus.mycompany.io/repository/updated-maven-releases/com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war' "
}
Warning:
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [pass, user]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
To avoid the Jenkins security warning as suggested in https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#interpolation-of-sensitive-environment-variables , I have modified my code as below. But with different permutation and combination of single and double quotes it is not working. It is throwing Bad substitution error:
withCredentials([usernamePassword(credentialsId: 'Jenkins-Nexus-tech', passwordVariable: 'pass', usernameVariable: 'user')]){
sh 'wget --user=$user --password=$pass \"https://nexus.mycompany.io/repository/updated-maven-releases/com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war\" '
}
I have tried with 3 single quotes. with this I am not getting the warning but it is unable to substitute the variables in the wget URL:
withCredentials([usernamePassword(credentialsId: 'Jenkins-Nexus-tech', passwordVariable: 'pass', usernameVariable: 'user')]){
sh '''wget --user=$user --password=$pass 'https://nexus.mycompany.io/repository/updated-maven-releases/com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war' '''
}
Error is as below as it can't substitute the values of the variable
[Pipeline] sh
+ wget --user=**** --password=**** https://nexus.mycompany.io/repository/updated-maven-releases/com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war
HTTP request sent, awaiting response... 404 com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war
ERROR 404: com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war.
It would be really great if someone can assist here.
The suggestion outlined in the page you mention suggests you do not use Groovy string interpolation to pass sensitive data to sh steps. The suggestion is to let the sh script instead get the values from the environment.
However, in your case you still need the value held in the params.AuthorizationServerVersion in your sh step.
Groovy string interpolation is only used when the string is within double quotes. String interpolation is turned off when the string is in single quotes.
When a string is in double quotes, you can escape $s using a backslash. This will prevent inserting the contents of a variable into the string when you don't want this to occur.
What you can do, then, is use double quotes, escape the $s on the variables you want to read from the environment, and leave the $s un-escaped on the variables you want to access directly from your Groovy code.
An example, using the withCredentials plugin:
withCredentials([string(credentialsId: 'mytoken', variable: 'THE_TOKEN')]) {
sh "some_command --that-uses \$THE_TOKEN --as-an-environment-variable --and-also-uses $params.A_PARAMETER"
}
Here, the token is injected into the environment as a secret, and is used in the sh script by escaping the $. This means the actual Unix shell (bash, zsh, or whatever) will get the literal $ and expand it as specified by that shell language.
The $ for $params.A_PARAMETER is not escaped, and is within double quotes, so Groovy will perform string interpolation and replace $params.A_PARAMETER with the value of the parameter at runtime as the string is constructed in memory, before the string is passed as an argument to the sh step.
In your specific case, this is what you can use (within your withCredentials block, of course):
sh "wget --user=\$user --password=\$pass 'https://nexus.mycompany.io/repository/updated-maven-releases/com/mycompany/project/authorization-server/${params.AuthorizationServerVersion}/authorization-server-${params.AuthorizationServerVersion}.war'"
Note that the $ on $user and $pass are escaped, but the $ on ${params.AuthorizationServerVersion} is not escaped.
I am writing a sensitive data to a .yml file as a string in my Jenkins groovy file. I have attached the .groovy file which would show what I am trying to achieve but it reveals the data in Pipeline steps which is also attached. Nothing shows in the console log. Value from AWS Parameter store is retrieved and passed as a string to a .yml file and this string value is shown in the pipeline steps. I am looking for a way to hide this from being shown in the Pipeline steps. I have tried using set +x but it is not hiding the string value in the pipeline steps. Any thoughts on how this can be achieved would be of great help.Screenshot
#!/usr/bin/env groovy
properties([
parameters([
string(name:'NODE_NAME', defaultValue: 'node', description: 'Node Name')
])
])
nodeName = params.NODE_NAME
node("${nodeName}") {
def AWS_REGION = "eu-west-2"
def paramPath = "path"
def paramName = "PASSWORD"
stage("Data from SSM") {
param = sh (script: "aws ssm get-parameter --name ${paramPath} --with-decryption --region ${AWS_REGION} | /var/jenkins_home/tools/jq .Parameter.Value | tr -d '\"'", returnStdout: true).trim()
sh('#!/bin/sh -e\n' + "echo '${paramName}: ${param}' >> vars.yml")
}
}
You can make use of credentials in jenkins. Go to Jenkins -> credentials -> global -> add credentials
Select either 'Username with password'/'secret text' depending on your need
This will generate a secret in jenkins
Then in your groovy file you can use the credentials as env variables which will not be printed by jenkins
withCredentials([usernamePassword(credentialsId: 'Your-Credential-ID', passwordVariable: 'pwdToken', usernameVariable: 'userToken')]){
//Use pwdToken as a variable here which will not be printed
}
I am retrieving the username password from the credentials plugin.
The values are then saved as environmental variables. I am using the password in later stages of the pipeline, in sh block as an argument for curl.
At this point the password is displayed in plaintext in the build logs. Is there a way to avoid this? I assumed by using the credentials plugin the password will be masked.
pipeline {
stages {
stage ('One') {
steps {
withCredentials([userNamePassword(credentialsId: 'my_cred', userNameVariable: 'User_Name', passwordVariable: 'Password')]){
env.User_Name = User_Name
env.Password = Password
}
}
}
stage ('Two') {
sh '''
curl -v -u ${User_Name}:${Password} ...
'''
}
}
}
Note: I am using the curl to upload a file to a remote host.
Thats true. The password will be displayed in plaintext.
The best way for your request, ist to use the HTTP Request Plugin.
You can pass credentials in "Authorization" in the Header inestead of URL.
I ended up using the curl inside the withCredentialsblock.
withCredentials([userNamePassword(credentialsId: 'my_cred', userNameVariable: 'User_Name', passwordVariable: 'Password')]){
sh '''
curl -v -u ${User_Name}:${Password} ...
'''
}
I am trying to replace the '/' from Git branch name with '_' in my jenkinsfile so that I can tag my docker image with the branch name. In bash the below command works fine
echo "${git_branch_name//\//_}"
But when use the above command in jenkinsfile as below it throws an error.
#!/usr/bin/env groovy
def commit_id
def imagetag
def branch_name
def git_branch_name
node('Nodename') {
stage('checkout') {
checkout (scm).$Branch_Param
sh "git rev-parse --short HEAD > .git/commit-id"
commit_id = readFile('.git/commit-id').trim()
sh "git rev-parse --abbrev-ref HEAD > .git/branch-name"
git_branch_name = readFile('.git/branch-name').trim()
branch_name= sh "echo ${git_branch_name//\//_}"
sh "echo ${commit_id}"
sh "echo ${branch_name}"
sh "echo Current branch is ${branch_name}"
}
}
WorkflowScript: 15: end of line reached within a simple string 'x' or "x" or /x/;
solution: for multi-line literals, use triple quotes '''x''' or """x""" or /x/ or $/x/$ # line 15, column 28.
sh "branch_name = echo ${git_branch_name//\//_}"
What am I doing wrong here? Should I use Groovy regular expression instead of shell? why is shell not being interpreted correctly?
Thank you
The issue is that you're asking Groovy itself to interpret the expression ${git_branch_name//\//_}, not the shell. Using double-quotes around the string you pass to the sh step is what causes that. So if you instead write the following, this first error will go away:
sh 'echo ${git_branch_name//\\//_}' // <- Note the single-quotes
Basically, always use single-quotes unless you specifically need to use groovy's string interpolation (see the very last echo at the bottom of this answer).
Interestingly, it seems when I tested I didn't need the shebang (#!/bin/bash) to specify bash as some comments suggest; this ${variable//x/y} replace syntax worked in an sh step as-is. I guess the shell spawned was bash. I don't know if that's always the case, or if our Jenkins box has been specifically setup that way.
Also note you need to escape the escape sequence ('\\/') because what you're passing to the sh step is a string literal in groovy code. If you don't add that extra backslash, the line passed to the shell to be interpreted by it will be echo ${git_branch_name////_}, which it won't understand.
But there are other issues as well. First, assigning the output of the sh step to branch_name as you do means branch_name will always equal null. To get the stdout from a line of shell code you need to pass the extra parameter returnStdout: true to sh:
branch_name = sh (
script: 'echo ${git_branch_name//\\//_}',
returnStdout: true
).trim () // You basically always need to use trim, because the
// stdout will have a newline at the end
For bonus points, we could wrap that sh call in a closure. I find myself using it often enough to make this a good idea.
// Get it? `sh` out, "shout!"
def shout = { cmd -> sh (script: cmd, returnStdout: true).trim () }
//...
branch_name = shout 'echo ${git_branch_name//\\//_}'
But finally, the major problem is that bash (or whatever shell is actually spawned) doesn't have access to groovy variables. As far as it knows, echo ${git_branch_name} outputs an empty string, and therefore so does echo ${git_branch_name//\//_}.
You have a couple of choices. You could skip the creation of .git/branch-name and just immediately output the string-replaced result of git rev-parse:
branch_name = shout 'name=$(git rev-parse --abbrev-ref HEAD) && echo ${name//\\//_}'
Or to simplify that further you could use groovy's string replace function rather than the bash syntax:
branch_name = shout ('git rev-parse --abbrev-ref HEAD').replace ('/', '_')
Personally, I find the latter quite a bit more readable. YMMV. So bringing it all together at last:
#!groovy
def shout = { cmd -> sh (script: cmd, returnStdout: true).trim () }
// Note that I'm not declaring any variables up here. They're not needed.
// But you can if you want, just to clearly declare the environment for
// future maintainers.
node ('Nodename') {
stage ('checkout') {
checkout (scm).$Branch_Param
commit_id = shout 'git rev-parse --short HEAD'
branch_name = shout ('git rev-parse --abbrev-ref HEAD').replace ('/', '_')
echo commit_id
echo branch_name
echo "The most recent commit on branch ${branch_name} was ${commit_id}"
}
}