stage ('Build image') {
steps {
withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'user', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {
sh "tempPass=\$(aws ecr get-login-password --region us-west-2)"
echo 'Hello------------------'
echo "${tempPass}"
echo 'Hello AFTER------------------'
}
}
}
Exception : groovy.lang.MissingPropertyException: No such property: pass for class: groovy.lang.Binding
I have tried echo "$tempPass" as well but does not work.
I tried this as well but variable value is coming in as null.
script {
tempPass=sh(script: "aws ecr get-login-password --region us-west-2")
}
Can anyone help here?
You need to return something from the sh step either returnStatus or returnStdout
https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
e.g.
latest_tag = sh(script: "aws ecr get-login-password --region us-west-2", returnStdout: true).trim()
Related
I have the following scriptive pipeline that adds a tag to an existingt ECR image in AWS
node("linux") {
stage("test") {
docker.withRegistry("https://0123456789.dkr.ecr.us-east-1.amazonaws.com", "myCreds") {
String rc = null
sh """
aws ecr batch-get-image --repository-name repo-name --image-ids imageTag=1.0.286 --query images[].imageManifest --output text > manifest.json
cat manifest.json
"""
try {
rc = sh(script: """
aws ecr put-image --repository-name repo-name --image-tag qa-1.0.286 --image-manifest file://manifest.json
""",
returnStdout: true).trim()
}
catch(err) {
println "rc=$rc"
}
}
}
}
When I run the pipeline, I get this in the console output.
+ aws ecr put-image --repository-name repo-name --image-tag qa-1.0.286 --image-manifest file://./manifest.json
An error occurred (ImageAlreadyExistsException) when calling the PutImage operation: Image with digest 'sha256:ff44828207c7c7df75a8be2e08057da438b4b7f3063acab34ea7ebbcb7dd50a6' and tag 'qa-1.0.286' already exists in the repository with name 'repo-name' in registry with id '0123456789'
[Pipeline] echo
rc=null
Why is rc=null instead of the An error occurred... string above it in the console output? I've used this way to capture a shell script outputs, but why doesn't it work here? What's the proper way to do it in this case?
The problem is that the shell step captures standard output and the aws client prints the message into standard error
You can forward the stderr into stdout with 2>&1 at the end of your command, for example:
aws ecr put-image --repository-name repo-name --image-tag qa-1.0.286 --image-manifest file://manifest.json 2>&1
But the other problem is that when the command inside sh fails, it throws an exception and it doesn't assign the value to the variable, so you need to make sure, that the command always succeeds, for example with adding || : (execute an empty command if the previous command fails)
The downside is that you will need to check the output variable to check if the command failed.
The snippet could then look like this:
String rc = null
sh """
aws ecr batch-get-image --repository-name repo-name --image-ids imageTag=1.0.286 --query images[].imageManifest --output text > manifest.json
cat manifest.json
"""
rc = sh(script:
"""
aws ecr put-image --repository-name repo-name --image-tag qa-1.0.286 --image-manifest file://manifest.json 2>&1 || :
""", returnStdout: true).trim()
if (rc.contains("error occurred")) {
// the command invocation failed
}
I have two groovy functions within a Jenkins pipeline that together logs into an ECR repo and builds a docker container. It looks like this:
def login() {
sh "aws ecr get-login --registry-ids <id> --region <region> --no-include-email"
sh "aws ecr get-login --region <region> --no-include-email"
}
def build(project, tag) {
login()
sh "docker build -t ${project}:${tag} ."
}
However, when I run this, I get pull access denied, as if I never logged in. I surmise this is because the aws ecr login commands ran in their own shells, and the build commands ran in another. Ideally, I'd like to leverage this kind of functional decomposition and other features of groovy, but run shell commands in one process/shell. Is this possible? How can I accomplish this?
The issue here is that aws ecr get-login returns a string containing the command to login to your registry. You will need to execute the result of that command, by assigning the output of the command to a variable and then executing that result, like so:
def login() {
loginRegistry = sh (script: "aws ecr get-login --registry-ids <id> --region <region> --no-include-email", returnStdout: true)
sh loginRegistry
loginRegion = sh (script: "aws ecr get-login --region <region> --no-include-email", returnStdout: true)
sh loginRegion
}
def build(project, tag) {
login()
sh "docker build -t ${project}:${tag} ."
}
I run my Jenkins build inside a docker container (node:latest).
But the enviroments variables are not defiend in the container:
$GIT_BRANCH, $GIT_COMMIT
So I got this error:
GitHub has been notified of this commit’s build result
groovy.lang.MissingPropertyException: No such property: GIT_BRANCH for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
I have a lot of variables I need to pass into the container. how to do that with Jenkins?
I looking for solution that inherit all environment variables that exist in Jenkins process and my host machine/docker
Here my Jenkinsfile:
throttle(['throttleDocker']) {
node('docker') {
wrap([$class: 'AnsiColorBuildWrapper']) {
try{
docker.image('node:latest').inside {
stage('Checkout SCM'){
checkout scm
}
stage('PS'){
sh 'node -v'
sh 'ls'
}
stage('Verify Branch') {
echo "$GIT_BRANCH"
echo "$GIT_COMMIT"
}
stage('Build'){
sh "npm run build"
sh 'ls'
}
stage('Test'){
sh 'echo "Test Stage inside container."'
}
}
Maybe this
node ('linux-slave') {
withEnv(['PATH=/usr/local/Cellar/coreutils/8.25/libexec/gnubin:/usr/local/Cellar/gnu-sed/4.2.2/libexec/gnubin:/Users/fbelzunc/cloudbees/support/support-shinobi-tools:/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home//bin:/Users/fbelzunc/bin:/usr/local/sbin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/usr/local/sbin:/usr/local/Cellar/ruby/2.1.0/lib/ruby/gems/2.1.0/gems/bundler-1.6.5/bin:/usr/local/Cellar/ruby/2.1.0/lib/ruby/gems/2.1.0/gems/beaker-1.16.0/bin:/usr/local/Cellar/ruby/2.1.0/bin/:/path/testing']) {
docker.image('maven:3.3.3-jdk-8').inside {
sh 'echo $PATH'
sh 'mvn --version'
}
}
sh 'echo $PATH'
}
I'm trying to leverage the Jenkins credentials plugin to store sensitive data which I want to inject into Secrets within my Kubernetes cluster. I have a JenkinsFile which is used in my project to define the steps and I've added the following code to pull a username/password from a credential and pass to shell script to replace a placeholder in a file with the actual file:
stages {
stage('Build') {
steps {
withCredentials([usernamePassword(credentialsId: 'creds-test', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh '''
echo $USERNAME
echo $PASSWORD
chmod +x secrets-replace.sh
./secrets-replace.sh USERNAME_PLACEHOLDER $USERNAME
./secrets-replace.sh PASSWORD_PLACEHOLDER $PASSWORD
'''
}
echo 'Building...'
sh './gradlew build --refresh-dependencies'
}
}
...
}
However whenever this runs all I ever get is the masked **** value back, even when I pass it to the shell script. Here is part of the build log:
Is there something I need to configure to get access to the unmasked value?
Write the variable to a file in jenkins. Go to the jenkins workspace and look inside the file. The token will be present in plain text there.
UPDATE
Further easy way will be to print the base64 encoded value of the credential and then decode it.
Like the others added above, you could actually write it to a file and then cat the file outside of the withCredentials. You should be fine with this. As below..
withCredentials([usernamePassword(credentialsId: 'creds-test', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh '''
echo $USERNAME > tmp
echo $PASSWORD >> tmp
'''
}
sh 'cat tmp'
This prints the actual credential values
Echoing straight from file didnt work for me so I tricked Jenkins like this to see the secret during debugging: Obviously, remove it right after debugging!
stage('Build') {
azureKeyVault(
credentialID: 'my-sp',
keyVaultURL: 'https://my-kv.vault.azure.net',
secrets: [
[envVariable: 'MY_SECRET', name: 'my-secret-name-in-azure-kv', secretType: 'Secret']
]
) {
sh '''
echo -n $MY_SECRET | base64 > tmpp
cat tmpp
'''
}
}
Consider manipulating the string
echo env.PASSWORD.toCharArray().join(' ');
like
stages {
stage('Build') {
steps {
withCredentials([usernamePassword(credentialsId: 'creds-test', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
script {
echo env.USERNAME.toCharArray().join(' ');
echo env.PASSWORD.toCharArray().join(' ');
}
sh '''
chmod +x secrets-replace.sh
./secrets-replace.sh USERNAME_PLACEHOLDER $USERNAME
./secrets-replace.sh PASSWORD_PLACEHOLDER $PASSWORD
'''
}
echo 'Building...'
sh './gradlew build --refresh-dependencies'
}
}
...
}
I'm facing one really weird problem
// Update the service
stage "Update Service"
def SERVICE_NAME = "currency-converter-search-srv"
def TASK_FAMILY = "currency-converter-search"
def TASK_REVISION = sh "aws --region us-east-1 ecs describe-task-definition --task-definition currency-converter-search | jq .taskDefinition.revision"
def DESIRED_COUNT = sh "aws --region us-east-1 ecs describe-services --services ${SERVICE_NAME} | jq .services[0].desiredCount"
if (DESIRED_COUNT == 0) {
DESIRED_COUNT = 1
}
sh "aws --region us-east-1 ecs update-service --cluster default --service ${SERVICE_NAME} --task-definition ${TASK_FAMILY}:${TASK_REVISION} --desired-count ${DESIRED_COUNT}"
this script fails and here below the log:
[Pipeline] stage (Update Service)
Entering stage Update Service
Proceeding
[Pipeline] sh
[workspace] Running shell script
+ jq .taskDefinition.revision
+ aws --region us-east-1 ecs describe-task-definition --task-definition currency-converter-search
13
[Pipeline] sh
[workspace] Running shell script
+ jq .services[0].desiredCount
+ aws --region us-east-1 ecs describe-services --services currency-converter-search-srv
0
[Pipeline] sh
[workspace] Running shell script
+ aws --region us-east-1 ecs update-service --cluster default --service currency-converter-search-srv --task-definition currency-converter-search:0 --desired-count 1
An error occurred (InvalidParameterException) when calling the UpdateService operation: revision must be between 1 and 2147483647
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 255
Finished: FAILURE
The reason is because TASK_REVISION variable is 0 but according to how it is processed is not zero but 13. do you know why this weird behaviour?
You can't assign the result of sh to a variable.
sh doesn't return anything meaningful... I think there's an issue for this, but there's no fix yet
The workaround seems to be to redirect the result to a file, then read that file