Create custom groovy closure to reuse shell() feature - jenkins

I am using Groogy DSL plugin in my Jenkins server. I realize this step is repeated in many places and jobs.
steps {
shell('''#!/bin/bash -ex
|aws s3 cp s3://${STACK_S3_BUCKET_NAME}/file myfile --region ${AWS_REGION}
|aws s3 cp s3://${STACK_S3_BUCKET_NAME}/otherfile .
|
|'''.stripMargin())
I am new using Groovy and I will like to create kind of custom Groovy Step or Closure to avoid this process, I will like to do something like:
awsS3cp {
from: '${ORIGIN}'
to: '${DESTINATION}'
}
Then implement something like this:
def awsS3cp { context ->
shell('''#!/bin/bash -ex
|aws s3 cp s3://$from $to
|'''.stripMargin())
}
Attempt 1
I did it in this way and it failed:
def awsS3cp(String from, String to) {
shell("""#!/bin/bash -ex
|echo 'copy from $from to $to'
|""".stripMargin())
}
def createEnvironmentJob = freeStyleJob( jobName )
createEnvironmentJob.with{
description( jobDescription )
steps {
awsS3cp ("S3-SOURCE-BUCKET","S3-TARGET-BUCKET")
}
}
The error output:
No signature of method: create_environment.shell() is applicable for argument types: (java.lang.String) values: [#!/bin/bash -ex
echo 'copy from S3-SOURCE-BUCKET to S3-TARGET-BUCKET'
]
Possible solutions: queue(java.lang.String), sleep(long), every(), grep(), job(java.lang.String), queue(javaposse.jobdsl.dsl.Job)
Finished: FAILURE

One option would be to define a function outside:
def awsS3cp(String from, String to) {
return sh("""#!/bin/bash -ex
|aws s3 cp s3://$from $to
|""".stripMargin())
}
Then, call it inside of the pipeline definition:
steps {
awsS3cp('source', 'destination')
}

Related

How do I use sh in Jenkins Global library

I am creating my own global library for Jenkins, which I have hosted on github, and to simplify some run-of-the-mill tasks, I wanted to add a function that returns the GIT tag.
Therefore I created something like this:
class Myclass{
static String getGitTag() {
return "${sh(returnStdout: true, script: 'git tag --sort version:refname | tail -1').trim()}"
}
}
... which results in this error:
No signature of method: static com.stevnsvig.jenkins.release.ReleaseUtil.sh()
So I'm left with two questions:
Is the solution to import the sh() library that Jenkins' groovy flavor obviously already has imported? (and if so how)
What is the best practice here? I am wondering why there isn't a GIT_TAG global variable when you use declarative pipelines, and something like this should (in my opinion) be easy as pie.
EDIT #1:
static String getGitTag() {
stdout = script.sh(script: "git tag --sort version:refname | tail -1", returnStdout: true)
return stdout.trim()
}
produces a similar error:
No signature of method: static com.stevnsvig.jenkins.release.ReleaseUtil.sh() is applicable for argument types: (java.util.LinkedHashMap) values: [[returnStdout:true, script:git tag --sort version:refname | tail -1]]
EDIT #2:
static String getGitTag() {
def stdout = "git tag --sort version:refname | tail -1".execute()
return stdout.in.text
}
completes, but the output is blank. Running the same command with pwd returns / which indicaes that the environment is not set, which makes sense, since all the commands running under Jenkins are designed to rununder pipelines
EDIT #3:
I went hunting for the import. Stumbled across the Jenkins CI project on github and started searching the many repositories. Found a promising one... and put a file called pwd.groovy in /vars with this content:
import org.jenkinsci.plugins.workflow.steps.durable_task.ShellStep
static String getPWD() {
def ret = ShellStep.sh(returnStdout: true, script: "git tag --sort version:refname | tail -1").trim()
echo "currently in ${ret}"
}
The error I got is a variation of the same. I guess since itsa plugin, the definition is different...
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: static org.jenkinsci.plugins.workflow.steps.durable_task.ShellStep.sh() is applicable ...
Option 1) Use Groovy execute to run cmd and get its output as below
tag = "git tag --sort version:refname | tail -1".execute().text
Option 2) Use Jenkins pipeline step sh.
One concept need to get clear: the context of sh is global function is when sh used directly inside Jenkinsfile.
In your case, sh is used outside the Jenkinsfile. To make better understand I give an example Jenkinsfile.
pipeline {
stages('foo') {
steps {
sh 'pwd'
// In above sh step, there is an implicit `this` which represents the
// global object for Jenkinsfile, you can image sh 'pwd' to this.sh 'pwd'
//
// Thus if you want to use `sh` outside Jenkinsfile, you must pass down the
// implicit `this` into the file where you used `sh`
}
}
}
To address your issue
// ReleaseUtil.groovy
static String getGitTag(steps) {
// here `steps` is the global object for Jenkinsfile
// you can use other pipeline step here by `steps`
steps.echo 'test use pipeline echo outside Jenkinsfile'
steps.withCredentials([steps.string(credentialsId: 'git_hub_auth', variable: 'GIT_AUTH_TOKEN')]) {
steps.echo '....'
steps.sh '....'
}
return steps.sh(returnStdout: true, script:"git tag --sort version:refname | tail -1").trim()
}
// Jenkinsfile
import com.stevnsvig.jenkins.release.ReleaseUtil
pipeline {
stages('foo') {
steps {
ReleaseUtil.getGitTag(this)
}
}
}

How to pass dynamic values to an environment block during the pipeline execution in Jenkins?

This is related to one question I asked before: Using groovy to parse JSON object in shell scripts for Jenkin
basically I will need to pass a dynamic value as returned from sh scripts to an environment block, so that the following stage can re-use that value and pass the version as a label to JIRA plugin called Xray. But I aware that I cannot pass dynamic values to an environment block during the pipeline execution. So, I think I am going to need try a different route for that, not sure if anyone could give me some tips please?
def setLatestAppVersionLabel() {
def response = sh(script: "curl --silent ${APP_ARTIFACTORY_URL}/${XRAY_PLATFORM}/builds/latest.json", returnStdout: true).trim() as String
def jsonResponse = readJSON text: response
LATEST_VERSION = jsonResponse.id
echo "LATEST_VERSION -> ${LATEST_VERSION}"
}
JSON response looks like that:
{"id":"1.0.0-6",
"version":"1.0.0",
"build":6,
"tag":"android-v1.0.0-6",
"commitHash":"5a78c4665xxxxxxxxxxe1b62c682f84",
"dateCreated":"2020-03-02T08:11:29.912Z"}
and there is an environment block where I would like to pass the value to one of the variable defined there#
environment {
AWS_DEFAULT_REGION = 'uk-xxx'
XRAY_ENVIRONMENT = 'e2e'
VERSION_KEY = 'id'
XRAY_PLATFORM = 'Android'
APP_ARTIFACTORY_URL = 'https://artifactory.example.com/mobile'
LATEST_VERSION = ''
}
If this path not working, what else could I use? Want to re-use the latest version taken from JSON response for the next stage in the pipeline to use.
Next stage looks like this:
stage('Import Result to Xray') {
when {
expression { return fileExists('xxx-executor/target/AndroidxxxxE2EResults/cucumber-reports/Cucumber.json')}
}
steps {
xrayResultsImport('xxx-executor/target/AndroidxxxxxE2EResults/cucumber-reports/Cucumber.json', 'xxx_ANDROID_E2E_xxxxxxx_Tests', XRAY_LABELS, ['E2E', 'Android', LATEST_VERSION], env.BUILD_URL)
}
}
Sorry I have to put xxxx to make this question general due to project confidentiality.
To put it simple, you want to use the version you fetched from a JSON response and want to use it in all stages of your Jenkins pipeline.
Ensure you've jq utility installed in your jenkins agent.
pipeline {
agent any
environment {
XRAY_LATEST_VERSION = ''
}
stages {
stage(‘Get Version ') {
steps {
script {
XRAY_LATEST_VERSION = sh(script: 'curl -s ${APP_ARTIFACTORY_URL}/${XRAY_PLATFORM}/builds/latest.json | jq .version | sed \'s/"//g\'', returnStdout: true).trim()
}
}
}
stage('Print'){
steps{
echo "${XRAY_LATEST_VERSION}"
}
}
}
}
You can use the variable ${XRAY_LATEST_VERSION} in any stages you want the and the value will be rendered across.

Unable to run shell command in Groovy in Jenkins

I am trying to get certain values from the slave by running shell commands such as :
git rev-parse HEAD
git config --get remote.origin.url
The method that I have tried to write for this is :
def executeCommand(String command) {
stdout = sh script: command, returnStdout: true
return stdout.trim()
}
Now when I try to run the first command :
output = executeCommand('git rev-parse HEAD')
I get the ERROR :
[Running] groovy "/Users/user-a/Documents/cmd.groovy"
Caught: groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
at cmd.executeCommand(cmd.groovy:2)
at cmd.run(cmd.groovy:6)
I also tried:
output = command.execute().text
But this returns nothing.
Im running out of ideas on how to run shell commands in Groovy in Jenkins and record the output.
MORE DETAILS
I am working with Jenkins shared Libraries. I have exposed a method in for my Jenkinsfile by the name getLatestBuildDetails(). This method is defined within my library. One of the actions within the method is to execute the git commands locally. So inorder to run any shell command locally, I have created the executeCommand function which takes the actual command to run as a String and executes it and returns the output to be used later by getLatestBuildDetails()
Library classes cannot directly call steps such as sh or git. They can however implement methods, outside of the scope of an enclosing class, which in turn invoke Pipeline steps, for example:
// src/org/foo/Zot.groovy
package org.foo;
def checkOutFrom(repo) {
git url: "git#github.com:jenkinsci/${repo}"
}
return this
Which can then be called from a Scripted Pipeline:
def z = new org.foo.Zot()
z.checkOutFrom(repo)
This approach has limitations; for example, it prevents the declaration of a superclass.
Alternately, a set of steps can be passed explicitly using this to a library class, in a constructor, or just one method:
package org.foo
class Utilities implements Serializable {
def steps
Utilities(steps) {this.steps = steps}
def mvn(args) {
steps.sh "${steps.tool 'Maven'}/bin/mvn -o ${args}"
}
}
When saving state on classes, such as above, the class must implement the Serializable interface. This ensures that a Pipeline using the class, as seen in the example below, can properly suspend and resume in Jenkins.
#Library('utils') import org.foo.Utilities
def utils = new Utilities(this)
node {
utils.mvn 'clean package'
}
If the library needs to access global variables, such as env, those should be explicitly passed into the library classes, or methods, in a similar manner.
Instead of passing numerous variables from the Scripted Pipeline into a library,
package org.foo
class Utilities {
static def mvn(script, args) {
script.sh "${script.tool 'Maven'}/bin/mvn -s ${script.env.HOME}/jenkins.xml -o ${args}"
}
}
The above example shows the script being passed in to one static method, invoked from a Scripted Pipeline as follows:
#Library('utils') import static org.foo.Utilities.*
node {
mvn this, 'clean package'
}
In your case you should write something like:
def getLatestBuildDetails(context){
//...
executeCommand(context, 'git rev-parse HEAD')
//...
}
def executeCommand(context, String command) {
stdout = script.sh(script: command, returnStdout: true)
return stdout.trim()
}
Jenkins file:
#Library('library_name') _
getLatestBuildDetails(this)
For more info see jenkins shared library documentation: https://jenkins.io/doc/book/pipeline/shared-libraries/
I am also using shared libraries. This is how I have used in my code:
String getMavenProjectName() {
echo "inside getMavenProjectName +++++++"
// mavenChartName = sh(
// script: "git config --get remote.origin.url",
// returnStdout: true
// ).trim()
def mavenChartName = sh returnStdout:true, script: '''
#!/bin/bash
GIT_LOG=$(env -i git config --get remote.origin.url)
basename "$GIT_LOG" .git; '''
echo "mavenChartName: ${mavenChartName}"
return mavenChartName
}
PS: Ignore the commented lines of code.
Try out the sh step instead of execute. :)
EDIT:
I would go with execute() or which I think it is even better, grgit.
I think you are not getting any output when you run cmd.execute().text because .text returns the standard output of the command and your command might only use the standard error as its output, you can check both:
def process = cmd.execute()
def stdOut = process.inputStream.text
def stdErr = process.errorStream.text

Groovy parameters are not visible in shell part of a Jenkinsfile

I'm facing a problem in Groovy script wherein the variable is not accessible in shell script part.
script-1:
def a=20;
println ("a is: $a");
output-1:
a is: 20
script-2:
def a=20;
println ("a is: $a");
sh '''echo a is $a''';
Output-2:
groovy.lang.MissingMethodException: No signature of method: Script1.sh() is applicable for argument types: (java.lang.String) values: [echo a is $a]
Possible solutions: use([Ljava.lang.Object;), is(java.lang.Object), run(), run(), any(), with(groovy.lang.Closure)
at Script1.run(Script1.groovy:3)
How can i get $a = 20 in the shell part sh. In other words what operations are required to pass the variable $a in shell script part.
I'm writing this script in context of a Jenkins pipeline where i'm facing a problem that groovy variables are not visible in the shell part.
this works:
pipeline {
agent any
stages {
stage('Example') {
steps {
script {
// a is accessible globally in the Jenkinsfile
a = 20
// b is only accessible inside this script block
def b = 22
sh "echo a is $a"
sh "echo b is $b"
}
}
}
}
post {
always {
sh "echo a is $a"
}
}
}
You should use double quote for the shell statement and not triple single quote.

Jenkins pipeline shell step

Trying to get this pipeline working..
I need to prepare some variables (list or string) in groovy, and iterate over it in bash. As I understand, groovy scripts run on jenkins master, but I need to download some files into build workspace, that's why I try to download them in SH step.
import groovy.json.JsonSlurper
import hudson.FilePath
pipeline {
agent { label 'xxx' }
parameters {
...
}
stages {
stage ('Get rendered images') {
steps {
script {
//select grafana API url based on environment
if ( params.grafana_env == "111" ) {
grafana_url = "http://xxx:3001"
} else if ( params.grafana_env == "222" ) {
grafana_url = "http://yyy:3001"
}
//get available grafana dashboards
def grafana_url = "${grafana_url}/api/search"
URL apiUrl = grafana_url.toURL()
List json = new JsonSlurper().parse(apiUrl.newReader())
def workspace = pwd()
List dash_names = []
// save png for each available dashboard
for ( dash in json ) {
def dash_name = dash['uri'].split('/')
dash_names.add(dash_name[1])
}
dash_names_string = dash_names.join(" ")
}
sh "echo $dash_names_string"
sh """
for dash in $dash_names_string;
do
echo $dash
done
"""
}
}
}
}
I get this error when run..
[Pipeline] End of Pipeline
groovy.lang.MissingPropertyException: No such property: dash for class: WorkflowScript
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:458)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.getProperty(DefaultInvoker.java:33)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at WorkflowScript.run(WorkflowScript:42)
Looks like I'm missing something obvious...
Escape the $ for the shell variable with a backslash, that should help:
for dash in $dash_names_string;
do
echo \$dash
done
the problem is on line three here:
for dash in $dash_names_string;
do
echo $dash
done
it's trying to find a $dash property in groovy-land and finding none. i can't actually think how to make this work vi an inline sh step (possibly not enough sleep), but if you save the relevant contents of your json response to a file and then replace those four lines with a shell script that reads the file and call it from the Jenkinsfile like sh './hotScript.sh', it will not try to evaluate that dollar value as groovy, and ought to at least fail in a different way. :)

Resources