I write the shared library for jenkins where I have a method that read configuration file (yaml) and should execute commands based on the input.
example for configuration file
commands:
- name: command 1
command: "sh 'ls -la'"
- name: command 2
command: "readYaml file: 'demo.yaml'"
the method code
def command_executor(config){
config.commands.each { command ->
this.script.echo "running ${command.name} command"
// This is my problem how to run the command
command.command.execute().text
}
}
The above example is define in my class and I call it from /var/my_command_executer.groovy file
How I can run any command from the string parameter?
I found the below solution:
Create temporary groovy file the predefined method name that call to the command.
Load the temporary file in method and call the method.
Something like
def command_executor(config){
config.commands.each { command ->
this.script.echo "running ${command.name} command"
this.script.writeFile file: "temp.groovy" text: """
def my_command_executor(){
${command.command}
}
"""
def temp_command_executor = load "temp.groovy"
temp_command_executor.my_command_executor()
}
}
Related
I'm trying to use bash variables in a shellScript in jetbrains space automation to no success.
My .space.kts is as follows;
job("mvn compile"){
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17"){
shellScript {
content = """
FOO="bar"
echo $FOO
"""
}
}
}
in the above i'd expect "bar" to be echoed, but instead im getting the following error when this tries to run;
Dsl file '/tmp/16487320722162400386/.space.kts' downloaded in 1736 ms
Compiling DSL script /tmp/16487320722162400386/.space.kts...
downloading /home/pipelines-config-dsl-compile-container/space-automation-runtime.jar ...
[SUCCESSFUL ] com.jetbrains#space-automation-runtime;1.1.100932!space-automation-runtime.jar (71ms)
Compilation failed in 8.652797664s.
ERROR Unresolved reference: FOO (.space.kts:9:23)
Cleaned up the output folder: /tmp/16487320722162400386
DSL processing failed: Compilation exited with non zero exit code: 2. Exit code: 102
I had planned on parsing the branch name from JB_SPACE_GIT_BRANCH and storing it in a variable to use in a call to mvn to build and tag a container using Jib
Is there anyway that i can use variables within the content of a shellScript? or should/ can this be done in a different way?
You need to replace $ by ${"$"}:
job("mvn compile") {
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17") {
shellScript {
content = """
FOO="bar"
echo ${"$"}FOO
"""
}
}
}
Or use a sh file file.sh like this:
FOO="bar"
echo $FOO
.
job("mvn compile") {
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17") {
shellScript {
content = """
./file.sh
"""
}
}
}
In gradle 7 I have created this method:
def shellCmd(String cmd) {
exec {
executable "sh"
args "-c", cmd
}
}
And this "pure" groovy version:
def shellCmd2(String cmd) {
def process = cmd.execute()
def output = new StringWriter(), error = new StringWriter()
process.waitForProcessOutput(output, error)
}
Which I call from another method e.g.:
def myMethod() {
shellCmd('ls -la')
}
I am now experimenting with getting it to work with multi-line (jenkins like) shell commands:
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [[ \$i == 'settings.gradle' ]]; then
echo "Found $i"
fi
done
"""
shellCmd(cmd)
}
but it fails with:
script '/home/user/samples/build.gradle': 47: Unexpected input: 'i' # line 47, column 5.
for i in $(ls -la); do
^
1 error
Probably I am breaking all the rules here but any input?
Also tried most of the suggestions here:
What's wrong with Groovy multi-line String?
but no luck so far.
Also based on the suggestion below I have tried to use shellCmd2 method (I am leaning towards using a plain groovy method for this to make it easier to debug outside of gradle) with:
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [ \$i = 'settings.gradle' ]; then
echo "Found \$i"
fi
done
"""
shellCmd2(cmd)
}
But that gives:
Caught: java.io.IOException: Cannot run program "for": error=2, No such file or directory
java.io.IOException: Cannot run program "for": error=2, No such file or directory
So seems the for keyword is now causing an issue.
There is nothing wrong with your multiline string. I had to change three things to make your code work:
Use single brackets [ and ]
Use single equal sign (=)
Escape the missing $i variable which was being interpreted as a Groovy variable.
You are using sh so you should only use POSIX compatible features.
Code:
def shellCmd(String cmd) {
exec {
executable "sh" // or use another shell like bash or zsh (less portable)
args "-c", cmd
}
}
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [ \$i = 'settings.gradle' ]; then
echo "Found \$i"
fi
done
"""
shellCmd(cmd)
}
You can compare with double brackets with shells like zsh and bash.
I'm trying to reference a secret file to run with newman under a sub directory of the workspace like so:
String integrationFile = "src/test/postman/integration.json"
String environmentFile = "src/test/postman/environment-dev.json"
String reportFile = "integrationReport.xml"
String reportArgs = "--reporters cli,junit --reporter-junit-export ${reportFile}"
node {
withCredentials([file(credentialsId: "${env.FILE_KEY}", variable: "FILE_PATH")]) {
String dataFile = "${env.FILE_PATH}"
dir('sub-dir') {
git branch: 'master',
credentialsId: "${config.GitHubKeyId}",
url: 'https://github.com/xxx/repo.git'
withEnv(["PATH=${tool 'nodejs-12.8.0'}/bin:${env.PATH}"]) {
try {
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d ${dataFile} ${reportArgs}")
} catch (error) {
throw error
} finally {
junit "${reportFile}"
}
}
}
}
}
}
But when I run the code above, Jenkins throws an error:
error: iteration data could not be loaded
ENOENT: no such file or directory, open '/var/jenkins_home/workspace/Platform'
The path looks to be truncated because when I run a pwd command before the node closure runs, the workspace should be:
/var/jenkins_home/workspace/Platform Management/JJob#2
My question is, why is Jenkins doing this? Do I need to format the variable of the secret another way? Or should I reference it differently?
I know the file exists because in another Jenkins pipeline that does not have a sub directory (dir("")), it works fine.
I see from the Jenkins docs about withCredentials shows that how the file is reference gets tricky when you move between directories, see here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Here are the things that I've tried:
${env.FILE_PATH}
${FILE_PATH}
$FILE_PATH
(all of the above with double and single quotes around the sh command)
Any help is appreciated,
Thanks!
Ok - after playing around alot more with it, I eventually added a double quote around the variable to keep the spaces. This SO helped me out: Jenkins coping with spaces in batch file arguments
In the newman command line script, I just had add double quotes within the param args, like so:
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d \"${dataFile}\" ${reportArgs}")
Here's my question, I want to compile jenkins pipeline into jar for shared library to import and use it, so that I can protect my jenkins pipeline source code.
As we know, shared library can use third-party libraries, I write my own third-party library and compile into jar then make shared library to import and use it, but I don't know how to use jenkins pipeline steps in my own third-party library.
Here's what I did:
I create my own third-party library write by groovy language and compile it into jar, source code like this:
// src/main/groovy/DemoLibrary.groovy
// I want this library to run jenkins pipeline step
package com.example
import org.jenkinsci.plugins.workflow.steps.durable_task.ShellStep
class DemoLibrary {
// this function can run command in jenkins master node
// runShell(script: script, cwd: cwd)
// return: process
def runShell(args) {
def cmd = args.script
if (args.cwd && args.cwd != "") {
cmd = "cd ${args.cwd} && ${cmd}"
}
def cmds = ['bash', '-c', cmd]
def proc = cmds.execute()
proc.waitFor()
if (proc.exitValue() != 0) {
throw new Exception("[ERROR] run shell error: ${proc.err.text}")
}
return proc
}
// i want this function to call jenkins "sh" step, but i don't know how to get StepContext in shared library
// runStepShell(script: script, context: context)
// return: stepExecution
def runStepShell(args) {
def shellStep = new ShellStep(args.script)
def stepExecution = shellStep.start(args.context)
retrun stepExecution
}
}
I create my shared library, source code like this:
// vars/demoLibrary.groovy
#Grab('com.example:demo-library:0.1.0')
#Field demoLib = new DemoLibrary()
def demoStage() {
docker.image("alpine:latest").inside("--user 1000:1000") {
def script = "hostname"
// run step "sh" can show the hostname of the docker container
sh script: script
// but run runShell show the hostname of the jenkins master
def proc = demoLib.runShell(script: script)
echo "${proc.text}"
// how can i get the docker stepContext to make my third-party library to run jenkins sh step?
demoLib.rrunStepShell(script: script, context: context)
}
}
Is it possible I can call jenkins steps in my own third-party library? This stucked me for several days. Thx
I am trying to get certain values from the slave by running shell commands such as :
git rev-parse HEAD
git config --get remote.origin.url
The method that I have tried to write for this is :
def executeCommand(String command) {
stdout = sh script: command, returnStdout: true
return stdout.trim()
}
Now when I try to run the first command :
output = executeCommand('git rev-parse HEAD')
I get the ERROR :
[Running] groovy "/Users/user-a/Documents/cmd.groovy"
Caught: groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
at cmd.executeCommand(cmd.groovy:2)
at cmd.run(cmd.groovy:6)
I also tried:
output = command.execute().text
But this returns nothing.
Im running out of ideas on how to run shell commands in Groovy in Jenkins and record the output.
MORE DETAILS
I am working with Jenkins shared Libraries. I have exposed a method in for my Jenkinsfile by the name getLatestBuildDetails(). This method is defined within my library. One of the actions within the method is to execute the git commands locally. So inorder to run any shell command locally, I have created the executeCommand function which takes the actual command to run as a String and executes it and returns the output to be used later by getLatestBuildDetails()
Library classes cannot directly call steps such as sh or git. They can however implement methods, outside of the scope of an enclosing class, which in turn invoke Pipeline steps, for example:
// src/org/foo/Zot.groovy
package org.foo;
def checkOutFrom(repo) {
git url: "git#github.com:jenkinsci/${repo}"
}
return this
Which can then be called from a Scripted Pipeline:
def z = new org.foo.Zot()
z.checkOutFrom(repo)
This approach has limitations; for example, it prevents the declaration of a superclass.
Alternately, a set of steps can be passed explicitly using this to a library class, in a constructor, or just one method:
package org.foo
class Utilities implements Serializable {
def steps
Utilities(steps) {this.steps = steps}
def mvn(args) {
steps.sh "${steps.tool 'Maven'}/bin/mvn -o ${args}"
}
}
When saving state on classes, such as above, the class must implement the Serializable interface. This ensures that a Pipeline using the class, as seen in the example below, can properly suspend and resume in Jenkins.
#Library('utils') import org.foo.Utilities
def utils = new Utilities(this)
node {
utils.mvn 'clean package'
}
If the library needs to access global variables, such as env, those should be explicitly passed into the library classes, or methods, in a similar manner.
Instead of passing numerous variables from the Scripted Pipeline into a library,
package org.foo
class Utilities {
static def mvn(script, args) {
script.sh "${script.tool 'Maven'}/bin/mvn -s ${script.env.HOME}/jenkins.xml -o ${args}"
}
}
The above example shows the script being passed in to one static method, invoked from a Scripted Pipeline as follows:
#Library('utils') import static org.foo.Utilities.*
node {
mvn this, 'clean package'
}
In your case you should write something like:
def getLatestBuildDetails(context){
//...
executeCommand(context, 'git rev-parse HEAD')
//...
}
def executeCommand(context, String command) {
stdout = script.sh(script: command, returnStdout: true)
return stdout.trim()
}
Jenkins file:
#Library('library_name') _
getLatestBuildDetails(this)
For more info see jenkins shared library documentation: https://jenkins.io/doc/book/pipeline/shared-libraries/
I am also using shared libraries. This is how I have used in my code:
String getMavenProjectName() {
echo "inside getMavenProjectName +++++++"
// mavenChartName = sh(
// script: "git config --get remote.origin.url",
// returnStdout: true
// ).trim()
def mavenChartName = sh returnStdout:true, script: '''
#!/bin/bash
GIT_LOG=$(env -i git config --get remote.origin.url)
basename "$GIT_LOG" .git; '''
echo "mavenChartName: ${mavenChartName}"
return mavenChartName
}
PS: Ignore the commented lines of code.
Try out the sh step instead of execute. :)
EDIT:
I would go with execute() or which I think it is even better, grgit.
I think you are not getting any output when you run cmd.execute().text because .text returns the standard output of the command and your command might only use the standard error as its output, you can check both:
def process = cmd.execute()
def stdOut = process.inputStream.text
def stdErr = process.errorStream.text