I am trying to generate some generic Groovy code for Jenkins but I seem to have trouble with multi line strings and extra white space. I've tried everything I could find by Googling but I can't seem to get it working.
My issue isn't related to simple multi line strings. I managed to trim white space by using the stripIndent() and stripMargin() methods for simple cases. My issue is caused by having interpolated methods inside my strings.
Groovy info: Groovy Version: 3.0.2 JVM: 13.0.2 Vendor: Oracle Corporation OS: Mac OS X
String method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".stripIndent().stripMargin()
}
static String simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".stripIndent().stripMargin()
}
static String method1() {
return """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".stripIndent().stripMargin()
}
String generateFullDeploymentPipelineCode() {
return """Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
This is what it prints(or writes to disk):
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Why the extra space around the import lines? I know the indentation method is supposed to trim all white space according to the least number of leading spaces, so that's why we use backslash (example here https://stackoverflow.com/a/19882917/7569335).
That works for simple strings, but it breaks down once use start using interpolation. Not with regular variables, just when you interpolate an entire method.
as variant - use just stripMargin() and only once on a final string
String method2(String tier, String jobName) {
return """\
|Map downstreamJobs = [:]
|stage ("${jobName}-${tier}-\${region}_${jobName}") {
| test
|}
"""
}
static String simpleLog() {
return """\
|script {
| def user = env.BUILD_USER_ID
|}
"""
}
static String method1() {
return """\
|import jenkins.model.Jenkins
|currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
return """\
|Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
result:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
another variant with trim() and stripIndent()
def method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".trim()
}
def simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".trim()
}
def method1() {
return """
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".trim()
}
def generateFullDeploymentPipelineCode() {
return """\
Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent()
}
println(generateFullDeploymentPipelineCode())
When you insert a string through interpolation you only indent the first line of it. The following lines of the inserted string will be indented differently, which messes everything up.
Using some lesser-known members of GString (namely .strings[] and .values[]), we can align the indentation of all lines of each interpolated value.
String method2(String tier, String jobName) {
indented """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
"""
}
String simpleLog() {
indented """\
script {
def user = env.BUILD_USER_ID
}
"""
}
String method1() {
indented """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
indented """\
Text here
${method1()}
${method2("test1", "test2")}
"""
}
println generateFullDeploymentPipelineCode()
//---------- Move the following code into its own script ----------
// Function to adjust the indentation of interpolated values so that all lines
// of a value match the indentation of the first line.
// Finally stripIndent() will be called before returning the string.
String indented( GString templ ) {
// Iterate over the interpolated values of the GString template.
templ.values.eachWithIndex{ value, i ->
// Get the string preceding the current value. Always defined, even
// when the value is at the beginning of the template.
def beforeValue = templ.strings[ i ]
// RegEx to match any indent substring before the value.
// Special case for the first string, which doesn't necessarily contain '\n'.
def regexIndent = i == 0
? /(?:^|\n)([ \t]+)$/
: /\n([ \t]+)$/
def matchIndent = ( beforeValue =~ regexIndent )
if( matchIndent ) {
def indent = matchIndent[ 0 ][ 1 ]
def lines = value.readLines()
def linesNew = [ lines.head() ] // The 1st line is already indented.
// Insert the indentation from the 1st line into all subsequent lines.
linesNew += lines.tail().collect{ indent + it }
// Finally replace the value with the reformatted lines.
templ.values[ i ] = linesNew.join('\n')
}
}
return templ.stripIndent()
}
// Fallback in case the input string is not a GString (when it doesn't contain expressions)
String indented( String templ ) {
return templ.stripIndent()
}
Live Demo at codingground
Output:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Conclusion:
Using the indented function, a clean Groovy syntax for generating code from GString templates has been achieved.
This was quite a learning experience. I first tried to do it completely different using the evaluate function, which turned out to be too complicated and not so flexible. Then I randomly browsed through some posts from mrhaki blog (always a good read!) until I discovered "Groovy Goodness: Get to Know More About a GString". This was the key to implementing this solution.
Related
My jenkins log is too big and often consoleFull did not load it completely.
So using groovy script fetching logs between timestamp like below.
JOBNAME = 'My_Jenkins_job'
BUILDID = '34'
def start = 23
def end = 25
def time_prefix = '2021-11-30T08:'
for (job in Jenkins.instance.getAllItems(Job.class)) {
if (job.name == JOBNAME) {
for (build in job.builds) {
if (build.id == BUILDID) {
def lines = build.logFile.readLines()
println '===================================================='
println "Log summary between ${time_prefix}${start}..${end}"
println '======================================================='
for (int i in start..end) {
def logmsg = lines.findAll {
it.contains("${time_prefix}${i}")
}
logmsg.each {
println it
}
}
}
}
}
}
But here I am using string and contains to parse the timings ,But if I can use datetime in range it will be good. How to convert the date time format in Jenkins to groovy and put it in a range.
I am trying to convert time and it fails.
String log_time="2021-12-01T09:19:54-07:00"
String timestamp = Date.parse("yyyy-MM-dd'T'HH:mm-ss':00'", log_time)
println(timestamp)
If I can convert the jenkins timestamp and put it in range operator I can extract the log between timestamp. How to do it?
This is working for me:
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
def log_time='2021-12-01T09:19:54-07:00'
def timestamp = LocalDateTime.parse(log_time, DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssz"))
println timestamp
I am using regex to grab a number from a string in my pipeline
it works ok as long that I have a match, but when there is no match I get an error
java.lang.IndexOutOfBoundsException: index is out of range 0..-1 (index = 0)
There error happens when i try to capture the group on following line
env.ChangeNr = chngnr[0][1]
How can i test if there isn't a match from my capture group ?
This is the pipeline
pipeline {
agent {
node {
label 'myApplicationNode'
}
}
environment {
GIT_MESSAGE = "${bat(script: "git log --no-walk --format=format:%%s ${GIT_COMMIT}", returnStdout: true)}".readLines().drop(2).join(" ")
}
stages {
stage('get_commit_msg'){
steps {
script {
def gitmsg=env.GIT_MESSAGE
def chngnr = gitmsg =~/([0-9]{1,8})/
env.ChangeNr = chngnr[0][1] /* put test if nothing is extracted */
}
}
}
}
}
In groovy when you use the =~ (find operator) it actually creates a java.util.regex.Matcher and therefore you can use any of its standard methods like find() or size(), so in your case you can jest use the size function to test if there are any matched patterns before you attempt to extract any groups:
def chngnr = gitmsg =~/([0-9]{1,8})/
assert chngnr.size() > 0
env.ChangeNr = chngnr[0][1]
Another nice option is to use the =~ operator in context of boolean, in this case, Groovy implicitly invokes the matcher.find() method, which means that the expression evaluates to true if any part of the string matches the pattern:
def chngnr = gitmsg =~/([0-9]{1,8})/
if(chngnr){
env.ChangeNr = chngnr[0][1]
}
else {
...
}
You can read more info on Groovy Regular Expressions Here
I have a map of key value pairs in Dart. I want to convert it to YAML and write into a file.
I tried using YAML package from dart library but it only provides methods to load YAML data from a file. Nothing is mentioned on how to write it back to the YAML file.
Here is an example:
void main() {
var map = {
"name": "abc",
"type": "unknown",
"internal":{
"name": "xyz"
}
};
print(map);
}
Expected output:
example.yaml
name: abc
type: unknown
internal:
name: xyz
How to convert the dart map to YAML and write it to a file?
It's a bit late of a response but for anyone else looking at this question I have written this class. It may not be perfect but it works for what I'm doing and I haven't found anything wrong with it yet. Might make it a package eventually after writing tests.
class YamlWriter {
/// The amount of spaces for each level.
final int spaces;
/// Initialize the writer with the amount of [spaces] per level.
YamlWriter({
this.spaces = 2,
});
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String write(dynamic yaml) {
return _writeInternal(yaml).trim();
}
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String _writeInternal(dynamic yaml, { int indent = 0 }) {
String str = '';
if (yaml is List) {
str += _writeList(yaml, indent: indent);
} else if (yaml is Map) {
str += _writeMap(yaml, indent: indent);
} else if (yaml is String) {
str += "\"${yaml.replaceAll("\"", "\\\"")}\"";
} else {
str += yaml.toString();
}
return str;
}
/// Write a list to a YAML string.
/// Pass the list in as [yaml] and indent it to the [indent] level.
String _writeList(List yaml, { int indent = 0 }) {
String str = '\n';
for (var item in yaml) {
str += "${_indent(indent)}- ${_writeInternal(item, indent: indent + 1)}\n";
}
return str;
}
/// Write a map to a YAML string.
/// Pass the map in as [yaml] and indent it to the [indent] level.
String _writeMap(Map yaml, { int indent = 0 }) {
String str = '\n';
for (var key in yaml.keys) {
var value = yaml[key];
str += "${_indent(indent)}${key.toString()}: ${_writeInternal(value, indent: indent + 1)}\n";
}
return str;
}
/// Create an indented string for the level with the spaces config.
/// [indent] is the level of indent whereas [spaces] is the
/// amount of spaces that the string should be indented by.
String _indent(int indent) {
return ''.padLeft(indent * spaces, ' ');
}
}
Usage:
final writer = YamlWriter();
String yaml = writer.write({
'string': 'Foo',
'int': 1,
'double': 3.14,
'boolean': true,
'list': [
'Item One',
'Item Two',
true,
'Item Four',
],
'map': {
'foo': 'bar',
'list': ['Foo', 'Bar'],
},
});
File file = File('/path/to/file.yaml');
file.createSync();
file.writeAsStringSync(yaml);
Output:
string: "Foo"
int: 1
double: 3.14
boolean: true
list:
- "Item One"
- "Item Two"
- true
- "Item Four"
map:
foo: "bar"
list:
- "Foo"
- "Bar"
package:yaml does not have YAML writing features. You may have to look for another package that does that – or write your own.
As as stopgap, remember JSON is valid YAML, so you can always write out JSON to a .yaml file and it should work with any YAML parser.
I ran into the same issue and ended up hacking together a simple writer:
// Save the updated configuration settings to the config file
void saveConfig() {
var file = _configFile;
// truncate existing configuration
file.writeAsStringSync('');
// Write out new YAML document from JSON map
final config = configToJson();
config.forEach((key, value) {
if (value is Map) {
file.writeAsStringSync('\n$key:\n', mode: FileMode.writeOnlyAppend);
value.forEach((subkey, subvalue) {
file.writeAsStringSync(' $subkey: $subvalue\n',
mode: FileMode.writeOnlyAppend);
});
} else {
file.writeAsStringSync('$key: $value\n',
mode: FileMode.writeOnlyAppend);
}
});
}
I have a string like "AAA_revision12" and I have to extract the substring before "_" for example "AAA". I tried some regexps, but they doesn't work in jenkins.
String stringParser(String inputString) {
inputString ? inputString.split(/_\d/)[0] : ''
}
$string = "AAA revision".split('-')
assert string[0]
Your question is very confusing. I presume you are after a groovy snippet which will return the substring. If so:
String stringParser(String inputString) {
inputString.split("_")[0]
}
As an example:
String input = "foo_bar"
desired = "foo"
assert desired == stringParser(input)
> True
Context:
I'm writing a jenkins file in which i want to read an Input_params.txt file, search for a keyword then, print the values of the keyword into an array and then print each element of the array.
The contents of input params (format is "KEY:VALUE") file are:
sh> cat Input_params.txt
SOME_DETAILS:1234-A0;1456-B1;2456-B0;3436-D0;4467-C0
Step-1: Store the SOME_DETAILS in an array:
Integer[] testArray = ["1234-A0", "1456-B1", "2456-B0" , "3436-D0" , "4467-C0"]
Step-2: print the elements of the array sequentially. For example:
testArray.each {
println "chip num is ${it}"
}
Sample code:
println ("Check if the "Key:Value" is present in the Input_params.txt \n");
if ( new File("$JobPath/Input_params.txt").text?.contains("SOME_DETAILS"))
{
println ("INFO: "Key:Value" is present in the info_file.txt \n");
>>>>> Code to write the "value" of line with key "SOME_DETAILS" into an array here.. <<<<<
}
I need help in writing the Code to write the "value" of line with key "SOME_DETAILS" into an array.
def testArray=[]
new File("/11/tmp/1.txt").splitEachLine("[:;]"){line->
if(line[0]=='SOME_DETAILS')testArray=line[1..-1]
}
println testArray
Maybe something like:
def theFile = new File("$JobPath/Input_params.txt")
def linesWithDetails = theFile.findAll {
it.contains 'SOME_DETAILS'
}