How to show timestamps in short format in Jenkins Blue Ocean? - jenkins

Using Timestamper plugin 1.11.2 with globally enabled timestamps, using the default format, I get the following console output:
00:00:41.097 Some Message
In Blue Ocean the output shows like:
[2020-04-01T00:00:41.097Z] Some Message
How can I make it so that Blue Ocean uses the short timestamp format? The long format is somewhat unreadable and clutters the details view of the steps.
I've looked at the Pipeline Options too, but there is only the timestamps option which doesn't have a parameter to specify the format.
Note: This question isn't a dupe, because it asks for differences in time zone only.

Edit:
⚠️ Unfortunately this workaround doesn't work in the context of node, see JENKINS-59575. Looks like I have to finally get my hands dirty with plugin development, to do stuff like that in a supported way.
Anyway, I won't delete this answer, as the code may still be useful in other scenarios.
Original answer:
As a workaround, I have created a custom ConsoleLogFilter. It can be applied as a pipeline option, a stage option or at the steps level. If you have the timestamp plugin installed, you should disable the global timestamp option to prevent duplicate timestamps.
Typically you would define the low-level code in a shared library. Here is a sample that can be copy-pasted right into the pipeline script editor (you might have to disable Groovy sandbox):
import hudson.console.LineTransformationOutputStream
import hudson.console.ConsoleLogFilter
import java.nio.charset.Charset
import java.nio.charset.StandardCharsets
pipeline{
agent any
/*
options{
// Enable timestamps for the whole pipeline, using default format
//withContext( myTimestamps() )
// Enable timestamps for the whole pipeline, using custom format
//withContext( myTimestamps( dateFormat: 'HH:mm:ss', prefix: '', suffix: ' - ' ) )
}
*/
stages {
stage('A') {
options {
// Enable timestamps for this stage only
withContext( myTimestamps() )
}
steps {
echo 'Hello World'
}
}
stage('B') {
steps {
echo 'Hello World'
// Enable timestamps for some steps only
withMyTimestamps( dateFormat: 'HH:mm:ss') {
echo 'Hello World'
}
}
}
}
}
//----- Code below should be moved into a shared library -----
// For use as option at pipeline or stage level, e. g.: withContext( myTimestamps() )
def myTimestamps( Map args = [:] ) {
return new MyTimestampedLogFilter( args )
}
// For use as block wrapper at steps level
void withMyTimestamps( Map args = [:], Closure block ) {
withContext( new MyTimestampedLogFilter( args ), block )
}
class MyTimestampedLogFilter extends ConsoleLogFilter {
String dateFormat
String prefix
String suffix
MyTimestampedLogFilter( Map args = [:] ) {
this.dateFormat = args.dateFormat ?: 'YY-MM-dd HH:mm:ss'
this.prefix = args.prefix ?: '['
this.suffix = args.suffix ?: '] '
}
#NonCPS
OutputStream decorateLogger( AbstractBuild build, OutputStream logger )
throws IOException, InterruptedException {
return new MyTimestampedOutputStream( logger, StandardCharsets.UTF_8, this.dateFormat, this.prefix, this.suffix )
}
}
class MyTimestampedOutputStream extends LineTransformationOutputStream {
OutputStream logger
Charset charset
String dateFormat
String prefix
String suffix
MyTimestampedOutputStream( OutputStream logger, Charset charset, String dateFormat, String prefix, String suffix ) {
this.logger = logger
this.charset = charset
this.dateFormat = dateFormat
this.prefix = prefix
this.suffix = suffix
}
#NonCPS
void close() throws IOException {
super.close();
logger.close();
}
#NonCPS
void eol( byte[] bytes, int len ) throws IOException {
def lineIn = charset.decode( java.nio.ByteBuffer.wrap( bytes, 0, len ) ).toString()
def dateFormatted = new Date().format( this.dateFormat )
def lineOut = "${this.prefix}${dateFormatted}${this.suffix}${lineIn}\n"
logger.write( lineOut.getBytes( charset ) )
}
}
Example output for stage "B":
Credits:
I got the idea from this answer.

Related

How to parse build.gradle just like Gradle 7 that supports variable references at end of interpolated strings not enclosed in curly brackets

NOTE: I have replicated this issue on Windows 10 and MacOS 12.6.2. I get the same results with Groovy 4.0.6 and Groovy 4.0.7 (and after updates: 3.0.9).
In order to automate some changes to build.gradle files that are written in Groovy, I'm using the official Apache Groovy parser 4.0.7 (which uses ANTLR 4.11.1) to parse the Groovy build.gradle files. I'm finding that the Groovy parser fails to parse build.gradle files that include variable references at the end of interpolated strings that are not enclosed in curly brackets. However, Gradle 7.5.1 successfully parses the same build.gradle file just fine with the unbracketed variable references at the end of interpolated strings.
To avoid the parser failures I am having to preprocess the build.gradle files to enclose the variable names in curly brackets when located at the end of an interpolated string. However, that adds complications and risks of RegExp pattern errors considering all the various scenarios I have to deal with. To some degree, that involves creating my own mini Groovy parser via RegExp.
Maybe Gradle is preprocessing the build.gradle file? Maybe Gradle 7.5.1 uses a different parser than the official Apache Groovy parser 4.0.7?
I would like to match what Gradle is doing to parse the build.gradle files as closely as possible to avoid these kinds of issues.
The following OneCompiler clip (https://onecompiler.com/groovy/3yuaw5jdv) shows both bracketed and unbracketed vars at the end of interpolated strings working for the following source in Groovy 2.7 (note: my program uses Groovy 4.0.7):
String name = "Joe"
println "Hello, ${name}"
println "Hello, $name "
println "Hello, $name/"
println "Hello, $name" // this fails in Groovy Parser 4.0.7
Output:
#1 Hello, Joe
#2 Hello, Joe
#3 Hello, Joe/
#4 Hello, Joe
I can paste the above code into a build.gradle file and run ./gradlew.bat build -x test without errors to get the expected output:
// build.gradle
plugins {
id 'java'
}
group 'org.example'
version '1.0-SNAPSHOT'
String name = "Joe"
println "#1 Hello, ${name}"
println "#2 Hello, $name "
println "#3 Hello, $name/"
println "#4 Hello, $name" // this fails in Groovy Parser 4.0.7
repositories {
mavenCentral()
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'
implementation 'org.apache.groovy:groovy:4.0.7'
}
test {
useJUnitPlatform()
}
Output:
PS C:\projects\groovy\groovy407> ./gradlew.bat build
> Configure project :
#1 Hello, Joe
#2 Hello, Joe
#3 Hello, Joe/
#4 Hello, Joe
BUILD SUCCESSFUL in 7s
4 actionable tasks: 4 executed
However, when I try to parse the above build.gradle file via the Groovy Parser 4.0.7, it can't parse the file and reports parser errors when a variable reference is at the end of an interpolated string and not enclosed in curly brackets.
The following Java class and Java test class demonstrate how the Groovy parser 4.0.7 handles the variable references at the end of interpolated strings not enclosed in curly brackets.
InterpolatedStringParseTester.java:
package org.example;
import groovyjarjarantlr4.v4.runtime.CharStreams;
import groovyjarjarantlr4.v4.runtime.CommonTokenStream;
import groovyjarjarantlr4.v4.runtime.InputMismatchException;
import org.apache.groovy.parser.antlr4.GroovyLexer;
import org.apache.groovy.parser.antlr4.GroovyParser;
import java.rmi.UnexpectedException;
public class InterpolatedStringParseTester {
public String upgrade(String contents) throws UnexpectedException{
try {
GroovyLexer groovyLexer = new GroovyLexer(CharStreams.fromString(contents));
CommonTokenStream tokens = new CommonTokenStream(groovyLexer);
GroovyParser groovyParser = new GroovyParser(tokens);
GroovyParser.CompilationUnitContext tree = groovyParser.compilationUnit();
if (tree.exception instanceof InputMismatchException) {
throw new UnexpectedException("failed to parse contents: ", tree.exception);
}
} catch (Exception e) {
e.printStackTrace();
throw new UnexpectedException(e.getLocalizedMessage());
}
return "";
}
}
InterpolatedStringParseTesterTest.java:
package org.example;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.rmi.UnexpectedException;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
class InterpolatedStringParseTesterTest {
String bracketedVarInStrInClass_OK = "class AClass { String x=\"y\" \n" +
"String a=\"${x}\" }";
String unbracketedVarInStrInClass_ParserErr = "class AClass { String x=\"y\" \n" +
"String a=\"$x\" }";
String closureWithVarInStr_OK = "{ String x-> println \"${x}\" }";
String closureWithUnbracketedVarInStrFailsParse_ThrowsException = "{ String x-> println \"$x\" }";
String helloJoe4Ways = "String name = \"Joe\"\n" +
"println \"#1 Hello, ${name}\"\n" +
"println \"#2 Hello, $name \"\n" +
"println \"#3 Hello, $name/\"\n" +
"println \"#4 Hello, $name\" // this fails in Groovy Parser 4.0.7\n";
#Test
void bracketedVarInStrInClassTest_OK() throws UnexpectedException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
assertEquals("", interpolatedStringParseTester.upgrade(bracketedVarInStrInClass_OK));
}
#Test
void unbracketedVarInStrInClassTest_NoException_ButParserErr() throws UnexpectedException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
assertEquals("", interpolatedStringParseTester.upgrade(unbracketedVarInStrInClass_ParserErr));
// does not throw an error, but outputs error: line 2:13 mismatched input ' }' expecting {GStringEnd, GStringPart}
}
#Test
void closureWithVarInStrTest_OK() throws UnexpectedException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
assertEquals("", interpolatedStringParseTester.upgrade(closureWithVarInStr_OK));
}
#Test
void closureWithUnbracketedVarInStrTest_ThrowsException() throws UnexpectedException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
assertThrows(UnexpectedException.class,
() -> {
interpolatedStringParseTester.upgrade(closureWithUnbracketedVarInStrFailsParse_ThrowsException);
/*
Throws:
line 1:21 missing RBRACE at '"$'
java.rmi.UnexpectedException: failed to parse contents: ; nested exception is:
groovyjarjarantlr4.v4.runtime.InputMismatchException
*/
});
}
#Test
void helloJoe4WaysTest_ThrowsInputMismatchException() throws UnexpectedException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
assertThrows(UnexpectedException.class,
() -> {
interpolatedStringParseTester.upgrade(helloJoe4Ways);
});
/**
* this results in:
* line 5:25 token recognition error at: ' // this fails in Groovy Parser 4.0.7\n'
* line 5:8 mismatched input '"#4 Hello, $' expecting {<EOF>, ';', NL}
* java.rmi.UnexpectedException: failed to parse contents: ; nested exception is:
*/
}
#Test
void buildGradleParse_ThrowsInputMismatchException() throws IOException {
InterpolatedStringParseTester interpolatedStringParseTester = new InterpolatedStringParseTester();
Path path = Paths.get("build.gradle");
Stream<String> lines = Files.lines(path);
String buildGradleContents = lines.collect(Collectors.joining("\n"));
lines.close();
assertThrows(UnexpectedException.class,
() -> {
interpolatedStringParseTester.upgrade(buildGradleContents);
});
}
}
UPDATE:
Below is the stderr output along with the stacktrace of the exception that is returned (i.e., returned in the tree.exception field as opposed to being thrown) by GroovyParser.CompilationUnitContext tree = groovyParser.compilationUnit(); in the unit test named helloJoe4WaysTest_ThrowsInputMismatchException:
line 5:25 token recognition error at: ' // this fails in Groovy Parser 4.0.7\n'
line 5:8 mismatched input '"#4 Hello, $' expecting {<EOF>, ';', NL}
groovyjarjarantlr4.v4.runtime.InputMismatchException
at groovyjarjarantlr4.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:492)
at groovyjarjarantlr4.v4.runtime.Parser.match(Parser.java:213)
at org.apache.groovy.parser.antlr4.GroovyParser.compilationUnit(GroovyParser.java:368)
at org.example.InterpolatedStringParseTester.upgrade(InterpolatedStringParseTester.java:18)
at org.example.InterpolatedStringParseTesterTest.lambda$helloJoe4WaysTest_ThrowsInputMismatchException$1(InterpolatedStringParseTesterTest.java:71)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:55)
at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:37)
at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3082)
at org.example.InterpolatedStringParseTesterTest.helloJoe4WaysTest_ThrowsInputMismatchException(InterpolatedStringParseTesterTest.java:69)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy2.stop(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
UPDATE:
On Windows 10, Gradle is using:
Gradle version: 7.4
Groovy version: 3.0.9
On MacOS 12.6.1 Gradle is using:
Gradle version: 7.4
Groovy version: 3.0.9
When I have more time I will try to re-run these tests with the parser that comes with Groovy 3.0.9 and 3.0.10 to match what Gradle is using.
UPDATE:
Since learning that Gradle 7.4 uses Groovy 3.0.9, I have re-run the same tests with Groovy 3.0.9 that result in the same exception when the variable reference is not wrapped in curl brackets and is at the end of a double-quoted interpolated string.
The following is the dependency in build.gradle to make use of the Groovy parse v3.0.9:
implementation 'org.codehaus.groovy:groovy-all:3.0.9'
I created a simple Java project to test Groovy 3.0.9.
Below is the code and output that demonstrates the difference between "with curl brackets" and "without curly brackets" around variable references that appear at the end of interpolated strings:
Main.java:
package org.example;
import groovyjarjarantlr4.v4.runtime.CharStreams;
import groovyjarjarantlr4.v4.runtime.CommonTokenStream;
import org.apache.groovy.parser.antlr4.GroovyLexer;
import org.apache.groovy.parser.antlr4.GroovyParser;
public class Main {
public static void main(String[] args) {
Main main = new Main();
boolean useBrackets = args.length> 0 && "y".equalsIgnoreCase(args[0]);
String brackets = "buildscript { def v=\"1.2.3\"; dependencies { classpath { \"grp:pkg:${v}\" } } }";
String noBrackets = "buildscript { def v=\"1.2.3\"; dependencies { classpath { \"grp:pkg:$v\" } } }";
String groovyCode = useBrackets ? brackets : noBrackets;
System.out.println("Groovy code:" + groovyCode);
main.runVisitor(groovyCode);
}
void runVisitor(String buildGradleContents) {
GroovyLexer groovyLexer = new GroovyLexer(CharStreams.fromString(buildGradleContents));
CommonTokenStream tokens = new CommonTokenStream(groovyLexer);
GroovyParser groovyParser = new GroovyParser(tokens);
GroovyParser.CompilationUnitContext tree = groovyParser.compilationUnit();
if (tree.exception != null) {
tree.exception.printStackTrace();
throw tree.exception;
}
TestVisitor testVisitor = new TestVisitor();
String results = testVisitor.visit(tree);
System.out.println("results: "+ results);
}
}
TestVisitor.java:
package org.example;
import groovyjarjarantlr4.v4.runtime.Token;
import org.apache.groovy.parser.antlr4.GroovyParser;
import org.apache.groovy.parser.antlr4.GroovyParserBaseVisitor;
public class TestVisitor extends GroovyParserBaseVisitor<String>{
public String visitIdentifier(GroovyParser.IdentifierContext ctx) {
Token startToken = ctx.getStart();
if (startToken != null) {
String startTxt = startToken.getText();
}
return visitChildren(ctx);
}
}
OUTPUT:
# WITH curly brackets
PS C:\projects\research\gradle-tooling> java -jar .\build\libs\gradle-tooling-1.0-SNAPSHOT.jar y
Groovy code:buildscript { def v="1.2.3"; dependencies { classpath { "grp:pkg:${v}" } } }
results: null
# WITHOUT curly brackets
PS C:\projects\research\gradle-tooling> java -jar .\build\libs\gradle-tooling-1.0-SNAPSHOT.jar n
Groovy code:buildscript { def v="1.2.3"; dependencies { classpath { "grp:pkg:$v" } } }
line 1:12 mismatched input '{' expecting {<EOF>, StringLiteral, GStringBegin, 'as', 'in', 'trait', 'var', IntegerLiteral, FloatingPointLiteral, BooleanLiteral, 'null', ';', Capitalized
Identifier, Identifier, NL}
groovyjarjarantlr4.v4.runtime.InputMismatchException
at groovyjarjarantlr4.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:492)
at groovyjarjarantlr4.v4.runtime.Parser.match(Parser.java:213)
at org.apache.groovy.parser.antlr4.GroovyParser.compilationUnit(GroovyParser.java:362)
at org.example.Main.runVisitor(Main.java:28)
at org.example.Main.main(Main.java:21)
Exception in thread "main" groovyjarjarantlr4.v4.runtime.InputMismatchException
at groovyjarjarantlr4.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:492)
at groovyjarjarantlr4.v4.runtime.Parser.match(Parser.java:213)
at org.apache.groovy.parser.antlr4.GroovyParser.compilationUnit(GroovyParser.java:362)
at org.example.Main.runVisitor(Main.java:28)
at org.example.Main.main(Main.java:21)

Use pipeline method in jenkins class

I would like to make a resilient slack notification class for my Jenkins instance.
Currently I just call the slackSend function from the Jenkins slack plugin. However if the call to slack fails, so does my build. This means I have a very hard dependency on Slack that I am not ok with.
I have managed to get the code to work but it is rather ugly and I would like to know if there is a better way to do this.
Firstly I have class defined in src/something/slack/slackHandler.groovy
The code looks a little like this:
package something.slack
public class slackHandler {
private String threadId = ""
private String channelName = ""
private Boolean silent = false
private Closure sender
private Logger logger
public silence(Boolean trigger) {
this.silent = trigger
}
public sendMessage(String msg) {
if (threadId == "") {
def (slackResponse, ok) = this.sendMessageTo(this.channelName, msg)
if (!ok) {
// We have tried to send a on channel. But the send failed so we can not determine the threadId.
// We should at this point store the messages and try send it again on the next call to slack.
return
}
this.setThreadId(slackResponse.threadId)
} else {
def (slackResponse, ok) = this.sendMessageTo(this.threadId, msg)
if (!ok){
// We tried to send on a threadId, it failed. We have the threadId so we can leave the slackResponse alone.
// We should at this point store the messages and try send it again on the next call to slack.
return
}
}
}
public sendMessageTo(String channel, String msg) {
return this.trySend(channel, msg)
}
private trySend(String to, String msg) {
if (this.silent) {
return [[threadId: "nothing"], true]
}
try {
return [sender(to, msg), true]
} catch (e) {
// These do not work :(
println("There wasn an error sending the slack message. Error $e")
println("Message being sent: $msg")
return [null, false]
}
}
}
This is the part I'm not happy about. To use the above code I need to create and pass a closure to send the messages to slack because the slackSend function is not available inside my class. I can not find out how to give it the ability to use this function or access the slack class. The println calls are also not showing up in the Jenkins console which is a problem for this class as the fallback is to log to the console. This makes me think that I am missing some sort of context that I need to give my class.
This is how I currently use the class:
def slackSender = new slackHandler(
[
channelName:"rd-bots",
sender: {to, msg -> return slackSend(channel: to, message: msg)}
]
)
slackSender.sendMessage("Hello :wave:")
Can someone please tell me if the is a way to pass on the context or if what I have done is the only way? Also why don't the println calls appear in the Jenkins log?
You can include a try-catch in your pipeline code around your call to your slack handler.
Here's the documentation for catchError: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#catcherror-catch-error-and-set-build-result-to-failure
I think it should look something like this in a pipeline:
#Library('shared-lib#main')_
pipeline{
agent any
options {
timestamps()
}
environment {
Custom_Variables = 'stuff'
}
stages{
stage ('blah') {
steps {
catchError{
def slackSender = new slackHandler(
[
channelName:"rd-bots",
sender: {to, msg -> return slackSend(channel: to, message: msg)}
]
)
slackSender.sendMessage("Hello :wave:")
}
}
}
}
}
After a bit of digging and looking at different questions for other things where context is needed I managed to find an answer.
When running in Jenkins the this value will give the context of your running environment. You can pass that on to something else.
Code updated to look like this:
public class slackHandler {
private String threadId = ""
private String channelName = ""
private Boolean silent = false
private ctx
public silence(Boolean trigger) {
this.silent = trigger
}
public setThreadId(String id) {
this.threadId = id
}
public sendMessage(String msg) {
if (threadId == "") {
def (slackResponse, ok) = this.sendMessageTo(this.channelName, msg)
if (!ok) {
// We have tried to send a on channel. But the send failed so we can not determine the threadId.
// We should at this point store the messages and try send it again on the next call to slack.
return
}
this.setThreadId(slackResponse.threadId)
} else {
def (slackResponse, ok) = this.sendMessageTo(this.threadId, msg)
if (!ok){
// We tried to send on a threadId, it failed. We have the threadId so we can leave the slackResponse alone.
// We should at this point store the messages and try send it again on the next call to slack.
return
}
}
}
public sendMessageTo(String channel, String msg) {
return this.trySend(channel, msg)
}
private trySend(String to, String msg) {
if (this.silent) {
return [[threadId: "nothing"], true]
}
def slackResponse = ctx.slackSend(channel: to, message: msg)
if (slackResponse != null) {
return [slackResponse, true]
} else {
ctx.echo("There was an error sending slack message sent: $msg")
return [null, false]
}
}
}
Used like this:
import com.proquoai.slack.slackHandler
def slackSender = new slackHandler(
[
channelName:"trashroom10120123",
ctx: this
]
)
node ('docker') {
stage('Send Slack Messages') {
slackSender.sendMessage("Hello :wave:")
slackSender.sendMessage("It's running :go_dance:")
}
stage('Send out of band messages') {
slackSender.sendMessageTo("rd-bots", ":ship-it:")
}
}
As a side note, the slackSend function appears to swallow the error and simply doesn't return a slackResponse. Therefore using a try/catch block didn't actually help in determining if slack sending failed.

How to Jenkins Groovy scripting for live fetching of Docker image + authentication

I have a script groovy, this script for live fetching of docker image,
I want to add the authentication function with the private repository, but I am not familiar with groovy, who can help me, thanks
import groovy.json.JsonSlurper
// Set the URL we want to read from, it is MySQL from official Library for this example, limited to 20 results only.
docker_image_tags_url = "https://registry.adx.abc/v2/mysql/tags/list"
try {
// Set requirements for the HTTP GET request, you can add Content-Type headers and so on...
def http_client = new URL(docker_image_tags_url).openConnection() as HttpURLConnection
http_client.setRequestMethod('GET')
// Run the HTTP request
http_client.connect()
// Prepare a variable where we save parsed JSON as a HashMap, it's good for our use case, as we just need the 'name' of each tag.
def dockerhub_response = [:]
// Check if we got HTTP 200, otherwise exit
if (http_client.responseCode == 200) {
dockerhub_response = new JsonSlurper().parseText(http_client.inputStream.getText('UTF-8'))
} else {
println("HTTP response error")
System.exit(0)
}
// Prepare a List to collect the tag names into
def image_tag_list = []
// Iterate the HashMap of all Tags and grab only their "names" into our List
dockerhub_response.results.each { tag_metadata ->
image_tag_list.add(tag_metadata.name)
}
// The returned value MUST be a Groovy type of List or a related type (inherited from List)
// It is necessary for the Active Choice plugin to display results in a combo-box
return image_tag_list.sort()
} catch (Exception e) {
// handle exceptions like timeout, connection errors, etc.
println(e)
}
The problem has been resolved, thank you everyone for your help
// Import the JsonSlurper class to parse Dockerhub API response
import groovy.json.JsonSlurper
// Set the URL we want to read from, it is MySQL from official Library for this example, limited to 20 results only.
docker_image_tags_url = "https://registry.adx.vn/v2/form-be/tags/list"
try {
// Set requirements for the HTTP GET request, you can add Content-Type headers and so on...
def http_client = new URL(docker_image_tags_url).openConnection() as HttpURLConnection
http_client.setRequestMethod('GET')
String userCredentials = "your_user:your_passwd";
String basicAuth = "Basic " + new String(Base64.getEncoder().encode(userCredentials.getBytes()));
http_client.setRequestProperty ("Authorization", basicAuth);
// Run the HTTP request
http_client.connect()
// Prepare a variable where we save parsed JSON as a HashMap, it's good for our use case, as we just need the 'name' of each tag.
def dockerhub_response = [:]
// Check if we got HTTP 200, otherwise exit
if (http_client.responseCode == 200) {
dockerhub_response = new JsonSlurper().parseText(http_client.inputStream.getText('UTF-8'))
} else {
println("HTTP response error")
System.exit(0)
}
// Prepare a List to collect the tag names into
def image_tag_list = []
// Iterate the HashMap of all Tags and grab only their "names" into our List
dockerhub_response.tags.each { tag_metadata ->
image_tag_list.add(tag_metadata)
}
// The returned value MUST be a Groovy type of List or a related type (inherited from List)
// It is necessary for the Active Choice plugin to display results in a combo-box
return image_tag_list.sort()
} catch (Exception e) {
// handle exceptions like timeout, connection errors, etc.
println(e)
}
here is the result

Jenkins gives error on configuring Amazon EC2 cloud

I'm trying to setup a Jenkins server with EC2 plugin such that all builds are executed by the EC2 instances, which act as agents for the master server.
While trying to save the cloud configuration for 'Amazon EC2' cloud in the Jenkins master, it fails with an exception. Checking in the Jenkins logs, I see this exception:
Caught unhandled exception with ID f6d45d51-fb00-4d1c-a474-0a55dd5ee710
org.kohsuke.stapler.WrongTypeException: Got type array but no lister class found for type class java.lang.String
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:724)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:478)
at org.kohsuke.stapler.RequestImpl.instantiate(RequestImpl.java:787)
Caused: java.lang.IllegalArgumentException: Failed to convert the instanceCapStr parameter of
the constructor public hudson.plugins.ec2.AmazonEC2Cloud
(java.lang.String,boolean,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.List,java.lang.String,java.lang.String)
at org.kohsuke.stapler.RequestImpl.instantiate(RequestImpl.java:789)
at org.kohsuke.stapler.RequestImpl.access$200(RequestImpl.java:83)
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:678)
Caused: java.lang.IllegalArgumentException: Failed to instantiate class hudson.plugins.ec2.AmazonEC2Cloud from
{
"cloudName":"ec2-cloud",
"includeUser":["false","false"],
"credentialsId":"",
"useInstanceProfileForCredentials":true,
"altEC2Endpoint":"",
"region":"eu-west-1",
"sshKeysCredentialsId":"jenkins-slave-ssh-key",
"instanceCapStr":["",""],
"noDelayProvisioning":false,
"roleArn":"",
"roleSessionName":"",
"templates":{
"description":"Amazon Linux 2 AMI",
"ami":"ami-0bb3fad3c0286ebd5",
"type":"T2Micro",
"ebsOptimized":true,
"monitoring":true,
"t2Unlimited":false,
"zone":"",
"securityGroups":"",
"remoteFS":"/var/lib/jenkins",
"remoteAdmin":"ec2-user",
"":"0",
"amiType":{
"rootCommandPrefix":"",
"slaveCommandPrefix":"",
"slaveCommandSuffix":"",
"sshPort":"22",
"stapler-class":"hudson.plugins.ec2.UnixData",
"$class":"hudson.plugins.ec2.UnixData"
},
"labelString":"ec2",
"mode":"EXCLUSIVE",
"idleTerminationMinutes":"30",
"initScript":"",
"tmpDir":"",
"userData":"",
"numExecutors":"",
"jvmopts":"",
"stopOnTerminate":false,
"subnetId":"",
"useDedicatedTenancy":false,
"name":"","value":""
},
"minimumNumberOfInstances":"0",
"minimumNumberOfSpareInstances":"0",
"iamInstanceProfile":"arn:aws:iam::xxxxxxxxxxxx:instance-profile/jenkins_server_role",
"deleteRootOnTermination":true,
"useEphemeralDevices":true,
"customDeviceMapping":"",
"launchTimeoutStr":"",
"associatePublicIp":false,
"connectionStrategy":"PRIVATE_IP",
"connectBySSHProcess":false,
"hostKeyVerificationStrategy":"CHECK_NEW_HARD",
"maxTotalUses":"-1",
"nodeProperties":{"stapler-class-bag":"true"}
},"stapler-class":"hudson.plugins.ec2.AmazonEC2Cloud","$class":"hudson.plugins.ec2.AmazonEC2Cloud"}
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:681)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:478)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:474)
at hudson.model.Descriptor.newInstance(Descriptor.java:598)
I do see the property 'Instance cap' in two different locations in the Jenkins UI. My understanding is that one is for configuring a max limit on the total number of instances allowed in the entire cloud,
and the other property describes a max limit on the number of instances for the particular AMI.
Is this a Jenkins issue? Or is it something wrong with the configuration I've provided?
NOTE: I'm okay with providing the configuration as code rather than via the Jenkins UI. If anyone is able to provide the same configuration through code, that is also fine.
I eventually ended up configuring this using a groovy script run in the Script Console in the Jenkins UI. The groovy script I used is:
import hudson.model.*
import jenkins.model.*
import hudson.plugins.ec2.*
import com.amazonaws.services.ec2.model.InstanceType
def instance = Jenkins.getInstance()
def ec2_cloud_name = 'ec2-cloud'
def ec2_instance_cap = 5
def worker_description = 'jenkins-slave running in ec2 instance'
def worker_label_string = 'ec2-slave'
def ami_id = 'ami-xxxxxxxxxxxxxxxxx'
def security_groups = 'sg-xxxxxxxxxxxxxxxxxx'
def subnet_id = 'subnet-xxxxxxxx'
def instance_type = 't2.micro'
def instance_profile_arn = 'arn:aws:iam::xxxxxxxxxxxx:instance-profile/jenkins_server_role'
def number_of_executors = 2
def ec2_tags = [
new EC2Tag('Name', 'jenkins-slave-instance')
]
def priv_key_txt = '''
-----BEGIN RSA PRIVATE KEY-----
<My Private key>
-----END RSA PRIVATE KEY-----
'''
def worker_ami = new SlaveTemplate(
// String ami
ami_id,
// String zone
'',
// SpotConfiguration spotConfig
null,
// String securityGroups
security_groups,
// String remoteFS
'',
// InstanceType type
InstanceType.fromValue(instance_type),
// boolean ebsOptimized
false,
// String labelString
worker_label_string,
// Node.Mode mode
Node.Mode.NORMAL,
// String description
worker_description,
// String initScript
'',
// String tmpDir
'',
// String userData
'',
// String numExecutors
"${number_of_executors}",
// String remoteAdmin
'',
// AMITypeData amiType
new UnixData(null, null, null, null),
// String jvmopts
'',
// boolean stopOnTerminate
false,
// String subnetId
subnet_id,
// List<EC2Tag> tags
ec2_tags,
// String idleTerminationMinutes
'30',
// int minimumNumberOfInstances
0,
// int minimumNumberOfSpareInstances
0,
// string instanceCapStr
'3',
// string iamInstanceProfile
'arn:aws:iam::xxxxxxxxxxxx:instance-profile/jenkins_server_role',
// boolean deleteRootOnTermination
true,
// boolean useEphemeralDevices
true,
// boolean useDedicatedTenancy
false,
// String launchTimeoutStr
'1800',
// boolean associatePublicIp
false,
// String customDeviceMapping
'',
// boolean connectBySSHProcess
false,
// boolean monitoring
false,
// boolean t2Unlimited
false,
// Enum connectionStrategy
ConnectionStrategy.PRIVATE_IP,
// int maxTotalUses
3,
// List<? extends NodeProperty<?>> nodeProperties
[],
// HostKeyVerificationStrategyEnum
HostKeyVerificationStrategyEnum.CHECK_NEW_HARD
)
def new_cloud = new AmazonEC2Cloud(
// String cloudName
ec2_cloud_name,
// boolean useInstanceProfileForCredentials
true,
// String credentialsId
'',
// String region
'eu-west-1',
// String privateKey
priv_key_txt,
// String sshKeysCredentialsId
'jenkins-slave-ssh-key',
// String instanceCapStr
"3",
// List<? extends SlaveTemplate> templates
[worker_ami],
// String roleArn
null,
// String roleSessionName
null
)
instance.clouds.add(new_cloud)
Strange thing is, after creating the config by running this script, now I am able to edit and save the created config via the Jenkins UI.

Jenkins Pipeline Execute Multiple FreeStyleProjects in Parallel [duplicate]

The script is not iterating through all the values of the 'modules' array.
class Module {
public String name = '';
public Boolean isCustom = false;
public Module(String name, Boolean custom){
this.name = name;
this.isCustom = custom;
}
}
//creates array from the ext_module env var
modules = [];
EXT_MODULE.split(',').each {
modules.add(new Module(it, false));
}
println modules;
modules.each {
println "MODULE NAME ::::: ${it.name}"
if(it.isCustom)
{
println "install custom";
} else {
println "install non custom";
}
};
This is the result of the run. The array shows 4 elements, but the code inside the .each black only executes once.
Running: Print Message
[Module#71f09325, Module#e1ddb41, Module#7069a674, Module#1f68f952]
Running: Print Message
MODULE NAME ::::: puppetlabs-ntp
Running: Print Message
install non custom
Running: End of Workflow
Finished: SUCCESS
The messages "Running: Print Message" and "Running: End of Workflow" indicate that you are using the new workflow plugin: https://wiki.jenkins-ci.org/display/JENKINS/Workflow+Plugin. This plugin currently has a bug causing at least some Groovy iterations involving a closure to be aborted after one iteration: https://issues.jenkins-ci.org/browse/JENKINS-26481
The workaround is to simply use an old school for loop (code below).
Also, NonCPS is another workaround.
There is an open issue for this matter. See here: https://issues.jenkins-ci.org/browse/JENKINS-26481
Update, Oct 24th, 2016
/**
* Dumps environment varibles to the log, using an old school for loop.
*/
import com.cloudbees.groovy.cps.NonCPS
def version = '1.0'
#NonCPS
def dumpEnvVars() {
def str = "Dumping build environment variables...\n"
for (Map.Entry<String, String> entry : currentBuild.build().environment) {
str += " ${entry.key} = ${entry.value}\n"
}
echo str
}
return this;
As of yesterday, the new Pipeline plugin was delivered in version 2.0 and correct this problem.
.each closures now work, but .collect still only iterate once.

Resources