My jenkins log is too big and often consoleFull did not load it completely.
So using groovy script fetching logs between timestamp like below.
JOBNAME = 'My_Jenkins_job'
BUILDID = '34'
def start = 23
def end = 25
def time_prefix = '2021-11-30T08:'
for (job in Jenkins.instance.getAllItems(Job.class)) {
if (job.name == JOBNAME) {
for (build in job.builds) {
if (build.id == BUILDID) {
def lines = build.logFile.readLines()
println '===================================================='
println "Log summary between ${time_prefix}${start}..${end}"
println '======================================================='
for (int i in start..end) {
def logmsg = lines.findAll {
it.contains("${time_prefix}${i}")
}
logmsg.each {
println it
}
}
}
}
}
}
But here I am using string and contains to parse the timings ,But if I can use datetime in range it will be good. How to convert the date time format in Jenkins to groovy and put it in a range.
I am trying to convert time and it fails.
String log_time="2021-12-01T09:19:54-07:00"
String timestamp = Date.parse("yyyy-MM-dd'T'HH:mm-ss':00'", log_time)
println(timestamp)
If I can convert the jenkins timestamp and put it in range operator I can extract the log between timestamp. How to do it?
This is working for me:
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter
def log_time='2021-12-01T09:19:54-07:00'
def timestamp = LocalDateTime.parse(log_time, DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssz"))
println timestamp
Related
I am using regex to grab a number from a string in my pipeline
it works ok as long that I have a match, but when there is no match I get an error
java.lang.IndexOutOfBoundsException: index is out of range 0..-1 (index = 0)
There error happens when i try to capture the group on following line
env.ChangeNr = chngnr[0][1]
How can i test if there isn't a match from my capture group ?
This is the pipeline
pipeline {
agent {
node {
label 'myApplicationNode'
}
}
environment {
GIT_MESSAGE = "${bat(script: "git log --no-walk --format=format:%%s ${GIT_COMMIT}", returnStdout: true)}".readLines().drop(2).join(" ")
}
stages {
stage('get_commit_msg'){
steps {
script {
def gitmsg=env.GIT_MESSAGE
def chngnr = gitmsg =~/([0-9]{1,8})/
env.ChangeNr = chngnr[0][1] /* put test if nothing is extracted */
}
}
}
}
}
In groovy when you use the =~ (find operator) it actually creates a java.util.regex.Matcher and therefore you can use any of its standard methods like find() or size(), so in your case you can jest use the size function to test if there are any matched patterns before you attempt to extract any groups:
def chngnr = gitmsg =~/([0-9]{1,8})/
assert chngnr.size() > 0
env.ChangeNr = chngnr[0][1]
Another nice option is to use the =~ operator in context of boolean, in this case, Groovy implicitly invokes the matcher.find() method, which means that the expression evaluates to true if any part of the string matches the pattern:
def chngnr = gitmsg =~/([0-9]{1,8})/
if(chngnr){
env.ChangeNr = chngnr[0][1]
}
else {
...
}
You can read more info on Groovy Regular Expressions Here
I am newbie to jenkins pipeline scripting and i am just trying to concatenate date to string getting below No Such Property exception. Dont know where am doing wrong. Could some one please help me to resolve this
def generateRandomText(){
def temp = ""
try{
Date date = new Date()
String datePart = date.format("ddHHmmssSSS")
temp = "abcde" + datepart
echo "printing ... $temp"
return temp
}
catch(theError){
echo "Error getting while generating random text: {$theError}"
}
return temp
}
MissingPropertyException means the variable wasn't declared.
There were some errors in your code:
You used echo, which doesn't exist in Groovy. Use one of the print functions instead. On the code below I used println
The datePart variable was mispelled
Here's your code fixed:
def generateRandomText(){
def temp = ""
try{
Date date = new Date()
String datePart = date.format("ddHHmmssSSS")
temp = "abcde" + datePart
println "printing ... $temp"
return temp
}
catch(theError){
println "Error getting while generating random text: {$theError}"
}
return temp
}
generateRandomText()
Output on groovyConsole:
printing ... abcde21195603124
Result: abcde21195603124
See Groovy's documentation.
I am trying to generate some generic Groovy code for Jenkins but I seem to have trouble with multi line strings and extra white space. I've tried everything I could find by Googling but I can't seem to get it working.
My issue isn't related to simple multi line strings. I managed to trim white space by using the stripIndent() and stripMargin() methods for simple cases. My issue is caused by having interpolated methods inside my strings.
Groovy info: Groovy Version: 3.0.2 JVM: 13.0.2 Vendor: Oracle Corporation OS: Mac OS X
String method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".stripIndent().stripMargin()
}
static String simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".stripIndent().stripMargin()
}
static String method1() {
return """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".stripIndent().stripMargin()
}
String generateFullDeploymentPipelineCode() {
return """Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
This is what it prints(or writes to disk):
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Why the extra space around the import lines? I know the indentation method is supposed to trim all white space according to the least number of leading spaces, so that's why we use backslash (example here https://stackoverflow.com/a/19882917/7569335).
That works for simple strings, but it breaks down once use start using interpolation. Not with regular variables, just when you interpolate an entire method.
as variant - use just stripMargin() and only once on a final string
String method2(String tier, String jobName) {
return """\
|Map downstreamJobs = [:]
|stage ("${jobName}-${tier}-\${region}_${jobName}") {
| test
|}
"""
}
static String simpleLog() {
return """\
|script {
| def user = env.BUILD_USER_ID
|}
"""
}
static String method1() {
return """\
|import jenkins.model.Jenkins
|currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
return """\
|Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
result:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
another variant with trim() and stripIndent()
def method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".trim()
}
def simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".trim()
}
def method1() {
return """
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".trim()
}
def generateFullDeploymentPipelineCode() {
return """\
Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent()
}
println(generateFullDeploymentPipelineCode())
When you insert a string through interpolation you only indent the first line of it. The following lines of the inserted string will be indented differently, which messes everything up.
Using some lesser-known members of GString (namely .strings[] and .values[]), we can align the indentation of all lines of each interpolated value.
String method2(String tier, String jobName) {
indented """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
"""
}
String simpleLog() {
indented """\
script {
def user = env.BUILD_USER_ID
}
"""
}
String method1() {
indented """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
indented """\
Text here
${method1()}
${method2("test1", "test2")}
"""
}
println generateFullDeploymentPipelineCode()
//---------- Move the following code into its own script ----------
// Function to adjust the indentation of interpolated values so that all lines
// of a value match the indentation of the first line.
// Finally stripIndent() will be called before returning the string.
String indented( GString templ ) {
// Iterate over the interpolated values of the GString template.
templ.values.eachWithIndex{ value, i ->
// Get the string preceding the current value. Always defined, even
// when the value is at the beginning of the template.
def beforeValue = templ.strings[ i ]
// RegEx to match any indent substring before the value.
// Special case for the first string, which doesn't necessarily contain '\n'.
def regexIndent = i == 0
? /(?:^|\n)([ \t]+)$/
: /\n([ \t]+)$/
def matchIndent = ( beforeValue =~ regexIndent )
if( matchIndent ) {
def indent = matchIndent[ 0 ][ 1 ]
def lines = value.readLines()
def linesNew = [ lines.head() ] // The 1st line is already indented.
// Insert the indentation from the 1st line into all subsequent lines.
linesNew += lines.tail().collect{ indent + it }
// Finally replace the value with the reformatted lines.
templ.values[ i ] = linesNew.join('\n')
}
}
return templ.stripIndent()
}
// Fallback in case the input string is not a GString (when it doesn't contain expressions)
String indented( String templ ) {
return templ.stripIndent()
}
Live Demo at codingground
Output:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Conclusion:
Using the indented function, a clean Groovy syntax for generating code from GString templates has been achieved.
This was quite a learning experience. I first tried to do it completely different using the evaluate function, which turned out to be too complicated and not so flexible. Then I randomly browsed through some posts from mrhaki blog (always a good read!) until I discovered "Groovy Goodness: Get to Know More About a GString". This was the key to implementing this solution.
I’m trying to push stage data from Jenkins pipeline to influx db and I get issue as below
Issue: In jenkin builds console output, after each stage, I see : java.io.NotSerializableException: sun.net.www.protocol.http.HttpURLConnection
Jenkins pipeline 2.150.2
InfluxDB 1.6.2
Any suggestions would be helpful. I'm new to this topic.
Note: I have commented #NonCPS annotation. If I uncomment then it sends only first stage data and exits and fails to iterate for loop which has 20 stages data.
//Maps for Field type columns
myDataField1 = [:]
myDataField2 = [:]
myDataField3 = [:]
//Maps for Custom Field measurements
myCustomDataFields1 = [:]
myCustomDataFields2 = [:]
myCustomDataFields3 = [:]
//Maps for Tag type columns
myDataTag1 = [:]
myDataTag2 = [:]
myDataTag3 = [:]
//Maps for Custom Tag measurements
myCustomDataTags1 = [:]
myCustomDataTags2 = [:]
myCustomDataTags3 = [:]
//#NonCPS
def pushStageData() {
def url_string = "${JENKINS_URL}job/ENO_ENG_TP/job/R421/13/wfapi/describe"
def get = new URL(url_string).openConnection();
get.addRequestProperty ("User-Agent","Mozilla/4.0");
get.addRequestProperty("Authorization", "Basic dZXZvceDIwMTk=");
//fetching the contents of the endpoint URL
def jsonText = get.getInputStream().getText();
//converting the text into JSON object using
JsonSlurperClassic
def jsonObject = new JsonSlurperClassic().parseText(jsonText)
// Extracting the details of all the stages present in that particular build number
for (int i=0; i<jsonObject.stages.size()-1; i++){ //size-1 to ignore the post stage
//populating the field type columns of InfluxDB measurements and pushing them to the map called myDataField1
def size = jsonObject.stages.size()-1
myDataField1['result'] = jsonObject.stages[i].status
myDataField1['duration'] =
jsonObject.stages[i].durationMillis
myDataField1['stage_name'] = jsonObject.stages[i].name
//populating the tag type columns of InfluxDB
measurements and pushing them to the map called
myDataTag1
myDataTag1['result_tag'] = jsonObject.stages[i].status
myDataTag1['stage_name_tag'] = jsonObject.stages[i].name
//assigning field type columns to the measurement called CustomData
myCustomDataFields1['CustomData'] = myDataField1
//assigning tag type columns to the measurement called CustomData
myCustomDataTags1['CustomData'] = myDataTag1
//Push the data into influx instance
try
{ step([$class: 'InfluxDbPublisher', target: 'jenkins_data', customPrefix: null, customDataMapTags: myCustomDataTags1]) }
catch (err)
{ println ("pushStagData exception: " + err) }
}
}
Expected: I want to push each stages of jenkins pipeline data to influx db.
Try to use jenkins' standard methods instead of creating objects manually. You can use httpRequest instead of URL. This should fix exception you are getting. You can also use readJson instead of JsonSlurperClassic (latter is optional since classic slurper is actually serializable).
you just need to explicitly set get to null to make sure there's none non serializable object being instantiated.
def jsonText = get.getInputStream().getText();
get = null;
I need to parse a text (output from a svn command) in order to retrieve a number (svn revision).
This is my code. Note that I need to retrieve all the output stream as a text to do other operations.
def proc = cmdLine.execute() // Call *execute* on the strin
proc.waitFor() // Wait for the command to finish
def output = proc.in.text
//other stuff happening here
output.eachLine {
line ->
def revisionPrefix = "Last Changed Rev: "
if (line.startsWith(revisionPrefix)) res = new Integer(line.substring(revisionPrefix.length()).trim())
}
This code is working fine, but since I'm still a novice in Groovy, I'm wondering if there were a better idiomatic way to avoid the ugly if...
Example of svn output (but of course the problem is more general)
Path: .
Working Copy Root Path: /svn
URL: svn+ssh://svn.company.com/opt/svnserve/repos/project/trunk
Repository Root: svn+ssh://svn.company.com/opt/svnserve/repos
Repository UUID: 516c549e-805d-4d3d-bafa-98aea39579ae
Revision: 25447
Node Kind: directory
Schedule: normal
Last Changed Author: ubi
Last Changed Rev: 25362
Last Changed Date: 2012-11-22 10:27:00 +0000 (Thu, 22 Nov 2012)
I've got inspiration from the answer below and I solved using find(). My solution is:
def revisionPrefix = "Last Changed Rev: "
def line = output.readLines().find { line -> line.startsWith(revisionPrefix) }
def res = new Integer(line?.substring(revisionPrefix.length())?.trim()?:"0")
3 lines, no if, very clean
One possible alternative is:
def output = cmdLine.execute().text
Integer res = output.readLines().findResult { line ->
(line =~ /^Last Changed Rev: (\d+)$/).with { m ->
if( m.matches() ) {
m[ 0 ][ 1 ] as Integer
}
}
}
Not sure it's better or not. I'm sure others will have different alternatives
Edit:
Also, beware of using proc.text. if your proc outputs a lot of stuff, then you could end up blocking when the inputstream gets full...
Here is a heavily commented alternative, using consumeProcessOutput:
// Run the command
String output = cmdLine.execute().with { proc ->
// Then, with a StringWriter
new StringWriter().with { sw ->
// Consume the output of the process
proc.consumeProcessOutput( sw, System.err )
// Make sure we worked
assert proc.waitFor() == 0
// Return the output (goes into `output` var)
sw.toString()
}
}
// Extract the version from by looking through all the lines
Integer version = output.readLines().findResult { line ->
// Pass the line through a regular expression
(line =~ /Last Changed Rev: (\d+)/).with { m ->
// And if it matches
if( m.matches() ) {
// Return the \d+ part as an Integer
m[ 0 ][ 1 ] as Integer
}
}
}