When I run the following code on the Jenkins workflow (Jenkins 1.609.1 ,workflow 1.8) I get error of 'NotSerializableException' (also below).
However, if I move the "build job" outside the "for" scope it works fine (the job is activated). Any ideas why this behavior?
node('master') {
ws('/opt/test) {
def file = "/ot.property"
def line = readFile (file)
def resultList = line.tokenize()
for(item in resultList )
{
build job: 'testjob_1'
}
}
}
Got error:
Running: End of Workflow
java.io.NotSerializableException: java.util.ArrayList$Itr
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
.....
I thnk it is because it's trying to serialize the unserializable item iterator on resultList as soon as it hits the build job step. See here for guidance on use of nonserializable variables:
https://github.com/jenkinsci/workflow-plugin/blob/master/TUTORIAL.md#serialization-of-local-variables
As a workaround to safely iterate using the workflow plugin, you need to us C-style loops. Try this instead:
for ( int i = 0; i < resultList.size; i++ ) {
etc...
According to CloudBees Platform Help page:
By design the pipeline can only keep records of Serializable objects. If you still need to keep an intermediate variable with a non serializable object, you need to hide it into a method and annotate this method with #NonCPS.
So you should transform your code into a function with #NonCPS helper method.
Related Jenkins bug: JENKINS-27421.
Related
I'm currently trying to automate the versioning of my maven project with a Jenkins (version 2.190.3) job and following the SemVer. So I have my Jenkinsfile and I'm doing something like that:
script {
def version = "1.2.4"
def pattern = ~/(\d{1,3})\.(\d{1,3})\.\d{1,4}$/
def newVersion = version.replaceFirst(pattern) { _,major,minor -> "${major}.${(minor as int) + 1}.0"}
}
The expectation is to have 1.3.0 in newVersion.
The code seems OK, working on web console but when I'm using Jenkins I have the following error:
java.lang.NullPointerException: Cannot execute null+1
Am I doing something wrong ?
Almost the exact answer to your question is given in this post: Jenkins groovy regex match string : Error: java.io.NotSerializableException: java.util.regex.Matcher
What it comes down to is that the script executed by Jenkins is kind of groovy, but not exactly executed as it is. It is transformed first to a serializable state (can be suspended, saved to file, transported, restored, resumed).
This doesn't work with certain objects that have state, but aren't serializable, including java.util.regex.Matcher, which is working under the hood of your regular expression. If you put this code in a method marked #NonCPS, the code is not transformed, and (more of less) executed as plain groovy.
#NonCPS
def foo() {
def version = "1.2.4"
def pattern = ~/(\d{1,3})\.(\d{1,3})\.\d{1,4}$/
def newVersion = version.replaceFirst(pattern) { _,major,minor -> "${major}.${(minor as int) + 1}.0"}
println "Version ${version} -> new ${newVersion}"
}
script {
foo()
}
Jenkins allows us to grab a Singelton of the running instance with jenkins.get(). I'm trying to build a class in my shared-library that CRUD's the cloud providers. My code looks like this.
#NonCPS
def create(){
Jenkins jenkins = Jenkins.getInstance()
// logic to create cloud
jenkins.clouds.add(tmpCloud)
jenkins.save()
}
#NonCPS
def delete(){
Jenkins jenkins = Jenkins.getInstance()
def newlist = jenkins.clouds.findAll{ it.getDisplayName() != cloud }
if(newlist){
jenkins.clouds.clear()
}
for ( int i = 0; i < newlist.size; i++ ) {
jenkins.clouds.add(newlist[i])
}
jenkins.save()
}
If I run just the create() function the code works as expected, same if I run just the delete(), but if I run both in the same job like this.
cloud.create()
cloud.delete()
Then only the create will work, the delete wont error but it wont do anything either. Is it possible to reinitialize a singleton? I feel like that is what I need to do. I have tried saving the jenkins instance as a field of my class, but that leads to a hudson serializable error. I dont see a way to pass the jenkins singleton around because of CPS issues.
I'm working on a jenkinsfile and I'm getting and exception in the third stage:
an exception which occurred:
in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv#7bbae4fb
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.CaseEnv#6896a2e3
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv#605ccbbc
in field com.cloudbees.groovy.cps.impl.CallEnv.caller
in object com.cloudbees.groovy.cps.impl.FunctionCallEnv#7b8ef914
in field com.cloudbees.groovy.cps.Continuable.e
in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable#11e73f3c
in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
in object org.jenkinsci.plugins.workflow.cps.CpsThread#b2df9bb
in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup#2b30596a
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup#2b30596a
Caused: java.io.NotSerializableException: java.util.regex.Matcher
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
I've been reading about it and I know I can't create non-serializable variables. So, I think it has to be with this part of my code:
def artifact_name = sh (
script: "ls -b *.jar | head -1",
returnStdout: true
).trim()
def has_snapshot = artifact_name =~ /-TEST\.jar/
if (has_snapshot) {
//Do something
}
My question is, how do I define that two variables in order to avoid that exception?
Your problem is this line:
def has_snapshot = artifact_name =~ /-TEST\.jar/
The =~ is the Groovy find operator. It returns a java.util.regex.Matcher instance, which is not Serializable. If Jenkins decides to pause your script after you have stored the result in a local variable that is serialized by Jenkins that is when you get the exception. This can be easily tested by immediately adding a sleep(1) step after your invocation and watch as that same exception is thrown.
To resolve this, you should :
Not store the java.util.regex.Matcher result in CPS transformed code
Move the usage into a #NonCPS annotated method or use the match operator (==~) which returns a boolean (if it fits your use case)
Building on the accepted answer I came up with this solution:
def hello = {
def matcher = ("Hello" =~ /Hello/)
matcher.find()
return matcher.group()
}.call()
I guess the stability of this is not so good, but I assume the likeliness of this failing to be very low. So if the impact of this code failing is also low it might be reasonable risk management to use this code.
The following seems to fit the case of running in a NonCPS context, but I am not 100% sure. It definitely is working though
#NonCPS def hello = {
def matcher = ("Hello" =~ /Hello/)
matcher.find()
return matcher.group()
}
hello = hello.call()
println hello
The accepted answer is certainly correct. In my case, I was trying to parse some JSON from an API response like so:
#NonCPS
def parseJson(rawJson) {
return new groovy.json.JsonSlurper().parseText(rawJson)
}
All this does is return a JsonSlurper that can then be used to walk down your JSON structure, like so:
def jsonOutput = parseJson(createIssueResponse)
echo "Jira Ticket Created. Key: ${jsonOutput.key}"
This snippet actually worked fine in my script, but later on in the script, it was using the jsonOutput.key to make a new web request. As stated in the other answer, if the script pauses when you have something stored into a local variable that cannot be serialized, you will get this exception.
When the script attempted to make the web request, it would pause (presumably because it was waiting for the request to respond), and the exception would get thrown.
In my scenario, I was able to fix this by doing this instead:
def ticketKey = parseJson(createIssueResponse).key.toString()
echo "Jira Ticket Created. Key: ${ticketKey}"
And later on when the script attempts to send the web request, it no longer throws the exception. Now that no JsonSlurper object is present in my running script when it is paused, it works fine. I previously assumed that because the method was annotated with #NonCPS that its returned object was safe to use, but that is not true.
I have a jenkins build that needs to get the filenames for all files checked in within a changeset.
I have installed groovy on the slave computer and configured Jenkins to use it. I am running the below script that should return the names (or so I assume as this may be wrong as well) and print to the console screen however I am getting this error:
groovy.lang.MissingPropertyException: No such property: paths for class: hudson.plugins.tfs.model.ChangeSet
Here is the Groovy System Script:
import hudson.plugins.tfs.model.ChangeSet
// work with current build
def build = Thread.currentThread()?.executable
// get ChangesSets with all changed items
def changeSet= build.getChangeSet()
def items = changeSet.getItems()
def affectedFiles = items.collect { it.paths }
// get file names
def fileNames = affectedFiles.flatten().findResults
fileNames.each {
println "Item: $it" // `it` is an implicit parameter corresponding to the current element
}
I am very new to Groovy and Jenkins so if its syntax issue or if I'm missing a step please let me know.
I don't know the version of jenkins you are using but according to the sourcecode of ChangeSet that you can find here I suggest you to replace line 9 with:
def affectedFiles = items.collect { it.getAffectedPaths() }
// or with the equivalent more groovy-idiomatic version
def affectedFiles = items.collect { it.affectedPaths }
Feel free to comment the answer if there will be more issues.
When using the Jenkins pipeline plugin, the build fails with a java.io.NotSerializableException error, like below:
java.io.NotSerializableException: org.codehaus.groovy.control.ErrorCollector
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
...
Caused by: an exception which occurred:
in field collector
in field abnormal
in field outcome
in field body
in field step
in field thread
in field this$0
in field returnAddress
in field parent
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup#6ae7e4f1
...
Finished: FAILURE
This happens when I use a custom library with some import statements. I've tried several things, like encapsulating the call in a method with #NonCPS, but the error remains.
Pipeline script
#!groovy
#Library('utils')
pipeline {
agent any
stages {
stage('Run Script') {
script {
myScript param1
}
}
}
}
vars/myScript.groovy
import com.company.jenkins.utils
def call(String param = "test") {
def libScript = LibScript(this)
libScript.printMessage("Hello World")
}
src/com/company/jenkins/utils/LibScript
package com.company.jenkins.utils;
// This import works fine
import groovy.json.*
// This one fails
import groovyx.net.http.RESTClient
class LibScript implements Serializable {
def steps
def client
LibScript(steps) { this.steps = steps }
def printMessage(String message) { steps.echo "Saying: " message }
// This also fails
#NonCPS
def doSomething() { client = new groovyx.net.http.RESTClient( 'https://somehost/' ) }
}
Versions used:
Jenkins: 2.19.3
Pipeline plugin: 2.5
Pipeline Shared Groovy Libraries Plugin: 2.7
This error will come because of the compilation error. This is not a good stacktrace to be get. But you are having a non serializable object as one of your variable or in the middle of the string as well ( can have variable value as '.."$var"..' ). For this, I am attaching two links. Please go through them and you will understand what has gone wrong. Please run step by step to understand where went wrong.
1) https://issues.jenkins-ci.org/browse/JENKINS-40109
2) https://coderwall.com/p/zvsh5q/jenkins-load-command-can-t-found-why-i-have-a-java-io-notserializableexception
As the accepted answer states, this (non-descriptive) error indicates some sort of compilation error. The issue then is tracking down what the compilation error is, since the error message tells you basically nothing.
It appears that getting this non-descriptive compilation error message is only an issue when using declarative pipelines. If you were to switch to a scripted pipeline, you would get a more specific error message.
However, you can get more descriptive errors out of declarative pipelines if you wrap the code in question with a try catch block. As an example from some of my code, I narrowed down the NotSerializableException from this block of pipeline code:
post {
always {
notifyOnBuildResults currentBuild : currentBuild
}
}
I changed the code to this
post {
always {
script {
try {
notifyOnBuildResults currentBuild : currentBuild
} catch (Exception e) {
echo e.toString()
}
}
}
}
and was able to see the specific compilation error.