Jenkins node disappear after adding new node
I add the new node through Jenkins GUI and I have about 118 active nodes. After the addition the new node appears but one of the already existing nodes disappears and any job running on the disappeared node starts to throw the following java exception
Execution of script [machine_clean.groovy] failed - java.lang.NullPointerException: Cannot
invoke method getChannel() on null object
org.jenkinsci.plugins.scriptler.util.GroovyScript$ScriptlerExecutionException: java.lang.NullPointerException: Cannot invoke method getChannel() on null object
at org.jenkinsci.plugins.scriptler.util.GroovyScript.call(GroovyScript.java:132)
at hudson.remoting.LocalChannel.call(LocalChannel.java:47)
at org.jenkinsci.plugins.scriptler.builder.ScriptlerBuilder.perform(ScriptlerBuilder.java:242)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:522)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: java.lang.NullPointerException: Cannot invoke method getChannel() on null object
at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:47)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:34)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at hudson.slaves.SlaveComputer$getChannel.call(Unknown Source)
at Script1.run(Script1.groovy:54)
at org.jenkinsci.plugins.scriptler.util.GroovyScript.call(GroovyScript.java:123)
Is there a limit on the number of supported node/executors? any idea how to fix this?
Related
I have two Jenkins jobs, one called "UpStreamJob" and the other one is its downstream job called "GetUpstreamJob".
Upstream job is configured to kick-off downstream job as its post build action.
Downstream job is configured with "execute groovy script" option which points to a groovy script
whose main purpose is to get the upstream job name that triggered it and then write the job name to a property file(which I am yet to implement).
Groovy script :
#GrabResolver(name='jenkins', root='http://repo.jenkins-ci.org/public/')
#Grapes([
#Grab(group='org.jenkins-ci.main', module='jenkins-core', version='2.167')
])
import hudson.model.*
def writeToPropertyFile(){
def causes = Run.getCauses()
for (cause in causes)
{
printCausesRecursively(cause)
}
}
def printCausesRecursively(cause) {
if (cause.class.toString().contains("UpstreamCause")) {
println "This job was caused by " + cause.toString()
for (upCause in cause.upstreamCauses) {
printCausesRecursively(upCause)
}
} else {
println "Root cause : " + cause.toString()
}
}
writeToPropertyFile()
The error I get in Jenkins console log is :
Started by upstream project "UpStreamJob" build number 11
originally caused by:
Started by user xyz
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on Server123 in workspace D:\Jenkins\workspace\GetUpstreamJob
[GetUpstreamJob] $ D:\Jenkins\tools\hudson.plugins.groovy.GroovyInstallation\Groovy-2-5-7\bin\groovy.bat D:\Asif\getUpstreamJob.groovy
Caught: java.lang.NoClassDefFoundError: org/xmlpull/v1/XmlPullParserException
java.lang.NoClassDefFoundError: org/xmlpull/v1/XmlPullParserException
at hudson.util.XStream2.getDefaultDriver(XStream2.java:109)
at hudson.util.XStream2.<init>(XStream2.java:113)
at hudson.model.Run.<clinit>(Run.java:2471)
at getUpstreamJob.writeToPropertyFile(getUpstreamJob.groovy:10)
at getUpstreamJob.run(getUpstreamJob.groovy:32)
Caused by: java.lang.ClassNotFoundException: org.xmlpull.v1.XmlPullParserException
... 5 more
Build step 'Execute Groovy script' marked build as failure
Notifying upstream projects of job completion
Finished: FAILURE
EDIT 1 : START
Below is the error that the Jenkins console log shows, when I try running the script by copy-pasting the script's code in the "Execute System Groovy script":
Started by upstream project "UpStreamJob" build number 15
originally caused by:
Started by user Malick, Asif
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on SERVERNAMEXYZ123 in workspace D:\Jenkins\workspace\GetUpstreamJob
ERROR: Build step failed with exception
groovy.lang.MissingMethodException: No signature of method: java.lang.Class.getCauses() is applicable for argument types: () values: []
Possible solutions: getClasses(), getClass(), getName(), metaClass(groovy.lang.Closure), metaClass(groovy.lang.Closure), getFields()
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:153)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
at org.kohsuke.groovy.sandbox.impl.Checker$checkedCall.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at Script1.writeToPropertyFile(Script1.groovy:10)
at Script1$writeToPropertyFile.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
at org.kohsuke.groovy.sandbox.impl.Checker$checkedCall.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at Script1.run(Script1.groovy:32)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runScript(GroovySandbox.java:162)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SecureGroovyScript.evaluate(SecureGroovyScript.java:368)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SecureGroovyScript.evaluate(SecureGroovyScript.java:312)
at hudson.plugins.groovy.SystemGroovy.run(SystemGroovy.java:95)
at hudson.plugins.groovy.SystemGroovy.perform(SystemGroovy.java:59)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1853)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
Build step 'Execute system Groovy script' marked build as failure
Notifying upstream projects of job completion
Finished: FAILURE
EDIT 2 : END
Please let me know in case I missed out on any details. Part of the logic is taken from the internet.
I have a multijob which has 6 child job. During test, I found that the multijob always show failed even all sub jobs succeed. The error message is:
Finished Build : 1.0.0.1 of Job : Sub_Job_6 with status : SUCCESS
FATAL: Channel "unknown": Remote call on JNLP4-connect connection from build_machine/10.224.113.202:51903 failed. The channel is closing down or has closed down
java.nio.channels.ClosedChannelException
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:179)
at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on JNLP4-connect connection from build_machine/10.224.113.202:51903 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:945)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at com.tikal.jenkins.plugins.multijob.MultiJobBuild$MultiJobRunnerImpl.run(MultiJobBuild.java:148)
at hudson.model.Run.execute(Run.java:1727)
at com.tikal.jenkins.plugins.multijob.MultiJobBuild.run(MultiJobBuild.java:76)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
I found the root cause. One of my sub job is to power off, power on the build machine( a reboot actually). After reboot, Jenkins client will re-connect to Jenkins server. The server may think it's a new conversation, the old one has stopped with some exception. So the final result will show failed.
After I removed this sub job, the multijob showed success finally.
I got a failure after all the build procedure were finished without any errors.
And I don't know why below error has occurred.
This error has marked the status of a job as a failure even all the tests were successfully finished.
FATAL: Remote call on JNLP4-connect connection from socautovm1.xxx.local/192.168.114.144:65033 failed
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from socautovm1.xxx.local/192.168.114.144:65033
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
java.lang.NoClassDefFoundError: Could not initialize class hudson.slaves.SlaveComputer
at hudson.util.ProcessTree.get(ProcessTree.java:399)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Unknown Source)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from socautovm1.xxx.local/192.168.114.144:65033 failed
at hudson.remoting.Channel.call(Channel.java:961)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
It seems that the agent is down. Check in which node the job runs and check its status. You may need to restart the node or even free some space in order to get it back online if the current status is offline.
I restarted the jenkins agent service and it resolved the issue. (/service/jenkins-agent)
I was having a similar issue when trying to run a Job on Jenkins that fetches the list of containers on a Slave node.
Whenever I run the job I get the error below:
ERROR: Issue with creating launcher for agent Linux1(Docker). The agent is being disconnected
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: hudson.remoting.ChannelClosedException: Channel "unknown": Remote call on Linux1(Docker) failed. The channel is closing down or has closed down
Here's how I fixed it:
I simply restarted the Jenkins service on the main/master node.
For Windows servers:
Search for Services and then locate the Jenkins service and restart it.
For Linux servers:
You might need to use Systemd (sudo systemctl restart jenkins)
That's all.
I hope this helps
I have this error:
java.io.IOException: Remote call on build4 failed
at hudson.remoting.Channel.call(Channel.java:789)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:953)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:540)
at hudson.model.Run.execute(Run.java:1741)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.util.jna.GNUCLibrary
at hudson.plugins.signal_killer.SignalKiller.sendSignal(SignalKiller.java:45)
at hudson.plugins.signal_killer.SignalKiller.kill(SignalKiller.java:29)
at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:216)
at hudson.util.ProcessTree$Windows$1.killRecursively(ProcessTree.java:427)
at hudson.util.ProcessTree$Windows.killAll(ProcessTree.java:487)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:965)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:956)
at hudson.remoting.UserRequest.perform(UserRequest.java:120)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at hudson.remoting.Engine$1$1.run(Engine.java:62)
at java.lang.Thread.run(Unknown Source)
at ......remote call to build4(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:252)
at hudson.remoting.Channel.call(Channel.java:781)
... 6 more
after my build. This error marked build as FAILED.
But I need to forcibly mark it as SUCCESS, because I got successful result of build - *.exe link of my distrib.
I've tried to use Jenkins Text Finder plugin:
Settings of JTF
It was looking for my regex in console output and mark build successful but it didn't work as I expected - it found regex, but didn't mark build successful.
Is there another way to solve this problem?
This isn't possible — the build result can only be marked as worse than the existing result, and therefore the Text Finder plugin can only set your build as UNSTABLE or FAILED.
It would appear that you need to wait for a resolution to JENKINS-20481. You should add as much detail to that ticket as you can to help.
I have 2 jenkins servers because my 2 builds have some incompatible system requirements.
I setup a new node for one of the servers and migrated the jobs from the other server and set them up to run on the node.
The node runs the job just fine and even archives the artifacts (they are linked from the job) but the job throws and exception and gets marked as a failure.
** Below is the output from the jobs **
Completed build, now archiving <-- I print this out at the end of my last build step
FATAL: Remote call on ops-1-jenkins-android-10-186.fam.io failed
java.io.IOException: Remote call on ops-1-jenkins-android-10-186.fam.io failed
at hudson.remoting.Channel.call(Channel.java:748)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:940)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:556)
at hudson.model.Run.execute(Run.java:1745)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:89)
at hudson.model.Executor.run(Executor.java:240)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.slaves.SlaveComputer
at hudson.util.ProcessTree.getKillers(ProcessTree.java:151)
at hudson.util.ProcessTree$OSProcess.killByKiller(ProcessTree.java:212)
at hudson.util.ProcessTree$UnixProcess.kill(ProcessTree.java:557)
at hudson.util.ProcessTree$UnixProcess.killRecursively(ProcessTree.java:564)
at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:488)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:952)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:943)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:328)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:701)
Thanks Lovato!
After I updated all the plugins and jenkins things worked just fine.
We got this error: FATAL: java.io.IOException: Remote call on pm_jenprodslave_1 failed. We found the slave server was unresponsive and had to reboot it. Once it came back the build works again.