Jenkins build faild on slave node with `java.io.EOFException` - jenkins

The stacktrace is below:
Evacuated stdout
Starting Selenium nodes on ci2
March 18, 2019 11:04:00 AM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARN: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.Git$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
March 18, 2019 11:04:03 AM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARN: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Slave JVM has not reported exit code. Is it still running?
[03/18/19 11:04:06] Launch failed - cleaning up connection
ERROR: Connection terminated
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:140)
at hudson.remoting.Command.readFrom(Command.java:126)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:36)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
[03/18/19 11:04:06] [SSH] Connection closed。
I use JDK8 and jenkins 2.164.1 on Ubuntu 16.04.6
How to fix this?

This issue has been bugging us as well. There seems to be a temporary work around, not a solution. Just set the remote root directory of the slave node configuration on jenkins server to a new path, and remoting will take care of the rest.. but, the issue seem to reoccur now and then. We don't know the root cause yet. Any word is more than welcome.

Related

Jenkins master can't connect to Subversion, but slave can

I am using Subversion plugin in Jenkins. My master machine (Windows) is unable to connect to SVN server (using Apache Subversion v1.7) via svn+ssh throwing exception
Running as SYSTEM
Building on master in workspace c:\Program Files (x86)\Jenkins\w4ManagerWorkspace
Updating svn+ssh://10.74.1.41/data/subversion/svn/tools/testComplete/trunk/W4Manager at revision '2022-12-01T10:31:08.582 +0100' --quiet
Using sole credentials arpc_ta/****** in realm ‘svn+ssh://10.74.1.41’
ERROR: Failed to update svn+ssh://10.74.1.41/data/subversion/svn/tools/testComplete/trunk/W4Manager
org.tmatesoft.svn.core.SVNException: svn: E210002: There was a problem while connecting to 10.74.1.41:22
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:145)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:79)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1282)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.testConnection(SVNRepositoryImpl.java:100)
at org.tmatesoft.svn.core.io.SVNRepository.getRepositoryUUID(SVNRepository.java:268)
at org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.createRepository(SvnRepositoryAccess.java:103)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepository(SvnNgRepositoryAccess.java:211)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:210)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:159)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1064)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1040)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1013)
at hudson.FilePath.act(FilePath.java:1172)
at hudson.FilePath.act(FilePath.java:1155)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:960)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:883)
at hudson.scm.SCM.checkout(SCM.java:505)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1213)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:637)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:509)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:431)
Caused by: java.io.IOException: There was a problem while connecting to 10.74.1.41:22
at com.trilead.ssh2.Connection.connect(Connection.java:781)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openConnection(SshHost.java:225)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openSession(SshHost.java:153)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshSessionPool.openSession(SshSessionPool.java:85)
at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:122)
... 35 more
Caused by: java.io.IOException: **Key exchange was not finished, connection is closed.**
at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95)
at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237)
at com.trilead.ssh2.Connection.connect(Connection.java:732)
... 39 more
Caused by: java.io.IOException: **Fatal error during MAC startup!**
at com.trilead.ssh2.transport.KexManager.finishKex(KexManager.java:298)
at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:571)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
at java.lang.Thread.run(Unknown Source)
ERROR: Subversion update failed
java.io.IOException: Fatal error during MAC startup!
at com.trilead.ssh2.transport.KexManager.finishKex(KexManager.java:298)
at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:571)
at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
at java.lang.Thread.run(Unknown Source)
Caused: java.io.IOException: Key exchange was not finished, connection is closed.
at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95)
at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237)
at com.trilead.ssh2.Connection.connect(Connection.java:732)
Caused: java.io.IOException: There was a problem while connecting to 10.74.1.41:22
at com.trilead.ssh2.Connection.connect(Connection.java:781)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openConnection(SshHost.java:225)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshHost.openSession(SshHost.java:153)
at org.tmatesoft.svn.core.internal.io.svn.ssh.SshSessionPool.openSession(SshSessionPool.java:85)
at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:122)
Caused: org.tmatesoft.svn.core.SVNException: svn: E210002: There was a problem while connecting to 10.74.1.41:22
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at org.tmatesoft.svn.core.internal.io.svn.SVNSSHConnector.open(SVNSSHConnector.java:145)
at org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:79)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1282)
at org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.testConnection(SVNRepositoryImpl.java:100)
at org.tmatesoft.svn.core.io.SVNRepository.getRepositoryUUID(SVNRepository.java:268)
at org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.createRepository(SvnRepositoryAccess.java:103)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepository(SvnNgRepositoryAccess.java:211)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:210)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:159)
Caused: hudson.scm.subversion.UpdaterException: failed to perform svn update
Caused: java.io.IOException
at hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:214)
at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1064)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1040)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1013)
at hudson.FilePath.act(FilePath.java:1172)
at hudson.FilePath.act(FilePath.java:1155)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:960)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:883)
at hudson.scm.SCM.checkout(SCM.java:505)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1213)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:637)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:509)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:431)
[Checks API] No suitable checks publisher found.
Finished: FAILURE
The same Jenkins project run from slave machine is successful. On my slave machine Jenkins is running as windows service.
Running as SYSTEM
Building remotely on slaveMachine (slave_1) in workspace c:\Program Files (x86)\Jenkins\w4ManagerWorkspace
Updating svn+ssh://10.74.1.41/data/subversion/svn/tools/testComplete/trunk/W4Manager at revision '2022-12-01T12:30:51.663 +0100' --quiet
Using sole credentials arpc_ta/****** in realm ‘svn+ssh://10.74.1.41’
Using sole credentials arpc_ta/****** in realm ‘svn+ssh://10.74.1.41’
At revision 7401
In projects configuration in Jenkins is shown, that something is not in order with message "Unable to access the repository"
However, when I run the same project on my slave machine, the connection is successful, and Jenkins does not show in console any error. svn update command finishes successfully. Both my master and slave machines are Windows server VMs with. Connection from master to SVN from TortoiseSVN is successful, and I am able to checkout the repository. I used SVN few years without problems, Subversion plugin was just installed to Jenkins.
Found similar issues e.g. Jenkins Master-Slave: Key exchange was not finished, connection is closed or https://support.tmatesoft.com/t/svn-e210002-svnkit-doesnt-connect-to-remote-repository/2480/13, with solution to edit config file /etc/ssh/sshd_config on svn server, but I don't think that can be the issue, since from slave machine the connection is successful. I checked also Java version, they are the same on both windows servers.
Tried also different credential types, public key authentication and user/pw from scm console of Jenkins [jenkinsUrl]/scm/SubversionSCM/enterCredential but it throws the same error, just from the line Fatal error during MAC startup!
edit: On SVN server in /var/log/secure there is Connection closed message from my master machine (10.74.37.1), no other error or details. From slave machine (10.74.37.28) the connection is accepted
Dec 5 14:47:11 xlcppt28 sshd[24808]: Connection closed by 10.74.37.1 port 57664 [preauth]
Dec 5 14:47:13 xlcppt28 sshd[24811]: Accepted password for arpc_ta from 10.74.37.28 port 57941 ssh2
Dec 5 14:47:13 xlcppt28 sshd[24811]: pam_unix(sshd:session): session opened for user arpc_ta by (uid=0)
Problem was resolved with updating Jenkins and then every plugin in Jenkins (as my Jenkins server is offline, that wasn't 2 clicks..) I also updated Java.

Cannot get Jenkins to start and run properly

I am following the "Getting Started" guide from the Jenkins documentation; however, I have not been able to get Jenkins to run properly. When I run the command java -jar jenkins.war --httpPort=8080 I get a series of warnings/errors and Jenkins will not run as expected. I am not able to install any plugins or complete the very next step of the getting started guide.
The first warning I receive is for deleting a temp file. It appears to fail because there is no file at that location.
Jan 27, 2018 11:58:13 PM Main deleteWinstoneTempContents
WARNING: Failed to delete the temporary Winstone file
/var/folders/mg/r5m253mx4hz75sk1f_gn5xg80000gn/T/winstone/jenkins.war
The next warning I get is a Jetty warning. I have no idea what the issue is here, as I am not familiar with Jetty.
Jan 27, 2018 11:58:16 PM
org.eclipse.jetty.server.handler.ContextHandler setContextPath
WARNING: Empty contextPath
The next warning is,
Jan 27, 2018 11:58:34 PM jenkins.slaves.DeprecatedAgentProtocolMonitor
initializerCheck
WARNING: This Jenkins instance uses deprecated Remoting protocols: CLI-
connect,CLI2-connect,JNLP-connect,JNLP2-connectIt may impact stability
of the instance. If newer protocol versions are supported by all system
components (agents, CLI and other clients), it is highly recommended to
disable the deprecated protocols.
And finally, I get a lengthy stacktrace regarding some type of Hudson or certification issue.
Jan 27, 2018 11:58:37 PM hudson.model.UpdateCenter updateDefaultSite
WARNING: Upgrading Jenkins. Failed to update the default Update Site
'default'. Plugin upgrades may fail.
javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at hudson.model.DownloadService.loadJSON(DownloadService.java:167)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:190)
at hudson.model.UpdateCenter.updateDefaultSite(UpdateCenter.java:2194)
at jenkins.install.SetupWizard.init(SetupWizard.java:184)
at jenkins.install.InstallState$3.initializeState(InstallState.java:105)
at jenkins.model.Jenkins.setInstallState(Jenkins.java:1037)
at jenkins.install.InstallUtil.proceedToNextStateFrom(InstallUtil.java:98)
at jenkins.model.Jenkins.<init>(Jenkins.java:927)
at hudson.model.Hudson.<init>(Hudson.java:86)
at hudson.model.Hudson.<init>(Hudson.java:82)
at hudson.WebAppMain$3.run(WebAppMain.java:233)
Caused by: sun.security.validator.ValidatorException: PKIX path
building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to
find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
... 23 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:146)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
... 29 more
As I said, I am following their instructions word for word and they're very simple, but it just won't work. Any help that you can provide would be much appreciated. Thanks.
It can depend on your exact setup and OS: this was seen for instance in a docker setup, with jenkinsci/docker issue 422, although this Dockerfile works.
The issue includes:
I was facing the same problem and I fixed it by removing option '--preferedClassLoader' in environment variable JENKINS_ARGS.
That being said, your local Jenkins instance might want to connect to https site (for instance to check for plugins update).
If that is the case, you would need to import the certificate of the remote site into your JDK keystore.

Unable to connect to slave from master. "Invalid encoded sequence encountered:"

I have Box Master and Box Slave in AWS EC2 instances. I created jenkins user in Box Slave and I copied the master's public keys to slave. Now I created a new node in Jenkins Master. However, when I connect to slave using Launch agent via execution of command on master using command ssh -tt jenkins#10.15.0.10, it gives me the following error:
just before slave Services-Slave gets launched ...
executing pre-launch scripts ...
[06/26/17 16:25:28] Launching agent
$ ssh -tt jenkins#10.15.0.10
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-1020-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
5 packages can be updated.
0 updates are security updates.
Last login: Mon Jun 26 20:19:51 2017 from 10.15.0.5
<===[JENKINS REMOTING CAPACITY]===>To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
jenkins#ip-10-94-0-63:~$ <===[JENKINS REMOTING CAPACITY]===ERROR: Unable to launch the agent for Services-Slave
java.io.IOException: Invalid encoded sequence encountered: 08 08 08 08
at hudson.remoting.BinarySafeStream$1._read(BinarySafeStream.java:194)
at hudson.remoting.BinarySafeStream$1.read(BinarySafeStream.java:80)
at hudson.remoting.BinarySafeStream$1.read(BinarySafeStream.java:97)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at hudson.remoting.BinarySafeStream$1._read(BinarySafeStream.java:189)
at hudson.remoting.BinarySafeStream$1.read(BinarySafeStream.java:125)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at hudson.remoting.BinarySafeStream$1._read(BinarySafeStream.java:189)
at hudson.remoting.BinarySafeStream$1.read(BinarySafeStream.java:125)
at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2338)
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2351)
at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:3092)
at java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:2892)
at java.io.ObjectInputStream.readUTF(ObjectInputStream.java:1075)
at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:684)
at java.io.ObjectInputStream.readClassDescriptor(ObjectInputStream.java:833)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1609)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at hudson.remoting.Capability.read(Capability.java:140)
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:391)
at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:310)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:389)
at hudson.slaves.CommandLauncher.launch(CommandLauncher.java:132)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:262)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
If I run the same command from my username from master it is able to ssh successfully. Any idea why this is happening?
I tried giving the .pem file
I also did sudo -u jenkins. Nothing works.
Several things were going wrong here too, while configuring master-node communication. Somewhere it seemed like, master is caching the configurations for nodes. Sometimes removing and adding the node did work!
But in the end, this helped every-time.
https://docs.google.com/document/d/1Qq-EkiUnC5x8BuM4AZWo-yRUQTrkberzz8JfdCM6yuc/edit?pli=1

Jenkins slave cannot connect with master: Incorrect acknowledgement sequence

After update of our Jenkins master installation to its latest LTS version 2.46.3 one of its slaves (Windows 7 machine, 32-bit) cannot connect with the master.
The error we're getting is:
java -jar slave.jar -jnlpUrl https://<jenkins-name>/computer/<node-name>/slave-agent.jnlp -secret <secret-value>
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up slave: node-name
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://<jenkins-name>/]
Jun 22, 2017 1:19:05 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP3-connect, JNLP-connect, CLI2-connect, Ping, CLI-connect, JNLP4-connect, JNLP2-c
onnect]
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: <jenkins-name>
Agent port: <jenkins-port>
Identity: <id:en:ti:ty>
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to <jenkins-name>:9150
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Jun 22, 2017 1:19:05 PM org.jenkinsci.remoting.protocol.impl.AckFilterLayer abort
WARNING: [JNLP4-connect connection to <our-proxy>/10.253.0.11:81] Incorrect acknowledgement sequence, expected 0x0003414333 got 0x4854545044
Jun 22, 2017 1:19:05 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowled
gement sent
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:385)
at hudson.remoting.Engine.run(Engine.java:287)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecvClosed(AckFilterLayer.java:280)
at org.jenkinsci.remoting.protocol.FilterLayer.abort(FilterLayer.java:164)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.abort(AckFilterLayer.java:130)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecv(AckFilterLayer.java:258)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:669)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48)
We spent a lot of time trying to fix the problem. Unfortunately without success.
Do you have an idea what could have caused the problem and how can it be solved?
We recently hit this issue with our AWS-based Jenkins using JNLP for remote integration testing. The remote slave would call back to the Jenkins master, which failed with a similar error. The issue ended up being a dynamically generated AWS ELB of type HTTP (because the Kubernetes ELB provisioner presently doesn't support multi-protocol ELBs) for the Jenkins Master. We had to manually change the JNLP ingress port type of the ELB to TCP, while the web interface ingress 'instance port' was protocol HTTP and 'load balancer' was protocol HTTPS.
Is the Jenkins master instance running behind a load balancer? I had the same issue when my instance was running behind an Application Load Balancer in AWS.
If so, then the acknowledgement sequence could get modified because of differing protocols in the Load balancer. JNLP requires TCP connection on port 50000 by default.
If your setup is on AWS, you could try creating a private hosted zone in Route53 with an Alias record for your Jenkins instance's private IP address.
For e.g: jenkins.example.com -> your Jenkins instance's private IP
Then, in Jenkins UI -> Manage Jenkins -> Configure System -> Manage nodes and clouds -> Configure clouds -> (under advanced settings)
Tunnel connection through : jenkins.example.com:50000
This avoids your slave agents to have to go through the load balancer to connect to the Jenkins Master.
I encounter this kind of problem on gcp, jenkins master behind load balance, almost the same as Sidharth Ramesh reply.
in configuration -> manage jenkins -> configure global security, in the 'agent' part, you must config a specific port, never choose random. I choose 50222 as example,
below is agent protocols: there is a checkbox of "Inbound TCP Agent Protocol/4 (TLS encryption)", we must make enable. if not, there is an error message: "server reports protocol jnlp4-connect not supported skipping"
open the firewall of port from jenkins slave to jenkins master vm internal ip.
enjoy
You need to check the secret key of the node is intact. If not proper, you have to download the slave.jar and also Run agent from command line with new jar file.
java -jar slave.jar -jnlpUrl http://<ipaddress>:8080/computer/<computername>/slave-agent.jnlp -secret 340d54sdrgtjj334kelkahsdjkf83f1c5120dc2fb74939fcdb7f05e1926049f8d7991
Also to check the java version installed is > 7
This happened to us when a Windows Update or some other silent background update messed with the slave's environment variables. HTTPS_PROXY and HTTP_PROXY had to be re-added, and once that was done we were back in business.
The message:
Incorrect acknowledgement sequence ...
happened for me when I had incorrectly configured a value for the Java property hudson.TcpSlaveAgentListener.port as the same port number as the HTTP port used by Jenkins. The TcpSlaveAgentListener javadoc indicates that is a misconfiguration when it says:
Aside from the HTTP endpoint, Jenkins runs TcpSlaveAgentListener that listens on a TCP socket. Historically this was used for inbound connection from agents (hence the name), but over time it was extended and made generic, so that multiple protocols of different purposes can co-exist on the same socket. (emphasis added)
If the HTTP port was 8080 and the hudson.TcpSlaveAgentListener.port was also 8080, then my JNLP agents failed to connect. As soon as I assigned another value to hudson.TcpSlaveAgentListener.port (like 50000) and restarted Jenkins, my JNLP agents were able to connect.
The stack trace on the failing JNLP agent was:
INFO: Trying protocol: JNLP4-connect
Mar 02, 2019 3:49:29 PM org.jenkinsci.remoting.protocol.impl.AckFilterLayer abort
WARNING: [JNLP4-connect connection to agent.example.com/172.16.16.113:8080] Incorrect acknowledgement sequence, expected 0x000341434b got 0x485454502f
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:614)
at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecvClosed(AckFilterLayer.java:280)
at org.jenkinsci.remoting.protocol.FilterLayer.abort(FilterLayer.java:164)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.abort(AckFilterLayer.java:130)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecv(AckFilterLayer.java:258)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:668)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$2200(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:283)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Unknown Source)
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to testing-a.markwaite.net:8080
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP3-connect not supported, skipping
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Mar 02, 2019 3:49:29 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)
at hudson.remoting.Engine.innerRun(Engine.java:639)
at hudson.remoting.Engine.run(Engine.java:474)
I had this issue, and i found a solution.
I have a jenkins master deployed via the jenkins helm chart on an EKS,
and its exposed with an ingress controller,
which is behind an ALB.
I tried to connect an inbound agent node and i had this error above.
solution in short: just tick the "Use WebSocket" option in the agent node config
(Manage Jenkins ==> Manage nodes and clouds ==> choose your inbound agent node ==> Configure ==> tick the "Use WebSocket")
after i did it - the agent could connect and the error was gone.
this is the most elegant solution, i believe its also more secured because when you use it you dont need to keep the 50000 tcp port open, you can just keep using the main port of jenkins (443 usually i guess),
Note: you do need to make sure that the agent will have access to the jenkins main port (usually 443 or 80).
this is how i found this solution:
i found this:
https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-setup-guide/configure-ports-jnlp-agents
which led me to this:
https://github.com/jenkinsci/jep/blob/master/jep/222/README.adoc
they explain there that when you expose jenkins through a load balancer, then you better use the websocket option (and even if not, using websocket is still better, because the websocket is more secured then the jnlp and the extra tcp port)

Dataflow exception - "RuntimeException: java.io.IOException: Premature EOF"

Kicked off our pipeline (id: 2015-05-24_21_45_40-9907197960194601343), and it failed with the below error. Looks like a problem with reading SideInputs.
Did something change in the service?
May 25, 2015, 2:53:09 PM
(ac187ee1445808a5): java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Premature EOF at
com.google.cloud.dataflow.sdk.runners.worker.SideInputUtils$ReaderIterator.hasNext(SideInputUtils.java:127) at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43)
Update: Another job with the same problem: 2015-05-25_00_50_13-9599484476849123094
Update: Another job with the same problem: 2015-06-04_18_22_47-8603825216881045184
We've not seen this error since. Seems to have been fixed on the Dataflow service side.

Resources