I'm having an issue with one of my docker containers connecting to my jenkins master. This used to work fine for several months but something must have changed either in Jenkins or our corporate firewall rules that I haven't been able to pinpoint.
Jenkins communicates with Docker host on port 4243 for Docker API.
I have the JNLP port fixed to 50724. My container is using jenkins/jnlp-slave as the base image. I'm using the Yet Another Docker Plugin.
Jenkins is able to start the container but it fails to establish the JNLP4 connnection. This is the error from docker logs of the container:
Feb 19, 2019 7:49:42 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: YAD Singapore Docker-ead378f6bce7
Feb 19, 2019 7:49:42 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Feb 19, 2019 7:49:42 AM hudson.remoting.jnlp.Main createEngine
WARNING: Certificate validation for HTTPs endpoints is disabled
Feb 19, 2019 7:49:42 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.29
Feb 19, 2019 7:49:42 AM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Feb 19, 2019 7:49:42 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://jenkins-master.work.com/]
Feb 19, 2019 7:49:42 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver openURLConnection
WARNING: HTTPs certificate check is disabled for the endpoint.
Feb 19, 2019 7:49:43 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: jenkins-master.work.com
Agent port: 50724
Identity: 3c:1d:86:85:6a:18:a1:bd:89:a7:a9:aa:1b:6b:0c:20
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-master.work.com:50724
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:614)
at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecvClosed(AckFilterLayer.java:280)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1800(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-master.work.com:50724
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP3-connect not supported, skipping
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Feb 19, 2019 7:49:43 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)
at hudson.remoting.Engine.innerRun(Engine.java:639)
at hudson.remoting.Engine.run(Engine.java:474)
Jenkins logs has this:
Feb 19, 2019 7:49:27 AM INFO com.github.kostyasha.yad.DockerCloud provision
Asked to provision load: '1', for: 'sing-slave-docker' label
Feb 19, 2019 7:49:27 AM INFO com.github.kostyasha.yad.DockerCloud provision
Will provision 'jnlp-slave-ssh', for label: 'sing-slave-docker', in cloud: 'YAD Singapore Docker'
Feb 19, 2019 7:49:28 AM INFO com.github.kostyasha.yad.DockerCloud addProvisionedSlave
Provisioning 'jnlp-slave-ssh' number '0' on 'YAD Singapore Docker'; Total containers: '0'
Feb 19, 2019 7:49:37 AM INFO hudson.slaves.NodeProvisioner$2 run
jnlp-slave-ssh provisioning successfully completed. We have now 3 computer(s)
Feb 19, 2019 7:49:37 AM INFO com.github.kostyasha.yad.launcher.DockerComputerJNLPLauncher launch
Starting connection command for ead378f6bce7616b7264de0605747f3299a4c750118c161d68e25bb99ea64b2c...
Feb 19, 2019 7:49:43 AM WARNING hudson.TcpSlaveAgentListener$ConnectionHandler run
Connection #703 failed
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)
Feb 19, 2019 7:49:43 AM WARNING hudson.TcpSlaveAgentListener$ConnectionHandler run
Connection #704 failed
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:192)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)
Feb 19, 2019 7:49:44 AM WARNING hudson.TcpSlaveAgentListener$ConnectionHandler run
Connection #705 failed
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)
Now I have another docker host that is not behind a firewall using the same docker image and it is able to connect and run my build. That's where I figure it has to be an issue with the firewall. But looking at the logs of the successful connection, I'm confused about what ports are actually being used. I know jenkins->docker on port 4243 for Docker API. JNLP port fixed to 50724. The container exposes port 4200 and is mapped to port 49810.
d442c6d53a1b jnlp-slave-ssh "/bin/sh -cxe 'cat <<" 0.0.0.0:49810->4200/tcp sleepy_liskov
But in the jenkins log it shows that it connects on some other port 56602:
Asked to provision load: '1', for: 'lewi-slave-docker' label
Feb 19, 2019 12:36:07 AM INFO com.github.kostyasha.yad.DockerCloud provision
Will provision 'jnlp-slave-ssh', for label: 'lewi-slave-docker', in cloud: 'YAD Lewisville Docker'
Feb 19, 2019 12:36:07 AM INFO com.github.kostyasha.yad.DockerCloud addProvisionedSlave
Provisioning 'jnlp-slave-ssh' number '0' on 'YAD Lewisville Docker'; Total containers: '0'
Feb 19, 2019 12:36:17 AM INFO hudson.slaves.NodeProvisioner$2 run
jnlp-slave-ssh provisioning successfully completed. We have now 4 computer(s)
Feb 19, 2019 12:36:17 AM INFO com.github.kostyasha.yad.launcher.DockerComputerJNLPLauncher launch
Starting connection command for d442c6d53a1b0a3ffa3f55732bceb112f3efacd1078313744cffb6d6c44eae21...
Feb 19, 2019 12:36:20 AM WARNING hudson.TcpSlaveAgentListener$ConnectionHandler run
Connection #562 failed
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:244)
Feb 19, 2019 12:36:20 AM INFO hudson.TcpSlaveAgentListener$ConnectionHandler run
Accepted JNLP4-connect connection #563 from lewi-docker.work.com/10.180.168.192:56602
What is port 56602 used for? This port is also random. When I run it again it shows up as 57820, etc.
Anything else I can look at or try?
OK, after much back and forth it was a firewall issue that was blocking the Agent Port 50724.
Related
I created a jenkins master in my OKE cluster (a single node cluster). I wanted to try using Jenkins kubernetes plugin and connect jnlp-slaves.
Following are my kuberenetes cloud configurations.
Jenkins tunnel and URL info
Jenkins version that Im using is Jenkins 2.319.3. When I start the build, a pod is creating in my cluster and gives an error.
Following is the log of the created pod.
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: kube-agent-s8tw6
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Mar 17, 2022 5:49:10 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 4.11.2
Mar 17, 2022 5:49:10 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using /home/jenkins/agent/remoting as a remoting work directory
Mar 17, 2022 5:49:10 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
INFO: Both error and output logs will be printed to /home/jenkins/agent/remoting
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://138.2.75.190:30000/]
Mar 17, 2022 5:49:10 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Mar 17, 2022 5:49:10 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: http://138.2.75.190
Agent port: 50000
Identity: 5b:f2:ef:be:d1:98:05:91:83:3d:a5:61:b6:1b:93:e5
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to http://138.2.75.190:50000
Mar 17, 2022 5:49:10 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: null
java.nio.channels.UnresolvedAddressException
at java.base/sun.nio.ch.Net.checkAddress(Unknown Source)
at java.base/sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
at java.base/java.nio.channels.SocketChannel.open(Unknown Source)
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:206)
at hudson.remoting.Engine.connectTcp(Engine.java:880)
at hudson.remoting.Engine.innerRun(Engine.java:757)
at hudson.remoting.Engine.run(Engine.java:540)
I checked the ports and I disabled firewall in my node. Im still getting the error. Can someone help me to solve this?
I'm running Jenkins on Kubernetes with a dynamic slave pod provisioning setup.
It used to work well for a long time but somehow started to giving this issue from yesterday.
This is the log that I'm getting from the slave pods
Warning: SECRET is defined twice in command-line arguments and the environment variable
Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: default-jxrsk
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Jun 26, 2021 10:48:16 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.40
Jun 26, 2021 10:48:16 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using /home/jenkins/remoting as a remoting work directory
Jun 26, 2021 10:48:16 AM org.jenkinsci.remoting.engine.WorkDirManager setupLogging
INFO: Both error and output logs will be printed to /home/jenkins/remoting
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://jenkins.jenkins:8080]
Jun 26, 2021 10:48:16 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jun 26, 2021 10:48:16 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: jenkins-agent
Agent port: 50000
Identity: 04:b1:b3:70:fe:e4:89:1b:30:62:4d:c9:6a:53:8a:3b
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-agent:50000
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 04:b1:b3:70:fe:e4:89:1b:30:62:4d:c9:6a:53:8a:3b
Jun 26, 2021 10:48:16 AM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins-agent/10.245.115.232:50000] Local headers refused by remote: Unknown client name: default-jxrsk
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: default-jxrsk
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:579)
at hudson.remoting.Engine.run(Engine.java:490)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: default-jxrsk
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:98)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
... 7 more
Jun 26, 2021 10:48:16 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:664)
at hudson.remoting.Engine.innerRun(Engine.java:604)
at hudson.remoting.Engine.run(Engine.java:490)
And getting this from the master Jenkins
2021-06-26 10:47:57.232+0000 [id=73] INFO o.j.r.p.i.ConnectionHeadersFilterLayer#onRecv: [JNLP4-connect connection from 10.244.75.115/10.244.75.115:48028] Refusing headers from remote: Unknown client name: default-t09rx
2021-06-26 10:48:01.652+0000 [id=29] INFO o.c.j.p.k.KubernetesCloud#provision: Excess workload after pending Kubernetes agents: 1
2021-06-26 10:48:01.653+0000 [id=29] INFO o.c.j.p.k.KubernetesCloud#provision: Template for label null: default
2021-06-26 10:48:11.659+0000 [id=73] INFO o.c.j.p.k.KubernetesLauncher#launch: Created Pod: jenkins/default-jxrsk
2021-06-26 10:48:11.660+0000 [id=73] WARNING o.c.j.p.k.KubernetesLauncher#launch: Error in provisioning; agent=KubernetesSlave name: default-jxrsk, template=PodTemplate{inheritFrom='', name='default', namespace='', hostNetwork=false, label='jenkins-jenkins-slave', serviceAccount='default', nodeSelector='doks.digitalocean.com/node-pool=k8s-generic', nodeUsageMode=NORMAL, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], volumes=[HostPathVolume [mountPath=/var/run/docker.sock, hostPath=/var/run/docker.sock]], containers=[ContainerTemplate{name='jnlp', image='myrepo/myimage:jenkins_jnlp-3.40-1-helm3', alwaysPullImage=true, workingDir='/home/jenkins', command='', args='${computer.jnlpmac} ${computer.name}', resourceRequestCpu='1000m', resourceRequestMemory='4096Mi', resourceLimitCpu='1000m', resourceLimitMemory='4096Mi', envVars=[ContainerEnvVar [getValue()=http://jenkins.jenkins:8080, getKey()=JENKINS_URL], KeyValueEnvVar [getValue()=jenkins-agent:50000, getKey()=JENKINS_TUNNEL]], livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe#2e38f1da}], imagePullSecrets=[org.csanchez.jenkins.plugins.kubernetes.PodImagePullSecret#5a6fc800]}
java.lang.NoSuchMethodError: io.fabric8.kubernetes.client.dsl.PodResource.watch(Ljava/lang/Object;)Ljava/lang/Object;
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:134)
at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:294)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-06-26 10:48:11.661+0000 [id=73] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent default-jxrsk
Any idea? I was trying to Google it for a whole day, but didn't find anything really helpful.
Need your help guys. Thanks.
Self reply:
I was able to provision slave pods again after I upgraded my Jenkins and Kubernetes plugin to the latest.
The version I'm at is Jenkins 2.289.1, and Kubernetes plugin 1.30.0.
I had a similar issue once. I can't recall the exact details but it either had to do with an update to the jcasc plugin or Jenkins itself. To fix it I updated the args specified for the agent to be "^${computer.jnlpmac} ^${computer.name}". From what I recall this was to avoid some premature interpolation.
Upgrading Kubernetes plugin(1.30.11) to the latest fixed the same issue in Jenkins 2.289.3
Looking for some help here!
I am running Jenkins(v2.134) as a docker container managed by Rancher(v1.6.16); and i am using haproxy(HA-Proxy version 1.6.3 2015/12/25) as my loadbalancer. Jenkins JNLP port is configured as 50000. haproxy got the rule for TCP port forwarding(8081 > 50000)
My slave (macmini) is in different network and behind firewall (This network can make reach my Jenkins Infra). I am using "Tunnel connection through" property and specified port ":8081". The port 8081 is already opened in the firewall.
My Jenkins Java version is -
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
My Slave Java version is -
Sun JDK "1.8.0_151"
Here are the agent logs -
*INFO: Setting up agent: my-slave-01*
Sep 26, 2018 2:48:50 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Sep 26, 2018 2:48:50 PM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.23
Sep 26, 2018 2:48:50 PM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /Users/jenkins/.jenkins/cache/jars
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://<My Jenkins URL>/]
Sep 26, 2018 2:48:51 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
*INFO: Agent discovery successful*
*Agent address: <My Jenkins DNS Name>*
*Agent port: 8081*
Identity: b5:c7:33:8d:9c:97:41:3f:e1:b1:b5:31:25:ea:b5:2e
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to <My Jenkins DNS>:8081
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Sep 26, 2018 2:48:51 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: b5:c7:33:8d:9c:97:41:3f:e1:b1:b5:31:25:ea:b5:2e
Sep 26, 2018 2:48:52 PM hudson.remoting.jnlp.Main$CuiListener status
*INFO: Connected*
Sep 26, 2018 2:48:54 PM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNING: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.envinject.EnvInjectComputerListener$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Sep 26, 2018 2:49:49 PM hudson.remoting.jnlp.Main$CuiListener status
*INFO: Terminated*
Any help will be appreciated.
Is there is a way to get more verbose logs?
I found the problem; it was with my load balancer(haproxy) default timeout for TCP connections (if no traffic) which was 50000ms.
https://cbonte.github.io/haproxy-dconv/configuration-1.5.html
However the Jenkins ping interval to slave i.e.hudson.slaves.ChannelPinger.pingInterval was 5 minutes (dafault).
https://wiki.jenkins.io/display/JENKINS/Features+controlled+by+system+properties
Increasing the load balancer timeout i.e. > 5 minutes solved the issue.
I am running jenkins master on kubernetes as docker container with docker image (jenkinsci/blueocean:1.5.0).
I am trying to add a slave node (ec2 ubuntu instace installed java version "1.8.0_171") via web start on fixed port.
I would able to telnet from my salve node to the master node
(telnet
connected:
but when I am trying to add my slave code to master node with java -jar ......
its keep on failing.
here are the logs below from jenkins slave node:
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: jenkins_slave
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
May 07, 2018 1:05:18 AM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using /opt/jenkins/remoting as a remoting work directory
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://<jenkins_master_url>/]
May 07, 2018 1:05:18 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: <jenkins_master_url>
Agent port: M<fixed port number>
Identity: df:0a:5c:07:b1:32:51:72:60:9f:eb:61:dc:45:68:24
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to <jenkins_masterUrl:fixedportnumber>
May 07, 2018 1:05:18 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:609)
at hudson.remoting.Engine.run(Engine.java:469)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Connection closed before acknowledgement sent
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.onRecvClosed(AckFilterLayer.java:280)
at org.jenkinsci.remoting.protocol.FilterLayer.abort(FilterLayer.java:164)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer.access$000(AckFilterLayer.java:43)
at org.jenkinsci.remoting.protocol.impl.AckFilterLayer$1.run(AckFilterLayer.java:176)
at org.jenkinsci.remoting.protocol.IOHub$DelayedRunnable.run(IOHub.java:958)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1$1.run(Engine.java:94)
at java.lang.Thread.run(Thread.java:748)
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-dev-aws.twg.co.nz:50000
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP3-connect not supported, skipping
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
May 07, 2018 1:05:29 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:670)
at hudson.remoting.Engine.innerRun(Engine.java:634)
at hudson.remoting.Engine.run(Engine.java:469)
is there any I am missing or if its a bug, is there any work around
I',m struggling to connect my windows slave to the Jenkins server (Linux), but with no luck. Tried many ways, resolved few, and finally, I'm stuck with a connection timeout. I see some of the post mentioning similar problems has no response.
Exception log
C:\jenkinsbin>java -jar slave.jar -jnlpUrl https://<jenkins.com>/comput
er/SFDC%20Node/slave-agent.jnlp -secret <****> -workDir "C:/jenkinsslavesfdc"
Feb 22, 2018 2:16:40 AM org.jenkinsci.remoting.engine.WorkDirManager initializeW
orkDir
INFO: Using C:\jenkinsslavesfdc\remoting as a remoting work directory
Both error and output logs will be printed to C:\jenkinsslavesfdc\remoting
Feb 22, 2018 2:16:41 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up slave: SFDC Node
Feb 22, 2018 2:16:41 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Feb 22, 2018 2:16:41 AM org.jenkinsci.remoting.engine.WorkDirManager initializeW
orkDir
INFO: Using C:\jenkinsslavesfdc\remoting as a remoting work directory
Feb 22, 2018 2:16:41 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://<jenkins.com>/]
Feb 22, 2018 2:16:43 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver
resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, JNLP-conn
ect, Ping, JNLP2-connect]
Feb 22, 2018 2:16:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: <jenkins.com>
Agent port: 55202
Identity: 80:e0:51:b6:65:64:7b:78:ce:ea:7d:c2:9d:27:5b:2a
Feb 22, 2018 2:16:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Feb 22, 2018 2:16:43 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.com:55202
Feb 22, 2018 2:17:14 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.com:55202 (retrying:2)
java.io.IOException: Failed to connect to jenkins.com:55202
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoin
t.java:243)
at hudson.remoting.Engine.connect(Engine.java:667)
at hudson.remoting.Engine.innerRun(Engine.java:528)
at hudson.remoting.Engine.run(Engine.java:451)
Caused by: java.net.ConnectException: Connection timed out: connect
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Unknown Source)
at sun.nio.ch.Net.connect(Unknown Source)
at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
at java.nio.channels.SocketChannel.open(Unknown Source)
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoin
t.java:204)
... 3 more
Few observations from my end:
Slave is able to ping jenkins.com
Port 55202 added under firewall (inbound & outbound) rule to allow connections in slave.
Port 55202 is listening in master
Also, since I’m using windows as my slave, is it suggested to use only java web start launch?
Tried traceroute in slave, but very inconsistent.
Is iptables in master are restricted for any non-identical subnets? I guess the master is not responding and hence connection timeout?
Rohit,
Can you check if this is relevant in your case?
Connecting Jenkins slave agent on windows failed with connection timeout
W.r.t. 2: You can completely turn off firewall if you are working in a same secured network.