Docker containers containers unable to talk to each other - docker

Question

I am trying to create to create a docker swarm and containers created on different machines are unable to talk to each other, I have done a lot of research, but still I am unable to figure out whats happening
Swarm command : docker swarm init --default-addr-pool 30.30.0.0/16
Docker compose file :
version: '3.9'
services:
chrome:
image: selenium/node-chrome:4.1.2-20220217
shm_size: 2gb
networks:
- mynet
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
deploy:
replicas: 1
entrypoint: bash -c 'SE_OPTS="--host $$HOSTNAME" /opt/bin/entry_point.sh'
selenium-hub:
image: selenium/hub:4.1.2-20220217
networks:
- mynet
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
networks:
mynet:
driver: overlay
attachable: true
After this stack is deployed, chrome service is unable to find selenium-hub service.
Docker network inspect on Worker node looks like this
[
{
"Name": "grid_mynet",
"Id": "dm4nx67j7tc6309fzp6rs7cl6",
"Created": "2022-03-08T09:57:17.654456805-05:00",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "30.30.3.0/24",
"Gateway": "30.30.3.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"bdf55ddb60384c47733a104f9e2478b3a1276383b2f02883b4e4bd082dbc9667": {
"Name": "grid_chrome.1.8iuwofyuk53xosbz3hmnk0ylb",
"EndpointID": "9c77c6b827e6ce224db3a8d4ee17590e96f258a20c4083be7a6b28cdc3af4065",
"MacAddress": "02:42:1e:1e:03:03",
"IPv4Address": "30.30.3.3/24",
"IPv6Address": ""
},
"lb-grid_mynet": {
"Name": "grid_mynet-endpoint",
"EndpointID": "29ba13bf815cb98bd507ec3a4934cfc3b3b1d7f819c139d0a8a1576bbfb7e935",
"MacAddress": "02:42:1e:1e:03:04",
"IPv4Address": "30.30.3.4/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4099"
},
"Labels": {
"com.docker.stack.namespace": "grid"
},
"Peers": [
{
"Name": "4bb6a3117f30",
"IP": "10.211.55.18"
}
]
}
]
Docker network inspect on Manager node looks like this
[
{
"Name": "grid_mynet",
"Id": "dm4nx67j7tc6309fzp6rs7cl6",
"Created": "2022-03-08T09:57:19.105558268-05:00",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "30.30.3.0/24",
"Gateway": "30.30.3.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"e6c571aad3215675727603e786ecf2902a0c9423dc8f5888523b8cb4f3a0fbb2": {
"Name": "grid_selenium-hub.1.klcnq45z2im4p1fpj2aoaxqq8",
"EndpointID": "7689edd38aae48c1e087ddebcc2dde7ad6f01fb1d7bfc979a1ae86d8b9de474f",
"MacAddress": "02:42:1e:1e:03:06",
"IPv4Address": "30.30.3.6/24",
"IPv6Address": ""
},
"lb-grid_mynet": {
"Name": "grid_mynet-endpoint",
"EndpointID": "fa6f18959fe04bf25f9e15970983999eb2d04f61f372408ac5dabdcb246e0751",
"MacAddress": "02:42:1e:1e:03:07",
"IPv4Address": "30.30.3.7/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4099"
},
"Labels": {
"com.docker.stack.namespace": "grid"
},
"Peers": [
{
"Name": "f3a79e1a65b5",
"IP": "10.211.55.16"
}
]
}
]
/etc/host file on chrome container looks like this
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
30.30.3.3 bdf55ddb6038
Pinging chrome from chrome container produce this result
seluser#bdf55ddb6038:/$ ping chrome
PING chrome (30.30.3.2) 56(84) bytes of data.
64 bytes from 30.30.3.2 (30.30.3.2): icmp_seq=1 ttl=64 time=0.077 ms
64 bytes from 30.30.3.2 (30.30.3.2): icmp_seq=2 ttl=64 time=0.078 ms
Pinging Selenium-hub from chrome container produces this
seluser#bdf55ddb6038:/$ ping selenium-hub
ping: selenium-hub: Name or service not known
/etc/hosts file on selenium-hub container looks
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
30.30.3.6 e6c571aad321
Pinging chrome from selenium-hub container produce this result
seluser#e6c571aad321:/$ ping chrome
ping: chrome: No address associated with hostname
seluser#e6c571aad321:/$ ^C
seluser#e6c571aad321:/$ ping grid_chrome
ping: grid_chrome: Name or service not known
Pinging selenium-hub from selenium-hub container produce this result
seluser#e6c571aad321:/$ ping selenium-hub
PING selenium-hub (30.30.3.5) 56(84) bytes of data.
64 bytes from 30.30.3.5 (30.30.3.5): icmp_seq=1 ttl=64 time=0.084 ms
64 bytes from 30.30.3.5 (30.30.3.5): icmp_seq=2 ttl=64 time=0.079 ms
Chrome container have this error in the end
Caused by: java.net.UnknownHostException: selenium-hub
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
at zmq.io.net.tcp.TcpAddress.resolve(TcpAddress.java:132)
... 37 more