Docker manager 1 grants access 2 times out - docker

Question

I work with 2 managers and 4 worker nodes.
root#ist-manager-1:~# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
xx antep-db-1 Ready Active 20.10.23
xx antep-manager-1 Ready Active Leader 20.10.23
xx antep-worker-1 Ready Active 20.10.23
xx ist-db-1 Ready Active 20.10.23
xx * ist-manager-1 Ready Active Reachable 20.10.23
xx ist-worker-1 Ready Active 20.10.23
I have a microservice running on laravel and octane that I wrote myself.
The yaml file I used to boot the service
version: '3.2'
services:
service:
image: xx/auth-service-roudrunner:1.0.2
tty: true
networks:
- kong-net
deploy:
placement:
constraints:
- node.role == worker
- node.labels.type == worker
environment:
- DB_HOST=auth_db
- DB_PORT=3306
- DB_DATABASE=auth-db
- DB_USERNAME=root
- DB_PASSWORD=xx
db:
image: mysql
tty: true
networks:
- kong-net
deploy:
placement:
constraints:
- node.role == worker
- node.labels.type == db
environment:
- MYSQL_ROOT_PASSWORD=xxx
- MYSQL_DATABASE=auth-db
networks:
kong-net:
external: true
My problem is:
When I send a curl request to auth_service:8000 on ist-manager-1, the response is received.
When I send a curl request to auth_service:8000 on antep-manager-1, the connection times out.
I can ping from two managers. I can share port directly and access over IP.
ist-manager-1 curl:
root#ist-manager-1:~# docker run --network kong-net --rm curlimages/curl:7.87.0 --max-time 15 -L -v http://auth_service:8000
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.0.6.40:8000...
* Connected to auth_service (10.0.6.40) port 8000 (#0)
> GET / HTTP/1.1
> Host: auth_service:8000
> User-Agent: curl/7.87.0-DEV
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Cache-Control: no-cache, private
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 27 Jan 2023 20:58:43 GMT
< Set-Cookie: XSRF-TOKEN=dd; expires=Fri, 27 Jan 2023 22:58:43 GMT; Max-Age=7200; path=/; samesite=lax
< Set-Cookie: xx_auth_service_session=yy; expires=Fri, 27 Jan 2023 22:58:43 GMT; Max-Age=7200; path=/; httponly; samesite=lax
< Content-Length: 6
<
{ [6 bytes data]
100 6 100 6 0 0 645 0 --:--:-- --:--:-- --:--:-- 666
* Connection #0 to host auth_service left intact
antep-manager-1 curl:
root#antep-manager-1:~# docker run --network kong-net --rm curlimages/curl:7.87.0 --max-time 15 -L -v http://auth_service:8000
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.0.6.40:8000...
0 0 0 0 0 0 0 0 --:--:-- 0:00:14 --:--:-- 0* Connection timed out after 15000 milliseconds
0 0 0 0 0 0 0 0 --:--:-- 0:00:15 --:--:-- 0
* Closing connection 0
curl: (28) Connection timed out after 15000 milliseconds
While the request can be sent through one manager on Docker without any problems, the request over the other manager times out.