I try to connect elasticsearch on Gitlab CI for running my test but always got connection refused
here my script
services:
- name: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
script:
- curl "http://$(cat /etc/hosts |grep $HOSTNAME|cut -d$'\t' -f1):9200"
Wtih $(cat /etc/hosts |grep $HOSTNAME|cut -d$'\t' -f1) I can get IP address
I try to use alias: elasticsearch but when I curl "http://elasticsearch:9200" also got error Could not resolve host
I have following docker-compose.yml redis config.
version: '3.5'
services:
db:
image: redis:latest
command: redis-server --bind 0.0.0.0 --appendonly yes --protected-mode no
ports:
- target: 6379
published: 6379
protocol: tcp
mode: ingress
There are two hosts leader-0 (manager) and redis-0 (worker)
> root#leader-0:~# docker node ls
ID HOSTNAME STATUS
46tmallxr4l8xr7i90vlwntjq * leader-0 Ready
mofbedj4sqlxgnyatbxhlokc7 redis-0 Ready
Redis host redis-0 exposes 6379 port on the localhost as expected:
> root#redis-0:~# redis-cli -h 127.0.0.1 ping
PONG
but 6379 is not available on the manager (although it should):
> root#leader-0:~# redis-cli -h 127.0.0.1 ping
Could not connect to Redis at 127.0.0.1:6379: Connection timed out
Interesting part is:
Connection timed out (not refused).
redis-cli -h 127.0.0.1 ping on other workers hosts works as expected (returns PONG).
Docker overlay mash network should expose 6379 port on the local interface on each host, but it looks like something went wrong and I messed up figuring out what exactly.
Other services on the manager host works properly (I can
curl http://localhost:${SERVICE_PORT}/).
Manager host has the same firewall rules as worker hosts (+ additional ports opened)
I try to setup traefik with a Docker Swarm. I have to VMs - one manger-node and one worker-node.
In addition I have created a external network with:
docker network create --driver=overlay proxy-net
I start traefik as a service within my manager-node with the following docker-compose.yml file:
version: '3'
services:
traefik:
image: traefik:v1.4.4
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- $PWD/management/traefik/traefik.toml:/etc/traefik/traefik.toml
ports:
- 80:80
- 8100:8080
deploy:
placement:
constraints:
- node.role == manager
networks:
default:
external:
name: proxy-net
My traefik.toml file looks like this:
Debug : "DEBUG"
defaultEntryPoints = ["http"]
[entryPoints]
[entryPoints.http]
address = ":80"
[web]
address = ":8080"
[docker]
watch = true
swarmmode = true
domain = "mydomain.com"
exposedbydefault = true
When I now start a new service (e.g. emilevauge/whoami) with:
docker service create \
--name whoami1 \
--publish mode=host,target=80,published=8002 \
--network proxy-net \
--label traefik.docker.network=proxy-net \
--label traefik.frontend.rule=Host:whoami.mydomain.com \
--label traefik.port=8002 \
emilevauge/whoami
The service is seen by the traefik web frontend. So at first every thing looks fine. I can access the service directly on my worker node on port 8002.
But traefik does not seem to be able to proxy this service. When I browse my endpoint URL (whomai.mydomain.com) I get the answer:
Bad Gateway
The traefik logfile (logLevel=DEBUG) shown messages like this:
proxy_traefik.1.zl50yv6got5f#tocidoc001 time="2017-12-03T20:09:28Z" level=debug msg="Filtering container without port and no traefik.port label swarmpit_app.1 : strconv.Atoi: parsing "": invalid syntax"
proxy_traefik.1.zl50yv6got5f#tocidoc001 time="2017-12-03T20:09:28Z" level=debug msg="Filtering container without port and no traefik.port label proxy_traefik.1 : strconv.Atoi: parsing "": invalid syntax"
proxy_traefik.1.zl50yv6got5f#tocidoc001 time="2017-12-03T20:09:28Z" level=debug msg="Filtering container without port and no traefik.port label swarmpit_db.1 : strconv.Atoi: parsing "": invalid syntax"
proxy_traefik.1.zl50yv6got5f#tocidoc001 time="2017-12-03T20:09:28Z" level=debug msg="Validation of load balancer method for backend backend-whoami1-whoami1-whoami1 failed: invalid load-balancing method ''. Using default method wrr."
proxy_traefik.1.zl50yv6got5f#tocidoc001 time="2017-12-03T20:09:28Z" level=debug msg="Configuration received from provider docker: {"backends":{"backend-whoami1-whoami1-whoami1":{"servers":{"service-0":{"url":"http://10.0.1.5:8002","weight":0}},"loadBalancer":{"method":"wrr"}}},"frontends":{"frontend-whoami1-whoami1-whoami1":{"entryPoints":["http"],"backend":"backend-whoami1-whoami1-whoami1","routes":{"service-whoami1":{"rule":"Host:whoami.mydomain.com"}},"passHostHeader":true,"priority":0,"basicAuth":[],"headers":{}}}}"
I played around several hours with different configurations. I also read the very concise documentation about traefik and docker-swarm. But I don't get any idea what I'm doing wrong.
Can any body help me with some tips how to better understand the problem?
I think it is not working because you Træfik service is not on the same docker network as your whoami1 service.
You should try to add proxy-net network to your Træfik service in your compose file.
There is a warning in Træfik documentation at the end of this page https://docs.traefik.io/configuration/backends/docker/
when running inside a container, Træfik will need network access through:
docker network connect <network> <traefik-container>
As already mentioned, they need to be in the same overlay network which is not ingress. The ingress network is only for manager nodes.
Further more, your traefik service is not assigned to the proxy-net network. You're creating proxy-net in your traefik config part, but don't assigned it to it
version: '3'
services:
traefik:
image: traefik:v1.4.4
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- $PWD/management/traefik/traefik.toml:/etc/traefik/traefik.toml
ports:
- 80:80
- 8100:8080
networks:
- proxy-net
deploy:
placement:
constraints:
- node.role == manager
networks:
proxy-net:
driver: overlay
Further more, you should create a config with docker config create. Otherwise with $PWD/management/traefik/traefik.toml you need to copy the traefik.toml file to every manager node.
Append your compose file with
configs:
traefik_conf_v1:
file: ./traefik.toml
and your traefik part with
configs:
- source: traefik_conf_v1
target: /etc/traefik/traefik.toml
Now back to your problem.
What's your service is missing is the label to the backend. Otherwise traefik doesn't know where the service is running (network assignment isn't enough!).
docker service create \
--name whoami1 \
--publish mode=host,target=80,published=8002 \
--network proxy-net \
--label traefik.backen=whoami1 \
--label traefik.docker.network=proxy-net \
--label traefik.frontend.rule=Host:whoami.mydomain.com \
--label traefik.port=8002 \
emilevauge/whoami
This should work. And when it does, stop publishing ports of your services. That makes everything complicated when you're in a hurry and need to scale. Remember, work balancing is handle by the swarm itself.
And yeah, dynamic flexible reverse proxys is still a problem nowadays :)
Remember, you got your entry points on manager nodes with traefik, but not on the worker nodes.
I finally I solved this issue. It was actually not a Traefik problem.
The problem was, that both VMs from my provider have the same private IPv4 address.
To register and join the docker-swarm it is important to provide the public IPv4 addresses with the option --advertise-addr
To register the swarm I have to run:
docker swarm init --advertise-addr [manager-ip-address]
to join the swarm by a worker-node also the public IPv4 address need to be set explicitly:
docker swarm join \
--token SWMTKN-1-xxxxxxxxxxxxxxxxxxxx-xxxxxxxx \
--advertise-addr [worker-ip-address]\
[manager-ip-address]:2377
I would say that your setup of service labels was wrong. Traefik redirects requests to swarm service port so it should go to port 80, not to published port 8002. I think that correct service create command should be:
docker service create \
--name whoami1 \
--publish mode=host,target=80,published=8002 \
--network proxy-net \
--label traefik.docker.network=proxy-net \
--label traefik.frontend.rule=Host:whoami.mydomain.com \
--label traefik.port=80 \
emilevauge/whoami
And publishing the 80 port for whoami service is not needed.
I have specified port-mapping in docker-compose, but it is still not working, i still have to access site using the port no specified in expose
below is my docker-compose.yml
version: '2'
networks:
default:
external:
name: nat
services:
website:
build:
context: '.'
dockerfile: "./iis.dockerfile"
ports:
- 3000:8081
and the corrosponding dockerfile
FROM microsoft/iis
RUN ["powershell.exe", "Install-WindowsFeature NET-Framework-45-ASPNET"]
RUN ["powershell.exe", "Install-WindowsFeature Web-Asp-Net45"]
ADD www/ c:\\webapp
EXPOSE 8081
RUN powershell New-Website -Name 'web-app' -Port 8081 -PhysicalPath 'c:\webapp' -ApplicationPool '.NET v4.5'
I have to access app using: http://192.168.105.33:8081/, if I do it using port 3000 it does not work.
Is there anything missing in my above configuration?? OS: windows server 2016, using windows containers with hyper-v and docker-compose up -d to get containers up and running...
docker-compose inspect gives me below output
"Ports": {
"8081/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "3000"
}
]
},
docker ps gives me below output
eb7aa1e74b7f website_website "C:\\ServiceMonitor..." 14 minutes ago Up 14 minutes 0.0.0.0:3000->
8081/tcp website_website_1
dokcer-compose ps gives me below output
Name Command State Ports
--------------------------------------------------------------------------------
website_website_1 C:\ServiceMonitor.exe w3svc Up 0.0.0.0:3000->8081/tcp
So I should be able to access it on host using port 3000.
Maybe you're launching your container with docker-compose run? There is a difference in port mapping between docker-compose run and docker-compose up [-d] [service] In this case the port configuration will be ignored by design
You can use --service-ports flag or manually expose them using -p flag
I have 2 docker containers managed with docker-compose and can't seem to properly use webpack to proxy some request to the backend api.
docker-compose.yml :
version: '2'
services:
web:
build:
context: ./frontend
ports:
- "80:8080"
volumes:
- ./frontend:/16AGR/frontend:rw
links:
- back:back
back:
build:
context: ./backend
expose:
- "8080"
ports:
- "8081:8080"
volumes:
- ./backend:/16AGR/backend:rw
the service web is a simple react application served by a webpack-dev-server.
the service back is a node application.
I have no problem to access either app from my host :
$ curl localhost
> index.html
$ curl localhost:8081
> Hello World
I can also ping and curl the back service from the web container :
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
73ebfef9b250 16agr_web "npm run start" 37 hours ago Up 13 seconds 0.0.0.0:80->8080/tcp 16agr_web_1
a421fc24f8d9 16agr_back "npm start" 37 hours ago Up 15 seconds 0.0.0.0:8081->8080/tcp 16agr_back_1
$ docker exec -it 73e bash
$ root#73ebfef9b250:/16AGR/frontend# curl back:8080
Hello world
However i have a problem with the proxy.
Webpack is started with
webpack-dev-server --display-reasons --display-error-details --history-api-fallback --progress --colors -d --hot --inline --host=0.0.0.0 --port 8080
and the config file is
frontend/webpack.config.js :
var webpack = require('webpack');
var config = module.exports = {
...
devServer: {
//redirect api calls to backend server
proxy: {
'/api': {
target: "back:8080",
secure: false
}
}
}
...
}
When i try to request /api/test with a link in my app for exemple i get a generic error, the link and google did not help much :(
[HPM] Error occurred while trying to proxy request /api/test from localhost to back:8080 (EINVAL) (https://nodejs.org/api/errors.html#errors_common_system_errors)
I suspect some weird thing because the proxy is on the container and the request is on localhost but I don't really have an idea to solve this.
I think I managed to tacle the problem.
Just had to change the webpack configuration with the following
devServer: {
//redirect api calls to backend server
proxy: {
'/api': {
target: {
host: "back",
protocol: 'http:',
port: 8080
},
ignorePath: true,
changeOrigin: true,
secure: false
}
}
}