I have simple docker-compose aka:
version: '3'
services:
app:
container_name: app
ports:
- 8081:8081
db:
container_name: db
ports:
- 5432:5432
And by default this containers are created in default(brige) network.
The app has db connection property: jdbc:postgresql://db/some_db, and everythig works perfectly. But from time to time I want the app to connecto to other db, that is running on my host machine, not in docker container.
The main problem is that I can not change my connection properties. And, ideally, I do not want to run new container with some additional options every time I want to switch the db host (but restart is ok)
Hence my question: what is the best way to achive this? Is it possible to set up additional route for containers host resolving? For exapmle, if db container is unreachable, then route to host.
You can access host services from your host.
See "How to access host port from docker container":
ip addr show docker0
docker.for.mac.localhost # docker 17.06+ June 2017
if db container is unreachable, then route to host.
That is a job for an orchestrator.
For instance, with kubernetes, you can associate an external load balencer, which could be tuned to redirect traffic to your pod, unless it is not accessible.
Related
I am new to docker and have trouble setting up the network between the containers to not allow unnecessary connections from outside.
I have a Docker running on a VPS with three containers on a remote IP 123.xxx.xxx.xxx
container name published ports IP adress
sqldb 3306:3306 172.xxx.xxx.4
applet1 80:3306 172.xxx.xxx.5
applet2 4444:4444 172.xxx.xxx.3
One is database and two are java apps. The trouble I am having right now is that when I create the containers the ports on the containers become exposed to the global internet so my database sqldb is exposed by 123.xxx.xxx.xxx:3306
Right now ny java apps are connect through JDBC like so jdbc:mysql://172.xxx.xxx.4:3306/db.
I am trying to accomplish the following:
port 80 on host so 123.xxx.xxx.xxx connects to java app applet1.
The goal is to give applet1 the ability to connect to sqldb and also applet2 but I don't wan't unecessary ports to be exposed to the whole internet. Preferably that internal URIs would be left but connections from outside (apart from SSH on port 22 and TCP on port 80) would be forbidden for ports 4444, 3306. Also, I don't yet know how to use docker-compose so if possible how can I solve it without it?
*I have heard you can connect to containers by writing container names like that: have not had success with it yet jdbc:mysql://sqldb/db.
If all your containers are running on the same docker bridge network, you don't need to expose any ports for them to communicate with each other.
Docker Compose is a particularly good tool for organising several containers like this as it automatically configures a network for you
# docker-compose.yaml
version: '3.9'
services:
sqldb:
image: sqldb
applet1:
image: applet1
ports:
- '80:3306' # you sure about this container port?
depends_on:
- sqldb
applet2:
image: applet2
depends_on:
- sqldb
Now only your applet1 container will have a host port mapping. Both applets will be able to connect to any other service within the network on their container ports.
I have a case where I need to call external API from docker container, but can only do that by its URL. In order to do that I'm mapping this URL to container gateway and that's working, but I need it to be dynamic, because I need to run this docker-compose on different devices and from what I see, the gateways are different.
version: '3'
services:
pdf-service:
image: $IMAGE:latest
container_name: pdf-$LOCALE
environment:
- NODE_ENV=production
- LOCALE=${LOCALE}
- API_URL=${API_URL}
extra_hosts:
- ${API_HOST}:172.24.0.1
tty: true
restart: always
ports:
- ${PORT}:8124
At the moment I've hardcoded it as you can see - 172.24.0.1, which is container's gateway. I've found out about something like host.gateway, but have no idea how to use it correctly. Also I've read that it's not working in production? My production environment is Debian 10 with Docker v. 18.09.1 and docker-compose v.1.21.0.
The IP 172.24.0.1 is internal to Docker. When you add an extra host you need to map a name to a public IP.
From your host run ping <api_host> to get the public IP. Use that IP in your docker-compose.yml instead of 172.24.0.1.
If the service you want to reach runs also in Docker, then it must expose a port on the host in order for you to reach it. So then the IP you need is the local network IP of your host (or the public one if for some reason you need to).
If you had to reach an IP that is Docker internal then you wouldn't have to declare an extra_host. You would just put the containers/services on the same network and refer each other by name.
I think you can use network, container can join many network, and if two container in a same network, they can find another by host (service name)
I have created the following docker compose file:
version: '2'
services:
testApp:
image: nginx
hostname: myHost
ports:
- "8080:80"
networks:
- test
networks:
test:
driver: bridge
From outside the container, I can open the web page with localhost:8080. But if I try to open the web page via the defined hostname, it doesn't work.
Can anyone tell me how I can fix this issue?
Is there also a way to connect to the containers ip from the outside?
Thanks in advance.
Other containers on the test network would be able to reference it by that hostname, but not your host machine. You are binding port 8080 on your machine to port 80 on the container, so any external system that you would want to access the website would need to connect to your host machine on 8080 (as you did with localhost:8080).
How you do that depends on your networking, but for example if you know the ip or hostname of your local machine you can (probably) connect from another device on the same home network (your phone? Another computer?) using http://{ip-of-your-host}:8080. Exposing that to the internet from within a home network typically requires port forwarding on your router, and optionally a domain name.
The main point though is that the hostname in your compose is only relevant to other containers connecting to the same docker network (test). Outside of that, systems would need to make a connection to 8080 on your host machine.
In recent versions docker-compose automatically creates a new network for the services it creates. Basically, every docker-compose setup is getting its own IP range, so that in theory I could call my services on the network's IP address with the predefined ports. This is great when developing multiple projects at the same time, since there is then no need to change the ports in docker-compose.yml (i.e. I can run multiple nginx projects at the same time on port 8080 on different interfaces)
However, this does not work as intended: every exposed port is still exposed on 0.0.0.0 and thus there are port conflicts with multiple projects. It is possible to put the bind IP into docker-compose.yml, however this is a killer for portability -- not every developer on the team uses the same OS or works on the same projects, therefore it's not clear which IP to configure.
It's be great to define the IP to bind the containers to in terms of the network created for this particular project. docker-compose should both know which network it created as well as its IP, so this shouldn't be a problem, however I couldn't find an easy way to do it. Is there a way or is this something yet to be implemented?
EDIT: An example of a port conflict: imagine two projects, each with an application server running on port 8080 and a MySQL database running on port 3306, both respectively exposed as "8080:8080" and "3306:3306". Running the first one with docker-compose creates a network called something like app1_network with an IP range of 172.18.0.0/16. Every exposed port is exposed on 0.0.0.0, i.e. on 127.0.0.1, on the WAN address, on the default bridge (172.17.0.0/16) and also on the 172.18.0.0/16. In this case I can reach my application server of all of 127.0.0.1:8080, 172.17.0.1:8080, 172.18.0.1:8080 and als on $WAN_IP:8080. If I start the second application now, it starts a second network app2_network 172.19.0.0/16, but still tries to bind every exposed port on all interfaces. Those ports are of course already taken (except for 172.19.0.1). If there had been a possibility to restrict each application to its network, application 1 would have available at 172.18.0.1:8080 and the second at 172.19.0.1:8080 and I wouldn't need to change port mappings to 8081 and 3307 respectively to run both applications at the same time.
In your service configuration, in docker-compose.yml:
ports:
- "127.0.0.1:8001:8001"
Reference: https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports
You can publish a port to a single IP address on the host by including the IP before the ports:
docker run -p 127.0.0.1:80:80 -d nginx
The above runs nginx on the loopback interface. You can use a similar port mapping inside of a docker-compose.yml file. e.g.:
ports:
- "127.0.0.1:80:80"
docker-compose doesn't have any special abilities to infer which network interface to use based on the docker network. You'd need to specify the unique IP address to use in each compose file, and that IP needs to be for a network interface on your host. For a developer machine, that IP may change as DHCP gives the laptop/workstation new addresses.
Because of the difficulty implementing your goal, most would either map different ports on the host to different containers, so 13307:3307 for container a, 23307:3307 for container b, 33307:3307 for container c, or whatever number scheme makes sense for you. And when dealing with HTTP traffic, then using a reverse proxy like traefik often makes the most sense.
It can be achieved by configuring network in docker-compose file.
Please consider below two docker-compose files. There is still drawback of needing to specify subnet unique across all project you work on at the same time. On the other hand you need to know which service you connecting too - this is why it cannot assign it dynamically.
my-project.yaml:
services:
nginx:
networks:
- my-project-network
image: nginx
ports:
- 80:80
networks:
my-project-network:
driver_opts:
com.docker.network.bridge.host_binding_ipv4: "172.20.0.1"
ipam:
config:
- subnet: "172.20.0.0/16"
my-other-project.yaml
services:
nginx:
networks:
- my-other-project-network
image: nginx
ports:
- 80:80
networks:
my-other-project-network:
driver_opts:
com.docker.network.bridge.host_binding_ipv4: "172.21.0.1"
ipam:
config:
- subnet: "172.21.0.0/16"
Note: that if you have other service binding to *:80 like for instance apache running on host - it will also bind on docker-compose networks' interfaces and you will not be able to use this port.
To run above two services:
docker-compose -f my-project.yaml up -d
docker-compose -f my-other-project.yaml up -d
I've following coder-compose configuration:
version: '2'
services:
nginx:
build: ./nginx
links:
- tomcat1:tomcat1
- tomcat2:tomcat2
- tomcat3:tomcat3
ports:
- "80:80"
tomcat1:
build: ./tomcat
ports:
- "8080"
tomcat2:
build: ./tomcat
ports:
- "8080"
tomcat3:
build: ./tomcat
ports:
- "8080"
So, the question is, how to get access to the host network from the linked container(s):tomcat1, tomcat2, tomcat3. Here is the diagram:
Update
Seems, my diagram doesn't help much. Nginx is a load balancer, Tomcat 1-3 are application nodes. Deployed web. app needs to get access to internet resource.
Internet access is by default active on all containers (in bridge mode). All you need to check is if the http(s)_proxy variables are set if you are behind a proxy.
If your question if how to access docker host from a container (and not the reverse: access a container from the local docker host), then you would need to inspect the routing table of a container: see "From inside of a Docker container, how do I connect to the localhost of the machine?"
export DOCKER_HOST_IP=$(route -n | awk '/UG[ \t]/{print $2}')
There is a recent (June 2016) effort to add a adding a dockerhost as an entry in /etc/hosts of all running containers: issue 23177.
Update March 2020: this issue has been closed, and redirect to PR 40007: "Support host.docker.internal in dockerd on Linux"
This PR allows containers to connect to Linux hosts by appending a special string "host-gateway" to --add-host e.g. "--add-host=host.docker.internal:host-gateway" which adds host.docker.internal DNS entry in /etc/hosts and maps it to host-gateway-ip
This PR also add a daemon flag call host-gateway-ip which defaults to
the default bridge IP
Docker Desktop will need to set this field to the Host Proxy IP so DNS requests for host.docker.internal can be routed to VPNkit
This will be in Docker for Linux (and Docker Desktop, which runs the Linux daemon, although inside a lightweight VM).
Difference between this and the current implementation on Docker Desktop is that;
the current Docker Desktop implementation is in a part of the code-base that's proprietary (i.e., part of how Docker Desktop is configured internally)
this code could be used by the Docker Desktop team in future as well (to be discussed)
this PR does not set up the "magic" host.docker.internal automatically on every container, but it can be used to run a container that needs this host by adding docker run --add-host host.docker.internal:host-gateway
(to be discussed); setting that "magic" domain automatically on containers that are started could be implemented by adding an option for this in the ~/.docker/config.json CLI configuration file.