I have a case where I need to call external API from docker container, but can only do that by its URL. In order to do that I'm mapping this URL to container gateway and that's working, but I need it to be dynamic, because I need to run this docker-compose on different devices and from what I see, the gateways are different.
version: '3'
services:
pdf-service:
image: $IMAGE:latest
container_name: pdf-$LOCALE
environment:
- NODE_ENV=production
- LOCALE=${LOCALE}
- API_URL=${API_URL}
extra_hosts:
- ${API_HOST}:172.24.0.1
tty: true
restart: always
ports:
- ${PORT}:8124
At the moment I've hardcoded it as you can see - 172.24.0.1, which is container's gateway. I've found out about something like host.gateway, but have no idea how to use it correctly. Also I've read that it's not working in production? My production environment is Debian 10 with Docker v. 18.09.1 and docker-compose v.1.21.0.
The IP 172.24.0.1 is internal to Docker. When you add an extra host you need to map a name to a public IP.
From your host run ping <api_host> to get the public IP. Use that IP in your docker-compose.yml instead of 172.24.0.1.
If the service you want to reach runs also in Docker, then it must expose a port on the host in order for you to reach it. So then the IP you need is the local network IP of your host (or the public one if for some reason you need to).
If you had to reach an IP that is Docker internal then you wouldn't have to declare an extra_host. You would just put the containers/services on the same network and refer each other by name.
I think you can use network, container can join many network, and if two container in a same network, they can find another by host (service name)
Related
I have the basic docker-compose.yml shown below for an apache server. And I was wondering if there was a way to configure this apache server, which is currently accessible using 0.0.0.0:8889 OR localhost:8889, so is it accessible using a custom host name, such as: local.foobar.dev for example?
version: '3'
services:
snappyweb:
image: php:7.0-apache
ports:
- "8889:80"
volumes:
- ./:/var/www/html
I assume you want this to only work for your local dev environment.
The easiest and safest way is to use an application that will essentially trick your local browser into thinking a URL of choice is a certain IP address. That IP could be a locahost:8000.
For this I use GasMask (osX) or your can use Host File Manager (Windows).
If you need to reach your container from your host using a custom name, you will need to do the mapping between local.foobar.dev and the desired IP manually by adding the following line to /etc/hosts
127.0.0.1 local.foobar.dev
For communication between containers, you can create a docker network and add the containers to this network. Then containers can reach each other using container names.
I created two Dockerfiles to run frontend and backend in a web application. When I run docker-compose.yaml file, web application front-end is opened of web browser. But I cannot login to the system. I think there is a problem with connecting those containers. Following is my docker-compose.yaml file. What can I do to resolve this problem ?
version: '2'
services:
gulp:
build: './ui'
ports:
- "3000:4000"
python:
build: '.'
ports:
- "5000:5000"
You need to use --links to enable communication between containers and you should use their DNS network alias like http://python:5000
Containers within a docker-compose file are part of one network by default. And one container can access other container using their host name.
Host name can be defined in docker-compose file using hostname. And if hostname is not defined, then the service name is considered the hostname.
Internally, docker containers can talk to each other by referring to each other at their hostname. Like in your case, gulp can access python at http://python:5000 and that would be possible even if you did not declare ports. This all is happening all because it is internal to the docker system.
From outside, if you want to connect to any of the services, then you can define ports, as you did and then access those services at the defined port number.
In recent versions docker-compose automatically creates a new network for the services it creates. Basically, every docker-compose setup is getting its own IP range, so that in theory I could call my services on the network's IP address with the predefined ports. This is great when developing multiple projects at the same time, since there is then no need to change the ports in docker-compose.yml (i.e. I can run multiple nginx projects at the same time on port 8080 on different interfaces)
However, this does not work as intended: every exposed port is still exposed on 0.0.0.0 and thus there are port conflicts with multiple projects. It is possible to put the bind IP into docker-compose.yml, however this is a killer for portability -- not every developer on the team uses the same OS or works on the same projects, therefore it's not clear which IP to configure.
It's be great to define the IP to bind the containers to in terms of the network created for this particular project. docker-compose should both know which network it created as well as its IP, so this shouldn't be a problem, however I couldn't find an easy way to do it. Is there a way or is this something yet to be implemented?
EDIT: An example of a port conflict: imagine two projects, each with an application server running on port 8080 and a MySQL database running on port 3306, both respectively exposed as "8080:8080" and "3306:3306". Running the first one with docker-compose creates a network called something like app1_network with an IP range of 172.18.0.0/16. Every exposed port is exposed on 0.0.0.0, i.e. on 127.0.0.1, on the WAN address, on the default bridge (172.17.0.0/16) and also on the 172.18.0.0/16. In this case I can reach my application server of all of 127.0.0.1:8080, 172.17.0.1:8080, 172.18.0.1:8080 and als on $WAN_IP:8080. If I start the second application now, it starts a second network app2_network 172.19.0.0/16, but still tries to bind every exposed port on all interfaces. Those ports are of course already taken (except for 172.19.0.1). If there had been a possibility to restrict each application to its network, application 1 would have available at 172.18.0.1:8080 and the second at 172.19.0.1:8080 and I wouldn't need to change port mappings to 8081 and 3307 respectively to run both applications at the same time.
In your service configuration, in docker-compose.yml:
ports:
- "127.0.0.1:8001:8001"
Reference: https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports
You can publish a port to a single IP address on the host by including the IP before the ports:
docker run -p 127.0.0.1:80:80 -d nginx
The above runs nginx on the loopback interface. You can use a similar port mapping inside of a docker-compose.yml file. e.g.:
ports:
- "127.0.0.1:80:80"
docker-compose doesn't have any special abilities to infer which network interface to use based on the docker network. You'd need to specify the unique IP address to use in each compose file, and that IP needs to be for a network interface on your host. For a developer machine, that IP may change as DHCP gives the laptop/workstation new addresses.
Because of the difficulty implementing your goal, most would either map different ports on the host to different containers, so 13307:3307 for container a, 23307:3307 for container b, 33307:3307 for container c, or whatever number scheme makes sense for you. And when dealing with HTTP traffic, then using a reverse proxy like traefik often makes the most sense.
It can be achieved by configuring network in docker-compose file.
Please consider below two docker-compose files. There is still drawback of needing to specify subnet unique across all project you work on at the same time. On the other hand you need to know which service you connecting too - this is why it cannot assign it dynamically.
my-project.yaml:
services:
nginx:
networks:
- my-project-network
image: nginx
ports:
- 80:80
networks:
my-project-network:
driver_opts:
com.docker.network.bridge.host_binding_ipv4: "172.20.0.1"
ipam:
config:
- subnet: "172.20.0.0/16"
my-other-project.yaml
services:
nginx:
networks:
- my-other-project-network
image: nginx
ports:
- 80:80
networks:
my-other-project-network:
driver_opts:
com.docker.network.bridge.host_binding_ipv4: "172.21.0.1"
ipam:
config:
- subnet: "172.21.0.0/16"
Note: that if you have other service binding to *:80 like for instance apache running on host - it will also bind on docker-compose networks' interfaces and you will not be able to use this port.
To run above two services:
docker-compose -f my-project.yaml up -d
docker-compose -f my-other-project.yaml up -d
In docker-compose legacy yml if you link a service it used to create an environment variable servicename_PORT which you could use to discover the port of a linked container. In the new v2 format we have user defined networks which add the service name to the internal DNS and so we can connect to linked services, but how do we find the port a linked service exposes? The only way I can think of is to create an environment variable for each linked service where I can put the port, but then I will have the same port twice in the docker-compose: once in the expose section of the service itself and once as an environment variable in the service that connects to it. Is there a more DRY way of discovering the exposed port?
For this, you usually use a registrator + service-discover, this means, a service like https://www.consul.io / registrator
Basically, this adds an API for you to either watch a kv store for you service defintions ( port / ip ) which then can be random, or even use DNS included in consul. The latter wont help with ports, thats what you use a registry for.
If you want to dodge this best-practice way. mount the docker socket and use docker inspect <servicename> to find the port.
services:
other:
container_name: foo
image: YYYY
theonedoingthelookup:
image: ZZZZ
volumes:
- /var/run/docker.sock:/var/run/docker.sock
You will need to have the docker cli tool installed in the container, then run this inside the ZZZZ container
docker inspect YYYY
Use some grep / awk / filters to extract the information you need
It's whatever port the service is running as in the container. Port mappings don't apply to container <-> container communication, only host <-> container communication.
For example:
version: '2'
services:
a:
...
networks:
- my-net
b:
...
networks:
- my-net
networks:
my-net:
Let's say a is running a webserver at port 8080, b would be able to hit it by sending a request to a:8080.
Currently in the company where I am working on they have a central development server which contains a LAMP environment. Each developer has access to the application as: developer_username.domain.com. The application we're working on uses licenses and the licenses are generated under each domain and are tied to the domain only meaning I can't use license from other developer.
The following example will give you an idea:
developer_1.domain.com ==> license1
developer_2.domain.com ==> license2
developer_n.domain.com ==> licenseN
I am trying to dockerize this enviroment at least having PHP and Apache in a container and I was able to create everything I need and it works. Take a look to this docker-compose.yml:
version: '2'
services:
php-apache:
env_file:
- dev_variables.env
image: reypm/php55-dev
build:
context: .
args:
- PUID=1000
- PGID=1000
ports:
- "80:80"
- "9001:9001"
extra_hosts:
- "dockerhost:xxx.xxx.xxx.xxx"
volumes:
- ~/var/www:/var/www
That will build what I need but the problem comes when I try to access the server because I am using http://localhost and then the license won't work and I won't be able to use the application.
The idea is to access as developer_username.domain.com, so my question is: is this a work that should be done on the Dockerfile or the Docker Compose I mean at image/container level let's say by setting up a ENV var perhaps or is this a job for /etc/hosts on the host running the Docker?
tl;dr
No! Docker doesn't do that for you.
Long answer:
What you want to do is to have a custom hostname on the machine hosting docker mapped to a container in Docker compose network. right?
Let's take a step back and see how networking in docker works:
By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.
This network is not equal to your host network and without explicit ports exporting (for a specific container) you wouldn't have access to this network. All exposing does, is that:
The exposed port is accessible on the host and the ports are available to any client that can reach the host.
From now on you can put a reverse proxy (like nginx) or you can edit /etc/hosts to define how clients can access the host (i.e. Docker host, the machine running Docker compose).
The hostname is defined when you start the container, overwriting anything you attempt to put inside the image. At a high level, I'd recommend doing this with a mix of custom docker-compose.yml and a volume per developer, but each running an identical image. The docker-compose.yml can include the hostname and domain setting. Then everything else that needs to be hostname specific on the filesystem, and the license itself, should point to files on the volume. Lastly, include an entrypoint that does the right thing if a new hostname is started with a default or empty volume, populating it with the new hostname data and prompting for the license.