I am using a docker-compose file and I need to scale specific containers by using the command
docker-compose scale taskcontainer=3
Unfortunately this is not possible due to the fact that I need to expose a specific port from all internal containers and I need a port range on host to make it happen. For example consider the following snippet
taskcontainer:
image: custom-container
ports:
- "9251:9249"
when I scale to 3 containers I need to export port 9249 from internal containers to 3 ports on host e.g. 9251-9253 (to avoid port clash on host) and I need that functionality dynamically because I do not know in advance how many containers will be used for scaling.
Is there a way to achieve that functionality?
Related
One thing I am trying to accomplish is that I have a single port exposed from the container(8001).I want to map this container port with a host port. But I want make this host port as randomly selected port from a given port range(8081-8089). Below is the syntax
ports:
- "8081-8089:8001"
If I use docker-compose v1.29.2 , its working as expected(Selecting single random port within given range). But when I try to use docker-compose v2 its not mapping a single port instead its running container on all ports of given range(8081-8089).
I haven't been able to find a way to do this in the Docker documentation. Do we have any flag which enables this functionality in docker-compose v2? or is it not supported in docker compose v2?
There's no way to do this in Linux in general. You can ask the OS to use a specific port or you can ask the OS to pick a port for you, but you can't constrain the automatically-chosen port.
In a Compose context you can omit the host port to let the OS choose the port
ports:
- "8001" # container port only
and you will need to use docker-compose port to find out what the port number actually is. That will be any free port number, and there is no way to limit it to a specific range.
From the Docker documentation:
--publish or -p flag. Publish a container's port(s) to the host.
--expose. Expose a port or a range of ports.
--link. Add link to another container. Is a legacy feature of Docker. It may eventually be removed.
I am using docker-compose with several networks. I do not want to publish any ports to the host, yet when I use expose, the port is then exposed to all the networks that container is connected to. It seems that after a lot of testing and reading I cannot figure out how to limit this to a specific network.
For example in this docker-compose file with where container1 joins the following three networks: internet, email and database.
services:
container1:
networks:
- internet
- email
- database
Now what if I have one specific port that I want to expose to ONLY the database network, so NOT to the host machine and also NOT to the email and internet networks in this example? If I would use ports: on container1 it is exposed to the host or I can bind it to a specific IP address of the host. *I also tried making a custom overlay network, giving the container a static IPv4 address and trying to set the ports in that format in ports: like - '10.8.0.3:80:80', but that also did not work because I think the binding can only happen to a HOST IP address. If i use expose: on container1 the port will be exposed to all three networks: internet, email and database.
I am aware I can make custom firewall ruling but it annoys me that I cannot write such simple config in my docker-compose file. Also, maybe something like 80:10.8.0.3:80 (HOST_IP:HOST_PORT:CONTAINER_IP:CONTAINER_PORT) would make perfect sense here (did not test it).*
Am I missing something or is this really not possible in Docker and Docker-compose?
Also posted here: https://github.com/docker/compose/issues/8795
No, container to container networking in docker is one-size-fits-many. When two containers are on the same network, and ICC has not been disabled, container-to-container communication is unrestricted. Given Docker's push into the developer workflow, I don't expect much development effort to change this.
This is handled by other projects like Kubernetes by offloading the networking to a CNI where various vendors support networking policies. This may be iptables rules, eBPF code, some kind of sidecar proxy, etc to implement it. But it has to be done as the container networking is setup, and docker doesn't have the hooks for you to implement anything there.
Perhaps you could hook into docker events and run various iptables commands for containers after they've been created. The application could also be configured to listen on the specific IP address for the network it trusts, but this requires injecting the subnet you trust and then looking up your container IP in your entrypoint, non-trivial to script up, and I'm not even sure it would work. Otherwise, this is solved by either restructuring the application so components that need to be on a less secure network are minimized, by hardening the sensitive ports, or switching the runtime over to something like Kubernetes with a network policy.
Things that won't help:
Removing exposed ports: this won't help since expose is just documentation. Changing exposed ports doesn't change networking between containers, or between the container and host.
Links: links are a legacy feature that adds entries to the host file when the container is created. This was replaced by creating networks with DNS resolution of other containers.
Removing published ports on the host: This doesn't impact container to container communication. The published port with -p creates a port forward from the host to the container, which you do want to limit, but containers can still communicate over a shared network without that published port.
The answer to this for me was to remove the -p command as that binds the container to the host and makes it available outside the host.
If you don't specify -p options. The container is available on all the networks it is connected to. On whichever port or ports the application is listening on.
It seems the -P forces the container on to the host and binds it to the port specified.
In your example if you don't use -p when staring "container1". "container1" would be available to the networks: internet, email, database with all its ports but not outside the host.
Do I have to expose the port if I am using the ports config?
In the docker-compose.yml below, do I have to keep expose 2022 or can I remove it? Is there a difference between them?
myproject-app:
build: ../myproject-app
container_name: myproject-app
image: myproject-app
expose:
- 2022
ports:
- 2022:2022
volumes:
- ../myproject-app/:/home/myproject/myproject-app/
- /home/myproject/myproject-app/node_modules
Exposed ports needed for another service, if you want to connect two services inside docker, so service with exposed port will be available inside docker net.
Ports property (ports:) make service port available on your host machine, so you can connect it with your OS net.
So you can delete expose property.
"Expose" means basically nothing in modern Docker. There is pretty much no reason to put an expose: line in your docker-compose.yml file. It's considered polite to include an EXPOSE line in your Dockerfile to document what port(s) your service will listen on, but it's not strictly necessary.
In modern Docker, with named networks, any container can connect to any port of any other container on the same network, provided a process is listening there. Before there were named networks, one container had to explicitly "link" to another to be able to call it, and then it could only reach the exposed ports of the target container. This setup is considered obsolete now (you never need links: either).
Plain Docker has an option (docker run -P, with a capital P) to publish all exposed ports on random host ports. Compose doesn't have an equivalent option. Ports that are exposed but not published also show up in the docker ps output. But those are really the only things "expose" does at all.
If you tell docker-compose to scale a service, and do NOT expose its ports,
docker-compose scale dataservice=2
There will be two IPs in the network that the dns name dataservice will resolve to. So, services that reach it by hostname will load balance.
I would also like to do this to the edge proxy as well. The point would be that
docker-compose scale edgeproxy=2
Would cause edgeproxy to resolve to one of 2 possible IP Addresses.
But the semantics of exposing ports is wrong for this. If I expose:
8443:8443
Then it will try to bind each edgeproxy to be bound to host 8443. What I want is more like:
0.0.0.0:8443:edgeproxy:8443
Where when you try to come into the docker network via host 8443, it randomly selects an edgeproxy:8443 IP to bind the incoming TCP connection to.
Is there an alternative to just do a port-forward? I want a port that can get me in to talk to any ip that will resolve as edgeproxy.
This is provided by swarm mode. You can enable a single node swarm cluster with:
docker swarm init
And then deploy your compose file as a stack with:
docker stack deploy -c docker-compose.yml $stack_name
There are quite a few differences from docker compose including:
Swarm doesn't build images
You manage the target state with docker service commands, trying to stop a container with docker stop won't work since swarm will restart it
The compose file needs to be in a v3 syntax
Networks will be an overlay network, and not attachable by containers outside of swarm, by default
One of the main changes is that exposed ports are published on an ingress network managed by swarm mode, and connections are round robin load balanced to your containers. You can also define a replica count inside the compose file, eliminating the need to run a scale command.
See more at: https://docs.docker.com/engine/swarm/
What is the difference between ports and expose options in docker-compose.yml?
According to the docker-compose reference,
Ports is defined as:
Expose ports. Either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen).
Ports mentioned in docker-compose.yml will be shared among different services started by the docker-compose.
Ports will be exposed to the host machine to a random port or a given port.
My docker-compose.yml looks like:
mysql:
image: mysql:5.7
ports:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
-------------------------------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 0.0.0.0:32769->3306/tcp
Expose is defined as:
Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.
Ports are not exposed to host machines, only exposed to other services.
mysql:
image: mysql:5.7
expose:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
---------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 3306/tcp
Edit
In recent versions of Dockerfile, EXPOSE doesn't have any operational impact anymore, it is just informative. (see also)
ports:
Activates the container to listen for specified port(s) from the world outside of the docker(can be same host machine or a different machine) AND also accessible world inside docker.
More than one port can be specified (that's is why ports not port)
expose:
Activates container to listen for a specific port only from the world inside of docker AND not accessible world outside of the docker.
More than one port can be specified
Ports
This section is used to define the mapping between the host server and Docker container.
ports:
- 10005:80
It means the application running inside the container is exposed at port 80. But external system/entity cannot access it, so it need to be mapped to host server port.
Note: you have to open the host port 10005 and modify firewall rules to allow external entities to access the application.
They can use
http://{host IP}:10005
something like this
EXPOSE
This is exclusively used to define the port on which application is running inside the docker container.
You can define it in dockerfile as well. Generally, it is good and widely used practice to define EXPOSE inside dockerfile because very rarely anyone run them on other port than default 80 port
Ports
The ports section will publish ports on the host. Docker will set up a forward for a specific port from the host network into the container. By default, this is implemented with a userspace proxy process (docker-proxy) that listens on the first port, and forwards into the container, which needs to listen on the second point. If the container is not listening on the destination port, you will still see something listening on the host, but get a connection refused if you try to connect to that host port, from the failed forward into your container.
Note, the container must be listening on all network interfaces since this proxy is not running within the container's network namespace and cannot reach 127.0.0.1 inside the container. The IPv4 method for that is to configure your application to listen on 0.0.0.0.
Also note that published ports do not work in the opposite direction. You cannot connect to a service on the host from the container by publishing a port. Instead you'll find docker errors trying to listen to the already-in-use host port.
Expose
Expose is documentation. It sets metadata on the image, and when running, on the container too. Typically, you configure this in the Dockerfile with the EXPOSE instruction, and it serves as documentation for the users running your image, for them to know on which ports by default your application will be listening. When configured with a compose file, this metadata is only set on the container. You can see the exposed ports when you run a docker inspect on the image or container.
There are a few tools that rely on exposed ports. In docker, the -P flag will publish all exposed ports onto ephemeral ports on the host. There are also various reverse proxies that will default to using an exposed port when sending traffic to your application if you do not explicitly set the container port.
Other than those external tools, expose has no impact at all on the networking between containers. You only need a common docker network, and connecting to the container port, to access one container from another. If that network is user created (e.g. not the default bridge network named bridge), you can use DNS to connect to the other containers.
I totally agree with the answers before.
I just like to mention that the difference between expose and ports is part of the security concept in docker. It goes hand in hand with the networking of docker.
For example:
Imagine an application with a web front-end and a database back-end.
The outside world needs access to the web front-end (perhaps on port
80), but only the back-end itself needs access to the database host
and port. Using a user-defined bridge, only the web port needs to be
opened, and the database application doesn’t need any ports open,
since the web front-end can reach it over the user-defined bridge.
This is a common use case when setting up a network architecture in docker.
So for example in a default bridge network, not ports are accessible from the outer world.
Therefor you can open an ingresspoint with "ports". With using "expose" you define communication within the network. If you want to expose the default ports you don't need to define "expose" in your docker-compose file.