In Docker we all know how to expose ports, EXPOSE instruction publishes and -p or -P option to expose during runtime. When we use "docker inspect" or "docker port" to see the port mappings and these configs output are pulled /var/lib/docker/containers/container-id/config.v2.json.
The question I got is when we expose port how does Docker actually changes the port in container, say the Apache or Nginx, say we can have the installation anywhere in the OS or file path, how does Docker finds the correct conf file(Apache /etc/httpd/conf/httpd.conf) to change if I suppose Docker does this on the line "Listen 80" or Listen "443" in the httpd.conf file. Or my whole understanding of Docker is in stake:)
Any help is appreciated.
"docker" does not change anything in the internal configuation of the container (or the services it provides).
There are three different points where you can configure ports
the service itself (for instance nginx) inside the image/container
EXPOSE xxxx in the Dockerfile (ie at build time of the image)
docker run -p 80:80 (or the respective equivalent for docker compose) (ie at the runtime of the container)
All three are (in principle) independent of each other. Ie, you can have completely different values in each of them. But in practice, you will have to adjust them to each other to get a working system.
We know, EXPOSE xxxx in the dockerfile doesn't actually publish any port at runtime, but just tells the docker service, that that specific container will listen to port xxxx at runtime. You can see this as sort of documentation for that image. So it's your responsibility as creator of the Dockerfile to provide the correct value here. Because anyone using that image, will probaby rely on that value.
But regardless, of what port you have EXPOSEd (or not, EXPOSE is completely optional) you still have to publish that port when you run the container (for instance when using docker run via -p aaaa:xxxx).
Now let us assume you have an nginx image which has the nginx service configured to listen to port 8000. Regardless of what you define with EXPOSE or -p aaaa:xxxx, that nginx service will always listen to port 8000 only and nothing else.
So if you now run your container with docker run -p 80:80, the runtime will bind port 80 of the host to port 80 of the container. But as there is no service listening on port 80 within the container, you simply won't be able to contact your nginx service on port 80. And you also won't be able to connect to nginx on port 8000, because it hasn't been published.
So in a typical setup, if your service in the container is configured to listen to port 8000, you should also EXPOSE 8000 in your dockerfile and use docker run -p aaaa:8000 to bind port aaaa of your host machine to port 8000 of your container, so that you will be able to connect to the nginx service via http://hostmachine:aaaa
Related
I'm experimenting with Dockerfiles, and I think I understand most of the logic. However, I don't see the difference between "exposing" and "publishing" a port in this context.
All the tutorials I have seen first include the EXPOSE command in the Dockerfile:
...
EXPOSE 8080
...
They then build an image from this Dockerfile:
$ docker build -t an_image - < Dockerfile
And then publish the same port as above when running the image:
$ docker run -d -p 8080 an_image
or publish all ports using
$ docker run -d -P an_image
What is the point of exposing a port in the Dockerfile, if it will be published anyway? Would there ever be a need to expose a port first, and not publish it later? Effectively, I would like to specify all the ports that I will use in the Dockerfile when creating the image, and then not bother with them again, running them simply with:
$ docker run -d an_image
Is this possible?
Basically, you have three (four) options:
Neither specify EXPOSE nor -p
Only specify EXPOSE
Specify EXPOSE and -p
Only specify -p which implicitly does EXPOSE
If you specify neither EXPOSE nor -p, the service in the container will only be accessible from inside the container itself.
If you EXPOSE a port, the service in the container is not accessible from outside Docker, but from inside other Docker containers. So this is good for inter-container communication.
If you EXPOSE and -p a port, the service in the container is accessible from anywhere, even outside Docker.
If you do -p, but do not EXPOSE, Docker does an implicit EXPOSE. This is because if a port is open to the public, it is automatically also open to other Docker containers. Hence -p includes EXPOSE. This is effectively same as 3).
The reason why both are separated is IMHO because:
choosing a host port depends on the host and hence does not belong to the Dockerfile (otherwise it would be depending on the host),
and often it's enough if a service in a container is accessible from other containers.
The documentation explicitly states:
The EXPOSE instruction exposes ports for use within links.
It also points you to how to link containers, which basically is the inter-container communication I talked about.
Short answer:
EXPOSE is a way of documenting
--publish (or -p) is a way of mapping a host port to a running container port
Notice below that:
EXPOSE is related to Dockerfiles ( documenting )
--publish is related to docker run ... ( execution / run-time )
Exposing and publishing ports
In Docker networking, there are two different mechanisms that directly involve network ports: exposing and publishing ports. This applies to the default bridge network and user-defined bridge networks.
You expose ports using the EXPOSE keyword in the Dockerfile or the --expose flag to docker run. Exposing ports is a way of documenting which ports are used, but does not actually map or open any ports. Exposing ports is optional.
You publish ports using the --publish or --publish-all flag to docker run. This tells Docker which ports to open on the container’s network interface. When a port is published, it is mapped to an available high-order port (higher than 30000) on the host machine, unless you specify the port to map to on the host machine at runtime. You cannot specify the port to map to on the host machine when you build the image (in the Dockerfile), because there is no way to guarantee that the port will be available on the host machine where you run the image.
from: Docker container networking
Update October 2019: the above piece of text is no longer in the docs but an archived version is here: docs.docker.com/v17.09/engine/userguide/networking/#exposing-and-publishing-ports
Maybe the current documentation is the below:
Published ports
By default, when you create a container, it does not publish any of its ports to the outside world. To make a port available to services outside of Docker, or to Docker containers which are not connected to the container's network, use the --publish or -p flag. This creates a firewall rule which maps a container port to a port on the Docker host.
and can be found here: docs.docker.com/config/containers/container-networking/#published-ports
Also,
EXPOSE
...The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published.
from: Dockerfile reference
Service access when EXPOSE / --publish are not defined:
At #Golo Roden's answer it is stated that::
"If you do not specify any of those, the service in the container will not be accessible from anywhere except from inside the container itself."
Maybe that was the case at the time the answer was being written, but now it seems that even if you do not use EXPOSE or --publish, the host and other containers of the same network will be able to access a service you may start inside that container.
How to test this:
I've used the following Dockerfile. Basically, I start with ubuntu and install a tiny web-server:
FROM ubuntu
RUN apt-get update && apt-get install -y mini-httpd
I build the image as "testexpose" and run a new container with:
docker run --rm -it testexpose bash
Inside the container, I launch a few instances of mini-httpd:
root#fb8f7dd1322d:/# mini_httpd -p 80
root#fb8f7dd1322d:/# mini_httpd -p 8080
root#fb8f7dd1322d:/# mini_httpd -p 8090
I am then able to use curl from the host or other containers to fetch the home page of mini-httpd.
Further reading
Very detailed articles on the subject by Ivan Pepelnjak:
Exposed ports
Published ports
See the official documentation reference: https://docs.docker.com/engine/reference/builder/#expose
The EXPOSE allows you to define private (container) and public (host) ports to expose at image build time for when the container is running if you run the container with -P.
$ docker help run
...
-P, --publish-all Publish all exposed ports to random ports
...
The public port and protocol are optional, if not a public port is specified, a random port will be selected on host by docker to expose the specified container port on Dockerfile.
A good pratice is do not specify public port, because it limits only one container per host ( a second container will throw a port already in use ).
You can use -p in docker run to control what public port the exposed container ports will be connectable.
Anyway, If you do not use EXPOSE (with -P on docker run) nor -p, no ports will be exposed.
If you always use -p at docker run you do not need EXPOSE but if you use EXPOSE your docker run command may be more simple, EXPOSE can be useful if you don't care what port will be exposed on host, or if you are sure of only one container will be loaded.
You expose ports using the EXPOSE keyword in the Dockerfile or the
--expose flag to docker run. Exposing ports is a way of documenting which
ports are used, but does not actually map or open any ports. Exposing ports
is optional.
Source: github commit
Most people use docker compose with networks. The documentation states:
The Docker network feature supports creating networks without the need to expose ports within the network, for detailed information see the overview of this feature).
Which means that if you use networks for communication between containers you don't need to worry about exposing ports.
EXPOSE keyword lets owner to inform others that which ports are going to be used by the container mainly.
You can publish any port even if you don't specify the port in EXPOSE.
For example we create a Dockerfile with nginx image that exposes port 1234
FROM nginx:latest
EXPOSE 1234
and build it
docker build -t porttest .
And run it with publishing 80 port to localhost:80
docker run -p 80:80 porttest
When you go localhost:80, you will see nginx default page.
Nginx default page
expose - will only allow that specific port to connect with container and it will use as "inter container communication " only
-p ( publish ) will map the host port with container port ( which you have already exposed in step 1 or in docker file ) and once you expose it , you will need to map it so we use publish , it will then access container outside world/internet.
EXPOSE is used to map local port container port
ie : if you specify expose in docker file like
EXPOSE 8090
What will does it will map localhost port 8090 to container port 8090
The Dockerfile command EXPOSE and the docker run argument --expose tells docker that the port must be exposed.
When publishing ports with -p, you can map an outer host port to a different inner container port, e.g. docker run -p 8080:80, where 8080 is the host port and 80 is the container port.
My question is, does EXPOSE refer to the inner container port or the outer host port?
The EXPOSE instruction in a Dockerfile refers to the container port.
The EXPOSE instruction documents the port on which an application inside the container is expected to be listening. The important word there is "documents". It does not change the behavior of docker running your container, it does not publish the port, and does not impact the ability to connect between containers.
Whether or not you expose the port, you need to separately publish the port to access it from outside the container network. And whether or not you expose the port, you can connect between containers on the same docker network.
There are various tools that can use this image metadata to automatically discover your application. This includes the -P flag to publish all container ports on random high numbered host ports. You will also see reverse proxies (like traefik) use this when querying the docker engine to determine the default port to use for your container.
EXPOSE just means those exposed ports of this current container are available/exposed to all containers that are in the same network.
According to the documentation, the EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. So it refers to the inner container.
EXPOSE allow communication between the container and other containers in the same network. But it does not allow communication with the host machine, or containers in another network! In order to permit that, you need to publish the port, with -p option.
I'm confused with the Docker doc's EXPOSE vs -p. Here's what I understand: EXPOSE serves for inter-container communication only and cannot be accessed from the outside.
But the developer guide writes "The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published."
Does it mean that if I have EXPOSE as 8001 and -p as 8002, the -p flag will overwrite EXPOSE and the container will be published on 8002? I guess I'm confused by the word "publish".
The developer documentation you quoted is accurate, the EXPOSE entry in the Dockerfile is documentation from the developer creating the image to the users running the image.
If you use -P then docker will publish all exposed ports on the host (note that uppercase P, different from the lowercase option).
For container to container communication, you only need to be in the same docker network. Containers talk on the port the application is listening on directly, no exposing of ports or publishing to the host needed.
Publishing the port is done at runtime with the -p option and maps a port from the host to one in the container to make it available outside of docker. Exposing ports has no impact on publishing with -p, it's just documentation, and metadata in the image.
EXPOSE means that your container will be accessible inside docker network by this port
-p flag means all above but for your outside communication
For example:
EXPOSE 8001 - exposing inside docker network
-p 8002:8001 - making port from 8001 inside to 8002 outside
I am new to docker, using https://github.com/mattrayner/docker-lamp
I've read about the docker run command but still not quite getting the -p option. Is there a way to make it tell Apache to listen on a non-standard port?
I have succeeded in starting it on the default port 80, then re-configuring/re-loading Apache, from within the container, to bind itself to port 8080. But in that scenario I can't access the container's Apache from outside it via localhost:8080. (If that makes sense.)
I simply want to develop something using PHP 5.6 without disturbing anything else on my local setup, which is running PHP 7.0. If there's another way to achieve the same end, I'm good with that too.
The -p or --publish option is a host:container port mapping specifically so that you don't have to change what may already be running inside the container.
If the container is already running on port 80 but you want to access it externally (via your host or laptop) via port 8080, then you can simple run with -p 8080:80 which will map your host port 8080 to the container port 80.
Multiple containers can run and use port 80 on the same host (since the containers have their own IP address on the Docker network). But you can only expose one port at a time.
For example, if you had 3 containers you wanted to run and all of them were listening on port 80, you could start the first with -p 8080:80, the second with -p 8082:80, and the third with -p 8084:80.
The -p section of https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p---expose does into this a bit deeper.
I'm experimenting with Dockerfiles, and I think I understand most of the logic. However, I don't see the difference between "exposing" and "publishing" a port in this context.
All the tutorials I have seen first include the EXPOSE command in the Dockerfile:
...
EXPOSE 8080
...
They then build an image from this Dockerfile:
$ docker build -t an_image - < Dockerfile
And then publish the same port as above when running the image:
$ docker run -d -p 8080 an_image
or publish all ports using
$ docker run -d -P an_image
What is the point of exposing a port in the Dockerfile, if it will be published anyway? Would there ever be a need to expose a port first, and not publish it later? Effectively, I would like to specify all the ports that I will use in the Dockerfile when creating the image, and then not bother with them again, running them simply with:
$ docker run -d an_image
Is this possible?
Basically, you have three (four) options:
Neither specify EXPOSE nor -p
Only specify EXPOSE
Specify EXPOSE and -p
Only specify -p which implicitly does EXPOSE
If you specify neither EXPOSE nor -p, the service in the container will only be accessible from inside the container itself.
If you EXPOSE a port, the service in the container is not accessible from outside Docker, but from inside other Docker containers. So this is good for inter-container communication.
If you EXPOSE and -p a port, the service in the container is accessible from anywhere, even outside Docker.
If you do -p, but do not EXPOSE, Docker does an implicit EXPOSE. This is because if a port is open to the public, it is automatically also open to other Docker containers. Hence -p includes EXPOSE. This is effectively same as 3).
The reason why both are separated is IMHO because:
choosing a host port depends on the host and hence does not belong to the Dockerfile (otherwise it would be depending on the host),
and often it's enough if a service in a container is accessible from other containers.
The documentation explicitly states:
The EXPOSE instruction exposes ports for use within links.
It also points you to how to link containers, which basically is the inter-container communication I talked about.
Short answer:
EXPOSE is a way of documenting
--publish (or -p) is a way of mapping a host port to a running container port
Notice below that:
EXPOSE is related to Dockerfiles ( documenting )
--publish is related to docker run ... ( execution / run-time )
Exposing and publishing ports
In Docker networking, there are two different mechanisms that directly involve network ports: exposing and publishing ports. This applies to the default bridge network and user-defined bridge networks.
You expose ports using the EXPOSE keyword in the Dockerfile or the --expose flag to docker run. Exposing ports is a way of documenting which ports are used, but does not actually map or open any ports. Exposing ports is optional.
You publish ports using the --publish or --publish-all flag to docker run. This tells Docker which ports to open on the container’s network interface. When a port is published, it is mapped to an available high-order port (higher than 30000) on the host machine, unless you specify the port to map to on the host machine at runtime. You cannot specify the port to map to on the host machine when you build the image (in the Dockerfile), because there is no way to guarantee that the port will be available on the host machine where you run the image.
from: Docker container networking
Update October 2019: the above piece of text is no longer in the docs but an archived version is here: docs.docker.com/v17.09/engine/userguide/networking/#exposing-and-publishing-ports
Maybe the current documentation is the below:
Published ports
By default, when you create a container, it does not publish any of its ports to the outside world. To make a port available to services outside of Docker, or to Docker containers which are not connected to the container's network, use the --publish or -p flag. This creates a firewall rule which maps a container port to a port on the Docker host.
and can be found here: docs.docker.com/config/containers/container-networking/#published-ports
Also,
EXPOSE
...The EXPOSE instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published.
from: Dockerfile reference
Service access when EXPOSE / --publish are not defined:
At #Golo Roden's answer it is stated that::
"If you do not specify any of those, the service in the container will not be accessible from anywhere except from inside the container itself."
Maybe that was the case at the time the answer was being written, but now it seems that even if you do not use EXPOSE or --publish, the host and other containers of the same network will be able to access a service you may start inside that container.
How to test this:
I've used the following Dockerfile. Basically, I start with ubuntu and install a tiny web-server:
FROM ubuntu
RUN apt-get update && apt-get install -y mini-httpd
I build the image as "testexpose" and run a new container with:
docker run --rm -it testexpose bash
Inside the container, I launch a few instances of mini-httpd:
root#fb8f7dd1322d:/# mini_httpd -p 80
root#fb8f7dd1322d:/# mini_httpd -p 8080
root#fb8f7dd1322d:/# mini_httpd -p 8090
I am then able to use curl from the host or other containers to fetch the home page of mini-httpd.
Further reading
Very detailed articles on the subject by Ivan Pepelnjak:
Exposed ports
Published ports
See the official documentation reference: https://docs.docker.com/engine/reference/builder/#expose
The EXPOSE allows you to define private (container) and public (host) ports to expose at image build time for when the container is running if you run the container with -P.
$ docker help run
...
-P, --publish-all Publish all exposed ports to random ports
...
The public port and protocol are optional, if not a public port is specified, a random port will be selected on host by docker to expose the specified container port on Dockerfile.
A good pratice is do not specify public port, because it limits only one container per host ( a second container will throw a port already in use ).
You can use -p in docker run to control what public port the exposed container ports will be connectable.
Anyway, If you do not use EXPOSE (with -P on docker run) nor -p, no ports will be exposed.
If you always use -p at docker run you do not need EXPOSE but if you use EXPOSE your docker run command may be more simple, EXPOSE can be useful if you don't care what port will be exposed on host, or if you are sure of only one container will be loaded.
You expose ports using the EXPOSE keyword in the Dockerfile or the
--expose flag to docker run. Exposing ports is a way of documenting which
ports are used, but does not actually map or open any ports. Exposing ports
is optional.
Source: github commit
Most people use docker compose with networks. The documentation states:
The Docker network feature supports creating networks without the need to expose ports within the network, for detailed information see the overview of this feature).
Which means that if you use networks for communication between containers you don't need to worry about exposing ports.
EXPOSE keyword lets owner to inform others that which ports are going to be used by the container mainly.
You can publish any port even if you don't specify the port in EXPOSE.
For example we create a Dockerfile with nginx image that exposes port 1234
FROM nginx:latest
EXPOSE 1234
and build it
docker build -t porttest .
And run it with publishing 80 port to localhost:80
docker run -p 80:80 porttest
When you go localhost:80, you will see nginx default page.
Nginx default page
expose - will only allow that specific port to connect with container and it will use as "inter container communication " only
-p ( publish ) will map the host port with container port ( which you have already exposed in step 1 or in docker file ) and once you expose it , you will need to map it so we use publish , it will then access container outside world/internet.
EXPOSE is used to map local port container port
ie : if you specify expose in docker file like
EXPOSE 8090
What will does it will map localhost port 8090 to container port 8090