I have a dockerfile of a ruby app that runs a puma server on port 5000. (It EXPOSE the port and also uses the RUN command to run puma -p 5000.
In my deployment.yaml I have to set containerPort to be 5000 to match this port.
This seems odd to me that my configuration lists the port in 2 different places. If I need to change the port it means I am changing the configuration in multiple places which is against the principles of the 12 factor app where configs are all in the same place.
Is there a way to set the port in only 1 place?
In your deployment.yaml you actually don't have to specify the containerPort; all ports are exposed. From the docs:
ports ContainerPort array
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network. Cannot be updated.
Related
I'm learning Kurbernetes and Docker at the moment, KinD in particular. To start with, I just want to run docker run --rm --name <container's name> -p 8080:80 -d <image name> to create a container from the image.
I know that ports are used in the TCP/IP protocol (or Internet Protocol) to address a specific program (software). Port 80 is a default port to run web servers.
Now, my question is why 8080 or why 5000? How to determine which port should be an OUTSIDE port in this case? Is it just random or are there any rule/restrictions?
For the docker run -p option (and Compose ports:), for the first port number you can pick any port that isn't already in use on your host system. As you've noted, port 80 is the standard HTTP port, and what gets used in http://hostname/ URLs without an explicit port number. Various frameworks use port 3000 or 5000 or 8000 or 8080 as their default but none of them is "standard" or "special" in any way.
The second port number must be the port number the server process is listening on. The server process must be listening on the special 0.0.0.0 "all address" address, if that's a configurable option; if it's listening on 127.0.0.1 (as many developer-oriented servers do by default) it will not be reachable from outside its container. This number often is included in an EXPOSE line in the Dockerfile, but that directive has no other effect. (There is no reason to include a docker run --expose option or Compose expose: block and it's always safe to delete it.)
There is no particular requirement that the two ports match. If you want to use host port 8888 because it's available, and your application is a Node application using the default Express port 3000, it will work to
docker run -p 8888:3000 ...
If you really don't care you can use docker run -p with only the container port number, but this is unusual. docker port will tell you what port Docker chose.
docker run -p 3000 --name my-container ...
docker port my-container 3000
You mention Kubernetes in here as well. In Kubernetes, all communications between Pods go through a Service, in effect an in-cluster load balancer. I'd recommend always making the Service use the "normal" port for whatever protocol you're using, port 80 for unencrypted HTTP. Each Service has its own in-cluster IP address so there's no risk of conflict between Services or Pods. If you're using a NodePort-type Service to make it accessible from outside the cluster, you are usually constrained to using ports 30000 through 32767.
apiVersion: v1
kind: Service
metadata: { name: the-service-name }
spec:
selector: { ... }
ports:
- port: 80 # for HTTP, regardless of how the service is implemented
targetPort: http # matching the Pod's `containerPorts:` name
# nodePort: 30080 # if the Service has type: NodePort, optional
Now calls from another Pod through this Service can use http://the-service-name/ as the URL with the default port.
port 8080 is normally used to host your personal webserver/service and it is alternate of port 80
you can also use other ports instead of 8080.
when some one try to connect to your webserver from outside, and if you use port 8080, they don't need to specify port number because by default it will look for port 8080.
If you use any other port number, when someone try to connect to your webservice/server from outside, they should specify the custom port number you specified to access your webservice/server
As David very nicely explained, you can use any of the 65535 ports your OS is not already using. However there are two additional limits:
Usually ports up to 1023 are not accessible from user space. As the docker daemon runs on root privileges you would not have to care, yet it is wise practice to go above.
Then every operating system has a range of ephemeral ports. These are ports that will be used by the operating system for outgoing TCP connections (yes, they also need a port). This range varies across operating systems. If you want to be sure you are able to start a container on some port (that you think is available), ensure the OS will not start using it based on whatever other process needs a connection. Thus better choose a port outside the ephemeral port range.
In short, on Linux systems you want to pick a value between 1024 and 32767.
I have a container exposing a web app through the 3000 port and another one witch access it by docker dns.
I want to access this container using the 80 port without modifying the web app and without direct exposing it to the host (aka --publish). Basically internally forward the 80 port to the 3000 port.
Is it possible to do it using docker without modifying the container to have socat or something?
No, Docker doesn’t have this capability. The only port remapping is when a port is published outside of Docker space using the docker run -p option, and this never affects inter-service communication. Your only options here are to change the server configuration to listen on port 80, or to change the client configuration to include the explicit port 3000.
(Kubernetes Services do have this capability, and I tend to remap an unprivileged port from a given Pod to the standard HTTP port in a Service, but that’s not a core Docker capability at all.)
What is the difference between ports and expose options in docker-compose.yml?
According to the docker-compose reference,
Ports is defined as:
Expose ports. Either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen).
Ports mentioned in docker-compose.yml will be shared among different services started by the docker-compose.
Ports will be exposed to the host machine to a random port or a given port.
My docker-compose.yml looks like:
mysql:
image: mysql:5.7
ports:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
-------------------------------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 0.0.0.0:32769->3306/tcp
Expose is defined as:
Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.
Ports are not exposed to host machines, only exposed to other services.
mysql:
image: mysql:5.7
expose:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
---------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 3306/tcp
Edit
In recent versions of Dockerfile, EXPOSE doesn't have any operational impact anymore, it is just informative. (see also)
ports:
Activates the container to listen for specified port(s) from the world outside of the docker(can be same host machine or a different machine) AND also accessible world inside docker.
More than one port can be specified (that's is why ports not port)
expose:
Activates container to listen for a specific port only from the world inside of docker AND not accessible world outside of the docker.
More than one port can be specified
Ports
This section is used to define the mapping between the host server and Docker container.
ports:
- 10005:80
It means the application running inside the container is exposed at port 80. But external system/entity cannot access it, so it need to be mapped to host server port.
Note: you have to open the host port 10005 and modify firewall rules to allow external entities to access the application.
They can use
http://{host IP}:10005
something like this
EXPOSE
This is exclusively used to define the port on which application is running inside the docker container.
You can define it in dockerfile as well. Generally, it is good and widely used practice to define EXPOSE inside dockerfile because very rarely anyone run them on other port than default 80 port
Ports
The ports section will publish ports on the host. Docker will set up a forward for a specific port from the host network into the container. By default, this is implemented with a userspace proxy process (docker-proxy) that listens on the first port, and forwards into the container, which needs to listen on the second point. If the container is not listening on the destination port, you will still see something listening on the host, but get a connection refused if you try to connect to that host port, from the failed forward into your container.
Note, the container must be listening on all network interfaces since this proxy is not running within the container's network namespace and cannot reach 127.0.0.1 inside the container. The IPv4 method for that is to configure your application to listen on 0.0.0.0.
Also note that published ports do not work in the opposite direction. You cannot connect to a service on the host from the container by publishing a port. Instead you'll find docker errors trying to listen to the already-in-use host port.
Expose
Expose is documentation. It sets metadata on the image, and when running, on the container too. Typically, you configure this in the Dockerfile with the EXPOSE instruction, and it serves as documentation for the users running your image, for them to know on which ports by default your application will be listening. When configured with a compose file, this metadata is only set on the container. You can see the exposed ports when you run a docker inspect on the image or container.
There are a few tools that rely on exposed ports. In docker, the -P flag will publish all exposed ports onto ephemeral ports on the host. There are also various reverse proxies that will default to using an exposed port when sending traffic to your application if you do not explicitly set the container port.
Other than those external tools, expose has no impact at all on the networking between containers. You only need a common docker network, and connecting to the container port, to access one container from another. If that network is user created (e.g. not the default bridge network named bridge), you can use DNS to connect to the other containers.
I totally agree with the answers before.
I just like to mention that the difference between expose and ports is part of the security concept in docker. It goes hand in hand with the networking of docker.
For example:
Imagine an application with a web front-end and a database back-end.
The outside world needs access to the web front-end (perhaps on port
80), but only the back-end itself needs access to the database host
and port. Using a user-defined bridge, only the web port needs to be
opened, and the database application doesn’t need any ports open,
since the web front-end can reach it over the user-defined bridge.
This is a common use case when setting up a network architecture in docker.
So for example in a default bridge network, not ports are accessible from the outer world.
Therefor you can open an ingresspoint with "ports". With using "expose" you define communication within the network. If you want to expose the default ports you don't need to define "expose" in your docker-compose file.
As we expose the container port in dockerfile itself then what is the use of container port in kubernetes yaml. What does it actually do. Is it mandatory to mention the container port in yaml file or we need not to mention in when we expose it in docker file.
Anyways, we will be using target port the map the container port with pod
ports:
- containerPort: 80
ports :
containerPortList of ports to expose from the container. Exposing a
port here gives the system additional information about the network
connections a container uses, but is primarily informational. Not
specifying a port here DOES NOT prevent that port from being exposed.
Any port which is listening on the default "0.0.0.0" address inside
a container will be accessible from the network. Cannot be
updated.
container-core
So it is exactly same with docker EXPOSE instruction. Both are informational. If you don’t configure ports in Kubernetes deployment, you can still access to the ports using Pod IP inside the cluster. You can create a service to access the ports externally without configuring ports in the deployment. But it is good to configure. It will help you or others to understand the deployment configuration better.
The EXPOSE instruction does not actually publish the port. It
functions as a type of documentation between the person who builds the
image and the person who runs the container, about which ports are
intended to be published.
.docker-reference-builder
What is the difference between ports and expose options in docker-compose.yml?
According to the docker-compose reference,
Ports is defined as:
Expose ports. Either specify both ports (HOST:CONTAINER), or just the container port (a random host port will be chosen).
Ports mentioned in docker-compose.yml will be shared among different services started by the docker-compose.
Ports will be exposed to the host machine to a random port or a given port.
My docker-compose.yml looks like:
mysql:
image: mysql:5.7
ports:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
-------------------------------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 0.0.0.0:32769->3306/tcp
Expose is defined as:
Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.
Ports are not exposed to host machines, only exposed to other services.
mysql:
image: mysql:5.7
expose:
- "3306"
If I do docker-compose ps, it will look like:
Name Command State Ports
---------------------------------------------------------------
mysql_1 docker-entrypoint.sh mysqld Up 3306/tcp
Edit
In recent versions of Dockerfile, EXPOSE doesn't have any operational impact anymore, it is just informative. (see also)
ports:
Activates the container to listen for specified port(s) from the world outside of the docker(can be same host machine or a different machine) AND also accessible world inside docker.
More than one port can be specified (that's is why ports not port)
expose:
Activates container to listen for a specific port only from the world inside of docker AND not accessible world outside of the docker.
More than one port can be specified
Ports
This section is used to define the mapping between the host server and Docker container.
ports:
- 10005:80
It means the application running inside the container is exposed at port 80. But external system/entity cannot access it, so it need to be mapped to host server port.
Note: you have to open the host port 10005 and modify firewall rules to allow external entities to access the application.
They can use
http://{host IP}:10005
something like this
EXPOSE
This is exclusively used to define the port on which application is running inside the docker container.
You can define it in dockerfile as well. Generally, it is good and widely used practice to define EXPOSE inside dockerfile because very rarely anyone run them on other port than default 80 port
Ports
The ports section will publish ports on the host. Docker will set up a forward for a specific port from the host network into the container. By default, this is implemented with a userspace proxy process (docker-proxy) that listens on the first port, and forwards into the container, which needs to listen on the second point. If the container is not listening on the destination port, you will still see something listening on the host, but get a connection refused if you try to connect to that host port, from the failed forward into your container.
Note, the container must be listening on all network interfaces since this proxy is not running within the container's network namespace and cannot reach 127.0.0.1 inside the container. The IPv4 method for that is to configure your application to listen on 0.0.0.0.
Also note that published ports do not work in the opposite direction. You cannot connect to a service on the host from the container by publishing a port. Instead you'll find docker errors trying to listen to the already-in-use host port.
Expose
Expose is documentation. It sets metadata on the image, and when running, on the container too. Typically, you configure this in the Dockerfile with the EXPOSE instruction, and it serves as documentation for the users running your image, for them to know on which ports by default your application will be listening. When configured with a compose file, this metadata is only set on the container. You can see the exposed ports when you run a docker inspect on the image or container.
There are a few tools that rely on exposed ports. In docker, the -P flag will publish all exposed ports onto ephemeral ports on the host. There are also various reverse proxies that will default to using an exposed port when sending traffic to your application if you do not explicitly set the container port.
Other than those external tools, expose has no impact at all on the networking between containers. You only need a common docker network, and connecting to the container port, to access one container from another. If that network is user created (e.g. not the default bridge network named bridge), you can use DNS to connect to the other containers.
I totally agree with the answers before.
I just like to mention that the difference between expose and ports is part of the security concept in docker. It goes hand in hand with the networking of docker.
For example:
Imagine an application with a web front-end and a database back-end.
The outside world needs access to the web front-end (perhaps on port
80), but only the back-end itself needs access to the database host
and port. Using a user-defined bridge, only the web port needs to be
opened, and the database application doesn’t need any ports open,
since the web front-end can reach it over the user-defined bridge.
This is a common use case when setting up a network architecture in docker.
So for example in a default bridge network, not ports are accessible from the outer world.
Therefor you can open an ingresspoint with "ports". With using "expose" you define communication within the network. If you want to expose the default ports you don't need to define "expose" in your docker-compose file.