This is a followup to an earlier question that I had asked, "https://stackoverflow.com/questions/72046646/does-docker-persist-the-resolv-conf-from-the-physical-etc-resolv-conf-in-the-co".
I've been testing with containers on 2 different machines, and using "--network host" and from that earlier thread in that case it is using a default "Host" mode network named "host"(?).
Since with "host" mode networking, the container and the app inside the container are basically on the same IP as the physical host where the container is running, under what (example) scenarios would you actually want to create a named "host" mode network and then have container use that named "host" mode network?
What would the advantages/differences be between using the custom/named "host" mode network vs. just using "--network host"?
It seems like both situations (using "--network host" vs. "create network xyz" where xyz is a named host network, and then doing the container "docker run --network xyz" would functionally be the same?
Sorry for the newbie question :( and thanks again in advance.
Jim
I don't think you can create a host-mode named network, and if you did, there'd be no reason to use it. If you need host networking – and you almost certainly don't – use docker run --net host or Compose network_mode: host.
But really, you don't need host networking.
With standard Docker networking, you can use docker run -p to publish individual ports out to the host. You get a choice to not publish a given port, and can remap the port. This also means that if, for example, you're running three services each with their own PostgreSQL server, there's no conflict over the single port 5432.
The cases where you actually need it are pretty limited. If an application listens on a very large number of ports or it doesn't listen on a predictable port then the docker run -p mechanism doesn't work well. If it needs to actively manage the host network then it needs to be given access to it (and it might be better run outside a container). If you've hard-coded localhost in your application, then in Docker your database isn't usually there (configuration via environment variables would be better).
Related
I'm trying to build and run a simple Docker container (using docker-compose to do this) on a GCP Instance (Ubuntu 20.04), and it seems that the container cannot access the internet, unless I run it using
docker run --net=host [...]
or use in my docker-compose.yml something like:
service:
build:
...
network: host
network_mode: host
...
I'm wondering why it is so, that a simple docker container on a standard GCP instance with Ubuntu 20.04 should require some specific configuration to access Internet, and why I see almost no mention of this while searching for this issue on the web.
Am I doing something wrong, is there a better way to do this?
See Container networking for Docker and the principle is applied consistently across other container runtimes too.
Using --net=host or network_mode: host binds container(s) to the host's network.
Rather than broadly publishing all of a container's or service's ports to the host network (and thus making them host public), you can be more precise using --publish=[HOST-PORT]:[CONTAINER-PORT] or ports to expose container ports as host ports (and potentially remap these too).
One (of several advantages) to the not-published-by-default behavior is that you must take a second step to publish a container's ports to a host where there is increased possibility that the service may be accessed (via its ports) by undesired actors.
From the Docker documentation:
--publish or -p flag. Publish a container's port(s) to the host.
--expose. Expose a port or a range of ports.
--link. Add link to another container. Is a legacy feature of Docker. It may eventually be removed.
I am using docker-compose with several networks. I do not want to publish any ports to the host, yet when I use expose, the port is then exposed to all the networks that container is connected to. It seems that after a lot of testing and reading I cannot figure out how to limit this to a specific network.
For example in this docker-compose file with where container1 joins the following three networks: internet, email and database.
services:
container1:
networks:
- internet
- email
- database
Now what if I have one specific port that I want to expose to ONLY the database network, so NOT to the host machine and also NOT to the email and internet networks in this example? If I would use ports: on container1 it is exposed to the host or I can bind it to a specific IP address of the host. *I also tried making a custom overlay network, giving the container a static IPv4 address and trying to set the ports in that format in ports: like - '10.8.0.3:80:80', but that also did not work because I think the binding can only happen to a HOST IP address. If i use expose: on container1 the port will be exposed to all three networks: internet, email and database.
I am aware I can make custom firewall ruling but it annoys me that I cannot write such simple config in my docker-compose file. Also, maybe something like 80:10.8.0.3:80 (HOST_IP:HOST_PORT:CONTAINER_IP:CONTAINER_PORT) would make perfect sense here (did not test it).*
Am I missing something or is this really not possible in Docker and Docker-compose?
Also posted here: https://github.com/docker/compose/issues/8795
No, container to container networking in docker is one-size-fits-many. When two containers are on the same network, and ICC has not been disabled, container-to-container communication is unrestricted. Given Docker's push into the developer workflow, I don't expect much development effort to change this.
This is handled by other projects like Kubernetes by offloading the networking to a CNI where various vendors support networking policies. This may be iptables rules, eBPF code, some kind of sidecar proxy, etc to implement it. But it has to be done as the container networking is setup, and docker doesn't have the hooks for you to implement anything there.
Perhaps you could hook into docker events and run various iptables commands for containers after they've been created. The application could also be configured to listen on the specific IP address for the network it trusts, but this requires injecting the subnet you trust and then looking up your container IP in your entrypoint, non-trivial to script up, and I'm not even sure it would work. Otherwise, this is solved by either restructuring the application so components that need to be on a less secure network are minimized, by hardening the sensitive ports, or switching the runtime over to something like Kubernetes with a network policy.
Things that won't help:
Removing exposed ports: this won't help since expose is just documentation. Changing exposed ports doesn't change networking between containers, or between the container and host.
Links: links are a legacy feature that adds entries to the host file when the container is created. This was replaced by creating networks with DNS resolution of other containers.
Removing published ports on the host: This doesn't impact container to container communication. The published port with -p creates a port forward from the host to the container, which you do want to limit, but containers can still communicate over a shared network without that published port.
The answer to this for me was to remove the -p command as that binds the container to the host and makes it available outside the host.
If you don't specify -p options. The container is available on all the networks it is connected to. On whichever port or ports the application is listening on.
It seems the -P forces the container on to the host and binds it to the port specified.
In your example if you don't use -p when staring "container1". "container1" would be available to the networks: internet, email, database with all its ports but not outside the host.
For running a container we can specify --net=host to enable host networking, which allows the container shares the host’s networking namespace. But what is the practical use case for this?
I've found it useful in two situations:
You have a server process that listens on a very large number of ports, or does not use a consistent port, so the docker run -p option is impractical or impossible.
You have a process that needs to examine or manage the host network environment. (Its wire protocol somehow depends on sending the host's IP address; it's a service-discovery system and you want it to advertise both Docker and non-Docker services running on the host.)
Host networking disables one of Docker's important isolation systems. If you run a container with host networking, you can't use features like port remapping and you can't accept inbound connections from other containers using the container name as a host name. In both of these cases, running the server outside Docker might be more appropriate.
In SO questions I frequently see --net host suggested as a hack to get around programs that have 127.0.0.1 hard-coded as the location of a database or another external resource. This isn't usually necessary, and adding a layer of configuration (environment variables work well) and the standard Docker networking setup is better practice.
I was going through docker official docs to understand the difference between user-defined and default bridge. Link to specific page - https://docs.docker.com/network/bridge/
In first point of section "Differences between user-defined bridges and the default bridge", it is stated that
If you run the same application stack on the default bridge network,
you need to open both the web port and the database port, using the -p
or --publish flag for each.
I don't understand this specific text, as to why it is need to explicitly publish(-p) required port of database container when it will be used only by some other container connected to the same bridge.
My existing understanding is that, unless explicitly blocked, containers connected to the docker0 can freely communicate with each other.
So, this extract has confused me. Can somebody help ?
If you take away one thing from that page, it's that you should always docker create network and then docker run --net containers on that network, if you're using plain Docker commands. (Docker Compose does this automatically for you; Kubernetes's networking model is fundamentally different.)
If you docker run a container without a --net option then you wind up using a backwards-compatiblitiy networking mode. In this mode (the "default bridge network") from the page you cite containers cannot communicate with each other by default. Your two options are for the server to publish a port (docker run -p) and the client to connect to the published port on the host, or for the server to expose a port (almost always done with an EXPOSE directive in the Dockerfile) and the client to --link to it.
There's no real reason to be using this "default" mode at this point, and in practice the paragraph you cite shouldn't matter except for fairly old scripted Docker setups.
Question: How can I change a Prometheus container's host address from the default 0.0.0.0:9090 to something like 192.168.1.234:9090?
Background: I am trying to get a Prometheus container to install and start in a production environment on a remote server. Since the server uses an IP other than Prometheus's default (0.0.0.0), I need to update the host address that the Prometheus container uses. If I don't, I can't sign-in to the UI and see any of the metrics. The IP of the remote server is provided by the user during the app's installation.
From what I understand from Prometheus's config document and the output of ./prometheus -h, the host address is immutable and therefore needs to be updated using the --web.listen-address= command-line flag. My problem is I don't know how to pass that flag to my Prometheus container; I can't simply run ./prometheus --web.listen-address="<remote-ip>:9090" because that's not a Docker command. And I can't pass it to the docker run ... command because Docker doesn't recognize that flag.
Environment:
Using SaltStack for config management
I cannot use Docker Swarm (i.e. each container must use its own Dockerfile)
You don't need to change the containerized prometheus' listen address. The 0.0.0.0/0 is the anynet inside the container.
By default, it won't even be accessible from your hosts network, let alone any surrounding networks (like the Internet).
You can map it to a port on a hosts interface though. The command for that looks somewhat like this:
docker run --rm -p 8080:9090 prom/prometheus
which would expose the service at 127.0.0.1:8080 on your host
You can do that with a public (e.g. internet-facing) interface as well, although i'd generally advise against exposing containers like this, due to numerous operational implications, which are somewhat beyond the scope of this answer. You should at least consider a reverse-proxy setup, where the users are only allowed to talk to some heavy-duty webserver which then communicates with prometheus, instead of letting them access your backend directly, even if this is just a small development deployment.
For general considerations on productionizing container setups, i suggest this.
Despite it's clickbaity title, this is a useful read.