I wrote a simple peer to peer system, in which, when starting a node
the node looks for a free port and then makes its service accessible on that port,
it registers its URL including the port to a central server, so that other nodes know how to connect to it.
I have the impression this is a typical kind of task that docker is useful for, so I thought of making a container for these peers (my previous experience with docker has only been to write a hello world container).
Ideally I would map (publish) my exposed port to a host port from within the container using the same code that I am running now, but I could imagine that is simply not possible, and I could get around that by starting the image using a script that checks for availability of ports and then runs the container on an appropriate free host port. If the first is possible however, that would be even better. To be explicit, I do something like the following in Python
port = 5001
while not port_is_free(port):
port += 1
The second part really has to be taken care of from within the container. Assume that it has been started with the command docker run -p 5005:80 p2p-node then I need to find out the published port 5005 that the exposed port 80 is mapped to from within the container.
Searching this site and the internet it looks like more people are interested in doing the same, but I couldn't find a solution, nor an affirmation that this simply cannot be done.
So this is the main question I want to ask: how can I see which published ports my exposed ports are mapped to from within a running docker container?
Some of your requirements are not clear to me.
However if you want to know only which host port is mapped with your container's port, you can simply pass an environment variable, -e VAR=val. Just an idea
Start container:
docker run -p 5005:80 -e HOST_PORT=5005 p2p-node
Access the variable from container
echo $HOST_PORT
there is docker-py, a python library of docker.
It is not clear why you want the host port. In docker, containers can communicate with each other without having to expose ports on host machines.
As long as the peer apps are containerized, you don't need the expose port. The containers can be connected via a Docker network and the internal port can be used for communication between the containers.
Related
Context, I'm currently dockerizing an application in windows containers, the application
will connect to a Sql Server database from outside the container, normally working with linux containers I could use host driver, but since that is not available in windows containers. How could I connect to that database outside my windows container?
So, the answers provided before are all valid. I'd just add that while Host network is not available on Windows, you can still use the same concept - albeit a bit different.
The native network driver on Windows is Network Address Translation. With that driver, the container will get a private IP address and the ports from the container host can be mapped to the ports on the container, by use of the docker run -p 8080:80, for example.
That way, if you want to continue to use the option to call the localhost between the app container and the database container you can. You just need to specify the port: localhost:8080. Note that if the host is not using that port, you can even map it directly, such as: docker run -p 80:80. The caveat here is: The container host cannot be using the port already, and you can't map the same port to another container. So, if you need another instance, you can map to something like: docker run -p 81:80.
I blogged about this here: https://cda.ms/4nB
If we have two applications app1.py and app2.py both running in docker container as flask services with following commands:
docker run -p 5000:5002 app1.py
docker run -p 9000:5002 app2.py
Is it possible to keep the same docker port 5002 for both containers?
Secondly, if I use app.run(host='0.0.0.0', port=5000, debug=True) in flask endpoint.py file which is used for image building, is port=5000 the docker port in container or the port available externally on host?
Yes, each container runs in an isolated network namespace, so every one can listen on the same port and they won’t conflict. Inside your application code the port you listen on is the container-internal port, and unless you get told in some other way (like an HTTP Host: header) you have no way of knowing what ports you’ve been remapped to externally.
Is it possible to keep the same docker port 5002 for both containers?
Yes, of course. Typically every container runs in an isolate network namespace, which means containers cannot communicate with each unless they are configured to do so. What confuses you maybe that containers do communicate with each other well by default, which should thank Docker network default setting. But there are still other use cases. You may see more about The container Network Model and network namespace here.
Is port=5000 the port in container or the port valid externally on host?
It’s the port in container with no doubt. We could notice it is a user-defined argument for function run() in Flask. Since Flask application runs in container, so 5000 will be the port which Flask app would listen on in container.
We should map it out if we wanna access 5000 on host(outside of container). The flag -p would help you.
Hope this helps~
Is it possible to expose docker ports dynamically, once a container is launched?
If not, what is the best practice to achieve something like this?
The use-case would be like:
I need to expose Java JMX Port momentarily to configure some application, and then close those ports (from the container), without actually closing the JMX Service, or modifying the Java application.
Thanks
With Weave network for Docker any port your application might open would be accessible from inside the network with no external intervention, unlike the aforementioned ambassador patter. However, those would be only accessible from the subnet the application is on. You the ports you statically expose will remain NATed by Docker too and would be externally accessible, but ephemeral once would be internal-only.
Never found a way to open ports dynamically.
Would it help to open a static port and start/stop a tunnel inside the container that forwards to the JMX Port.
socat or ssh comes to mind, e.g.
ssh -L<static-sourceport>:<targetserver>:<jmx-targetport> localhost
That's an interesting use case. I'm assuming you mean "publish docker ports dynamically". If so, you could look into something like the grand ambassador pattern, where you'd do something like:
Start your java/jmx container with only a static exposed port (nothing published to the host).
Bring up your ambassador container which publishes your jmx port temporarily.
When you're done, remove your ambassador.
I haven't fully understood the way port forwarding works with docker.
My scenario looks like this:
I have a Dockerfile that exposes a port (in my case it's 8000)
I have built an image using this Dockerfile (by using "docker build -t test_docker")
Now I created several containers by using "docker run -p 808X:8000 -d test_docker"
The host reacts on calling its IP with the different ports I have assigned on "docker run"
What exactly does this EXPOSE command do in the Dockerfile? I understood that the docker daemon itself handles the network connections and while calling "docker run" I also tell what image should be used...
Thanks
OK, I think I understood the reason.
If you are listening on ports within your application, you need to expose exactly this port. E.g.
HttpServer.bind('127.0.0.1', 8000).then((server) {...}
will need "EXPOSE 8000". Like this you can listen to several to several ports in your app but then need to expose them all.
Am I right?
Exposing ports in your dockerfile allows you to spin up a container using the -P(See here) flag on the docker run command.
A simple use case might be that you have nginx sitting on port 80 on a load balancing server, and it's going to load balance that traffic across a few docker conatiners sitting on a coreos docker server. Since each of your apps use the same port, 8000, you wouldn't be able to get to them individually. So docker would map each container to a high random, and non conflicting port on the host. So when you hit 49805, it goes to container 1s 8000, and when you hit 49807, it goes to container 2s 8000.
It seems that the preferred way to expose services to other Docker containers is container linking, which sets some environment variables that you then have to use in your application code to look up host names and port numbers:
psql -h $PG_PORT_5432_TCP_ADDR -p $PG_PORT_5432_TCP_PORT
Is there a reason this is not done via port forwarding in a way that is transparent to the application? So that in the same way that I can just run my web server inside the container on standard port 80 and have Docker figure out what actual port to use, I could just be doing
psql -h 0.0.0.0 # no -p necessary, we use the default port
The port forwarding would be set up when I start docker, just like with server ports.
This is possible! It has actually be proposed by the CoreOS team; you can read more in the following blog post:
http://coreos.com/blog/Jumpers-and-the-software-defined-localhost/
Docker will soon allow to start a container sharing the network namespace of another container; it will help with those scenarios (and in the short term, it will allow to do what you suggest very easily).
Project Atomic is also following this approach:
http://www.projectatomic.io/docs/inter-container-networking/
Geard uses iptables to enable containers to connect to each other. Network namespaces allows adding iptables rules to the network namespace of a container. The basic idea is to make remote endpoints appear as if they were local to a container. For example the database container could be made to appear to be running locally inside the application container.