Is it possible to expose docker ports dynamically, once a container is launched?
If not, what is the best practice to achieve something like this?
The use-case would be like:
I need to expose Java JMX Port momentarily to configure some application, and then close those ports (from the container), without actually closing the JMX Service, or modifying the Java application.
Thanks
With Weave network for Docker any port your application might open would be accessible from inside the network with no external intervention, unlike the aforementioned ambassador patter. However, those would be only accessible from the subnet the application is on. You the ports you statically expose will remain NATed by Docker too and would be externally accessible, but ephemeral once would be internal-only.
Never found a way to open ports dynamically.
Would it help to open a static port and start/stop a tunnel inside the container that forwards to the JMX Port.
socat or ssh comes to mind, e.g.
ssh -L<static-sourceport>:<targetserver>:<jmx-targetport> localhost
That's an interesting use case. I'm assuming you mean "publish docker ports dynamically". If so, you could look into something like the grand ambassador pattern, where you'd do something like:
Start your java/jmx container with only a static exposed port (nothing published to the host).
Bring up your ambassador container which publishes your jmx port temporarily.
When you're done, remove your ambassador.
Related
I have some docker containers talking together through docker bridge networks. They cannot be accessed from outside (I was said) as they are launched from a script with a default command which does not include 'expose' nor '-p' option. I cannot change that script.
I would like to connect to one of this containers which runs a server and listens for requests on port 8080. I tried connecting that bridge to a newly created docker bridge network, but i did not succede.
Now I am thinking of creating a new container and letting it talk to the server one (through bridge networks). As it is a new contaienr I can use the 'expose' or '-p' options, so it would be able to talk to the host machine.
Is it a good idea? How can I forward every request made to that container to the server one and get responses back to the host machine then?
Thanks
Within the default docker network, all ports are exposed. So you only need a container that exposes a port to the host machine and is in the same network as the other containers you have already created.
This is a relatively normal pattern. You can use a reverse proxy like nginx to achieve something like this.
There are some containers that automate this process:
https://github.com/jwilder/nginx-proxy
If you have no control over the other containers though, you will need to write the proxy config by hand.
If the container to which you are trying to connect is an http server, you may be able to use a ready-made container image that can work as an http forwarder (e.g., nginx - it is relatively easy to configure it as an http forwarder).
If you need plain tcp forwarding, you could make a container running 'socat' (socat can work as a tcp forwarder).
NOTE: in either case, you will be exposing a listener that wasn't meant to be on a public address. Do take measures not to allow unauthorized connections.
I am deploying a eureka server on a VM(say host external IP is a.b.c.d) as a docker image. Trying this in 2 ways.
1.I am running the docker image without explicit port mapping : docker run -p 8671 test/eureka-server
Then running docker ps command shows the port mapping as : 0.0.0.0:32769->8761/tcp
Try accessing the eureka server from outside of the VM with http://a.b.c.d:32769 , its not available.
2.I am running the docker image with explicit port mapping : docker run -p 8761:8761 test/eureka-server
Then running docker ps command shows the port mapping as : 0.0.0.0:8761->8761/tcp
Try accessing the eureka server from outside of the VM with http://a.b.c.d:8761 , its available.
Why in the first case the eureka server is not available from out side the host machine even if there is a random port(32769) assigned by docker.
Is it necessary to have explicit port mapping to have docker app available from external network ?
Since you're looking for access from the outside world to the host via the mapped port you'll need to ensure that the source traffic is allowed to reach that port on the host and given protocol. I'm not a network security specialist, but I'd suggest that opening up an entire range of ports simply because you don't know which port docker will pick would be a bad idea. If you can, I'd say pick a port and explicitly map it and ensure the firewall allows access to that port from the appropriate source address(es) e.g. ALLOW TCP/8671 in from 10.0.1.2/32 as an example - obviously your specific address range will vary on your network configuration. Docker compose may help you keep this consistent (as will other orchestration technologies like Kubernetes). In addition if you use cloud hosting services like AWS you may be able to leverage VPC security groups to help you whitelist source traffic to the port without knowing all possible source IP addresses ahead of time.
You either have the firewall blocking this port, or from wherever you are making the requests, for certain ports your outgoing traffic is disabled, so your requests never leave your machine.
Some companies do this. They leave port 80, 443, and couple of more for their intranet, and disable all other destination ports.
I wrote a simple peer to peer system, in which, when starting a node
the node looks for a free port and then makes its service accessible on that port,
it registers its URL including the port to a central server, so that other nodes know how to connect to it.
I have the impression this is a typical kind of task that docker is useful for, so I thought of making a container for these peers (my previous experience with docker has only been to write a hello world container).
Ideally I would map (publish) my exposed port to a host port from within the container using the same code that I am running now, but I could imagine that is simply not possible, and I could get around that by starting the image using a script that checks for availability of ports and then runs the container on an appropriate free host port. If the first is possible however, that would be even better. To be explicit, I do something like the following in Python
port = 5001
while not port_is_free(port):
port += 1
The second part really has to be taken care of from within the container. Assume that it has been started with the command docker run -p 5005:80 p2p-node then I need to find out the published port 5005 that the exposed port 80 is mapped to from within the container.
Searching this site and the internet it looks like more people are interested in doing the same, but I couldn't find a solution, nor an affirmation that this simply cannot be done.
So this is the main question I want to ask: how can I see which published ports my exposed ports are mapped to from within a running docker container?
Some of your requirements are not clear to me.
However if you want to know only which host port is mapped with your container's port, you can simply pass an environment variable, -e VAR=val. Just an idea
Start container:
docker run -p 5005:80 -e HOST_PORT=5005 p2p-node
Access the variable from container
echo $HOST_PORT
there is docker-py, a python library of docker.
It is not clear why you want the host port. In docker, containers can communicate with each other without having to expose ports on host machines.
As long as the peer apps are containerized, you don't need the expose port. The containers can be connected via a Docker network and the internal port can be used for communication between the containers.
I have a docker container that listens on a socket, lets say its udp port 20000. (this is iot udp data coming in)
This app can (and should) be loadbalanced.
I publish it to my docker host and expose the port. Docker assigns some random port on the docker host.
I need to add this container to the pool on my loadbalancer, which sits outside of the docker network.
How do I automate this? Any time a new instance of this container starts I need to add it to the pool. When it dies I need to remove it from the pool.
The pattern which worked for me is to use 2 pieces:
Registrator, which detects containers as they go on-line and register them in a kind of service registry (e.g. Consul).
Load-balancer that is aware (watches) of services registered and changing its configuration accordingly. In my particular case it was HAProxy supported by Consul Template, which did a great job in automating all this stuff.
In general, this pattern can vary in details, but usually it will be something like that.
I am learning Micro-services architecture by writing a small web app. The app has the following components, each of which will be hosted by a docker container.
In my API Gateway which is written in NodeJS, there is some place I will call:
request('http://service_b_ip_addr:port/get_service_b', callback);
However, both service_b_ip_addr and port are not known until Marathon has the Service B's docker container created.
With some Service Discovery mechanism, such as mesos-dns or marathon-lb, I guess that I could just change service_b_ip_addr to something like service_b.marathon.com.
But I've no idea how should I put the port in my program.
Thanks in advance for your help.
PS:
I am using BRIDGED network mode given that multiple instances of a Service could locate on the same Mesos slave. So port is a NATted random number.
Take a look at this answer.
If you use marathon-lb then there is no need to pass a port because it's a proxy and it will know where service is just by name.
If you use mesos-dns you should make a SRV request to get ip and port. In node you can do it with dns.resolveSrv(hostname, callback) but your DNS must be exposed on defaul (53) port and supports SRV request (mesos-dns supports it).