I'm using docker-compose.yml version 2. One of my containers doesn't see another one and I'm trying to debug this.
Previously, docker-compose relied on links mechanism for containers to interact with each other. It was implemented via dynamic modification of /etc/hosts file.
Starting from docker-compose version 2, docker-compose relies upon Docker DNS mechanism. Now /etc/hosts is kept intact and containers query their internal Docker DNS to find each other.
I wonder, how do I query Docker DNS from within a container for debugging purposes? I want to make sure that it properly resolves the hostname of other containers in its network and that they are known by their aliases.
Related
I have a server application (that I cannot change) that, when you connect as a client, will give you other URLs to interact with. Those URLs are also part of the same server so the URL advertised uses the hostname of a docker container.
We are running in a mixed economy (some docker containers, some regular applications). We actually need to set up where we have the server running as a docker application on a single VM, and that server will be accessed by non-docker clients (as well as docker clients not running on the same docker network).
So you have a server hostname (the docker container) and a docker hostname (the hostname of the VM running docker).
The client's initial connection is to: dockerhostname:1234 but when the server sends URLs to the client, it sends: serverhostname:5678 ... which is not resolvable by the client. So far, we've addressed this by adding "server hostname " to the client's /etc/hosts file but this is a pain to maintain.
I have also set the --hostname of the server docker container to the same name as the docker host and it has mostly worked but I've seen where a docker container running on the same docker network as the server had issues connecting to the server.
I realize this is not an ideal docker setup. We're migrating from a history of delivering as rpm's to delivering containers .. but it's a slow process. Our company has lots of applications.
I'm really curious if anyone has advice/lessons learned with this situation. What is the best solution to my URL problem? (I'm guessing it is the /etc/hosts we're already doing)
You can do port-mapping -p 8080:80
How you build and run your container?
With a shell command, dockerfile or yml file?
Check this:
docker port
Call this and it will work:
[SERVERIP][PORT FROM DOCKERHOST]
To work with hostnames you need DNS or use hosts file.
The hosts file solution is not a good idea, it's how the internet starts in the past ^^
If something change you have to change all hosts files on every client!
Or use a static ip for your container:
docker network ls
docker network create my-network
docker network create --subnet=172.18.0.0/16 mynet123
docker run --net mynet123 --ip 172.18.0.22 -it ubuntu bash
Assign static IP to Docker container
You're describing a situation that requires a ton of work. The shortest path to success is your "adding things to /etc/hosts file" process. You can use configuration management, like ansible/chef/puppet to only have to update one location and distribute it out.
But at that point, you should look into something called "service discovery." There are a ton of ways to skin this cat, but the short of it is this. You need some place (lazy mode is DNS) that stores a database of your different machines/services. When a machine needs to connect to another machine for a service, it asks that database. Hence the "service discovery" part.
Now implementing the database is the hardest part of this, there are a bunch of different ways, and you'll need to spend some time with your team to figure out what is the best way.
Normally running an internal DNS server like dnsmasq or bind should get you most of the way, but if you need something like consul that's a whole other conversation. There are a lot of options, and the best thing to do is research, and audit what you actually need for your situation.
I've had db and server container, both running in the same network. Can ping db host by its container id with no problem.
When I set a hostname for db container manually (-h myname), it had an effect ($ hostname returns set host), but I can't ping that hostname from another container in the same network. Container id still pingable.
Although it works with no problem in docker compose.
What am I missing?
Hostname is not used by docker's built in DNS service. It's a counterintuitive exception, but since hostnames can change outside of docker's control, it makes some sense. Docker's DNS will resolve:
the container id
container name
any network aliases you define for the container on that network
The easiest of these options is the last one which is automatically configured when running containers with a compose file. The service name itself is a network alias. This lets you scale and perform rolling updates without reconfiguring other containers.
You need to be on a user created network, not something like the default bridge which has DNS disabled. This is done by default when running containers with a compose file.
Avoid using links since they are deprecated. And I'd only recommend adding host entries for external static hosts that are not in any DNS, for container to container, or access to other hosts outside of docker, DNS is preferred.
I've found out, that problem can be solved without network using --add-host option. Container's IP can be gain using inspect command.
But when containers in the same network, they are able to access each other via it names.
As stated in the docker docs, if you start containers on the default bridge network, adding -h myname will add this information to
/etc/hosts
/etc/resolv.conf
and the bash prompt
of the container just started.
However, this will not have any effect to other independent containers. (You could use --link to add this information to /etc/hosts of other containers. However, --link is deprecated.)
On the other hand, when you create a user-defined bridge network, docker provides an embedded DNS server to make name lookups between containers on that network possible, see Embedded DNS server in user-defined networks. Name resolution takes the container names defined with --name. (You
will not find another container by using its --hostname value.)
The reason, why it works with docker-compose is, that docker-compose creates a custom network for you and automatically names the containers.
The situation seems to be a bit different, when you don't specify a name for the container yourself. The run reference says
If you do not assign a container name with the --name option, then the daemon generates a random string name for you. [...] If you specify a name, you can use it when referencing the container within a Docker network.
In agreement with your findings, this should be read as: If you don't specify a custom --name, you cannot use the auto-generated name to look up other containers on the same network.
I've currently got two fairly vanilla CentOS 7 boxes that are running under a Docker Swarm, one a master and the other joined to it. In that swarm, I want to have a stack running that will essentially be my Plex / multimedia system. I've got the docker_compose.yml file for that linked. I can deploy the file to the swarm using the following command:
docker stack deploy --compose-file docker-compose.yml plexsystem
That works fine, it deploys the containers like you would expect. The issue I'm having is that the containers do not have external internet access, so if the container needs to download any files or interact with APIs, they fail. I interact with containers with docker exec -it container /bin/bash and try to ping out, and it always fails.
What do I need to add or change to my docker-compose.yml file so that my networking can work and I can finally get my stack working as it should. I've been banging my head with this one and I cannot figure out how to get swarm networking. Thank you very much!
Checkout your docker daemon file and look for the iptables parameter configuration. Most probably it is set to false and hence it is not able to access the internet.
--iptables=false prevents the Docker daemon from adding iptables rules. If multiple daemons manage iptables rules, they may overwrite rules set by another daemon. Be aware that disabling this option requires you to manually add iptables rules to expose container ports. If you prevent Docker from adding iptables rules, Docker will also not add IP masquerading rules, even if you set --ip-masq to true. Without IP masquerading rules, Docker containers will not be able to connect to external hosts or the internet when using network other than default bridge.
Okay so in Vagrant/VVV you can assign different hostnames to your different projects so when you go to http://myproject-1.dev your website shows up.
This is very convenient if you are working on dozens of projects at the same time, As far as I know such thing is not possible in docker (it can't touch hosts file), My question is, is there something similar we can do in Docker? Some automated tool maybe?
Using docker for windows.
Hostnames can map many containers together. In docker compose, there's a hostname option. But that's only within the Docker network bridge, not available to the host
Docker isn't a VM (although it runs within one in Windows).
You can edit your hosts file to have the HyperVisor available, but you're supposed to have the host ports forwarded into the container.
Use localhost, not any hostname.
If you prefer your Vagrant patterns, continue using it, but provision Docker containers from it, or use Docker Machine
I have an existing app that comprises of 4 docker containers running on the same host. They have been linked together using the link command.
However, after some upgrades of docker, the link behaviour has been deprecated, and changed it seems. We are having issues where containers are loosing the link to each other now.
So, docker says to use the new Network feature over linked containers. But I can't see how this works.
If 2 containers are in the same network, are the same ENV vars automatically exposed on the containers as if they were linked?
Or is the hosts file updated with the correct container name / ip addresses ? Even after a docker restart ?
I can't see in the docs how a container can find the location of another in its network?
Also, compose looks to have a simple set up for linking containers, and may automate some of this - would compose be the way to go for defining multi container apps? Or is it too soon to run it in production?
Does compose support multiple host configuration as well?
at some point in the future we will probably need to move one of the containers to a different host....
If 2 containers are in the same network, are the same ENV vars automatically exposed on the containers as if they were linked?
no, you would now have to use the container names as their hostnames. The new network feature has no idea which ports will be used. Think of this as 2 computers plugged on the same network hub. Both can address the other one by its hostname.
is the hosts file updated with the correct container name / ip addresses ? Even after a docker restart ?
yes, /etc/hosts files for all containers which are part of a network will be updated live by the docker engine.
I can't see in the docs how a container can find the location of another in its network?
Using the container name. See the Connect containers section of the Work with network commands doc:
Once connected, the containers can communicate using another container’s IP address or name.
Also, compose looks to have a simple set up for linking containers, and may automate some of this - would compose be the way to go for defining multi container apps? Or is it too soon to run it in production?
Compose supports the new network feature as beta by offering the --x-networking option. You should not use it in production yet (current Compose version is 1.5).
Furthermore, the current implementation is a bit inconvenient as we must use the full container name which is composed of the project name + _ + container name + _1. The documentation says the next version (current one is 1.5) will improve this so that we should not have to worry about the project name to address containers.
Does compose support multiple host configuration as well?
Yes, in conjonction with Swarm as detailed in the overlay network documentation