I'm wondering why, with docker-compose, the container cannot resole host-name while it's working with docker ? The host is on different physical machine but on the same network.
My Dockerfile
FROM openjdk:8-jre-alpine
CMD ping -c 2 myhost
My docker-compose.yml
version: '3.3'
services:
net:
build: .
image: test/myimage:1.0
container_name: mycontainer
After a docker-compose build I tried
docker run -it test/myimage:1.0
PING myhost (10.20.78.13): 56 data bytes
64 bytes from 10.20.78.13: seq=0 ttl=250 time=0.720 ms
64 bytes from 10.20.78.13: seq=1 ttl=250 time=0.515 ms
but
docker-compose up
Recreating mycontainer ...
Recreating mycontainer ... done
Attaching to mycontainer
mycontainer | ping: bad address 'myhost'
mycontainer exited with code 1
What can I do to have it working ?
Edit1
Using cat /etc/resolv.conf instead of ping -c 2 myhost
docker-compose up
Recreating mycontainer ...
Recreating mycontainer ... done
Attaching to mycontainer
mycontainer | search myorg.intra
mycontainer | nameserver 127.0.0.11
mycontainer | options ndots:0
mycontainer exited with code 0
and
docker run -it test/myimage:1.0
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.19.128.55
nameserver 10.19.142.23
nameserver 10.19.128.56
search myorg.intra
You need use the dns entries in your compose also
version: '3.3'
services:
net:
build: .
image: test/myimage:1.0
container_name: mycontainer
dns:
- 10.19.128.55
- 10.19.142.23
- 10.19.128.56
dns_search: myorg.intra
If you don't want to specify them per container then you can even set them directly at the daemon level.
Create or edit /etc/docker/daemon.json and below entries in the JSON file
{
"dns": ["10.19.128.55", "10.19.142.23", "10.19.128.56"],
"dns-search": ["myorg.intra"],
}
Restart the docker service using service docker restart. Then you should not need the entries in the docker-compose. Your original docker-compose should work
There is a subtle difference in network settings when running docker commands directly vs. performing some docker-compose runs. With docker-compose the containers are attached to some user-defined network which is a bridged network but not with the same configuration as the default bridged network. The latter one does some special configuration to be backwards compatible to the versions where these networking features didn't exist like today.
I assume that when running from compose, you are missing some dns settings that are available on your host machine and thus you aren't able to resolve the other host on your network. Please have a look at the differences here and here. For a first check, you could compare the output when running cat /etc/resolv.conf instead of your ping command for both ways and see what you need to add to make it work from compose.
Related
I would like to use a standard way of running my docker containers. I have have been keeping a docker_run.sh file, but docker-compose.yml looks like a better choice. This seems to work great until I try to access my website running in the container. The ports don't seem to be set up correctly.
Using the following docker_run.sh, I can access the website at localhost. I expected the following docker-compose.yml file to have the same results when I use the docker-compose run web command.
docker_run.sh
docker build -t web .
docker run -it -v /home/<user>/git/www:/var/www -p 80:80/tcp -p 443:443/tcp -p 3316:3306/tcp web
docker-compose.yml
version: '3'
services:
web:
image: web
build: .
ports:
- "80:80"
- "443:443"
- "3316:3306"
volumes:
- "../www:/var/www"
Further analysis
The ports are reported as the same in docker ps and docker-compose ps. Note: these were not up at the same time.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
<id> web "/usr/local/scripts/…" About an hour ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:3307->3306/tcp <name>
$ docker-compose ps
Name Command State Ports
---------------------------------------------------------------------------------------------------------------
web /usr/local/scripts/start_s ... Up 0.0.0.0:3316->3306/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
What am I missing?
As #richyen suggests in a comment, you want docker-compose up instead of docker-compose run.
docker-compose run...
Runs a one-time command against a service.
That is, it's intended to run something like a debugging shell or a migration script, in the overall environment specified by the docker-compose.yml file, but not the standard command specified in the Dockerfile (or the override in the YAML file).
Critically to your question,
...docker-compose run [...] does not create any of the ports specified in the service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports to be created and mapped to the host, specify the --service-ports flag.
Beyond that, the docker run command you show and the docker-compose.yml file should be essentially equivalent.
You don't run docker-compose.yamls the same way that you would run a local docker image that you have either installed or created on your machine. docker-compose files are typically launched running the command docker-compose up -d to run in detached mode. Then when you run docker ps you should see it running. You can also run docker-compose ps as you did above.
I am trying to run bamboo on server using docker containers. When i running on local machine work normally and volume save datas successfully. But when i run same docker compose file on server, volume data not save my datas.
docker-compose.yml
version: '3.2'
services:
bamboo:
container_name: bamboo-server_test
image: atlassian/bamboo-server
volumes:
- ./volumes/bamboo_test_vol:/var/atlassian/application-data/bamboo
ports:
- 8085:8085
volumes:
bamboo_test_vol:
Run this compose file on local machine
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
916c98ca1a9d atlassian/bamboo-server "/entrypoint.sh" 24 minutes ago Up 24 minutes 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/bamboo_test_vol/
$ ls
bamboo.cfg.xml logs
localhost:8085
Run this compose file on server
$ ssh <name>#<ip_address>
password for <name>:
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
38b77e1b736f atlassian/bamboo-server "/entrypoint.sh" 12 seconds ago Up 11 seconds 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/
$ cd bamboo_test_vol/
$ ls
$ # VOLUME PATH IS EMPTY
server_ip:8085
I didn't have this problem when I tried the same process for jira-software. Why can't it work through the bamboo server even though I use the exact same compose file?
I had the same problem when I wanted to upgrade my Bamboo server instance with my mounted host volume for the bamboo-home directory.
The following was in my docker-compose file:
version: '2.2'
bamboo-server:
image: atlassian/bamboo-server:${BAMBOO_VERSION}
container_name: bamboo-server
environment:
TZ: 'Europe/Berlin'
restart: always
init: true
volumes:
- ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
- "8085:8085"
- "54663:54663"
When i started with docker-compose up -d bamboo-server, the container never took the files from the host system. So I tried it first without docker-compose, following the instructions of Atlassian Bamboo with the following command:
docker run -v ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
The following error message was displayed:
docker: Error response from daemon: create ./bamboo/bamboo-server/data: "./bamboo/bamboo-server/data" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
So I converted the error message and took the absolute path:
docker run -v /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
After the successful start, I switched to the docker container via SSH and all files were as usual in the docker directory.
I transferred the whole thing to the docker-compose file and took the absolute path in the volumes section. Subsequently it also worked with the docker-compose file.
My docker-compose file then looked like this:
[...]
init: true
volumes:
- /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
[...]
Setting up a containerized Bamboo Server is not supported for these reasons;
Repository-stored Specs (RSS) are no longer processed in Docker by default. Running RSS in Docker was not possible because;
there is no Docker capability added on the Bamboo server by default,
the setup would require running Docker in Docker.
I have the following docker-compose.yml file:
version: '3'
services:
frontend:
image: alpine
command: tail -f /dev/null
networks:
- shared
- default
backend:
image: alpine
command: tail -f /dev/null
networks:
- shared
- default
networks:
shared:
external: true
Based on the file from above I create two projects which use the same network (shared) and the same service names (frontend and backend):
docker-compose -p foo up -d
docker-compose -p bar up -d
Does the DNS of docker make sure that docker-compose -p foo exec frontend ping backend only resolves to the backend container in project foo and vice versa for project bar?
According to https://github.com/docker/compose/issues/4645, the resolve order in this case in non deterministic. Since the network is being converted to unordered dict in golang, the order is not preserved. Which implies https://github.com/docker/libnetwork/blob/master/sandbox.go#L593 the order of endpoints being queried don't match the order of network.
The solution is to define https://docs.docker.com/compose/compose-file/compose-file-v2/#priority if using docker-compose version 2. Or fully qualified dns name as service.network such as backend.foo_default or backend.shared.
Based on your setup I have used nslookup to find out whether the DNS resolution is isolated or not.
$ docker-compose -p foo exec frontend nslookup backend
Name: backend
Address 1: 172.19.0.2 foo_backend_1.shared
Address 2: 172.19.0.4 bar_backend_1.shared
As you can see from the output above, backend resolves to both of the containers.
If you use docker swarm you can qualify hostnames with the service name to disambiguate containers. But I don't believe docker-compose does this.
I am trying to run docker networking with compose in docker 1.9. I know this is still experimental in 1.9, but I'd like to know if my use case could work.
Basically and to simplify, I have two servers, each one being in a container. Let's call them S1 and S2. S1 is exposed globally and must be accessible from S2. I want to run S2 through docker-compose with the --x-networking flag (the reason why I want this is that S2 is actually a bit more complexe than what is assumed here, having several containers, and I want them to run in a single subnetwork). S1 can run with or without compose.
What I understand from docker networks is that any container can reach other from the same network, or can reach anything that is "globally" exposed through host port mapping, right?
So my scenario is:
I start manually S1 with port mapping such as "-p 7210:7202" (7202 is exposed in dockerfile)
S2 is created from a simple compose file and gone up with flag --x-networking
For my test case I just created a very minimalistic compose file, such as:
S2:
build: .
ports:
- "8080:80"
Results:
S1 is NOT visible from S2 under "localhost" (this is quite expected)
S1 is NOT visible from S2 under "172.17.0.1" (= interface docker0)
I would have expected to be able to reach it under 172.17.0.1, since S1 uses docker0 as I understand.
Moreover, if I start S2 without compose but manually with "docker run", then I can access S1 using 172.17.0.1
So why doesn't it work with compose? Is it a limitation due to networking features being still experimental?
Note:
This is old content. The --x-networking has been removed in docker-compose
1.6. Additionally Docker compose has a new file format.
The documentation states that the networking feature is experimental in Docker compose.
Note: Compose’s networking support is experimental, and must be
explicitly enabled with the docker-compose --x-networking flag.
It's Docker 1.9 that actually implements the new feature:
https://docs.docker.com/engine/userguide/networking/dockernetworks/
Perhaps an example would help.
Example
└── demo
└── docker-compose.yml
I'm using docker 1.9.0 and docker-compose 1.5.0
docker-compose.yml
Declare two containers callled "web1" and "web2". Doesn't matter what image in this case I'm running tomcat.
web1:
image: tomcat:8.0
ports:
- 8080
web2:
image: tomcat:8.0
ports:
- 8080
Create the containers
Start this using docker compose
$ cd demo
$ docker-compose --x-networking up -d
Creating network "demo" with driver "None"
Creating demo_web2_1
Creating demo_web1_1
Note how a network called "demo" is created when you specified the new --x-networking parameter.
Demonstrate how discovery works
Each container is run on the "demo" network that was created and each container is placed as an entry in the other hosts file.
$ docker-compose ps
Name Command State Ports
---------------------------------------------------------------
demo_web1_1 catalina.sh run Up 0.0.0.0:32773->8080/tcp
demo_web2_1 catalina.sh run Up 0.0.0.0:32772->8080/tcp
$ docker exec -it demo_web1_1 cat /etc/hosts
..
172.18.0.2 demo_web2_1
$ docker exec -it demo_web2_1 cat /etc/hosts
..
172.18.0.3 demo_web1_1
Run an additional container outside of compose
Start another container, and specify you want it to be attached to the "demo" network:
$ docker run --net demo --name helloworld -d tomcat:8.0
And see how the hosts files in the other containers is updated automatically
$ docker exec -it demo_web1_1 cat /etc/hosts
..
172.18.0.2 demo_web2_1
172.18.0.4 helloworld
$ docker exec -it demo_web2_1 cat /etc/hosts
172.18.0.3 demo_web1_1
172.18.0.4 helloworld
$ docker exec -it helloworld cat /etc/hosts
172.18.0.2 demo_web2_1
172.18.0.3 demo_web1_1
Suppose I have a docker compose file with two containers. Both reference each other in their /etc/hosts file. Container A has a reference for container B and vice versa. And all of this happens automatically. Now I want to add one or more hostnames to B in A's hosts file. How can I go about doing this? Is there a special way I can achieve this in Docker Compose?
Example:
172.0.10.166 service-b my-custom-hostname
Yes. In your compose file, you can specify network aliases.
services:
db:
networks:
default:
aliases:
- database
- postgres
In this example, the db service could be reached by other containers on the default network using db, database, or postgres.
You can also add aliases to running containers using the docker network connect command with the --alias= option.
Docker compose has an extra_hosts feature that allows additional entries to be added to the container's host file.
Example
docker-compose.yml
web1:
image: tomcat:8.0
ports:
- 8081:8080
extra_hosts:
- "somehost:162.242.195.82"
- "otherhost:50.31.209.229"
web2:
image: tomcat:8.0
ports:
- 8082:8080
web3:
image: tomcat:8.0
ports:
- 8083:8080
Demonstrate host file entries
Run docker compose with the new docker 1.9 networking feature:
$ docker-compose --x-networking up -d
Starting tmp_web1_1
Starting tmp_web2_1
Starting tmp_web3_1
and look at the hosts file in the first container. Shows the other containers, plus the additional custom entries:
$ docker exec tmp_web1_1 cat /etc/hosts
..
172.18.0.4 web1
172.18.0.2 tmp_web2_1
172.18.0.3 tmp_web3_1
50.31.209.229 otherhost
162.242.195.82 somehost
If I understand your question correctly, you can pass a host name referenced in your host's /etc/hosts file via --add-host flag :
$ docker run ... --add-host="droid"
Your host's /etc/hosts would need the following entry:
xx.xx.xx.xx droid
Of course, xx.xx.xx.xx will need to be reachable from inside the container you just started using the 'docker run' command. You can have one or more --add-host="xyz".
More details about --add-host here:
http://docs.docker.com/v1.8/reference/commandline/run/