I'm trying to run two services using docker compose. Here is my docker-compose.yml:
version: '3'
services:
data:
build: ./ML_py
ports:
- "5000:5000"
node:
build: ./node_server
ports:
- "8080:8080"
This is all identical to the docker-compose beginner documentation, and when I run docker-compose up a network is created for the set of services. However, when I run docker ps, the containers have no mapped ports. Shouldn't the port values specified by 'ports' take care of this? Anyone know what might cause this?
Related
Scenario:
With the following docker-compose.yaml
version: 3
services:
helloworld:
image: hello-world
container_name: hello-world
whoami:
image: containous/whoami
container_name: whoami
containers are started with docker-compose up
docker-compose.yaml is then edited to expose a port
version: 3
services:
helloworld:
image: hello-world
container_name: hello-world
whoami:
image: containous/whoami
container_name: whoami
ports:
- 10000:80
whoami is restarted via docker-compose restart whoami
Problem: the port is not exposed.
My question: what is the correct command to restart a container (previouly started as part of a docker-compose up) so that its (modified) definition in docker-compose.yaml is taken into account?
Note: restarting everything with docker-compose down && docker-compose up correctly exposes the port. What I want to avoid is to interfere with other running containers when modifying a single one.
Only another docker-compose up seems to work.
According to docker-compose up documentation:
If there are existing containers for a service, and the service’s configuration or image was changed after the container’s creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes).
What is the use of container_name in docker-compose.yml file? Can I use it as hostname which is nothing but the service name in docker-compose.yml file.
Also when I explicitly write hostname under services does it override the hostname represented by service name?
hostname: just sets what the container believes its own hostname is. In the unusual event you got a shell inside the container, it might show up in the prompt. It has no effect on anything outside, and there’s usually no point in setting it. (It has basically the same effect as hostname(1): that command doesn’t cause anything outside your host to know the name you set.)
container_name: sets the actual name of the container when it runs, rather than letting Docker Compose generate it. If this name is different from the name of the block in services:, both names will be usable as DNS names for inter-container communication. Unless you need to use docker to manage a container that Compose started, you usually don’t need to set this either.
If you omit both of these settings, one container can reach another (provided they’re in the same Docker Compose file and have compatible networks: settings) using the name of the services: block and the port the service inside the container is listening in.
version: '3'
services:
redis:
image: redis
db:
image: mysql
ports: [6033:3306]
app:
build: .
ports: [12345:8990]
env:
REDIS_HOST: redis
REDIS_PORT: 6379
MYSQL_HOST: db
MYSQL_PORT: 3306
The easiest answer is the following:
container_name: This is the container name that you see from the host machine when listing the running containers with the docker container ls command.
hostname: The hostname of the container. Actually, the name that you define here is going to the /etc/hosts file:
$ exec -it myserver /bin/bash
bash-4.2# cat /etc/hosts
127.0.0.1 localhost
172.18.0.2 myserver
That means you can ping machines by that names within a Docker network.
I highly suggest set these two parameters the same to avoid confusion.
An example docker-compose.yml file:
version: '3'
services:
database-server:
image: ...
container_name: database-server
hostname: database-server
ports:
- "xxxx:yyyy"
web-server:
image: ...
container_name: web-server
hostname: web-server
ports:
- "xxxx:xxxx"
- "5101:4001" # debug port
you can customize the image name to build & container name during docker-compose up for this, you need to mention like below in docker-compose.yml file.
It will create an image & container with custom names.
version: '3'
services:
frontend_dev:
stdin_open: true
environment:
- CHOKIDAR_USEPOLLING=true
build:
context: .
dockerfile: Dockerfile.dev
image: "mycustomname/sample:v1"
container_name: mycustomname_sample_v1
ports:
- '3000:3000'
volumes:
- /app/node_modules
- .:/app
I have a flask service running through docker-compose on port 5000. Similarly, I have a different go service running through another docker-compose on port 8000. The Golang service needs to call a flask API running on 5000. I am facing trouble in getting the go service to call flask service. I have tried adding docker-network but failed. What are the pros and cons of running both the services through different docker-compose as compared to single docker-compose? (I have not been able to successfully run them in a single docker-compose, btw). docker ps running both the containers.
Flask Docker compose
version: '3' # version of compose format
services:
bidders:
build:
dockerfile: Dockerfile
context: .
volumes:
- .:/usr/src/bidders # mount point
ports:
- 5000:5000 # host:container
Go Docker Compose
version: '3'
services:
auctions:
container_name: auctions
build: .
command: go run main.go
volumes:
- .:/go/src/auctions
working_dir: /go/src/auctions
ports:
- "8000:8000"
Third Nwtwork Docker-compose.yml
#docker-compose.yml
version: '3'
networks:
- second_network
networks:
second_network:
driver: bridge
With a single docker-compose.yml it will be easier to make both services inside the same network. So what was the issue you got while doing this ? Also make sure that your flask and go application both are binding to 0.0.0.0 from the code itself and not 127.0.0.1 so you can reach them from outside the container.
With two docker-compose.yml you have two options:
Create a network through one of these files and make the other container which in another file join this external network.
Create a network using docker network create and define an external network in both files for your containers
There is a similar question that you can check it's answer from here with example included
You can check Networking in Compose for more information
I can't find a real definition of what a docker-compose file is.
Is it correct to say this:
A docker-compose file is a YAML file that allows us to deploy multiples Docker containers at the same time.
I'd like to be able to explain a bit better what a docker-compose file is.
A docker-compose.yml is a config file for Docker Compose.
It allows to deploy, combine, and configure multiple docker containers at the same time. The Docker "rule" is to outsource every single process to its own Docker container.
Take for example a simple web application: You need a server, a database, and PHP. So you can set three docker containers with Apache2, PHP, and MySQL.
The advantage of Docker Compose is easy configuration. You don't have to write a big bunch of commands into Bash. You can predefine it in the docker-compose.yml:
db:
image: mysql
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: example_db
MYSQL_USER: root
MYSQL_PASSWORD: rootpw
php:
image: php
ports:
- "80:80"
- "443:443"
volumes:
- ./SRC:/var/www/
links:
- db
As you can see in my example, I define port forwarding, volumes for external data, and links to the other Docker container. It's fast, reproducible, and not that hard to understand.
The Docker Compose file format is formally specified which enables docker-compose.yml files being executed with something else than Docker, Podman for example.
Docker Compose is a tool that allows you to deploy and manage multiple containers at the same time.
A docker-compose.yml file contains instructions on how to do that.
In this file, you instruct Docker Compose for example to:
From where to take the Dockerfile to build a particular image
Which ports you want to expose
How to link containers
Which ports you want to bind to the host machine
Docker Compose reads that file and executes commands.
It is used instead of all optional parameters when building and running a single docker container.
Example:
version: '2'
services:
nginx:
build: ./nginx
links:
- django:django
- angular:angular
ports:
- "80:80"
- "8000:8000"
- "443:443"
networks:
- my_net
django:
build: ./django
expose:
- "8000"
networks:
- my_net
angular:
build: ./angular2
links:
- django:django
expose:
- "80"
networks:
- my_net
networks:
my_net:
external:
name: my_net
This example instructs Docker Compose to:
Build nginx from path ./nginx
Links angular and django containers (so their IP in the Docker network is resolved by name)
Binds ports 80, 443, 8000 to the host machine
Add it to network my_net
(so all 3 containers are in the same network and therefore accessible from each other)
Then something similar is done for the django and angular containers.
If you would use just Docker commands, it would be something like:
docker build --name nginx .
docker run --link django:django angular:angular --expose 80 443 8000 --net my_net nginx
So while you probably don't want to type all these options and commands for each image/container, you can write a docker-compose.yml file in which you write all these instructions in a human-readable format.
I have two different Docker containers and each has a different image. Each app in the containers uses non-conflicting ports. See the docker-compose.yml:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
ports:
- "6473:6473"
- "6474:6474"
- "1812:1812"
depends_on:
- postgres
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
postgres:
container_name: postgres.dev
hostname: postgres.dev
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
I can cURL each image successfully from the host machine (Mac OS), e.g. curl -k https://localhost:6473/service_a/api/version works. What I'd like to do is to be able to refer to postgres container from the service_a container via localhost as if these two containers were one and they share the same localhost. I know that it's possible if I use the hostname postgres.dev from inside the service_a container, but I'd like to be able to use localhost. Is this possible? Please note that I am not very well versed in networking or Docker.
Mac version: 10.12.4
Docker version: Docker version 17.03.0-ce, build 60ccb22
I have done quite some prior research, but couldn't find a solution.
Relevant: https://forums.docker.com/t/localhost-and-docker-compose-networking-issue/23100/2
The right way: don't use localhost. Instead use docker's built in DNS networking and reference the containers by their service name. You shouldn't even be setting the container name since that breaks scaling.
The bad way: if you don't want to use the docker networking feature, then you can switch to host networking, but that turns off a very key feature and other docker capabilities like the option to connect containers together in their own isolated networks will no longer work. With that disclaimer, the result would look like:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
network_mode: "host"
depends_on:
- postgres
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
postgres:
container_name: postgres.dev
image: postgres:9.6
network_mode: "host"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
Note that I removed port publishing from the container to the host, since you're no longer in a container network. And I removed the hostname setting since you shouldn't change the hostname of the host itself from a docker container.
The linked forum posts you reference show how when this is a VM, the host cannot communicate with the containers as localhost. This is an expected limitation, but the containers themselves will be able to talk to each other as localhost. If you use a VirtualBox based install with docker-toolbox, you should be able to talk to the containers by the virtualbox IP.
The really wrong way: abuse the container network mode. The mode is available for debugging container networking issues and specialized use cases and really shouldn't be used to avoid reconfiguring an application to use DNS. And when you stop the database, you'll break your other container since it will lose its network namespace.
For this, you'll likely need to run two separate docker-compose.yml files because docker-compose will check for the existence of the network before taking any action. Start with the postgres container:
version: "2"
services:
postgres:
container_name: postgres.dev
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
Then you can make a second service in that same network namespace:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
network_mode: "container:postgres.dev"
ports:
- "6473:6473"
- "6474:6474"
- "1812:1812"
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
Specifically for Mac and during local testing, I managed to get the multiple containers working using docker.for.mac.localhost approach. I documented it http://nileshgule.blogspot.sg/2017/12/docker-tip-workaround-for-accessing.html