I am getting started with docker and docker-compose. I have the tutorials and I use docker-compose.yml file to run one of my sites in my local machine.
I can see my site running by going to http://localhost
My problem now is trying to run more than one site. If one of my sites is running and I try to run another site using docker-compose up -d I get the following error.
$ docker-compose up -d
Creating network "exampleCOM_default" with driver "bridge"
Creating exampleCOMphp-fpm ...
Creating exampleCOMmariadb ... error
ERROR: for exampleCOMmariadb Cannot start service db: driver failed programming external connectivity on endpoint exampleCOMmariadb (999572f33113c9fce034b4ed72aaCreating exampleCOMphp-fpm ... done
eady allocated
Creating exampleCOMnginx ... error
ERROR: for exampleCOMnginx Cannot start service nginx: driver failed programming external connectivity on endpoint exampleCOMnginx (9dc04f8b06825d7ff535afb1101933be7435c68f4350f845c756fc93e1a0322c): Bind for 0.0.0.0:443 failed: port is already allocated
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint exampleCOMmariadb (999572f33113c9fce034b4ed72aa072708f6f477eb2af8ad614c0126ca457b64): Bind for 0.0.0.0:3306 failed: port is already allocated
ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on endpoint exampleCOMnginx (9dc04f8b06825d7ff535afb1101933be7435c68f4350f845c756fc93e1a0322c): Bind for 0.0.0.0:443 failed: port is already allocated
Encountered errors while bringing up the project.
This is my docker-compose file. I am using LEMP stack (PHP, NGINX, MARIADB)
version: '3'
services:
db:
container_name: ${SITE_NAME}_mariadb
build:
context: ./mariadb
volumes:
- ./mariadb/scripts:/docker-entrypoint-initdb.d
- ./.data/db:/var/lib/mysql
- ./logs/mariadb:/var/log/mysql
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
ports:
- '${MYSQL_PORT:-3306}:3306'
command:
'mysqld --innodb-flush-method=fsync'
networks:
- default
restart: always
nginx:
container_name: ${SITE_NAME}_nginx
build:
context: ./nginx
args:
- 'php-fpm'
- '9000'
volumes:
- ${APP_PATH}:/var/www/app
- ./logs/nginx/:/var/log/nginx
ports:
- "80:80"
- "443:443"
depends_on:
- php-fpm
networks:
- default
restart: always
php-fpm:
container_name: ${SITE_NAME}_php-fpm
build:
context: ./php7-fpm
args:
TIMEZONE: ${TIMEZONE}
volumes:
- ${APP_PATH}:/var/www/app
- ./php7-fpm/config/php.ini:/usr/local/etc/php/php.ini
environment:
DB_HOST: db
DB_PORT: 3306
DB_DATABASE: ${MYSQL_DATABASE}
DB_USERNAME: ${MYSQL_USER}
DB_PASSWORD: ${MYSQL_PASSWORD}
networks:
- default
restart: always
networks:
default:
driver: bridge
The host port you have mapped to is preventing you from starting another instance of the service even though the docker-compose creates a private network.
You can solve this problem by using random host ports assigned by docker-compose.
The ports entry in docker-compose is
ports
host_port:container_port
If you specify only the container port host port is randomly assigned. See here
You can provide the host_port values in ranges.
In below example, i've run the nginx containers and started multiple nginx containers that are automatically exposed to host ports based on the range values [30000-30005].
Command:
docker run -p 30000-30005:80 --name nginx1 -d nginx
Output:
9083d5fc97e0 nginx ... Up 2 seconds 0.0.0.0:30001->80/tcp nginx1
f2f9de1efd8c nginx ... Up 24 seconds 0.0.0.0:30000->80/tcp nginx
Related
How do you launch Postgres from Docker, using docker-compose?
My docker-compose.yml looks like:
version: "3.6"
services:
db:
container_name: db
image: postgres:14-alpine
environment:
- POSTGRES_USER=test
- POSTGRES_PASSWORD=test
- POSTGRES_DB=test
ports:
- "5432:5432"
command: -c fsync=off -c synchronous_commit=off -c full_page_writes=off --max-connections=200 --shared-buffers=4GB --work-mem=20MB
tmpfs:
- /var/lib/postgresql
web:
container_name: web
build:
context: ..
dockerfile: test_tools/Dockerfile
shm_size: '2gb'
volumes:
- /dev/shm:/dev/shm
depends_on:
- db
This is a simple test environment to mimic a web server and a database server.
Yet when I build this, it fails with:
Creating db ... error
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint db (bdaebf844ee8ddd593b6bc75733d8aa6196112b62f7909be060017a9a33b3c34): Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
Why is my Postgres container trying to allocate a port on the host?
I do have Postgres running on port 5432 of the host, but why would this be interfering? These are just test containers that only need to talk to each other, and should not be accessible to the host, much less allocate host ports.
I've confirmed with docker ps -a that there are no other containers that might also be consuming port 5432.
ports:
- 5432
will start your Postgres, but on a random (free) host port.
Try to map postgres to different port on host for example
ports:
5432:15432
will make your db works on port 15432 on your host.
Below is the working docker-compose file in v2 spec:
version: '2'
volumes:
webroot:
driver: local
services:
app: # Launch uwsgi application server
build:
context: ../../
dockerfile: docker/release/Dockerfile
links:
- dbc
volumes:
- webroot:/var/www/someapp
environment:
DJANGO_SETTINGS_MODULE: someapp.settings.release
MYSQL_HOST: dbc
MYSQL_USER: todo
MYSQL_PASSWORD: passwd
command:
- uwsgi
- "--socket /var/www/someapp/someapp.sock"
- "--chmod-socket=666"
- "--module someapp.wsgi"
- "--master"
- "--die-on-term"
test: # Run acceptance test cases
image: shamdockerhub/someapp-specs
links:
- nginx
environment:
URL: http://nginx:8000/todos
JUNIT_REPORT_PATH: /reports/acceptance.xml
JUNIT_REPORT_STACK: 1
command: --reporter mocha-jenkins-reporter
nginx: # Start nginx web server that forwards https packets to uwsgi server
build:
context: .
dockerfile: Dockerfile.nginx
ports:
- "8000:8000"
links:
- app
volumes:
- webroot:/var/www/someapp
dbc: # Launch MySQL server
image: mysql:5.6
hostname: dbr
expose:
- "3306"
environment:
MYSQL_DATABASE: someapp
MYSQL_USER: todo
MYSQL_PASSWORD: passwd
MYSQL_ROOT_PASSWORD: passwd
agent: # Ensure DB server is runnin
image: shamdockerhub/ansible
links:
- dbc
environment:
PROBE_HOST: "dbc"
PROBE_PORT: "3306"
command: ["probe.yml"]
where entries
MYSQL_HOST: dbc
PROBE_HOST: "dbc"
does not look intuitive, because the hostname is set to dbr in dbc service
1)
app service fails with below error on using MYSQL_HOST: dbr
django.db.utils.OperationalError: (2005, "Unknown MySQL server host 'dbr' (0)")
2)
agent service also fails in below ansible code when PROBE_HOST: "dbr"
set_fact:
probe_host: "{{ lookup('env', 'PROBE_HOST') }}"
local_action: >
wait_for host={{ probe_host }}
1)
Why these two services are failing with value dbr?
2)
How to make these two services work with MYSQL_HOST: dbr
and PROBE_HOST: "dbr"?
that is how Docker works because the hostname is not unique and that will lead to a problem if you give two containers the same hostname therefore compose will always use the service name for DNS resolution
Setting hostname: is equivalent to the hostname(8) command on plain Linux: it changes what the container thinks its own hostname is, but doesn't affect anything outside the container that might try to reach it. On plain Linux running hostname dbr won't change an external DNS server or other machines' /etc/hosts files, for example. Setting the hostname might affect a shell prompt, in the unusual case of getting an interactive shell inside a container; it has no effect on networking.
Within a single Docker Compose file, if you have no special configuration for networks:, any container can reach any other container using the name of its block in the YAML file. In your file, app, nginx, test, dbc, and agent are valid hostnames. If you manually specify a container_name: I believe that will also be reachable; network aliases as suggested in #asolanki's answer give yet another name; and the deprecated links: option would give still another. All of these are in addition to the standard name Compose gives you.
Networking in Compose has some reasonable explanations of all of this.
In your example, dbr is not a valid hostname. dbc is the Compose service name of the container, but nothing from the previous listing causes a hostname dbr to exist. It happens to be the name you'll see in the prompt if you docker-compose exec dlc sh but nobody else thinks that container has that name.
As a specific corollary to "links: is deprecated", the form of links: you have does absolutely nothing. links: [dbc] makes the container that would otherwise be visible under the name dbc visible to that specific container as that same name. You could use it to give an alternate name to a container from the point of view of a client, but I wouldn't.
Your docker-compose.yml file doesn't have any networks: blocks, and so Compose will create a default network and attach all of the containers to it. This is totally fine and I would not recommend changing it. If you do declare multiple networks, the other requirement here is that the client and server need to be on the same network to reach each other. (Containers without a networks: block implicitly have networks: [default].)
If you want to reference the service by another name you can use network alias.
Modified compose file to use network alias
version: '2'
volumes:
webroot:
driver: local
services:
app: # Launch uwsgi application server
build:
context: ../../
dockerfile: docker/release/Dockerfile
links:
- dbc
volumes:
- webroot:/var/www/someapp
environment:
DJANGO_SETTINGS_MODULE: someapp.settings.release
MYSQL_HOST: dbc
MYSQL_USER: todo
MYSQL_PASSWORD: passwd
command:
- uwsgi
- "--socket /var/www/someapp/someapp.sock"
- "--chmod-socket=666"
- "--module someapp.wsgi"
- "--master"
- "--die-on-term"
networks:
new:
aliases:
- myapp
test: # Run acceptance test cases
image: shamdockerhub/someapp-specs
links:
- nginx
environment:
URL: http://nginx:8000/todos
JUNIT_REPORT_PATH: /reports/acceptance.xml
JUNIT_REPORT_STACK: 1
command: --reporter mocha-jenkins-reporter
networks:
- new
nginx: # Start nginx web server that forwards https packets to uwsgi server
build:
context: .
dockerfile: Dockerfile.nginx
ports:
- "8000:8000"
links:
- app
volumes:
- webroot:/var/www/someapp
networks:
- new
dbc: # Launch MySQL server
image: mysql:5.6
hostname: dbr
expose:
- "3306"
environment:
MYSQL_DATABASE: someapp
MYSQL_USER: todo
MYSQL_PASSWORD: passwd
MYSQL_ROOT_PASSWORD: passwd
networks:
new:
aliases:
- dbr
agent: # Ensure DB server is runnin
image: shamdockerhub/ansible
links:
- dbc
environment:
PROBE_HOST: "dbc"
PROBE_PORT: "3306"
command: ["probe.yml"]
networks:
- new
networks:
new:
I'm trying to map a port from my container, to a port on the host following the docs but it doesn't appear to be working.
After I run docker-compose -f development.yml up --force-recreate I get no errors. But if I try to reach the frontend service using localhost:8081 the network is unreachable.
I used docker inspect to view the IP and tried to ping that and still nothing.
Here is the docker-compose file I am using. And I doing anything wrong?
development.yml
version: '3'
services:
frontend:
image: nginx:latest
ports:
- "8081:80"
volumes:
- ./frontend/public:/var/www/html
api:
image: richarvey/nginx-php-fpm:latest
ports:
- "8080:80"
restart: always
volumes:
- ./api:/var/www/html
environment:
APPLICATION_ENV: development
ERRORS: 1
REMOVE_FILES: 0
links:
- db
- mq
db:
image: mariadb
restart: always
volumes:
- ./data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: dEvE10pMeNtMoDeBr0
mq:
image: rabbitmq:latest
restart: always
environment:
RABBITMQ_DEFAULT_USER: developer
RABBITMQ_DEFAULT_PASS: dEvE10pMeNtMoDeBr0
You are using docker toolbox. Docker toolbox uses docker machine. In Windows with docker toolbox, you are running under a virtualbox with its own IP, so localhost is not where your containers live. You will need to go 192.168.99.100:8081 to find your frontend.
As per the documentation on docker machine(https://docs.docker.com/machine/get-started/#run-containers-and-experiment-with-machine-commands):
$ docker-machine ip default
192.168.99.100
I have a website hosted on shared hosting on production. The website connects to the database via localhost in the code. In my docker-compose I have a php:5.6-apache and mysql:5.6 instance.
Is there any way to tell docker-compose to have port 3306 on the web container port forwarded to 3306 on the db container, so that when the web container tries to connect to localhost on 3306 it gets sent to db on 3306 and also share port 80 on the web container to the outside world?
Current docker-compose.yml:
version: "3"
services:
web:
build: .
#image: php:5.6-apache
ports:
- "8080:80"
environment:
- "APP_LOG=php://stderr"
- "LOG_LEVEL=debug"
volumes:
- .:/var/www/html
network_mode: service:db # See https://stackoverflow.com/a/45458460/95195
# networks:
# - internal
working_dir: /var/www
db:
image: mysql:5.6
ports:
- "3306:3306"
environment:
- "MYSQL_XXXXX=*****"
volumes:
- ./provision/mysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
# networks:
# - internal
networks:
internal:
driver: bridge
Current error:
ERROR: for web Cannot create container for service web: conflicting options: port publishing and the container type network mode
Yes it is possible. You need to use the network_mode option. See the below example
version: '2'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
ports:
- "80:80"
- "3306:3306"
app:
image: ubuntu:16.04
command: bash -c "apt update && apt install -y telnet && sleep 10 && telnet localhost 3306"
network_mode: service:db
outputs
app_1 | Trying 127.0.0.1...
app_1 | Connected to localhost.
app_1 | Escape character is '^]'.
app_1 | Connection closed by foreign host.
network_mode: service:db instructs docker to not assign the app services it own private network. Instead let it join the network of db service. So any port mapping that you need to do, needs to happen on the db service itself.
The way I usually use it is different, I create a base service which runs a infinite loop and the db and app service both are launched on base service network. All ports mapping need to happen at the base service.
I am trying to build my airflow using docker and rabbitMQ. I am using rabbitmq:3-management image. And I am able to access rabbitMQ UI, and API.
In airflow I am building airflow webserver, airflow scheduler, airflow worker and airflow flower. Airflow.cfg file is used to config airflow.
Where I am using broker_url = amqp://user:password#127.0.0.1:5672/ and celery_result_backend = amqp://user:password#127.0.0.1:5672/
My docker compose file is as follows
version: '3'
services:
rabbit1:
image: "rabbitmq:3-management"
hostname: "rabbit1"
environment:
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "user"
RABBITMQ_DEFAULT_PASS: "password"
RABBITMQ_DEFAULT_VHOST: "/"
ports:
- "5672:5672"
- "15672:15672"
labels:
NAME: "rabbitmq1"
webserver:
build: "airflow/"
hostname: "webserver"
restart: always
environment:
- EXECUTOR=Celery
ports:
- "8080:8080"
depends_on:
- rabbit1
command: webserver
scheduler:
build: "airflow/"
hostname: "scheduler"
restart: always
environment:
- EXECUTOR=Celery
depends_on:
- webserver
- flower
- worker
command: scheduler
worker:
build: "airflow/"
hostname: "worker"
restart: always
depends_on:
- webserver
environment:
- EXECUTOR=Celery
command: worker
flower:
build: "airflow/"
hostname: "flower"
restart: always
environment:
- EXECUTOR=Celery
ports:
- "5555:5555"
depends_on:
- rabbit1
- webserver
- worker
command: flower
I am able to build images using docker compose. However, I am not able to connect my airflow scheduler to rabbitMQ. I am getting following error:
consumer: Cannot connect to amqp://user:**#localhost:5672//: [Errno
111] Connection refused.
I have tried using 127.0.0.1 and localhost both.
What I am doing wrong ?
From within your airflow containers, you should be able to connect to the service rabbit1. So all you need to do is to change amqp://user:**#localhost:5672//: to amqp://user:**#rabbit1:5672//: and it should work.
Docker compose creates a default network and attaches services that do not explicitly define a network to it.
You do not need to expose the 5672 & 15672 ports on rabbit1 unless you want to be able to access it from outside the application.
Also, generally it is not recommended to build images inside docker-compose.
I solved this issue by installing rabbitMQ server into my system with command sudo apt install rabbitmq-server.