Today I switched from "Docker Toolbox" to "Docker for Mac", because Docker now has finally write-access to my User directory (which doesn't worked with "Docker Toolbox") - Yay!
But this change also includes that all containers now running under my localhost and not under Docker's IP as before (e.g. 192.168.99.100).
Since my localhost listens to various ports by default (80, 443, ...) and I don't want to always add new created ports, that doesn't conflict with the standard one's, to my local dev domains (e.g. example.dev:8443), I wonder how to run my containers as before.
I read about network configs and tried a lot of things (creating a new host network, exposing ports with an IP in front of it, ...), but didn't got it working.
What kind of config do I need to run my app container with the IP 192.168.99.100? Thats my docker-compose.yml so far.
version: '2'
services:
app:
build:
context: .
dockerfile: Dockerfile
depends_on:
- mysql
- redis
- memcached
ports:
- 80:80
- 443:443
- 22:22
- 3000:3000
- 3001:3001
volumes:
- ./app/:/app/
- /tmp/debug/:/tmp/debug/
- ./:/docker/
volumes_from:
- storage
# cap and privileged needed for slowlog
cap_add:
- SYS_PTRACE
privileged: true
env_file:
- etc/environment.yml
- etc/environment.development.yml
mysql:
build:
context: docker/mysql/
dockerfile: MariaDB-10
ports:
- 3306:3306
volumes_from:
- storage
volumes:
- ./data/mysql:/var/lib/mysql
- /tmp/debug/:/tmp/debug/
env_file:
- etc/environment.yml
- etc/environment.development.yml
redis:
build: docker/redis/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
memcached:
build: docker/memcached/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
storage:
build: docker/storage/
volumes:
- /storage
You need to declare "networks:" for each of your services:
e.g.
version: '2'
services:
app:
image: xxxx:xxx
ports:
- "80:80"
networks:
- my-network
mysql:
image: xxxx:xxx
networks:
- my-network
networks:
my-network:
driver: bridge
Then from side your app configuration, you can use "mysql" as the hostname of database server.
You can define a network in your compose file, then add any services to the network.
https://docs.docker.com/compose/networking/
But I would suggest you just use different ports now that you are running natively. I.e. 8080:80
Related
I have 3 docker-compose files. One to start the kafka and the other two are consumer and producer. Added external_links in the other docker-compose files to kafka, but still unable to access kafka from inside containers. From outside the container, I can access through localhost:9092, but what about inside docker container.
# docker-compose1.yml
version: "3.6"
services:
zookeeper:
image: 'docker.io/bitnami/zookeeper:3.7'
container_name: zookeeper
ports:
- '2181:2181'
volumes:
- 'zookeeper_data:/bitnami'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'docker.io/bitnami/kafka:3'
container_name: kafka
ports:
- '9092:9092'
volumes:
- 'kafka_data:/bitnami'
environment:
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_LISTENERS=PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
- KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
- KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_ADVERTISED_HOST_NAME=localhost
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_AUTO_CREATE_TOPICS_ENABLE=true
depends_on:
- zookeeper
volumes:
zookeeper_data:
external: true
kafka_data:
external: true
# docker-compose2.yml
version: "3.6"
services:
web:
hostname: ocp-transmitter
image: 'ocp/transmitter'
command: bash -c "bundle install && foreman start"
ports:
- '3000:3000'
volumes:
- .:/app:cached
stdin_open: true
tty: true
external_links:
- kafka
First, remove these, they are deprecated
- KAFKA_ADVERTISED_HOST_NAME=localhost
- KAFKA_ADVERTISED_PORT=9092
Second, read the bitnami image documentation more carefully, all the Kafka properties start with KAFKA_CFG_, then read the section about internal/external listeners
The linked answer(s) are correct Communication between multiple docker-compose projects
Run docker network create with a name to setup an external bridge network separately from Compose, then add networks section to each service in that network (Zookeeper, Kafka, and your Kafka clients). Then make sure it's external
networks:
example-net:
external: true
Then you'd use kafka:29092 in your apps, not localhost, and not port 9092
So, I've setup several container apps that use MariaDB as their db backend, using docker-compose.
Containers are setup as needed and therefore MariaDB gets installed each time on every container that uses the db.
For example, I have some containers (PHPMyAdmin, NGiNX-PM, etc.) that use MariaDB, and they, in turn, have a version of it installed within their container. I also have a separate container (MariaDB) that I would rather have shared amongst the other containered apps and, thereby, I'd only have to maintain one version of the db.
I've searched for a solution, but no luck. Needless to say, I'm a noob at docker.
The only thing I can come up with is that all the apps need to be installed through the same docker-compose.yaml file to use the same db? That would make for a very long file if I had many containers running, and I'd prefer to have a directory per app and all the app's contents available in this one location.
I'm sure there is a way, I just haven't been able to figure it out.
So this is what I've tried:
The following setup is what I've tried but I am unable to get it to work:
(/docker/apps/mariadb/mariadb.yml)
version: '3.9'
networks:
NET:
external: true
services:
#############################################################################################
# MariaDB (docker-compose -f mariadb.yml up -d) #
#############################################################################################
mariadb:
image: jsurf/rpi-mariadb:latest
restart: unless-stopped
environment:
- TZ=${TIMEZONE}
- MYSQL_DATABASE=dockerApps
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
volumes:
- $HOME/docker/apps/mariadb/db:/var/lib/mysql
expose:
- '3306'
networks:
- NET
(/docker/apps/nginxpm/nginxpm.yml)
version: '3.9'
networks:
NET:
external: true
services:
#############################################################################################
# NGiNX Proxy Manager (docker-compose -f nginxpm.yml up -d) #
#############################################################################################
nginxpm:
container_name: NGiNX_Proxy_Manager
image: 'jc21/nginx-proxy-manager:latest'
ports:
- '80:80'
- '81:81'
- '443:443'
volumes:
- ./config.json:/app/config/production.json
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
networks:
- NET
depends_on:
- mariadb
(/docker/apps/phpmyadmin/phpmyadmin.yml)
version: "3.9"
networks:
NET:
external: true
services:
#############################################################################################
# phpMyAdmin (docker-compose up -d -OR- docker-compose -f phpmyadmin.yml up -d) #
#############################################################################################
phpmyadmin:
image: phpmyadmin:latest
container_name: phpMyAdmin
restart: unless-stopped
environment:
PMA_HOST: mariadb
PMA_USER: root
PMA_PASSWORD: ${MYSQL_PASSWORD}
volumes:
# Must add ServerName directive to end of file "ServerName 127.0.0.1"
- $HOME/docker/apps/phpmyadmin/apache2.conf:/etc/apache2/apache2.conf
ports:
- '8004:80'
networks:
- NET
Any help in this matter is greatly appreciated.
Ok, so after some more reading and testing, I've found the answer to my issue. I was assuming that "depends_on" was supposed to connect the containers, somehow. Not true!
I found that "external_links" is the correct way of connecting them.
So, my final docker-compose file looks like this:
(/docker/apps/nginxpm/nginxpm.yml)
version: '3.9'
networks:
NET:
external: true
services:
#############################################################################################
# NGiNX Proxy Manager (docker-compose -f nginxpm.yml up -d) #
#############################################################################################
nginxpm:
container_name: NGiNX_Proxy_Manager
image: 'jc21/nginx-proxy-manager:latest'
ports:
- '80:80'
- '81:81'
- '443:443'
volumes:
- ./config.json:/app/config/production.json
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
networks:
- NET
external_links:
- mariadb
I use docker compose to build up the services, i want to set the mysql to be static, i tried to use networks directive but it didn't work and the errors say the ip already occupied, every this i restart the windows, the mysql ip was changed, sometimes 172.18.0.3,or 172.18.0.4, anyone know how to assign a static ip to the container? here is the yml
version: '3.7'
services:
nginx:
container_name: hki_nginx
image: nginx:latest
ports:
- 80:80
- 4433:443
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
- ./nginx/conf:/etc/nginx/conf.d
- ./src:/var/www
links:
- php
- php72
php:
container_name: hki_php
image: php:5.6-fpm-ext1
volumes:
- ./src:/var/www
- ./php/php.ini:/usr/local/etc/php/php.ini
- ./php/php-fpm.conf:/usr/local/etc/php-fpm.d/www.conf
#- ./php/phpfpm/:/usr/local/etc/php-fpm.d/
php72:
container_name: web_php
image: php:7.2-fpm-ext2
volumes:
- ./src:/var/www
- ./php72/php.ini:/usr/local/etc/php/php.ini
- ./php72/php-fpm.conf:/usr/local/etc/php-fpm.d/www.conf
#- ./php/phpfpm/:/usr/local/etc/php-fpm.d/
mysql:
container_name: hki_mysql
image: mysql:5.7
volumes:
- ./mysql/data:/var/lib/mysql
- ./mysql/my.cnf:/etc/mysql/conf.d/my.cnf
- ./mysql/init:/docker-entrypoint-initdb.d/
ports:
- 3306:3306
environment:
- MYSQL_ROOT_PASSWORD=*Abcd1234
- MYSQL_USER=abc
- MYSQL_PASS=*Abcd1234
#networks:
#default:
#ipv4_address: 172.18.0.3
This is because you commented the part that assigns a static IP to your mysql container
mysql:
...
#networks:
#default:
#ipv4_address: 172.18.0.3
If you take away the #, it will have a static IP.
And you might have forgotten the top-level network section in your docker-compose.yml as the official doc setting static IP states
networks:
app_net:
ipam:
driver: default
config:
- subnet: 172.18.0.0/24
I have the following docker-compose configuration:
version: '2'
services:
nginx:
image: 'nginx:latest'
expose:
- '80'
- '8080'
container_name: nginx
ports:
- '80:80'
- '8080:8080'
volumes:
- '/home/ubuntu/nginx.conf:/etc/nginx/nginx.conf'
networks:
- default
restart: always
inmates:
image: 'xxx/inmates:mysql'
container_name: 'inmates'
expose:
- '3000'
env_file: './inmates.env'
volumes:
- inmates_documents_images:/data
- inmates_logs:/logs.log
networks:
- default
restart: always
we19:
image: 'xxx/we19:dev'
container_name: 'we19'
expose:
- '3000'
env_file: './we19.env'
volumes:
- we19_logs:/logs.log
networks:
- default
restart: always
desktop:
image: 'xxx/desktop:dev'
container_name: 'desktop'
expose:
- '3000'
env_file: './desktop.env'
volumes:
- desktop_logs:/logs.log
networks:
- default
restart: always
volumes:
inmates_documents_images:
inmates_logs:
desktop_logs:
we19_logs:
Assume I did docker-compose up -d --buiild.
Now the 4 containers (services) are runnig.
Now, I want to update ./desktop.env file with new content. Is there any possible way to reset only desktop container with the new env file? Or docker-compose restart is neccessary?
Basically I'm trying to restart only desktop container with the new env file but keep all 3 others container up running without restarting them.
Extract from docker-compose up --help
[...]
If there are existing containers for a service, and the service's configuration or image was changed after the container's creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes). To prevent Compose from picking up changes, use the --no-recreate flag.
[...]
Usage: up [options] [--scale SERVICE=NUM...] [SERVICE...]
[...]
The following command should do the trick in your case.
docker-compose up -d desktop
If not, see the documentation for other options you can use to meet your exact requirement (e.g. --force-recreate, --renew-anon-volumes, ...)
in docker-compose.yml,
What is the difference between in following ports notations?
ports:
- "5000:5000"
resp:
ports:
- "8080"
or no ports at all.
For example in following docker-compose.yml, the mongodb service must be exposing a port to communicate with node service, but no port is specified
services:
node:
build:
context: .
dockerfile: node.dockerfile
ports:
- "3000:3000"
networks:
- nodeapp-network
depends_on:
- mongodb
mongodb:
image: mongo
networks:
- nodeapp-network
networks:
nodeapp-network:
driver: bridge
source: https://github.com/DanWahlin/NodeExpressMongoDBDockerApp
However in these docker-compose.yml, there are ports awlays specified with either 27017:27017 or 8080 notation.
services:
nginx:
container_name: nginx
image: ${DOCKER_ACCT}/nginx
build:
context: .
dockerfile: .docker/nginx.${APP_ENV}.dockerfile
links:
- node1:node1
- node2:node2
- node3:node3
ports:
- "80:80"
- "443:443"
networks:
- codewithdan-network
node1:
container_name: node-codewithdan-1
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
node2:
container_name: node-codewithdan-2
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
node3:
container_name: node-codewithdan-3
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
mongo:
container_name: mongo
image: ${DOCKER_ACCT}/mongo
build:
context: .
dockerfile: .docker/mongo.dockerfile
ports:
- "27017:27017"
env_file:
- ./.docker/env/mongo.${APP_ENV}.env
networks:
- codewithdan-network
redis:
container_name: redis
image: ${DOCKER_ACCT}/redis
build:
context: .
dockerfile: .docker/redis.${APP_ENV}.dockerfile
ports:
- "6379"
networks:
- codewithdan-network
networks:
codewithdan-network:
driver: bridge
source: https://github.com/DanWahlin/CodeWithDanDockerServices
Can you explain the difference?
Typical Docker containers run a long-running server listening on some TCP port. Other containers on the same Docker network can reach that container using the container’s name (docker run --name, container_name: directive) as a DNS name and the port the server is running on. In Docker Compose, Compose creates a Docker network per Compose YAML file, and also makes services available under their key in the YAML file. This works even if no ports: are specified.
So, for instance, if your docker-compose.yml file says
services:
mongo:
image: mongo
others:
env:
MONGODB_HOST: mongo
MONGODB_PORT: 27017
then the MongoDB container will be reachable on that host name and (default) port, even though it doesn’t explicitly have a ports:.
If you do declare a ports: then the container will be reachable from outside Docker space. If you only have one port it’s the port number of the server, and Docker picks the host port; this isn’t useful in most cases (but it’s guaranteed to not hit a port conflict). If you have two ports they’re the host port and internal service port. You can also specify a host IP address to bind(2) to.
Presence or absence of ports: doesn’t affect inter-dontainer communication. Always use the container’s name (or Docker-compose.yml service name) and the “internal” port number the server is listening on.
Either specify both ports (HOST:CONTAINER), or just the container port (an ephemeral host port is chosen). So in your case 8080 is container port
ports:
- "3000"
- "3000-3005"
- "8000:8000"
- "9090-9091:8080-8081"
- "49100:22"
- "127.0.0.1:8001:8001"
- "127.0.0.1:5000-5010:5000-5010"
- "6060:6060/udp"
From here
The ephemeral port range is configured by /proc/sys/net/ipv4/ip_local_port_range kernel parameter, typically ranging from 32768 to 61000.
Either way, you should be able to peek at what Docker has accomplished in your network stack by examining your NAT tables. from here
In docker compose by default no ports will be created in case they collide with already opened ports