What if have 10 instance of any container which needs presistant storage, so on 10 instance how docker will manage volume for them, i've defined volume in docker-compose.yml
i didn't find anything regarding this that what will happen when multiple instance will run?
1. will docker create new folders for each instance or
2. share same folder to all of them (this will lead data corrption)?
here is my sample docker-compose.yml
version: '2'
services:
consul:
#image: myappteam/consul:3.4.0
build: ./consul
container_name: consul
hostname: consul
domainname: consul
restart: always
volumes:
- myapp-data:/data/consul
consului:
#image: myappteam/consul-ui:3.4.0
build: ./consul-ui
container_name: consul-ui
hostname: consul-ui
domainname: consul-ui
ports:
- 8500:8500
restart: always
volumes:
- myapp-data:/data/consului
nginx:
#image: myappteam/nginx:3.4.0
build: ./nginx
container_name: nginx
hostname: nginx
domainname: nginx
ports:
- "80:80"
volumes:
- myapp-logs:/logs/nginx_access_logs
- myapp-logs:/logs/nginx_error_logs
restart: always
volumes:
myapp-data:
myapp-logs:
myapp-bundle:
myapp-source:
so in above example, myapp-data is plan i want to have all data so i was thinking when i'll increase instance of nginx, consul, will they use same myapp-data volume or create new volume? because if it will use same instance then data will corrupted because two instance will write same files..
so in that case what should i do?
Related
I have two conteiners:
docker-compose.yml
version: '3.8'
services:
db:
image: postgres:14.1
container_name: postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
......
network_mode: bridge
web:
container_name: web
build: .
........
network_mode: bridge
external_links:
- postgres
depends_on:
- db
volumes:
postgres_data:
name: postgres_data
After docker-compose up, when i recreate only one container - "db", all works, but i can not connect to conteiner "web", i get error: "Failure
Cannot link to a non running container: /postgres AS /web/postgres".
In conteiner "web" i call db as host=postgres.
What am I doing wrong?
The external_links: setting is obsolete and you don't need it. You can just remove it with no adverse consequences.
network_mode: bridge and container_name: are also unnecessary, though they shouldn't specifically cause problems; still, I'd delete them. What you show can be reduced to
version: '3.8'
services:
db:
image: postgres:14.1
volumes:
- postgres_data:/var/lib/postgresql/data/
......
web:
build: .
........
depends_on:
- db
volumes:
postgres_data: # empty
Since Compose creates a network named default for you and attaches containers to it, your application container can still reach the database container using the hostname db. Networking in Compose in the Docker documentation describes this further.
Problem:
When having two docker-compose files / projects with the same services, under the same network, when you spin up t he second compose project, the DNS name for the service gets overwritten.
eg:
App 1
version: "3.1"
services:
db:
image: mysql:8.0
container_name: monolith-db
networks:
- my-network-name
webserver:
image: nginx:alpine
container_name: monolith-webserver
networks:
- my-network-name
phpfpm:
container_name: monolith-phpfpm
networks:
- my-network-name
networks:
my-network-name:
external: true
App 2
version: "3.1"
services:
db:
image: mysql:8.0
container_name: ms-auth-db
networks:
- my-network-name
webserver:
image: nginx:alpine
container_name: ms-auth-webserver
networks:
- my-network-name
phpfpm:
container_name: ms-auth-phpfpm
networks:
- my-network-name
networks:
my-network-name:
external: true
If you start App 1, the services inside can connect to their declared services by service name as hostname, for example, in my config I have database-host: db
However, when I do docker-compose -p ms-auth --env-file .env -f infra/local/docker-compose.yml up -d then db hostname now points to App 2's db service.
The solution is to use the container_name as hostname
e.g. instead of connecting to db, configure App 1' config files to use the hostname monolith-db, and for pointing from App 1 to App 2, also use container name as hostname, e.g. ms-auth-host: ms-auth-webserver
I have two containers defined in a docker-compose yaml file that need to talk to each other, but they can't.
version: "3.9"
networks:
localdev:
driver: 'bridge'
services:
master-db:
image: mysql:8.0
container_name: master-db
hostname: master-db
command: --default-authentication-plugin=mysql_native_password
restart: always
ports:
- "4000:3306"
networks:
- localdev
page-store:
hostname: page-store
build:
context: .
dockerfile: Dockerfile.page_store
container_name: page-store
ports:
- "2020:2020"
networks:
- localdev
links:
- master-db
In the page-store Python Flask microservice, I try to access the MySQL database by using its hostname of master-db, but the name cannot resolve.
You should be able to connect each other using respective service names. master-db and page-store removing hostname
As per Official guide you may have to define master-db,page-store in container's /etc/hosts, if you want to use hostname: page-store etc.
Please refer this SO thread.
Also using --links may not be the best option.
I already have a docker-compose.yml file like this:
version: "3.1"
services:
memcached:
image: memcached:alpine
container_name: dl-memcached
redis:
image: redis:alpine
container_name: dl-redis
mysql:
image: mysql:5.7.21
container_name: dl-mysql
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: dl-phpmyadmin
environment:
- PMA_ARBITRARY=1
- PMA_HOST=dl-mysql
- PMA_PORT=3306
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
restart: always
ports:
- 8002:80
volumes:
- /application
links:
- mysql
elasticsearch:
build: phpdocker/elasticsearch
container_name: dl-es
volumes:
- ./phpdocker/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "8003:9200"
webserver:
image: nginx:alpine
container_name: dl-webserver
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/nginx/nginx.conf:/etc/nginx/conf.d/default.conf
- ./logs:/var/log/nginx:delegated
ports:
- "9003:80"
php-fpm:
build: phpdocker/php-fpm
container_name: dl-php-fpm
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/php-fpm/php-ini-overrides.ini:/etc/php/7.2/fpm/conf.d/99-overrides.ini
- ./../docker/php-fpm/certs/store_stock/:/usr/local/share/ca-certificates/
- ./logs:/var/log:delegated # nginx logs
- /application/var/cache
environment:
XDEBUG_CONFIG: remote_host=host.docker.internal
PHP_IDE_CONFIG: "serverName=dl"
node:
build:
dockerfile: dl/phpdocker/node/Dockerfile
context: ./../
container_name: dl-node
working_dir: /application
ports:
- "8008:3000"
volumes:
- ./../:/application:cached
tty: true
My goal is to have 2 isolate environments working at the same time in the same server with the same docker-compose file? I wonder if it's possible?
I want to be able to stop and update one env. while the other one is still running and getting the traffic.
Maybe I need another approach in my case?
There are a couple of problems with what you're trying to do. If your goal is to put things behind a load balancer, I think that rather than trying to start multiple instances of your project, a better solution would be to use the scaling features available to docker-compose. In particular, if your goal is to put some services behind a load balancer, you probably don't want multiple instances of things like your database.
If you combine this with a dynamic front-end proxy like Traefik, you can make the configuration largely automatic.
Consider a very simple example consisting of a backend container running a simple webserver and a traefik frontend:
---
version: "3"
services:
webserver:
build:
context: web
labels:
traefik.enable: true
traefik.port: 80
traefik.frontend.rule: "PathPrefix:/"
frontend:
image: traefik
command:
- --api
- --docker
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
ports:
- "80:80"
- "127.0.0.1:8080:8080"
If I start it like this, I get a single backend and a single frontend:
docker-compose up
But I can also ask docker-compose to scale out the backend:
docker-compose up --scale webserver=3
In this case, I get a single frontend and three backend servers. Traefik will automatically discover the backends and will round-robin connections between them. You can download this example and try it out.
Caveats
There are a few aspects of your configuration that would need to change in order to make this work (and in fact, you would need to change them even if you were to create multiple instances of your project as you have proposed in your question).
Conflicting paths
Take for example the configuration of your webserver container:
volumes:
- ./logs:/var/log/nginx:delegated
If you start two instances of this service, both containers will mount ./logs on /var/log/nginx. If they both attempt to write to /var/log/nginx/access.log, you're going to have problems.
The easiest solution here is to avoid bind mounts for things like log directories (and any other directories to which you will be writing), and instead use named docker volumes.
Hardcoding container names
In some places, you are hardcoding the container name, like this:
mysql:
image: mysql:5.7.21
container_name: dl-mysql
This will cause problems if you attempt to start multiple instances of this project or multiple instances of the mysql container. Don't statically set the container name.
Deprecated links syntax
Your configuration is using the deprecated links syntax:
links:
- mysql
Don't do that. In modern docker, containers on the same network can simply refer to each other by name. In other words, if your compose configuration has:
mysql:
image: mysql:5.7.21
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
Other containers in your compose stack can simply use the hostname mysql to refer to this service.
You won't be able to run same compose file on a host without changing the port mappings because that will cause port conflict. I'd recommend creating a base compose file and using extends to override port mappings for different environments.
I am deploying a small stack onto a UCP
One of the issues I am facing is naming the container for service1.
I need to have a static name for the container, since it's utilized by mycustomimageforservice2
The container_name option is ignored when deploying a stack in swarm mode with a (version 3) Compose file.
I have to use version: 3 compose files.
version: "3"
services:
service1:
image: dockerhub/service1
ports:
- "8080:8080"
container_name: service1container
networks:
- mynet
service2:
image: myrepo/mycustomimageforservice2
networks:
- mynet
restart: on-failure
networks:
mynet:
What are my options?
You can't force a containerName in compose as its designed to allow things like scaling a service (by updating the number of replicas) and that wouldn't work with names.
One service can access the other using servicename (http://serviceName:internalServicePort) instead and docker will do the rest for you (such as resolving to an actual container address, load balancing between replicas....).
This works with the default network type which is overlay
You can face your problem linking services in docker-compose.yml file.
Something like:
version: "3"
services:
service1:
image: dockerhub/service1
ports:
- "8080:8080"
networks:
- mynet
service2:
image: myrepo/mycustomimageforservice2
networks:
- mynet
restart: on-failure
links:
- service1
networks:
mynet:
Using links arguments in your docker-compose.yml you will allow some service to access another using the container name, in this case, service2 would establish a connection to service1 thanks to the links parameter. I'm not sure why you use a network but with the links parameter would not be necessary.
container_name option is ignored when deploying a stack in swarm mode since container names need to be unique.
https://docs.docker.com/compose/compose-file/#container_name
If you do have to use version 3 but don't work with swarms, you can add --compatibility to your commands.
Specify a custom container name, rather than a generated default name.
container_name: my-web-container
see this in the full docker-compose file
version: '3.9'
services:
node-ecom:
build: .
image: "node-ecom-image:1.0.0"
container_name: my-web-container
ports:
- "4000:3000"
volumes:
- ./:/app:ro
- /app/node_modules
- /config/.env
env_file:
- ./config/.env
know more