docker-compose creating multiple set of services - docker

I'm trying to create 3 mattermost services on 1 AWS EC2 machine.
Let me explain further with more texts:
When I run docker-compose up -d, I get a service whose structure is like this:
How can I modify the docker related scripts so that I can create 3 sets of service?
I've tried docker-compose up --scale app=3 --scale web=3 --scale db=3. But I can't find any way to specify different port for each of the App container.
The only solution I've found is:
Create 3 copies of mattermost-docker folder.
Change the App port and database connection information.
Run docker-compose up -d 3 times in mattermost-docker1, mattermost-docker2, and mattermost-docker3 separately.
But this solution creates a lot of duplicated files. I don't like it.
Anyone knows how to create 3 sets of mattermost services?

You need to specify a port range in docker compose:
For example, for 10 container scalling:
version: '3'
services:
web:
...
ports:
- "80-90:443"
app:
...
ports:
- "8000-8010"
Note that you don't need to change port inside container (443, 444, 445). You can use the same, and furthermore that's recommended, because although you use different containers in a port range, is easier if they use the same nginx configuration.

This starts sets of services using 3 separate databases (nginx and mongo used as example).
version: '3'
services:
web1:
container_name: web1
image: nginx:latest
ports:
- 8080:8080
app1:
container_name: app1
image: nginx:latest
ports:
- "8081:8081"
db1:
container_name: db1
image: mongo
ports:
- 27017
web2:
container_name: web2
image: nginx:latest
ports:
- 8082:8082
app2:
container_name: app2
image: nginx:latest
ports:
- "8083:8083"
db2:
container_name: db2
image: mongo
ports:
- 27018
web3:
container_name: web3
image: nginx:latest
ports:
- 8084:8084
app3:
container_name: app3
image: nginx:latest
ports:
- "8085:8085"
db3:
container_name: db3
image: mongo
ports:
- 27019
Local Test:
NAMES STATUS PORTS IMAGE
db2 Up About a minute 27017/tcp, 0.0.0.0:32803->27018/tcp mongo
web1 Up About a minute 80/tcp, 0.0.0.0:32802->8080/tcp nginx:latest
db1 Up About a minute 0.0.0.0:32801->27017/tcp mongo
app1 Up About a minute 80/tcp, 0.0.0.0:32800->8081/tcp nginx:latest
app3 Up About a minute 80/tcp, 0.0.0.0:32798->8085/tcp nginx:latest
db3 Up About a minute 27017/tcp, 0.0.0.0:32799->27019/tcp mongo
app2 Up About a minute 80/tcp, 0.0.0.0:32797->8083/tcp nginx:latest
web3 Up About a minute 80/tcp, 0.0.0.0:32796->8084/tcp nginx:latest
web2 Up About a minute 80/tcp, 0.0.0.0:32795->8082/tcp nginx:latest

Related

Why can't I access to my local docker-compose from Browser?

I've run my docker-compose file trying to dockerize pgadmin for Postgres but my browser cannot connect to pgadmin on url localhost:8080.
This is the docker-compose file that I am running
version: '3'
services:
db:
container_name: postgres_container
image: postgres
restart: always
environment:
POSTGRES_DB: postgres_db
POSTGRES_USER: admin
POSTGRES_PASSWORD: secret
PGDATA: /var/lib/postgresql/data
ports:
- "5432:5432"
volumes:
- db-data:/var/lib/postgresql/data
pgadmin:
container_name: pgadmin4_container
image: dpage/pgadmin4:5.5
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: admin#admin.com
PGADMIN_DEFAULT_PASSWORD: secret
PGADMIN_LISTEN_PORT: 80
ports:
- "8080:80"
volumes:
- pgadmin-data:/var/lib/pgadmin
volumes:
db-data:
pgadmin-data:
This is my docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6a6a588f639 dpage/pgadmin4:5.5 "/entrypoint.sh" 3 hours ago Up 9 minutes 443/tcp, 0.0.0.0:8080->80/tcp pgadmin4_container
ad6fe3349717 postgres "docker-entrypoint.s" 3 hours ago Up 9 minutes 0.0.0.0:5432->5432/tcp postgres_container
When I try to connect to it from browser to localhost:8080 it says Connection attempt failed
I am using almost the same docker compose file on Windows 10 with WSL 2 and can connect immediately using Firefox build 100 browser to localhost:8080.
My difference is that the image for pgadmin4 in the compose file is image: dpage/pgadmin4
the latest not v5.5
I solved this problem. Turns out that DOCKER_HOST variable was set to 192.168.99.100:2376. You can see it by running command echo $DOCKER_HOST
I just ran my docker container into this port instead of localhost and everything worked fine.
docker run -d -p 192.168.99.100:9411:9411 openzipkin/zipkin
And I was able to access my docker container through the browser by url http://192.168.99.100:9411/ Thank you very much everyone

How to deploy the same compose in different stacks and adapt container names automatically?

I am trying to design a docker-compose.yml file that will allow me to easily launch environments to develop inside of. Sometimes I would like to have 2 or more of these up at the same time but doing it naively I get ERROR: for pdb Cannot create container for service pdb: Conflict. The container name "/pdb" is already in use by container ... (even if they are on different stacks).
version: '3.4'
services:
pdb:
hostname: "pdb"
container_name: "pdb"
image: "postgres:latest"
(...other services...)
Is there a way to automatically name these in a distinguishable but systematic way? For example something like this:
version: '3.4'
services:
pdb:
hostname: "${stack_name}_pdb"
container_name: "${stack_name}_pdb"
image: "postgres:latest"
(...other services...)
EDIT: Apparently this is a somewhat service specific question so here is the complete compose file just in case...
version: '3.4'
services:
rmq:
hostname: "rmq"
container_name: "rmq"
image: "rabbitmq:latest"
networks:
- "fakenet"
ports:
- "5672:5672"
healthcheck:
test: "rabbitmq-diagnostics -q ping"
interval: 30s
timeout: 30s
retries: 3
pdb:
hostname: "pdb"
container_name: "pdb"
image: "postgres:latest"
networks:
- "fakenet"
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: ******
POSTGRES_USER: postgres
POSTGRES_DB: test_db
volumes:
- "./deploy/pdb:/docker-entrypoint-initdb.d"
- "./data/dbase:/var/lib/postgresql/data"
healthcheck:
test: ["CMD-SHELL", "pg_isready"]
interval: 10s
timeout: 5s
retries: 5
workenv:
hostname: "aiida"
container_name: "aiida"
image: "aiida_workenv:v0.1"
expose:
- "8888" # AiiDa Lab
- "8890" # Jupyter Lab
- "5000" # REST API
ports: # local:container
- 8888:8888
- 8890:8890
- 5000:5000
volumes:
- "./data/codes:/home/devuser/codes"
- "./data/aiida:/home/devuser/.aiida"
depends_on:
pdb:
condition: service_healthy
rmq:
condition: service_healthy
networks:
- "fakenet"
environment:
LC_ALL: "en_US.UTF-8"
LANG: "en_US.UTF-8"
PSQL_HOST: "pdb"
PSQL_PORT: "5432"
command: "tail -f /dev/null"
networks:
fakenet:
driver: bridge
Just don't manually set container_name: at all. Compose will automatically assign a name based on the current project name. Similarly, you don't usually need to set hostname: (RabbitMQ is one extremely specific exception, if you're using that).
If you do need to publish ports out of your Compose setup to be able to access them from the host system, the other obvious pitfall is that the first ports: number must be unique across the entire host. You can specify a single number for ports: to let Docker pick the host port, though you'll need to look it up later with docker-compose port.
version: '3.8'
services:
pdb:
image: "postgres:latest"
# no hostname: or ports:
app:
build: .
environment:
PGHOST: pdb
ports:
- 3000 # container-side port, Docker picks host port
docker-compose -p myname -d up
docker-compose -p myname port app 3000

How to add image name to docker-compose up

I have a docker-compose file that I use the image block in the service to name. For example
version: '3'
services:
redis:
image: redis
restart: unless-stopped
ports:
- "6379"
worker:
image: worker:production
build: .
user: root
command: celery -A ExactEstate worker --loglevel=info
env_file: ./.env.prod
restart: unless-stopped
links:
- redis
depends_on:
- redis
beats:
image: beats:production
build: .
user: root
command: celery --pidfile= -A ExactEstate beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
env_file: ./.env.prod
restart: unless-stopped
links:
- redis
depends_on:
- redis
web:
image: web:production
build: .
user: root
command: daphne -b 0.0.0.0 -p 8000 ExactEstate.asgi:application
ports:
- "8000:8000"
env_file: ./.env.prod
restart: unless-stopped
links:
- redis
- worker
- beats
depends_on:
- redis
- worker
- beats
This gives a docker ps of:
0ad78269a9ce beats:production "celery --pidfile= -…" 7 minutes ago Up 7 minutes exactestate_beats_1
1a44f7c98b50 worker:production "celery -A ExactEsta…" 7 minutes ago Up 7 minutes exactestate_worker_1
f3a09723ba66 redis "docker-entrypoint.s…" 7 minutes ago Up 7 minutes 0.0.0.0:32769->6379/tcp exactestate_redis_1
Let's suppose I also have built containers from a different compose file (i.e. staging) How can I use docker-compose to on pull up the exact service/image I want?
For example: docker-compose up web:production or docker-compose up web:staging
You can achieve this by using environment variables. Variables for docker-compose up could be passed as .env file or set by export (or set on Windows) command (docker documentation).
web:
image: web:production
Should be changed to
web:
image: web:${ENV}
And then you can run your application by running
$ export ENV=production && docker-compose up
Or you can create .env file containing line ENV=production. Then you can simply run application with docker-compose up.

How to setup Redis Commander and Docker?

I've tried the 'with docker' docs here but it's not working from the localhost:7000, localhost:8081, or any other port I use. What am I missing?
REDIS_PORT=6379
### Redis ################################################
redis:
container_name: redis
hostname: redis
build: ./redis
volumes:
- ${DATA_PATH_HOST}/redis:/data
ports:
- "${REDIS_PORT}:6379"
networks:
- backend
### REDISCOMMANDER ################################################
redis-commander:
container_name: rediscommander
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379
ports:
- "7000:80"
networks:
- frontend
- backend
depends_on:
- redis
Docker ps gives me:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
042c9a2e918a rediscommander/redis-commander:latest "/usr/bin/dumb-init …" About a minute ago Up About a minute (healthy) 8081/tcp, 0.0.0.0:7000->80/tcp rediscommander
86bc8c1ca5ff laradock_redis "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:6379->6379/tcp redis
Docker logs rediscommander gives me:
$ docker logs rediscommander
Creating custom redis-commander config '/redis-commander/config/local-production.json'.
Parsing 1 REDIS_HOSTS into custom redis-commander config '/redis-commander/config/local-production.json'.
node ./bin/redis-commander
Using scan instead of keys
No Save: false
listening on 0.0.0.0:8081
access with browser at http://127.0.0.1:8081
Redis Connection redis:6379 using Redis DB #0
Redis commader is listening on port 8081 in container. That is why you should change port binding to
ports:
- "7000:8081"
in redis commander block and access it via localhost:7000.

How to name a volume using a docker-compose.yml file?

I'm new to Docker and I'm trying to find out how to set the name of the created data volume. Currently the directory is automatically named as a long hash under /var/libs/docker which is far from user friendly.
I'm attempting to set up a development environment for MODX as shown here:
https://github.com/modxcms/docker-modx
Currently my docker-compose.yml file is as follows:
web:
image: modx
links:
- db:mysql
ports:
- 80:80
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: example
ports:
- 3306:3306
command: mysqld --sql-mode=NO_ENGINE_SUBSTITUTION
myadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- 8080:8080
This works perfectly but I'm unsure as to how to name the data volume that I would edit directly with my IDE.
(As a side question, does it have to be created under /var/libs/docker ? Or is there a way of setting it to a directory in my home folder?)
Update:
Thanks to the help from #juliano I've updated my docker-compose.yml file to:
version: '2'
services:
web:
image: modx
volumes:
- html:/home/muzzstick/dev/modxdev
links:
- db:mysql
ports:
- 80:80
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: example
ports:
- 3306:3306
command: mysqld --sql-mode=NO_ENGINE_SUBSTITUTION
myadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- 8080:8080
volumes:
html:
external: false
Unfortunately this seems to stop the web container from running.
db and myadmin containers show they're running ok.
There weren't any errors... if I type docker start docker_web_1 it appears to start but docker ps -a shows it exited as soon as it started.
Update 2
Running docker-compose up -d appears to run without issue. But then as you can see below, the web container exits as soon as it's created.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a1dd6d8ac94e modx "/entrypoint.sh apach" 10 seconds ago Exited (1) 5 seconds ago docker_web_1
ee812ae858dc phpmyadmin/phpmyadmin "/run.sh phpmyadmin" 10 seconds ago Up 5 seconds 80/tcp, 0.0.0.0:8080->8080/tcp docker_myadmin_1
db496134e0cf mysql "docker-entrypoint.sh" 11 seconds ago Up 10 seconds 0.0.0.0:3306->3306/tcp docker_db_1
Update 3
OK the error logs for this container shows:
error: missing MODX_DB_HOST and MYSQL_PORT_3306_TCP environment variables
Did you forget to --link some_mysql_container:mysql or set an external db
with -e MODX_DB_HOST=hostname:port?
This error appears to be originating from https://github.com/modxcms/docker-modx/blob/master/apache/docker-entrypoint.sh#L15-L20
Could it be something like linking is handled differently in docker-compose version 2?
To create a named data volume using the version 2 of compose files you will have a separated area:
version: '2'
services:
db:
image: postgres
volumes:
- amazingvolume:/var/lib/postgresql/data
volumes:
amazingvolume:
external: true
So you can define the volume name (amazingvolume), if it's external or not and under your service (db in this example) you can define which directory you gonna mount.
Just search in the docker documentation for hosted mounted volumes:
version: '2'
services:
web:
image: modx
environment:
- MYSQL_PORT_3306_TCP=3306
- MODX_DB_HOST=mysql:3306
volumes:
- /home/muzzstick/dev/modxdev/html:/var/www/html
links:
- db:mysql
ports:
- 80:80
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: example
ports:
- 3306:3306
command: mysqld --sql-mode=NO_ENGINE_SUBSTITUTION
myadmin:
image: phpmyadmin/phpmyadmin
links:
- db:db
ports:
- 8080:8080
Change /var/www/html to the directory where the html files will be inside the container. And also create the directory at the left in your host and give read permission to all users.
Regards

Resources