Docker Compose merge arrays for YAML aliases and anchors - docker

I have the following broken docker-compose file
version: '3.4'
x-vols1: &vols-1
- /home/:/home/
x-vols2: &vols-2
- /tmp/:/tmp/
services:
app1:
container_name: app1
image: app1
volumes:
<<: *vols-1
app2:
container_name: app2
image: app2
volumes:
<<: *vols-1
<<: *vols-2
This fails with the following error
$ docker-compose -f test.yaml config
ERROR: yaml.constructor.ConstructorError: while constructing a mapping
in "./test.yaml", line 14, column 13
expected a mapping for merging, but found scalar
in "./test.yaml", line 4, column 7
Question 1: How can I merge arrays in docker-compose? The syntax that I am trying to use is the one for merging dicts
Question 2: If there is no way to merge arrays, is there a workaround?
Use case: I have multiple services, some of them map some volumes, others map other volumes, others map all volumes. I would like to not repeat myself.
Thank you!

The Yaml merge syntax is for merging mappings, not for arrays. For more on that, see this issue. However, if you are just adding single volumes, you don't need to merge anything. Just insert the alias as an array entry:
version: '3.4'
x-vols1: &vols-1
"/home/:/home/"
x-vols2: &vols-2
"/tmp/:/tmp/"
services:
app1:
container_name: app1
image: app1
volumes:
- *vols-1
app2:
container_name: app2
image: app2
volumes:
- *vols-1
- *vols-2

The desired behavior can be achieved by using multiple docker-compose files, one for each volume. Note that the anchors and aliases are not required, but keeping them in to align with the question.
base.yaml
version: '3.4'
services:
app1:
container_name: app1
image: app1
app2:
container_name: app2
image: app2
vol1.yaml
version: '3.4'
x-vols1: &vols-1
volumes:
- /home/:/home/
services:
app1:
container_name: app1
image: app1
<<: *vols-1
app2:
container_name: app2
image: app2
<<: *vols-1
vol2.yaml
version: '3.4'
x-vols2: &vols-2
volumes:
- /tmp/:/tmp/
services:
app2:
container_name: app2
image: app2
<<: *vols-2
Verify as
$ docker-compose -f base.yaml -f vol1.yaml -f vol2.yaml config
Result
services:
app1:
container_name: app1
image: app1
volumes:
- /home:/home:rw
app2:
container_name: app2
image: app2
volumes:
- /home:/home:rw
- /tmp:/tmp:rw
version: '3.4'
Additional documentation https://docs.docker.com/compose/extends/

This is a slight variation on the currently accepted answer, which doesn't require you to change your docker-compose command line to include other files:
docker-compose.yml
version: '3.9'
services:
app1:
container_name: app1
image: app1
extends:
file: common-volumes.yml
service: app1
app2:
container_name: app2
image: app2
extends:
file: common-volumes.yml
service: app2
volumes:
- /tmp/:/tmp/
common-volumes.yml
x-vols1: &vols-1
volumes:
- /home/:/home/
services:
app1:
<<: *vols-1
app2:
<<: *vols-1
To view the resulting config:
docker compose config

Related

Converting a docker-compose.yml file with Kompose to K8S with a build step

I've the following docker-compose.yml file, I'd like to convert it to K8S configuration files.
version: '3.1'
services:
api-fiber:
build:
context: .
dockerfile: Dockerfile
ports:
- 8666:8666
volumes:
- ./:/var/app/current
networks:
- api-network
redis:
image: redis:latest
container_name: redis
restart: always
ports:
- 6379:6379
networks:
api-network:
driver: bridge
Kompose converts the resources to yaml, but skip the build section of the api-fiber specs. How to reference this step to make Kompose aware of it?

Need to up my docker file in ubuntu "docker-compose up -d"

services:
postgres:
container_name: 'lh-postgres'
image: 'postgres:13'
environment:
POSTGRES_PASSWORD: root
redis:
container_name: 'lh-redis'
image: 'redis:6'
nginx:
container_name: 'lh-nginx'
build: ./nginx
depends_on:
- php-fpm
volumes:
- ./src/lh-app:/var/www/html/app
- ./src/lh-api:/var/www/html/api
ports:
- "80:80"
- "443:443"
php-fpm:
container_name: 'lh-php'
image: docker.io/bitnami/php-fpm:8.0
user: '1000:1000'
build:
context: ./php-fpm
args:
- PHP_ENV= development
depends_on:
- postgres
- redis
volumes:
- ./src/lh-app:/var/www/html/app
- ./src/lh-api:/var/www/html/api
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services: 'postgres'
getting this error
I think you are missing some ENV vars.
This is our docker-compose.yml for Postgres
version: '3.9'
services:
db:
image: postgres:latest
restart: "no"
container_name: db
volumes:
- ./database:/var/lib/postgresql/data
ports:
- "8002:5432"
environment:
POSTGRES_PASSWORD: verySecurePassword34058
POSTGRES_USER: root
POSTGRES_DB: myDatabase
networks:
default:
external: true
name: our-network
Other parts of the application, (like Redis, the NodeJS App, etc) are in other docker-compose.yml files, But since they share the same network, they talk to each other.
You have not mentioned version in docker-composer.yml
version: '2'
services:
postgres:
container_name: 'lh-postgres'
image: 'postgres:13'
environment:
POSTGRES_PASSWORD: root
redis:
container_name: 'lh-redis'
image: 'redis:6'
You should include your docker and docker-compose version in the querstion to help us answer you.
It would also be wise to define the version: 'x' element at the top of your compose file.
You may be suffering from an old version of the cli, akin to this question:
docker-compose : Unsupported config option for services service: 'web'

equivalent of copy command to init the database via entry point

What is the equivalent of specifying the schema of a database in docker-compose.yml file.
database/Dockerfile
FROM mysql:5.7
COPY ./schema.sql /docker-entrypoint-initdb.d/
docker-compose.yml
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
Attempts
I've attempted with the following. Is it possible?
version: '3'
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
command: --init-file /database/schema.sql
volumes:
- ./init.sql:/database/schema.sql
So you have schema.sql somewhere on your host filesystem, let's assume it's on ./database/schema.sql. Then you should have such a compose file:
version: '3'
services:
database:
image: "mysql:5.7"
container_name: "mysql"
ports:
- "6603:3306"
volumes:
- ./database/schema.sql:/docker-entrypoint-initdb.d/init.sql
This image does not support any --init-file command. Instead, it accepts init scripts put under directory /docker-entrypoint-initdb.d.

How to enable DNS Load Balancing in a Docker Compose YAML file?

I have a simple docker-compose.yaml file with 2 ElasticSearch services:
version: '3'
services:
elasticsearch1:
image: elasticsearch:6.5.4
restart: always
elasticsearch2:
image: elasticsearch:6.5.4
restart: always
I want to enable DNS Load Balancing for them in the compose file, but I did not find a way to do this. I know that with docker run this is possible using --net-alias. What's the equivalent in Docker Compose?
The same functionality is there in a compose file:
version: '3'
services:
elasticsearch1:
image: elasticsearch:6.5.4
restart: always
networks:
default:
aliases:
- elasticsearch
elasticsearch2:
image: elasticsearch:6.5.4
restart: always
networks:
default:
aliases:
- elasticsearch
https://docs.docker.com/compose/compose-file/#aliases

How I can group containers in docker-compose?

I'm using this docker-compose.yml.
And I wanna make simpler and inherrit configuration, if its possible.
version: '2'
services:
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
web_one:
container_name: "web_one"
build:
context: ./
dockerfile: web.docker
volumes:
- ./../one:/var/www
environment:
- VIRTUAL_HOST=whoami_one.local
links:
- app_one
app_one:
container_name: "app_one"
build:
context: ./
dockerfile: app.docker
volumes:
- ./../one:/var/www
links:
- db
web_two:
container_name: "web_two"
build:
context: ./
dockerfile: web.docker
volumes:
- ./../two:/var/www
environment:
- VIRTUAL_HOST=whoami_two.local
links:
- app_two
app_two:
container_name: "app_two"
build:
context: ./
dockerfile: app.docker
volumes:
- ./../two:/var/www
links:
- db
I have 15 sites with same configuration.
Can I make config simpler? Like this:
version: '2'
services:
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
one:
extends:
file: common-services.yml
volumes:
- ./../one:/var/www
environment:
- VIRTUAL_HOST=whoami_one.local
two:
extends:
file: common-services.yml
volumes:
- ./../two:/var/www
environment:
- VIRTUAL_HOST=whoami_two.local
Or better?
Thank you!
UPDATE 31 Aug 2021 In the latest docker compose there's support for profiles https://docs.docker.com/compose/profiles/ so this problem would be perfectly solved by that new feature.
Another way is to create no-op services that depend on other services.
For example, in the following docker-compose.yml I have two namespaces, dev for services needed when developing the app, and metrics for services related to visualizing app metrics (since I'm not interested instarting those when developing).
version: "3"
services:
dev:
image: tianon/true
depends_on: ["postgres", "keycloak"]
metrics:
image: monroe/noop
depends_on: ["grafana"]
postgres: ...
keycloak: ...
grafana: ...

Resources