docker-compose variable format - docker

I have the following docker-compose file. I have created the .env file as well with the relevant variables. when I attempt to run docker-compose using this file it complains about the variables. I know in Ansible yaml I declare the vars inside the file and can call them anywhere. Not sure what what I am doing wrong.
---
version: "3"
services:
${hostname01}:
image: apline:3.12
restart: unless-stopped
container_name: ${hostname01}
environment:
volumes:
- ${hostname01}_Data:/var/www/html
ports:
- 80:80
- 443:443
services:
${hostname02}:
image: mariadb
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${mySQLRootPass}
volumes:
- ${hostname02}_Data:/var/lib/mysql
ports:
- 3306:3306
volumes:
${hostname01}_Data:
${hostname02}_Data:
driver: local

Related

Why are docker volumes not created?

I am running Flask and MySQL using docker-compose.yaml.
Although I have specified a volume, the volume has not been created.
Why is this?
docker-compose.yaml
version: '3'
services:
api:
build: python
container_name: api_server_flask
ports:
- "5000:5000"
tty: true
environment:
TZ: Asia/Tokyo
FLASK_APP: app.py
depends_on:
- db
networks:
- app_flask
db:
build: mysql
container_name: db_server_flask
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: pass
MYSQL_DATABASE: testdb
TZ: Asia/Tokyo
volumes:
- ./db-flask-data/:/var/lib/mysql
command: mysqld
networks:
- app_flask
volumes:
db-flask-data:
networks:
app_flask:
driver: bridge
You need to remove the relative path as follows:
volumes:
- db-flask-data/:/var/lib/mysql
It is explained here: https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose
Actually, you are creating a named volume under /var/lib/docker/volumes/db-flask-data when you are specifying the following command:
volumes:
db-flask-data:

Persist nifi data and volume

I want to make my nifi data volume and configuration persist means even if I delete container and docker compose up again I would like to keep what I built so far in my nifi. I try to mount volumes as follows in my docker compose file in volumes section nevertheless it doesn't work and my nifi processors are not saved. How can I do it correctly? Below my docker-compose.yaml file.
version: "3.7"
services:
nifi:
image: koroslak/nifi:latest
container_name: nifi
restart: always
environment:
- NIFI_HOME=/opt/nifi/nifi-current
- NIFI_LOG_DIR=/opt/nifi/nifi-current/logs
- NIFI_PID_DIR=/opt/nifi/nifi-current/run
- NIFI_BASE_DIR=/opt/nifi
- NIFI_WEB_HTTP_PORT=8080
ports:
- 9000:8080
depends_on:
- openldap
volumes:
- ./volume/nifi-current/state:/opt/nifi/nifi-current/state
- ./volume/database/database_repository:/opt/nifi/nifi-current/repositories/database_repository
- ./volume/flow_storage/flowfile_repository:/opt/nifi/nifi-current/repositories/flowfile_repository
- ./volume/nifi-current/content_repository:/opt/nifi/nifi-current/repositories/content_repository
- ./volume/nifi-current/provenance_repository:/opt/nifi/nifi-current/repositories/provenance_repository
- ./volume/log:/opt/nifi/nifi-current/logs
#- ./volume/conf:/opt/nifi/nifi-current/conf
postgres:
image: koroslak/postgres:latest
container_name: postgres
restart: always
environment:
- POSTGRES_PASSWORD=secret123
ports:
- 6000:5432
volumes:
- postgres:/var/lib/postgresql/data
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4:4.18
restart: always
environment:
- PGADMIN_DEFAULT_EMAIL=admin
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- 8090:80
metabase:
container_name: metabase
image: metabase/metabase:v0.34.2
restart: always
environment:
MB_DB_TYPE: postgres
MB_DB_DBNAME: metabase
MB_DB_PORT: 5432
MB_DB_USER: metabase_admin
MB_DB_PASS: secret123
MB_DB_HOST: postgres
ports:
- 3000:3000
depends_on:
- postgres
openldap:
image: osixia/openldap:1.3.0
container_name: openldap
restart: always
ports:
- 38999:389
# Mocked source systems
jira-api:
image: danielgtaylor/apisprout:latest
container_name: jira-api
restart: always
ports:
- 8000:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/jira-api.json
pipedrive-api:
image: danielgtaylor/apisprout:latest
container_name: pipedrive-api
restart: always
ports:
- 8100:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/pipedrive-api.yaml
restcountries-api:
image: danielgtaylor/apisprout:latest
container_name: restcountries-api
restart: always
ports:
- 8200:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/restcountries-api.json
volumes:
postgres:
nifi:
openldap:
metabase:
pgadmin:
Using Registry you can achieve that all changes you are doing or your nifi are committed to git. I.e. if you change some processor configuration, it will be reflected in your git repo.
As for flow files, you may need to fix volumes mappings.

Invalid docker.compose.yaml file

I am trying to get a couple of containers up and running, however I am running into some issues. I run this command:
docker-compose up -d --build itvdflab
and get this error
The Compose file './docker-compose.yaml' is invalid because:
Unsupported config option for services: 'itvdelab'
Unsupported config option for networks: 'itvdelabnw'
Here is the yaml file.
services:
itvdelab:
image: itversity/itvdelab
hostname: itvdelab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "cluster_util_db"
cluster_util_db:
image: postgres:13
ports:
- "6432:5432"
volumes:
- ./cluster_util_db_scripts:/docker-entrypoint-initdb.d
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
itvdflab:
build:
context: .
dockerfile: images/pythonsql/Dockerfile
hostname: itvdflab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "pg.itversity.com"
pg.itversity.com:
image: postgres:13
ports:
- "5432:5432"
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
networks:
itvdelabnw:
name: itvdelabnw
What changes do I need to make to get this working?
Your docker-compose.yml file is missing a version: line. Until very recently, this caused Docker Compose to interpret this as the original "version 1" Compose format, which doesn't have a top-level services: key and doesn't support Docker networks. The much newer Compose Specification claims that a version: key is optional, but in practice if you can't be guaranteed to use a very new version of Compose (built as a plugin to the docker binary) it's required. The most recent Compose file versions supported by the standalone Python docker-compose tool are 3.8 and 2.4 (you need the 2.x version for some resource-related constraints in non-Swarm installations).
# Add at the very beginning
version: '3.8'
Here is the revised copy:
version: '3.4'
services:
itvdelab:
image: itversity/itvdelab
hostname: itvdelab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "cluster_util_db"
cluster_util_db:
image: postgres:13
ports:
- "6432:5432"
volumes:
- ./cluster_util_db_scripts:/docker-entrypoint-initdb.d
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
itvdflab:
build:
context: .
dockerfile: images/pythonsql/Dockerfile
hostname: itvdflab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "pg.itversity.com"
pg.itversity.com:
image: postgres:13
ports:
- "5432:5432"
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
networks:
itvdelabnw:
name: itvdelabnw
and now I get the following error
ERROR: The Compose file './docker-compose.yaml' is invalid because:
services.pg.itversity.com.networks.itvdelabnw contains unsupported option: 'name'
for me work try different version. In my case work
version: '2.2'

Docker-Compose build image and use its .env file which are stored in another directory

I have a docker-compose file which contains a bunch of services. To that docker-compose file, I want to add another service now. The other service files (including its .env) are stored in another folder. I tried to build it like I show you below, but it isnt working. Where do I go wrong?
The docker-compose.yml is contained in the directory nft-trading-service, the other dockerfile which I am trying to include in this docker-compose.yaml is in its own folder nft-asset-updater.
So the structure looks like this
root/nft-trading-server (holding docker-compose.yml)
root/nft-asset-updater (holding its own Dockerfile and .env)
version: "3"
services:
nftapi:
env_file:
- .env
build:
context: .
ports:
- '5000:5000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
asset_update_service:
env_file:
- .env
build:
context: ../nft-asset-updater
dockerfile: .
ports:
- '9000:9000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
postgres:
container_name: postgres
image: postgres:latest
ports:
- "5432:5432"
volumes:
- /data/postgres:/var/lib/postgresql/data
env_file:
- docker.env
networks:
- postgres
pgadmin:
links:
- postgres:postgres
container_name: pgadmin
image: dpage/pgadmin4
ports:
- "8080:80"
env_file:
- docker.env
networks:
- postgres
networks:
postgres:
driver: bridge

docker-compose load .env from dynamic path

I have below structure
The project root .env defines the path to either ./docker/dev/.env or ./docker/prod/.env as depicted below:
environment=dev
dot_env_path=./docker/dev/.env
My docker-compose.yml contains the below service snippet:
services:
db:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file: ${dot_env_path} <--notice this variable points to the actual .env
environment:
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
ports:
- 3306:3306
expose:
- 3306
volumes:
- db-tmc:/var/lib/mysql
However, when I do docker-compose build I receive below warnings; it's not detecting the ${dot_env_path} and loading it's content
Any idea, much appreciated?
Dont use environment: if you are using the env_file option. It overrides the variables already added to the environment from the env_file: option
Use this instead.
services:
db:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file: ${dot_env_path}
ports:
- 3306:3306
expose:
- 3306
volumes:
- db-tmc:/var/lib/mysql

Resources