I have below structure
The project root .env defines the path to either ./docker/dev/.env or ./docker/prod/.env as depicted below:
environment=dev
dot_env_path=./docker/dev/.env
My docker-compose.yml contains the below service snippet:
services:
db:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file: ${dot_env_path} <--notice this variable points to the actual .env
environment:
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
- MYSQL_USER=${MYSQL_USER}
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
ports:
- 3306:3306
expose:
- 3306
volumes:
- db-tmc:/var/lib/mysql
However, when I do docker-compose build I receive below warnings; it's not detecting the ${dot_env_path} and loading it's content
Any idea, much appreciated?
Dont use environment: if you are using the env_file option. It overrides the variables already added to the environment from the env_file: option
Use this instead.
services:
db:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
env_file: ${dot_env_path}
ports:
- 3306:3306
expose:
- 3306
volumes:
- db-tmc:/var/lib/mysql
Related
I want to make my nifi data volume and configuration persist means even if I delete container and docker compose up again I would like to keep what I built so far in my nifi. I try to mount volumes as follows in my docker compose file in volumes section nevertheless it doesn't work and my nifi processors are not saved. How can I do it correctly? Below my docker-compose.yaml file.
version: "3.7"
services:
nifi:
image: koroslak/nifi:latest
container_name: nifi
restart: always
environment:
- NIFI_HOME=/opt/nifi/nifi-current
- NIFI_LOG_DIR=/opt/nifi/nifi-current/logs
- NIFI_PID_DIR=/opt/nifi/nifi-current/run
- NIFI_BASE_DIR=/opt/nifi
- NIFI_WEB_HTTP_PORT=8080
ports:
- 9000:8080
depends_on:
- openldap
volumes:
- ./volume/nifi-current/state:/opt/nifi/nifi-current/state
- ./volume/database/database_repository:/opt/nifi/nifi-current/repositories/database_repository
- ./volume/flow_storage/flowfile_repository:/opt/nifi/nifi-current/repositories/flowfile_repository
- ./volume/nifi-current/content_repository:/opt/nifi/nifi-current/repositories/content_repository
- ./volume/nifi-current/provenance_repository:/opt/nifi/nifi-current/repositories/provenance_repository
- ./volume/log:/opt/nifi/nifi-current/logs
#- ./volume/conf:/opt/nifi/nifi-current/conf
postgres:
image: koroslak/postgres:latest
container_name: postgres
restart: always
environment:
- POSTGRES_PASSWORD=secret123
ports:
- 6000:5432
volumes:
- postgres:/var/lib/postgresql/data
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4:4.18
restart: always
environment:
- PGADMIN_DEFAULT_EMAIL=admin
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- 8090:80
metabase:
container_name: metabase
image: metabase/metabase:v0.34.2
restart: always
environment:
MB_DB_TYPE: postgres
MB_DB_DBNAME: metabase
MB_DB_PORT: 5432
MB_DB_USER: metabase_admin
MB_DB_PASS: secret123
MB_DB_HOST: postgres
ports:
- 3000:3000
depends_on:
- postgres
openldap:
image: osixia/openldap:1.3.0
container_name: openldap
restart: always
ports:
- 38999:389
# Mocked source systems
jira-api:
image: danielgtaylor/apisprout:latest
container_name: jira-api
restart: always
ports:
- 8000:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/jira-api.json
pipedrive-api:
image: danielgtaylor/apisprout:latest
container_name: pipedrive-api
restart: always
ports:
- 8100:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/pipedrive-api.yaml
restcountries-api:
image: danielgtaylor/apisprout:latest
container_name: restcountries-api
restart: always
ports:
- 8200:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/restcountries-api.json
volumes:
postgres:
nifi:
openldap:
metabase:
pgadmin:
Using Registry you can achieve that all changes you are doing or your nifi are committed to git. I.e. if you change some processor configuration, it will be reflected in your git repo.
As for flow files, you may need to fix volumes mappings.
I have a docker-compose file which contains a bunch of services. To that docker-compose file, I want to add another service now. The other service files (including its .env) are stored in another folder. I tried to build it like I show you below, but it isnt working. Where do I go wrong?
The docker-compose.yml is contained in the directory nft-trading-service, the other dockerfile which I am trying to include in this docker-compose.yaml is in its own folder nft-asset-updater.
So the structure looks like this
root/nft-trading-server (holding docker-compose.yml)
root/nft-asset-updater (holding its own Dockerfile and .env)
version: "3"
services:
nftapi:
env_file:
- .env
build:
context: .
ports:
- '5000:5000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
asset_update_service:
env_file:
- .env
build:
context: ../nft-asset-updater
dockerfile: .
ports:
- '9000:9000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
postgres:
container_name: postgres
image: postgres:latest
ports:
- "5432:5432"
volumes:
- /data/postgres:/var/lib/postgresql/data
env_file:
- docker.env
networks:
- postgres
pgadmin:
links:
- postgres:postgres
container_name: pgadmin
image: dpage/pgadmin4
ports:
- "8080:80"
env_file:
- docker.env
networks:
- postgres
networks:
postgres:
driver: bridge
I use docker-compose up -d to start my services then I use docker-compose down to stop it
problem; it seems that my data is not persisted; the "${SQL_INIT}:/docker-entrypoint-initdb.d" is executed ever time
I have setup volume db_data for persistence; docker volume ls returns
local backend_mariadb-data
local docker_db_data
local docker_db_logs
here is my docker-compose
version: "3"
services:
backend:
container_name: backend
image: backend
restart: always
build: images/backend/
env_file: .env
ports:
- "8000:8000"
depends_on:
- mariadb
networks:
- app_network
mariadb:
container_name: mariadb
image: "mariadb:${MARIADB_VERSION}"
restart: 'always'
env_file: .env
volumes:
- "${SQL_INIT}:/docker-entrypoint-initdb.d"
- "db_data:${MARIADB_DATA_DIR}"
- "db_logs:${MARIADB_LOG_DIR}"
environment:
MYSQL_ROOT_PASSWORD: "${MYSQL_ROOT_PASSWORD}"
MYSQL_DATABASE: "${MYSQL_DATABASE}"
MYSQL_USER: "${MYSQL_USER}"
MYSQL_PASSWORD: "${MYSQL_PASSWORD}"
ports:
- "3306:3306"
networks:
- app_network
volumes:
db_data:
db_logs:
networks:
app_network:
with .env file
MARIADB_VERSION="latest"
MARIADB_DATA_DIR="/var/database/mariadb"
MARIADB_LOG_DIR="/var/logs/mariadb"
MYSQL_DATABASE="app"
MYSQL_USER="app"
MYSQL_PASSWORD="password"
MYSQL_ROOT_PASSWORD="password"
SQL_INIT="./database/dev"
Remove quotes in env file. Docker-Compose treats them as part of the value.
See last Bulletpoint
Besides that the default path for the data is /var/lib/mysql and for logging is /var/log/mysql (but disabled on default)
I am looking for a way to set global environment variables in my docker-compose.yml file so that I won't need to duplicate them. What is the best way to do it? As you can see some environment variables are repeated in multiple containers. I don't want to re-write them many times.
version: "3.4"
services:
test-db:
image: postgres-test:1.4.0
environment:
- POSTGRES_PASSWORD=password
- DB_NAME_1=dbname1
- DB_USER_1=dbuser1
- DB_PASS_1=dbpass1
- DB_NAME_2=dbname2
- DB_USER_2=dbuser2
- DB_PASS_2=dbpass2
volumes:
- test-db-data:/var/lib/postgresql/data
restart: on-failure
ports:
- 5432:5432
networks:
- test
test-key:
image: key-test:1.4.0
environment:
DB_VENDOR: POSTGRES
DB_ADDR: test-db
DB_DATABASE: dbname1
DB_USER: dbuser1
DB_PASSWORD: dbpass1
KEY_USER: admin
KEY_PASSWORD: admin
#KEY_LOGLEVEL: DEBUG
KEY_FRONTEND_URL: http://test-key:8080/auth
KEY_IMPORT: /test-key/test_realm.json
command:
- "-Djboss.as.management.blocking.timeout=3600"
ports:
- 8080:8080
volumes:
- $PWD/test_realm.json:/test-key/test_realm.json
networks:
- test
depends_on:
- test-db
test-bus:
image: bus-test:1.5.0
environment:
PYTEST_HOST: test-pytest
PYTEST_PORT: 8090
TEST_DB_PW: dbpass2
TEST_DB_USER: dbuser2
TEST_DB_NAME: dbname2
TEST_DB_HOST: test-db
TEST_KAF_HOST: test-kaf
TEST_KAF_PORT: 9092 # for a connection inside test-network
#TEST_SERVER_STATE: PRODUCTION
TEST_SERVER_STATE: DEVELOPMENT
ports:
- 5000:5000
networks:
- test
depends_on:
- test-db
- test-key
You can define your variables in an env-file named .env beside docker-compose.yml file.
.env
dbname1_key=dbname1
dbuser1_key=dbuser1
dbpass1_key=dbpass1
Then you can use the variable in docker-compose.yml file
docker-compose.yml
version: "3.4"
services:
test-db:
image: postgres-test:1.4.0
environment:
- POSTGRES_PASSWORD=password
- DB_NAME_1=${dbname1_key}
- DB_USER_1=${dbuser1_key}
- DB_PASS_1=${dbpass1_key}
- DB_NAME_2=dbname2
- DB_USER_2=dbuser2
- DB_PASS_2=dbpass2
volumes:
- test-db-data:/var/lib/postgresql/data
restart: on-failure
ports:
- 5432:5432
networks:
- test
...
I have the following docker-compose file. I have created the .env file as well with the relevant variables. when I attempt to run docker-compose using this file it complains about the variables. I know in Ansible yaml I declare the vars inside the file and can call them anywhere. Not sure what what I am doing wrong.
---
version: "3"
services:
${hostname01}:
image: apline:3.12
restart: unless-stopped
container_name: ${hostname01}
environment:
volumes:
- ${hostname01}_Data:/var/www/html
ports:
- 80:80
- 443:443
services:
${hostname02}:
image: mariadb
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${mySQLRootPass}
volumes:
- ${hostname02}_Data:/var/lib/mysql
ports:
- 3306:3306
volumes:
${hostname01}_Data:
${hostname02}_Data:
driver: local