I'm deploying my Django app using AWS code pipeline which is dockerized and I was storing my env variables inside an env file for local development but for the code pipeline I set them all inside environment variables but the variables are still getting None.
docker-compose.yml
version: "3.8"
services:
db:
container_name: db
image: "postgres"
restart: always
volumes:
- postgres-data:/var/lib/postgresql/data/
app:
container_name: app
build:
context: .
restart: always
volumes:
- static-data:/vol/web
depends_on:
- db
proxy:
container_name: proxy
build:
context: ./proxy
restart: always
depends_on:
- app
ports:
- 80:8000
volumes:
- static-data:/vol/static
volumes:
postgres-data:
static-data:
getting env variable in django like:
os.environ.get('FRONTEND_URL')
in my case, I put all env variables inside my AWS CodeBuild environment variables and call them in my docker environment as below.
version: "3.8"
services:
db:
container_name: db
image: "postgres"
restart: always
volumes:
- postgres-data:/var/lib/postgresql/data/
environment:
- VARIABLE_NAME: ${VARIABLE_NAME}
You can specify the env file in the docker compose itself along with relative path.
....
....
app:
container_name: app
build:
context: .
restart: always
env_file:
- <web-variables1.env>
volumes:
- static-data:/vol/web
depends_on:
- db
....
....
Refer more details at documentation
Related
I have a docker-compose file which contains a bunch of services. To that docker-compose file, I want to add another service now. The other service files (including its .env) are stored in another folder. I tried to build it like I show you below, but it isnt working. Where do I go wrong?
The docker-compose.yml is contained in the directory nft-trading-service, the other dockerfile which I am trying to include in this docker-compose.yaml is in its own folder nft-asset-updater.
So the structure looks like this
root/nft-trading-server (holding docker-compose.yml)
root/nft-asset-updater (holding its own Dockerfile and .env)
version: "3"
services:
nftapi:
env_file:
- .env
build:
context: .
ports:
- '5000:5000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
asset_update_service:
env_file:
- .env
build:
context: ../nft-asset-updater
dockerfile: .
ports:
- '9000:9000'
depends_on:
- postgres
networks:
- postgres
extra_hosts:
- "host.docker.internal:host-gateway"
restart: always
postgres:
container_name: postgres
image: postgres:latest
ports:
- "5432:5432"
volumes:
- /data/postgres:/var/lib/postgresql/data
env_file:
- docker.env
networks:
- postgres
pgadmin:
links:
- postgres:postgres
container_name: pgadmin
image: dpage/pgadmin4
ports:
- "8080:80"
env_file:
- docker.env
networks:
- postgres
networks:
postgres:
driver: bridge
services:
postgres:
container_name: 'lh-postgres'
image: 'postgres:13'
environment:
POSTGRES_PASSWORD: root
redis:
container_name: 'lh-redis'
image: 'redis:6'
nginx:
container_name: 'lh-nginx'
build: ./nginx
depends_on:
- php-fpm
volumes:
- ./src/lh-app:/var/www/html/app
- ./src/lh-api:/var/www/html/api
ports:
- "80:80"
- "443:443"
php-fpm:
container_name: 'lh-php'
image: docker.io/bitnami/php-fpm:8.0
user: '1000:1000'
build:
context: ./php-fpm
args:
- PHP_ENV= development
depends_on:
- postgres
- redis
volumes:
- ./src/lh-app:/var/www/html/app
- ./src/lh-api:/var/www/html/api
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services: 'postgres'
getting this error
I think you are missing some ENV vars.
This is our docker-compose.yml for Postgres
version: '3.9'
services:
db:
image: postgres:latest
restart: "no"
container_name: db
volumes:
- ./database:/var/lib/postgresql/data
ports:
- "8002:5432"
environment:
POSTGRES_PASSWORD: verySecurePassword34058
POSTGRES_USER: root
POSTGRES_DB: myDatabase
networks:
default:
external: true
name: our-network
Other parts of the application, (like Redis, the NodeJS App, etc) are in other docker-compose.yml files, But since they share the same network, they talk to each other.
You have not mentioned version in docker-composer.yml
version: '2'
services:
postgres:
container_name: 'lh-postgres'
image: 'postgres:13'
environment:
POSTGRES_PASSWORD: root
redis:
container_name: 'lh-redis'
image: 'redis:6'
You should include your docker and docker-compose version in the querstion to help us answer you.
It would also be wise to define the version: 'x' element at the top of your compose file.
You may be suffering from an old version of the cli, akin to this question:
docker-compose : Unsupported config option for services service: 'web'
Is there a way to specify an empty array in docker-compose.yml?
I tried ports: [] to no avail.
A JSON empty array literal should be valid YAML syntax, but when I use this, I get the following error from Docker:
services.myapp_migrations.ports must be a list
For more context:
I have a python application with the following docker-compose.yml:
version: "3.9"
services:
myapp: &myapp
build:
context: .
ports:
- 8000:8000
environment:
- MYAPP_DB_HOST=postgres
volumes:
- ./.env:/app/.env
working_dir: /app
depends_on:
- postgres
myapp_migrations:
<<: *myapp
command: ["migrate"]
restart: on-failure
ports: []
postgres:
image: postgres:12-alpine
environment:
- POSTGRES_PASSWORD=${MYAPP_DB_PASSWORD}
- POSTGRES_USER=${MYAPP_DB_USER}
- POSTGRES_DB=${MYAPP_DB_NAME}
As you can see, I'm trying to re-use the myapp definition with a YAML anchor, but I need to override the ports definition in myapp_migrations.
(I know I could use extension fields for this, but I'm trying to avoid that.)
I agree with #flyx this is probably a temporary bug.
You might be able to work around this by using docker-compose in place of docker compose. Because they are separate binaries, they are updated independently.
For example, given your yaml above:
docker compose (not working)
test % docker compose config
services.myapp_migrations.ports must be a list
docker-compose (working)
test % docker-compose config
WARNING: The MYAPP_DB_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The MYAPP_DB_USER variable is not set. Defaulting to a blank string.
WARNING: The MYAPP_DB_NAME variable is not set. Defaulting to a blank string.
services:
myapp:
build:
context: /Users/jvanus/test
depends_on:
postgres:
condition: service_started
environment:
MYAPP_DB_HOST: postgres
ports:
- published: 8000
target: 8000
volumes:
- /Users/jvanus/test/.env:/app/.env:rw
working_dir: /app
myapp_migrations:
build:
context: /Users/jvanus/test
command:
- migrate
depends_on:
postgres:
condition: service_started
environment:
MYAPP_DB_HOST: postgres
ports: []
restart: on-failure
volumes:
- /Users/jvanus/test/.env:/app/.env:rw
working_dir: /app
postgres:
environment:
POSTGRES_DB: ''
POSTGRES_PASSWORD: ''
POSTGRES_USER: ''
image: postgres:12-alpine
version: '3.9'
I have the following docker-compose file. I have created the .env file as well with the relevant variables. when I attempt to run docker-compose using this file it complains about the variables. I know in Ansible yaml I declare the vars inside the file and can call them anywhere. Not sure what what I am doing wrong.
---
version: "3"
services:
${hostname01}:
image: apline:3.12
restart: unless-stopped
container_name: ${hostname01}
environment:
volumes:
- ${hostname01}_Data:/var/www/html
ports:
- 80:80
- 443:443
services:
${hostname02}:
image: mariadb
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${mySQLRootPass}
volumes:
- ${hostname02}_Data:/var/lib/mysql
ports:
- 3306:3306
volumes:
${hostname01}_Data:
${hostname02}_Data:
driver: local
I have a value in a Dockerfile called ${APP_NAME}. What is it? If this were bash scripting, I would assume it to be some sort of variable but it hasn't been assigned a value anywhere. Is it a command line argument? If so, how would I pass it in when I wanted to call docker-compose with it?
For reference, the Docker file looks like this:
version: '2'
services:
nginx:
container_name: ${APP_NAME}_nginx
hostname: nginx
build:
context: ./containers/nginx
dockerfile: Dockerfile
ports:
- "80:80"
- "443:443"
volumes:
- .:/app
links:
- phpfpm
networks:
- backend
phpfpm:
container_name: ${APP_NAME}_phpfpm
hostname: phpfpm
expose:
- "9000"
build:
context: ./containers/php-fpm
dockerfile: Dockerfile
working_dir: /app
volumes:
- .:/app
links:
- mysql
networks:
- backend
mysql:
container_name: ${APP_NAME}_mysql
hostname: mysql
build:
context: ./containers/mysql
dockerfile: Dockerfile
volumes:
- ./storage/mysql:/var/lib/mysql
- ${MYSQL_ENTRYPOINT_INITDB}:/docker-entrypoint-initdb.d
environment:
- MYSQL_DATABASE=${DB_DATABASE}
- MYSQL_ROOT_PASSWORD=${DB_PASSWORD}
ports:
- "33061:3306"
expose:
- "3306"
networks:
- backend
networks:
backend:
driver: "bridge"
And actually, I'm probably going to have a lot of questions about docker because I've never really used it before so a reference to Dockerfile syntax would be helpful.
This means that there is probably somewhere in your project .env file which contains variables necessary for docker compose. You can find more about it at the official docker compose docs. It says that you can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file. Try to find more here: https://docs.docker.com/compose/compose-file/#variable-substitution