I have the .env file as yaml format, in other words I have development.yaml file which consists of following let's say:
server:
port: 3000
apiPrefix: api/v1
swagger:
title: Test
description: test description
version: 1.0
api: /docs
database:
name: test
autoLoadModels: true
synchronize: false
dialect: postgres
and inside docker-compose.yaml
version: '3.8'
services:
dev:
env_file:
- config/development.yaml
container_name: test_development_api_docker_container
image: test_development_api_docker_container:1.0.0
build:
context: .
target: testDevelopmentEnv
dockerfile: ./Dockerfile
command: npm run start:debug
ports:
- ${port}:${port}
networks:
- test_network
volumes:
- .:/Users/falyoun/dev/ite/test
- /Users/falyoun/dev/ite/test/node_modules
restart: unless-stopped
networks:
test_network:
In the above file I tried:
${port}
${server.port}
and all causes issues like
ERROR: Invalid interpolation format for "ports" option in service "dev": "${server.port}:${server.port}"
or
ERROR: The Compose file './docker-compose.yaml' is invalid because:
services.dev.ports contains an invalid type, it should be a number, or an object
Before you try anything, you should switch from:
docker-compose
to:
docker compose
The docker-compose tool has been deprecated, and is at this point a fairly stale version (1.25.X) as installed by apt install. Instead, you should remove docker-compose from your computer and install docker-compose-plugin (latest version as of today is 2.6.X).
Related
As far as I know, to use the env variable in docker-compose we need to create a .env file
PORT=9000
And then use it like this in docker-compose.yml:
services:
go:
ports:
- ${PORT}:${PORT}
But what if I have a config file in .yml file like this:
http:
host: "0.0.0.0"
port: "1020"
jwt_secret: "secret"
How to access it on the docker-compose? This doesn't work:
version: "3.8"
services:
go:
env_file: config.yml
build:
context: .
dockerfile: Dockerfile
command: /fetch-app
ports:
- ${http.port}:${http.port}
environment:
NODE_ENV: development
It is generates error:
invalid interpolation format for services.go.ports.[].
You may need to escape any $ with another $.
${http.port}:${http.port}
When building the docker file, I have the command:
CMD ["/app/database/updateLocalDocker.sh"]
The shell script should connect to the postgres service using liquibase but fails with the error connection refused...
When i comment out the above CMD and run the same script directory from the container via docker exec -t -i f42c4bbcd95d /bin/bash, it works fine.
The URL i'm trying to connect to is: jdbc:postgresql://localhost:5432/service_x"
I have a feeling that it's related to either the service not being started or a network issue, when trying to execute the CMD during the docker-compose build stage.
Any guidance would be much appreicated.
docker-compose.yml:
version: "3.8"
services:
db:
image: local.db
build:
context: .
ports:
- 15432:5432
environment:
POSTGRES_PASSWORD: password
networks:
- a
networks:
a:
name: a
external: true
To access your database from your localhost you need to use the port 15432 instead of 5432.
services:
db:
image: local.db
build:
context: .
ports:
- 15432:5432 <--- Here
environment:
POSTGRES_PASSWORD: password
networks:
- a
The first port is your host and the second is the port used in your container.
You can also access it with the container name and the port used in it.
Docker port mapping documentation :
https://docs.docker.com/config/containers/container-networking/
Instead of putting the command in the Dockerfile, you can directly put the command in the docker-compose file and remove CMD ["/app/database/updateLocalDocker.sh"].
docker-compose.yml
version: "3.8"
services:
db:
image: local.db
build:
context: .
command: sh -c "<Enter-your-command>"
ports:
- 15432:5432
environment:
POSTGRES_PASSWORD: password
networks:
- a
networks:
a:
name: a
external: true
If you have one command execute
command: <command>
OR
If you have more than one command, it should be separated by &&.
Syntax:
sh -c "<command-1> && <command-2> && <command-3>"
Docker noob here.
I have two files docker-compose.build.yml and docker-compose.up.yml in my docker folder. Following are the contents of both files..
docker-compose.build.yml
version: "3"
services:
base:
build:
context: ../
dockerfile: ./docker/Dockerfile.base
args:
DEBUG: "true"
image: ottertune-base
labels:
NAME: "ottertune-base"
web:
build:
context: ../
dockerfile: ./docker/Dockerfile.web
image: ottertune-web
depends_on:
- base
labels:
NAME: "ottertune-web"
volumes:
- ../server:/app
driver:
build:
context: ../
dockerfile: ./docker/Dockerfile.driver
image: ottertune-driver
depends_on:
- base
labels:
NAME: "ottertune-driver"
docker-compose.up.yml
version: "3"
services:
web:
image: ottertune-web
container_name: web
expose:
- "8000"
ports:
- "8000:8000"
links:
- backend
- rabbitmq
depends_on:
- backend
- rabbitmq
environment:
DEBUG: 'true'
ADMIN_PASSWORD: 'changeme'
BACKEND: 'postgresql'
DB_NAME: 'ottertune'
DB_USER: 'postgres'
DB_PASSWORD: 'ottertune'
DB_HOST: 'backend'
DB_PORT: '5432'
DB_OPTS: '{}'
MAX_DB_CONN_ATTEMPTS: 30
RABBITMQ_HOST: 'rabbitmq'
working_dir: /app/website
entrypoint: ./start.sh
labels:
NAME: "ottertune-web"
networks:
- ottertune-net
driver:
image: ottertune-driver
container_name: driver
depends_on:
- web
environment:
DEBUG: 'true'
working_dir: /app/driver
labels:
NAME: "ottertune-driver"
networks:
- ottertune-net
rabbitmq:
image: "rabbitmq:3-management"
container_name: rabbitmq
restart: always
hostname: "rabbitmq"
environment:
RABBITMQ_DEFAULT_USER: "guest"
RABBITMQ_DEFAULT_PASS: "guest"
RABBITMQ_DEFAULT_VHOST: "/"
expose:
- "15672"
- "5672"
ports:
- "15673:15672"
- "5673:5672"
labels:
NAME: "rabbitmq"
networks:
- ottertune-net
backend:
container_name: backend
restart: always
image: postgres:9.6
environment:
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: 'ottertune'
POSTGRES_DB: 'ottertune'
expose:
- "5432"
ports:
- "5432:5432"
labels:
NAME: "ottertune-backend"
networks:
- ottertune-net
networks:
ottertune-net:
driver: bridge
Nothing wrong with the dockerfiles, i just have a few doubts about this approach.
What purpose does having multiple files serve instead of just one docker-compose.yml?
How does docker-compose work when used with multiple files?
When i do docker-compose -f docker-compose.build.yml build --no-cache
Building base
Step 1/1 : FROM ubuntu:18.04
---> 775349758637
[Warning] One or more build-args [DEBUG] were not consumed
Successfully built 775349758637
Successfully tagged ottertune-base:latest
Building web
Step 1/1 : FROM ottertune-base
---> 775349758637
Successfully built 775349758637
Successfully tagged ottertune-web:latest
Building driver
Step 1/1 : FROM ottertune-base
---> 775349758637
Successfully built 775349758637
Successfully tagged ottertune-driver:latest
and then docker-compose up i get the error
rabbitmq is up-to-date
backend is up-to-date Starting web ... error
ERROR: for web Cannot start service web: OCI runtime create failed: container_linux.go:346:
starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or
directory": unknown
ERROR: for web Cannot start service web: OCI runtime create failed: container_linux.go:346:
starting container process caused "exec: \"./start.sh\": stat ./start.sh: no such file or
directory": unknown
ERROR: Encountered errors while bringing up the project.
this entrypoint start.sh is defined in the docker-compose.up.yml file which I didn't pass as an argument to
docker-compose build
So, why is the docker-compose up trying to run this entrypoint from a yml file which is not even passed during build? Really confused on this and didn't find much about it on google and stackoverflow.
If you docker-compose -f a.yml -f b.yml ..., Docker Compose merges the two YAML files. If you look at the two files you've posted, one has all of the run-time settings (ports:, environment:, ...), and if you happened to have the images already it would be enough to run the application. The second only has build-time settings (build:), but requires the source tree checked out locally to be able to run.
You probably need to specify both files on every docker-compose invocation
docker-compose -f docker-compose.build.yml -f docker-compose.up.yml up --build
It does seem like the author of these files intended for them to be run separately
docker-compose -f docker-compose.build.yml build
docker-compose -f docker-compose.up.yml up
but note that some of the run-time options in the build file, like volumes: to hide the application built into the image, will never take effect.
(You should be able to delete a large number of settings in the "up" YAML file that either duplicate what's in the image or that Docker Compose can provide for you: container_name:, expose:, links:, working_dir:, entrypoint:, networks:, and (probably) labels: are all unnecessary and can be deleted.)
What purpose does having multiple files serve instead of just one docker-compose.yml?
You can share configuration across environments. For example, I keep the common configuration such as the network and server in a docker-compose.yml. I keep my development environment specifics such as a server with automatic reload and debugging enabled in a docker-compose.override.yml. I keep the production-specific configs in a docker-compose.prod.yml. Then I can run docker-compose up --build for my development environment (Docker Compose uses docker-compose.yml and docker-compose.override.yml by default). And I can run my prod environment with docker-compose -f docker-compose.yml -f docker-compose.prod.yml up --build. You can read about this in the dedicated docs page.
How does docker-compose work when used with multiple files?
It takes the first file as the base file, and adds or replaces configs from subsequent files ot the base file. See the relevant docs.
When i do docker-compose -f docker-compose.build.yml build --no-cache ...
As for your last question, I can't really tell by what I've seen. But unlike Dockerfiles which need two commands (docker build and docker run), docker-compose only needs one. So when you do docker-compose up, it looks for a file named docker-compose.yml (and also docker-compose.override.yml if it's present).
In my Docker Instance, I created a volume called jokes. I'm trying to build out my service now with docker-compose up but I keep getting this error message:
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.db.volumes contains an invalid type, it should be an array
Here is my docker-compose.yml file:
version: '3.6'
services:
web:
build: .
command: puma
depends_on:
- db
environment:
DATABASE_URL: "postgres://postgres#db"
ports:
- "3000:3000"
volumes:
- "./:/app"
working_dir: /app
db:
image: "postgres:10.3-alpine"
volumes: "-jokes: /var/lib/postgresql/data"
volumes:
jokes: ~
How should I fix this?
How should I fix this?
As detailed in the official documentation you are required to give a list there (hence complaint about list), so move hyphen outside the quotes like so:
volumes:
- "jokes:/var/lib/postgresql/data"
I am trying to compose a stack using secrets
for development, i use local files in docker/secrets/FILE_NAME
I had this working in windows 10, but I'm struggling to get it to work under win7 toolbox.
I get an error:
Cannot create container for service db:
invalid volume specification:
'C:\project\docker\secrets\DB_USERNAME:/run/secrets/db-username:ro'
I was trying to set COMPOSE_CONVERT_WINDOWS_PATH but unfortunately, this does not change anything. I will get the same output with true or false.
Setting absolute paths did not help either.
Docker compose version 1.16.1 build 6d1ac219
Docker version 17.10.0-ce, build f4ffd25
My docker-compose.yml
version: '3.3'
services:
db:
image: postgres
volumes:
- db-data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/db-password
- POSTGRES_USER_FILE=/run/secrets/db-username
- POSTGRES_DB_FILE=/run/secrets/db-name
secrets:
- db-username
- db-password
- db-name
web:
build:
context: ./docker/machines/django/
args:
buildno: 1
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./Server:/Server
ports:
- "8000:8000"
depends_on:
- db
secrets:
- db-username
- db-password
- db-name
secrets:
db-username:
file: ./docker/secrets/DB_USERNAME
db-password:
file: ./docker/secrets/DB_PASSWORD
db-name:
file: ./docker/secrets/DB_NAME
volumes:
db-data:
driver: "local"