docker-compose.yml:
version: '3.7'
services:
db:
image: mysql:8.0
docker-compose.test.yml:
version: '3.7'
services:
db:
ports:
- 3306:3306
docker-compose config does not show the port. Why?
I'm trying to display the effective compose file, means: the merge result that is also used by docker-compose up.
docker-compose version 1.25.0
According to the documentation, docker-compose will only automatically find files named docker-compose.yml and docker-compose.override.yml:
By default, Compose reads two files, a docker-compose.yml and an optional docker-compose.override.yml file. By convention, the docker-compose.yml contains your base configuration. The override file, as its name implies, can contain configuration overrides for existing services or entirely new services.
If you want to use additional compose files, you need to specify them explicitly using -f <filename>:
docker-compose up -f docker-compose.yml -f docker-compose.test.yml
By default, docker-compose finds the docker-compose.yml file. The problem you are facing is happening because the port is configured in the other file and DockerCompose doesn't know anything about it. If you want to check the configuration of the other file, you need to pass it as a parameter:
docker-compose -f ./path/to/docker-compose.test.yml config
Or you can put the port configuration on the first file like this:
version: '3.7'
services:
db:
image: mysql:8.0
ports:
- 3306:3306
And it should work just fine.
Related
So I am trying to set up a few services in my local docker environment. One of the services, as defined in my docker-compose.yml file, is as follows:
version: "3"
services:
solr:
image: solr:latest
ports:
- "8983:8983"
container_name: solr
...which is pretty straightforward.
However, I also want to set up another service that has its own docker-compose.yml in it. If I simply specify a FROM in the Dockerfile for this service, it doesn't work as expected as the repo's docker-compose.yml includes services from other repositories as well.
What is the right way to configure such a service that has its own docker-compose.yml file?
Use extend in yml file to call another docker-compose file. When defining any service in docker-compose.yml, you can declare that you are extending another service like this:
web:
extends:
file: common-services.yml
service: webapp
Ref : https://docs.docker.com/compose/extends/#understand-the-extends-configuration
Note : extend option is removed from docker-compose version-3 and it is available only in version-2
Ref : https://docs.docker.com/compose/extends/#extending-services
I have docker-compose.yml like below:
version: '2'
services:
micro-service:
image: some/micro-service:${SERVICE_VERSION}
env_file:
- ../all-variables/${PROFILE}/micro_service.env
ports:
- "8085:8085"
And I have two files : dev.env and stage.env where SERVICE_VERSION and PROFILE are described.
Is there any way to specify concrete file when running docker-compose up
By default docker-compose takes .env file from current dir.
Is there a way to override it or another workaround ?
As mentioned in the answer in the comments there is no way to do that as it is directly coded into the source code to use .env.
However, there a couple ways to get similar behaviour.
The first way works natively with docker-compose, which would be to use docker-compose override files.
So in your case you could have your base docker-compose.yml file like this:
version: '2'
services:
micro-service:
image: some/micro-service:1.0.0
ports:
- "8085:8085"
Then you can define a docker-compose-dev.yml file:
version: '2'
services:
micro-service:
image: some/micro-service:dev
env_file:
- ../all-variables/dev/micro_service.env
Then you can run the following command
$ docker-compose up -f docker-compose.yml -f docker-compose-dev.yml up
If you do this the values in docker-compose-dev.yml will override those in docker-compose.yml. So instead of using image some/micro-service:1.0.0 it will use the image defined in docker-compose-dev.yml.
The second way would be to use docker-app. Which is a new experimental utility from the Docker team.
Basically you will create a dockerapp file that would look like this:
version: 0.0.1
name: app
---
version: '2'
services:
micro-service:
image: some/micro-service:${SERVICE_VERSION}
env_file:
- ../all-variables/${PROFILE}/micro_service.env
ports:
- "8085:8085"
---
SERVICE_VERSION: latest
PROFILE: default
Then if you convert your .env files to .yml render the compose file with the correct variables using docker-app.
docker-app render -f dev.yml | docker-compose -f - up
Hopefully this is helpful, I am going through a similar issue when working with multiple environments with docker-compose.
My docker compose configs look like this:
docker-compose.yml
version: '3.5'
services:
nginx:
ports:
- 8080:8080
docker-compose.prod.yml
version: '3.5'
services:
nginx:
ports:
- 80:80
Now, when I run command: docker-compose -f docker-compose.yml -f docker-compose.prod.yml up the nginx exposes on host machine two ports: 8000 and 80, because it merges ports properties:
version: '3.5'
services:
nginx:
ports:
- 8080:8080
- 80:80
Is there a way to override it? I want to expose only port 80.
This behaviour is documented at https://docs.docker.com/compose/extends/#adding-and-overriding-configuration
For the multi-value options ports, expose, external_links, dns, dns_search, and tmpfs, Compose concatenates both sets of values
Since the ports will be the concatenation of the ports in all your compose files, I would suggest creating a new docker-compose.dev.yml file which contains your development port mappings, removing them from the base docker-compose.yml file.
As Nikson says, you can name this docker-compose.override.yml to apply your development configuration automatically without chaining the docker-compose files. docker-compose.override.yml will not be applied if you manually specify another override file (e.g. docker-compose -f docker-compose.yml -f docker-compose.prod.yml)
it isn't possible a the moment but I found quite good way to fix this issue using the command yq.
You need to remove the ports from the original file.
Example:
Be careful this command will remove the nginx ports from your current docker-compose.yml (because of the -i option)
yq e -i 'del(.services.nginx.ports)' docker-compose.yml
You can execute this command on your deployment script or manually before your docker-compose up -d
There's also an open issue on docker-compose, that you may want to check once in a while.
Just keep the docker-compose.yml super simple and add the ports in another file docker-compose.develop.yml, then run it like docker-compose -f docker-compose.yml -f docker-compose.develop.yml up.
This way you can separate it from your docker-compose.override.yml file.
So you will have three files:
|- docker-compose.yml # no ports specified
|- docker-compose.override.yml # ports 8080:8080
|- docker-compose.develop.yml #ports 80:80
Refer to this post for longer explanation: https://mindbyte.nl/2018/04/04/overwrite-ports-in-docker-compose.html
I've faced the same problem. The proposed solution with docker-compose.override.yml sounds pretty well and is also an official one.
Although for some of my own projects I've applied the erb template engine to make docker-compose.yml.erb file compile for multiple environments. In short I use:
COMPOSE_TEMPLATE_ENV=production erb docker-compose.yml.erb > docker-compose.yml
COMPOSE_TEMPLATE_ENV=production erb docker-compose.yml.erb > docker-compose-production.yml
And then I can use the ENV['COMPOSE_TEMPLATE_ENV'] in my template and also the syntax of ERB, so only one file to configure and no worries about piplening them properly. Here's the short post article I've written about it
Use .override.yml file for overriding properties and a clear separation of properties need to be overridden
docker-compose.override.yml
Example:
version: '3.5'
services:
nginx:
ports:
- 80:80
Default:
docker-compose up
will use your docker-compose.yml and docker-compose.override.yml files
Reference: docker-compose multiple compose
I have a .env file with a large list of variables in the root of my node project. In the root is also my docker-compose.yml file:
version: '3'
services:
api:
build: .
env_file:
- .env
ports:
- 5000:5000
I'm trying to pass in these variables to Docker (from here), but when I run docker-compose up, I'm running into an error in my node app because it doesn't have access to these variables. What am I doing wrong? Thanks.
Change the file name to something different like node.env. The .env file is used by the docker-compose itself to define the content of the docker-compose file itself.
Example:
$ cat .env
TAG=v1.5
$ cat docker-compose.yml
version: '3'
services:
web:
image: "webapp:${TAG}"
The content of this file is not used inside the docker instance itself.
For production deployment, I don't want shared volumes. So, I have an override file but this does not remove the volumes.
Is there a way to remove shared volumes in an override file? I'd like to avoid having an override just for development, because that seems clunky to use.
This is my docker-compose.yml:
version: '2'
# other services defined here
services:
web:
build:
context: .
# other configuration
volumes:
- .:${APP_DIR}
And my docker-compose.prod.yml:
version: '2'
services:
web:
volumes: []
restart: always
I'm assuming you want to use docker-compose up to start the development version, and have a second config file to extend it for production.
If you want to make sure to override volumes completely, use a third config file, docker-compose.override.yml. Put all your volume definitions for development in there.
docker-compose up extends the base config with this file by default. But when you do something like docker-compose -f docker-compose.yml -f production.yml, the docker-compose.override.yml file won't be loaded, and you'll get only the volumes from the production file.
When merging a list entry in docker-compose, it adds new maps but doesn't remove existing volume mappings.
You can implement this by either making dev have the override file, or up to version 2.1 you can extend a common docker file rather than applying overrides which lets to devs point to a single file.
This could be your docker-compose.yml:
version: '2'
# other services defined here
services:
web:
extends:
file: docker-compose.prod.yml
service: web
build:
context: .
restart: no
volumes:
- .:${APP_DIR}
And your docker-compose.prod.yml would contain all the common configuration and prod settings:
version: '2'
services:
web:
# other configuration
restart: always