How do you name your docker-compose.yml files? Is there a convention to follow?
With Dockerfiles <purpose>.Dockerfile seems to work the best being instantly recognized by VSCode and PyCharm.
I like the idea of structuring docker/compose files into folders so that default names could be used, but as far as I know docker can't see files up the tree creating different problems.
According to the -f documentation:
If you don’t provide this flag on the command line, Compose [...] looks for a
docker-compose.yml and a docker-compose.override.yml file.
If you don't want to use the -f flag you can use the default name docker-compose.yml and override it with docker-compose.override.yml.
However, if you use -f, since you'll provide the filename, you can use whatever you want.
I think a good way to name them depending on the environment could be docker-compose.{env}.yml and place them at the top level directory:
docker-compose.prod.yml
docker-compose.dev.yml
docker-compose.test.yml
docker-compose.staging.yml
And you can use the default docker-compose.yml to define the base configuration that is common to all environments.
Related
I start to work with a new project using Docker and I can see there is no docker-compose.yml file there, but a few files like docker-compose.myname1.yml, docker-compose.myname2.yml instead. Trying to run docker-compose up I get an error:
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
So I wonder if that's possible for a Docker project to work without the docker-compose.yml file and if it is, what conditions should be met. Maybe that's the matter of some specific version or environment?
https://docs.docker.com/compose/reference/
"Options:
-f, --file FILE Specify an alternate compose file
(default: docker-compose.yml)"
so in your case that would be:
docker-compose -f dockcer-compose.myname1.yml up
docker-compose -f dockcer-compose.myname2.yml up
There are two different issues presented in your question:
The reason for getting an error
When running docker-compose up, you must be in a directory that contains docker-compose.yml (or '.yaml), as indicated by the error you are receiving.
The reason you are getting an error is either because there is no such file in the current directory, or (less likely) the environment variable COMPOSE_FILE is set to something else.
Different names for the docker-compose file
In order to use an alternative file, you can take one of these approaches:
Run docker-compose with the --file argument:
$ docker-compose --help
-f, --file FILE Specify an alternate compose file
(default: docker-compose.yml)
Set the COMPOSE_FILE environment variable prior to running any docker-compose command.
We have docker-compose-dev.yaml, docker-compose-staging.yaml, docker-compose-prod.yaml
We would like to refactor some of the service definitions inside docker-compose-common.yaml, so rest of the lifecycle specific definitions can inherit.
Is it possible to reuse service definitions across multiple docker-compose files, in the above fashion?
Yes, this is possible, please check the official documentation here.
The file containing the common definitions must be named docker-compose.yml, for the overrides you can keep the names you already use (-dev, -staging, etc.).
Just make sure you use the -f parameter when running docker compose:
docker-compose -f docker-compose.yaml -f docker-compose-prod.yaml up -d
I have a project with components base on Docker and orchestrated with docker-compose. Some of them are optional, and can be added at runtime.
I can think about two ways to achieve that:
Create a new serviceA.yml compose file and run it as a separate project
Add serviceA to my base compose.yml and run it again
What is the preferred option to do that?
I've also seen that you can combine docker-compose files with the extend keyword, but I don't think this can fit, since I have a variable number of services that I can add at runtime.
I usually end up having multiple yml files.
Then you can add several -f flags to docker-compose command to indicate which services to run.
For instance, having:
docker-compose.yml: containing basic services
docker-compose-additional-services.yml: containing additional/optional services.
You can execute to start:
docker-compose -f docker-compose.yml -f docker-compose-additional-services.yml up
And it will start all services.
Note, that all services are merged as if they were in a single file, so you can reference (depends_on, link) services from one file to the other.
I'm trying to wrap my head around Docker, but I'm having a hard time figuring it out. I tried to implement it in my small project (MERN stack), and I was thinking how do you distinct between development, (maybe staging), and production environments.
I saw one example where they used 2 Docker files, and 2 docker-compose files, (each pair for one env, so Dockerfile + docker-compose.yml for prod, Dockerfile-dev + docker-compose-dev.yml for dev).
But this just seems like a bit of an overkill for me. I would prefer to have it only in two files.
Also one of the problem is that e.g. for development I want to install nodemon globally, but not for poduction.
In perfect solution I imagine running something like that
docker-compose -e ENV=dev build
docker-compose -e ENV=dev up
Keep in mind, that I still don't fully get docker, so if you caught some of mine misconceptions about docker, you can point them out.
You could take some clues from "Using Compose in production"
You’ll almost certainly want to make changes to your app configuration that are more appropriate to a live environment. These changes may include:
Removing any volume bindings for application code, so that code stays inside the container and can’t be changed from outside
Binding to different ports on the host
Setting environment variables differently (e.g., to decrease the verbosity of logging, or to enable email sending)
Specifying a restart policy (e.g., restart: always) to avoid downtime
Adding extra services (e.g., a log aggregator)
The advice is then not quite similar to the example you mention:
For this reason, you’ll probably want to define an additional Compose file, say production.yml, which specifies production-appropriate configuration. This configuration file only needs to include the changes you’d like to make from the original Compose file.
docker-compose -f docker-compose.yml -f production.yml up -d
This overriding mechanism is better than trying to mix dev and prod logic in one compose file, with environment variable to try and select one.
Note: If you name your second dockerfile docker-compose.override.yml, a simple docker-compose up would read the overrides automatically.
But in your case, a name based on the environment is clearer.
Docker Compose will read docker-compose.yml and docker-compose.override.yml by default. Understanding-Multiple-Compose-Files
You can set a default docker-compose.yml and different overwrite compose file. For example, docker-compose.prod.yml docker-compose.test.yml. Keep them in the same place.
Then create a symbolic link named docker-compose.override.yml for each env.
Track docker-compose.{env}.yml files and add docker-compose.override.yml to .gitignore.
In prod env: ln -s ./docker-compose.prod.yml ./docker-compose.override.yml
In test env: ln -s ./docker-compose.test.yml ./docker-compose.override.yml
The project structure will then look like this:
project\
- docker-compose.yml # tracked
- docker-compose.prod.yml # tracked
- docker-compose.test.yml # tracked
- docker-compose.override.yml # ignored & linked to override composefile for current env
- src/
- ...
Then you have done. In each environment, you can use the compose-file with the same command docker-compose up
If you are not sure, use docker-compose config to check if it's been override properly.
I'm working on overriding default docker-compose.yml file using docker-compose.override.yml as shown in this link, And I can able to specify ports and volumes in the override file.
In the similar way, is it also possible to specify version of the image which needs to be deployed? If no, what is the best way to handle such circumstance where we need to specify different version for the image?
Any help on this would be great.
Docker is already having that feature. I tried to override image name with simple docker-compose, it is working.
For example,
docker-compose.yml with content,
my-httpd:
image: httpd:latest
ports:
- "1110:80"
And docker-compose.override.yml with content,
my-httpd:
image: httpd:2.4
After the execution of docker-compose -d, here is the docker ps info,
It uses ports from docker-compose.yml (not overrided) and image from docker-compose.override.yml as it is getting overridden here.
Note: It you have different names and location, you could use it like the following command instead of docker-compose -d,
docker-compose -f <DOCKER COMPOSE LOCATION> -f <OVERRIDE FILE LOCATION> up -d
Edit 2:
Overriding in the above manner will only replace the non array values and the array variables will be merged.
For example, if I have ports in both files which is an array, it will bind both the ports instead of taking ports value just from the overriding file unlike the image part (non array).
There is (now?) a better option for what you want to do. From
https://docs.docker.com/compose/environment-variables/
It’s possible to use environment variables in your shell to populate values inside a Compose file:
web:
image: "webapp:${TAG}"
For example you need to have 3 versions of docker:
docker-composer-master.yml
docker-composer-dev.yml
docker-composer-docker.yml
You can create 3 file docker-composer and add content as the below picture