I am using the same docker-compose.yml file for multiple projects. I am really lazy, so I don't want to start them with docker-compose -p $PROJECT_NAME up.
As of Docker version 17.06.0, is it possible to set the variable directly in the docker-compose.yml file?
UPDATE: You can now use the top-level name property in your docker-compose YAML file. This is available from Docker Compose v2.3.3
This is the result of the #745 proposal. An issue which persevered for about 8 years.
Previously:
Right now, we can use the .env file to set the custom project name like this:
COMPOSE_PROJECT_NAME=SOMEPROJECTNAME
It's not flexible, but it's better than nothing. Currently, there is an open issue regarding this as a proposal.
I know this question was asked a long time ago, but I ran into the same problem. There's a suggestion to add the feature https://github.com/docker/compose/issues/745, but they don't want to.
However, I have a Makefile in the root of the directory and then you can add something like in the Makefile:
.PHONY: container-name
container-name:
docker-compose -p $PROJECT_NAME up -d container-name
and then run make container-name
I know it isn't what you asked for, but could maybe make your life a bit easier.
220806 UPDATE: you can now use the top-level name property in your docker-compose YAML file.
This is the result of the #745 proposal.
Update as on Docker Compose version 2.3.3, name can be given in the compose file, but please note the following as per documentation compose-spec at github.com., Compose official documentation
Whenever project name is defined by top-level name or by some custom mechanism, it MUST be exposed for interpolation and environment variable resolution as COMPOSE_PROJECT_NAME.
name: stitch
services:
foo:
image: busybox
environment:
- COMPOSE_PROJECT_NAME
command: echo "I'm running ${COMPOSE_PROJECT_NAME}"
Previously proposed solution :
You can add them as your environment variables which are available through the session in which you are bringing up your containers using the docker compose.
Ie, if you wanted to use $PROJECT_NAME somewhere inside your docker-compose.yaml then if this variable has a value in your session, then it would be picked up.
Inside the yaml you can assign it to anything as you want it. You want as a commandline arg to some script, even that is also possible. ie,
working_dir: /opt
command: /bin/bash -c './script.sh ${PROJECT_NAME}'
volumes:
- /var/run/:/host/var/run/
I'm using
docker version : Docker version 17.09.0-ce, build afdb6d4
docker-compose version : docker-compose version 1.14.0, build c7bdf9e
Related
I have been using docker-compose, but noticed there is also a docker compose (without the dash).
I have not been able to quickly determine the differences between the two forms by googling.
Anyone?
docker compose's help:
docker-compose's help:
The docker compose (with a space) is a newer project to migrate compose to Go with the rest of the docker project. This is the v2 branch of the docker/compose repo. It's been first introduced to Docker Desktop users, so docker users on Linux didn't see the command. In addition to migrating to Go, it uses the compose-spec, and part of the rewrite may result in behavior differences.
The original python project, called docker-compose, aka v1 of docker/compose repo, has now been deprecated and development has moved over to v2. To install the v2 docker compose as a CLI plugin on Linux, supported distribution can now install the docker-compose-plugin package. E.g. on debian, I run apt-get install docker-compose-plugin.
Brandon Mitchell from docker's Captain Program replied to the github issue I opened on this as follows:
The docker/compose-cli project is in an in-between state, where it's not available in upstream releases of the docker-cli Linux packages, but is being included in Docker Desktop. The documentation pages normally follow what's in the docker/cli, so having this released to Desktop early puts the documentation in a difficult position. I'm going to raise this issue with the Docker team to see how they'd like to handle it.
Update: from docker github issue:
gtardif commented 2 days ago
compose command reference doc is now live
new docker-compose command reference
Quote from https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command
Compose V2 and the new docker compose command
Important
The new Compose V2,
which supports the compose command as part of the Docker CLI, is now available.
Compose V2 integrates compose functions into the Docker platform,
continuing to support most of the previous docker-compose features and flags.
You can run Compose V2 by replacing the hyphen (-) with a space,
using docker compose, instead of docker-compose.
In addition to what has been already said here, I have noticed an important difference between the two.
In our setup, the docker-compose.yml file is located in a template folder. This way we can run multiple instances of the same project based on the same template. The local instance has its own folder with its own .env file (and also its own volumes).
There is also a template .env file in the template folder : copied and adapted to the instance folder using a script.
In order to work, the docker-compose.yml file looks like this, in the template folder :
version: "3"
services:
wordpress:
image: wordpress
container_name: "${COMPOSE_PROJECT_NAME}_wordpress"
env_file:
- ${PWD}/.env
...
And the local instance .env file :
# compose file location
COMPOSE_FILE=../templateFolder/docker-compose.yml
# this instance name
COMPOSE_PROJECT_NAME=foo
In this context :
with docker-compose, the .env file is read in the instance location, which is expected
with docker compose, the .env file is read in the template location !
To override this, we had to rename the template .env file into dotEnv.
This behavior is very lightly described here : https://docs.docker.com/compose/#multiple-isolated-environments-on-a-single-host
If it not yet included in the docker installation, docker compose can be installed on Linux as CLI plugin.
COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r '.tag_name')
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/$COMPOSE_VERSION/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
See https://docs.docker.com/compose/cli-command/#installing-compose-v2
If you do not want to have changes, but desire the original legacy docker-compose functionality, also known as Compose standalone vs. Compose plugin, you can do the following:
# Run as root
VERSION=v2.12.2
curl -SL https://github.com/docker/compose/releases/download/$VERSION/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
chmod a+x /usr/local/bin/docker-compose
# Test it
docker-compose
This allows you to e.g. keep using docker-compose in shell scripts.
Check versions on this page.
I have been using docker-compose, but noticed there is also a docker compose (without the dash).
I have not been able to quickly determine the differences between the two forms by googling.
Anyone?
docker compose's help:
docker-compose's help:
The docker compose (with a space) is a newer project to migrate compose to Go with the rest of the docker project. This is the v2 branch of the docker/compose repo. It's been first introduced to Docker Desktop users, so docker users on Linux didn't see the command. In addition to migrating to Go, it uses the compose-spec, and part of the rewrite may result in behavior differences.
The original python project, called docker-compose, aka v1 of docker/compose repo, has now been deprecated and development has moved over to v2. To install the v2 docker compose as a CLI plugin on Linux, supported distribution can now install the docker-compose-plugin package. E.g. on debian, I run apt-get install docker-compose-plugin.
Brandon Mitchell from docker's Captain Program replied to the github issue I opened on this as follows:
The docker/compose-cli project is in an in-between state, where it's not available in upstream releases of the docker-cli Linux packages, but is being included in Docker Desktop. The documentation pages normally follow what's in the docker/cli, so having this released to Desktop early puts the documentation in a difficult position. I'm going to raise this issue with the Docker team to see how they'd like to handle it.
Update: from docker github issue:
gtardif commented 2 days ago
compose command reference doc is now live
new docker-compose command reference
Quote from https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command
Compose V2 and the new docker compose command
Important
The new Compose V2,
which supports the compose command as part of the Docker CLI, is now available.
Compose V2 integrates compose functions into the Docker platform,
continuing to support most of the previous docker-compose features and flags.
You can run Compose V2 by replacing the hyphen (-) with a space,
using docker compose, instead of docker-compose.
In addition to what has been already said here, I have noticed an important difference between the two.
In our setup, the docker-compose.yml file is located in a template folder. This way we can run multiple instances of the same project based on the same template. The local instance has its own folder with its own .env file (and also its own volumes).
There is also a template .env file in the template folder : copied and adapted to the instance folder using a script.
In order to work, the docker-compose.yml file looks like this, in the template folder :
version: "3"
services:
wordpress:
image: wordpress
container_name: "${COMPOSE_PROJECT_NAME}_wordpress"
env_file:
- ${PWD}/.env
...
And the local instance .env file :
# compose file location
COMPOSE_FILE=../templateFolder/docker-compose.yml
# this instance name
COMPOSE_PROJECT_NAME=foo
In this context :
with docker-compose, the .env file is read in the instance location, which is expected
with docker compose, the .env file is read in the template location !
To override this, we had to rename the template .env file into dotEnv.
This behavior is very lightly described here : https://docs.docker.com/compose/#multiple-isolated-environments-on-a-single-host
If it not yet included in the docker installation, docker compose can be installed on Linux as CLI plugin.
COMPOSE_VERSION=$(curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r '.tag_name')
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/$COMPOSE_VERSION/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
See https://docs.docker.com/compose/cli-command/#installing-compose-v2
If you do not want to have changes, but desire the original legacy docker-compose functionality, also known as Compose standalone vs. Compose plugin, you can do the following:
# Run as root
VERSION=v2.12.2
curl -SL https://github.com/docker/compose/releases/download/$VERSION/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
chmod a+x /usr/local/bin/docker-compose
# Test it
docker-compose
This allows you to e.g. keep using docker-compose in shell scripts.
Check versions on this page.
I have a package (bootstrap) that is included in multiple local projects. Example:
project1/:
src/...
tests/...
vendor/bootstrap/...
project2/:
src/...
tests/...
vendor/bootstrap/...
This package has its internal tests and static code analyzers that I want to run inside each projectX/vendor/bootstrap folder. The tests and analyzers are run from docker containers. I.e. bootstrap has docker-compose.yml with some configuration:
version: '3.7'
services:
cli:
build: docker/cli
working_dir: /app
volumes:
- ./:/app
tty: true
The problem is when I run something inside project1/vendor/bootstrap, then switch to project2/vendor/bootstrap and run something there, docker thinks that I execute containers from project1. I believe it's because of the same folder name as Docker Compose generates container names as [folder-name_service-name]. So when I run docker-compose exec cli sh it checks if there is a running container bootstrap_cli, but it can be created within another bootstrap folder of another project.
Example of docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
128c3e834df4 bootstrap_cli "docker-php-entrypoi…" 55 minutes ago Up 55 minutes bootstrap_cli
NAMES is the same for containers in all these projectX folders.
There is an option to add container_name: bootstrap_project1_cli, but it seems Docker Compose ignores it when searching for a running container.
So is it possible to differentiate containers of the same name and have all of them at the same time?
Have a look at this github issue:
https://github.com/docker/compose/issues/2120
There are two options to set the COMPOSE_PROJECT_NAME. Use the -p commandline flag or the COMPOSE_PROJECT_NAME environment variable. Both are documented here: https://docs.docker.com/compose/reference/overview/#compose-project-name
When you run docker-compose, it needs a project name for the containers. If you don't specify the -p option, docker-compose looks for an environment varaible named COMPOSE_PROJECT_NAME. If both are not set, it defaults to the current working directory. Thats the behaviour you have.
If you don't want to add a commandline parameter, you can specify the environment variable in your .env file inside the directory of your docker compose file. See https://docs.docker.com/compose/env-file/
docker-compose is basically just a wrapper around the docker cli. It provides some basic scoping for the services inside a compose file by prefixing the containers (networks and volumes, too) with the COMPOSE_PROJECT_NAME value. If not configured differently, this value corresponds to the directory name of the compose file. You can overwrite this by having according environment variables set. A simple solution would be to place an .env file into the bootstrap directories that contains an instruction like
COMPOSE_PROJECT_NAME=project1_bootstrap
which will lead to auto-generated container names like e.g. project1_bootstrap_cli_1
Details:
https://docs.docker.com/compose/reference/envvars/
https://docs.docker.com/compose/env-file/
We are using docker in a team of developers. We have on project all devs work on. Since we do not want to have one docker-compose.yml for each developer we use environment variables to pass the username to docker-compose. Inside docker-compose we have something like this
services:
myservice:
image: myimage
container_name: ${user}_myservice
This used to work very well for us but has stopped working lately. Assume there are two users. The first user runs docker-compose up myservice launching ${user1}_myservice. When the second user issues the same command, the second user will kill the container running under ${user1}_myservice and start ${user2}_myservice.
Somehow it seems that docker services are now linked directly and not only through the container_name variable as before.
We recently upgraded docker to Docker version 17.09.0-ce, build afdb6d4. I attribute the change to the "new" docker version. I have tried downgrading docker-compose to previous versions and it seems this is not related to docker-compose.
UPDATE
Inspired by the answer below we found the following workaround:
We set the env variable COMPOSE_PROJECT_NAME to be the username on login of the user on the host. Then we extend the service name in our docker-compose.yml files to be <proj>_<service>, thereby avoiding any conflicts between identical service names across projects.
Rather than mucking about with variables in docker-compose.yml, it's probably easier just to make use of the --project-name (-p) option to docker-compose.
Normally, docker-compose derives the project name from the name of the directory that contains your docker-compose.yaml file. So if two people try to start an application from a directory named myapp, they will end up with a conflict because both instances will attempt to use the same name.
However, if they were to run instead:
docker-compose --project-name ${USER}_myapp ...
Then docker-compose for each user would use different project names (like alice_myapp and bob_myapp) and there would be no conflict.
If people get tired of using the -p option, they could create a .env like this:
COMPOSE_PROJECT_NAME=alice_myapp
And this would have the same effect as specifying -p alice_myapp on the command line.
I'm wondering if I've stumbled on a bug or that there's something not properly documented about variable substitution on Windows in combination with Docker Machine and Compose (installed version of docker is 1.11.1).
If I run the "docker-compose up" command for a yml file that looks like this:
volumes:
- ${FOOBAR}/build/:/usr/share/nginx/html/
And this variable doesn't exist docker compose will correctly complain about it:
The foobar variable is not set. Defaulting to a blank string.
However, when I change it to an existing environment variable:
volumes:
- ${PROJECT_DIR}/build/:/usr/share/nginx/html/
It will then not properly start the container and displays the following error (trying to access the nginx container will give you a host is unreachable message):
ERROR: for nginx rpc error: code = 2 desc = "oci runtime error: could not synchronise with container process: not a directory"
If I run the echo command in the Docker Quickstart Terminal it will output the correct path that I've set in the environment variable. If I replace the ${PROJECT_DIR} with the environment variable value the container runs correctly.
I get the same type of error message if I try to use the environment variable for the official php image instead of the official nginx image. In both cases the docker compose file works if I substitute ${PROJECT_DIR} text with the content of the environment variable.
So is this a bug or am I missing something?
After some mucking about I've managed to get the containers to start correctly without error messages if I use the following (contains the full path to the local files):
volumes:
- ${PROJECT_DIR}:/usr/share/nginx/html/
The nginx container is then up and running though it cannot find the files then anymore. If I replace the variable with the path it contains it then can find the files again.
Above behaviour isn't consistent. When I added a second environment variable for substitution it gave the oci runtime error. Kept giving it when I removed that second variable and only started working again when I also removed the first variable. After that it suddenly accepted ${PROJECT_DIR}/build/ but still without finding files.
Starting a bash session to the nginx container shows that the mount point for the volume contains no files.
I'm really at a loss here what docker is doing and what it expects from me. Especially as I have no idea to what it is expanding the variables in the compose file.
In the end the conclusion is that variable substitution is too quirky on Windows with Docker Machine to be useful. However, there is an alternative to variable substitution.
If you need a docker environment that does the following:
Can deploy on different computers that don't run the same OS
Doesn't care if the host uses Docker natively or via Virtual Box (this can require path changes)
Then your best bet is to use extending.
First you create the docker-compose.yml file that contains the images you'll need. For example an php image with MySQL:
php:
image: 5.5-apache
links:
- php_db:mysql
- maildev:maildev
ports:
- 8080:80
php_db:
image: mariadb
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: examplepass
You might notice that there aren't any volumes defined in this docker-compose file. That is something we're going to define in a file called docker-compose.override.yml:
php:
volumes:
- /workspaces/Eclipse/project/:/var/www/html/
When you have both files in one directory docker-compose does something interesting. It combines them into one adding/overwriting settings in the docker-compose.yml with those present in docker-compose.override.yml.
Then when running the command docker-compose up it will result in a docker run that is configured for the machine you're working on.
You can get similar behaviour with custom files names if you change a few things in your docker-compose command:
docker-compose -f docker-compose.yml -f docker-compose.conf.yml up
The detail is that docker-compose can accept multiple compose files and it will combine them into one. This happens from left to right.
Both methods allows you to create a basic compose file that configures the containers you need. You then can override/add the settings you need for the specific computer you're running docker on.
The page Overview of docker-compose CLI has more details on how these commands work.