I'm working on overriding default docker-compose.yml file using docker-compose.override.yml as shown in this link, And I can able to specify ports and volumes in the override file.
In the similar way, is it also possible to specify version of the image which needs to be deployed? If no, what is the best way to handle such circumstance where we need to specify different version for the image?
Any help on this would be great.
Docker is already having that feature. I tried to override image name with simple docker-compose, it is working.
For example,
docker-compose.yml with content,
my-httpd:
image: httpd:latest
ports:
- "1110:80"
And docker-compose.override.yml with content,
my-httpd:
image: httpd:2.4
After the execution of docker-compose -d, here is the docker ps info,
It uses ports from docker-compose.yml (not overrided) and image from docker-compose.override.yml as it is getting overridden here.
Note: It you have different names and location, you could use it like the following command instead of docker-compose -d,
docker-compose -f <DOCKER COMPOSE LOCATION> -f <OVERRIDE FILE LOCATION> up -d
Edit 2:
Overriding in the above manner will only replace the non array values and the array variables will be merged.
For example, if I have ports in both files which is an array, it will bind both the ports instead of taking ports value just from the overriding file unlike the image part (non array).
There is (now?) a better option for what you want to do. From
https://docs.docker.com/compose/environment-variables/
It’s possible to use environment variables in your shell to populate values inside a Compose file:
web:
image: "webapp:${TAG}"
For example you need to have 3 versions of docker:
docker-composer-master.yml
docker-composer-dev.yml
docker-composer-docker.yml
You can create 3 file docker-composer and add content as the below picture
Related
I have a package (bootstrap) that is included in multiple local projects. Example:
project1/:
src/...
tests/...
vendor/bootstrap/...
project2/:
src/...
tests/...
vendor/bootstrap/...
This package has its internal tests and static code analyzers that I want to run inside each projectX/vendor/bootstrap folder. The tests and analyzers are run from docker containers. I.e. bootstrap has docker-compose.yml with some configuration:
version: '3.7'
services:
cli:
build: docker/cli
working_dir: /app
volumes:
- ./:/app
tty: true
The problem is when I run something inside project1/vendor/bootstrap, then switch to project2/vendor/bootstrap and run something there, docker thinks that I execute containers from project1. I believe it's because of the same folder name as Docker Compose generates container names as [folder-name_service-name]. So when I run docker-compose exec cli sh it checks if there is a running container bootstrap_cli, but it can be created within another bootstrap folder of another project.
Example of docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
128c3e834df4 bootstrap_cli "docker-php-entrypoi…" 55 minutes ago Up 55 minutes bootstrap_cli
NAMES is the same for containers in all these projectX folders.
There is an option to add container_name: bootstrap_project1_cli, but it seems Docker Compose ignores it when searching for a running container.
So is it possible to differentiate containers of the same name and have all of them at the same time?
Have a look at this github issue:
https://github.com/docker/compose/issues/2120
There are two options to set the COMPOSE_PROJECT_NAME. Use the -p commandline flag or the COMPOSE_PROJECT_NAME environment variable. Both are documented here: https://docs.docker.com/compose/reference/overview/#compose-project-name
When you run docker-compose, it needs a project name for the containers. If you don't specify the -p option, docker-compose looks for an environment varaible named COMPOSE_PROJECT_NAME. If both are not set, it defaults to the current working directory. Thats the behaviour you have.
If you don't want to add a commandline parameter, you can specify the environment variable in your .env file inside the directory of your docker compose file. See https://docs.docker.com/compose/env-file/
docker-compose is basically just a wrapper around the docker cli. It provides some basic scoping for the services inside a compose file by prefixing the containers (networks and volumes, too) with the COMPOSE_PROJECT_NAME value. If not configured differently, this value corresponds to the directory name of the compose file. You can overwrite this by having according environment variables set. A simple solution would be to place an .env file into the bootstrap directories that contains an instruction like
COMPOSE_PROJECT_NAME=project1_bootstrap
which will lead to auto-generated container names like e.g. project1_bootstrap_cli_1
Details:
https://docs.docker.com/compose/reference/envvars/
https://docs.docker.com/compose/env-file/
I have a docker image which is based off node:12. This image, along with others, are brought up via a docker-composer.yml file. The issue I am having is that I am unable to add extra_hosts to the container. In my yml file - I've got the following
my-app:
image: <image_name>
container_name: <container_name>
extra_hosts:
- "somehost:162.242.195.82"
This appears to work with other containers but not ones based of node:12. Is this functionality not available to images based off node:12? Or am I doing something wrong?
Check the base image of the node:12, you have other choices (Stretch, Alpine, Buster) https://hub.docker.com/_/node/
It is a very bad idea to alter the /etc/hosts of an image manually and to build a new image with an altered /etc/hosts containing specific infos.
Potential workaround:
Create a Dockerfile which uses node:12 as your parent image and then write your host into the /etc/hosts manually. Then update your compose setup to build from that Dockerfile.
As mentioned below, this is purely a hacky workaround. It may be worth trying the other node:12 distribution such as node:12-alpine.
I am using the same docker-compose.yml file for multiple projects. I am really lazy, so I don't want to start them with docker-compose -p $PROJECT_NAME up.
As of Docker version 17.06.0, is it possible to set the variable directly in the docker-compose.yml file?
UPDATE: You can now use the top-level name property in your docker-compose YAML file. This is available from Docker Compose v2.3.3
This is the result of the #745 proposal. An issue which persevered for about 8 years.
Previously:
Right now, we can use the .env file to set the custom project name like this:
COMPOSE_PROJECT_NAME=SOMEPROJECTNAME
It's not flexible, but it's better than nothing. Currently, there is an open issue regarding this as a proposal.
I know this question was asked a long time ago, but I ran into the same problem. There's a suggestion to add the feature https://github.com/docker/compose/issues/745, but they don't want to.
However, I have a Makefile in the root of the directory and then you can add something like in the Makefile:
.PHONY: container-name
container-name:
docker-compose -p $PROJECT_NAME up -d container-name
and then run make container-name
I know it isn't what you asked for, but could maybe make your life a bit easier.
220806 UPDATE: you can now use the top-level name property in your docker-compose YAML file.
This is the result of the #745 proposal.
Update as on Docker Compose version 2.3.3, name can be given in the compose file, but please note the following as per documentation compose-spec at github.com., Compose official documentation
Whenever project name is defined by top-level name or by some custom mechanism, it MUST be exposed for interpolation and environment variable resolution as COMPOSE_PROJECT_NAME.
name: stitch
services:
foo:
image: busybox
environment:
- COMPOSE_PROJECT_NAME
command: echo "I'm running ${COMPOSE_PROJECT_NAME}"
Previously proposed solution :
You can add them as your environment variables which are available through the session in which you are bringing up your containers using the docker compose.
Ie, if you wanted to use $PROJECT_NAME somewhere inside your docker-compose.yaml then if this variable has a value in your session, then it would be picked up.
Inside the yaml you can assign it to anything as you want it. You want as a commandline arg to some script, even that is also possible. ie,
working_dir: /opt
command: /bin/bash -c './script.sh ${PROJECT_NAME}'
volumes:
- /var/run/:/host/var/run/
I'm using
docker version : Docker version 17.09.0-ce, build afdb6d4
docker-compose version : docker-compose version 1.14.0, build c7bdf9e
Maybe I'm missing this when reading the docs, but is there a way to overwrite files on the container's file system when issuing a docker run command?
Something akin to the Dockerfile COPY command? The key desire here is to be able to take a particular Docker image, and spin several of the same image up, but with different configuration files. (I'd prefer to do this with environment variables, but the application that I'm Dockerizing is not partial to that.)
You have a few options. Using something like docker-compose, you could automatically build a unique image for each container using your base image as a template. For example, if you had a docker-compose.yml that look liked:
container0:
build: container0
container1:
build: container1
And then inside container0/Dockerfile you had:
FROM larsks/thttpd
COPY index.html /index.html
And inside container0/index.html you had whatever content you
wanted, then running docker-compose build would generate unique
images for each entry (and running docker-compose up would start
everything up).
I've put together an example of the above
here.
Using just the Docker command line, you can use host volume mounts,
which allow you to mount files into a container as well as
directories. Using my thttpd as an example again, you could use the
following -v argument to override /index.html in the container
with the content of your choice:
docker run -v index.html:/index.html larsks/thttpd
And you could accomplish the same thing with docker-compose via the
volume entry:
container0:
image: larsks/thttpd
volumes:
- ./container0/index.html:/index.html
container1:
image: larsks/thttpd
volumes:
- ./container1/index.html:/index.html
I would suggest that using the build mechanism makes more sense if you are trying to override many files, while using volumes is fine for one or two files.
A key difference between the two mechanisms is that when building images, each container will have a copy of the files, while using volume mounts, changes made to the file within the image will be reflected on the host filesystem.
This question is coming from an issue on the Docker's repository:
https://github.com/docker/compose/issues/942
I can't figure it out how to create a data container (no process running) with docker compose.
UPDATE: Things have changed in the last years. Please refer to the answer from #Frederik Wendt for a good and up-to-date solution.
My old answer: Exactly how to do it depends a little on what image you are using for your data-only-container. If your image has an entrypoint, you need to overwrite this in your docker-compose.yml. For example this is a solution for the official MySql image from docker hub:
DatabaseData:
image: mysql:5.6.25
entrypoint: /bin/bash
DatabaseServer:
image: mysql:5.6.25
volumes_from:
- DatabaseData
environment:
MYSQL_ROOT_PASSWORD: blabla
When you do a docker-compose up on this, you will get a container like ..._DatabaseData_1 which shows a status of Exited when you call docker ps -a. Further investigation with docker inspect will show, that it has a timestamp of 0. That means the container was never run. Like it is stated by the owner of docker compose here.
Now, as long as you don't do a docker-compose rm -v, your data only container (..._DatabaseData_1) will not loose its data. So you can do docker-compose stop and docker-compose up as often as you like.
In case you like to use a dedicated data-only image like tianon/true this works the same. Here you don't need to overwrite the entrypoint, because it doesn't exist. It seems like there are some problems with that image and docker compose. I haven't tried it, but this article could be worth reading in case you experience any problems.
In general it seems to be a good idea to use the same image for your data-only container that you are using for the container accessing it. See Data-only container madness for more details.
The other answers to this question are quite out of date, and data volumes have been supported for some time now. Example:
version: "3.9"
services:
frontend:
image: node:lts
volumes:
- myapp:/home/node/app
volumes:
myapp:
See
https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose for details and options.
A data only container (DOC) is a container that is created only to serve as a volume provider. The container itself has no function other than that other containers can mount it's volume by using the volumes_from directive.
The DOC has to run only once to create the volume. Other containers can reference the volumes in it even if it's stopped.
The OP Question:
The docker-compose.yml starts the DOC every time you do a docker-compose up. OP asks for an option to only create container and volume, and not run it, using some sort of an create_only: true option.
As mention in the issue from the OP's question:
you either create a data container with the same name as the one specified in the docker-compose.yml, and run docker-compose up --no-recreate (the one specified in docker-compose.yml won't be recreated).
or you run a container with a simple command which never returns.
Like: tail -f /dev/null