All we have such a dashboard in Docker Desktop, where we can look at containers that were started with Docker-Compose.
At the picture below I have 3 containers that were started with one docker-compose1.yml (frontend, backend, db).
The fourth container db-v4 that was created via another docker-compose2.yml file.
So my question is -- How can I separate then into this dashboard? I have tried to use different networks, but this didn't work out.
For example, first container will be in one group and other three are in another group
Normally it is sorted by project name. Per default the project name is the folder name where the yml-file is located. You may:
Use different folders for yml-files
Or set name by env:
#.env.1 listing:
COMPOSE_PROJECT_NAME=MyProject1
#.env.2 listing:
COMPOSE_PROJECT_NAME=MyProject2
# Run with
docker-compose --env-file .env.1 -f project1.yml up
docker-compose --env-file .env.2 -f project2.yml up
or using the --project-name flag, short -p
# Run with
docker-compose -p MyProject1 -f project1.yml up
docker-compose -p MyProject2 -f project2.yml up
By the way: It is not just optics. Docker uses this project name to share resources and handles it as one bundle. If they are separate logical units you SHOULD use different names.
Related
I want to recreate a service, including its volumes. The closest I got was the following commands:
docker-compose stop foo
docker-compose rm -f foo
docker-compose up --renew-anon-volumes -d foo
docker-compose start foo
The issue here is --renew-anon-volumes recreates all services that have anonymous volumes, not just foo's volumes. If I don't use --renew-anon-volumes, then I think I need a named volume to do docker volume rm myvolume. However, with named volumes, Docker Compose always prepends a project name. Since my script doesn't know the project name, I can't programmatically delete the volume. I can't enforce that the user uses a particular project name. I know I can set the project name using an environment variable, but there's no guarantee that the user won't run Docker Compose with a different project name.
I think there are 2 potential solutions:
Make --renew-anon-volumes only recreate the volumes for the service I specified
Use a named volume and somehow figure out the correct prefix
Are either of these doable, or is there another solution?
Many roads leading to Rome, depending on your prerequisites:
Do docker volume ls and regex the result for your named volume (just working if volume name is unique)
Use external volumes and volume create them with known names by bootstrap script before running docker-compose up (not working if volumes must be instantiated)
Set project name to a known value. Normally it takes the folder name, but can explicitly given in docker-compose command (-p NAME) or by environment variable (COMPOSE_PROJECT_NAME=NAME).
Setup a dummy compose-file just containing this single service with its volumes for your script. Doing a docker-compose -f 'your-down-file.yml' down -v which removes all named and anonymous volumes belonging to this service and docker-compose -f .. up on this file.
Edit (#DavidMaze):
You're right, docker compose recognizes that fact. But it does NOT remove it, just warning. If you want to remove all "orphans" you need the flag --remove-orphans.
But for some reasons the down does not remove volumes then, even if flag -v is given. This could be reported because it is not behaving like described.
And errata: the flag -f must go before up/down and not after!
docker-compose rm has a -v option to delete anonymous volumes attached to a container, and also a -s option to stop the container. For your particular use case it should be enough to:
docker-compose rm -s -f -v foo
docker-compose up -d foo
This will only help for anonymous volumes, that is, where the Compose file has volumes: with only a container path and there is no corresponding top-level volumes: entry. I don't immediately see a Compose option to list, remove, or otherwise manage named volumes that Compose created.
I'm new to the world of containers and specially when it comes to Docker Compose. I'm confused about some concepts and I can't find information about them.
Basically I want to know if I can handle settings in different "docker-compose.yml" files in a isolated manner. I explain better... I would like to know if I can up or stop resources referring to a specific "docker-compose.yml" individually.
PLUS:
To better explain my doubt I'll show you some conjectures about what I'm trying to explain.
It seems to me that it is possible to have multiple configurations for Docker Compose using different ".yml" files like the example below...
EXAMPLE
docker-compose -f docker-compose.a.yml -f docker-compose.b.yml up -d
... and that I can also handle each of these settings individually, such as stopping all the resources referring to a specific docker-compose.yml...
EXAMPLE
docker-compose -f docker-compose.b.yml stop
[Ref(s).: https://gabrieltanner.org/blog/docker-compose#using-multiple-docker-compose-files , https://stackoverflow.com/q/29835905/3223785 , https://stackoverflow.com/questions/42287426/docker-multiple-environments , https://runnable.com/docker/advanced-docker-compose-configuration ]
Yes, it is possible. I'm not exactly sure what you are trying to do, but to be able to manage the services using -f option the way that you described, there shouldn't be a service with the same name on multiple files.
For example, if you have a service called db in docker-compose.a.yml and one other db service in docker-compose.b.yml. The following command will only built one container for db service:
docker-compose -f docker-compose.a.yml -f docker-compose.b.yml up -d
Take a look at -p option. It will make a project with the services isolated inside it. Then you can manage them using following commands with the same docker-compose.yml file:
docker-compose -p foo up -d
docker-compose -p foo stop [service_name]
yes you can.
It is just a matter of preference but i usually create a folder for every project i have, each of them have a unique docker-compose.yml file in it with all its dependencies (frontend / database /redis)
Then to start a specific project i just go inside its folder and run docker-compose up. it then only starts this project without touching others.
you can also type this if you only want to start redis.
docker-compose up redis
All docker-compose subcommands (up, stop, down...) must be executed consuming a docker-compose<.SOME_OPT_VAL>.yml file.
This docker-compose.yml file must be in the folder where the docker-compose command is executed or must be informed via the -f flag. This way, these subcommands will be executed on the "services" (resources) defined in the docker-compose.yml file.
There is also the possibility of defining the service where a certain subcommand will be executed...
MODELS
docker-compose <SUBCOMAMND> <OPT_SERVICE_NAME>
docker-compose -f <DOCKER_COMPOSE_YML> <SUBCOMAMND> <OPT_SERVICE_NAME>
EXAMPLES
docker-compose stop api
docker-compose -f docker-compose.yml stop api
I have a few running docker containers created by executing docker-compose up.
Is there any way to get the exact file path of the corresponding docker-compose.yml file used to start these containers, just by inspecting the running containers?
As far as I can see, docker inspect CONTAINER_NAME does not provide this information, nor does docker-compose provide a method to get compose-related information from a running container.
What I'd like to do in a script:
list certain running containers on a docker host
get the corresponding docker-compose.yml file locations
use docker-compose to restart all containers of the corresponding docker-compose projects at once
The answer to this question seems to have changed with new versions of docker-compose.
There is a label "com.docker.compose.project.working_dir": "/var/opt/docker", that points to the directory where I started docker-compose. I have not checked if that is pwd or the actual location of the docker-compose.yml file.
This got me interesting information about docker-compose:
samuel#vmhost1:~$ docker inspect fc440a1afbaa | grep com.docker.compose
"com.docker.compose.config-hash": "89069285a4783b79b421ea84f2b652becbdee148fbad095a6d9d85aab67ececc",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "docker",
"com.docker.compose.project.config_files": "docker-compose.yml",
"com.docker.compose.project.working_dir": "/var/opt/docker",
"com.docker.compose.service": "jenkins",
"com.docker.compose.version": "1.25.0"
samuel#vmhost1:~$
I'm running docker-compose.yml configuration version 3.6
It is not currently possible.
As an alternative might find the following helpful:
Use docker ps -a | grep <certain_container>
Use locate docker-compose.yml and find the one that you want
Use docker-compose restart (do docker-compose to see option)
You can identify it using the inspect command, as follow:
docker inspect <container_id> | grep compose
Update: Since this was asked, docker compose v2 was released, which is written in Go and accessible from docker compose instead of docker-compose (there may also be a shim directing docker-compose to this new version depending on your install). This version now embeds the directory into the image labels that you can retrieve with:
docker container inspect ${container_name_or_id} \
--format '{{ index .Config.Labels "com.docker.compose.project.working_dir" }}'
This isn't perfect for the OP's request since there may be more than one compose file, the file could be located in a different directory from where compose was run, and it doesn't capture things like environment variables or profiles that may modify how compose starts the project. However I suspect it gets most people close enough to find the source.
If you're using an older version of compose, you can use one of the options in the original answer below:
As far as I can see, docker inspect CONTAINER_NAME does not provide
this information, nor does docker-compose provide a method to get
compose-related information from a running container.
From an already running container that you do not control, the information is not there. You can infer the location using bind mount directories if the container creates any host mounts to relative directories. Otherwise, it's possible to deploy containers without compose, and it's possible to use compose without a compose file on the filesystem (piped via stdin), and compose does not store these details on running containers for you.
What I'd like to do in a script:
list certain running containers on a docker host
get the corresponding docker-compose.yml file locations
use docker-compose to restart all containers of the corresponding docker-compose projects at once
If you just want to run a restart on all containers in the same project, you don't need the first two steps, or even docker-compose. Instead, you can run:
docker ps --filter "label=com.docker.compose.project=${your_compose_project}" -q \
| xargs docker restart
Which uses a label docker-compose adds to each project it deploys.
If you want to proactively store the compose file location for later use, you can inject that as a label in your compose file:
version: '2'
services:
test:
image: busybox
command: tail -f /dev/null
labels:
COMPOSE_PATH: ${PWD} # many Linux shells define the PWD variable
If your shell does not set a ${PWD} environment variable, you can start compose with:
PWD=$(pwd) docker-compose up -d
Then you can later inspect containers for this label's value with:
docker inspect --format '{{.Config.Labels.COMPOSE_PATH}}' ${your_container_id}
And you can chain a filter and inspect command together to find the path for a specific project:
docker ps --filter "label=com.docker.compose.project=${your_compose_project}" -q \
| xargs docker inspect --format '{{.Config.Labels.COMPOSE_PATH}}'
you know, your question turns to be a useful answer to the same issue I have.
I used docker inspect <containerID> and then it gave me the location that I should look into. specifically in these lines:
HostConfig": {
"Binds": [
....
...
],
If you mounted a local volume, e.g ./data then inspecting the container will give you the path, e.g docker inspect peertube_peertube_1 | jq .[0].HostConfig.Binds. It doesn't work for containers without volumes but it's rare enough.
I decided to change the project name of a docker composition:
$ docker-compose -p old_name up -d # Before
Starting old_name_web_1
$ docker-compose -p new_name up -d # After
Creating new_name_web_1
But I don't wanted to delete my containers, so I renamed them:
$ docker rename old_name_web_1 new_name_web_1
...
I thought docker-compose was based on container names, but it does not seem to be the case:
$ docker-compose -p new_name up -d
ERROR: for web Cannot create container for service web: Conflict. The name "/new_name_web_1" is already in use by container 4930deaabb[...]. You have to remove (or rename) that container to be able to reuse that name.
ERROR: Encountered errors while bringing up the project.
How can I relink my old containers to the new composition ?
It looks like you are using one of the newer versions of docker compose which tracks containers by labels assigned to them rather than by their names. That is why renaming the container didn't work.
Updating labels
You can check container's labels through the docker inspect command.
$ docker inspect --format='{{json .Config.Labels }}' container_name
The project name is the value of the 'com.docker.compose.project' label.
Moving an existing container to a new project is as easy as changing the value of that label. However it is not yet supported by Docker CLI. There is an open issue requesting that feature.
Workaround
It still can be achieved by directly editing the configuration file of that particular container. There you will find labels currently assigned to that container.
$ nano /var/lib/docker/containers/$container_id/config.v2.json
Assign the new project name to the 'com.docker.compose.project' label and save the file. Next you have to restart the daemon. Otherwise the changes will not be visible to docker.
$ systemctl daemon-reload
While it is true docker-compose reuse existing containers, this comment mentions:
docker-compose by default uses the folder name of the yml file as the project name, and prefix that name to all container names.
This could explain why docker-compose up did not pick up the new container name.
I am working with Docker and I have a web-app that requires the following:
Tomcat
PostgreSQL
MongoDB
To install item 2 and 3 I do the following:
I can run a command for PostgreSQL like :
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
For Mongodb I run:
docker run --name some-mongo -d mongo
For Tomcat, I have a Dockerfile with Tomcat and copying my war to the apps folder. I build the image using Docker and run it.
My question is whether there is a better way to coordinate this step by step via separate script? Is Docker compose the solution for this?
thanks
A Dockerfile is a recipe for building an image, which is a template for starting containers. To describe a system that is made of multiple containers using standardized images, you would use docker-compose, not a new Dockerfile. You would use a Dockerfile to customize a pre-existing docker image, like mysql or node or ubuntu, for some specific use.
docker-compose allows you to express multiple docker commands as a .yml file in a specific format.
You can then use docker-compose up to start the set of containers.
The docker-compose .yml file for your example might start looking somewhat like this
some-postgres:
environment:
POSTGRES_PASSWORD:mysecretpassword
image: postgres
some-mongo:
image: mongo
You would add links between the containers with a links: line. These and other details are in the docs.
Basically docker-compose is just a yaml file implementation of docker run.
As docker run has arguments passed to it, these exact same arguments are stipulated in docker compose in a yaml format instead of on the command line.
Docker compose supports multiple containers too.
Docker compose has a few other nice features such as docker-compose logs , this command gives logs of all containers started by compose.