This question is coming from an issue on the Docker's repository:
https://github.com/docker/compose/issues/942
I can't figure it out how to create a data container (no process running) with docker compose.
UPDATE: Things have changed in the last years. Please refer to the answer from #Frederik Wendt for a good and up-to-date solution.
My old answer: Exactly how to do it depends a little on what image you are using for your data-only-container. If your image has an entrypoint, you need to overwrite this in your docker-compose.yml. For example this is a solution for the official MySql image from docker hub:
DatabaseData:
image: mysql:5.6.25
entrypoint: /bin/bash
DatabaseServer:
image: mysql:5.6.25
volumes_from:
- DatabaseData
environment:
MYSQL_ROOT_PASSWORD: blabla
When you do a docker-compose up on this, you will get a container like ..._DatabaseData_1 which shows a status of Exited when you call docker ps -a. Further investigation with docker inspect will show, that it has a timestamp of 0. That means the container was never run. Like it is stated by the owner of docker compose here.
Now, as long as you don't do a docker-compose rm -v, your data only container (..._DatabaseData_1) will not loose its data. So you can do docker-compose stop and docker-compose up as often as you like.
In case you like to use a dedicated data-only image like tianon/true this works the same. Here you don't need to overwrite the entrypoint, because it doesn't exist. It seems like there are some problems with that image and docker compose. I haven't tried it, but this article could be worth reading in case you experience any problems.
In general it seems to be a good idea to use the same image for your data-only container that you are using for the container accessing it. See Data-only container madness for more details.
The other answers to this question are quite out of date, and data volumes have been supported for some time now. Example:
version: "3.9"
services:
frontend:
image: node:lts
volumes:
- myapp:/home/node/app
volumes:
myapp:
See
https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose for details and options.
A data only container (DOC) is a container that is created only to serve as a volume provider. The container itself has no function other than that other containers can mount it's volume by using the volumes_from directive.
The DOC has to run only once to create the volume. Other containers can reference the volumes in it even if it's stopped.
The OP Question:
The docker-compose.yml starts the DOC every time you do a docker-compose up. OP asks for an option to only create container and volume, and not run it, using some sort of an create_only: true option.
As mention in the issue from the OP's question:
you either create a data container with the same name as the one specified in the docker-compose.yml, and run docker-compose up --no-recreate (the one specified in docker-compose.yml won't be recreated).
or you run a container with a simple command which never returns.
Like: tail -f /dev/null
Related
I am bringing up my project dependencies using docker-compose. So far this used to work
docker-compose up -d --no-recreate;
However today I tried running the project again after couple of weeks and I was greeted with error message
Creating my-postgres ... error
ERROR: for my-postgres Cannot create container for service postgres: b'Conflict. The container name "/my-postgres" is already in use by container "dbd06bb1d99eda6f075ea688df16e8b355e559e1759f084dee8f3cddfc535b0b". You have to remove (or rename) that container to be able to reuse that name.'
ERROR: for postgres Cannot create container for service postgres: b'Conflict. The container name "/my-postgres" is already in use by container "dbd06bb1d99eda6f075ea688df16e8b355e559e1759f084dee8f3cddfc535b0b". You have to remove (or rename) that container to be able to reuse that name.'
ERROR: Encountered errors while bringing up the project.
My docker-compose.yml file is
postgres:
container_name: my-postgres
image: postgres:latest
ports:
- "15432:5432"
Docker version is
Docker version 19.03.1, build 74b1e89
Docker compose version is
docker-compose version 1.24.1, build 4667896b
Intended behavior of this call is to:
make the container if it does not exist
start the container if it exists
just chill and do nothing if the container is already started
Docker Compose normally assigns a container name based on its current project name and the name of the services: block. Specifying container_name: explicitly overrides this; but, it means you can’t launch multiple copies of the same Compose file with different project names (from different directories) because the container name you’ve explicitly chosen won’t be used.
You almost never care what the container name is explicitly. It only really matters if you’re trying to use plain docker commands to manipulate Compose-managed containers; it has no impact on inter-service communication. Just delete the container_name: line.
(For similar reasons you can almost always delete hostname: and links: sections if you have them with no practical impact on your overall system.)
In my case I moved the project in an other directory.
When I tryed to run docker-compose up it failed because of some conflicts.
With command docker system prune I resolved them.
It's caused by being in a different directory than when you last ran docker-compose up. One option is to change back to the original directory. Or if you've configured it as a systemd service you can use systemctl.
Well...the error message seems pretty straightforward to me...
The container name "/my-postgres" is already in use by container
If you just want to restart where you left, you should use docker-compose start.
Otherwise, just clean up your workspace before running it :
docker-compose down
docker-compose up -d
Remove --no-recreate flag from your docker-compose command. And execute the command again.
$docker-compose up -d
--no-recreate is using for preventing accedental updates.
If there are existing containers for a service, and the service’s configuration or image was changed after the container’s creation, docker-compose up picks up the changes by stopping and recreating the containers. To prevent Compose from picking up changes, use the --no-recreate flag.
official docker docs.Link
I had similar issue
dcdown --remove-orphans
That worked for me.
I have an application that performs elaboration over a data feed. The process is divided into tasks so I structured a docker-compose.yml file like this:
task1-service:
image: task1-image
task2-service:
image: task2-image
task3-service:
image: task3-image
Each task-service is triggered by the end of the previous and triggers the next, then it can exit. So there's no point to keep each service running.
I wonder if there's a solution to keep them all stopped, and start each service on demand when needed.
I don't know if docker compose is the correct solution, but I like the idea ok keeping the system described into one yml file. Anyway, other solutions are appreciated.
Thanks
Is possible to solve your approach in different ways, and one of them is with docker-compose.
First, you can start one concrete service (taskX-service) using docker-compose up -d <service_name>
You have more details in docker-compose up for only certain containers
Second, docker-compose also allows you to configure dependencies between containers. If you want to run them in order, you can specify it in depends_on: structure.
For example, to execute tasks 1, 2, and 3 in order you could use:
task1-service:
image: task1-image
task2-service:
image: task2-image
depends_on:
- task1-service
task3-service:
image: task3-image
depends_on:
- task2-service
Furthermore, this docker-compose.yml is compatible with first I said:
docker-compose up -d task1-service
docker-compose up -d task2-service (also launchs task1-service)
docker-compose up -d task3-service (also launchs task2 and 1 service)
If you don't specify any container, with docker-compose down stops all containers in compose file.
I hope it's useful for you.
What we want to do:
We want to use docker-compose to link one already running container (A) to another container (B) by container name. We use "external-link" as both containers are started from different docker-compose.yml files.
Problem:
Container B fails to start with the error although a container with that name is running.
ERROR: for container_b Cannot start service container_b: Cannot link to a non running container: /PREVIOUSLY_LINKED_ID_container_a_1 AS /container_b_1/container_a_1
output of "docker ps":
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
RUNNING_ID container_a "/docker-entrypoint.s" 15 minutes ago Up 15 minutes 5432/tcp container_a_1
Sample code:
docker-compose.yml of Container B:
container_b:
external_links:
- container_a_1
What differs this question from the other "how to fix"-questions:
we can't use "sudo service docker restart" (which works) as this is a production environment
We don't want to fix this every time manually but find the reason so that we can
understand what we are doing wrong
understand how to avoid this
Assumptions:
It seems like two instances of the container_a exist (RUNNING_ID and PREVIOUSLY_LINKED_ID)
This might happen because we
rebuilt the container via docker-compose build and
changed the forwarded external port of the container (80801:8080)
Comment
Do not use docker-compose down as suggested in the comments, this removes volumnes!
Docker links are deprecated so unless you need some functionality they provide or are on an extremely old version of docker, I'd recommend switching to docker networks.
Since the containers you want to connect appear to be started in separate compose files, you would create that network externally:
docker network create app_net
Then in your docker-compose.yml files, you connect your containers to that network:
version: '3'
networks:
app_net:
external:
name: app_net
services:
container_a:
# ...
networks:
- app_net
Then in your container_b, you would connect to container_a as "container_a", not "container_a_1".
As an aside, docker-compose down is not documented to remove volumes unless you pass the -v flag. Perhaps you are using anonymous volumes, in which case I'm not sure that docker-compose up would know where to find your data. A named volume is preferred. More than likely, your data was not being stored in a volume, which is dangerous and removes your ability to update your containers:
$ docker-compose down --help
By default, the only things removed are:
- Containers for services defined in the Compose file
- Networks defined in the `networks` section of the Compose file
- The default network, if one is used
Networks and volumes defined as `external` are never removed.
Usage: down [options]
Options:
--rmi type Remove images. Type must be one of:
'all': Remove all images used by any service.
'local': Remove only images that don't have a custom tag
set by the `image` field.
-v, --volumes Remove named volumes declared in the `volumes` section
of the Compose file and anonymous volumes
attached to containers.
--remove-orphans Remove containers for services not defined in the
Compose file
Problem
A have few projects on my computer, that all are developed.
I started working with docker few month ago and I share some config between projects.
I have clear container list each time. Why? Because usually my process need to look like:
Delete all containers docker rm $(docker ps -a -q)
Delete all images - docker rmi $(docker images -q)
run docker-compose up -d
The problem is everywhere, I have defined composer (like in config below). When I switch, but don't delete images/containers, then i have, that composer/composer container exist and don't start.
Of course, I use more services like that, but this is simples one.
docker-compose.yml
version: '2'
services:
php:
image: php:7.1.3-alpine
volumes:
- ./:/app
working_dir: /app
composer:
image: composer/composer
volumes_from:
- php
working_dir: /app
My env
Mac OSX 10.11.6
Docker for mac: 17.03.1-ce-mac12 (17661)
Till now
searched texts like : Docker multi use of containers / docker multi projects on local with same container
read some blog with configuration, but didn't get hint.
Summary
As the topic is to wide, to many pages.
Maybe I ommited something in understand of concept or config.
Would be nice to get some explanation and hints for good manage docker-compose.yml files like that and what was wrong in my process.
Thanks.
Your question isn't very clear, but it sounds like you're using docker-compose to bring the containers up but relying on docker rm/docker rmi to take them down. Try doing everything with Docker Compose. Bring services up:
docker-compose up -d
Take services down but let volumes persist:
docker-compose down
Take services down and destroy volumes:
docker-compose down --volumes
https://docs.docker.com/compose/reference/down/
For the compose file you posted, you shouldn't generally need to use docker rm/docker rmi.
I'm using an official MySQL docker image (https://github.com/docker-library/mysql/blob/3288a66368f16deb6f2768ce373ab36f92553cfa/5.6/Dockerfile) with docker-compose and I would like its data to be wiped out upon restart. The default is that it retains its data between container restarts.
Here's my docker-compose.yml:
version: "2"
services:
mydb:
image: mysql:5.6
environment:
MYSQL_ROOT_PASSWORD: foo
When I use docker inspect on my container it shows its location on the host machine. How can I instead have it store the data inside the container? I do not want it to persist.
When using docker-compose, the containers are not removed on docker-compose stop (or ctrl-c, or any other kind of interrupt/exit). Thus, if you're stopping the container, it's still going to exist the next time you start.
What you want is docker-compose down which, according to the docs will "Stop and remove containers, networks, images, and volumes". Note that only containers and networks are removed by default - you need to specify a command-line switch if you want to remove images or volumes.