We have various docker-compose files that create a service infrastructure. My question is if it is possible to only create the volumes defined in the compose file without creating the containers?
It might be possible without starting any services.
command can be provided like this.
docker-compose up --no-start --no-deps
Or it's possible to specify individual volume.
docker-compose up --no-deps -d --volume <volume-name>
You can use a docker-compose.override.yml file with invalid entrypoints or different image (use a small template image) for those containers, that way they'll die immediately.
docker-compose.override.yml:
<conatiner-name>:
image: hello-world
command: "/hello"
restart: "no"
Related
I'm running a docker-compose setup, and when I want to update files in my image I create a new docker image. Though the problem is; the file I'm editing is located in the persistent volume, meaning the Docker image itself will get the changes, but since I'm not deleting docker-compose volumes the volume will be used by the new image, hence the old file will be used by new image.
Running docker-compose down -v is not an options because I want to keep other existing files in the volume (logs etc.).
I want to know if it possible to do this without too much hacks, since I'm looking to automate this.
Example docker-compose.yml
version: '3.3'
services:
myService:
image: myImage
container_name: myContainer
volumes:
- data_volume:/var/data
volumes:
data_volume
NOTE: The process of doing change in my case:
docker-compose down
docker build -t myImage:t1 .
docker compose up -d
You could start a container, mount the volume and execute a command to delete single files. Something like
docker run -d --rm -v data_volume:/var/data myImage rm /var/data/[file to delete]
This is a section of my docker-compose file:
version: "3.2"
services:
cachable_service:
image: my/cachable_service:x.x
build:
context: .
dockerfile: cachable_service_Dockerfile
service_in_development:
image: my/service_in_development:latest
build:
context: .
dockerfile: Dockerfile
depends_on:
- cachable_service
How do I force only the image for the service_in_development to be always build from scratch so I get my latest code in there?
I've looked into the cache_from option in the docker-compose file reference, but I couldn't find any clear explanation on what exactly it does.
Clarifications:
In this question, I was asking about an option to force rebuild only specific services using an option in the docker-compose.yml file itself. But, if a commandline option let's me do this, I'd be happy with that as well for now.
Also, I'm talking about forcing a 'rebuild' of images. Not just 'recreation' of service.
You can use command below to rebuild all images:
docker-compose up --build --force-recreate
to force rebuild one service:
docker-compose up --build --force-recreate service-name-here
More useful parameters in documentation
Usage: up [options] [--scale SERVICE=NUM...] [SERVICE...]
Options:
-d, --detach Detached mode: Run containers in the background,
print new container names. Incompatible with
--abort-on-container-exit.
--no-color Produce monochrome output.
--quiet-pull Pull without printing progress information
--no-deps Don't start linked services.
--force-recreate Recreate containers even if their configuration
and image haven't changed.
--always-recreate-deps Recreate dependent containers.
Incompatible with --no-recreate.
--no-recreate If containers already exist, don't recreate
them. Incompatible with --force-recreate and -V.
--no-build Don't build an image, even if it's missing.
--no-start Don't start the services after creating them.
--build Build images before starting containers.
--abort-on-container-exit Stops all containers if any container was
stopped. Incompatible with -d.
-t, --timeout TIMEOUT Use this timeout in seconds for container
shutdown when attached or when containers are
already running. (default: 10)
-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving
data from the previous containers.
--remove-orphans Remove containers for services not defined
in the Compose file.
--exit-code-from SERVICE Return the exit code of the selected service
container. Implies --abort-on-container-exit.
--scale SERVICE=NUM Scale SERVICE to NUM instances. Overrides the
`scale` setting in the Compose file if present.
I am new to Docker and have docker-compose.yml which is containing many services and iI need to start one particular service. I have docker-compose.yml file with information:
version: '2'
services:
postgres:
image: ${ARTIFACTORY_URL}/datahub/postgres:${BUILD_NUMBER}
restart: "no"
volumes:
- /etc/passwd:/etc/passwd
volumes_from:
- libs
depends_on:
- libs
setup:
image: ${ARTIFACTORY_URL}/setup:${B_N}
restart: "no"
volumes:
- ${HOME}:/usr/local/
I am able to call docker-compose.yml file using command:
docker-compose -f docker-compose.yml up -d --no-build
But I need to start "setup service" in docker-compose file:
How can I do this?
It's very easy:
docker compose up <service-name>
In your case:
docker compose -f docker-compose.yml up setup -d
To stop the service, then you don't need to specify the service name:
docker compose down
will do.
Little side note: if you are in the directory where the docker-compose.yml file is located, then docker-compose will use it implicitly, there's no need to add it as a parameter.
You need to provide it in the following situations:
the file is not in your current directory
the file name is different from the default one, eg. myconfig.yml
As far as I understand your question, you have multiple services in docker-compose but want to deploy only one.
docker-compose should be used for multi-container Docker applications. From official docs :
Compose is a tool for defining and running multi-container Docker
applications.
IMHO, you should run your service image separately with docker run command.
PS: If you are asking about recreating only the container whose image is changed among the multiple services in your docker-compose file, then docker-compose handles that for you.
Up to recent version of Docker (v1.10), we were thought that we can use DOC: data-only containers. So I would create such DOC (based on e.g. busybox) and use --volumes-from to link it to my container. You can still read about this in Docker documentation.
With new version of docker, it is said that instead of DOC we should use named volumes. Here is an example of docker-compose.yml:
version: '2'
services:
elasticsearch:
image: elasticsearch:2.2.0
command: elasticsearch -Des.network.host=0.0.0.0
ports:
- "9201:9200"
volumes:
- "es-data:/usr/share/elasticsearch/data"
volumes:
es-data:
Here we created and use named volume es-data.
There is still not much documentation on this new feature. I am asking:
Can we replace DOC with named containers? How long volume is persisted? What if I remove the container that is using it?
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Can we replace DOC with named containers?
In many cases, yes, named containers will be a better option.
How long volume is persisted? What if I remove the container that is using it?
If you remove the container, the volume will still be there. The only way to remove the volume is to use docker-compose down -v or docker volume rm <volume name>.
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Instead of --volumes-from, you can use --volume=<volume name>.
Note that volumes created by docker-compose are always prefixed with the project name, so if you use it with a docker command the full name is actually <project_name>_es-data.
This question is coming from an issue on the Docker's repository:
https://github.com/docker/compose/issues/942
I can't figure it out how to create a data container (no process running) with docker compose.
UPDATE: Things have changed in the last years. Please refer to the answer from #Frederik Wendt for a good and up-to-date solution.
My old answer: Exactly how to do it depends a little on what image you are using for your data-only-container. If your image has an entrypoint, you need to overwrite this in your docker-compose.yml. For example this is a solution for the official MySql image from docker hub:
DatabaseData:
image: mysql:5.6.25
entrypoint: /bin/bash
DatabaseServer:
image: mysql:5.6.25
volumes_from:
- DatabaseData
environment:
MYSQL_ROOT_PASSWORD: blabla
When you do a docker-compose up on this, you will get a container like ..._DatabaseData_1 which shows a status of Exited when you call docker ps -a. Further investigation with docker inspect will show, that it has a timestamp of 0. That means the container was never run. Like it is stated by the owner of docker compose here.
Now, as long as you don't do a docker-compose rm -v, your data only container (..._DatabaseData_1) will not loose its data. So you can do docker-compose stop and docker-compose up as often as you like.
In case you like to use a dedicated data-only image like tianon/true this works the same. Here you don't need to overwrite the entrypoint, because it doesn't exist. It seems like there are some problems with that image and docker compose. I haven't tried it, but this article could be worth reading in case you experience any problems.
In general it seems to be a good idea to use the same image for your data-only container that you are using for the container accessing it. See Data-only container madness for more details.
The other answers to this question are quite out of date, and data volumes have been supported for some time now. Example:
version: "3.9"
services:
frontend:
image: node:lts
volumes:
- myapp:/home/node/app
volumes:
myapp:
See
https://docs.docker.com/storage/volumes/#use-a-volume-with-docker-compose for details and options.
A data only container (DOC) is a container that is created only to serve as a volume provider. The container itself has no function other than that other containers can mount it's volume by using the volumes_from directive.
The DOC has to run only once to create the volume. Other containers can reference the volumes in it even if it's stopped.
The OP Question:
The docker-compose.yml starts the DOC every time you do a docker-compose up. OP asks for an option to only create container and volume, and not run it, using some sort of an create_only: true option.
As mention in the issue from the OP's question:
you either create a data container with the same name as the one specified in the docker-compose.yml, and run docker-compose up --no-recreate (the one specified in docker-compose.yml won't be recreated).
or you run a container with a simple command which never returns.
Like: tail -f /dev/null