is there possible to somehow create persistent storage for containers, created with docker-compose and don't remove them even when running docker-compose down -v so they will be automaticly attached to their containers after again starting the docker-compose up -d ?
What I usually do is to use an external volume, something like:
$ docker volume create nodemodules
docker-compose.yml
version: '3.7'
services:
frontend:
image: node:11
volumes:
- nodemodules:/app/node_modules
volumes:
nodemodules:
external: true
Refer the docs for more info: https://docs.docker.com/compose/compose-file/#external
Related
I got docker compose:
version: '2'
services:
elasticsearch:
image: 'elasticsearch:7.9.1'
environment:
- discovery.type=single-node
ports:
- '9200:9200'
- '9300:9300'
volumes:
- /var/lib/docker/volumes/elastic_search_volume:/usr/share/elasticsearch/data:rw
When I run:
docker volume ls
I see no results. How to list unnamed volumes?
docker volume ls as you've shown it will list all of the volumes that exist.
However, in the docker-compose.yml file you show, you're not creating a named or anonymous volume. Instead, you're creating a bind mount to connect a host directory to the container filesystem space. These aren't considered "volumes" in a technical Docker sense, and a docker volume command won't show or manipulate those.
Reaching directly into /var/lib/docker usually isn't a best practice. It's better to ask Docker Compose to manage the named volume for you:
version: '2'
services:
elasticsearch:
volumes:
# No absolute host path, just the volume name
- elastic_search_volume:/usr/share/elasticsearch/data:rw
volumes:
elastic_search_volume:
# Without this line, Compose will create the volume for you.
# With this line, Compose expects it to already exist; you may
# need to manually `docker volume create elastic_search_volume`.
# external: true
I have existing docker-compose.yml file that runs on my Docker CE standalone server.
I would like to deploy this same configuration using the AWS ECS service. The documentation of the ecs-cli tool states that Docker Compose files can be used. Other (simpler) container configs have worked with my existing files.
With my configuration, this errors with:
ERRO[0000] Unable to open ECS Compose Project error="External option
is not supported"
FATA[0000] Unable to create and read ECS Compose Project
error="External option is not supported"
I am using "external" Docker volumes, so that they are auto-generated as required and not deleted when a container is stopped or removed.
This is a simplification of the docker-compose.yml file I am testing with and would allow me to mount the volume to a running container:
version: '3'
services:
busybox:
image: busybox:1.31.1
volumes:
- ext_volume:/path/in/container
volumes:
ext_volume:
external: true
Alternatively, I have read in other documentation to use the ecs-params.yml file in the same directory to pass in variables. Is this a replacement to my docker-compose.yml file? I had expected to leave it's syntax unchanged.
Working config (this was ensuring the container stays running, so I could ssh in and view the mounted drive):
version: '3'
services:
alpine:
image: alpine:3.12
volumes:
- test_docker_volume:/path/in/container
command:
- tail
- -f
- /dev/null
volumes:
test_docker_volume:
And in ecs-params.yml:
version: 1
task_definition:
services:
alpine:
cpu_shares: 100
mem_limit: 28000000
docker_volumes:
- name: test_docker_volume
scope: "shared"
autoprovision: true
I have a docker image "doc_image" and a docker volume "doc_volume". I want to spin up a container from the image where the volume is mounted into a specific point
If I do this with docker run like this:
docker run -d -p 5000:5000 -v doc_volume:/directory doc_image
then it runs flawlessly (I can see the expected files in /directory in interactive way). However, when I try to spin it up with docker-compose like with a docker-compose.yml like this:
version '3'
services:
my_service:
image: doc_image
volumes:
- doc_volume:/directory
volumes:
doc_volume:
there is nothing in /directory:
FileNotFoundError: [Errno 2] No such file or directory: '/directory/file.txt'
What went wrong here?
Add external property to volumes section:
version '3'
services:
my_service:
image: doc_image
volumes:
- doc_volume:/directory
volumes:
doc_volume:
external: true # << here we go
Your problem is that docker-compose creates another volume unless you explicitly tell him to use external one. External means creates not by means of docker-compose.
this is my docker-compose file
version: '3'
services:
mongo:
image: mongo:latest
ports:
- 27017:27017
volumes:
- dbdata:/data/db
volumes:
dbdata:
When I start and stop the mongo container I lose all data.
What am I missing out on ?
Try running docker volume create dbdata in order to create a persistent volume, redeploy your stack and it should work after that. I.e. the first reboot will cause a data wipe, but after that it should persist.
I hope this helps you
Don't know why it wasn't working but it is now solved
After running my docker-compose file I see the mydata volume when I run docker volume ls.
That volume persists through container destruction.
If I run this command the volume mounts and the container starts as expected with initialized state:
docker run --name gogs --net mk1net --ip 203.0.113.3 -v gogs-data:/data -d gogs/gogs
However if I run the corresponding docker-compose script the volume does not mount. The container still starts up, but without the state it reads on startup.
version: '3'
services:
gogs:
image: gogs/gogs
ports:
- "3000:3000"
volumes:
- gogs-data:/data
networks:
mk1net:
ipv4_address: 203.0.113.3
volumes:
gogs-data:
networks:
mk1net:
ipam:
config:
- subnet: 203.0.113.0/24
Any ideas?
Looking at your command, the gogs-data volume was defined outside the docker compose file, probably using something like:
docker volume create gogs-data
If so then you need to specify it as external inside your docker compose file like this:
volumes:
gogs-data:
external: true
You can also define a different name for your external volume and keep using current volume name inside your docker compose file to avoid naming conflicts, like for example, let's say your project is about selling cars so you want the external volume to be call selling-cars-gogs-data but want to keep it simple as gogs-data inside your docker compose file, then you can do this:
volumes:
gogs-data:
external:
name: selling-cars-gogs-data
Or even better using environment variable to set the volume name for a more dynamic docker compose design, like this:
volumes:
gogs-data:
external:
name: "${MY_GOGS_DATA_VOLUME}"
And then start your docker compose like this:
env MY_GOGS_DATA_VOLUME='selling-cars-gogs-data' docker-compose up
Hope this helps, here is also a link to the docker compose external volumes documentation in case you want to learn more: https://docs.docker.com/compose/compose-file/#external
You can make pretty much everything external, including container linking to connect to other docker compose containers.