I have a Docker container running on my Mac. This Docker container has a home folder like:
/home/my_user/my_project/
It is based on a Ubuntu OS image and running on my Mac. When I run this container, the container constantly updates a folder under the my_project folder. Now when I stop and remove this container, it just gets erased and when I start a new instance of the container, the process has to begin all over again, i.e., the container starts writing into the my_project folder, but the old files which it already wrote is completely lost.
How can I make the data written by the container be persistent even after a container delete / restart?
Docker persistent volumes is what I understand that I need, but how can I mount a local folder on my Mac such that the data is written and persisted? This container could run on a Windows machine, so how can I make a persistent volume across different OS?
You need to start your container with the -v flag. So if you were to mount the /home/my_user/my_project directory from the container onto the host to /srv/my_app/data for example, you'd need to use it as follows:
docker run -v /srv/my_app/data:/home/my_user/my_project IMAGE_NAME
There's also a difference between volumes and bind mounts, which I explained here
Related
So I have this remote folder /mnt/shared mounted with fuse. It is mostly available, except there shall be some disconnections from time to time.
The actual mounted folder /mnt/shared becomes available again when the re-connection happens.
The issue is that I put this folder into a docker volume to make it available to my app: /shared. When I start the container, the volume is available.
But if a disconnection happens in between, while the /mnt/shared repo on the host machine is available, the /shared folder is not accessible from the container, and I get:
user#machine:~$ docker exec -it e313ec554814 bash
root#e313ec554814:/app# ls /shared
ls: cannot access '/shared': Transport endpoint is not connected
In order to get it to work again, the only solution I found is to docker restart e313ec554814, which brings downtime to my app, hence is not an acceptable solution.
So my questions are:
Is this somehow a docker "bug" not to reconnect to the mounted folder when it is available again?
Can I execute this task manually, without having to restart the whole container?
Thanks
I would try the following solution.
If you mount the volume to your docker like so:
docker run -v /mnt/shared:/shared my-image
I would create an intermediate directory /mnt/base/shared and mount it to docker like so:
docker run -v /mnt/base/shared:/base/shared my-image
and I will also adjust my code to refer to the new path or creating a link from /base/shared to /shared inside the container
Explanation:
The problem is that the mounted directory /mnt/shared is probably deleted on host machine, when there is a disconnection and a new directory is created after connection is back. But, the container started running with directory mapping for the old directory which was deleted. By creating an intermediate directory and mapping to it instead you avoid this mapping issue.
Another solution that might work is to mount the directory using bind-propagation=shared
e.g:
--mount type=bind,source=/mnt/shared,target=/shared,bind-propagation=shared
See docker docs explaining bind-propogation
I have a directory in my Docker container, and I'm trying to make it available locally using -v screenshots:/srv/screenshots in my docker run command but it's not available.
Do I need to add something else to my command?
Host volumes are mapped from the host into the container, not the other way around. This is one way to have persistent storage (so the data don't disappear when the container is re-created).
You can copy the screenshot folder to your host with docker cp and map them in.
You will have your screenshots in the local screenshots folder. Mapping them in with -v screenshots:/srv/screenshots makes them appear in /srv/screenshots in the container, but these files are really on the host.
See: Mount a host directory as data volume
i am using native docker for mac and i have a small application running with docker container .
currently i am manually copying the data from my mac to docker container using docker cp command.
i want to make it dynamic, i want to put the data in my local directory which should get sync with docker container .
example:
mac local dir : users/vishnu/data/
which should get sync to
`<Docker-container-ID>:/opt/deploy/`
the container is already running ,i should not release the running container . i can only stop and start . is there a way ?? Thanks in advance
host mounted volume.
when you docker run you add a -v /Users/vishnu/data:/opt/deploy parameters.
if you need to add a mounted volume to your existing container, use the Kitematic UI. it's easier that way. but in general, you should add this when you docker run.
...
also, FYI - the idea that you can't delete a container is an anti-pattern with Docker. if you can't delete your container, because it would cause too many problems, you're doing something wrong. https://derickbailey.com/2017/04/05/what-i-learned-by-deleting-all-of-my-docker-images-and-containers/
I am new to docker containers and I and am trying to solve a problem I am facing right now.
These are my understanding based on limited knowledge.
When we create a docker container, Docker creates a local mount and use it as the root file system for the docker container.
Now, if I run any commands in the container from the host server using docker exec the docker is not using the mounted partition as the / file system for the container. I mean, it still pics up the binaries and env variables from the host server. Is there any option/alternate solution for making the docker use the original mounted directory for docker exec too ?
If I access/start the container with docker attach or docker run -i -t /bin/bash, I get the mounted directory as my / file system, which gives me an entirely independent environment from my host system. But this doesn't happen with the docker exec command.
Please help !!
You are operating under a misconception. The docker image only contains what was installed in it. This is usually a very cut down version of an operating system for efficiency reasons.
The docker container is started from an image - and that's a running version, which can change and store state - but may be discarded.
docker run starts a container from an image. You can run the same image multiple times to create completely different containers (which happen to have the same starting point for their content).
docker exec attaches to one of those containers to run a command. So you will only see the things inside it that ... were inside the image, or added post start (like log files). It has no vision of the host filesystem, and may not be the same OS - the only requirement is that it shares elements of the kernel ... although it usually has a selection of the commonly used binaries.
And when you run an image to create a container, you can specify a mount. One of the options when you do this is passing through a host filesystem, with e.g. -v /path/on/host:/path_in/container. But you don't have to, you can use data containers or use a docker volume mount instead. e.g. docker run -v /mount creates a mount point within the container, using the docker filesystem, which isn't part of the parent host. This can be used to make a data container with: docker create -v /path/to/data --name data_for_acontainer some_basic_image
And then mount volumes from that data container on a new one:
docker run -d --volumes-from data_for_acontainer some_app_image
Which will attach that data container onto the /path/to/data mount. But in neither case is the 'host' filesystem touched directly - this is the whole point of dockerising things.
Take a typical data only Docker container:
FROM stackbrew/busybox:latest
RUN mkdir /data
VOLUME /data
Now I have seen a great deal of them that are run like this:
docker run -name my-data data true
The true command exits as soon as it runs, and so does the container. But surprisingly it continues to serve the volume when you connect it with another container via --volumes-from my-data.
My question is, how does that work? How does a stopped container still allow access in it's volumes?
Volumes in docker are not a top-level thing. They are "simply" part of container's meta-data.
When you have VOLUME in your dockerfile or start a container with -v, Docker will create a directory in /var/lib/docker/volumes* with a random ID (this is the exact same process as creating an image with commit except it is empty) and add that random ID to the container's metadata.
When the container starts, Docker will mount-bind the directory /var/lib/docker/volumes/* at the given location for that volume.
When you use volumes-from, Docker will just lookup the volume id and the location from an other container, running or not and mount-bind the directory at the set location.
Volumes are not linked with the runtime, it is just directories that are mounted.
* With newer versions, Docker now uses the vfs driver for storage and /var/lib/docker/volumes/ is used only for metadatas like size, create time, etc. The actual data are stored in /var/lib/docker/vfs/dir/<volume id>