I created a docker container that shares a folder with its host using the following command:
docker run -dit -P -v $(pwd):/home/shared ubuntu
The container became unresponsive so I deleted it and pruned it using the following steps mentioned here:
docker rm $(docker ps -aq)
docker rmi $(docker images -q)
docker system prune
Now, I would like to delete the file that resides on the host and was created by the container. However, I keep getting this error msg:
rm file1
rm: remove write-protected regular file ‘file1’? Y
rm: cannot remove ‘file1’: Permission denied
I don't have sudo privilege on this host, can I do something about it, or do I need to ask the system administrator's help for this?
Thank you
The Docker daemon runs as root and your user(apparently in the docker group, given the lack of sudo and the file permission issues) has access to the API(typically exposed via a local unix socket). Through the docker API you should be able to delete the created files.
If it is truly a volume, you should simply be able to use the docker volume rm command.
docker volume rm <your-volume>
If you instead mounted a path from the host, you should be able to launch a new container to delete the file(s), re-using the same mount point(for instance, start a basic bash container with docker run, rm the specified files, and then remove the new container).
Related
I want to make a backup of all my containers and volumes, so the easiest way would be to copy /var/lib/docker to another location.
However this directory also includes all the images, and I don't want to include them since they all can easily be re-downloaded from public sources.
So how can I copy this directory while excluding the images?
You have to differentiate between container backup and vol backup:
Backing up a container, that is, its configurations like labels, envs, etc.
You do that by committing the container as an image:
$ docker container commit <container-name/id> <name-of-new-image>
Better give it also some metainfo:
$ docker container ... -m "very important container config state" -a "John Doe"
Backing up a volume
Let's say the volume of interest <my-vol> is bound to a container <other-container> - which may have been created like: docker container run -v /my-vol other-container ...
So first you have to bind the volume also to a newly created temporary container with the --volumes-from flag. With the -v option you mount a local path (of the host) into the container:
$ docker container run -rm --volumes-from <other-container> \
-v <dir/on/host>:<mountpath/in/container> \
<ubuntu/centos/whatever-base-image> tar cvf <mountpath/in/container>/backup.tar /<my-vol>
After completing the command the container stops and with that it will also be deleted because of the -rm option.
Whith all that the steps are:
bind the volume to a temp container
mount a hostpath into the container
make a tarbal (or whatever kind of backup)
of the volume in the container
container stops and is deleted after the backup command has finished
the backup tarbal is left on the mounted dir of the container host.
see also: https://docs.docker.com/storage/volumes/
Shell Command
.. the other - not recommended - way would be to do it just with os level commands:
shopt -s extglob
cp -r var/lib/docker/!(image) your/path/backup
For that you have to stop all involved containers to prevent read/write issues.
I need to reset a moodle docker to its initial state every 24 hours. This docker will be a running a demo site where users can login and carry out various setting changes and the site needs to reset itself every day. Does docker provide any such feature?
I searched for a docker reset command but it doesn't seem to be there yet.
Will such a process of removing and reinitiating docker container work?
docker rm -f $(docker ps -a -q)
docker volume rm $(docker volume ls -q)
docker-compose up -d
I should be able to do this programatically ofcourse, preferably using a shell script.
Yes you do not need to reset just recreate the container is enough but if you bind volumes with the host it will not work if there is anything that pick from persistent storage of the host in docker-compose up.
Write a bash script that will run every 1:00 AM or whatever time you want to create fresh container.
0 0 * * * create_container.sh
create_container.sh
#!/bin/bash
docker-compose rm -f
docker-compose up -d
or you can use your own script as well but if there is bind volumes the clear that files before creating the container.
rm -rf /path/to_host_shared_volume
docker rm -f $(docker ps -a -q)
.
.
.
As the behavour of -v is different it will create directory if not exist.
Or if you want to remove everything then you can use system-prune
#!/bin/bash
docker system prune -f -a --volumes
docker-compose up -d
Remove all unused containers, networks, images (both dangling and unreferenced), and volumes.
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all volumes not used by at least one container
- all images without at least one container associated to them
- all build cache
I tried to share data between the docker container and the host, for example by adding the parameter -v /Users/name/Desktop/Tutorials:/cntk/Tutorials to the docker run command, but I noticed that it also deletes all the files on the docker contained in /cntk/Tutorials.
My question is how to make the same link, but having instead all the files in /cntk/Tutorials copied to the host (at /Users/name/Desktop/Tutorials)
Thank you
Unfortunately that it is not possible, take a look here. That is because this is how mounting works in Linux.
It is not correct to say that the files were deleted. They are still present in the underlying image, but the act of mounting another directory at the same path has obscured them. They exist, but are not accessible in this condition.
One way you can accomplish this is by mounting a volume into your container at a different path, and then copying the container's files to that path. Something like this.
Mount a host volume using a different path than the one the container already has for the files you are interested in.
docker run -v /Users/name/Desktop/Tutorials:/cntk/Tutorials2 [...]
Now, execute a command that will copy the files already in the docker image, into the mounted volume from the outside host.
docker exec <container-id> cp -r /cntk/Tutorials /cntk/Tutorials2
The docker cp command allows you to copy files/folders on demand between host and the container:
docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-
docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH
docker cp ContainerName:/home/data.txt . <== copy from container to host
docker cp ./test.txt ContainerName:/test.txt <== copy from host to container
docker cp ContainerName:/test.txt ./test2.txt <== copy from container to host
For details run docker cp --help
I created a docker container and have an application running inside it. I created a second docker container (on the same host) with the same application running inside it. I need to create a few more containers this way. However, when I remove a container, I need to ensure that the dependencies it creates on the host are completely removed. How could this be achieved ?
Thanks,
Checkout the documentation of the docker rm command:
Usage: docker rm [OPTIONS] CONTAINER [CONTAINER...]
Remove one or more containers
Options:
-f, --force Force the removal of a running container (uses SIGKILL)
--help Print usage
-l, --link Remove the specified link
-v, --volumes Remove the volumes associated with the container
So use the "-v" option
Update
You can also use this command to cleanup volumes with no associated containers.
docker volume rm $(docker volume ls -qf dangling=true)
Credit: sceada
I have been trying to setup a graph database using orientdb. So I tried using volumes by the following command
docker run -d -p 2424:2424 -p 2480:2480 -v config:/orientdb/config -v database:/orientdb/databases -v backup:/orientdb/backup -e ORIENTDB_ROOT_PASSWORD=mypasswdhere orientdb:latest
My prime motive behind using volumes was to store data in database after I kill the container.
But I used this command frequently to start the server.
Now it has hogged my disk space so I guess it creates a new copy each time this command is executed.
Can someone indicate a correct way to use existing volumes to use stored data in docker and to clean up the redundant data recreated by frequent execution of this command?
You can create named volumes with docker volume create
$ docker volume create --name hello
$ docker run -d -v hello:/world busybox ls /world
That way, only one volume in /var/lib/docker/volumes will be used each time you launch that container.
See also "Mount a shared-storage volume as a data volume".
In the meantime, to remove dangling volumes:
docker volume ls -qf "dangling=true" | xargs docker volume rm
As far as I understand, you aren't re-using the container, instead you start a new one each time.
After the first run, you can stop and the restart it with docker stop/start commands.