We have noticed that our containers are taking up a lot of space, one of the reasons for this is the images.
We would like to move the images.
I know right now they are stored in
/var/lib/docker/graph/<id>/layer
Is there a way to move these to another location/persistent disk?
To move images to another drive or another server:
docker save image_name > image_name.tar
mv image_name.tar /somewhere/else/
Load it back into docker
docker load < image_name.tar
Reference.
Here's any easy way to move docker's data:
sudo service docker stop
sudo mv /var/lib/docker /a/new/location
sudo ln -s /a/new/location /var/lib/docker # Create a symbolic link
sudo service docker start
No need to change DOCKER_OPTS or use -g /path.
You can always mount /var/lib/docker to a different disk. Otherwise, you can start the daemon with -g /path in order to tell docker to use a different directory for storage.
Using the answer by #creack I did the following on my Ubuntu install to move my entire docker images/containers folder to a new location/disk. The great thing about doing this is that any new images that I install will then use the new disk location.
First stop the docker service:
sudo service docker stop
Then move the docker folder from the default location to your target location:
sudo mv /var/lib/docker /thenewlocation
Then edit the /etc/default/docker file, inserting/amending the following line which provides the new location as an argument for the docker service:
DOCKER_OPTS="-g /thenewlocation/docker"
Restart the docker service:
sudo service docker start
This worked 100% for me - all my images remained in tact.
Related
I am logged in as root
the file is config.jason is in ~/.docker directory and i just want to move it to the root folder
I don't think I fully understand the problem, but I will try to answer what I think you might be looking for.
Whenever you want to change any settings/files/information from the docker & docker/swarm files/folders you should stop the docker service, do the changes in config and such and then start the service back again.
For instance adjusting the /etc/docker/daemon.json:
sudo service docker stop
sudo vi /etc/docker/daemon.json
# make the changes
sudo service docker start
In case you want to MOVE any actual files or switch directories I personally use rsync to do so, which works GREAT.
Example:
sudo service docker stop
sudo rsync -aP /var/lib/docker/ /opt/docker
sudo service docker start
Hope it helps
I would like to have a Docker Volume that mounts to a container. This volume would need to be somewhere other than the default location of volumes, preferably somewhere on the Desktop. This is because I am running a web server and would like some directories to be editable by something like VSCode so I don't always have to go inside the container to edit a file. I am not going to be using Docker Compose and instead will be using a Docker File for the container. The functionality I'm going for is the following equivalent of Docker Compose, but in a Dockerfile or through docker run, whichever is easiest to accomplish:
volumes:
- <local-dir>:<container-dir>
This directory will need to be editable LIVE and using the Dockerfile ADD command will not suffice, because after building, the image gets put into a tar archive and cannot be accessed after that.
with this solution you can move even A live container to new partition:
Add a configuration file to tell the docker daemon what is the location of the data directory
Using your preferred text editor add a file named daemon.json under the directory /etc/docker. The file should have this content:
{
"data-root": "/path/to/your/docker"
}
Copy the current data directory to the new one
sudo rsync -aP /var/lib/docker/ /path/to/your/docker
Rename the old docker directory
sudo mv /var/lib/docker /var/lib/docker.old
Restart the docker daemon
sudo service docker start
resource: https://www.guguweb.com/2019/02/07/how-to-move-docker-data-directory-to-another-location-on-ubuntu/
You can mount a directory from your host inside your container when you launch the docker container, using -v or --volume
docker run -v /path/to/desktop/some-dir:/container-dir/path <docker-image>
Volumes specified in the Dockerfile, as you exemplified, will automatically create those volumes under /var/lib/docker/volumes/ every time a container is launched from that image, but it is NOT recommended have these volumes altered by non-Docker processes.
After freshly installing Ubuntu 18 I am receiving the following error when trying to launch a docker container that has a bind to a LVM (ext4) partition:
mkdir /storage: read-only file system
I have tried reinstalling the OS, reinstalling Docker and forcing the drive to mount as RW (everything that isn't docker can write to the drive).
The directory that is being bound is currently set to 777 permissions.
There seems to be almost no information available for this error.
I had same issue, but removed docker from snap and reinstall on following the official docker steps.
Remove docker from snap
snap remove docker
then remove the docker directory, and old version
rm -R /var/lib/docker
sudo apt-get remove docker docker-engine docker.io
install official docker: https://docs.docker.com/install/linux/docker-ce/ubuntu/
I hope this help for you!
Update 01/2021: while still pretty cool, Snaps don't always work. Specifically with the Docker Snap, it didn't work for Swarm mode, so I ditched it and installed Docker the recommended way.
Snaps are actually pretty cool, IMO, and think it's beneficial to run Docker within a Snap than installing it directly on the system. The fact that you're getting a read-only permissions error is a good thing. It means that a rogue container isn't able to wreak havoc on your base OS. That said, how to fix your issue.
The reason that this is coming up is that Snaps will expose the host OS as read-only so that Docker can see the host's files, but not modify them (hence the permission denied error). But there is a directory that the Docker Snap can write to: /var/snap/docker. Actually, a better directory that snap can write to is /home. I created /home/docker for container's to have persistent storage from the host system.
In your case, you wanted /storage to be writeable by Docker containers. I had a very similar use-case, which led me to this SO post. I solved this by mounting my storage within the docker snap directory /home/docker; the easiest example simply being a directory on the same filesystem:
mkdir -p /home/docker/<container name>/data
In my case, I created a ZFS dataset at the location above instead of simply mkdir'ing a directory.
Then, the container I ran could write to that with something like:
docker run -ti -v /home/docker/<container name>/data:/data [...]
Now you have the best of both worlds: Docker running in a contained Snap environment and persistent storage. 🙌🏽
To solve this, create/run you container with --privileged:
ex.:
docker run --privileged -i --name master --hostname k8s-master -d ubuntu:20.04
I want to put a Docker image onto a USB HD and then be able to plug that into any [Linux] machine that has Docker and run the image. How would I go about doing that?
So far, I've discovered that you can "export" a Docker image into a flat file, but it appears you can't do anything with it until you "import" it again. That's no good. My ultimate goal is to run this stuff from a boot CD, which obviously won't have any writable storage to "import" the data into.
So remember that Docker is a service running on your [Linux] machine.
What you can do is are the following options:
Build and run the Dockerfile located on your USB Drive
docker build -t my_image --file /path/to/Dockerfile/on/usb/drive . && docker container run -d my_image
Create a docker-compose file and run the docker-compose from the Dockerfile on your USB Drive
docker-compose up -d --build -f /path/to/Dockerfile/on/usb/drive
In the end, the container will always run on the host machine, but you can take that USB drive to any machine and run the Dockerfile anywhere
OK, so it appears there's two main locations that the Docker daemon uses:
/var/lib/docker holds all the Docker images.
/var/run/docker holds... actually I'm not sure.
The solution I came up with is this:
Delete (!!) the /var/lib/docker folder.
Create a symlink named /var/lib/docker which points to where you actually want the data stored.
Then (and only then) start the Docker daemon.
This seems to result in Docker storing its data where you tell it to. In particular, if you symlink to a folder on an external USB device, Docker will store its state there. You can then repeat this procedure on another machine (maybe one without Internet access) and access the image(s).
Mind you, this stores the entire state of the Docker daemon, not just one image. But I haven't yet found a way around that.
You also wouldn't want to do this to a "real" computer; I want this for a boot CD, where next time you reboot, all the changes to the filesystem will just disappear again.
Another possibility: It's possible to run two Docker daemons on the same host, and to pass images between them. So you could start one daemon running on USB storage, load the necessary image(s) into it, and then on another machine start Docker running on the same USB device.
To run an alternative Docker daemon, you need the following incantations:
containerd \
--state-dir /mnt/Docker/containerd \
--listen unix:///mnt/Docker/containerd.sock
dockerd \
--pidfile /mnt/Docker/dockerd.pid \
--data-root /mnt/Docker/Data \
--exec-root /mnt/Docker/Exec \
--containerd /mnt/Docker/containerd.sock \
--host unix:///mnt/Docker/dockerd.sock
For this to work, the directory /mnt/Docker needs to already exist. The other files, sockets and directories appear to get created automatically.
Both containerd and dockerd accept a --debug option that makes them output a lot more info to the console. Both of these are daemons, so the commands above never return.
Once the new dockerd is running, you can talk to it as normal if you manually specify the socket:
docker --host unix:///mnt/Docker/dockerd.sock info
You might want to define that as a shell alias to save some typing.
You can copy an image from the "normal" Docker daemon to the new one you just created like so:
docker save ubuntu:latest | docker --host unix:///mnt/Docker/dockerd.sock load
Disclaimer/Edit 2
Some years later, for everyone reading this question - If you are on Windows and want to use docker with linux containers, I highly recommend not using docker for windows at all and instead starting the entire docker environment inside a VM altogether. This Ext3 NTFS issue will break your neck on so many different levels that installing docker-machine might not even be worth the effort.
Edit:
I am using docker-machine which starts a boot2docker instance inside a Virtualbox VM with a shared folder on /c/Users from which you can mount volumes into your containers. The permissions of said volumes are the ones the question is about. The VMs are stored under /c/Users/tom/.docker/
I chose to use the docker-machine Virtualbox workflow over Hyper-V because I need VBox in my daily workflow and running Hyper-V and Virtualbox together on one system is not possible due to incompabilities between different Hypervisors.
Original question
I am currently trying to set up PHPMyAdmin in a container on windows but I can't change the permissions of the config.inc.php file.
I found: Cannot call chown inside Docker container (Docker for Windows) and thought this might be somewhat related but it appears to apply only to MongoDB.
This is my docker-compose.yml
version: "3"
services:
pma:
image: (secrect company registry)/phpmyadmin
ports:
- 9090:80
volumes:
- /c/Users/tom/projects/myproject/data/var/www/public/config.inc.php:/var/www/public/config.inc.php
now, when I docker exec -it [container] bash and change in the mounted directory, I try to run chmod on the config.inc.php but for some reason, it fails silently.
root#22a4bag43245: ls -la config.inc.php
-rw------- 1 root root 0 Aug 11 15:11 config.inc.php
root#22a4bag43245: chmod 655 config.inc.php
root#22a4bag43245: ls -la config.inc.php
-rw------- 1 root root 0 Aug 11 15:11 config.inc.php
Considering the linked answer, I thought I could just move the volume out of my Userhome but then vbox doesn't mount the folder at all.
How do I change the file permissions of /var/www/public/config.inc.php persistently?
I had the same problem of not being able to change ownership even after using chown. And as I researched, it was because of NTFS volumes being mounted inside ext filesystem. So I used another approach.
The volumes internal to docker are free from these problems. So you can mount your file on internal docker volume and then create a hard symlink to that file inside your local folder wherever you want:
sudo ln $(docker volume inspect --format '{{ .Mountpoint }}' <project_name>_<volume_name>) <absolute_path_of_destination>
This way you can have your files in desired place, inside docker and without any permission issues, and you will be able to modify the contents of file as in the normal volume mount due to hard symlink.
Here is a working implementation of this process which mounts and links a directory. In case you wanna know about the details, see possible fix section in issue.
EDIT
Steps to implement this approach:
Mount the concerned file in internal docker-volume(also known as named volumes).
Before making hardlink, make sure volumes and concerned file are present there. To ensure this, you should have run your container at least once before or if you want to automate this file creation, you can include a docker run which creates the required files and exits.
docker run --rm -itd \
-v "<Project_name>_<volume_name>:/absolute/path" \
<image> bash -c "touch /absolute/path/<my_file>"
This docker run will create volumes and required files. Here, container is my project name, by default, it is the name of the folder in which project is present and <volume_name> is the same as one which we want to use in our original container. <image> can be the same one which is already being used in your original containers.
Create a hardlink in your OS to the actual file location on your system. You can find the file location using docker volume inspect --format '{{ .Mountpoint }}' <project_name>_<volume_name>/<my_file>. Linux users can use ln in terminal and windows users can use mklink in command prompt.
In step 3 we have not used /absolute/path since the <volume_name> refers to that location already, and we just need to refer to the file.
Try one of the following:
If you can rebuild the image image: image: (secrect company registry)/docker-stretchimal-apache2-php7-pma then inside the docker file, add the following
USER root
RUN chmod 655 config.inc.php
Then you can rebuild the image and push it to the registry, and what you were doing should work. This should be your preferred solution, as you don't want to be manually changing the permissions everytime you start a new container
Try to exec using the user root explicitly
docker exec -it -u root [container] bash