After moving my data-root directory to an SD card my new docker data directory is /mnt/sd/var/lib/docker and everything is there, but docker/portainer do not see the images, containers, stacks, volumes..nothing
Also I cannot access the docker directory in that path and need to change permissions all the time.
I think docker may not have the permissions. Any ideas how to make my data appear again in docker/portainer?
I am changing the persmissions manually using sudo chmod -R 775 /mnt/sd/var/lib/docker but it has just a temporary effect.
Related
My situation is the following:
I am having a docker image/container in which I am compiling. I had to install some components to $HOME via the Dockerfile (so while creating the image).
Let's say one of those components is in ~/.config, but also other folders.
I would like to have the possibility to override the files in .config by mounting a home folder from the host on top of the one inside docker. Whenever you place a file in the mounted folder, it overrides the one which is already inside the container.
So in theory, this is exactly what an OverlayFS does, right? While the lower directory would be the one inside the Docker container, the upper directory would be the one on my Host.
Is there a way to accomplish that?
Until now I found the following related topics:
https://serverfault.com/questions/841238/how-to-use-overlayfs-with-docker-volumes
Drawback: The answer does only show how to use overlayfs on the host, but getting acccess to the lower container/image directory is not that self-explaining and also feels dirty.
Can I mount docker host directory as copy on write/overlay?
Drawback: Using mount -t overlay inside docker does not work on newer kernels because of the disabled overlay-on/over-overlay option
I also thought about manipulating the docker files on host directly, i.e. the directories where docker stores the files, but that feels a bit dirty.
To do so, I would declare VOLUME /home/user at the end of the Dockerfile. Then I would find my files of that directory in /var/lib/docker/volumes/user/_data. I could then create a overlayfs on my host, using that directory as lower, my other folder as upper. I could then remount that new directory using docker run --volume. Unfortunately this would involve su rights to access the /var/lib directory.
The other way around would be to bind-mount single files, but that's maybe a bit hackish too.
The problem
I messed up really badly, started a prestashop docker container (via compose) for production and forgot to add a persistent volume to it. I've set up my store and now all my data is dangling. I need to somehow convert my current docker container to one with persistent volume.
Things I've tried
I've copied all data from current container to a local folder, created a new compose file and everything would work if I chmod -R 777 whole directory but people go to hell for these types of crimes, so that's not an option.
As I'm not an expert, I have no other ideas.
The issue you have is most probably related to the user id / group id that is owning the files from the old container. When you move to the new container, if the IDs are not same you will have problem, so chmod -R 777 is fixing this problem, but the gates of hell are wide open.
It's better to understand which user id and group id should be the owners of the files and use chown and chgrp to set the right owners from the new container.
Alright,
So it was a pretty simple solution:
sudo su
go to the non-persistent container storage location which you can get by using docker inspect container take the long string hash 822adf62adsomethingsomething.
then cd /var/lib/docker/overlay2/822adf62adsomethingsomething/merged. There you have all your docker container files.
cp -a whateveryouwant /whereeveryouwant as cp -a keeps permissions and owners intact.
profit
I have a directory and various files and directories inside that directory.
The ownership and group varies of some files directories.
I created a Dockerfile. I have created exactly the same user and group with the same uid and gid in Dockerfile.
So the container created will also have the same user and group with the same uid and gid as host.
The Image is built successfully as also container. When I See inside the container the ownership of all the directories and files are (root root).
I need the exact same ownership for each and every file and directory as of host.
Please help.
Thanks in advance.
the root user will be always the owner of the files after copying the files to the docker image.
It is a best practice to fix the file permission after copying or adding files to the Docker images.
ONBUILD ADD . /application
ONBUILD RUN chown -R rails /application
you don't need to care about the file owners on the host system, because all files will be available inside the docker container and the only thing that you should do is to make sure that the user running in the container has access rights to the files.
The only case that file permission need to be aligned between the host and the docker containers is the case where both the container and the host are sharing some files.
I have nginx and wordpress in docker containers, which share a volume bind mount (i.e. mapped to files on disk) for /var/www/html.
At runtime I need to add files to the mapped directory, chown them, and for them to be usable. They appear in the volume, but their owner is wrong so they are unusable by nginx / wordpress.
(Typically the advice is to do this in the Dockerfile - but that doesn't apply in this case. I'm not modifying the app, I'm just adding to the volume, which is not ephemeral.)
When I stop and start the docker apps (nginx and wordpress), everything works. But it doesn't work while online.
The parent directory has chmod 2775 and chown -R bob:www-data. When I copy files in there at runtime, I am doing so as bob. But nginx still cannot access those new files (404).
So how do I copy and chown files into a bind mount at runtime?
You should check that you are not only using the same username, but also the same UID & GID (If you have the user bob both in docker and the host but UID & GID don't match you will get permissions issues).
There is a good explanation on that on this other post.
I have an application that saves the data in the container directory /var/lib/app-data.
This container directory should be backed up since the application makes changes to the /var/lib/app-data content regularly as it runs.
But need to backup only specific directories and files under /var/lib/app-data/
For ex Below are the files and directories should be backed up regularly under /var/lib/app-data/,so even if the container stopped, the new container still get this files and directories.
/var/lib/app-data/ui_log/
/var/lib/app-data/node_log/
/var/lib/app-data/users/mailer.xml
/var/lib/app-data/plugins/
Other files and directories should not be backed up to volumes.
Currently I could able to find out only one way to map docker volume like below.
docker run -v forkuidatavolume:/var/lib/app-data mohan/forkui:1.0
But above step backing up directories other than one mentioned above.
Is it possible to backup specific container files and directories using -v or any other way?
Thanks,
Mohan
You can put a list of files that you do not want to be added to your docker image at .dockerignore. For example:
.git
/tmp/*
/var/lib/app-data/ui_log/*
/var/lib/app-data/node_log/
/var/lib/app-data/users/mailer.xml
/var/lib/app-data/plugins/*
or you can use multiple volumes and link them to your tmp like this:
docker run -v /var/lib/app-data/ui_log/:/tmp/ui_log -v /var/lib/app-data/node_log/:/tmp/node_log ... mohan/forkui:1.0