attempt to change docker data-root fails - why - docker

Question

I am trying to set my docker storage dir as other than default, something I've done on other machines:
/etc/docker/daemon.json:
{
"data-root": "/mnt/x/y/docker_data"
}
where the storage dir looks like
jeremyr#snorble:~$ ls -ltr /mnt/x/y
total 4
drwxrwxrwx 11 jeremyr 5001 122 Mar 19 08:14 docker_data
with the daemon.json file in place, sudo systemctl restart docker hits Job for docker.service failed (without that daemon.json, docker restarts fine and docker run hello-world runs fine) . with the daemon.json in place, journalctl -xn shows
Mar 25 14:20:33 bolt88 systemd[1]: docker.service start request repeated too quickly, refusing to start.
Mar 25 14:20:33 bolt88 systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Mar 25 14:20:33 bolt88 systemd[1]: Unit docker.service entered failed state.
Mar 25 14:20:34 bolt88 sudo[23961]: jeremyr : TTY=pts/18 ; PWD=/home/jeremyr ; USER=root ; COMMAND=/bin/journalctl -xn
Mar 25 14:20:34 bolt88 sudo[23961]: pam_unix(sudo:session): session opened for user root by jeremyr(uid=0)
while systemctl status docker.service just shows code=exited, status=1/FAILURE
and in dmesg I see this:
1547:[Mon Mar 25 14:21:41 2019] aufs au_opts_verify:1570:dockerd[20714]: dirperm1 breaks the protection by the permission bits on the lower branch
1548-[Mon Mar 25 14:21:41 2019] device veth34d1dfd entered promiscuous mode
1549-[Mon Mar 25 14:21:41 2019] IPv6: ADDRCONF(NETDEV_UP): veth34d1dfd: link is not ready
1550-[Mon Mar 25 14:21:41 2019] IPv6: ADDRCONF(NETDEV_CHANGE): veth34d1dfd: link becomes ready
1551:[Mon Mar 25 14:21:41 2019] docker0: port 1(veth34d1dfd) entered forwarding state
1552:[Mon Mar 25 14:21:41 2019] docker0: port 1(veth34d1dfd) entered forwarding state
1553:[Mon Mar 25 14:21:41 2019] docker0: port 1(veth34d1dfd) entered disabled state
1554-[Mon Mar 25 14:21:41 2019] device veth34d1dfd left promiscuous mode
1555:[Mon Mar 25 14:21:41 2019] docker0: port 1(veth34d1dfd) entered disabled state
1556-[Mon Mar 25 14:21:59 2019] systemd-sysv-generator[20958]: Ignoring creation of an alias umountiscsi.service for itself
Docker version 17.05.0-ce, build 89658be, on a debian 8.8 setup .
Does anyone know why docker isn't allowing use of that dir as data-root?

Answer 1

TD;DR -- worked on Ubuntu 18.04 just before post
follow the instructions:
sudo systemctl stop docker
sudo rsync -axPS /var/lib/docker/ /mnt/x/y/docker_data #copy all existing data to new location
sudo vi /lib/systemd/system/docker.service # or your favorite text editor
in file docker.service find one line like this:
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
add --data-root /mnt/x/y/docker_data to it(on one line):
ExecStart=/usr/bin/dockerd --data-root /mnt/x/y/docker_data -H fd:// --containerd=/run/containerd/containerd.sock
save and quit, then
sudo systemctl daemon-reload
sudo systemctl start docker
docker info | grep "Root Dir"
last command should output: Docker Root Dir: /mnt/x/y/docker_data
that's it, should've done here.
The Too Long version, if you Do want to Read:
after some investigating, I found some outdated articles, include this one, they mentioned some confident solution, these are typical pages:
add -g option in docker.service
not working because -g and --graph Deprecated In Release: v17.05.0
add data-root in /etc/docker/daemon.json, the method tried by question author,
not working for some unknown reason
read those solution on about one dozen web pages, got the inspiration:
How To Change Docker Data Folder Configuration
not a very good solution -- not popular, , but the interesting part is below Update::
graph has been deprecated in v17.05.0 .You can use data-root instead.
Yeah, graph => data-root, and the --graph is just the long form of -g, so I tried this substitution in solution add -g option in docker.service, and Ta da ~

Answer 2

Something is off on the docker_data.
Solution:
remove the /etc/docker/daemon.json file.
start docker.
copy the /var/lib/docker contents to the path you've put in /etc/docker/daemon.json.
put back the file /etc/docker/daemon.json and restart docker.

Answer 3

Well, I'm not an expert of docker, but I see "dirperm1 breaks the protection by the permission bits on the lower branch" in your log. And I also see this.
"drwxrwxrwx 11 jeremyr 5001 122 Mar 19 08:14 docker_data"
As my understanding, docker daemon requires the access permission to the directory. Does 5001 mean "docker" group?
However, if you ran the daemon in root permission, then it shouldn't happen.

Answer 4

Check the docker version of your machine by
docker --version
I was facing the same issue, and it got solved after upgrading the docker to latest version which is available.
Even the documentation available on docker's official website have not mentioned anything like that.
Once you upgrade docker ,
Restart the docker by
systemctl restart docker
The error will be gone, and new changes will start reflecting.