What's the reason for the 42 layer limit in Docker? - docker

Question

At different places I found the information that a docker image can only consist of up to 42 layers. This seems to be a limitation of the used AUFS file system.
Can anybody tell me why this limit exists or does anybody have some documentation explaining this?

Answer 1

I'm starting to suspect that there isn't any such a hard limit.
Create the following python script, and name it "makeDockerfile.py"
with open("Dockerfile", "w") as file:
file.write("from alpine:3.8\n")
for i in range(0, 201):
file.write("run echo {i}\n".format(i=i))
then run
python makeDockerfile.py && docker build --tag manylayer . && docker run -it manylayer /bin/sh
You'll see that you are running a working container with > 200 layers.
(note, this was tested with linux containers on linux)
Note that this doesn't mean that this many layers are necessarily SUPPORTED, just that they are possible in some cases.
In fact, I've seen containers fail with far fewer than 42 layers, and removing any arbitrary layer seems to fix it. (see https://github.com/docker/for-win/issues/676#issuecomment-462991673 )
EDIT:
thaJeztah, maintainer of Docker, has this to say about it:
The "42 layer limit" on aufs was on older versions of aufs, but should no longer be the case.
However, the 127 layer limit is still there. This is due to a restriction of Linux not accepting more than X arguments to a syscall that's used.
Although this limit can be raised in modern kernels, it's not the default, so if we'd go beyond that maximum, an Image built on one machine may not be usable on another machine.
( see https://github.com/docker/docker.github.io/issues/8230 )

Answer 2

From the Docker BP documentation:
Minimize the number of layers
You need to find the balance between readability (and thus long-term maintainability) of the Dockerfile and minimizing the number of layers it uses. Be strategic and cautious about the number of layers you use.
They also give advices on how to avoid too many layers:
That way you can delete the files you no longer need after they’ve been extracted and you won’t have to add another layer in your image
[..]
Lastly, to reduce layers and complexity, avoid switching USER back and forth frequently.
TL;DR: the benefit of minimizing the number of layers can be likened to the benefit of minimizing the number of small files but rather have fewer bigger ones. A docker pull is also faster (try downloading 2048 files of 1kB or one file of 2MB) . and having fewer layers reduces the complexity of an image, hence the maintainability.
As for the 42 limit. Well... I guess they had to come up with a number and they pick this particular one ;)

Answer 3

This seems to be imposed primarily by AUFS (sfjro/aufs4-linux).
See PR 50 "Prohibit more than 42 layers in the core "
Hey, out of curiosity, what's the detailed rationale behind this?
People have been asking how to bypass the 42 layers limit, and we've always told them "wait for device mapper!" so... what should we tell them now?
We can't allow more than 42 layers just yet. Devicemapper indeed supports it, but then, if you push a 50 layer image to the registry, most people won't be able to use it. We'll enable this once we support more than 42 layers for AUFS.
PR 66 was supposed to remove that limit
This thread reported:
The 42 limit comes from the aufs code itself.
My guess is that it's because of stack depth: if you stack these kernel data structures too many levels deep, you might trigger an overflow somewhere. The hardcoded limit is probably a pragmatic way to avoid that.