Docker: using SSH in builds with Buildkit - docker

Question

Following the documentation I'm trying to pass an SSH key to my container. This is my original Dockerfile
# syntax=docker/dockerfile:experimental
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.6
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh git clone git#github.com:USER/REPO.git
and this works
DOCKER_BUILDKIT=1 docker build --ssh default=~/github .
However, if I try to install anything with apt:
# syntax=docker/dockerfile:experimental
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.6
RUN apt update
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh git clone git#github.com:USER/REPO.git
I get the following error:
[+] Building 1.8s (7/9)
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 306B 0.0s
=> resolve image config for docker.io/docker/dockerfile:experimental 1.1s
=> CACHED docker-image://docker.io/docker/dockerfile:experimental#sha256:de85b2f3a3e8a2f7fe48e8e84a65f6fdd5cd5183afa6412fff9caa6871649c44 0.0s
=> [internal] load metadata for docker.io/tiangolo/uvicorn-gunicorn-fastapi:python3.6 0.0s
=> CACHED [1/4] FROM docker.io/tiangolo/uvicorn-gunicorn-fastapi:python3.6 0.0s
=> ERROR [2/4] RUN apt update 0.4s
------
> [2/4] RUN apt update:
#7 0.352
#7 0.352 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#7 0.352
#7 0.359 Reading package lists...
#7 0.375 E: Could not get lock /var/lib/apt/lists/lock - open (13: Permission denied)
#7 0.375 E: Unable to lock directory /var/lib/apt/lists/
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = failed to build LLB: executor failed running [/bin/sh -c apt update]: runc did not terminate sucessfully
However, the second Dockerfile actually works if Buildkit is disabled. Any suggestions on what might be the problem?

Answer 1

I had this exact same issue. For me the solution was to upgrade Docker. I had this issue with 19.03.11 which my Ubuntu install was pulling in as a snap. 20.10.1 (latest as of this post) worked for me.
More info here: https://github.com/moby/moby/issues/39106#issuecomment-752246367
edit: Unfortunately this doesn't work when the build is run non-interactively (for example, as a systemd-based CI agent) - at least for me.