microk8s in docker container - docker

Question

For the automatic testing of my code I would like to add a microk8s in my docker-compose setup for testing. Hence, I'd like to install microk8s in docker environment
I got snap running in the docker container using the following commands from the web
FROM ubuntu:18.04
ENV container docker
ENV PATH /snap/bin:$PATH
ADD snap /usr/local/bin/snap
RUN apt-get update
RUN apt-get install -y snapd squashfuse fuse
RUN systemctl enable snapd
STOPSIGNAL SIGRTMIN+3
CMD [ "/sbin/init" ]
snap script
!/bin/sh -e
while ! kill -0 $(pidof snapd) 2>/dev/null; do
echo "Waiting for snapd to start."
sleep 1
done
/usr/bin/snap $#
and building docker
docker build -t snapd .
and run
docker run --name=snapd -ti -d --tmpfs /run --tmpfs /run/lock --tmpfs /tmp --privileged -v /lib/modules:/lib/modules:ro snapd
up to here everything is fine.
However if try to install microk8s via snap it fails
snap install microk8s --classic --channel=1.18/stable
2020-04-27T14:22:39Z INFO Waiting for restart...
error: cannot perform the following tasks:
- Run install hook of "microk8s" snap if present (run hook "install": execv failed: Permission denied)
checking snap systemctl status snapd.service gives me
Apr 27 15:14:32 8985fc7fc5cb snapd[489]: helpers.go:961: cannot retrieve info for snap "microk8s": cannot find installed snap "microk8s" at revision 1341: missing file /sn
ap/microk8s/1341/meta/snap.yaml
Apr 27 15:14:33 8985fc7fc5cb snapd[489]: helpers.go:105: error trying to compare the snap system key: system-key versions not comparable
Apr 27 15:14:33 8985fc7fc5cb snapd[489]: helpers.go:961: cannot retrieve info for snap "microk8s": cannot find installed snap "microk8s" at revision 1341: missing file /sn
ap/microk8s/1341/meta/snap.yaml
Apr 27 15:14:33 8985fc7fc5cb systemd[1]: Started Snappy daemon.
Apr 27 15:15:08 8985fc7fc5cb snapd[489]: handlers.go:495: Reported install problem for "microk8s" as e11fe0c4-8899-11ea-a8e2-fa163ee63de6 OOPSID

Answer 1

I found a (for me) somewhat satisfying answer is to use k3s
A German description can be found here
The key is the following docker-compose.yml
version: '3'
services:
server:
image: rancher/k3s:v0.8.1
command: server --disable-agent
environment:
- K3S_CLUSTER_SECRET=somethingtotallyrandom
- K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml
- K3S_KUBECONFIG_MODE=666
volumes:
- k3s-server:/var/lib/rancher/k3s
# get the kubeconfig file
- .:/output
ports:
- 6443:6443
node:
image: rancher/k3s:v0.8.1
tmpfs:
- /run
- /var/run
privileged: true
environment:
- K3S_URL=https://server:6443
- K3S_CLUSTER_SECRET=somethingtotallyrandom
ports:
- 31000-32000:31000-32000
worker:
image: rancher/k3s:v0.8.1
tmpfs:
- /run
- /var/run
privileged: true
environment:
- K3S_URL=https://server:6443
- K3S_CLUSTER_SECRET=somethingtotallyrandom
volumes:
k3s-server: {}

Answer 2

(Just for completeness but I don't accept this answer, if so it my own but suggested by #David Maze) You can use kind Kubernetes in Docker. it was primarily developed for testing docker itself.