docker run error(operation not support) after load busybox images on embeded playform - docker

Question

I built a docker form source for my embedded platform(armv7) and install them successfully. But I got an error(docker: Error response from daemon: operation not supported) when trying load arm32v7/busybox image
I built docker(Version: 18.09.4) and its dependencies with buildroot and install it successfully
2.Then I run dockerd with command "dockerd -s overlay &"
3.And I load armv7/busybox image which was saved from my notebook.
(As my embeded board was limited by gateway and can't connect the Internet, I use my notebook save image for my embeded board)
4.finally I load the image successfully and run it failed with "docker: Error response from daemon: operation not supported"
5.run docker info on board output like this
# docker info
WARN[1970-01-02T02:46:52.315076016Z] Could not get operating system name: Error opening /usr/lib/os-release: open /usr/lib/os-release: no such file or directory
WARN[1970-01-02T02:46:52.328707599Z] failed to retrieve docker-init version: exec: "docker-init": executable file not found in $PATH
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 18.09.4
Storage Driver: overlay
Backing Filesystem: tmpfs
Supports d_type: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version:
runc version: v1.0.0-rc7
init version: N/A (expected: )
Kernel Version: 4.9.37
OSType: linux
Architecture: armv7l
CPUs: 2
Total Memory: 498.3MiB
Name: (none)
ID: VS4X:CF4M:UNNY:QKJW:2PKQ:5ZUL:7N7C:24AK:O2VN:ZRLM:BUZO:5BHO
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
WARNING: the overlay storage-driver is deprecated, and will be removed in a future release.
1.on my notebook
leo#ubuntu:~$ sudo docker pull arm32v7/busybox
Using default tag: latest
latest: Pulling from arm32v7/busybox
2b35d97f9c81: Pulling fs layer
2b35d97f9c81: Pull complete
Digest: sha256:6b1ef683f696c503a0afb93d06684a9a70f8f793a90220eb0da569be116ce1d1
Status: Downloaded newer image for arm32v7/busybox:latest
leo#ubuntu:~$ sudo docker save arm32v7/busybox -o arm_busybox.tar
2.on my embeded board
# rz
rz waiting to receive.
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring arm_busybox.tar...
100% 1141 KB 1141 KB/sec 00:00:01 0 Errors
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
arm32v7/busybox latest 94491159407b Less than a second ago 944kB
# docker^C
# docker images rmi 94491159407b
"docker images" requires at most 1 argument.
See 'docker images --help'.
Usage: docker images [OPTIONS] [REPOSITORY[:TAG]]
List images
# docker rmi 94491159407b
Untagged: arm32v7/busybox:latest
Deleted: sha256:94491159407b8aca603c71cdcaf164be9e63ed17239b2b081ce3a6bf765f6e8a
Deleted: sha256:7dc29058f73fc3fe174d0cc6795bee0a4130ba807f971e6e0cf4a88d77a9fe53
#
# ls
arm_busybox.tar
# docker load -i arm_busybox.tar
7dc29058f73f: Loading layer [==================================================>] 1.16MB/1.16MB
Loaded image: arm32v7/busybox:latest
#
# docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
arm32v7/busybox latest 94491159407b Less than a second ago 944kB
# docker run arm32v7/busybox
docker: Error response from daemon: operation not supported.
See 'docker run --help'.
# docker run arm32v7/busybox ls
docker: Error response from daemon: operation not supported.
See 'docker run --help'.
# docker run arm32v7/busybox /bin/sh
docker: Error response from daemon: operation not supported.
See 'docker run --help'.
docker run arm32v7/busybox should return successfully. but get an error response.
why?

Answer 1

After CONFIG_TMPFS_XATTR and CONFIG_TMPFS_POSIX_ACL flags switch to [=y] (before compiling kernel),docker run don't return error, but block !!
I enable debug mode when starting dockerd. there is log of dockerd
# dockerd -D -s overlay
...
DEBU[1970-01-02T07:55:03.848247951Z] form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":null,"Domainname":"","Entrypoint":null,"Env":[],"HostConfig":{"AutoRemove":false,"Binds":null,"BlkioDeviceReadBps":null,"BlkioDeviceReadIOps":null,"BlkioDeviceWriteBps":null,"BlkioDeviceWriteIOps":null,"BlkioWeight":0,"BlkioWeightDevice":[],"CapAdd":null,"CapDrop":null,"Cgroup":"","CgroupParent":"","ConsoleSize":[0,0],"ContainerIDFile":"","CpuCount":0,"CpuPercent":0,"CpuPeriod":0,"CpuQuota":0,"CpuRealtimePeriod":0,"CpuRealtimeRuntime":0,"CpuShares":0,"CpusetCpus":"","CpusetMems":"","DeviceCgroupRules":null,"Devices":[],"DiskQuota":0,"Dns":[],"DnsOptions":[],"DnsSearch":[],"ExtraHosts":null,"GroupAdd":null,"IOMaximumBandwidth":0,"IOMaximumIOps":0,"IpcMode":"","Isolation":"","KernelMemory":0,"Links":null,"LogConfig":{"Config":{},"Type":""},"MaskedPaths":null,"Memory":0,"MemoryReservation":0,"MemorySwap":0,"MemorySwappiness":-1,"NanoCpus":0,"NetworkMode":"default","OomKillDisable":false,"OomScoreAdj":0,"PidMode":"","PidsLimit":0,"PortBindings":{},"Privileged":false,"PublishAllPorts":false,"ReadonlyPaths":null,"ReadonlyRootfs":false,"RestartPolicy":{"MaximumRetryCount":0,"Name":"no"},"SecurityOpt":null,"ShmSize":0,"UTSMode":"","Ulimits":null,"UsernsMode":"","VolumeDriver":"","VolumesFrom":null},"Hostname":"","Image":"94491159407b","Labels":{},"NetworkingConfig":{"EndpointsConfig":{}},"OnBuild":null,"OpenStdin":false,"StdinOnce":false,"Tty":false,"User":"","Volumes":{},"WorkingDir":""}
DEBU[1970-01-02T07:55:03.907315451Z] container mounted via layerStore: &{/var/lib/docker/overlay/9efba7d0f8b20b508274373120276bfcead4c0afdc98ac6c813a41e64c61df9e/merged 0x2bdd510 0x2bdd510}
DEBU[1970-01-02T07:55:03.946627243Z] Calling POST /v1.39/containers/77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417/attach?stderr=1&stdout=1&stream=1
DEBU[1970-01-02T07:55:03.947147743Z] attach: stdout: begin
DEBU[1970-01-02T07:55:03.953483993Z] Calling POST /v1.39/containers/77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417/wait?condition=next-exit
DEBU[1970-01-02T07:55:03.954236410Z] attach: stderr: begin
DEBU[1970-01-02T07:55:03.964652451Z] Calling POST /v1.39/containers/77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417/start
DEBU[1970-01-02T07:55:03.981005285Z] container mounted via layerStore: &{/var/lib/docker/overlay/9efba7d0f8b20b508274373120276bfcead4c0afdc98ac6c813a41e64c61df9e/merged 0x2bdd510 0x2bdd510}
DEBU[1970-01-02T07:55:03.982641076Z] Assigning addresses for endpoint boring_montalcini's interface on network bridge
DEBU[1970-01-02T07:55:03.982754368Z] RequestAddress(LocalDefault/172.17.0.0/16, <nil>, map[])
DEBU[1970-01-02T07:55:03.982917743Z] Request address PoolID:172.17.0.0/16 App: ipam/default/data, ID: LocalDefault/172.17.0.0/16, DBIndex: 0x0, Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:2 Serial:false PrefAddress:<nil>
DEBU[1970-01-02T07:55:04.013580701Z] Assigning addresses for endpoint boring_montalcini's interface on network bridge
INFO[1970-01-02T07:55:04.015169910Z] /etc/resolv.conf does not exist
INFO[1970-01-02T07:55:04.015304618Z] No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]
INFO[1970-01-02T07:55:04.015357285Z] IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]
DEBU[1970-01-02T07:55:04.018542701Z] Programming external connectivity on endpoint boring_montalcini (dcf05afcaeb79ee2fc5fc4a6aceda37c6083f3a19fe608fabb4bb8d91f7e5f1b)
DEBU[1970-01-02T07:55:04.019416368Z] EnableService 77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417 START
DEBU[1970-01-02T07:55:04.019500118Z] EnableService 77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417 DONE
DEBU[1970-01-02T07:55:04.022324743Z] bundle dir created bundle=/var/run/docker/containerd/77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417 module=libcontainerd namespace=moby root=/var/lib/docker/overlay/9efba7d0f8b20b508274373120276bfcead4c0afdc98ac6c813a41e64c61df9e/merged
DEBU[1970-01-02T07:55:04.030072118Z] event published ns=moby topic="/containers/create" type=containerd.events.ContainerCreate
INFO[1970-01-02T07:55:04.060624576Z] shim containerd-shim started address="/containerd-shim/moby/77d7e164a3a10e34bb25ff59852bdfac9c962639dedf543614d71107567f6417/shim.sock" debug=true pid=1383
DEBU[0000] registering ttrpc server
DEBU[0000] serving api on unix socket socket="[inherited from parent]"

Answer 2

mv root dir to a persistent path, not tmpfs, then：
stop dockerd
mv /var/lib/docker /root/docker
start dockerd
docker run arm32v7/busybox