Raspberry Pi 4B... Issues with docker not running because of aufs - docker

To begin with I bought a raspberry pi 4b. I installed berry boot on the sd and connected a SSD 480gb. I managed to get it working with a Tv as a monitor hdmi group = 1 etc.
I then set up ssh with macbook so I can access my pi with my labtop or use the desktop raspbian.
Followed a 4 month old youtube video (relatively new).
https://www.youtube.com/watch?v=tx-Hfq5Hc6E
I used the commands as follows:
//install docker
#curl -sSL https://get.docker.com/ |sh
//Set up so root isn't used as user - recommended
#sudo usermod -aG docker pi
//https://hub.docker.com/r/michaelmiklis/rpi-monitor (last updated 2 years ago)...
//used quick start command for project (this worked for the video just 4 months ago)
#docker run --device=/dev/vchiq --device=/dev/vcsm --volume=/opt/vc:/opt/vc --volume=/boot:/boot --volume=/sys:/dockerhost/sys:ro --volume=/etc:/dockerhost/etc:ro --volume=/proc:/dockerhost/proc:ro --volume=/usr/lib:/dockerhost/usr/lib:ro -p=8888:8888 --name="rpi-monitor" -d michaelmiklis/rpi-monitor:latest
this command produces an error response:
#docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/b46dae086de8d3cf89a07e9079fb311d3989768f482418fbfdcf3c886aa5f516-init: mount target=/var/lib/docker/aufs/mnt/b46dae086de8d3cf89a07e9079fb311d3989768f482418fbfdcf3c886aa5f516-init data=br:/var/lib/docker/aufs/diff/b46dae086de8d3cf89a07e9079fb311d3989768f482418fbfdcf3c886aa5f516-init=rw:/var/lib/docker/aufs/diff/987d36feb1d5eda275ebd7191449f82db9921b8ea21d9f4d1089529b5bdcf30f=ro+wh:/var/lib/docker/aufs/diff/19ff179d7ce8587e1174b2cbba701a46910d7a5a4dbc62e9c58498cff5eb9bba=ro+wh:/var/lib/docker/aufs/diff/13be75a7cabe4da1b971b1fb8b25b6bc6fdb6ab13456790dfc25da23e97b9aa0=ro+wh:/var/lib/docker/aufs/diff/8b2d2a79ba8e0f4235d3ce623f6cda8486f7293af60abe24623f9fd70b2f7613=ro+wh:/var/lib/docker/aufs/diff/776dcea3b43ecd880a459ec7182305eea7d9779dab0fe45b69d3558a5401fc7b=ro+wh:/var/lib/docker/aufs/diff/a3f67653f5d5507a1ca3a4627563bd916be979ae1593dda8d0073f1f152b01a3=ro+wh:/var/lib/docker/aufs/diff/3176f9f6ed8047e376bacf9ee42370158329d66a8cfe7afd7e1a4d65d5022698=ro+wh:/var/lib/docker/aufs/diff/7427fa9add426dee44acf434935d13c6fddaac36ad222b12037b8a0a8529c222=ro+wh:/var/lib/docker/aufs/diff/27ffbeb11ca61262f65d85dfd748190b9b64223112e7faaf5f7f69eb47ada066=ro+wh:/var/lib/docker/aufs/diff/6781db1c1025f9d601bb0c4dd54df180c8fbda8d0632cc77835574b74a3f7179=ro+wh:/var/lib/docker/aufs/diff/1fa0fdfb5625eb000d3f6424b84dd8d9c19fb3ce721789277a72f2588f547659=ro+wh:/var/lib/docker/aufs/diff/a3da4ba9bfe80df4efc955929d4fa0b63d0dbf922b483f9d582507adf38e0192=ro+wh:/var/lib/docker/aufs/diff/b027240667462a4ff2e540b0cbdbbd78fdf3000c1ecdae734efc0ef794bc90f0=ro+wh:/var/lib/docker/aufs/diff/7aea6d1beaa5a57f1eac445759c1acfa563b9a8c87ce592d61797caa92547f38=ro+wh:/var/lib/docker/aufs/diff/e3ff1cda15a224b9c3af5a1a772f3373635abd031c9a4546f157b4550b7df245=ro+wh:/var/lib/docker/aufs/diff/e751ee6e4e942642958f1d9459e5b707d3970b53d70e5ae1986a70cbd2973435=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument.
See 'docker run --help'.
I have read about the my issue except in regards to the raspberry pi and haven't been able to workout a solution please help!
BTW I have tried uninstalling docker and installing it again (sudo apt remove docker-ce). If it is a driver issue I don't know how to tell docker to not use aufs but to only use ?"overlay2"?
Remember my pi boots from sd but runs raspbian os on my ssd (there might be an issue there with docker maybe). I very much appreciate any help.

Related

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]. AFTER installing nvidia-docker2

I followed the instructions to install the nvidia-docker2 from the official documentation https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html
Whenever I run their test example:
sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
I still get the error:
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]. 3
I rebooted but still no effect.
I am on Ubuntu 22.04 with my nvidia drivers updated.
Nvidia-smi works on the machine but not working using docker
EDIT (SOLVED): Finally I found out what was going on.
When reinstalling, it was working, however if rebooting, it was going again to a previous state where it was not working.
This was due to the installation of another docker service installed using "snapd" so I had to purge completely docker:
sudo snap remove docker and after I could "Reinstall everything" and it finally is stable, even after rebooting
Unfortunately I was not able to "Fix" properly the issue so I purge all docker package and all nvidia container packages and reinstalled everything and now it works!!
Good old methods work fine :)
you need to restart the docker daemon :
sudo systemctl restart docker
if the problem still occurs install the nvidia-container-toolkit then restart docker daemon.

nvidia-smi gives an error inside of a docker container

Sometimes I can't communicate with my Nvidia GPUs inside a docker container when I came back to my workplace from home, even though the previously launched process that utilizes GPUs is running well. The running process (training a neural network via Pytorch) is not affected by the disconnection but I cannot launch a new process.
nvidia-smi gives Failed to initialize NVML: Unknown Error and torch.cuda.is_available() returns False likewise.
I met two different cases:
nvidia-smi works fine when it is done at the host machine. In this case, the situation can be solved by restarting the docker container via docker stop $MYCONTAINER followed by docker start $MYCONTAINER at the host machine.
nvidia-smi doesn't work at the host machine nor nvcc --version, throwing Failed to initialize NVML: Driver/library version mismatch and Command 'nvcc' not found, but can be installed with: sudo apt install nvidia-cuda-toolkit error. Strange point is that the current process still runs well. In this case, installing the driver again or rebooting the machine solves the problem.
However, these solutions require stopping all current processes. It would be unavailable when I should not stop the current process.
Does somebody has suggestion for solving this situation?
Many thanks.
(sofwares)
Docker version: 20.10.14, build a224086
OS: Ubuntu 22.04
Nvidia driver version: 510.73.05
CUDA version: 11.6
(hardwares)
Supermicro server
Nvidia A5000 * 8
(pic1) nvidia-smi not working inside of a docker container, but worked well on the host machine.
(pic2) nvidia-smi works after restarting a docker container, which is the case 1 I mentioned above
For the problem of Failed to initialize NVML: Unknown Error and having to restart the container, please see this ticket and post your system/package information there as well:
https://github.com/NVIDIA/nvidia-docker/issues/1671
There's a workaround on the ticket, but it would be good to have others post their configuration to help fix the issue.
Downgrading containerd.io to 1.6.6 works as long as you specify no-cgroups = true in /etc/nvidia-container-runtime/config.toml and specify the devices to docker run like docker run --gpus all --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidia-modeset:/dev/nvidia-modeset --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools --device /dev/nvidiactl:/dev/nvinvidiactl --rm -it nvidia/cuda:11.4.2-base-ubuntu18.04 bash
so sudo apt-get install -y --allow-downgrades containerd.io=1.6.6-1 and sudo apt-mark hold containerd.io to prevent the package from being updated. So do that, edit the config file, and pass all of the /dev/nvidia* devices in to docker run.
For the Failed to initialize NVML: Driver/library version mismatch issue, that is caused by the drivers updating but you haven't rebooted yet. If this is a production machine, I would also hold the driver package to stop that from auto-updating as well. You should be able to figure out the package name from something like sudo dpkg --get-selections "*nvidia*"

IOTA "one-click-tangle" via docker linux containers - ERROR 1

I am trying to run a private tangle on my computer through linux docker containers.
Therefore I followed the guide over at https://wiki.iota.org/chrysalis-docs/tutorials/one_click_private_tangle
Every step succeeded up until we tried to execute
./private_tangle.sh install
This reports
Error 1
as seen in the screenshot below:
We do net get any further information, is anyone familiar with this error, or has any clue how to get some more information on the error so that we can at least have a clue where to look?
Some further information:
After executing docker ps -a we see that not a single container is running.
I am running on a windows 10 machine
I execute the commands from within ubuntu (version 20.04)
Ubuntu, docker-desktop and docker-desktop-data are all running WSL2
Docker integration with ubuntu is activated
I thought the error could maybe come from no hornet node initially being installed, so I installed a hornet node successfully, according the guide that https://wiki.iota.org/chrysalis-docs/tutorials/one_click_private_tangle. This changed nothing to the Error.
The version of docker and docker-compose are compliant with the requirements
If any more details are needed to help me solve this problem, please let me know.
I used the documentation (https://wiki.iota.org/chrysalis-docs/tutorials/one_click_private_tangle) to install these containers on my local ubuntu 18.04.
My docker version is: 20.10.12
And docker-compose version is: 1.29.2
By following the steps of the tutorial I managed to successfully start all of the containers without trouble.
My guess here would be that the permission of the 'private-tangle.sh' are not correct or that there is permission problem on the docker level.
You should start with checking the permission level of the private-tangle.sh script by using $ls -l
Here is my output -rwxrwxr-x 1 ben ben 9413 Jan 11 11:28 private-tangle.sh
It could also be due to the docker rights if you have to use sudo when executing a docker command it will give some troubles when executing the script.
You need to add yourself to a docker group to be able to run docker commands without sudo. You can do this by running sudo usermod -aG docker $USER with damiaan-vh as $user.
Solution from source https://stackoverflow.com/posts/70665394/edit
Suggesting to downgrade ubuntu version to 18.04 for more stable version.
For reinstalling the docker and docker-compose programs follow this documentations
(docker: https://docs.docker.com/engine/install/ubuntu/ )
(docker-compose: https://docs.docker.com/compose/install/ )

How can I use an ueye camera from IDS with ROS through a docker?

I have a docker container containing ubuntu 20.04 and ROS Noetic and I'm trying to control a Ueye camera from IDS with it. For this, I downloaded the ueye_cam package for ROS:
http://wiki.ros.org/ueye_cam
and the image_pipeline package:
http://wiki.ros.org/image_pipeline
I also downloaded the package corresponding to my camera on the IDS website and followed the tutorial for the installation:
https://www.thorlabs.com/software/THO/ThorCam/Linux/Readme_uEye_Linux_4.90.06.html
When I try to run $ roslaunch ueye_cam rgb8.launch
I get the following error message:
[ERROR] [1626351965.262425900]: No UEye cameras are connected
[ERROR] [1626351965.262560700]: Hint: make sure that the IDS camera daemon (/etc/init.d/ueyeusbdrc) is running
[ERROR] [1626351965.262743000]: Failed to initialize [camera]
I ran:
$ /usr/bin/ueyeusbd start
and got :
IDS ueyeusbd 4.94.1557 64bit build Apr 30 2021 08:04:32 ( cmdl cap scq )
and ran :
$ /etc/init.d/ueyeusbdrc start
and got the error:
Starting ueyeusbd... IDS ueyeusbd 4.94.1557 64bit build Apr 30 2021 08:04:32 ( cmdl cap scq )
ueyeusbd is not running.
I tried following this solution:
How can I run a service inside a docker container to get feed from a IDS uEye camera using gstreamer?
but it doesn't work because I don't have /var on my windows host.
I also tried:
systemctl start ueyeusbdrc
but I got this error:
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Does anyone know if it is possible to control an ueye camera from IDS, connected to a windows host, through a docker containing ROS1?
If not, do you know if it is possible to control an ueye camera with a ROS2 package because I already have the corresponding ROS2 drivers?
The Solution that did work for me is the following
Add a few options to the docker command.
Adding these arguments to the docker command solved the issue:
...
--privileged -v /dev/bus/usb:/dev/bus/usb
--ipc=host
--pid=host
-v /var/run:/var/run
-v /usr:/usr
...
The other key part is to install the IDS software on the host machine.
and make sure the system service for the IDS uEye camera is running in the host
(sudo systemctl start ueyeusbdrc)
and enable the docker feature via the IDS-manager software on the host
Part of the solution was inspired from this post
How can I run a service inside a docker container to get feed from a IDS uEye camera using gstreamer?
Hope this will help
Btw tested this solution only with ROS(1)

Getting docker daemon not running error

I am trying to get docker working on my sys
However, not able to
Steps to reproduce the issue that I am facing:
Installed EPEL on 6.5 RHEL
Installed docker-io
Able to run "docker" command
When running "docker run -i -t fedora /bin/bash" command using without root, getting below error:
FATA[0000] Post http:///var/run/docker.sock/v1.17/containers/create:
dial unix /var/run/docker.sock
http:///var/run/docker.sock/v1.17/containers/create:%20dial%20unix%20/var/run/docker.sock :
permission denied.
Are you trying to connect to a TLS-enabled daemon without TLS?
However, with root getting diff error like below for the same command:
FATA[0000] Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
A RHEL 6.5, also termed Update 5, is from 21 November 2013 and comes with kernel 2.6.32-431.
That seems quite an old kernel for docker to be installed and run successfully. Docker would need ideally a 3.10+ kernel.
Although Adrian Mouat mentions in the comments that the Red Hat Enterprise Linux 6.5 (64-bit) or later is supported
You will need 64 bit RHEL 6.5 or later, with a RHEL 6 kernel version 2.6.32-431 or higher as this has specific kernel fixes to allow Docker to work.
So make sure the docker daemon is started:
sudo service docker start
Then try some sudo docker commands:
sudo docker run -i -t fedora /bin/bash
The doc mentions:
If you get a Cannot start container error mentioning SELinux or permission denied, you may need to update the SELinux policies.
This can be done using sudo yum upgrade selinux-policy and then rebooting.

Resources