How to run OpenCL + OpenGL inside a Docker container? - docker

The aim is to run an OpenCL/OpenGL (interop) app inside a docker container. But I have not been successful yet.
Intro
I have laptop with an NVidia graphics card so I thought leveraging on NVidia Dockerfiles [1,2] would be a good starting point.
The following Dockerfile:
# Dockerfile to run OpenGL app
FROM nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},display
RUN apt-get update && apt-get install -y --no-install-recommends \
mesa-utils && \
rm -rf /var/lib/apt/lists/*
works quite well, and I was able to run glxgears.
Running OpenCL on its own container was no big deal either:
# Dockerfile to run OpenCL app
FROM nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
ocl-icd-libopencl1 \
clinfo && \
rm -rf /var/lib/apt/lists/*
RUN mkdir -p /etc/OpenCL/vendors && \
echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
and clinfo successfully shows information about my device.
Attempt
Finally here's my attempt at creating a container with both OpenGL and OpenCL drivers:
# Dockerfile mixing OpenGL and OpenCL
FROM nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES},display
RUN apt-get update && apt-get install -y --no-install-recommends \
mesa-utils \
ocl-icd-libopencl1 \
clinfo && \
rm -rf /var/lib/apt/lists/*
RUN mkdir -p /etc/OpenCL/vendors && \
echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
And now, although clinfo still prints OpenCL device information, glxgears on the other hand fails with the following error:
Error: couldn't get an RGB, Double-buffered visual
Any idea how to make this work? Thanks in advance.
References
[1] https://gitlab.com/nvidia/opencl/blob/ubuntu16.04/devel/Dockerfile
[2] https://gitlab.com/nvidia/opencl/blob/ubuntu16.04/runtime/Dockerfile

ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
You forgot the capability display.

What did work for me is the following
STEP 1: added at the end of the Dockerfile the following two lines
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility,display
STEP 2: to run the container
$ sudo xhost +local:root
$ docker run --gpus all -it --rm --name container_name \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=$DISPLAY \
-e QT_X11_NO_MITSHM=1 \
--net=host \
image_name bash

Related

Tmux as Entrypoint deactivates unicode characters

I'm building a Docker image including a ready to use terminal with all my usual tools.
I'm running a 2020 Macbook Air M1 running Monterey 12.5.1.
I'd like to start the container directly in a tmux session, but the characters display behavior is inconsistent.
When ENTRYPOINT is ["zsh"] and I execute tmux in the interactive container, the characters are as expected :
and when executing tmux :
but when changing the ENTRYPOINT to ["zsh", "-c", "tmux"] :
Here is my Dockerfile :
FROM ubuntu:22.04
ARG USER=ben
ENV GROUP=${USER}
ENV HOME=/home/${USER}
ENV TMUX_SESSION_NAME=devops
RUN groupadd ${GROUP}
RUN useradd -m -g ${GROUP} ${USER}
RUN apt-get update -y && apt-get upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata
RUN apt-get install -y \
ca-certificates \
curl \
git \
wget \
docker \
vim \
fzf \
zsh \
fd-find \
zsh-syntax-highlighting \
tmux \
locales \
locales-all
RUN usermod -s /bin/zsh ${USER}
# Configuring locales
RUN ln -fs /usr/share/zoneinfo/Europe/Paris /etc/localtime \
&& dpkg-reconfigure --frontend noninteractive tzdata
USER ${USER}
WORKDIR /home/${USER}
# Oh-My-Zsh configuration
RUN wget https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh -O - | zsh || true
# ZSH plugins
RUN git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k
RUN git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-${HOME}/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
RUN git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-${HOME}/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
COPY --chown=${USER}:${GROUP} zshrc ${HOME}/.zshrc
COPY --chown=${USER}:${GROUP} tmux.conf ${HOME}/.tmux.conf
COPY --chown=${USER}:${GROUP} p10k.zsh ${HOME}/.p10k.zsh
# ENTRYPOINT ["zsh", "-c", "tmux"]
ENTRYPOINT ["zsh"]
I couldn't find the reason for this behavior, but I investigated starting tmux directly from zsh and not in the ENTRYPOINT, and the solution that solved my issue was to set the environment variables ZSH_TMUX_AUTOSTART=true.
Thank you all for your help !

docker over virtualbox cannot start

I have a docker file that creates a valid image that runs on my Ubuntu 18.04.
For compatibility with other machines, I've tried to run the docker in a Virtual Box Ubuntu machine (and avoid any configuration errors that may occur).
my docker run command line:
docker run -id --net=host --rm --privileged --gpus=all --env="NVIDIA_DRIVER_CAPABILITIES=all" --env="DISPLAY" -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix:rw -v /run/user/1000/gdm/Xauthority:/root/.Xauthority --env="QT_X11_NO_MITSHM=1" --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v /home/git/:/git --name nirge_sim nirge-sim:1.0
The base docker file:
FROM gazebo:gzserver9-bionic
# nvidia-container-runtime
ENV NVIDIA_VISIBLE_DEVICES \
${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES \
${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics
# install Utilities
RUN apt-get update -y && apt-get install -y apt-utils curl ca-certificates wget \
&& rm -rf /var/lib/apt/lists/*
# install gazebo packages
RUN apt-get update -y && apt-get install -y --allow-unauthenticated --no-install-recommends \
libgazebo9-dev \
&& rm -rf /var/lib/apt/lists/*
# install ros packages
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
RUN curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | apt-key add -
RUN apt-get update && apt-get install -y --allow-unauthenticated \
ros-melodic-desktop-full \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --allow-unauthenticated --no-install-recommends \
ros-melodic-gazebo-ros-pkgs ros-melodic-gazebo-ros-control \
ros-melodic-gazebo-plugins ros-melodic-gazebo-ros ros-melodic-gazebo-ros\
ros-melodic-simulators \
&& rm -rf /var/lib/apt/lists/*
# final config for ros
RUN echo 'source /opt/ros/melodic/setup.bash' >> /root/.bashrc
RUN echo 'export LIBGL_ALWAYS_INDIRECT=1' >> /root/.bashrc
CMD ["bash"]
So this works on my Host, but not on my hosted host via virtual box.
the error is:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: nvml error: driver not loaded: unknown.
Would appreciate any advice on this issue.
It appears that one of the dependencies (Gazebo) requires a dedicated GPU, one that is not simulated as a part of VirtualBox.
Nvidia cards tend to work well in Ubuntu
sourced from original site

Elastic user password is not working for my docker image

When I am using the elasticsearch official docker image ELASTIC_PASSWORD env variable is working good
docker run -dti -e ELASTIC_PASSWORD=my_own_password -e discovery.type=single-node elasticsearch:7.8.0
But when I build my own customized docker image the ELASTIC_PASSWORD is not working can you please help me on this
Here is my Docker file
FROM ubuntu:18.04
ENV \
REFRESHED_AT=2020-06-20
###############################################################################
# INSTALLATION
###############################################################################
### install prerequisites (cURL, gosu, tzdata, JDK for Logstash)
RUN set -x \
&& apt update -qq \
&& apt install -qqy --no-install-recommends ca-certificates curl gosu tzdata openjdk-11-jdk-headless \
&& apt clean \
&& rm -rf /var/lib/apt/lists/* \
&& gosu nobody true \
&& set +x
### set current package version
ARG ELK_VERSION=7.8.0
### install Elasticsearch
# predefine env vars, as you can't define an env var that references another one in the same block
ENV \
ES_VERSION=${ELK_VERSION} \
ES_HOME=/opt/elasticsearch
ENV \
ES_PACKAGE=elasticsearch-${ES_VERSION}-linux-x86_64.tar.gz \
ES_GID=991 \
ES_UID=991 \
ES_PATH_CONF=/etc/elasticsearch \
ES_PATH_BACKUP=/var/backups \
KIBANA_VERSION=${ELK_VERSION}
RUN DEBIAN_FRONTEND=noninteractive \
&& mkdir ${ES_HOME} \
&& curl -O https://artifacts.elastic.co/downloads/elasticsearch/${ES_PACKAGE} \
&& tar xzf ${ES_PACKAGE} -C ${ES_HOME} --strip-components=1 \
&& rm -f ${ES_PACKAGE} \
&& groupadd -r elasticsearch -g ${ES_GID} \
&& useradd -r -s /usr/sbin/nologin -M -c "Elasticsearch service user" -u ${ES_UID} -g elasticsearch elasticsearch \
&& mkdir -p /var/log/elasticsearch ${ES_PATH_CONF} ${ES_PATH_CONF}/scripts /var/lib/elasticsearch ${ES_PATH_BACKUP}
As I think in order to achieve this functionality (set ELASTIC_PASSWORD from command line and it works) for your own container you need to re-configure Elasticsearch startup script. It's not a trivial task.
For example here is docker-entrypoint.sh from official docker image.
https://github.com/elastic/elasticsearch/blob/master/distribution/docker/src/docker/bin/docker-entrypoint.sh
You can see that script do all 'hidden' work to allow us to run it by only command.

Error :- Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "

I am asking for a massive favor. I was stuck below the issue for the last couple of days. If someone helps then that would be great. Going back to the issue. I have installed a docker and docker container using the following code (Docker-Apache spark).
Docker File:-
FROM debian:stretch
MAINTAINER Getty Images "https://github.com/gettyimages"
RUN apt-get update \
&& apt-get install -y locales \
&& dpkg-reconfigure -f noninteractive locales \
&& locale-gen C.UTF-8 \
&& /usr/sbin/update-locale LANG=C.UTF-8 \
&& echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
&& locale-gen \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Users with other locales should set this in their derivative image
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RUN apt-get update \
&& apt-get install -y curl unzip \
python3 python3-setuptools \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& easy_install3 pip py4j \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# http://blog.stuart.axelbrooke.com/python-3-on-spark-return-of-the-pythonhashseed
ENV PYTHONHASHSEED 0
ENV PYTHONIOENCODING UTF-8
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
# JAVA
RUN apt-get update \
&& apt-get install -y openjdk-8-jre \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# HADOOP
ENV HADOOP_VERSION 3.0.0
ENV HADOOP_HOME /usr/hadoop-$HADOOP_VERSION
ENV HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
ENV PATH $PATH:$HADOOP_HOME/bin
RUN curl -sL --retry 3 \
"http://archive.apache.org/dist/hadoop/common/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz" \
| gunzip \
| tar -x -C /usr/ \
&& rm -rf $HADOOP_HOME/share/doc \
&& chown -R root:root $HADOOP_HOME
# SPARK
ENV SPARK_VERSION 2.4.1
ENV SPARK_PACKAGE spark-${SPARK_VERSION}-bin-without-hadoop
ENV SPARK_HOME /usr/spark-${SPARK_VERSION}
ENV SPARK_DIST_CLASSPATH="$HADOOP_HOME/etc/hadoop/*:$HADOOP_HOME/share/hadoop/common/lib/*:$HADOOP_HOME/share/hadoop/common/*:$HADOOP_HOME/share/hadoop/hdfs/*:$HADOOP_HOME/share/hadoop/hdfs/lib/*:$HADOOP_HOME/share/hadoop/hdfs/*:$HADOOP_HOME/share/hadoop/yarn/lib/*:$HADOOP_HOME/share/hadoop/yarn/*:$HADOOP_HOME/share/hadoop/mapreduce/lib/*:$HADOOP_HOME/share/hadoop/mapreduce/*:$HADOOP_HOME/share/hadoop/tools/lib/*"
ENV PATH $PATH:${SPARK_HOME}/bin
RUN curl -sL --retry 3 \
"https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/${SPARK_PACKAGE}.tgz" \
| gunzip \
| tar x -C /usr/ \
&& mv /usr/$SPARK_PACKAGE $SPARK_HOME \
&& chown -R root:root $SPARK_HOME
WORKDIR $SPARK_HOME
CMD ["bin/spark-class", "org.apache.spark.deploy.master.Master"]
Command:
ubuntu#ip-123.43.11.136:~$ sudo docker run -it --rm -v $(pwd):/home/ubuntu sparkimage /home/ubuntu bin/spark-submit ./count.py
Got Error below
Error :- Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/home/ubuntu\": permission denied": unknown.
Can some help me what could be the issue? I have gone through several links but no luck still not able to resolve the issue.
ERRO[0001] error waiting for the container: context cancelled
Everything passes after image sparkimage are considered as an argument to the Docker Entrypoint.
For example
Entrypoint ["node"]
so when you start
docker run -it my_image app.js
Now here app.js will be the argument for the node which will start app.js and docker will treat them like node app.js.
So you are passing invalid option to the image in your docker run command as there is no entrypoint in your Dockefile and command become
CMD ["/home/ubuntu bin/spark-submit ./count.py"]
That's is its throw error for /home/ubuntu permission denied.
You can try these two combinations.
Etnrypoint ["bin/spark-class", "org.apache.spark.deploy.master.Master"]
with the run command.
sudo docker run -it --rm -v $(pwd):/home/ubuntu sparkimage /home/ubuntu bin/spark-submit ./count.py
OR
CMD ["bin/spark-class", "org.apache.spark.deploy.master.Master","/home/ubuntu bin/spark-submit ./count.py"]
with docker run command
sudo docker run -it --rm -v $(pwd):/home/ubuntu sparkimage
The issue has been resolved. the corrected right mount path and executed and it was working fine without any issue.

Running GUI in docker (no ssh, no VNC)

TL;DR: root is not supposed to run GUI app, set a regular user to do so.
I'm trying to run arduino IDE (downloaded, not the package) from within a Docker. I wrote the Dockerfile as follow:
FROM ubuntu:14.04
MAINTAINER Mael Auzias <docker#mael.auzias.net>
ENV HOME /home/arduino
ENV USER arduino
RUN apt-get update && apt-get install -y \
libx11-6 libxext-dev libxrender-dev libxtst-dev \
--no-install-recommends \
&& useradd --create-home --home-dir $HOME $USER \
&& chown -R $USER:$USER $HOME
ADD arduino-1.6.6-linux64.tar.xz $HOME
WORKDIR $HOME/arduino-1.6.6
USER $USER
ENTRYPOINT ["/bin/bash"]
I spent time to understand how does Jessica Frazelle usually starts her graphical containers to rightly start mine with the command:
$docker run --name arduino --rm -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix 25af73b6cb3c ./arduino
No protocol specified
Picked up JAVA_TOOL_OPTIONS:
No protocol specified
Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
I installed strace and check with xeyes what was wrong, and I get the following error:
connect(3, {sa_family=AF_LOCAL, sun_path=#"/tmp/.X11-unix/X0"}, 20) = -1 ECONNREFUSED (Connection refused)
Did anyone experience this? Can any point me out some doc or see what I'm doing wrong?
Any help would be welcome.
PS: as specified in the title I do not want to use ssh or VNC. No cryptography should be used nor network when a unix socket is faster and enough.
Solution
Got some news...
As the user root I cannot start graphical application. When I su regular-user and start xterm or xeyes it works. I don't really understand why though :/
Here is the working Dockerfile, tested on Fedora 23.
The application must not be ran as root so it starts using X.
Note that, unrelated to this issue, a Java option has been removed from the bash file arduino (so it starts properly).
After a docker build -t arduino-1.6.6 ., docker run --name arduino --rm -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix arduino-1.6.6 ./arduino start the arduino IDE.
You will not be able to upload any code into an arduino without adding a --device or -v to share the /dev/ttyUSB0.
FROM ubuntu:14.04
MAINTAINER Mael Auzias <docker#mael.auzias.net>
ENV HOME /home/arduino
ENV USER arduino
RUN apt-get update && apt-get install -y \
libx11-6 libxext-dev libxrender-dev libxtst-dev \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --create-home --home-dir $HOME $USER \
&& chown -R $USER:$USER $HOME
ADD arduino-1.6.6-linux64.tar.xz $HOME
RUN sed -i 's/"-Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel"//g' /home/arduino/arduino-1.6.6/arduino
WORKDIR $HOME/arduino-1.6.6
USER $USER
ENTRYPOINT ["/bin/bash"]
Got some news...
As the user root I cannot start graphical application. When I su regular-user and start xterm or xeyes it works. I don't really understand why though :/
Here is the working Dockerfile, tested on Fedora 23.
The application must not be ran as root so it starts using X.
Note that, unrelated to this issue, a Java option has been removed from the bash file arduino (so it starts properly).
After a docker build -t arduino-1.6.6 ., docker run --name arduino --rm -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix arduino-1.6.6 ./arduino start the arduino IDE.
You will not be able to upload any code into an arduino without adding a --device or -v to share the /dev/ttyUSB0.
FROM ubuntu:14.04
MAINTAINER Mael Auzias <docker#mael.auzias.net>
ENV HOME /home/arduino
ENV USER arduino
RUN apt-get update && apt-get install -y \
libx11-6 libxext-dev libxrender-dev libxtst-dev \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --create-home --home-dir $HOME $USER \
&& chown -R $USER:$USER $HOME
ADD arduino-1.6.6-linux64.tar.xz $HOME
RUN sed -i 's/"-Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel"//g' /home/arduino/arduino-1.6.6/arduino
WORKDIR $HOME/arduino-1.6.6
USER $USER
ENTRYPOINT ["/bin/bash"]

Resources