All new to docker, i'm trying to build an image of a software, during the
RUN apt-get install -y xxx command i'm encourtering issues :
Setting up lvm2 (2.02.95-8) ...
Setting up LVM Volume Groups... /dev/mapper/control: open failed:Operation not permitted
Failure to communicate with kernel device-mapper driver.
Check that device-mapper is available in the kernel.
No volume groups found
/dev/mapper/control: open failed: Operation not permitted
Failure to communicate with kernel device-mapper driver.
Check that device-mapper is available in the kernel.
No volume groups found
what could cause this issue ?
my distrib is a Debian7, maybe should i try this on a more recent distros ?
here is the Dockerfile :
#installation d'hynesim
FROM debian:wheezy
RUN echo $(whoami)
RUN echo "exit 0" > /usr/sbin/policy-rc.d
RUN apt-get update && apt-get install -y curl
RUN echo 'deb [arch=amd64] http://repository.hynesim.org/debian wheezy 2.2 backports' >> /etc/apt/sources.list && \
echo 'deb-src [arch=amd64] http://repository.hynesim.org/debian wheezy 2.2 backports' >> /etc/apt/sources.list
RUN curl -o - https://repository.hynesim.org/debian/hynesim.asc | apt-key add - && apt-get update && apt-get install -y \
hynesim-node \
hynesim-glacier
Related
I got a K8S+DinD issue:
launch Kubernetes cluster
start a main docker image and a DinD image inside this cluster
when running a job requesting GPU, got error could not select device driver "nvidia" with capabilities: [[gpu]]
Full error
http://localhost:2375/v1.40/containers/long-hash-string/start: Internal Server Error ("could not select device driver "nvidia" with capabilities: [[gpu]]")
exec to the DinD image inside of K8S pod, nvidia-smi is not available.
Some debugging and it seems it's due to the DinD is missing the Nvidia-docker-toolkit, I had the same error when I ran the same job directly on my local laptop docker, I fixed the same error by installing nvidia-docker2 sudo apt-get install -y nvidia-docker2.
I'm thinking maybe I can try to install nvidia-docker2 to the DinD 19.03 (docker:19.03-dind), but not sure how to do it? By multiple stage docker build?
Thank you very much!
update:
pod spec:
spec:
containers:
- name: dind-daemon
image: docker:19.03-dind
I got it working myself.
Referring to
https://github.com/NVIDIA/nvidia-docker/issues/375
https://github.com/Henderake/dind-nvidia-docker
First, I modified the ubuntu-dind image (https://github.com/billyteves/ubuntu-dind) to install nvidia-docker (i.e. added the instructions in the nvidia-docker site to the Dockerfile) and changed it to be based on nvidia/cuda:9.2-runtime-ubuntu16.04.
Then I created a pod with two containers, a frontend ubuntu container and the a privileged docker daemon container as a sidecar. The sidecar's image is the modified one I mentioned above.
But since this post is 3 year ago from now, I did spent quite some time to match up the dependencies versions, repo migration over 3 years, etc.
My modified version of Dockerfile to build it
ARG CUDA_IMAGE=nvidia/cuda:11.0.3-runtime-ubuntu20.04
FROM ${CUDA_IMAGE}
ARG DOCKER_CE_VERSION=5:18.09.1~3-0~ubuntu-xenial
RUN apt-get update -q && \
apt-get install -yq \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable" && \
apt-get update -q && apt-get install -yq docker-ce docker-ce-cli containerd.io
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN set -eux; \
apt-get update -q && \
apt-get install -yq \
btrfs-progs \
e2fsprogs \
iptables \
xfsprogs \
xz-utils \
# pigz: https://github.com/moby/moby/pull/35697 (faster gzip implementation)
pigz \
# zfs \
wget
# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -x \
&& addgroup --system dockremap \
&& adduser --system -ingroup dockremap dockremap \
&& echo 'dockremap:165536:65536' >> /etc/subuid \
&& echo 'dockremap:165536:65536' >> /etc/subgid
# https://github.com/docker/docker/tree/master/hack/dind
ENV DIND_COMMIT 37498f009d8bf25fbb6199e8ccd34bed84f2874b
RUN set -eux; \
wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
chmod +x /usr/local/bin/dind
##### Install nvidia docker #####
# Add the package repositories
RUN curl -fsSL https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add --no-tty -
RUN distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && \
echo $distribution && \
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
tee /etc/apt/sources.list.d/nvidia-docker.list
RUN apt-get update -qq --fix-missing
RUN apt-get install -yq nvidia-docker2
RUN sed -i '2i \ \ \ \ "default-runtime": "nvidia",' /etc/docker/daemon.json
RUN mkdir -p /usr/local/bin/
COPY dockerd-entrypoint.sh /usr/local/bin/
RUN chmod 777 /usr/local/bin/dockerd-entrypoint.sh
RUN ln -s /usr/local/bin/dockerd-entrypoint.sh /
VOLUME /var/lib/docker
EXPOSE 2375
ENTRYPOINT ["dockerd-entrypoint.sh"]
#ENTRYPOINT ["/bin/sh", "/shared/dockerd-entrypoint.sh"]
CMD []
When I use exec to login into the Docker-in-Docker container, I can successfully run nvidia-smi (which previously return not found error then cannot run any GPU resource related docker run)
Welcome to pull my image at brandsight/dind:nvidia-docker
I'm running a command that worked until yesterday, and works locally on my local Docker, to install gcsfuse version 0.28.1:
E: Version '0.28.1' for 'gcsfuse' was not found
I tried it on the google cloud console too, and got the same error there.
Any suggestions or pointers?
Here's the original command:
export GCSFUSE_REPO=gcsfuse-lsb_release -c -s
&& echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
&& apt-get update && apt-get install -y gcsfuse=0.28.1 \
I tried the below commands on Google Cloud Shell as per this document reference and I was able to install GCSFuse successfully.
Add the gcsfuse distribution URL as a package source and import its public key using :
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Update the list of packages available and install gcsfuse using :
sudo apt-get update
sudo apt-get install gcsfuse
(Ubuntu before wily only) Add yourself to the fuse group, then log out and back in using :
sudo usermod -a -G fuse $USER
exit
I also found a line in this document which says “The following instructions set up apt-get to see updates to gcsfuse, and are supported for the bionic, artful, zesty, yakkety, xenial, and trusty releases of Ubuntu, and the jessie and stretch releases of Debian. (Run lsb_release -c to find your release codename.) Users of older releases should follow the instructions for other distributions below.
So you should also try and find the release codename first. If you are a user of an older release then these commands might not work for you. For that, follow the instructions/commands of installing older distributions, which is clearly differentiated and specified in this document link
I am not sure if it was your formatting but it looks like it was missing the release variable in your commands. Please try with the commands below:
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s` && \
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update -y && \
apt-get install -y gcsfuse=0.28.1 -V
If it does not work, here is the repo with all gcsfuse releases:
https://github.com/GoogleCloudPlatform/gcsfuse/releases/tag/v0.28.1
I have a docker file that creates a valid image that runs on my Ubuntu 18.04.
For compatibility with other machines, I've tried to run the docker in a Virtual Box Ubuntu machine (and avoid any configuration errors that may occur).
my docker run command line:
docker run -id --net=host --rm --privileged --gpus=all --env="NVIDIA_DRIVER_CAPABILITIES=all" --env="DISPLAY" -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix:rw -v /run/user/1000/gdm/Xauthority:/root/.Xauthority --env="QT_X11_NO_MITSHM=1" --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -v /home/git/:/git --name nirge_sim nirge-sim:1.0
The base docker file:
FROM gazebo:gzserver9-bionic
# nvidia-container-runtime
ENV NVIDIA_VISIBLE_DEVICES \
${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES \
${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics
# install Utilities
RUN apt-get update -y && apt-get install -y apt-utils curl ca-certificates wget \
&& rm -rf /var/lib/apt/lists/*
# install gazebo packages
RUN apt-get update -y && apt-get install -y --allow-unauthenticated --no-install-recommends \
libgazebo9-dev \
&& rm -rf /var/lib/apt/lists/*
# install ros packages
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
RUN curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | apt-key add -
RUN apt-get update && apt-get install -y --allow-unauthenticated \
ros-melodic-desktop-full \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --allow-unauthenticated --no-install-recommends \
ros-melodic-gazebo-ros-pkgs ros-melodic-gazebo-ros-control \
ros-melodic-gazebo-plugins ros-melodic-gazebo-ros ros-melodic-gazebo-ros\
ros-melodic-simulators \
&& rm -rf /var/lib/apt/lists/*
# final config for ros
RUN echo 'source /opt/ros/melodic/setup.bash' >> /root/.bashrc
RUN echo 'export LIBGL_ALWAYS_INDIRECT=1' >> /root/.bashrc
CMD ["bash"]
So this works on my Host, but not on my hosted host via virtual box.
the error is:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: nvml error: driver not loaded: unknown.
Would appreciate any advice on this issue.
It appears that one of the dependencies (Gazebo) requires a dedicated GPU, one that is not simulated as a part of VirtualBox.
Nvidia cards tend to work well in Ubuntu
sourced from original site
I am attempting to fuse a dir in a Docker image using gcsfuse. I am using a Debian stretch image, and having trouble working with the fuse package.
I have attempted to install fuse both via apt-get as well as build from the source via the git repo. Both have had their respective problems.
1: After apt-get I receive indication that fuse was successfully installed.
root#a7d6f712fab9:/queue# apt-get install fuse
Reading package lists... Done
Building dependency tree
Reading state information... Done
fuse is already the newest version (2.9.7-1+deb9u2).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
root#a7d6f712fab9:/queue# apt-get install libfuse-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libfuse-dev is already the newest version (2.9.7-1+deb9u2)
However when running modprobe fuse (what fails during the gcsfuse mount attempt):
root#a7d6f712fab9:/queue# modprobe fuse
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.9.125-linuxkit/modules.dep.bin'
modprobe: FATAL: Module fuse not found in directory /lib/modules/4.9.125-linuxkit
2: When using the tar.gz from source, meson is only available as version 0.37, whereas libfuse requires meson > 0.38 to build properly (from earlier versions).
Here's my Dockerfile:
FROM python:3.6-slim
RUN apt-get update \
&& apt-get install -y libfuse-dev \
curl \
gnupg \
apt-utils \
lsb-release \
kmod
RUN export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s` \
&& echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list \
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
RUN apt-get update \
&& apt-get install -y gcsfuse
COPY . /queue
WORKDIR /queue
I'd like modprobe fuse to actually work, or to understand how I can build fuse/modprobe in a way where the package is identified via modprobe.
Thanks!
Docker containers use the host kernel. So if a kernel module needs loading you have to load it on the host machine and not in Docker.
elastic search is successfully running on docker container. but i'm not able access in browser. i mapped ports correctly. but the problem is in docker container. in container elasticsearch is mapped with localhost
127.0.0.1:9200
Dokcerfile
FROM ubuntu:16.04
MAINTAINER Rajesh Gurram
RUN apt-get update && \
apt-get install -y net-tools curl wget gnupg
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:webupd8team/java && \
apt-get update && \
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get install -y oracle-java8-installer && apt-get clean
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
RUN apt-get install apt-transport-https
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add - && \
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list && \
apt update && apt install -y elasticsearch
RUN sed -i 's/#network.host: 192.168.0.1/network.host: 0.0.0.0/g' /etc/elasticsearch/elasticsearch.yml
EXPOSE 9200 9300
Run Below command on Host machine it will resolve the issue
$ sysctl -w vm.max_map_count=262144
If you want to use docker to get an instance of Elasticsearch, you can read the following Guide:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
You can also use docker images directly from elastic, if ubuntu is not a necessary base image:
https://www.docker.elastic.co/
If you want to upgrade to an ELK Stack later on, I recommend a docker volume for persistency purposes.