I use php:7.3.2-cli-alpine3.9 as my base image. I also need SSHFS installed inside container since PHP library I use relies on it. I know there are many answers with "if you install SSHFS inside container you are doing it wrong" but in my case I need this software installed inside container not on host.
In my Dockerfile I have
RUN apk update && apk add sshfs;
This command is executed without errors:
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.3-21-g265a28802e [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.3-15-g583c0d55e9 [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9764 distinct packages available
(1/12) Installing openssh-keygen (7.9_p1-r4)
(2/12) Installing openssh-client (7.9_p1-r4)
(3/12) Installing fuse-common (3.2.6-r1)
(4/12) Installing fuse3 (3.2.6-r1)
(5/12) Installing libffi (3.2.1-r6)
(6/12) Installing libintl (0.19.8.1-r4)
(7/12) Installing libuuid (2.33-r0)
(8/12) Installing libblkid (2.33-r0)
(9/12) Installing libmount (2.33-r0)
(10/12) Installing pcre (8.42-r1)
(11/12) Installing glib (2.58.1-r2)
(12/12) Installing sshfs (3.5.1-r0)
Executing busybox-1.29.3-r10.trigger
Executing glib-2.58.1-r2.trigger
OK: 25 MiB in 44 packages
But when I am trying to mount remote host
'/usr/bin/sshfs' -C -o reconnect -o 'Port=22' -o 'UserKnownHostsFile=/ssh/known_hosts' -o StrictHostKeyChecking=yes -o 'IdentityFile=/ssh/<FILE>' -o PasswordAuthentication=no '<USER>#<HOST>:/' '/fuse/'
I am getting:
fuse: device not found, try 'modprobe fuse' first
When I am running it I am getting:
modprobe: can't change directory to '/lib/modules': No such file or directory
Am I missing any packages? What else needs to be installed?
Thanks.
In order to run SSHFS inside container it requires privileged permissions.
Install SSHFS by adding this line in Dockerfile:
RUN apk update && apk add sshfs;
Run container:
docker run --privileged=true -it --rm --name alpine-app transfers-image
Related
So I am running into an error with the latest docker build from alpine. alpine:3.14.0 was released about a day ago and was trying to install libressl and libressl-dev and both seem to fail with the error below. My work around at the moment was to build using the alpine:3.12.0 as 3.12.0 seems to not have libretls installed. Although I would like to know how to fix this. I tried to remove libretls but that didn't work (error also below). Thanks
$ docker -v
Docker version 20.10.6, build 370c289
$ docker run --rm -it alpine /bin/ash
/ # apk add libressl-dev
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/5) Installing libressl3.3-libcrypto (3.3.3-r0)
(2/5) Installing libressl3.3-libssl (3.3.3-r0)
(3/5) Installing libressl3.3-libtls (3.3.3-r0)
ERROR: libressl3.3-libtls-3.3.3-r0: trying to overwrite usr/lib/libtls.so.20 owned by libretls-3.3.3-r0.
ERROR: libressl3.3-libtls-3.3.3-r0: trying to overwrite usr/lib/libtls.so.20.0.3 owned by libretls-3.3.3-r0.
(4/5) Installing pkgconf (1.7.4-r0)
(5/5) Installing libressl-dev (3.3.3-r0)
Executing busybox-1.33.1-r2.trigger
1 error; 41 MiB in 19 packages
/ #
/ # apk info libretls
libretls-3.3.3-r0 description:
port of libtls from libressl to openssl
libretls-3.3.3-r0 webpage:
https://git.causal.agency/libretls/
libretls-3.3.3-r0 installed size:
84 KiB
/ #
/ # apk del libretls
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/main: No such file or directory
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/community: No such file or directory
World updated, but the following packages are not removed due to:
libretls: busybox alpine-baselayout apk-tools
OK: 6 MiB in 14 packages
/ # exit
this is the upstream issue. Downgrading to alpine 3.13 works for now until the issue is fixed.
Normally docker images support fixed alpine versions (thanks to MrGlass comment). For example
python:3-alpine -> python:3-alpine3.13
php:7.4-fpm-alpine -> php:7.4-fpm-alpine3.13
I'm trying to build a docker image that uses nvidia hardware decoding in gstreamer and have encountered a strange problem with making the image.
The build process does not find the nvidia cuda related stuff while running docker build (or nvidia-docker build), but when I spin up the failed image as a container and do those very same steps from within the container everything works. I even saved the container as image which gave me a persistent image that works as intended.
Has anyone experienced similar problem and can shed some light on it?
Dockerfile:
FROM nvcr.io/nvidia/deepstream:3.0-18.11 AS base
ENV DEBIAN_FRONTEND noninteractive
#install some dependencies. NOTE - not removing apt cache for the MWE
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
libdc1394-22 \
tmux \
vim \
libjpeg-dev \
libpng-dev \
libpng12-dev \
cuda-toolkit-10-0 \
python3-setuptools \
python3-pip ninja-build pkg-config gobject-introspection gnome-devel bison flex libgirepository1.0-dev liborc-0.4-dev
RUN pip3 install meson && ldconfig
FROM base
#pull and make gstreamer:
RUN cd /tmp && mkdir gstreamer
RUN git clone https://github.com/GStreamer/gst-build.git /tmp/gstreamer \
&& cd /tmp/gstreamer \
&& git checkout tags/1.16.0 \
&& ./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure \
&& ninja -C build \
&& ninja install -C build
Testing:
build and run the container. Inside the container:
$ gst-inspect-1.0 nvdec
No such element or plugin 'nvdec'
$ cd /tmp/gstreamer
$ ./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure
$ ninja -C build
$ ninja install -C build
$ gst-inspect-1.0 nvdec
Factory Details:
Rank primary (256)
[... all plugin parameters show up]
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstNvDec
EDIT1
The image builds with no errors, only when I try to call gstreamer it is built with no acceleration. I noticed that in the build process the major difference is
meson.build:109:2: Exception: Problem encountered: The nvdec plugin was enabled explicitly, but required CUDA dependencies were not found.
which does not happen when building from within the container.
Lack of error is related, most likely, to the ninja+meson build system which looks for compatible packages, reports the exception, but doesn't throw it and continues as if nothing wrong happened
EDIT2
Answering comment:
To build it and get the error, just build the attached docker image:
sudo docker build -t gst16:latest . > build.log
This will dump all the output into the build.log file.
I don't have a docker registry that I could use for this and the docker image gets quite big by docker standards (~8 Gigs), but to produce successfully, it's fairly simple:
sudo docker run --runtime="nvidia" -ti gst16:latest /bin/bash
or
sudo nvidia-docker run -ti gst16:latest /bin/bash
which seems to work the same for me. Notice no --rm flag! From within the container:
#check if nvidia decoder plugin is there:
gst-inspect-1.0 nvdec
#fail!
#now build it from within:
cd /opt/gstreamer
./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure
ninja -C build
ninja install -C build
gst-inspect-1.0 nvdec
#success reported
Now to get the image, exit the container (ctrl+d) and in the host shell:
sudo docker container ls -a to view all containers including stopped ones
from gst16:latest get the CONTAINER_ID and copy it
sudo docker commit <CONTAINER_ID> gst16:manual and after a few seconds you should have the container saved as an image. Verify with sudo docker images
run the new image with sudo docker run --runtime=`nvidia` --rm -ti gst16:manual /bin/bash
from within the container try again the gst-inspect-1.0 nvdec to verify it's working
EDIT3
$ nvidia-docker --version
Docker version 18.09.0, build 4d60db4
I think I found the solution/reason
Writing it here in case someone finds themselves in similar situation, plus I hate finding old threads with similar problem and no answer or "nevermind, I solved it" as the only follow up
Docker build does not have any ties to nvidia runtime and gstreamer requires access to the full nvidia toolchain in order to build the plugins that need it. This is to be resolved with gstreamer 1.18 but until then, there is no way to build gstreamer with nvidia codecs in docker build.
The workaround:
Build image with all dependencies.
Run a container of said image using runtime="nvidia" but don't use --rm flag
In the container, build gstreamer and install it as normally.
Verify with gst-inspect-1.0
Commit the container as new image: docker commit <container_name> <temporary_image_name>
Tag the temporary image properly.
I'm trying to deploy local image in minikube.
This is what I'm doing:
minikube start
eval $(minikube docker-env)
Trying to build image in minikube (build image -t my-image .), but it gets stuck at 2. fetching packages... line
I don't have any HTTP_PROXY, HTTPS_PROXY, NO_PROXY set. Anybody had such an issue?
Edit. This is my Dockerfile. It works fine. I can build, push, pull from remote docker registry. It only doesn't work, when I build image in minikube.
FROM node:10.16.2-alpine
WORKDIR /app/server
ADD package.json .
RUN apk --no-cache add --virtual .build build-base python && \
yarn install && \
npm rebuild bcrypt --build-from-source && \
yarn cache clean && \
apk del .build
ADD . .
EXPOSE 3000
CMD [ "yarn", "start" ]
This is docker build log in minikube:
Sending build context to Docker daemon 292MB
Step 1/7 : FROM node:10.16.2-alpine
---> 4f877c96a193
Step 2/7 : WORKDIR /app/server
---> Using cache
---> 71cd2676c364
Step 3/7 : ADD package.json .
---> Using cache
---> af5f79d46abf
Step 4/7 : RUN apk --no-cache add --virtual .build build-base python && yarn install && npm rebuild bcrypt --build-from-source && yarn cache clean && apk del .build
---> Running in c38695ed65b7
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/27) Installing binutils (2.31.1-r2)
(2/27) Installing libmagic (5.36-r0)
(3/27) Installing file (5.36-r0)
(4/27) Installing gmp (6.1.2-r1)
(5/27) Installing isl (0.18-r0)
(6/27) Installing libgomp (8.3.0-r0)
(7/27) Installing libatomic (8.3.0-r0)
(8/27) Installing mpfr3 (3.1.5-r1)
(9/27) Installing mpc1 (1.0.3-r1)
(10/27) Installing gcc (8.3.0-r0)
(11/27) Installing musl-dev (1.1.20-r5)
(12/27) Installing libc-dev (0.7.1-r0)
(13/27) Installing g++ (8.3.0-r0)
(14/27) Installing make (4.2.1-r2)
(15/27) Installing fortify-headers (1.0-r0)
(16/27) Installing build-base (0.5-r1)
(17/27) Installing libbz2 (1.0.6-r7)
(18/27) Installing expat (2.2.7-r0)
(19/27) Installing libffi (3.2.1-r6)
(20/27) Installing gdbm (1.13-r1)
(21/27) Installing ncurses-terminfo-base (6.1_p20190105-r0)
(22/27) Installing ncurses-terminfo (6.1_p20190105-r0)
(23/27) Installing ncurses-libs (6.1_p20190105-r0)
(24/27) Installing readline (7.0.003-r1)
(25/27) Installing sqlite-libs (3.28.0-r0)
(26/27) Installing python2 (2.7.16-r1)
(27/27) Installing .build (0)
Executing busybox-1.29.3-r10.trigger
OK: 211 MiB in 43 packages
yarn install v1.17.3
info No lockfile found.
[1/4] Resolving packages...
warning bugsnag#2.4.3: All projects should upgrade to our universal JS notifier: "#bugsnag/js". See https://github.com/bugsnag/bugsnag-js/blob/master/UPGRADING.md for more details.
warning bugsnag > cuid > core-js#1.2.7: core-js#<2.6.8 is no longer maintained. Please, upgrade to core-js#3 or at least to actual version of core-js#2.
warning node-mailer#0.1.1: node-mailer is not maintained
warning jest > jest-cli > prompts > kleur#2.0.2: Please upgrade to kleur#3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
warning jest > jest-cli > jest-environment-jsdom > jsdom > left-pad#1.3.0: use String.prototype.padStart()
warning nest-cli > exists-sync#0.0.3: Please replace with usage of fs.existsSync
warning supertest > superagent#3.8.3: Please note that v5.0.1+ of superagent removes User-Agent header by default, therefore you may need to add it yourself (e.g. GitHub blocks requests without a User-Agent header). This notice will go away with v5.0.2+ once it is released.
[2/4] Fetching packages...
info There appears to be trouble with your network connection. Retrying...
You can try to increase --network timeout parameter, adapting it in yarn command line within the shared Dockerfile as #Marc ABOUCHACRA mentioned in the comment:
yarn install --network-timeout 1000000
The issue seems to be very common and not related to minikube VM, just search for the details in this Stack thread.
I have a problem to use the docker rstudio-image rocker/rstudio proposed
on https://www.rocker-project.org/ (docker containers for R). Since I am a beginner with both docker and RStudio, I suspect the problem comes from me and does not deserve a bug report:
I open a proper terminal with 'Docker Quickstart Terminal'
where I run the image with docker run -d -p 8787:8787 -e DISABLE_AUTH=true -v <...>:/home/rstudio/<...> --name rstudio rocker/rstudio
in my browser I then get a nice RStudio instance at the address http://192.168.99.100:8787
but in this instance I can't install several packages such as xml2. I get the message:
Using PKG_CFLAGS=
Using PKG_LIBS=-lxml2
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libxml-2.0 was not found. Try installing:
* deb: libxml2-dev (Debian, Ubuntu, etc)
* rpm: libxml2-devel (Fedora, CentOS, RHEL)
* csw: libxml2_dev (Solaris)
If libxml-2.0 is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘xml2’
* removing ‘/usr/local/lib/R/site-library/xml2’
Warning in install.packages :
installation of package ‘xml2’ had non-zero exit status
I don't know whether xml2 is on the image but the file libxml-2.0.pc does exist on my laptop in the directory /opt/local/lib/pkgconfig and pkg-config is in /opt/local/bin. So I tried linking these pkg paths when running
the image (to see what happen when I play with the image environment
in RStudio), adding options -v
/opt/local/lib/pkgconfig:/home/rstudio/lib/pkgconfig -v
/opt/local/bin:/home/rstudio/bin to the run command. But it doesn't work: for some reason
I don't see the content of lib/pkgconfig in RStudio...
Also the RStudio instance does not accept root/sudo commands so I can't
use tools such as apt-get in the RStudio terminal
so, what's the trick ?
Libraries on your laptop (the host for docker) are not available for docker containers. You should create a custom image with required libraries, create a Dockerfile like this:
FROM rocker/rstudio
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
libxml2-dev # add any additional libraries you need
CMD ["/init"]
Above I added the libxml2-dev but you can add as many libraries as you need.
Then build your image using this command (you need to execute below command in directory there you created Dockerfile):
docker build -t my_rstudio:0.1 .
Then you can start your container:
docker run -d -p 8787:8787 -e DISABLE_AUTH=true --name rstudio my_rstudio:0.1
(you can add any additional arguments like -v to above).
I have a project running on Docker with docker-compose for dev environment.
I want to get it running on GitLabCI with a gitlab-ci-multi-runner "Docker mode" instance.
Here is my .gitlab-ci.yml file:
image: soullivaneuh/docker-bash
before_script:
- apk add --update bash curl
- curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
- chmod +x /usr/local/bin/docker-compose
- ./configure
- docker-compose up -d
Note that soullivaneuh/docker-bash image is just a docker image with bash installed.
The script fails on docker-compose up -d command:
gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image soullivaneuh/docker-bash ...
Pulling docker image soullivaneuh/docker-bash:latest ...
Running on runner-1ee5079f-project-3-concurrent-1 via sd-59984...
Fetching changes...
Removing app/config/parameters.yml
Removing docker-compose.env
HEAD is now at 5c5e7ff remove docker service
From https://git.dummy.net/project/project
5c5e7ff..45e643d docker-ci -> origin/docker-ci
Checking out 45e643dd as docker-ci...
Previous HEAD position was 5c5e7ff... remove docker service
HEAD is now at 45e643d... Remove docker info commands
$ apk add --update bash curl
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
OK: 10 MiB in 28 packages
$ curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
$ ./configure
$ docker-compose up -d
bash: line 30: /usr/local/bin/docker-compose: No such file or directory
ERROR: Build failed with: exit code 1
I have absolutly no idea why this is failing.
Thanks for help.
The No such file or directory is misleading. I've received that many times while trying to run dynamically linked binaries using alpine linux (which it appears you are using).
The problem (as I understand it) is that the binary was compiled and linked against glibc, but alpine uses musl, not glibc.
You could use ldd /usr/local/bin/docker-compose to tell you which libraries are missing (or run it with strace if all else fails).
To get it working, it might be easier to install from python source (https://docs.docker.com/compose/install/#install-using-pip), which is what the official compose image does (https://github.com/docker/compose/blob/master/Dockerfile.run).
Or you could use an image built on debian or some other distro that uses glibc.