I have managed to install github cli in container but I am unable to run it as a random user (open shift does runs containers as random user due to security)
What directories should I create/chmod/chown beforehand to make it work?
THANK YOU!
docker file:
RUN wget https://github.com/cli/cli/releases/download/v1.0.0/gh_1.0.0_linux_amd64.deb
RUN dpkg -i gh_*_linux_amd64.deb
RUN chmod g+rwx -R /usr/local/bin/gh # <- not enough
RUN gh version
container log:
...
Step 7/28 : RUN gh version
---> Running in 09d1ffbfcc02
Welcome to GitHub CLI!
To authenticate, please run `gh auth login`.
You can also set the GITHUB_TOKEN environment variable, if preferred.
The command '/bin/sh -c gh version' returned a non-zero code: 4
`/` is not writable.
...
there is a bug in the docs, it is
gh --version
not
gh version
also RUN chmod g+rwx -R /usr/local/bin/ before mentioned command helps.
Related
EDIT: documentation given by the informatic administration was shitty, old version of singularity, now the order of arguments is different and the problem is solved.
To make my tool more portable, and because I have to use it on a cluster, I have to put my bioinformatics tool at disposal for docker. Tool is located here. The docker hub is 007ptar007/metadbgwas, if you want to experience with it. The Dockerfile is in the repo, and to make it easier to everyone :
FROM ubuntu:latest
ENV DEBIAN_FRONTEND=noninteractive
USER root
COPY ./install_docker.sh ./
RUN chmod +x ./install_docker.sh && sh ./install_docker.sh
ENTRYPOINT ["/MetaDBGWAS/metadbgwas.sh"]
ENV PATH="/MetaDBGWAS/:${PATH}"
And the install_docker.sh script contains :
apt-get update
apt install -y libgatbcore-dev libhdf5-dev libboost-all-dev libpstreams-dev zlib1g-dev g++ cmake git r-base-core
Rscript -e "install.packages(c('ape', 'phangorn'))"
Rscript -e "install.packages('https://raw.githubusercontent.com/sgearle/bugwas/master/build/bugwas_1.0.tar.gz', repos=NULL, type='source')"
git clone --recursive https://github.com/Louis-MG/MetaDBGWAS.git
cd MetaDBGWAS
sed -i "51i#include <limits>" ./REINDEER/blight/robin_hood.h #temporary fix for REINDEER compilation
sh install.sh
The problem :
My tool parses the command line, and needs a verbose (-v, or --verbose) argument. It also needs to reject unknown arguments; anything that isn't used by the tool causes the help message to be printed in the standard output and exits. To use the tool, I need to mount volumes were the data is; using -v /path/to/files:/input option:
singularity run docker://007ptar007/metadbgwas --volumes '/path/to/data:/inputd/:/input' --files /input --strains /input/strains --threads 8 --output ~/output
But my tool sees this as a bad -v option value or the --volume as an unknown option. I can't change this on my tool. How do I solve this conflict ?
You need to put any arguments intended for singularity - such as the volume mounting - before the name of the image you want to run (e.g. the docker image you specify in your command):
singularity run -v '/path/to/data:/input' docker://007ptar007/metadbgwas --files /input --strains /input/strains --threads 8 --output ~/output
I'm trying to build a docker image that uses nvidia hardware decoding in gstreamer and have encountered a strange problem with making the image.
The build process does not find the nvidia cuda related stuff while running docker build (or nvidia-docker build), but when I spin up the failed image as a container and do those very same steps from within the container everything works. I even saved the container as image which gave me a persistent image that works as intended.
Has anyone experienced similar problem and can shed some light on it?
Dockerfile:
FROM nvcr.io/nvidia/deepstream:3.0-18.11 AS base
ENV DEBIAN_FRONTEND noninteractive
#install some dependencies. NOTE - not removing apt cache for the MWE
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
libdc1394-22 \
tmux \
vim \
libjpeg-dev \
libpng-dev \
libpng12-dev \
cuda-toolkit-10-0 \
python3-setuptools \
python3-pip ninja-build pkg-config gobject-introspection gnome-devel bison flex libgirepository1.0-dev liborc-0.4-dev
RUN pip3 install meson && ldconfig
FROM base
#pull and make gstreamer:
RUN cd /tmp && mkdir gstreamer
RUN git clone https://github.com/GStreamer/gst-build.git /tmp/gstreamer \
&& cd /tmp/gstreamer \
&& git checkout tags/1.16.0 \
&& ./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure \
&& ninja -C build \
&& ninja install -C build
Testing:
build and run the container. Inside the container:
$ gst-inspect-1.0 nvdec
No such element or plugin 'nvdec'
$ cd /tmp/gstreamer
$ ./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure
$ ninja -C build
$ ninja install -C build
$ gst-inspect-1.0 nvdec
Factory Details:
Rank primary (256)
[... all plugin parameters show up]
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstNvDec
EDIT1
The image builds with no errors, only when I try to call gstreamer it is built with no acceleration. I noticed that in the build process the major difference is
meson.build:109:2: Exception: Problem encountered: The nvdec plugin was enabled explicitly, but required CUDA dependencies were not found.
which does not happen when building from within the container.
Lack of error is related, most likely, to the ninja+meson build system which looks for compatible packages, reports the exception, but doesn't throw it and continues as if nothing wrong happened
EDIT2
Answering comment:
To build it and get the error, just build the attached docker image:
sudo docker build -t gst16:latest . > build.log
This will dump all the output into the build.log file.
I don't have a docker registry that I could use for this and the docker image gets quite big by docker standards (~8 Gigs), but to produce successfully, it's fairly simple:
sudo docker run --runtime="nvidia" -ti gst16:latest /bin/bash
or
sudo nvidia-docker run -ti gst16:latest /bin/bash
which seems to work the same for me. Notice no --rm flag! From within the container:
#check if nvidia decoder plugin is there:
gst-inspect-1.0 nvdec
#fail!
#now build it from within:
cd /opt/gstreamer
./setup.py -Dgtk_doc=disabled -Dgst-plugins-bad:nvdec=enabled -Dgst-plugins-bad:nvenc=enabled -Dgst-plugins-bad:iqa=disabled -Dgst-plugins-bad:bluez=disabled --reconfigure
ninja -C build
ninja install -C build
gst-inspect-1.0 nvdec
#success reported
Now to get the image, exit the container (ctrl+d) and in the host shell:
sudo docker container ls -a to view all containers including stopped ones
from gst16:latest get the CONTAINER_ID and copy it
sudo docker commit <CONTAINER_ID> gst16:manual and after a few seconds you should have the container saved as an image. Verify with sudo docker images
run the new image with sudo docker run --runtime=`nvidia` --rm -ti gst16:manual /bin/bash
from within the container try again the gst-inspect-1.0 nvdec to verify it's working
EDIT3
$ nvidia-docker --version
Docker version 18.09.0, build 4d60db4
I think I found the solution/reason
Writing it here in case someone finds themselves in similar situation, plus I hate finding old threads with similar problem and no answer or "nevermind, I solved it" as the only follow up
Docker build does not have any ties to nvidia runtime and gstreamer requires access to the full nvidia toolchain in order to build the plugins that need it. This is to be resolved with gstreamer 1.18 but until then, there is no way to build gstreamer with nvidia codecs in docker build.
The workaround:
Build image with all dependencies.
Run a container of said image using runtime="nvidia" but don't use --rm flag
In the container, build gstreamer and install it as normally.
Verify with gst-inspect-1.0
Commit the container as new image: docker commit <container_name> <temporary_image_name>
Tag the temporary image properly.
I am trying to set up a Docker image (my Dockerfile is available here, sorry for the french README: https://framagit.org/Gwendal/firefox-icedtea-docker) with an old version of Firefox and an old version of Java to run an old Java applet to start a VPN. My image does work and successfully allows me to start the Java applet in Firefox.
Unfortunately, the said applet then tries to run the following command in the container (I've simply removed the --config part from the command as it does not matter here):
INFO: launching '/usr/bin/pkexec sh -c /usr/sbin/openvpn --config ...'
Then the applet exits silently with an error. While investigating, I've tried running a command with pkexec with the same Docker image, and it gives me this result:
$ sudo docker-compose run firefox pkexec /firefox/firefox-sdk/bin/firefox-bin -new-instance
**
ERROR:pkexec.c:719:main: assertion failed: (polkit_unix_process_get_start_time (POLKIT_UNIX_PROCESS (subject)) > 0)
But I don't know polkit at all and cannot understand this error.
EDIT: A more minimal way to reproduce the problem is with this Dockerfile:
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get install -y policykit-1
And then run:
$ sudo docker build -t pkexec-test .
$ sudo docker run pkexec-test pkexec echo Hello
Which leads here again to:
ERROR:pkexec.c:719:main: assertion failed: (polkit_unix_process_get_start_time (POLKIT_UNIX_PROCESS (subject)) > 0)
Should I conclude that pkexec cannot work in a docker container? Or is there any way to make this command work?
Sidenote: I have no control whatsoever on the Java applet that I try to run, it is a horrible and very dated proprietary black box that I am supposed to use at work, for which I have no access to the source code, and that I must use as is.
I have solved my own problem by replacing pkexec by sudo in the docker image, and by allowing passwordless sudo.
Given an ubuntu docker image where a user called developer was created and configured with a USER statement, add these lines:
# Install sudo and make 'developer' a passwordless sudoer
RUN apt-get install sudo
ADD ./developersudo /etc/sudoers.d/developersudo
# Replacing pkexec by sudo
RUN rm /usr/bin/pkexec
RUN ln -s /usr/bin/sudo /usr/bin/pkexec
with the file developersudo containing:
developer ALL=(ALL) NOPASSWD:ALL
This replaces any call to pkexec made in a process running in the container, by a call to sudo without any password prompt, which works nicely.
I have a project running on Docker with docker-compose for dev environment.
I want to get it running on GitLabCI with a gitlab-ci-multi-runner "Docker mode" instance.
Here is my .gitlab-ci.yml file:
image: soullivaneuh/docker-bash
before_script:
- apk add --update bash curl
- curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
- chmod +x /usr/local/bin/docker-compose
- ./configure
- docker-compose up -d
Note that soullivaneuh/docker-bash image is just a docker image with bash installed.
The script fails on docker-compose up -d command:
gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image soullivaneuh/docker-bash ...
Pulling docker image soullivaneuh/docker-bash:latest ...
Running on runner-1ee5079f-project-3-concurrent-1 via sd-59984...
Fetching changes...
Removing app/config/parameters.yml
Removing docker-compose.env
HEAD is now at 5c5e7ff remove docker service
From https://git.dummy.net/project/project
5c5e7ff..45e643d docker-ci -> origin/docker-ci
Checking out 45e643dd as docker-ci...
Previous HEAD position was 5c5e7ff... remove docker service
HEAD is now at 45e643d... Remove docker info commands
$ apk add --update bash curl
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
OK: 10 MiB in 28 packages
$ curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
$ ./configure
$ docker-compose up -d
bash: line 30: /usr/local/bin/docker-compose: No such file or directory
ERROR: Build failed with: exit code 1
I have absolutly no idea why this is failing.
Thanks for help.
The No such file or directory is misleading. I've received that many times while trying to run dynamically linked binaries using alpine linux (which it appears you are using).
The problem (as I understand it) is that the binary was compiled and linked against glibc, but alpine uses musl, not glibc.
You could use ldd /usr/local/bin/docker-compose to tell you which libraries are missing (or run it with strace if all else fails).
To get it working, it might be easier to install from python source (https://docs.docker.com/compose/install/#install-using-pip), which is what the official compose image does (https://github.com/docker/compose/blob/master/Dockerfile.run).
Or you could use an image built on debian or some other distro that uses glibc.
I'm using Docker and Arch Linux inside the Docker-Container.
Introducing makepkg-4.2.0 my Installation Command with yaourt were broken like described here: https://github.com/archlinuxfr/yaourt/issues/67
The Problem is, that yaourt should be run as non-root user. But as yaourt wants also to install the Package in every Case, after it has built it, root user is needed or a user that has the Permission to install Packages.
So my Question ist how to solve this? I want to install a Package from AUR inside the Docker, because no official Package exists yet.
Until now i was using Arch Linux, pacman and yaourt.
So the Command,
RUN yaourt -S --noconfirm aur/taskd
that installs taskd, worked before makepkg-4.2.0:
With the new makepkg Version building the Docker fails with the following Error from yaourt:
makepkg: invalid option '--asroot'
If i change the user to a non-root User and try to install the Package i get a Command prompt in my automated build asking for the Root-Password for actually installing the Package.
Password: su: Authentication information cannot be recovered
Password: su: Authentication information cannot be recovered
Password: su: Authentication information cannot be recovered
The command [/bin/sh -c yaourt -S --noconfirm aur/taskd] returned a non-zero code: 1
Without polluting to many offtopic Lines spread over two Dockerfiles, the interesting Portion of the Dockerfile looks like:
FROM kaot/arch_linux_base:latest
MAINTAINER Kaot
RUN useradd --no-create-home --shell=/bin/false yaourt && usermod -L yaourt
RUN yaourt -S --noconfirm aur/taskd
ENTRYPOINT ["/controlcenter/controlcenter.sh"]
CMD ["cc:start"]
If found a Solution that let yaourt only download the Info how to build the Package, then invoke makepkg itself, both with an non-root User and afterwards install the build Package with the root User and pacman.
The Portion of the Dockerfile looks like this
RUN mkdir -p /tmp/Package/ && chown yaourt /tmp/Package
USER yaourt
RUN cd /tmp/Package && pwd && ls -al && yaourt --getpkgbuild aur/taskd && cd taskd && makepkg --pkg taskd
USER root
RUN pacman -U --noconfirm /tmp/Package/taskd/taskd-1.1.0-1-x86_64.pkg.tar.xz
With some Variables, further Enhancements could be achieved, but in Principle this works.