Questions during setup in Docker build freeze the build process - docker

Issue
I am trying to install some drivers on a Docker image. During the procedure, one of the installation asks me for some input from command line. This obviously freezes the procedure. How can I solve this?
How to reproduce the issue
Start a shell in the base image: docker run -it nvcr.io/nvidia/deepstream:6.0-triton
Run the following commands:
export DEBIAN_FRONTEND noninteractive
# Set some variables to download the proper header modules
export VERSION="2.83.18%2Brev1.dev"
export BALENA_MACHINE_NAME="genericx86-64-ext"
# Set variables for the Yocto version of the OS
export YOCTO_VERSION=5.10.43
export YOCTO_KERNEL=${YOCTO_VERSION}-yocto-standard
# Set variables to download proper NVIDIA driver
export NVIDIA_DRIVER_VERSION=470.86
export NVIDIA_DRIVER=NVIDIA-Linux-x86_64-${NVIDIA_DRIVER_VERSION}
# Install some prereqs
apt install -y git wget unzip build-essential libelf-dev bc libssl-dev bison flex software-properties-common
mkdir -p /usr/src/kernel_source
cd /usr/src/kernel_source
# Causes a pipeline to produce a failure return code if any command errors.
# Normally, pipelines only return a failure if the last command errors.
#SHELL ["/bin/bash", "-o", "pipefail", "-c"]
# Download the kernel source then prepare kernel source to build a module.
curl -fsSL "https://files.balena-cloud.com/images/${BALENA_MACHINE_NAME}/${VERSION}/kernel_source.tar.gz" \
| tar xz --strip-components=2 && \
make -C build modules_prepare -j"$(nproc)"
The last command will have the following output, and then it will freeze waiting for user input:
make: Entering directory '/usr/src/kernel_source/build'
SYNC include/config/auto.conf.cmd
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
LEX scripts/kconfig/lexer.lex.c
HOSTCC scripts/kconfig/expr.o
YACC scripts/kconfig/parser.tab.[ch]
HOSTCC scripts/kconfig/symbol.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/util.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/lexer.lex.o
HOSTCC scripts/kconfig/parser.tab.o
HOSTLD scripts/kconfig/conf
*
* Restart config...
*
*
* BPF based packet filtering framework (BPFILTER)
*
BPF based packet filtering framework (BPFILTER) (BPFILTER) [Y/n/?] y
bpfilter kernel module with user mode helper (BPFILTER_UMH) [M/n/y/?] (NEW)
Notes
It seems that export DEBIAN_FRONTEND=noninteractive is not doing the job.
Looking at make options there doesn't seem to be an option to skip the question, but I guess I am wrong. How can I fix this?

Related

Singularity arguments conflict with my bioinformatics tool arguments

EDIT: documentation given by the informatic administration was shitty, old version of singularity, now the order of arguments is different and the problem is solved.
To make my tool more portable, and because I have to use it on a cluster, I have to put my bioinformatics tool at disposal for docker. Tool is located here. The docker hub is 007ptar007/metadbgwas, if you want to experience with it. The Dockerfile is in the repo, and to make it easier to everyone :
FROM ubuntu:latest
ENV DEBIAN_FRONTEND=noninteractive
USER root
COPY ./install_docker.sh ./
RUN chmod +x ./install_docker.sh && sh ./install_docker.sh
ENTRYPOINT ["/MetaDBGWAS/metadbgwas.sh"]
ENV PATH="/MetaDBGWAS/:${PATH}"
And the install_docker.sh script contains :
apt-get update
apt install -y libgatbcore-dev libhdf5-dev libboost-all-dev libpstreams-dev zlib1g-dev g++ cmake git r-base-core
Rscript -e "install.packages(c('ape', 'phangorn'))"
Rscript -e "install.packages('https://raw.githubusercontent.com/sgearle/bugwas/master/build/bugwas_1.0.tar.gz', repos=NULL, type='source')"
git clone --recursive https://github.com/Louis-MG/MetaDBGWAS.git
cd MetaDBGWAS
sed -i "51i#include <limits>" ./REINDEER/blight/robin_hood.h #temporary fix for REINDEER compilation
sh install.sh
The problem :
My tool parses the command line, and needs a verbose (-v, or --verbose) argument. It also needs to reject unknown arguments; anything that isn't used by the tool causes the help message to be printed in the standard output and exits. To use the tool, I need to mount volumes were the data is; using -v /path/to/files:/input option:
singularity run docker://007ptar007/metadbgwas --volumes '/path/to/data:/inputd/:/input' --files /input --strains /input/strains --threads 8 --output ~/output
But my tool sees this as a bad -v option value or the --volume as an unknown option. I can't change this on my tool. How do I solve this conflict ?
You need to put any arguments intended for singularity - such as the volume mounting - before the name of the image you want to run (e.g. the docker image you specify in your command):
singularity run -v '/path/to/data:/input' docker://007ptar007/metadbgwas --files /input --strains /input/strains --threads 8 --output ~/output

Building a multi-architecture Docker image, but Dockerfile requires different packages depending on architecture

I am experimenting with docker's buildx and noticed that everything seems to be straight forward except for one thing. My Dockerfile needs to pull certain packages depending on the architecture.
For example, here's a piece of the Dockerfile:
FROM XYZ
# Set environment variable for non-interactive install
ARG DEBIAN_FRONTEND=noninteractive
# Run basic commands to update the image and install basic stuff.
RUN apt update && \
apt dist-upgrade -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" && \
apt autoremove -y && \
apt clean -y && \
...
# Install amazon-ssm-agent
mkdir /tmp/ssm && \
curl https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/debian_amd64/amazon-ssm-agent.deb -o /tmp/ssm/amazon-ssm-agent.deb && \
As you can see from above, the command is set to pull down the Amazon SSM agent using a hard-coded link.
What's the best way to approach this? Should I just modify this Dockerfile to create a bunch of if conditions?
Docker automatically defines a set of ARGs for you when you're using the BuildKit backend (which is now the default). You need to declare that ARG, and then (within the RUN command) you can use an environment variable $TARGETOS to refer to the target operating system (the documentation suggests linux or windows).
FROM ...
# Must be explicitly declared, and after FROM
ARG TARGETOS
# Then it can be used like a normal environment variable
RUN curl https://s3.amazonaws.com/ec2-downloads-$TARGETOS/...
There is a similar $TARGETPLATFORM if you need to build either x86 or ARM images, but its syntax doesn't necessarily match what's in this URL. If $TARGETPLATFORM is either amd64 or arm, you may need to reconstruct the Debian architecture string. You can set a shell variable within a single RUN command and it will last until the end of that command, but no longer.
ARG TARGETPLATFORM
RUN DEBARCH="$TARGETPLATFORM"; \
if [ "$DEBARCH" = "arm" ]; then DEBARCH=arm64; fi; \
curl .../debian-$DEBARCH/...

Accepting license terms while running Dockerfile

I am having trouble building an image on Docker when trying to install Berryconda. I need to accept the terms and conditions before I can continue (as shown in the image below). Has anyone experienced this issue before and knows how to tackle this scenario?
Berryconda T&C
Berryconda installation portion of the Dockerfile
RUN : \
&& echo "Installing Berryconda" \
&& wget -O berryconda3-2.0.0.sh "https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh" \
&& chmod +x berryconda3-2.0.0.sh \
&& ./berryconda3-2.0.0.sh /usr/bin/berryconda3
You should be able to run the Berryconda install script with the -b argument to execute the script in batch mode.
-b run install in batch mode (without manual intervention),
it is expected the license terms are agreed upon
The available command line arguments for the installer are described near the top of the install script.

How to set breakpoint in Dockerfile itself?

Searching up the above shows many results about how to set breakpoints for apps running in docker containers, yet I'm interested in setting a breakpoint in the Dockerfile itself, such that the docker build is paused at the breakpoint. For an example Dockerfile:
FROM ubuntu:20.04
RUN echo "hello"
RUN echo "bye"
I'm looking for a way to set a breakpoint on the RUN echo "bye" such that when I debug this Dockerfile, the image will build non-interactively up to the RUN echo "bye" point, exclusive. After then, I would be able to interactively run commands with the container. In the actual Dockerfile I have, there are RUNs before the breakpoint that change the file system of the image being built, and I want to analyze the filesystem of the image at the breakpoint by being able to interactively run commands like cd / ls / find at the time of the breakpoint.
You can't set a breakpoint per se, but you can get an interactive shell at an arbitrary point in your build sequence (between steps).
Let's build your image:
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu:20.04
---> 1e4467b07108
Step 2/3 : RUN echo "hello"
---> Running in 917b34190e35
hello
Removing intermediate container 917b34190e35
---> 12ebbdc1e72d
Step 3/3 : RUN echo "bye"
---> Running in c2a4a71ae444
bye
Removing intermediate container c2a4a71ae444
---> 3c52993b0185
Successfully built 3c52993b0185
Each of the lines that says ---> 0123456789ab with a hex ID has a valid image ID. So from here you can
docker run --rm -it 12ebbdc1e72d sh
which will give you an interactive shell on the partial image resulting from the first RUN command.
There's no requirement that the build as a whole succeed. If a RUN step fails, you can use this technique to get an interactive shell on the image immediately before that step and re-run the command by hand. If you have a very long RUN command, you may need to break it into two to be able to get a debugging shell at a specific point within the command sequence.
I don't think this is possible directly - that feature has been discussed and rejected.
What I generally do to debug a Dockerfile is to comment all of the steps after the "breakpoint", then run docker build followed by docker run -it image bash or docker run -it image sh (depending on whether you have bash installed inside the container).
Then, I have an interactive shell, and I can run commands to debug why later stages are failing.
I agree that being able to set a breakpoint and poke around would be a handy feature, though.
You can run commands in intermediate containers using Remote shell debugging tricks.
Make sure your container images include basic utilities like netcat (nc) and fuser. These utilities enable "calling home" from any intermediate container image. At home you'll answer calls with netcat (or socat). This netcat will send your commands to containers, and print their outcomes. This debugging approach will work even on Dockerfiles that are built on unknown worker nodes somewhere in cloud.
Example:
FROM debian:testing-slim
# Set environment variables for calling home from breakpoints (BP)
ENV BP_HOME=<IP-ADDRESS-OF-YOUR-HOST>
ENV BP_PORT=33720
ENV BP_CALLHOME='BP_FIFO=/tmp/$BP.$BP_HOME.$BP_PORT; (rm -f $BP_FIFO; mkfifo $BP_FIFO) && (echo "\"c\" continues"; echo -n "($BP) "; tail -f $BP_FIFO) | nc $BP_HOME $BP_PORT | while read cmd; do if test "$cmd" = "c" ; then echo -n "" >$BP_FIFO; sleep 0.1; fuser -k $BP_FIFO >/dev/null 2>&1; break; else eval $cmd >$BP_FIFO 2>&1; echo -n "($BP) " >$BP_FIFO; fi; done'
# Install needed utils (netcat, fuser)
RUN apt update && apt install -y netcat psmisc
# Now you are ready to run "eval $BP_CALLHOME" wherever you want to call home.
RUN BP=before-hello eval $BP_CALLHOME
RUN echo "hello"
RUN BP=after-hello eval $BP_CALLHOME
RUN echo "bye"
Start waiting for and answering calls from a Dockerfile before launching a Docker build. On home host run nc -k -l -p 33720 (alternatively socat STDIN TCP-LISTEN:33720,reuseaddr,fork).
This is how above example looks like at home:
$ nc -k -l -p 33720
"c" continues
(before-hello) echo *
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
(before-hello) id
uid=0(root) gid=0(root) groups=0(root)
(before-hello) c
"c" continues
(after-hello)
...
The recent (May 2022) project ktock/buildg offers breakpoints.
See "Interactive debugger for Dockerfile" from Kohei Tokunaga
buildg is a tool to interactively debug Dockerfile based on BuildKit.
Source-level inspection
Breakpoints and step execution
Interactive shell on a step with your own debugigng tools
Based on BuildKit (needs unmerged patches)
Supports rootless
The command break, b LINE_NUMBER sets a breakpoint.
Example:
$ buildg.sh debug --image=ubuntu:22.04 /tmp/ctx
WARN[2022-05-09T01:40:21Z] using host network as the default
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.1s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 195B done
#2 DONE 0.1s
#3 [internal] load metadata for docker.io/library/busybox:latest
#3 DONE 3.0s
#4 [build1 1/2] FROM docker.io/library/busybox#sha256:d2b53584f580310186df7a2055ce3ff83cc0df6caacf1e3489bff8cf5d0af5d8
#4 resolve docker.io/library/busybox#sha256:d2b53584f580310186df7a2055ce3ff83cc0df6caacf1e3489bff8cf5d0af5d8 0.0s done
#4 sha256:50e8d59317eb665383b2ef4d9434aeaa394dcd6f54b96bb7810fdde583e9c2d1 772.81kB / 772.81kB 0.2s done
Filename: "Dockerfile"
2| RUN echo hello > /hello
3|
4| FROM busybox AS build2
=> 5| RUN echo hi > /hi
6|
7| FROM scratch
8| COPY --from=build1 /hello /
>>> break 2
>>> breakpoints
[0]: line 2
>>> continue
#4 extracting sha256:50e8d59317eb665383b2ef4d9434aeaa394dcd6f54b96bb7810fdde583e9c2d1 0.0s done
#4 DONE 0.3s
...
From PR 24:
Add --cache-reuse option which allows sharing the build cache among invocation of buildg debug to make the 2nd-time debugging faster.
This is useful to speed up running buildg multiple times for debugging an errored step.
Note that breakpoints on cached steps are ignored as of now.
Because of this limitation, this feature is optional as of now. We should fix this limitation and make it the default behaviour in the future.
Man, Docker makes things hard. Here's a workaround I cooked up:
Insert FROM scratch where you want the break point.
Run docker build . --stage=<n-1> where <n> is the number of FROM commands before your "breakpoint". Eg, if it's a single stage build, use --stage=0.
Alternatively, if you have already named the stage where you want the break point with FROM <image> AS <stage> then you can use --stage=<stage> instead.
Docker has cached all your successful layers anyway (even if you can't see them), and because the FROM "breakpoint" comes before the (potentially unsuccessful) point of interest, the build should all come from cache and be very fast.
So for example, if my Dockerfile looks like this:
FROM debian:bullseye AS build
RUN apt-get update && apt-get install -y \
build-essential cmake ninja-build \
libfontconfig1-dev libdbus-1-dev libfreetype6-dev libicu-dev libinput-dev libxkbcommon-dev libsqlite3-dev libssl-dev libpng-dev libjpeg-dev libglib2.0-dev
<SNIP lots of other setup commands>
ADD my_source.tar.xz /
WORKDIR /my_source
RUN ./configure -option1 -option2
RUN cmake --build . --parallel
RUN cmake --install .
FROM alpine
COPY --from=build /my_build /my_build
...
Then I can add a "breakpoint" like this:
FROM debian:bullseye AS build
RUN apt-get update && apt-get install -y \
build-essential cmake ninja-build \
libfontconfig1-dev libdbus-1-dev libfreetype6-dev libicu-dev libinput-dev libxkbcommon-dev libsqlite3-dev libssl-dev libpng-dev libjpeg-dev libglib2.0-dev
<SNIP lots of other setup commands>
ADD my_source.tar.xz /
WORKDIR /my_source
#### BREAKPOINT ###
FROM scratch
#### BREAKPOINT ###
RUN ./configure -option1 -option2
RUN cmake --build . --parallel
RUN cmake --install .
FROM alpine
COPY --from=build /my_build /my_build
...
and trigger it with docker build . --stage=build

Succesfully created a virtualenv (using "mkproject") in Dockerfile, but can't run "workon" properly

Edit: Solved- typo
I have a Dockerfile that successfully creates a virtualenv using virtualenvwrapper (along with setting up a heap of "standard" settings/packages in our normal environment). I am using the resulting image as a "base image" for further use. All good so far. However, the following Dockerfile (based of the first image, "base_image_14.04") falls down at the last line:
FROM base_image_14.04
USER root
RUN DEBIAN_FRONTEND=noninteractive \
apt-get update && apt-get install -y \
libproj0 libproj-dev \
libgeos-c1v5 libgeos-dev \
libjpeg62 libjpeg-dev \
zlib1g zlib1g-dev \
libfreetype6 libfreetype6-dev \
libgdal20 libgdal-dev \
&& rm -rf /var/lib/apt/lists
USER webdev
RUN ["/bin/bash", "-ic", "mkproject maproxy"]
EXPOSE 80
WORKDIR $PROJECT_HOME/mapproxy
ADD ./requirements.txt .
RUN ["/bin/bash", "-ic", "workon mapproxy && pip install -r requirements.txt"]
The "mkproject mapproxy" works fine. If I comment out the last line it builds successfully and I can spin up the container and run "workon mapproxy" manually, not a problem. But when I try and build with the last line, it gives a workon error:
ERROR: Environment 'mapproxy' does not exist. Create it with 'mkvirtualenv mapproxy'.
workon is being called, but for some reason it can't find the mapproxy virtualenv.
WORKON_HOME & PROJECT_HOME both exist (defined in the parent image) and point to the correct locations (and are used successfully by "mkproject mapproxy").
So why is workon returning an error when the mapproxy virtualenv exists? The same error happens when I isolate that last line into a third Dockerfile building on the second.
Solved: It was a simple typo. mkproject maproxy instead of mapproxy. :sigh:
I am trying to build a docker image and am running into similar problems.
First question was why use a virtual env in docker? The main reason in a nutshell is to minimize effort to migrate an existing and working approach into a docker container. I will eventually use docker-compose, but I wanted to start by getting my feet wet with it all in a single docker container.
In my first attempt I installed almost everything with apt-get, including uwsgi. I installed my app "globally" with pip3. The app has command line functionality and a separate flask web app, hence the need for uwsgi. The command line functionality works, but when I make a request of the flask app uwsgi / python has a problem with locale: Fatal Python error: Py_Initialize: Unable to get the locale encoding and ImportError: No module named 'encodings
I have stripped away all my app specific additions to narrow down the problem. This is the Dockerfile I'm using:
# Docker image definition for testing
FROM ubuntu:xenial
# Create a user
RUN useradd -G sudo -ms /bin/bash tester
RUN echo 'tester:password' | chpasswd
WORKDIR /home/tester
# Skipping apt-get update to save some build time. Some are kept
# to insure they are the same as on host setup.
RUN apt-get install -y python3 python3-dev python3-pip \
virtualenv virtualenvwrapper sudo nano && \
apt-get clean -qy
# After above, can we use those installed in rest of Dockerfile?
# Yes, but not always, such as with virtualenvwrapper. What about
# virtualenv? How do you "source" the script? Doesn't appear to be
# installed, as bash complains "source needs a single parameter"
ENV VIRTUALENVWRAPPER_PYTHON /usr/bin/python3
ENV VIRTUALENVWRAPPER_VIRTUALENV /usr/bin/virtualenv
RUN ["/bin/bash", "-c", "source", "/usr/share/virtualenvwrapper/virtualenvwrapper.sh"]
# Create a virtualenv so uwsgi can find locale
# RUN mkdir /home/tester/.virtualenv && virtualenv -p`which python3` /home/bts_tools/.virtualenv/bts_tools
RUN mkvirtualenv -p`which python3` bts_tools && \
workon bts_tools && \
pip3 --disable-pip-version-check install --upgrade bts_tools
USER tester
ENTRYPOINT ["/bin/bash"]
CMD ["--login"]
The build fails on the line I try to source the virtualenvwrapper script. Bash complains source needs an argument - the file to be sourced. So I comment out the RUN lines and it builds without error. When I run the resulting container I see all the additions to the ENV that virtualenvwrapper makes (you can see all of them by executing the "set" command without any args), and the script to be sourced is there too.
So my question is why doesn't docker find them? How does the docker build process work if the results of any previous RUNs or ENVs aren't applied for subsequent use in the Dockerfile? I know some things are applied and work, for example if you apt-get nginx you can refer to /etc/nginx or alter things under that folder. You can create a user and set it's password or cd into its home folder for example. If I move the WORKDIR before the RUN useradd -G I see a warning from useradd the home folder already exists. I tried to use the "time" program to time how long it takes to do various things in the Dockerfile and docker complains it can't find 'time'.
So what exactly is going on? I have spent the last 3 days trying to figure this out. It just shouldn't be this difficult. What am I missing?
Parts of the bts_tools flask app worked when I wasn't using virtual envs. Most of the app didn't work, and the issue was this locale problem. Since everything works on the host outside of docker, and after trying to alter the PATH, PYTHONHOME, PYTHONPATH in my uwsgi start script to overcome the dreaded "locale encoding" fatal error, I decided to try to replicate the host setup as closely as possible since that didn't have the locale issue. When I have had that problem before I could run dpkg-reconfigure python3 or fix with changes to PATH or ENV settings. If you google the problem you'll see many people have difficulties with python & locale. It's almost enough reason to avoid using python!
I posted this elsewhere about locale issue, if it helps.

Resources