I have the following Dockerfile:
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y \
build-essential \
ca-certificates \
gcc \
git \
libpq-dev \
make \
python-pip \
python2.7 \
python2.7-dev \
ssh \
&& apt-get autoremove \
&& apt-get clean
ARG SSH_PRIVATE_KEY
RUN mkdir /root/.ssh/
RUN echo "${SSH_PRIVATE_KEY}" > /root/.ssh/id_rsa
RUN touch /root/.ssh/known_hosts
RUN ssh-keyscan http://bitbuckrt.org >> /root/.ssh/known_hosts
RUN pip install git+ssh://git#bitbucket.org/repo.git
I am building the Docker image from this Dockerfile using the following command:
docker build -t myimage:v1 --build-arg SSH_PRIVATE_KEY="ssh-rsa jkdfjgklfsgnkljgxdfeheflkfkl/hkskkdhgtgshshsh/... " .
However, it is not building my image. I get the following error:
"docker build" requires exactly 1 argument.
What could be the issue? How to correctly pass the SSH_PRIVATE_KEY while building the image?
Assign your private key to a bash variable and use it in the command. Reading the key from a file while assigning to the variable is the safest. The special chars inside the key might be screwing the command if it has, say, a quotation mark. Ex:
PKEY=$(<key.txt)
docker build -t myimage:v1 --build-arg SSH_PRIVATE_KEY=$PKEY .
Related
I am new to docker and not an IT-specialist. I try to install Kartoza Geoserver in Docker on Heroku, but so far no success. Does anyone has experience with this and can explain me the settings in the dockerfile and the steps specifically for a Heroku install?
So far I tried a build with a modified dockerfile but I always get the same error (in the log trail) when opening/launching geoserver on Heroku:
"Error: groupadd: cannot open /etc/group".
I guess it is an permission/privileges issue.
Any sharing of experience on modifying the docker file so that the image is read by Heroku would be helpfull.
Modifying the settings in the dockerfile:
Removed the port forwarding from dockerfile
Add RUN adduser -D myuser USER myuser to dockerfile
Result dockerfile:
#--------- Generic stuff all our Dockerfiles should start with so we get caching ------------
ARG IMAGE_VERSION=9.0.65-jdk11-openjdk-slim-buster
ARG JAVA_HOME=/usr/local/openjdk-11
FROM tomcat:$IMAGE_VERSION
LABEL maintainer="Tim Sutton<tim#linfiniti.com>"
ARG GS_VERSION=2.22.0
ARG WAR_URL=https://downloads.sourceforge.net/project/geoserver/GeoServer/${GS_VERSION}/geoserver-${GS_VERSION}-war.zip
ARG STABLE_PLUGIN_BASE_URL=https://sourceforge.net/projects/geoserver/files/GeoServer
ARG DOWNLOAD_ALL_STABLE_EXTENSIONS=1
ARG DOWNLOAD_ALL_COMMUNITY_EXTENSIONS=1
ARG HTTPS_PORT=8443
ENV DEBIAN_FRONTEND=noninteractive
#Install extra fonts to use with sld font markers
RUN adduser -D myuser
USER myuser
RUN set -eux; \
apt-get update; \
apt-get -y --no-install-recommends install \
locales gnupg2 wget ca-certificates rpl pwgen software-properties-common iputils-ping \
apt-transport-https curl gettext fonts-cantarell lmodern ttf-aenigma \
ttf-bitstream-vera ttf-sjfonts tv-fonts libapr1-dev libssl-dev \
wget zip unzip curl xsltproc certbot cabextract gettext postgresql-client figlet gosu gdal-bin; \
# Install gdal3 - bullseye doesn't build libgdal-java anymore so we can't upgrade
curl https://deb.meteo.guru/velivole-keyring.asc | apt-key add - \
&& echo "deb https://deb.meteo.guru/debian buster main" > /etc/apt/sources.list.d/meteo.guru.list \
&& apt-get update \
&& apt-get -y --no-install-recommends install gdal-bin libgdal-java; \
dpkg-divert --local --rename --add /sbin/initctl \
&& (echo "Yes, do as I say!" | apt-get remove --force-yes login) \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*; \
# verify that the binary works
gosu nobody true
ENV \
JAVA_HOME=${JAVA_HOME} \
DEBIAN_FRONTEND=noninteractive \
GEOSERVER_DATA_DIR=/opt/geoserver/data_dir \
GDAL_DATA=/usr/share/gdal \
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/tomcat/native-jni-lib:/usr/lib/jni:/usr/local/apr/lib:/opt/libjpeg-turbo/lib64:/usr/lib:/usr/lib/x86_64-linux-gnu" \
FOOTPRINTS_DATA_DIR=/opt/footprints_dir \
GEOWEBCACHE_CACHE_DIR=/opt/geoserver/data_dir/gwc \
CERT_DIR=/etc/certs \
RANDFILE=/etc/certs/.rnd \
FONTS_DIR=/opt/fonts \
GEOSERVER_HOME=/geoserver \
EXTRA_CONFIG_DIR=/settings \
COMMUNITY_PLUGINS_DIR=/community_plugins \
STABLE_PLUGINS_DIR=/stable_plugins
WORKDIR /scripts
ADD resources /tmp/resources
ADD build_data /build_data
ADD scripts /scripts
RUN echo $GS_VERSION > /scripts/geoserver_version.txt ;\
chmod +x /scripts/*.sh;/scripts/setup.sh \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN echo 'figlet -t "Kartoza Docker GeoServer"' >> ~/.bashrc
WORKDIR ${GEOSERVER_HOME}
ENTRYPOINT ["/bin/bash", "/scripts/entrypoint.sh"]
Then build it with argument --platform linux/amd64 (I use arm64 architecture). Then pushed it to Heroku. All the time I get the same error.
I am trying to copy a file into my docker container but the command fails. The file is in the same directory as the Dockerfile, so I don't understand the reason for the error.
I'd appreciate any help or advice. Thanks beforehand.
This is the code:
FROM ubuntu:20.04 as builder
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get install -y \
build-essential \
cmake \
software-properties-common \
libopencv-dev
RUN add-apt-repository -y ppa:chrberger/libcluon
RUN apt-get update
RUN apt-get install -y libcluon
ADD . /opt/sources
WORKDIR /opt/sources
RUN mkdir build && \
cd build && \
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/tmp/dest .. && \
make && make install
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update --fix-missing
RUN apt-get install -y \
libopencv-core4.2 \
libopencv-imgproc4.2 \
libopencv-video4.2 \
libopencv-calib3d4.2 \
libopencv-features2d4.2 \
libopencv-objdetect4.2 \
libopencv-highgui4.2 \
libopencv-videoio4.2 \
libopencv-flann4.2 \
libopencv-dnn-dev \
python3-opencv
WORKDIR /usr/bin
COPY --from=builder /tmp/dest /usr
COPY --from=builder yolov3-tiny_obj.cfg /params
ENTRYPOINT ["/usr/bin/opendlv-perception-helloworld"]
Could you please clarify which line in your Dockerfile causes the error message?
Is the file you are trying to copy from your working directory yolov3-tiny_obj.cfg?
If that is the case, it fails because you specify to copy it from the builder stage.
The line should probably look like this:
COPY yolov3-tiny_obj.cfg /params
I'm trying to build a Docker image including a very particular configuration of OpenCV with CUDA and GPU support.
The build succeeds, and if I make install it from the same context that built the image, it works with no problems.
The problem happens when I try to use a multi stage build, to avoid keeping all the dependencies needed to build OpenCV. Before you continue reading, what follows might actually be an XY problem, if you have a better solution on how to copy OpenCV build artifacts (including Python bindings!) in a Docker multistage build, that is my actual intent.
Now for my attempted solution and the struggle I have:
I run COPY --from=requirements /opencv /opencv and it works and it apparently copies everything in the right path (I checked the filesystem). But, when I run from the build folder make install, I get this CMake error:
CMake Error: The source directory "" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:2724: recipe for target 'cmake_check_build_system' failed
make: *** [cmake_check_build_system] Error 1
Again, the same command, from the same folder, but without multistage build, works.
Here is my Dockerfile:
# Stage 1: Build
FROM nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04 AS requirements
# Install dependencies
RUN echo "deb http://es.archive.ubuntu.com/ubuntu eoan main universe" | tee -a /etc/apt/sources.list
RUN apt-get update && apt-get -y upgrade
RUN apt-get -y install build-essential cmake unzip pkg-config libjpeg-dev libpng-dev libtiff-dev libavcodec-dev \
libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libgtk-3-dev libatlas-base-dev \
gfortran python3-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libxvidcore-dev x264 \
libx264-dev libfaac-dev libmp3lame-dev libtheora-dev libfaac-dev libmp3lame-dev libvorbis-dev \
libjpeg-dev libpng-dev libtiff-dev git python3-pip libtbb-dev libprotobuf-dev protobuf-compiler \
libgoogle-glog-dev libgflags-dev libgphoto2-dev libeigen3-dev libhdf5-dev wget libtbb-dev gcc-8 g++-8 llvm \
python3-venv libgirepository1.0-dev
# Install my project requirements
WORKDIR /venv
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"
ADD requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
# Build OpenCV
WORKDIR /opencv
RUN wget https://github.com/opencv/opencv/archive/4.4.0.zip && mv 4.4.0.zip opencv.zip && unzip opencv.zip && rm opencv.zip
RUN wget https://github.com/opencv/opencv_contrib/archive/4.4.0.zip && mv 4.4.0.zip opencv_contrib.zip && unzip opencv_contrib.zip && rm opencv_contrib.zip
WORKDIR /opencv/opencv-4.4.0/build
ENV SITE_PACKAGES /venv/lib/python3.7/site-packages
ENV EXTRA_MODULES /opencv/opencv_contrib-4.4.0/modules
ENV CUDA_ARCH 7.5
ADD docker/build_opencv.sh .
RUN ./build_opencv.sh
# Stage 2: runtime
FROM nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04
RUN apt-get update && apt-get -y upgrade
RUN apt-get -y install build-essential cmake python3-venv
# Install OpenCV
COPY --from=requirements /opencv /opencv
WORKDIR /opencv/opencv-4.4.0/build
RUN make install && ldconfig
# build fails here and the rest is specific to my project so I've ommitted it
The build_opencv.sh script has this options:
#!/bin/bash
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_C_COMPILER=/usr/bin/gcc-8 \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D INSTALL_C_EXAMPLES=OFF \
-D WITH_TBB=ON \
-D WITH_CUDA=ON \
-D BUILD_opencv_cudacodec=OFF \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
-D WITH_V4L=ON \
-D WITH_QT=OFF \
-D WITH_OPENGL=ON \
-D WITH_GSTREAMER=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_PC_FILE_NAME=opencv.pc \
-D OPENCV_ENABLE_NONFREE=ON \
-D OPENCV_PYTHON3_INSTALL_PATH=$SITE_PACKAGES \
-D OPENCV_EXTRA_MODULES_PATH=$EXTRA_MODULES \
-D PYTHON_EXECUTABLE=/usr/bin/python3 \
-D WITH_CUDNN=ON \
-D OPENCV_DNN_CUDA=ON \
-D CUDA_ARCH_BIN=$CUDA_ARCH \
-D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2 \
-D WITH_GTK_2_X=OFF \
-D BUILD_EXAMPLES=OFF ..
make -j16
You need at least numpy in your requirements.txt file.
In order to reproduce the issue, a minimal setup would have this structure:
- docker
- Dockerfile
- build_opencv.sh
- requirements.txt
Build using from the root of the build context:
docker build -t opencvmultistage:latest -f docker/Dockerfile .
Am I doing something wrong? Maybe CMake has some weird cache that I'm not copying to the new image and makes the build fail?
For the sake of clarity, if I add make install in the build_opencv.sh script it works, but I have OpenCV installed in the build context and not the runtime, which is not what I pretend to do. make install runs in the same directory, and the same files should be present, so I don't really know what's going on.
It is simpler to run cmake & make and make install in the same stage and then copy the install folders. It will allow to not have any build tools like cmake or build-essential in the final docker image.
We will use a custom CMAKE_INSTALL_PREFIX so that OpenCV binaries are installed to a directory and we can copy it straight to the next stage. Using a custom prefix will avoid having to copy CUDA installation or development libraries no longer required. Then we will run ldconfig on that directory to link the libraries as usual.
Modify the build script to use a custom CMAKE_INSTALL_PREFIX:
mkdir /prefix
cmake -D CMAKE_BUILD_TYPE=RELEASE \
# all compiler flags...
-D CMAKE_INSTALL_PREFIX=/prefix
Modifying the Dockerfile
to run make install in stage 1
# Stage 1: Build
FROM nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04 AS requirements
...
ADD build_opencv.sh .
RUN ./build_opencv.sh && make install
copy the installation in stage 2
# Stage 2: runtime
FROM nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04
RUN apt-get update && apt-get -y upgrade
RUN apt-get -y install build-essential python3-venv
# Install OpenCV
COPY --from=requirements /prefix /prefix
COPY --from=requirements /venv /venv
ENV PATH="/venv/bin:$PATH"
RUN ldconfig /prefix
I have some server and clients who are communicating using gRPC/golang. Now I want to containerize my application but the size of the docker image containing goland execution and grpc support is larger(more than 1GB). I would like to decrease the size of the docker image.
The required golang version is 1.9 and higher. Here is the Dockerfile script is given. If there is other way please suggest it.
FROM golang:1.11
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
CMD ["go", "run", "helloworld/greeter_server/main.go"]
try to make a multistage docker image like this
# Compile stage
FROM golang:1.11 as build-env
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
RUN go build -o server helloworld/greeter_server/main.go
# Making image
FROM alpine:3.8 AS host
RUN apk add --no-cache \
ca-certificates
COPY --from=build-env /helloworld/server /
# copy any other files you need
WORKDIR /
EXPOSE 8000
CMD ["server"]
You can try to use distroless base images and multi-stage builds. That might help you.
What do I have to do to run the Turtlebot 3 Simulation as described in http://emanual.robotis.com/docs/en/platform/turtlebot3/simulation/ on a debian:buster system using docker?
The steps for debian:stretch using repositories in http://wiki.ros.org/lunar/Installation/Debian are not working with debian:buster, see https://github.com/ros-infrastructure/rospkg/issues/125
I finally found a solution to my question.
Create a Dockerfile as follows:
FROM osrf/ros:kinetic-desktop-full-jessie
RUN apt-get update && apt-get install -y --no-install-recommends screen
RUN apt-get install -y --no-install-recommends \
ros-kinetic-joy \
ros-kinetic-teleop-twist-joy \
ros-kinetic-teleop-twist-keyboard \
ros-kinetic-laser-proc \
ros-kinetic-rgbd-launch \
ros-kinetic-depthimage-to-laserscan \
ros-kinetic-rosserial-arduino \
ros-kinetic-rosserial-python \
ros-kinetic-rosserial-server \
ros-kinetic-rosserial-client \
ros-kinetic-rosserial-msgs \
ros-kinetic-amcl \
ros-kinetic-map-server \
ros-kinetic-move-base \
ros-kinetic-urdf \
ros-kinetic-xacro \
ros-kinetic-compressed-image-transport \
ros-kinetic-rqt-image-view \
ros-kinetic-gmapping \
ros-kinetic-navigation
RUN mkdir -p /root/catkin_ws/src/ \
&& cd /root/catkin_ws/src/ \
&& git clone https://github.com/ROBOTIS-GIT/turtlebot3_msgs.git \
&& git clone https://github.com/ROBOTIS-GIT/turtlebot3.git \
&& git clone https://github.com/ROBOTIS-GIT/turtlebot3_simulations.git
RUN /bin/bash -c "source /opt/ros/kinetic/setup.bash; cd /root/catkin_ws && /opt/ros/kinetic/bin/catkin_make && /opt/ros/kinetic/bin/catkin_make -DCMAKE_INSTALL_PREFIX=/opt/ros/kinetic install"
RUN apt-get install -y --no-install-recommends vim bash-completion sudo
RUN apt-get install -y --no-install-recommends apt-utils
RUN useradd --create-home --shell /bin/bash robo
RUN echo "robo ALL=(ALL:ALL) NOPASSWD:ALL" > /etc/sudoers.d/robo && chmod 0440 /etc/sudoers.d/robo
COPY ./start_simu.sh /usr/local/bin
RUN chmod 755 /usr/local/bin/start_simu.sh
USER robo
WORKDIR /home/robo
RUN rm -rf /var/lib/apt/lists/*
Add a file start_simu.sh in the same directory, containing:
#!/bin/bash
screen -dmS turtlebot_fake /bin/bash -c "source /opt/ros/kinetic/setup.bash;env TURTLEBOT3_MODEL=burger roslaunch turtlebot3_fake turtlebot3_fake.launch"
sleep 2
screen -S turtlebot_fake -X screen /bin/bash -c "source /opt/ros/kinetic/setup.bash;env TURTLEBOT3_MODEL=burger roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch"
source "/opt/ros/$ROS_DISTRO/setup.bash"
exec "/bin/bash"
Now built your docker image using sudo docker build --tag ros:turtlebot3_fake_node .
Run the image:
xhost +local:root
sudo docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume="/tmp/.X11-unix:/tmp/.X11-uni
x:rw" ros:turtlebot3_fake_node /usr/local/bin/start_simu.sh
If image is stopped, do xhost -local:root.
The simulation is running in screen, connect to it via screen -R.