I am planning to run a customized Rstudio with both tidyverse, rmarkdown and rstan pre-installed on Digital Ocean's Docker 17.03.0-ce on 16.04.
On Docker Hub, there is already a pre-built image: Rocker/verse that has tidyverse and rmarkdown functionality. So my plan was to simply get the latest version of Rocker/verse and add a new layer of rstan. I use the dockerfile of jonzelner/rstan and jrnold/rstan and update them as below:
FROM rocker/verse:latest
# Install essentials
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
clang-3.6
# Global site-wide config
RUN mkdir -p $HOME/.R/ \
&& echo "\nCXX=clang++ -ftemplate-depth-256\n" >> $HOME/.R/Makevars \
&& echo "CC=clang\n" >> $HOME/.R/Makevars
# Install rstan
RUN install2.r --error \
inline \
RcppEigen \
StanHeaders \
rstan \
KernSmooth
# Config for rstudio user
RUN mkdir -p /home/rstudio/.R/ \
&& echo "\nCXX=clang++ -ftemplate-depth-256\n" >> /home/rstudio/.R/Makevars \
&& echo "CC=clang\n" >> /home/rstudio/.R/Makevars \
&& echo "CXXFLAGS=-O3\n" >> /home/rstudio/.R/Makevars \
&& echo "\nrstan::rstan_options(auto_write = TRUE)" >> /home/rstudio/.Rprofile \
&& echo "options(mc.cores = parallel::detectCores())" >> /home/rstudio/.Rprofile
# Install loo
RUN install2.r --error \
matrixStats \
loo
However, my attempt to build this image (docker build -t image_name .) ended in error. The results are provided below:
Sending build context to Docker daemon 2.56 kB
Step 1/6 : FROM rocker/verse:latest
---> 7ad7d994bffd
Step 2/6 : RUN apt-get update && apt-get install -y --no-install-recommends clang-3.6
---> Using cache
---> 77473c759438
Step 3/6 : RUN mkdir -p $HOME/.R/ && echo "\nCXX=clang++ -ftemplate-depth-256\n" >> $HOME/.R/Makevars && echo "CC=clang\n" >> $HOME/.R/Makevars
---> Using cache
---> a5b06d28b9fe
Step 4/6 : RUN install2.r --error inline RcppEigen StanHeaders rstan KernSmooth
---> Running in 86b52d5a76f3
trying URL 'https://cran.rstudio.com/src/contrib/inline_0.3.14.tar.gz'
Content type 'unknown' length 18002 bytes (17 KB)
==================================================
downloaded 17 KB
* installing *source* package ‘inline’ ...
** package ‘inline’ successfully unpacked and MD5 sums checked
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (inline)
The downloaded source packages are in
‘/tmp/downloaded_packages’
trying URL 'https://cran.rstudio.com/src/contrib/RcppEigen_0.3.2.9.0.tar.gz'
Content type 'unknown' length 1209128 bytes (1.2 MB)
==================================================
downloaded 1.2 MB
* installing *source* package ‘RcppEigen’ ...
** package ‘RcppEigen’ successfully unpacked and MD5 sums checked
** libs
clang++ -ftemplate-depth-256 -I/usr/local/lib/R/include -DNDEBUG -I/usr/local/include -I"/usr/local/lib/R/site-library/Rcpp/include" -I../inst/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppEigen.cpp -o RcppEigen.o
/bin/bash: clang++: command not found
/usr/local/lib/R/etc/Makeconf:141: recipe for target 'RcppEigen.o' failed
make: *** [RcppEigen.o] Error 127
ERROR: compilation failed for package ‘RcppEigen’
* removing ‘/usr/local/lib/R/site-library/RcppEigen’
Error in install.packages(pkgs = f, lib = lib, repos = if (isMatchingFile(f)) NULL else rep, :
installation of package ‘RcppEigen’ had non-zero exit status
The command '/bin/sh -c install2.r --error inline RcppEigen StanHeaders rstan KernSmooth' returned a non-zero code: 1
Some similar questions about a package having non-zero exit status have to do with memory, so I tried to increase the Docker memory on my local machine, but still no luck. Is there any thing I can do to successfully build this image?
As you see in the error output, the installation process cannot find the compiler:
/bin/bash: clang++: command not found
Be aware that this is the output from trying to install clang-3.6 in a rocker/verse container:
root#8ea0d98e67c3:/# apt-get install -y --no-install-recommends clang-3.6
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'python-clang-3.6' for regex 'clang-3.6'
0 upgraded, 0 newly installed, 0 to remove and 15 not upgraded.
It appears that this image doesn't come with Clang 3.6 in its repositories. And this name instead points to python-clang-3.6; Clang bindings for the Python programming language.
Unless you absolutely want 3.6, I would install the plain clang package that points to 3.5, and then re-run the Docker build process.
apt-get install clang
Related
I am having an issue where I'm trying to build in docker, I am getting an error saying GLIBC_2.34 not found (required by /usr/local/lib/bashlib.so)
I am doing this on WSL (Ubuntu) on Windows 11
The contents of my Makefile in the assets folder is as follows
all: bashlib.so
bashlib.so: processhider.c
gcc -Wall -fPIC -shared -o bashlib.so processhider.c -ldl
.PHONY clean:
rm -f bashlib.so
I issue make all and the bashlib.so file gets created. Running ldd --version brings ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35
I go up a folder (I do the above) to create the bashlib.so needed for the next bit.
The processhider.c file mentioned above is taken from https://github.com/gianlucaborello/libprocesshider/blob/master/processhider.c
So now I go into the parent folder and run sudo make build and get the following error:
...
=> ERROR [11/40] RUN groupadd init -g 1050 0.3s
------
> [11/40] RUN groupadd init -g 1050:
#15 0.277 /bin/sh: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/local/lib/bashlib.so)
------
executor failed running [/bin/sh -c groupadd init -g 1050]: exit code: 1
make: *** [Makefile:8: build] Error 1
The content of the Makefile is as follows
TAG?=cheeseballs/$(shell basename `pwd`):latest
all: build
build:
docker build -t "${TAG}" .
run: build
docker run --rm -e RESOURCE_ID=00000000-0000-4000-0000-000000000000 -ti "${TAG}"
clean:
docker rmi "${TAG}"
The contents of the Dockerfile are below (up to and including the line where it errors)
# Pull base image.
FROM debian:bullseye
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y man
# Uncomment the line below to set a root password for testing purposes
# RUN bash -c 'echo -e "testing\ntesting" | passwd root'
# No changes should need to be made here
RUN python3 -m pip install
RUN python3 -m pip install libtmux
RUN python3 -m pip install tmuxp
RUN python3 -m pip install pyinstaller
COPY assets/bashlib.so /usr/local/lib/bashlib.so
RUN chmod 755 /usr/local/lib/bashlib.so
RUN echo /usr/local/lib/bashlib.so >> /etc/ld.so.preload
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
RUN groupadd init -g 1050
...
I don't know where to go from here - any suggestions?
I am trying to install glibc on Alpine Linux. I am running Alpine Linux in the Docker. Here are the steps I am using:
docker pull alpine
docker run -it alpine /bin/sh
apk add --no-cache make gcc linux-headers bsd-compat-headers gawk bison binutils coreutils diffutils gettext bash grep sed texinfo perl
wget https://ftp.gnu.org/gnu/glibc/glibc-2.28.tar.gz
tar -xzf glibc-2.28.tar.gz
cd glibc-2.28
mkdir glibc-build
cd glibc-build
../configure --prefix=/usr \
--disable-profile --enable-add-ons \
--libexecdir=/usr/bin --with-headers=/usr/include \
--enable-static-pie
cat > /etc/ld.so.conf << "EOF"
# Begin /etc/ld.so.conf
/usr/local/lib
/opt/lib
# End /etc/ld.so.conf
EOF
make
make install
I am getting following error on 11th step:
/usr/lib/gcc/x86_64-alpine-linux-musl/6.4.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lssp_nonshared
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:129: /glibc-2.28/glibc-build/elf/sotruss-lib.so] Error 1
make[2]: Leaving directory '/glibc-2.28/elf'
make[1]: *** [Makefile:258: elf/subdir_lib] Error 2
make[1]: Leaving directory '/glibc-2.28'
make: *** [Makefile:9: all] Error 2
If I try to add --disable-shared flag than another errors occur.
The error could be solved by adding libc-dev with the following command: apk add --no-cache libc-dev. But this way I would have two C libraries but I need my application to use glibc specifically.
UPDATE
If I run apk add --no-cache libc-dev, make command passes successfully but make install fails with the following error:
Execution of gcc failed!
The script has found some problems with your installation!
Please read the FAQ and the README file and check the following:
Did you change the gcc specs file (necessary after upgrading from
Linux libc5)?
Are there any symbolic links of the form libXXX.so to old libraries?
Links like libm.so -> libm.so.5 (where libm.so.5 is an old library) are wrong,
libm.so should point to the newly installed glibc file - and there should be
only one such link (check e.g. /lib and /usr/lib)
You should restart this script from your build directory after you've
fixed all problems!
Btw. the script doesn't work if you're installing GNU libc not as your
primary library!
Eventually, I changed few steps in order to build glibc on Alpine Linux.
Here are the steps that worked for me:
docker pull alpine
docker run -it alpine /bin/sh
wget https://ftp.gnu.org/gnu/glibc/glibc-2.28.tar.gz
tar -xzf glibc-2.28.tar.gz
cd glibc-2.28
mkdir glibc-build
cd glibc-build
apk add --no-cache make gcc gawk bison linux-headers libc-dev
../configure --prefix=/usr \
--disable-profile --enable-add-ons \
--libexecdir=/usr/lib --with-headers=/usr/include \
--without-cvs --enable-static-pie
cat > /etc/ld.so.conf << "EOF" # Begin /etc/ld.so.conf
/usr/local/lib
/opt/lib
/usr/lib
/usr/lib64
/usr/libexec
# End /etc/ld.so.conf
EOF
make
make install
I hope these steps will work for everybody else also.
Installing musl-dev should get you past that build error, too, so you'll be building glibc against musl.
The musl-dev package has its own /usr/lib/libssp_nonshared.a:
https://pkgs.alpinelinux.org/contents?file=libssp_nonshared.a&path=&name=musl-dev&branch=v3.8&repo=main&arch=x86_64
I'm trying to build MariaDB on a Debian based container. But it cannot find a z library.
Here is the container image script:
FROM debian
RUN apt-get update
RUN apt-get install -y libncurses-dev
RUN apt-get install -y build-essential;
RUN apt-get install -y cmake;
COPY mariadb-10.1.18.tar.gz /usr/bin/
WORKDIR /usr/bin/
RUN gzip -d mariadb-10.1.18.tar.gz
RUN tar -xvf mariadb-10.1.18.tar
RUN ln -s mariadb-10.1.18 mariadb
WORKDIR /usr/bin/mariadb/
RUN mkdir install; mkdir install/data; mkdir install/var; mkdir install/etc; mkdir install/tmp
RUN cd /usr/bin/mariadb/;
RUN rm -f CMakeCache.txt;
RUN cmake \
-DCMAKE_INSTALL_PREFIX=/usr/bin/mariadb/install \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DMYSQL_DATADIR=/usr/bin/mariadb/install/data \
-DDOWNLOAD_BOOST=1 \
-DWITH_BOOST=/usr/bin/mariadb/install/boost \
-DMYSQL_UNIX_ADDR=/usr/bin/mariadb/install/tmp/mariadb.sock
And the console output:
[ 77%] Building CXX object storage/tokudb/PerconaFT/ft/CMakeFiles/ft.dir/log_print.cc.o
Linking CXX shared library libft.so
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
storage/tokudb/PerconaFT/ft/CMakeFiles/ft.dir/build.make:1323: recipe for target 'storage/tokudb/PerconaFT/ft/libft.so' failed
make[2]: *** [storage/tokudb/PerconaFT/ft/libft.so] Error 1
CMakeFiles/Makefile2:3438: recipe for target 'storage/tokudb/PerconaFT/ft/CMakeFiles/ft.dir/all' failed
make[1]: *** [storage/tokudb/PerconaFT/ft/CMakeFiles/ft.dir/all] Error 2
Makefile:147: recipe for target 'all' failed
make: *** [all] Error 2
The command '/bin/sh -c make' returned a non-zero code: 2
either change the container script and specify CMake parameter -DWITH_ZLIB=bundled to use zlib sources in extra/zlib or install zlib by apt-get install zlib-devel
I have amended a script from https://github.com/jayrambhia/Install-OpenCV/blob/master/Ubuntu/2.4/opencv2_4_10.sh to try to install OpenCV 2.4.13 onto a vm running Ubuntu 14.04 where I have sudo permission. I'm new to openCV, cmake & make so any help getting this script to work would be appreciated as I have to install it on 20 vm's.
After a few minutes of running the script returns with an error saying can't
/bin/sh: 1: cd: can't cd to /home/myaccount/setups/OpenCV/opencv-2.4.13/build
and
make[2]: *** [3rdparty/libtiff/CMakeFiles/libtiff.dir/depend] Error 2
make[1]: *** [3rdparty/libtiff/CMakeFiles/libtiff.dir/all] Error 2
The last few lines of the output is included below and the script is before that.
Any ideas why the script wont complete successfully or how to correct the permissions issue?
Extra Details
I'm not sure if these are relevant but as I dig into the problem I'll update this section
cmake version 2.8
Makerfile mentions # The shell in which to execute make rules. SHELL
= /bin/sh but my terminal reports echo $0 as bash
when I run make install V=1 it builds 100% but then reports
-- Install configuration: "RELEASE" CMake Error at cmake_install.cmake:36 (FILE): file cannot create directory: /usr/local/include/opencv2. Maybe need administrative privileges
But when I run sudo make install V=1 I get can't cd error above
Adding Shebang #!/bin/bash to the start of my script didn't solve it
Running the script on a fresh ubuntu local machine alows the script to run but I need to get it running on the hosted vm
umask of setups folder (where script runs and creates a directory is 0022 . Permissions for setups folder is 755, group = Domain Users, Owner = myaccount Thats where the source files folder gets created as well as the build folder which is also 755 after it is created
Script
arch=$(uname -m)
if [ "$arch" == "i686" -o "$arch" == "i386" -o "$arch" == "i486" -o "$arch" == "i586" ]; then
flag=1
else
flag=0
fi
echo "Installing OpenCV 2.4.13"
mkdir OpenCV
cd OpenCV
echo "Removing any pre-installed ffmpeg and x264"
sudo apt-get -y remove ffmpeg x264 libx264-dev
echo "Installing unzip"
sudo apt-get -y install unzip
echo "Installing Dependenices"
sudo apt-get -y install libopencv-dev
sudo apt-get -y install build-essential checkinstall cmake pkg-config yasm
sudo apt-get -y install libtiff4-dev libjpeg-dev libjasper-dev
sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev
sudo apt-get -y install python-dev python-numpy
sudo apt-get -y install libtbb-dev libeigen3-dev
sudo apt-get -y install libqt4-dev libgtk2.0-dev
sudo apt-get -y install libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-core-dev
sudo apt-get -y install x264 v4l-utils ffmpeg
sudo apt-get -y install libgtk2.0-dev
echo "Downloading OpenCV 2.4.13"
if ! [ -f "OpenCV-2.4.13.zip" ]; then
wget -O OpenCV-2.4.13.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.13/opencv-2.4.13.zip/download
fi
echo "Installing OpenCV 2.4.13"
if ! [ -d "opencv-2.4.13" ]; then
unzip OpenCV-2.4.13.zip
fi
rm OpenCV-2.4.13.zip
cd opencv-2.4.13
rm -rf build
mkdir build
cd build
cmake -D CUDA_ARCH_BIN=3.2 -D CUDA_ARCH_PTX=3.2 -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D BUILD_TIFF=ON -D WITH_QT=ON -D WITH_OPENGL=ON ..
make -j$(nproc)
sudo make install
sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig
echo "OpenCV 2.4.13 ready to be used"
Command Output
[100%] Build Java tests
Buildfile: /home/myaccount/setups/OpenCV/opencv-2.4.13/build/modules/java/test/.build/build.xml
build:
compile:
[mkdir] Created dir: /home/myaccount/setups/OpenCV/opencv-2.4.13/build/modules/java/test/.build/build/classes
[javac] Compiling 104 source files to /home/myaccount/setups/OpenCV/opencv-2.4.13/build/modules/java/test/.build/build/classes
jar:
[mkdir] Created dir: /home/myaccount/setups/OpenCV/opencv-2.4.13/build/modules/java/test/.build/build/jar
[jar] Building jar: /home/myaccount/setups/OpenCV/opencv-2.4.13/build/modules/java/test/.build/build/jar/opencv-test.jar
BUILD SUCCESSFUL
Total time: 6 seconds
[100%] Built target opencv_test_java
[sudo] password for myaccount:
Sorry, try again.
[sudo] password for myaccount:
/bin/sh: 1: cd: can't cd to /home/myaccount/setups/OpenCV/opencv-2.4.13/build
make[2]: *** [3rdparty/libtiff/CMakeFiles/libtiff.dir/depend] Error 2
make[1]: *** [3rdparty/libtiff/CMakeFiles/libtiff.dir/all] Error 2
make: *** [all] Error 2
OpenCV 2.4.13 ready to be used
myaccount#vm-20161023-002:~/setups$
What popped up into my head to explain the lack of permission problem: if your /home is on NFS, root may have nobody's permission on files there. There are quite a few related questions on StackExchange.
My solution would be not to try installing third-party software as root in the first place. Executing as root, your make install can do anything, including adding or overwriting files that are already managed by your software package manager. This may confuse the package manager or even render your system unusable. It's better to install the software somewhere else; /usr/local and /opt were created for this purpose. In order to make sure your installation procedure doesn't deviate from this, you can create a different user (which I call local), chown -R these directories to that user, and install using sudo -u local instead of sudoing to root. This will be fine for most installations, and should you run into one that tries to do something requiring root permissions (such as writing files to /etc or /usr/bin, restarting system services, doing tricky things with permissions, etc.), it will fail with an error message, allowing you to decide whether you want this before it has already happened. I have no idea whether OpenCV requires any such steps.
I have very little experience with docker, but I'm trying to create automated build for my library using gitlab-ci and a docker image.
I want it to be build for raspberrypi so I found an docker image (here) and I have created a github repository with Dockerfile.
In next step I have created an automated build in dockerhub, but build fails with following error:
Build failed: rpc error: code = 2 desc = "oci runtime error: exec format error"
I couldn't find any help in the documentation and also nothing on google so I'm asking:
Do you know what causes this error?
My Dockerfile:
FROM resin/rpi-raspbian:wheezy-20160518
MAINTAINER lewicki.pk#gmail.com
RUN apt-get update && apt-get install -y \
automake \
cmake \
g++ \
git \
libboost-all-dev \
libi2c-dev \
libjsoncpp-dev \
libpcre3-dev \
libssl-dev \
libtool \
make \
openssl \
pkg-config \
wget \
&& apt-get clean
And the log from docker:
Starting build of index.docker.io/piotrlewicki/gitlab-ci_rpi-image:latest...
Step 1 : FROM resin/rpi-raspbian:wheezy-20160518 ---> aefd4527968f
Step 2 : MAINTAINER lewicki.pk#gmail.com
---> Running in a463aace5c5a
---> c5eedf34e43c Removing intermediate container a463aace5c5a
Step 3 : RUN apt-get update && apt-get install -y automake cmake g++ git libboost-all-dev libi2c-dev libjsoncpp-dev libpcre3-dev libssl-dev libtool make openssl pkg-config wget && apt-get clean
---> Running in 318905fb5329 Removing intermediate container 318905fb5329
rpc error: code = 2 desc = "oci runtime error: exec format error"
It's not an issue.
Docker Hub simply does not support image builds for other architectures than x86.
Mine was ARM.
Below response from Docker support:
The image that your build is based upon,
resin/rpi-raspbian:wheezy-20160518, is an ARM-based image. Cross
architecture image builds are not supported on Docker Hub. Only
x86-based images can be built in Docker Hub/Cloud.