I need to build a docker image of fio with librbd so that I can set the ioengine=rbd. I am building for arch aarch64. When I run fio with ioengine=rbd I get an error saying unable to load librbd. Any help here really appreciated.
Here is my Dockerfile:
# syntax=docker/dockerfile:1
From alpine
RUN apk add --update-cache fio librbd --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
I build and push my docker image:
$ sudo docker build . -t manjo8/fio
Sending build context to Docker daemon 2.048kB
Step 1/2 : From alpine
---> bb3de5531c18
Step 2/2 : RUN apk add --update-cache fio librbd --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
---> Running in 68e44788feb0
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/19) Installing libaio (0.3.112-r1)
(2/19) Installing numactl (2.0.14-r0)
(3/19) Installing fio (3.28-r1)
(4/19) Installing libblkid (2.37.2-r0)
(5/19) Installing libbz2 (1.0.8-r1)
(6/19) Installing libgcc (10.3.1_git20210424-r2)
(7/19) Installing xz-libs (5.2.5-r0)
(8/19) Installing boost1.76-iostreams (1.76.0-r0)
(9/19) Installing boost1.76-thread (1.76.0-r0)
(10/19) Installing libstdc++ (10.3.1_git20210424-r2)
(11/19) Installing fmt (7.1.3-r1)
(12/19) Installing eudev-libs (3.2.10-r0)
(13/19) Installing librados (16.2.6-r1)
(14/19) Installing argon2-libs (20190702-r1)
(15/19) Installing device-mapper-libs (2.02.187-r1)
(16/19) Installing json-c (0.15-r1)
(17/19) Installing libuuid (2.37.2-r0)
(18/19) Installing cryptsetup-libs (2.3.6-r0)
(19/19) Installing librbd (16.2.6-r6)
Executing busybox-1.33.1-r3.trigger
OK: 26 MiB in 33 packages
Removing intermediate container 68e44788feb0
---> 1534fb8d9308
Successfully built 1534fb8d9308
Successfully tagged manjo8/fio:latest
When I run the docker image I get an error saying its unable to load librdb.
fio-5c6dc65894-bh7dt
fio: file:ioengines.c:127, func=dlopen, error=Error loading shared library rbd: No such file or directory
fio: engine rbd not loadable
fio: failed to load engine
If you look at the build log for fio, it says that it is building fio without support for Rados Block Device engine:
Rados Block Device engine no
So, the only thing to do is to build it from source. Using the build script from alpine, plus some googling for how to build fio with rbd support, I came up with this:
From alpine
ENV fio_version=3.28
RUN set -vuo pipefail \
&& apk add --no-cache libaio-dev librados librbd \
&& apk add --no-cache --virtual .build-deps linux-headers zlib-dev coreutils gcc make musl-dev ceph-dev \
&& cd /tmp \
&& wget https://github.com/axboe/fio/archive/fio-$fio_version.tar.gz \
&& tar -xzf fio-$fio_version.tar.gz \
&& rm /tmp/fio-$fio_version.tar.gz \
&& cd /tmp/fio-fio-$fio_version \
&& ./configure \
--prefix=/usr \
--disable-native \
&& make -j \
&& make install \
&& rm -rf /tmp/fio-fio-$fio_version \
&& apk del --no-network .build-deps
I'm only able to test it in a limited way, given that I don't have a ceph cluster, but I'm able to get past the error you were having.
I also spent some effort optimizing the build so it wouldn't waste space by including build-time dependencies into the image. (This is the --virtual .build-deps and apk del .build-deps stages you see above.)
Related
I am trying with https://github.com/d4rkstar/kong-konga-keycloak when I run sudo docker-compose up -d kong it send me:
The command '/bin/sh -c luarocks install --pin lua-resty-jwt' returned a non-zero code: 132
Watching dockerfile it contains:
FROM kong:2.0.4-alpine
LABEL description="Alpine + Kong 2.0.4 + kong-oidc plugin"
USER root
RUN apk update && apk add git unzip luarocks
RUN luarocks install --pin lua-resty-jwt 0.2.2-0
RUN luarocks install kong-oidc
USER kong
This is the output when I install manually:
/ # apk add luarocks
(1/2) Installing curl (7.79.1-r0)
(2/2) Installing luarocks (2.4.4-r1)
Executing busybox-1.31.1-r9.trigger
OK: 67 MiB in 38 packages
/ # luarocks
Illegal instruction (core dumped)
My platform is: ubuntu 20 where I am running docker 20.10.10 and docker-compose 1.29.2.
2 days after
I replaced apk add luarocks by ( How to install gdbserver package on Alpine Docker image? )
RUN apk add --no-cache make
RUN apk add --no-cache linux-headers
RUN apk add --no-cache texinfo
RUN apk add --no-cache gcc
RUN apk add --no-cache g++
RUN apk add --no-cache gfortran
# install gdb
# RUN apk add --no-cache gdb
RUN mkdir gdb-build ;\
cd gdb-build;\
wget http://ftp.gnu.org/gnu/gdb/gdb-7.11.tar.xz;\
tar -xvf gdb-7.11.tar.xz;\
cd gdb-7.11;\
./configure --prefix=/usr;\
make;\
make -C gdb install;\
cd ..;\
rm -rf gdb-build/;
then now I do not get error while I build.
I get the same error when I tried to migrate:
docker-compose run --rm kong kong migrations bootstrap
and kong never start, but now luarocks works fine inside container.
Thank you
I faced with range case, when I was try to install some bundle in locally I faced with erro, but when I try install the same bundle in test server everything installed without error. I use docker-compose and install bundle inside in image. docker-copose and other docker file with all dependecies absolutelty the same, all in git.
composer require league/flysystem-bundle
and in locally I faced with that
Using version dev-master for league/flysystem-bundle
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Restricting packages listed in "symfony/symfony" to "5.0.*"
Installation failed, reverting ./composer.json to its original content.
[RuntimeException]
Could not load package ezsystems/ezplatform in http://repo.packagist.org: [
UnexpectedValueException] Could not parse version constraint dev-load-varni
sh-only-when-used as ^2.0#dev: Invalid version string "^2.0#dev"
[UnexpectedValueException]
Could not parse version constraint dev-load-varnish-only-when-used as ^2.0#
dev: Invalid version string "^2.0#dev"
locally composer version Composer version 1.10.11 2020-09-08 16:53:44
and test server
/var/www/symfony # composer require league/flysystem-bundle
Using version dev-master for league/flysystem-bundle
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Restricting packages listed in "symfony/symfony" to "5.0.*"
Prefetching 3 packages 🎶 💨
- Downloading (100%)
Package operations: 3 installs, 0 updates, 0 removals
- Installing league/mime-type-detection (1.4.0): Loading from cache
- Installing league/flysystem (1.x-dev 53f16fd): Loading from cache
- Installing league/flysystem-bundle (dev-master 525845a): Loading from cache
Package easycorp/easy-log-handler is abandoned, you should avoid using it. No replacement was suggested.
Package zendframework/zend-code is abandoned, you should avoid using it. Use laminas/laminas-code instead.
Package zendframework/zend-eventmanager is abandoned, you should avoid using it. Use laminas/laminas-eventmanager instead.
Writing lock file
Generating autoload files
20 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
Symfony operations: 1 recipe (c67222ac592a52b7dec1c2cd56763685)
- WARNING league/flysystem-bundle (>=1.0): From github.com/symfony/recipes-contrib:master
The recipe for this package comes from the "contrib" repository, which is open to community contributions.
Review the recipe at https://github.com/symfony/recipes-contrib/tree/master/league/flysystem-bundle/1.0
Do you want to execute this recipe?
[y] Yes
[n] No
[a] Yes for all packages, only for the current installation session
[p] Yes permanently, never ask again for this project
(defaults to n):
ocramius/package-versions: Generating version class...
ocramius/package-versions: ...done generating version class
Executing script cache:clear [OK]
Executing script assets:install public [OK]
test server composer version Composer version 1.10.10 2020-08-03 11:35:19
my dockerfile
FROM alpine:edge
LABEL maintainer="Vincent Composieux <vincent.composieux#gmail.com>"
RUN apk add --update --no-cache \
coreutils \
yarn \
php7-fpm \
php7-apcu \
php7-ctype \
php7-curl \
php7-dom \
php7-gd \
php7-iconv \
php7-imagick \
php7-json \
php7-intl \
php7-mcrypt \
php7-fileinfo\
php7-mbstring \
php7-opcache \
php7-openssl \
php7-pdo \
php7-pdo_mysql \
php7-mysqli \
php7-pdo_pgsql \
php7-pgsql \
php7-xml \
php7-zlib \
php7-phar \
php7-tokenizer \
php7-session \
php7-simplexml \
php7-xdebug \
php7-zip \
php7-xmlwriter \
make \
curl \
zlib-dev \
libxml2-dev \
rabbitmq-c-dev \
oniguruma-dev \
php7-pecl-amqp \
php7-amqp \
php7-redis
RUN apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/ php7-pecl-mongodb
RUN echo "$(curl -sS https://composer.github.io/installer.sig) -" > composer-setup.php.sig \
&& curl -sS https://getcomposer.org/installer | tee composer-setup.php | sha384sum -c composer-setup.php.sig \
&& php composer-setup.php && rm composer-setup.php* \
&& chmod +x composer.phar && mv composer.phar /usr/bin/composer
COPY symfony.ini /etc/php7/conf.d/
COPY symfony.ini /etc/php7/cli/conf.d/
COPY xdebug.ini /etc/php7/conf.d/
COPY symfony.pool.conf /etc/php7/php-fpm.d/
CMD ["php-fpm7", "-F"]
WORKDIR /var/www/symfony
EXPOSE 9001
Why in the same time I faced with differnt version composer. Composer installed by the same way, by the same Dockerfile. How to fix this problem ?
I don't belive, how it's possible, this problem don't should be appear when using docker structure.. ?
Looks like composer 1.10.11 is broken. You can switch to 1.10.10 like this:
composer-setup.php --version=1.10.10
Confirmed that error from composer 1.10.11. You should down to composer v1.10.10.
composer self-update 1.10.10
You can use self-update to downgrade composer version
Now you can update your composer version to 1.10.12
In this version you don't have this error.
I have been requested to move an Elixir/Phoenix app to Docker, with which I have no prior experience. The app uses non-latest versions of Elixir and Phoenix so I have had to diverge from the code online which generally focuses on latest versions. That led me to write this Dockerfile
# FROM bitwalker/alpine-elixir:latest
FROM bitwalker/alpine-elixir:1.3.4
MAINTAINER Paul Schoenfelder <paulschoenfelder#gmail.com>
# Important! Update this no-op ENV variable when this Dockerfile
# is updated with the current date. It will force refresh of all
# of the base images and things like `apt-get update` won't be using
# old cached versions when the Dockerfile is built.
ENV REFRESHED_AT=2017-07-26 \
# Set this so that CTRL+G works properly
TERM=xterm
# Install NPM
RUN \
mkdir -p /opt/app && \
chmod -R 777 /opt/app && \
apk update && \
apk --no-cache --update add \
git make g++ wget curl inotify-tools \
nodejs nodejs-current-npm && \
npm install npm -g --no-progress && \
update-ca-certificates --fresh && \
rm -rf /var/cache/apk/*
# Add local node module binaries to PATH
ENV PATH=./node_modules/.bin:$PATH \
HOME=/opt/app
# Install Hex+Rebar
RUN mix local.hex --force && \
mix local.rebar --force
WORKDIR /opt/app
CMD ["/bin/sh"]
<then it goes on to add some elixir depedencies>
On running
sudo docker build -t phoenix .
I'm ending up with this error and wondering how to get around it? Noting 'current' in the title I'm wondering whether using an older version of nodejs, and if so, how to do that? Beyond that I am open to any and all suggestions
ERROR: unsatisfiable constraints:
nodejs-current-npm (missing):
required by: world[nodejs-current-npm]
musl-1.1.14-r14:
breaks: musl-dev-1.1.14-r15[musl=1.1.14-r15]
That looks like bitwalker/alpine-elixir issue 5:
when using tagged images, you may sometimes need to explicitly upgrade packages, as the installed packages are at the versions found when building the image.
Generally it's as simple as adding apk --update upgrade before any commands which install packages.
Indeed, when you compare the old elixir 1.4.4-based Dockerfile, and the latest one, you will see an upgrade first in the latter:
apk --no-cache --update upgrade && \
apk add --no-cache --update --virtual .elixir-build \
...
Try and add that to your Dockerfile.
I'm trying to run a simple python app that communicates with kafka. I'm looking to use an alpine container for it. Here's my current dockerfile (it's not optimal... just trying to get things working for now).
FROM python:3.6-alpine
MAINTAINER Ashic Mahtab (ashic#live.com)
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk update && apk --no-cache add librdkafka
COPY requirements.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements.txt
COPY api /usr/src/app/api
COPY static /usr/src/app/static
CMD ["python", "api/index.py"]
The requirements file has confluent-kafka in it. The build fails with
OK: 8784 distinct packages available
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
so:libcrypto.so.41 (missing):
required by:
librdkafka-0.9.4-r1[so:libcrypto.so.41]
librdkafka-0.9.4-r1[so:libcrypto.so.41]
librdkafka-0.9.4-r1[so:libcrypto.so.41]
so:libssl.so.43 (missing):
required by:
librdkafka-0.9.4-r1[so:libssl.so.43]
librdkafka-0.9.4-r1[so:libssl.so.43]
librdkafka-0.9.4-r1[so:libssl.so.43]
My questions are a) is there a way to get this working without building inside the container? It'd be good enough if I could simply copy the library over to alpine. Or even if I could copy librdkafka over. b) If not, how can I get libssl and libcryto.so working?
here a way to make it work:
FROM python:3.6-alpine
MAINTAINER Ashic Mahtab (ashic#live.com)
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
RUN sed -i -e 's/v3\.4/edge/g' /etc/apk/repositories \
&& apk upgrade --update-cache --available \
&& apk --no-cache add librdkafka
COPY requirements.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements.txt
COPY api /usr/src/app/api
COPY static /usr/src/app/static
CMD ["python", "api/index.py"]
Here an explanation:
python:3.6-alpine image are based on alpine linux 3.4: Dockerfile
So you first need to correctly move to edge alpine branch: Edge
This is done by the line:
RUN sed -i -e 's/v3\.4/edge/g' /etc/apk/repositories \
&& apk upgrade --update-cache --available \
Then you can install librdkafka and the dependencies.
I maintain the ucalgary/python-librdkafka image that extends the official python:3.6-alpine image and installs librdkafka from its source releases. You're welcome to use the image, or take a look at the Dockerfile to see how it's built.
Since you're on edge anyway, you could also just use python3 from there:
% cat Dockerfile
FROM alpine:edge
MAINTAINER Gerd Flaig <stackoverflow#gerd.flaig.name>
RUN apk update && apk add --no-cache python3 librdkafka && \
python3 -m ensurepip && \
rm -r /usr/lib/python*/ensurepip && \
pip3 install --upgrade pip setuptools && \
rm -r /root/.cache
% docker build .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM alpine:edge
edge: Pulling from library/alpine
71c5a0cc58e4: Pull complete
Digest: sha256:99588bc8883c955c157d18fc3eaa4a3c1400c223e6c7cabca5f600a3e9f8d5cd
Status: Downloaded newer image for alpine:edge
---> 8914de95a28d
Step 2/3 : MAINTAINER Gerd Flaig <stackoverflow#gerd.flaig.name>
---> Running in 658b056c4e16
---> a3b2485fabb0
Removing intermediate container 658b056c4e16
Step 3/3 : RUN apk update && apk add --no-cache python3 librdkafka && python3 -m ensurepip && rm -r /usr/lib/python*/ensurepip && pip3 install --upgrade pip setuptools && rm -r /root/.cache
---> Running in d7c3f3b30d89
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
v3.5.0-4449-g8922925e5c [http://dl-cdn.alpinelinux.org/alpine/edge/main]
v3.5.0-4450-gb7b4122d6f [http://dl-cdn.alpinelinux.org/alpine/edge/community]
OK: 8348 distinct packages available
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
(1/16) Installing libressl2.5-libcrypto (2.5.3-r1)
(2/16) Installing libgcc (6.3.0-r3)
(3/16) Installing libressl2.5-libssl (2.5.3-r1)
(4/16) Installing libstdc++ (6.3.0-r3)
(5/16) Installing librdkafka (0.9.4-r1)
(6/16) Installing libbz2 (1.0.6-r5)
(7/16) Installing expat (2.2.0-r0)
(8/16) Installing libffi (3.2.1-r3)
(9/16) Installing gdbm (1.12-r0)
(10/16) Installing xz-libs (5.2.3-r0)
(11/16) Installing ncurses-terminfo-base (6.0-r7)
(12/16) Installing ncurses-terminfo (6.0-r7)
(13/16) Installing ncurses-libs (6.0-r7)
(14/16) Installing readline (6.3.008-r5)
(15/16) Installing sqlite-libs (3.18.0-r0)
(16/16) Installing python3 (3.6.1-r1)
Executing busybox-1.26.2-r0.trigger
OK: 75 MiB in 27 packages
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/lib/python3.6/site-packages
Requirement already up-to-date: pip in /usr/lib/python3.6/site-packages
Collecting setuptools
Downloading setuptools-35.0.2-py2.py3-none-any.whl (390kB)
Collecting packaging>=16.8 (from setuptools)
Downloading packaging-16.8-py2.py3-none-any.whl
Collecting appdirs>=1.4.0 (from setuptools)
Downloading appdirs-1.4.3-py2.py3-none-any.whl
Collecting six>=1.6.0 (from setuptools)
Downloading six-1.10.0-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools)
Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
Installing collected packages: pyparsing, six, packaging, appdirs, setuptools
Found existing installation: setuptools 28.8.0
Uninstalling setuptools-28.8.0:
Successfully uninstalled setuptools-28.8.0
Successfully installed appdirs-1.4.3 packaging-16.8 pyparsing-2.2.0 setuptools-35.0.2 six-1.10.0
---> cfb1033ceec0
Removing intermediate container d7c3f3b30d89
Successfully built cfb1033ceec0
I have very little experience with docker, but I'm trying to create automated build for my library using gitlab-ci and a docker image.
I want it to be build for raspberrypi so I found an docker image (here) and I have created a github repository with Dockerfile.
In next step I have created an automated build in dockerhub, but build fails with following error:
Build failed: rpc error: code = 2 desc = "oci runtime error: exec format error"
I couldn't find any help in the documentation and also nothing on google so I'm asking:
Do you know what causes this error?
My Dockerfile:
FROM resin/rpi-raspbian:wheezy-20160518
MAINTAINER lewicki.pk#gmail.com
RUN apt-get update && apt-get install -y \
automake \
cmake \
g++ \
git \
libboost-all-dev \
libi2c-dev \
libjsoncpp-dev \
libpcre3-dev \
libssl-dev \
libtool \
make \
openssl \
pkg-config \
wget \
&& apt-get clean
And the log from docker:
Starting build of index.docker.io/piotrlewicki/gitlab-ci_rpi-image:latest...
Step 1 : FROM resin/rpi-raspbian:wheezy-20160518 ---> aefd4527968f
Step 2 : MAINTAINER lewicki.pk#gmail.com
---> Running in a463aace5c5a
---> c5eedf34e43c Removing intermediate container a463aace5c5a
Step 3 : RUN apt-get update && apt-get install -y automake cmake g++ git libboost-all-dev libi2c-dev libjsoncpp-dev libpcre3-dev libssl-dev libtool make openssl pkg-config wget && apt-get clean
---> Running in 318905fb5329 Removing intermediate container 318905fb5329
rpc error: code = 2 desc = "oci runtime error: exec format error"
It's not an issue.
Docker Hub simply does not support image builds for other architectures than x86.
Mine was ARM.
Below response from Docker support:
The image that your build is based upon,
resin/rpi-raspbian:wheezy-20160518, is an ARM-based image. Cross
architecture image builds are not supported on Docker Hub. Only
x86-based images can be built in Docker Hub/Cloud.