Adding libvips to a Rails 7 Docker image - ruby-on-rails

I'm new to Docker and trying to create a Dockerfile for this new Rails 7 app. I'm using vips instead of imagemagick for the memory benefits.
and my local machine is a mac so brew install vips takes care of my non docker development flow, but it hasn't gone so well using the ruby-vips gem, or installing from source.
Running $ docker compose up results in:
/usr/local/bundle/gems/ffi-1.15.5/lib/ffi/library.rb:145:in block in ffi_lib': Could not open library 'vips.so.42': vips.so.42: cannot open shared object file: No such file or directory. (LoadError)
With the following docker-compose.yml:
version: "3.9"
services:
db:
image: postgres
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
and a Dockerfile:
FROM ruby:3.0.1
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN gem install ruby-vips
RUN bundle install
# Add a script to be executed every time the container starts.
COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
# Configure the main process to run when running the image
CMD ["rails", "server", "-b", "0.0.0.0"]
I've also tried installing from source (https://www.libvips.org/install.html) install of using ruby-vips with no luck.

TLDR; ruby-vips need libvips42 installed on your docker image.
Update your Dockerfile to use the following:
RUN apt-get update -qq && apt-get install -y --no-install-recommends nodejs postgresql-client libvips42
PS: run docker compose down and docker compose up --build to force a rebuild of your docker images.

I don't think you're actually installing libvips in your dockerfile. Try this:
FROM ruby:3.0.1
RUN apt-get update -qq \
&& apt-get install -y nodejs postgresql-client
RUN apt install -y --no-install-recommends libvips42
WORKDIR /myapp
...
However, this will install the libvips that comes with buster, and it's 8.7.x from five years ago (!!). Debian does not move quickly.
I would build current libvips from source. Something like this:
# based on buster
FROM ruby:3.0.1
RUN apt-get update && apt-get install -y \
build-essential \
unzip \
wget \
git \
pkg-config
# stuff we need to build our own libvips ... this is a pretty random selection
# of dependencies, you'll want to adjust these
RUN apt-get install -y \
glib-2.0-dev \
libexpat-dev \
librsvg2-dev \
libpng-dev \
libgif-dev \
libjpeg-dev \
libexif-dev \
liblcms2-dev \
liborc-dev
ARG VIPS_VERSION=8.12.2
ARG VIPS_URL=https://github.com/libvips/libvips/releases/download
RUN apt-get install -y \
wget
RUN cd /usr/local/src \
&& wget ${VIPS_URL}/v${VIPS_VERSION}/vips-${VIPS_VERSION}.tar.gz \
&& tar xzf vips-${VIPS_VERSION}.tar.gz \
&& cd vips-${VIPS_VERSION} \
&& ./configure --disable-deprecated \
&& make -j 4 V=0 \
&& make install
RUN gem install ruby-vips
That won't include support for GIF save, or for formats like HEIC or PDF. You'll probably want to adjust it a bit. And of course you should not build packages in your deployment docker image, you'd want to do that in a separate dockerfile.
Hopefully ruby-vips deployment will become more automated over the next few months now that's it's the rail7 default. It's rather manual right now.

Related

Docker runs on WIndows and only on one of two Linux systems

I have a docker image that I have built that runs on my windows laptop as expected. When I copy and load it on to one of my two Linux systems I get this error when I run docker logs:
Error: 'docker/semantic_search_django/gunicorn.conf' doesn't exist
When I inspect the running container on Windows I can see that "missing" file! Furthermore, if I copy and load the same docker image to my second Linux system, it runs as expected.
This issue just happened today. I've been having success on all 3 systems for the past couple of months until today. Any suggestions would be greatly appreciated. Both Linux systems are running Ubuntu 18.04.5 LTS.
I've tried renamed the images, I've stopped and started the docker daemon, I've even restarted both Linux boxes.
Here are the commands I have used:
docker pull my.artifactory.com/ciee_ssrdjango
docker-compose up -d
My docker-compose.yml
version: "3.8"
services:
web:
image: m.artifactory.com/ciee_ssrdjango
env_file:
- proxy.env
- django.env
container_name: ciee_ssrdjango
volumes:
- query-results-volume:/code
expose:
- "${SSRDJANGO_PORT}"
extra_hosts:
dbhost: ${POSTGRES_DOCKER_IP}
depends_on:
- db
networks:
- ssr_network
networks:
ssr_network:
external: true
volumes:
postgresql-volume:
external: true
query-results-volume:
external: true
My Dockerfile:
FROM ubuntu:18.04
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
COPY ./requirements.txt /requirements.txt
#prevents being asked to set TZ
ARG DEBIAN_FRONTEND=noninteractive
RUN apt update -y && \
apt -y upgrade && \
apt install -y python3-pip && \
apt install -y build-essential libssl-dev libffi-dev libpq-dev python3-dev && \
apt install -y software-properties-common python3.8
RUN python3 -m pip install --upgrade pip setuptools wheel
ENV TZ=US/Eastern
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update -y & apt install gcc libxml2-dev libxslt-dev postgresql postgresql-contrib postgresql-plpython-10 --no-install-recommends unixodbc-dev unixodbc libpq-dev -y
RUN mkdir /code # && mkdir /code/ciee
RUN pip install nltk
RUN export PATH=~/.local/bin:$PATH
RUN pip install -r /requirements.txt
COPY . /code/
WORKDIR /code
RUN useradd -m user && chmod 777 /home/user && mkdir /code/query_results && chmod 777 /code/query_results
USER user
CMD ["gunicorn", "semantic_search_django.wsgi:application", "--config", "docker/semantic_search_django/gunicorn.conf", "--keep-alive", "600"]
Here's the thing, I've been using these files and commands successfully for many weeks.
I can make one assumption. You are mounting query-results-volume into /code directory in container and your conf file is located inside it. The volume persists between containers – that's the nature of the volumes. So, somehow, the file in question (or even the folder) has been removed from the volume on the problem machine and now container can not get it.

Generating PHP library with Dockerized gRPC

I'm trying to build a gRPC PHP Client and gRPC NodeJs Server in docker. But the problem is I can't install protoc-gen-php-grpc to my docker server. When I try to run this run this makefile:
proto_from_within_container:
# PHP
protoc /var/www/protos/smellycat.proto \
--php_out=/var/www/php-client/src \
$(: 👇 generate server interface) \
--php-grpc_out=/var/www/php-client/src \
$(: 👇 generates the client code) \
--grpc_out=/var/www/php-client/src \
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin \
--proto_path /var/www/protos
proto:
powershell rm -r -fo php-client/src -ErrorAction SilentlyContinue
powershell New-Item -ItemType Directory -Path php-client/src -Force -ErrorAction SilentlyContinue
docker-compose run grpc-server make proto_from_within_container
With this command: make proto
Getting this error message after docker containers builded:
protoc /var/www/protos/smellycat.proto \
--php_out=/var/www/php-client/src \
\
--php-grpc_out=/var/www/php-client/src \
\
--grpc_out=/var/www/php-client/src \
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin \
--proto_path /var/www/protos
protoc-gen-php-grpc: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--php-grpc_out: protoc-gen-php-grpc: Plugin failed with status code 1.
Makefile:4: recipe for target 'proto_from_within_container' failed
make: *** [proto_from_within_container] Error 1
This is my docker-compose file
version: "3"
services:
grpc-server:
container_name: grpc-server
build:
context: .
dockerfile: Dockerfile-server
working_dir: /var/www
volumes:
- .:/var/www
grpc-client:
image: php:7.4-cli
container_name: grpc-client
build:
context: .
dockerfile: Dockerfile-client
working_dir: /var/www
volumes:
- .:/var/www
command: bash -c [php php_client.php && composer install]
And this is my grpc-server docker file:
FROM node:latest
ENV DEBIAN_FRONTEND=noninteractive
#Versions
ARG PROTOBUF_VERSION=3.14.0
ARG PHP_GRPC_VERSION=1.34.0
# Utils
RUN apt-get update -yqq \
&& apt-get install -yqq wget unzip zlib1g-dev git autoconf libtool automake build-essential software-properties-common curl zip \
&& rm -rf /var/lib/apt/lists/*
# Protobuf
RUN mkdir -p /protobuf
RUN cd /protobuf \
&& wget https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip -O protobuf.zip \
&& unzip protobuf.zip && rm protobuf.zip
# grpc PHP (generate client)
RUN apt-get update -yqq && apt-get upgrade -yqq
RUN apt-get install php php-dev php-pear phpunit zlib1g-dev -yqq
RUN pecl install grpc-${PHP_GRPC_VERSION}
RUN cd /protobuf && git clone -b v${PHP_GRPC_VERSION} https://github.com/grpc/grpc \
&& cd /protobuf/grpc && git submodule update --init
RUN cd /protobuf/grpc && make grpc_php_plugin
ENV PATH "/protobuf/bin:${PATH}"
ENV PATH "/protobuf/grpc/bins/opt:${PATH}"
# NPM Installation
WORKDIR /var/www
COPY . /var/www
RUN npm install
CMD ["node", "server.js"]
Do you have any advice?
After a lot of search and readings, I finally managed to build a full application that communicates with each other.
The problem was at the Makefile, at this step:
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin
I was assigning the wrong path for grpc_php_plugin.
There is my new dockerfile:
FROM php:7.4-cli
# Environment variables
ENV DEBIAN_FRONTEND=noninteractive
# Utils
RUN apt-get update -yqq && \
apt-get upgrade -yqq && \
apt-get install -y unzip build-essential git software-properties-common curl pkg-config zip zlib1g-dev
# Composer installation
COPY --from=composer:latest /usr/bin/composer /usr/local/bin/composer
# Install grpc and probuf with pecl
RUN pecl install grpc && pecl install protobuf
# Enable grpc and protobuf extensions in php.ini file
RUN echo starting && \
docker-php-ext-enable grpc && \
docker-php-ext-enable protobuf
# Install cmake
RUN apt-get update -yqq && apt-get -y install cmake
# Install grpc_php_plugin and protoc
RUN git clone -b v1.36.2 https://github.com/grpc/grpc && \
cd grpc && git submodule update --init && \
mkdir cmake/build && cd cmake/build && \
cmake ../.. && make protoc grpc_php_plugin
# Setting node, protoc and grpc_php_plugin paths
ENV PATH "/grpc/cmake/build:${PATH}"
ENV PATH "/grpc/cmake/build/third_party/protobuf:${PATH}"
# Moving client folder to vm
WORKDIR /var/www
COPY ./client /var/www
# Packages
RUN composer install
# Generate php libraries from proto file
RUN make proto
CMD [ "php", "./handler.php" ]
For my full application, click.

Speed up image building docker with docker compose

i am a newby in docker and I need an installation based on ubuntu of logstash.
I try the official logstash image and without success I can't run it so I decided to build my own installation based on my needs.
It works well but takes a lot of time to build it.
I wonder how can I improve (speed up) my building
This is my Dockerfile
FROM ubuntu:18.04
# RUN adduser --disabled-password --gecos "" ubuntu
# RUN usermod -aG sudo ubuntu
# RUN echo "ubuntu ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN apt-get update && \
apt-get autoclean && \
apt-get autoremove
RUN echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-7.x.list
RUN apt-get install curl ca-certificates apt-utils wget apt-transport-https default-jre gnupg apt-transport-https software-properties-common -y
# RUN update-alternatives --config java
ENV LANG C.UTF-8
ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
ENV PATH $JAVA_HOME/bin:$PATH
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
RUN apt update
RUN apt install ruby-full -y
RUN ruby --version
RUN apt install jruby -y
#install jruby (last version)
RUN jruby --version
RUN apt install logstash
#install plugins and bundles.
RUN cd /usr/share/logstash && gem install bundler
COPY logstash-pcap-test.conf /usr/share/logstash/
RUN mkdir /home/logstash/ && mkdir /home/logstash/traffic
COPY /traffic-example/* /home/logstash/traffic/
WORKDIR /usr/share/logstash
CMD ["/bin/bash","-c","bin/logstash -f logstash-pcap-test.conf --config.reload.automatic"]
And this is my docker-compose
version: "3"
services:
logstash_test:
build:
context: .
dockerfile: container/Dockerfile
image: logstash_test:latest
container_name: logstash_test
hostname: logstash_test
ports:
- 9600:9600
- 8089:8089
networks:
- elknetwork
networks:
elknetwork:
driver: bridge
Any thoughts?

How to map the host OS file to the container at the time of container build process

docker-compose.yml:
version: '3'
services:
ezmove:
volumes:
- /host-dir:/home/container-dir
build:
context: .
args:
BRANCH: develop
Dockerfile:
FROM appcontainers/ubuntu:xenial
MAINTAINER user <user>
RUN apt-get update -y --no-install-recommends \
&& apt-get install -y --no-install-recommends python3.5-minimal python3.5-venv \
&& apt-get install -y --no-install-recommends git \
&& apt-get install -y --no-install-recommends python-pip \
&& pip install --upgrade pip \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /home/container-dir
WORKDIR /home/container-dir
RUN /bin/bash - c "sh ./script.sh"
At the time of build the docker container How to map local directory to container
When $ docker-compose up, it will starts to build container but after installation of the packag dependancies it will try to execute the script.sh file but got error "FILE NOT FOUND! "
Tried:
Not want todo git clone inside docker continer
Not want to store source code inside the continer
So, how to map the host OS file to the container at build time
you lack some COPY or ADD in your Dockerfile in order to copy your script.sh in your image.
Check the docs
https://docs.docker.com/engine/reference/builder/#add
https://docs.docker.com/engine/reference/builder/#copy
By the way, Docker is about isolation, so a running container should be isolated from the host, and certainly not access the host OS.

How to save my installations on Ubuntu image into Docker

Docker Codes
# Import Ubuntu image to Docker
docker pull ubuntu:16.04
docker run -it ubuntu:16.04
# Instsall Python3 and pip3
apt-get update
apt-get install -y python3 python3-pip
# Install Selenium
pip3 install selenium
# Install BeautifulSoup4
pip3 install beautifulsoup4
# Install library for PhantomJS
apt-get install -y wget libfontconfig
# Downloading and installing binary
mkdir -p /home/root/src && cd &_
tar jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2
cd phantomjs-2.1.1-linux-x86_64/bin/
cp phantomjs /usr/local/bin/
# Installing font
apt-get install -y fonts-nanum*
Question
I am trying to import Ubuntu image to docker and install serveral packages inscluding python3, pip3, bs4, and PhantomJs. Then I want to save all this configurations in Docker as "ubuntu-phantomjs". As I am currently on Ubuntu image, anything that starts with 'docker' command do not work. How could I save my image?
Here is the dockerfile:
# Import Ubuntu image to Docker
FROM ubuntu:16.04
# Install Python3, pip3, library and fonts
RUN apt-get update && apt-get install -y \
python3 \
python3-pip \
wget libfontconfig \
fonts-nanum*
&& rm -rf /var/lib/apt/lists/*
RUN pip3 install selenium beautifulsoup4
# Downloading and installing binary
RUN mkdir -p /home/root/src && cd &_ tar jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2 && cd phantomjs-2.1.1-linux-x86_64/bin/ && cp phantomjs /usr/local/bin/
Now after saving the code in file named dockerfile, open a terminal in the same directory as the one where file is stored, and run following command:
$ docker build -t ubuntu-phantomjs .
-t means that the target is ubuntu-phantomjs and . means that the context for docker is the current directory. The above dockerfile is not a standard one, and does not follow all good practices mentioned here. You can change this file according to your needs, read the documentations for more help.

Resources