After following the tutorial at: https://devcenter.heroku.com/articles/build-docker-images-heroku-yml it was not clear which contents we should put in the Dockerfile.
For a Rails app what do we need in our Dockerfile?
There is a Rails 5 example that you can base your solution: https://github.com/jahangiranwari/rails5-docker-heroku/
For a Rails 6 project I've used the following Dockerfile:
FROM ruby:2.6.6
# Install node & yarn
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get install -y nodejs
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -y yarn
# Install base deps or additional (e.g. tesseract)
ARG INSTALL_DEPENDENCIES
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends ${INSTALL_DEPENDENCIES} \
build-essential libpq-dev git \
&& apt-get clean autoclean \
&& apt-get autoremove -y \
&& rm -rf \
/var/lib/apt \
/var/lib/dpkg \
/var/lib/cache \
/var/lib/log
# Install deps with bundler
RUN mkdir /app
WORKDIR /app
COPY Gemfile* /app/
ARG BUNDLE_INSTALL_ARGS
RUN gem install bundler:2.1.4
RUN bundle config set without 'development test'
RUN bundle install ${BUNDLE_INSTALL_ARGS} \
&& rm -rf /usr/local/bundle/cache/* \
&& find /usr/local/bundle/gems/ -name "*.c" -delete \
&& find /usr/local/bundle/gems/ -name "*.o" -delete
COPY . /app/
# Compile assets
ARG RAILS_ENV=development
RUN if [ "$RAILS_ENV" = "production" ]; then SECRET_KEY_BASE=$(rake secret) bundle exec rake assets:precompile; fi
And this heroku.yml file:
build:
docker:
web: Dockerfile
config:
BUNDLE_INSTALL_ARGS: --jobs 10 --retry=3
RAILS_ENV: production
# Put extra deps here
INSTALL_DEPENDENCIES: curl openssh-server python
run:
web: bundle exec puma -C config/puma.rb
Update 20-1-22:
It works for Rails 7
Related
I am trying to run a nextflow pipeline which uses an older version of nextflow (21.04.3) and java version 8. Since I have to use this pipeline on a remote server, therefore I can only use singularity.
As this nextflow pipeline also uses singularity pull calls therefore I need the singularity installed inside the docker image as well. Then, I can convert this image docker image to a singularity image and then I can move it to the remote server.
I am trying to install singularity inside dockerfile but I am getting errors,
This is the dockerfile that I am using,
FROM python:3.8.9-slim
LABEL authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
RUN apt-get install -y singularity-container
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow- io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
These are the errors I am getting
Step 9/17 : RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee
/etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --
keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
---> Running in afc3dcbbd1ee
--2022-03-17 17:40:19-- http://neuro.debian.net/lists/xenial.us-ca.full
Resolving neuro.debian.net (neuro.debian.net)... 129.170.233.11
Connecting to neuro.debian.net (neuro.debian.net)|129.170.233.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 262
Saving to: ‘STDOUT’
0K 100% 18.4M=0s
deb http://neurodeb.pirsquared.org data main contrib non-free
#deb-src http://neurodeb.pirsquared.org data main contrib non-free
deb http://neurodeb.pirsquared.org xenial main contrib non-free
#deb-src http://neurodeb.pirsquared.org xenial main contrib non-free
2022-03-17 17:40:19 (18.4 MB/s) - written to stdout [262/262]
/bin/sh: 1: apt-key: not found
The command '/bin/sh -c wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update'
returned a non-zero code: 127
I there a way to install singularity using a dockerfile ?
Thanks
I made some changes in the dockerfile based on the method to install singularity in linux given here.
The complete dockerfile with which I was able to run successfully nextflow, java and singularity within singularity is given below,
FROM python:3.8.9-slim
LABEL
authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN apt-get update && apt-get install -y \
build-essential \
libssl-dev \
uuid-dev \
libgpgme11-dev \
squashfs-tools \
libseccomp-dev \
wget \
pkg-config \
procps
# Download Go source version 1.16.3, install them and modify the PATH
ENV VERSION=1.16.3
ENV OS=linux
ENV ARCH=amd64
RUN wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz && \
echo 'export PATH=$PATH:/usr/local/go/bin' | tee -a /etc/profile
# Download Singularity from version 3.7.3 (security version)
ENV VERSION=3.7.3
RUN wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz
# Compile Singularity sources and install it
RUN export PATH=$PATH:/usr/local/go/bin && \
cd singularity && \
./mconfig --without-suid && \
make -C ./builddir && \
make -C ./builddir install
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow-io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
The file named requirements.txt used in the above dockerfile is given below,
click
GitPython
jinja2
jsonschema
packaging
prompt_toolkit>=3.0.3
pyyaml
pytest-workflow
questionary>=1.8.0
requests_cache
requests
rich>=10.0.0
tabulate
Following is my dockerfile
FROM ruby:2.7.1-slim as Builder
ARG FOLDERS_TO_REMOVE
ARG BUNDLE_WITHOUT
ENV GEM_HOME="/vendor/bundle"
ENV PATH $GEM_HOME/bin:$GEM_HOME/gems/bin:$PATH
RUN apt-get update && apt-get install -y gnupg2
ADD https://dl.yarnpkg.com/debian/pubkey.gpg /tmp/yarn-pubkey.gpg
RUN apt-key add /tmp/yarn-pubkey.gpg && rm /tmp/yarn-pubkey.gpg
RUN echo 'deb http://dl.yarnpkg.com/debian/ stable main' > /etc/apt/sources.list.d/yarn.list
RUN apt-get update && apt-get install -qq -y --fix-missing --no-install-recommends \
build-essential \
libpq-dev \
git \
tzdata \
libgeos-dev \
nodejs \
yarn
WORKDIR /code
COPY Gemfile* package.json yarn.lock ./
RUN yarn install --check-files
RUN gem install bundler:2.1.4 -N && \
bundle config set without $BUNDLE_WITHOUT && \
bundle config set deployment 'true' && \
bundle config set no-cache 'true' && \
bundle install -j4 --retry 3
COPY . .
EXPOSE 3000
CMD ["bundle", "exec", "rails", "server"]
When I build the image and run it I get
bundler: command not found: rails
When I remove the line bundle config set deployment 'true' it works perfectly fine.
What could be the problem?
I have this Dockerfile. When I try to run it, in the Composer Update line it returns an error, that extensions are not installed.
That is because of GRPC not being activated on the php.ini
My question is, how can I activated it via terminal?
FROM php:7.2-apache
WORKDIR /var/www/html
COPY . ./
RUN apt-get update && apt-get install -y -q nodejs npm curl unzip git rake ruby-ronn zlib1g-dev libpng-dev && apt-get clean
RUN apt-get install php7.2=dev php-pear phpunit
RUN curl -sS https://getcomposer.org/installer | php
RUN mv composer.phar /usr/local/bin/composer
RUN pecl install grpc
RUN docker-php-ext-install mbstring
RUN docker-php-ext-install zip
RUN docker-php-ext-install gd
RUN npm install
RUN npm install -g gulp-cli
RUN composer update
RUN gulp --env=production
EXPOSE 80 443
you forgot to activate the grcp extension ,you can use the code under the comment #install protoc to activated,and you will get message if it has activated or not by RUN php -r "echo extension_loaded('grpc') ? 'yes' : 'no';"
I think this way is better and more concise:
FROM php:7-apache
RUN apt-get update && apt-get install -y -q git rake ruby-ronn zlib1g-dev && apt-get clean
# install composer
RUN cd /usr/local/bin && curl -sS https://getcomposer.org/installer | php
RUN cd /usr/local/bin && mv composer.phar composer
RUN pecl install grpc
#install protoc
RUN mkdir -p /tmp/protoc && \
curl -L https://github.com/google/protobuf/releases/download/v3.2.0/protoc-3.2.0-linux-x86_64.zip > /tmp/protoc/protoc.zip && \
cd /tmp/protoc && \
unzip protoc.zip && \
cp /tmp/protoc/bin/protoc /usr/local/bin && \
cd /tmp && \
rm -r /tmp/protoc && \
docker-php-ext-enable grpc
RUN php -r "echo extension_loaded('grpc') ? 'yes' : 'no';"
WORKDIR /app
COPY . /app
RUN composer install
RUN npm install
RUN npm install -g gulp-cli
RUN gulp --env=production
EXPOSE 8181
I am hitting issues running a start script (eg npm run gulp-dist) for my container as specified in my docker compose file. I traced the issue down to a node version compatibility issue which has led me to some confusion.
If I enter the container with docker-compose run workspace bash and then run node -v I get back v10.5.0 as expected (and what my script requires).
Yet if in docker-compose I set command: node -v it prints v4.2.6 when bringing up the container with docker-compose up workspace.
So I'm wondering where are the commands run that I specify in docker-compose (I thought they were run in the container once it had started). And how do I run a command in the container - I want to specify it in docker-compose as I run a different command in two different docker-compose files (one for dev env, one for production).
Note: My dev machine has node version 11, so I have no idea where four is.
Also, if run docker-compose run workspace bash and then run the original script, it works fine - it is just failing when run as a docker-compose command.
Here's my dockerfile (sorry, it's big):
# FROM laradock/workspace:1.8-71
# copied the contents of the above laradock workspace
# dockerfile and replaced put here directly.
FROM phusion/baseimage:latest
MAINTAINER Mahmoud Zalt <mahmoud#zalt.me>
RUN DEBIAN_FRONTEND=noninteractive
RUN locale-gen en_US.UTF-8
ENV LANGUAGE=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8
ENV LC_CTYPE=en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV TERM xterm
# Add the "PHP 7" ppa
RUN apt-get install -y software-properties-common && \
add-apt-repository -y ppa:ondrej/php
#
#--------------------------------------------------------------------------
# Software's Installation
#--------------------------------------------------------------------------
#
# Install "PHP Extentions", "libraries", "Software's"
RUN apt-get update && \
apt-get install -y --allow-downgrades --allow-remove-essential \
--allow-change-held-packages \
php7.1-cli \
php7.1-common \
php7.1-curl \
php7.1-intl \
php7.1-json \
php7.1-xml \
php7.1-mbstring \
php7.1-mcrypt \
php7.1-mysql \
php7.1-pgsql \
php7.1-sqlite \
php7.1-sqlite3 \
php7.1-zip \
php7.1-bcmath \
php7.1-memcached \
php7.1-gd \
php7.1-dev \
pkg-config \
libcurl4-openssl-dev \
libedit-dev \
libssl-dev \
libxml2-dev \
xz-utils \
libsqlite3-dev \
sqlite3 \
git \
curl \
vim \
nano \
postgresql-client \
&& apt-get clean
#####################################
# Composer:
#####################################
# Install composer and add its bin to the PATH.
RUN curl -s http://getcomposer.org/installer | php && \
echo "export PATH=${PATH}:/var/www/vendor/bin" >> ~/.bashrc && \
mv composer.phar /usr/local/bin/composer
# Source the bash
RUN . ~/.bashrc
#
# other - workspace specific config
#
RUN apt-get -y update && \
apt-get install pkg-config libmagickwand-dev -y && \
pecl install imagick
#####################################
# Non-Root User:
#####################################
# Add a non-root user to prevent files being created with root permissions on host machine.
ENV PUID 1000
ENV PGID 1000
RUN groupadd -g ${PGID} laradock && \
useradd -u ${PUID} -g laradock -m laradock && \
apt-get update -yqq
#####################################
# Set Timezone
#####################################
ARG TZ=UTC
ENV TZ ${TZ}
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
#####################################
# Composer:
#####################################
# Add the composer.json
COPY ./composer.json /home/laradock/.composer/composer.json
# Make sure that ~/.composer belongs to laradock
RUN chown -R laradock:laradock /home/laradock/.composer
USER laradock
# Check if global install need to be ran
ARG COMPOSER_GLOBAL_INSTALL=false
ENV COMPOSER_GLOBAL_INSTALL ${COMPOSER_GLOBAL_INSTALL}
RUN if [ ${COMPOSER_GLOBAL_INSTALL} = true ]; then \
# run the install
composer global install \
;fi
USER root
#####################################
# Node / NVM:
#####################################
# Check if NVM needs to be installed
ARG NODE_VERSION=10.5.0
ENV NODE_VERSION 10.5.0
ENV NVM_DIR /home/laradock/.nvm
RUN curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash && \
. $NVM_DIR/nvm.sh && \
nvm install ${NODE_VERSION} && \
nvm use ${NODE_VERSION} && \
npm install -g gulp bower vue-cli \
;fi
# link node and nodejs
RUN ln -s /usr/bin/nodejs /usr/bin/node
# Wouldn't execute when added to the RUN statement in the above block
# Source NVM when loading bash since ~/.profile isn't loaded on non-login shell
RUN echo "" >> ~/.bashrc && \
echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.bashrc && \
echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm' >> ~/.bashrc \
;fi
# install required things
RUN apt-get update && apt-get install apt-transport-https && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
apt-get update && apt-get install -y --allow-unauthenticated yarn mysql-client
# Add NVM binaries to root's .bashrc
USER root
RUN apt-get install npm -y
# set npm registry address
RUN npm config set registry http://registry.npmjs.org/
#
#--------------------------------------------------------------------------
# Final Touch
#--------------------------------------------------------------------------
#
# Clean up
USER root
RUN apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Set default work directory
WORKDIR /var/www
# # copy in our code, so as not to rely on a volume in prod
COPY . /var/www
# ensure directories we need are writable
RUN chmod -R o+w /var/www/user-api-laravel/storage
RUN chmod -R o+w /var/www/user-api-laravel/bootstrap/cache
RUN chmod -R o+w /var/www/auto/storage
RUN chmod -R o+w /var/www/auto/bootstrap/cache
# install php project dependencies
RUN cd /var/www/user-api-laravel && composer install
RUN cd /var/www/auto && composer install
WORKDIR /var/www
USER root
# install auto-scalar deps
RUN cd /var/www/auto-scaler && npm i
# php.ini for cli
ADD ./php-cli.ini /etc/php/7.1/cli/php.ini
And relevant part of docker-compose:
workspace:
build:
context: ./www-workspace
args:
- TZ=${WORKSPACE_TIMEZONE}
- NODE_VERSION=${WORKSPACE_NODE_VERSION}
command: [bash, -c, "cd /var/www/spa && npm run dist-prod"]
Still don't know what context the commands run in, but made mine work. It was due to node being installed via NVM. Or at least when I installed, as #Noogen suggested, via curl -sL https://deb.nodesource.com/setup_10.x | sudo bash - I could then run commands against my container and they would have access to the correct node version. I had to settled for a lower node version (not 10.5.0 as I could specify with NVM) but in the end it worked so no worries.
The following Dockerfile it's supposed to start sidekiq when the container it's started:
FROM phusion/baseimage:0.9.18
ENV RUBY_MAJOR "2.0"
ENV RUBY_VERSION "2.0.0-p643"
ENV RUBYGEMS_VERSION "2.5.2"
ENV BUNDLER_VERSION "1.11.2"
ENV NODE_VERSION "5.5.0"
ENV BOWER_VERSION "1.7.6"
ENV APT_PACKAGES " \
git imagemagick \
gcc g++ make patch binutils libc6-dev \
libjemalloc-dev libffi-dev libssl-dev libyaml-dev zlib1g-dev libgmp-dev \
libxml2-dev libxslt1-dev libpq-dev libreadline-dev libsqlite3-dev htop \
"
ENV APT_REMOVE_PACKAGES "openssh-server"
RUN apt-get update && apt-get -y dist-upgrade
RUN apt-get install -y $APT_PACKAGES
RUN apt-get remove --purge -y $APT_REMOVE_PACKAGES
RUN apt-get autoremove --purge -y
WORKDIR /tmp
RUN curl -o ruby.tgz \
"https://cache.ruby-lang.org/pub/ruby/${RUBY_MAJOR}/ruby-${RUBY_VERSION}.tar.gz" && \
tar -xzf ruby.tgz && \
cd ruby-${RUBY_VERSION} && \
./configure --enable-shared --with-jemalloc --disable-install-doc && \
make -j4 && \
make install && \
rm /usr/local/lib/libruby-static.a
ENV GEM_SPEC_CACHE "/tmp/gemspec"
RUN echo 'gem: --no-document' > $HOME/.gemrc
RUN gem update --system ${RUBYGEMS_VERSION}
RUN gem install -v ${BUNDLER_VERSION} bundler
RUN curl https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-x64.tar.gz \
|tar -xz -C /usr --strip-components=1
RUN npm install bower#${BOWER_VERSION} -g
RUN rm /etc/my_init.d/00_regen_ssh_host_keys.sh
RUN rm -r /etc/service/sshd
RUN useradd -m app
RUN mkdir /home/app/webapp && chown app:app -R /home/app
RUN rm -rf /tmp/* /var/tmp/* /var/lib/apt /var/lib/dpkg /usr/share/man /usr/share/doc
WORKDIR /home/app/webapp
# Copy the Gemfile as well as the Gemfile.lock and install
# the RubyGems. This is a separate step so the dependencies
# will be cached unless changes to one of those two files
# are made.
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5 --without development test
# Copy the main application.
COPY . ./
COPY .cron/daily_run_app /etc/cron.daily/
RUN chmod a+x /etc/cron.daily/daily_run_app
# Precompile Rails assets
RUN bundle exec rake assets:precompile
EXPOSE 3000
COPY .conf_files/sidekiq.sh /etc/service/sidekiq/run
COPY .conf_files/appserver.sh /etc/service/appserver/run
# Start puma
RUN chmod a+x /etc/service/appserver/run
ENTRYPOINT /etc/service/appserver/run
#ENTRYPOINT bundle exec puma -C config/puma.rb
# Start sidekiq
#ENTRYPOINT bundle exec sidekiq -L log/sidekiq.log
#RUN chmod a+x /etc/service/sidekiq/run
#ENTRYPOINT /etc/service/sidekiq/run
The script /etc/service/appserver/run:
#!/bin/bash
SIDEKIQ_THREADS=${SIDEKIQ_THREADS:-16}
cd /home/app/webapp
exec chpst -u app bundle exec puma -C config/puma.rb \
2>&1 |logger -t appserver
SIDEKIQ_CONFIG="/home/app/webapp/config/sidekiq.yml"
exec chpst -u root bundle exec sidekiq -t 5 -c $SIDEKIQ_THREADS -C $SIDEKIQ_CONFIG \
2>&1 |logger -t sidekiq
You can enquequed jobs because you can see that in the sidekiq dasboard, but they are not going to be processed because sidekiq it's not started, but it should be started with the Dockerfile and Script that are shown above. So when you type manually:
bundle exec sidekiq
Right here sidekiq it's started and the jobs now could be processed. what could be happening?