I know there are a lot of questions with similar title but none of them is similar to my issue. My problem is simple, I'm trying to install a few packages in my dockerfile using the command install_packages. This results in a permission denied error stating /bin/sh: 1: install_packages: Permission denied
I've tried adding USER 0 and USER root but none of them worked. Below is my Dockerfile. What am I doing wrong?. Also saw a similar issue in github but the solution provided there didn't work for me.
Similar Issue: https://github.com/bitnami/bitnami-docker-wordpress/issues/230
Dockerfile: (Errors out in the sixth command)
FROM docker.io/bitnami/minideb:buster
LABEL maintainer "Bitnami <containers#bitnami.com>"
USER root
ENV HOME="/" \
OS_ARCH="amd64" \
OS_FLAVOUR="debian-10" \
OS_NAME="linux"
COPY prebuildfs /
# Install required system packages and dependencies
RUN install_packages ca-certificates curl ghostscript gzip imagemagick libc6 libgcc1 libgmp-dev libjemalloc-dev libncurses5-dev libncurses6 libreadline-dev libreadline7 libssl1.1 libstdc++6 libtinfo6 libxml2-dev libxslt1-dev procps tar zlib1g zlib1g-dev
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "ruby" "2.6.6-0" --checksum ecadce77e40822926c69f76cc11437954dd068793548c3f0a21007f9c4fafed3
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "gosu" "1.12.0-0" --checksum 582d501eeb6b338a24f417fededbf14295903d6be55c52d66c52e616c81bcd8c
RUN . /opt/bitnami/scripts/libcomponent.sh && component_unpack "fluentd" "1.10.4-0" --checksum 1428d81be002b7124db38326c73ebe9980e5b3c94a15e3054cdbdff9f8aa3979
RUN apt-get update && apt-get upgrade -y && \
apt-get install build-essential -y && \
rm -r /var/lib/apt/lists /var/cache/apt/archives
COPY rootfs /
RUN /opt/bitnami/scripts/fluentd/postunpack.sh
ENV BITNAMI_APP_NAME="fluentd" \
BITNAMI_IMAGE_VERSION="1.10.4-debian-10-r1" \
GEM_HOME="/opt/bitnami/fluentd" \
PATH="/opt/bitnami/ruby/bin:/opt/bitnami/common/bin:/opt/bitnami/fluentd/bin:$PATH"
#GEM Installation
WORKDIR /opt/bitnami/fluentd
RUN gem install jwk-tool
RUN fluent-gem install 'fluent-plugin-azure-storage-append-blob'
#RUN fluent-gem install 'fluent-plugin-encrypt'
RUN gem install fluent-plugin-jwt-filter && \
gem install json -v "~> 1.8" && \
gem install fluent-plugin-kubernetes && \
gem install fluent-plugin-kubernetes_metadata_filter && \
gem uninstall tzinfo -v 2.0.2
#RUN jwk_tool -g -k key
COPY ./key.pub /opt/bitnami/fluentd/key.pub
EXPOSE 24224 5140
WORKDIR /opt/bitnami/fluentd
#USER 1001
ENTRYPOINT [ "/opt/bitnami/scripts/fluentd/entrypoint.sh" ]
CMD [ "/opt/bitnami/scripts/fluentd/run.sh" ]
The "install_packages" file within the user path directory might not be set as executable by default. You can try making it executable "chmod +x /usr/sbin/install_packages" and try again. It worked for me.
Related
I am using Poetry to install a python project using Poetry in a Docker container. Below you can find my Docker file, which used to work fine until recently when I switched to a new version of Poetry (1.2.1) and the new recommended Poetry installer:
# pull official base image
FROM ubuntu:20.04
ENV PATH = "${PATH}:/home/poetry/bin"
ENV APP_HOME=/home/app/web
RUN apt-get -y update && \
apt upgrade -y && \
apt-get install -y \
python3-pip \
curl \
netcat \
gunicorn && \
rm -fr /var/lib/apt/lists
# alias python2 to python3
RUN ln -s /usr/bin/python3 /usr/bin/python
# Install Poetry
RUN mkdir -p /home/poetry && \
curl -sSL https://install.python-poetry.org | POETRY_HOME=/home/poetry python -
# Cleanup
RUN apt-get remove -y curl && \
apt-get clean
RUN pip install --upgrade pip && \
pip install cryptography && \
pip install psycopg2-binary
# create directory for the app user
# create the app user
# create the appropriate directories
RUN adduser --system --group app && \
mkdir -p $APP_HOME/static-incdtim && \
mkdir -p $APP_HOME/mediafiles
# copy project
COPY . $APP_HOME
WORKDIR $APP_HOME
# Install Python packages
RUN poetry config virtualenvs.create false
RUN poetry install --only main
# copy entrypoint-prod.sh
COPY ./entrypoint.incdtim.prod.sh $APP_HOME/entrypoint.sh
RUN chmod a+x $APP_HOME/entrypoint.sh
# chown all the files to the app user
RUN chown -R app:app $APP_HOME
# change to the app user
USER app
# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.sh"]
The poetry install works fine, I attached to a running container and run it myself and found that it works without problems. However, when I open a Python console and try to import a module (django) which is installed by the Poetry project, the module is not found. Please note that I am installing my project in the system environment (poetry config virtualenvs.create false). I verified, and there is only one version of python installed in the docker container. The specific error I get when trying to import a python module installed by Poetry is: ModuleNotFoundError: No module named xxxx
Although this is not an answer, it is too long to fit within the comment section. It is rather a piece of advice:
declare your ENV at the top of the Dockerfile to make it easier to read.
merge the multiple RUN commands together to avoid creating useless intermediate layers. In the particular case of apt-get install, this will also prevent you from installing a package which dates back from the first "apt-get update". Indeed, since the command line has not changed Docker will not re-execute the command and thus not refresh the package list..
avoid making a copy of all the files in "." when you previously copy some specific files to specific places..
Here, you Dockerfile could rather look like:
# pull official base image
FROM ubuntu:20.04
ENV PATH = "${PATH}:/home/poetry/bin"
ENV HOME=/home/app
ENV APP_HOME=/home/app/web
RUN apt-get -y update && \
apt upgrade -y && \
apt-get install -y \
python3-pip \
curl \
netcat \
gunicorn && \
rm -fr /var/lib/apt/lists
# alias python2 to python3
RUN ln -s /usr/bin/python3 /usr/bin/python
# Install Poetry
RUN mkdir -p /home/poetry && \
curl -sSL https://install.python-poetry.org | POETRY_HOME=/home/poetry python -
# Cleanup
RUN apt-get remove -y \
curl && \
apt-get clean
RUN pip install --upgrade pip && \
pip install cryptography && \
pip install psycopg2-binary
# create directory for the app user
# create the app user
# create the appropriate directories
RUN mkdir -p /home/app && \
adduser --system --group app && \
mkdir -p $APP_HOME/static-incdtim && \
mkdir -p $APP_HOME/mediafiles
WORKDIR $APP_HOME
# copy project
COPY . $APP_HOME
# Install Python packages
RUN poetry config virtualenvs.create false && \
poetry install --only main
# copy entrypoint-prod.sh
RUN cp $APP_HOME/entrypoint.incdtim.prod.sh $APP_HOME/entrypoint.sh && \
chmod a+x $APP_HOME/entrypoint.sh && \
chown -R app:app $APP_HOME
# change to the app user
USER app
# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.sh"]
UPDATE:
Let's get back to your question. Having your program running okay when you "run it yourself" does not mean all the dependencies are met. Indeed, this can mean that your module has not been imported yet (and thus has not triggered the ModuleNotFoundError exception yet).
In order to validate this theory, you can either:
create a simple application which imports the failing module and then quits. If the import succeeds then there is something weird indeed.
list the installed modules with poetry show --latest. If the package is listed, then there is something weird indeed.
If none of the above indicates the module is installed, that just means the module is not installed and you should update your Dockerfile to install it.
NOTE: I do not know much about poetry, but you may want to have a list external dependencies to be met for your application. In the case of pip3, the list is expressed as a file named requirement.txt and can be installed with pip3 install -r requirement.txt.
It turns out this is known a bug in Poetry: https://github.com/python-poetry/poetry/issues/6459
I am trying to run a nextflow pipeline which uses an older version of nextflow (21.04.3) and java version 8. Since I have to use this pipeline on a remote server, therefore I can only use singularity.
As this nextflow pipeline also uses singularity pull calls therefore I need the singularity installed inside the docker image as well. Then, I can convert this image docker image to a singularity image and then I can move it to the remote server.
I am trying to install singularity inside dockerfile but I am getting errors,
This is the dockerfile that I am using,
FROM python:3.8.9-slim
LABEL authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
RUN apt-get install -y singularity-container
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow- io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
These are the errors I am getting
Step 9/17 : RUN wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee
/etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --
keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update
---> Running in afc3dcbbd1ee
--2022-03-17 17:40:19-- http://neuro.debian.net/lists/xenial.us-ca.full
Resolving neuro.debian.net (neuro.debian.net)... 129.170.233.11
Connecting to neuro.debian.net (neuro.debian.net)|129.170.233.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 262
Saving to: ‘STDOUT’
0K 100% 18.4M=0s
deb http://neurodeb.pirsquared.org data main contrib non-free
#deb-src http://neurodeb.pirsquared.org data main contrib non-free
deb http://neurodeb.pirsquared.org xenial main contrib non-free
#deb-src http://neurodeb.pirsquared.org xenial main contrib non-free
2022-03-17 17:40:19 (18.4 MB/s) - written to stdout [262/262]
/bin/sh: 1: apt-key: not found
The command '/bin/sh -c wget -O- http://neuro.debian.net/lists/xenial.us-ca.full | tee /etc/apt/sources.list.d/neurodebian.sources.list && \ apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \ apt-get update'
returned a non-zero code: 127
I there a way to install singularity using a dockerfile ?
Thanks
I made some changes in the dockerfile based on the method to install singularity in linux given here.
The complete dockerfile with which I was able to run successfully nextflow, java and singularity within singularity is given below,
FROM python:3.8.9-slim
LABEL
authors="phil.ewels#scilifelab.se,erik.danielsson#scilifelab.se" \
description="Docker image containing requirements for the nfcore tools"
# Do not pick up python packages from $HOME
ENV PYTHONNUSERSITE=1
# Update pip to latest version
RUN python -m pip install --upgrade pip
# Install dependencies
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
# Install Nextflow dependencies
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y git \
&& apt-get install -y wget
# Create man dir required for Java installation
# and install Java
RUN mkdir -p /usr/share/man/man1 \
&& apt-get install -y openjdk-11-jre \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
# Install Singularity
RUN apt-get update && apt-get install -y \
build-essential \
libssl-dev \
uuid-dev \
libgpgme11-dev \
squashfs-tools \
libseccomp-dev \
wget \
pkg-config \
procps
# Download Go source version 1.16.3, install them and modify the PATH
ENV VERSION=1.16.3
ENV OS=linux
ENV ARCH=amd64
RUN wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \
tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \
rm go$VERSION.$OS-$ARCH.tar.gz && \
echo 'export PATH=$PATH:/usr/local/go/bin' | tee -a /etc/profile
# Download Singularity from version 3.7.3 (security version)
ENV VERSION=3.7.3
RUN wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
tar -xzf singularity-${VERSION}.tar.gz
# Compile Singularity sources and install it
RUN export PATH=$PATH:/usr/local/go/bin && \
cd singularity && \
./mconfig --without-suid && \
make -C ./builddir && \
make -C ./builddir install
# Setup ARG for NXF_VER ENV
ARG NXF_VER=""
ENV NXF_VER ${NXF_VER}
# Install Nextflow
RUN wget https://github.com/nextflow-io/nextflow/releases/download/v21.04.3/nextflow | bash \
&& mv nextflow /usr/local/bin \
&& chmod a+rx /usr/local/bin/nextflow
# Add the nf-core source files to the image
COPY . /usr/src/nf_core
WORKDIR /usr/src/nf_core
# Install nf-core
RUN python -m pip install .
# Set up entrypoint and cmd for easy docker usage
CMD [ "." ]
The file named requirements.txt used in the above dockerfile is given below,
click
GitPython
jinja2
jsonschema
packaging
prompt_toolkit>=3.0.3
pyyaml
pytest-workflow
questionary>=1.8.0
requests_cache
requests
rich>=10.0.0
tabulate
I am running a container from this dockerfile:
FROM node:14
RUN apt update && apt upgrade
RUN apt install -y wget build-essential libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev
RUN wget https://www.python.org/ftp/python/3.9.4/Python-3.9.4.tgz
RUN tar xzf Python-3.9.4.tgz
RUN cd Python-3.9.4 \
&& ./configure --enable-optimizations \
&& make install
RUN alias python=python3
RUN wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh
RUN bash Mambaforge-$(uname)-$(uname -m).sh -b
RUN ln -s /root/mambaforge/bin/conda /usr/bin/conda
RUN conda init
RUN conda create -n myenv nodejs=14 r=4 r-import r-dplyr r-tidyr
# USER node
With the above code, I have access to conda in the shell. But if I uncomment the last line, I get:
conda: Permission denied
I have tried:
chmod -R a+rwx /usr/bin/conda
and
RUN chown -R node:node /usr/bin/conda
RUN chmod 755 /usr/bin/conda
I found on the internet, none of which worked. What should I do to give the node user the required permission?
So, I need to develop and run a specific gui app on Ubuntu. Application is based on OpenGL and freeglut. I'd like to develop from macbook, so I tried to use the vs code remote container feature. It worked well, until I needed to run this app. I installed a lot of various gl packages inside container, ran it with xquarts and etc.
And now I met the problem with glut: freeglut (my_app): ERROR: Internal error <FBConfig with necessary capabilities not found> in function fgOpenWindow. As I found there, it's a known bug with glut and remote connection. And it's known from 2009! So does anyone know, how to get rid of it? Or I will never run my app inside docker?
My current Dockerfile:
FROM nvidia/cudagl:9.0-devel-ubuntu16.04
# This Dockerfile's base image has a non-root user with sudo access. Use the "remoteUser"
# property in devcontainer.json to use it. On Linux, the container user's GID/UIDs
# will be updated to match your local UID/GID (when using the dockerFile property).
# See https://aka.ms/vscode-remote/containers/non-root-user for details.
ARG USERNAME=user
ENV DEBIAN_FRONTEND noninteractive
# Adding new architecture in package base
RUN dpkg --add-architecture i386
# Installg tools
RUN apt-get update \
&& apt-get -y install sudo software-properties-common zsh wget
# Installing latest git
RUN add-apt-repository ppa:git-core/ppa \
&& apt-get update \
&& apt-get -y install git
# Install C++ tools
RUN apt-get -y install build-essential gcc-multilib g++-multilib cmake cppcheck valgrind
# Install some dependencies
RUN apt-get -y install libc6-dev-i386 libgl1-mesa-dev libglu1-mesa-dev freeglut3 freeglut3-dev libjpeg-turbo8-dev libpng12-dev mesa-utils
# Install dependencies with another architecture
RUN apt-get -y install libgl1-mesa-dev:i386 libglu1-mesa-dev:i386 freeglut3:i386 freeglut3-dev:i386 libpng12-0:i386
# Install nvidia driver
RUN apt-get install -y binutils xserver-xorg-video-all
# Adding new user
RUN groupadd --gid 2000 $USERNAME \
&& useradd --uid 2000 --gid $USERNAME --shell /bin/zsh --create-home $USERNAME \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME
# Clean up
RUN apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
USER $USERNAME
ENV DEBIAN_FRONTEND interactive
I don't know how, but now it's working! I've tried to use so many things, so I don't know what exactly helped me. So I'll try to tell everything:
Docker base image become Ubuntu 14.04. I've already tried it before, but it didn't work then.
I've downgraded XQuartz to version 2.7.8
I've pasted this in terminal: defaults write org.macosforge.xquartz.X11 enable_iglx -bool true
Just in case, here is my new Dockerfile:
FROM ubuntu:14.04
# This Dockerfile's base image has a non-root user with sudo access. Use the "remoteUser"
# property in devcontainer.json to use it. On Linux, the container user's GID/UIDs
# will be updated to match your local UID/GID (when using the dockerFile property).
# See https://aka.ms/vscode-remote/containers/non-root-user for details.
ARG USERNAME=user
ENV DEBIAN_FRONTEND noninteractive
# Adding new architecture in package base
RUN dpkg --add-architecture i386
# Install tools
RUN apt-get update \
&& apt-get -y install sudo software-properties-common zsh wget
# Install last git
RUN add-apt-repository ppa:git-core/ppa \
&& apt-get update \
&& apt-get -y install git
# Install C++ tools
RUN apt-get -y install build-essential gcc-multilib g++-multilib cmake cppcheck valgrind
# Install some dependencies
RUN apt-get -y install libc6-dev-i386 libgl1-mesa-dev libglu1-mesa-dev freeglut3 freeglut3-dev libjpeg-turbo8-dev libpng12-dev mesa-utils
# Install dependencies with another architecture
RUN apt-get -y install libgl1-mesa-dev:i386 libglu1-mesa-dev:i386 freeglut3:i386 freeglut3-dev:i386 libpng12-0:i386
# Install nvidia driver
RUN apt-get install -y binutils xserver-xorg-video-all
# Adding new user
RUN groupadd --gid 2000 $USERNAME \
&& useradd --uid 2000 --gid $USERNAME --shell /bin/zsh --create-home $USERNAME \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME
# Clean up
RUN apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
USER $USERNAME
ENV DEBIAN_FRONTEND interactive
I just made a very simple Docker file in my terminal, basically I did the following:
mkdir pgrouted
cd pgrouted
touch Dockerfile
Now I open the Docker file in the nano editor, and I add the following commands to the Docker file:
FROM ubuntu
MAINTAINER Gautam <gautamx07#yahoo.com>
LABEL Description="pgrouting excercise" Vendor="skanatek" Version="1.0"
ENV BBOX="-122.8,45.4,-122.5,45.6"
# Add pgRouting launchpad repository
RUN sudo apt-add-repository -y ppa:ubuntugis/ppa
RUN sudo apt-add-repository -y ppa:georepublic/pgrouting
RUN sudo apt-get update
# Install pgRouting package (for Ubuntu 14.04)
RUN sudo apt-get install postgresql-9.3-pgrouting
# Install osm2pgrouting package
RUN sudo apt-get install osm2pgrouting
# Install workshop material (optional, but maybe slightly outdated)
RUN sudo apt-get install pgrouting-workshop
# For workshops at conferences and events:
# Download and install from http://trac.osgeo.org/osgeo/wiki/Live_GIS_Workshop_Install
RUN wget --no-check-certificate https://launchpad.net/~georepublic/+archive/pgrouting/+files/pgrouting-workshop_2.0.6-ppa1_all.deb
RUN sudo dpkg -i pgrouting-workshop_2.0.6-ppa1_all.deb
# Review: Not sure weather this should be in the dockerfile
RUN cp -R /usr/share/pgrouting/workshop ~/Desktop/pgrouting-workshop
# Log in as user "user"
RUN psql -U postgres
# Create routing database
RUN CREATE DATABASE routing;
# Add PostGIS functions
RUN CREATE EXTENSION postgis;
# Add pgRouting core functions
CREATE EXTENSION pgrouting;
# Download using Overpass XAPI (larger extracts possible than with default OSM API)
wget --progress=dot:mega -O "sampledata.osm" "http://www.overpass-api.de/api/xapi?*[bbox=${BBOX}][#meta]"
The entire Dockerfile can be see HERE at a glance.
Now when I try to build the Dockerfile, like so:
docker build -t gautam/pgrouted:v1 .
The Dockerfile runs and then I get the below error:
Step 4 : RUN sudo apt-add-repository -y ppa:ubuntugis/ppa
---> Running in c93c3c5fd5e8
sudo: apt-add-repository: command not found
The command '/bin/sh -c sudo apt-add-repository -y ppa:ubuntugis/ppa' returned a non-zero code: 1
Why am I getting this error?
apt-add-repository is just not in the base Ubuntu image. You'll first need to install it. try apt-get install software-properties-common
By the way, you don't need to use sudo in the Dockerfile because the commands run as root by default unless you change to another user with the USER command.
Add these lines before running apt-add-repository command
RUN apt-get update && \
apt-get install -y software-properties-common && \
rm -rf /var/lib/apt/lists/*
thats worked for me:
RUN apt-get update --fix-missing && \
apt-get install -y software-properties-common && \
rm -rf /var/lib/apt/lists/* && \
add-apt-repository ppa:ondrej/php && \
apt install -y nginx php7.4-fpm php7.4-mysql php7.4-curl net-tools telnet php7.4-gd php-mail php7.4 php7.4-common php7.4-sqlite3 php7.4-curl php7.4-intl php7.4-mbstring php7.4-xmlrpc php7.4-mysql php7.4-gd php7.4-xml php7.4-cli php7.4-zip php7.4-soap unzip && \
rm -rf /var/lib/apt/lists/* && \
apt clean