Docker doesn't find file - docker

I'm working on a project that uses a Docker image for a specific feature, other than that I don't need docker at all so I don't understand much about it. The issue is that Docker doesn't finds a file that is actually in the folder and the build process breaks.
When trying to create the image using docker build -t project/render-worker . the error is this:
Step 18/23 : RUN bin/composer-install && php composer-setup.php --install-dir=/bin && php -r 'unlink("composer-setup.php");' && php /bin/composer.phar global require hirak/prestissimo
---> Running in 695db3bf2f02
/bin/sh: 1: bin/composer-install: not found
The command '/bin/sh -c bin/composer-install && php composer-setup.php --install-dir=/bin && php -r 'unlink("composer-setup.php");' && php /bin/composer.phar global require hirak/prestissimo' returned a non-zero code: 127
As mentioned the file composer-install does exist and this is what's in it:
#!/bin/sh
EXPECTED_SIGNATURE="$(wget -q -O - https://composer.github.io/installer.sig)"
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
ACTUAL_SIGNATURE="$(php -r "echo hash_file('SHA384', 'composer-setup.php');")"
if [ "$EXPECTED_SIGNATURE" != "$ACTUAL_SIGNATURE" ]
then
echo 'ERROR: Invalid installer signature'
rm composer-setup.php
fi
Basically this is to get composer as you can see.
This is the Docker file:
FROM php:7.2-apache
RUN echo 'deb http://ftp.debian.org/debian stretch-backports main' > /etc/apt/sources.list.d/backports.list
RUN apt-get update
RUN apt-get install -y --no-install-recommends \
libpq-dev \
libxml2-dev \
ffmpeg \
imagemagick \
wget \
git \
zlib1g-dev \
libpng-dev \
unzip \
mencoder \
parallel \
ruby-dev
RUN apt-get -t stretch-backports install -y --no-install-recommends \
libav-tools \
&& rm -rf /var/lib/apt/lists/*
RUN docker-php-ext-install \
pcntl \
pdo_pgsql \
pgsql \
soap \
gd \
zip
RUN gem install compass
RUN a2enmod rewrite
ENV APACHE_RUN_USER root
ENV APACHE_RUN_GROUP root
EXPOSE 80
WORKDIR /app
COPY . /app
# Configuring apache to run the symfony app
COPY config/docker/apache.conf /etc/apache2/sites-enabled/000-default.conf
RUN echo "export DATABASE_URL" >> /etc/apache2/envvars \
&& echo ". /etc/environment" >> /etc/apache2/envvars
RUN wget -cqO- https://nodejs.org/dist/v10.15.3/node-v10.15.3-linux-x64.tar.xz | tar -xJ
RUN cp -a node-v10.15.3-linux-x64/bin /usr \
&& cp -a node-v10.15.3-linux-x64/include /usr \
&& cp -a node-v10.15.3-linux-x64/lib /usr \
&& cp -a node-v10.15.3-linux-x64/share /usr/ \
&& rm -rf node-v10.15.3-linux-x64 node-v10.15.3-linux-x64.tar.xz
RUN bin/composer-install \
&& php composer-setup.php --install-dir=/bin \
&& php -r "unlink('composer-setup.php');" \
# Install prestissimo for dramatically faster `composer install`
&& php /bin/composer.phar global require hirak/prestissimo
RUN APP_ENV=prod APP_SECRET= DATABASE_URL= AWS_KEY= AWS_SECRET= AWS_REGION= MEDIA_S3_BUCKET= \
GIPHY_API_KEY= FACEBOOK_APP_ID= FACEBOOK_APP_SECRET= \
GOOGLE_API_KEY= GOOGLE_CLIENT_ID= GOOGLE_CLIENT_SECRET= STRIPE_SECRET_KEY= STRIPE_ENDPOINT_SECRET= \
THEYSAIDSO_API_KEY= REV_CLIENT_API_KEY= REV_USER_API_KEY= REV_API_ENDPOINT= RENDER_QUEUE_URL= \
CLOUDWATCH_LOG_GROUP_NAME= \
php /bin/composer.phar install --no-interaction --no-dev --prefer-dist --optimize-autoloader --no-scripts \
&& php /bin/composer.phar clear-cache
RUN npm install \
&& node_modules/bower/bin/bower install --allow-root \
&& node_modules/grunt/bin/grunt
# Don't allow it to keep logs around; they're emitted on STDOUT and sent to AWS
# CloudWatch from there, so we don't need them on disk filling up the space
RUN mkdir -p var/cache/prod && chmod -R 777 var/cache/prod
RUN mkdir -p var/log && ln -s /dev/null var/log/prod.log \
&& ln -s /dev/null var/log/prod.deprecations.log && chmod -R 777 var/log
CMD ["/usr/bin/env", "bash", "./bin/start_render_worker"]
Like I said, unfortunately I don't have the slightest idea of how docker works and what's going on, just that I need it. I'm running docker in Win10 Pro and to make matters even worst it is actually working for another dev running Win10. We tried a few things but we can't make it work. I tried cloning the repo in other locations with no success at all. Everything before this particular step runs correctly.
[EDIT]
As suggested by the users I ran RUN ls bin/ before the composer install line and this is the result:
Step 18/24 : RUN ls bin/
---> Running in 6cb72090a069
append_captions
capture
composer-install
concat_project_video
console
encode_frames
encode_frames_to_gif
format_video_for_concatenation
generate_meme_bar
image_to_video
install.sh
phpcs
phpunit
process_render_queue
publish_docker_image
run_animation_worker
run_render_worker
run_render_worker_osx
start_render_worker
update
Removing intermediate container 6cb72090a069
As you can see composer-install is there so this is quite baffling.
Also I checked and set the line ending sequence to LF and the result is the same error.
[SECOND EDIT]
I added COPY bin/composer-install /bin
Then RUN ls bin/
And the results are the same. The ls command finds the file but the error persists. Also adding a slash before bin doesn't change anything :(

Related

Run 'opentsdb' image as non-root

I'm trying to build a custom image of opentsdb to run as non-root user. Our k8s clusters have security policies that doesn't allow containers to run as root. I'm utilizing an existing Dockerfile from here https://hub.docker.com/r/petergrace/opentsdb-docker/dockerfile
Below is my Docker file where I have added extra step to create a new user 'opentsdb' and at the end running it as USER 'opentsdb'
FROM alpine:latest
ENV TINI_VERSION v0.18.0
ENV TSDB_VERSION 2.4.0
ENV HBASE_VERSION 1.4.4
ENV GNUPLOT_VERSION 5.2.4
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
ENV PATH $PATH:/usr/lib/jvm/java-1.8-openjdk/bin/
ENV ALPINE_PACKAGES "rsyslog bash openjdk8 make wget libgd libpng libjpeg libwebp libjpeg-turbo cairo pango lua"
ENV BUILD_PACKAGES "build-base autoconf automake git python3-dev cairo-dev pango-dev gd-dev lua-dev readline-dev libpng-dev libjpeg-turbo-dev libwebp-dev sed"
ENV HBASE_OPTS "-XX:+UseConcMarkSweepGC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"
ENV JVMARGS "-XX:+UseConcMarkSweepGC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -enableassertions -enablesystemassertions"
RUN addgroup opentsdb && adduser -D -u 100 -G opentsdb opentsdb
# Tini is a tiny init that helps when a container is being culled to stop things nicely
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-static-amd64 /tini
RUN chmod +x /tini
ENTRYPOINT ["/tini", "--"]
# Add the base packages we'll need
RUN apk --update add apk-tools \
&& apk add ${ALPINE_PACKAGES} \
# repo required for gnuplot \
--repository http://dl-cdn.alpinelinux.org/alpine/v3.0/testing/ \
&& mkdir -p /opt/opentsdb
WORKDIR /opt/opentsdb/
# Add build deps, build opentsdb, and clean up afterwards.
RUN set -ex && apk add --virtual builddeps ${BUILD_PACKAGES}
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN wget --no-check-certificate \
-O v${TSDB_VERSION}.zip \
https://github.com/OpenTSDB/opentsdb/archive/v${TSDB_VERSION}.zip \
&& unzip v${TSDB_VERSION}.zip \
&& rm v${TSDB_VERSION}.zip \
&& cd /opt/opentsdb/opentsdb-${TSDB_VERSION} \
&& echo "tsd.http.request.enable_chunked = true" >> src/opentsdb.conf \
&& echo "tsd.http.request.max_chunk = 1000000" >> src/opentsdb.conf
RUN cd /opt/opentsdb/opentsdb-${TSDB_VERSION} \
&& find . | xargs grep -s central.maven.org | cut -f1 -d : | xargs sed -i "s/http:\/\/central/https:\/\/repo1/g" \
&& find . | xargs grep -s repo1.maven.org | cut -f1 -d : | xargs sed -i "s/http:\/\/repo1/https:\/\/repo1/g" \
&& ./build.sh \
&& cp build-aux/install-sh build/build-aux \
&& cd build \
&& make install \
&& cd / \
&& rm -rf /opt/opentsdb/opentsdb-${TSDB_VERSION}
RUN cd /tmp && \
wget --no-check-certificate https://sourceforge.net/projects/gnuplot/files/gnuplot/${GNUPLOT_VERSION}/gnuplot-${GNUPLOT_VERSION}.tar.gz && \
tar xzf gnuplot-${GNUPLOT_VERSION}.tar.gz && \
cd gnuplot-${GNUPLOT_VERSION} && \
./configure && \
make install && \
cd /tmp && rm -rf /tmp/gnuplot-${GNUPLOT_VERSION} && rm /tmp/gnuplot-${GNUPLOT_VERSION}.tar.gz
RUN apk del builddeps && rm -rf /var/cache/apk/*
#Install HBase and scripts
RUN mkdir -p /data/hbase /root/.profile.d /opt/downloads
WORKDIR /opt/downloads
RUN wget -O hbase-${HBASE_VERSION}.bin.tar.gz http://archive.apache.org/dist/hbase/${HBASE_VERSION}/hbase-${HBASE_VERSION}-bin.tar.gz \
&& tar xzvf hbase-${HBASE_VERSION}.bin.tar.gz \
&& mv hbase-${HBASE_VERSION} /opt/hbase \
&& rm -r /opt/hbase/docs \
&& rm hbase-${HBASE_VERSION}.bin.tar.gz
# Add misc startup files
RUN ln -s /usr/local/share/opentsdb/etc/opentsdb /etc/opentsdb \
&& rm /etc/opentsdb/opentsdb.conf \
&& mkdir /opentsdb-plugins
ADD files/opentsdb.conf /etc/opentsdb/opentsdb.conf.sample
ADD files/hbase-site.xml /opt/hbase/conf/hbase-site.xml.sample
ADD files/start_opentsdb.sh /opt/bin/
ADD files/create_tsdb_tables.sh /opt/bin/
ADD files/start_hbase.sh /opt/bin/
ADD files/entrypoint.sh /entrypoint.sh
# Fix ENV variables in installed scripts
RUN for i in /opt/bin/start_hbase.sh /opt/bin/start_opentsdb.sh /opt/bin/create_tsdb_tables.sh; \
do \
sed -i "s#::JAVA_HOME::#$JAVA_HOME#g; s#::PATH::#$PATH#g; s#::TSDB_VERSION::#$TSDB_VERSION#g;" $i; \
done
RUN echo "export HBASE_OPTS=\"${HBASE_OPTS}\"" >> /opt/hbase/conf/hbase-env.sh
#4242 is tsdb, rest are hbase ports
EXPOSE 60000 60010 60030 4242 16010 16070
USER opentsdb
#HBase is configured to store data in /data/hbase, vol-mount it to persist your data.
VOLUME ["/data/hbase", "/tmp", "/opentsdb-plugins"]
CMD ["/entrypoint.sh"]
however the newly built image is throwing error and says permission denied for /opt/bin/ files. And the opentsdb is not getting deployed correctly.
On local using docker desktop, everything works fine using root, when I run below command
docker run -dp 4242:4242 petergrace/opentsdb-docker
Do i need to use any chown commands too ?
Could you help how to make opentsdb get deployed correctly using uid 100 ? Thanks in advance!

How to setup a older meteor version in dockerfile, and docker container

I have a project running with meteor and node.js in my local. The meteor version is 2.4, node.js version is 8.9.4, I have meteor/release file to make meteor version be 2.2 so that meteor and node can work together.
(base) xxx$ meteor --version
Meteor 2.4
(base) xxx$ node -v
v8.9.4
It seems fine so I deploy this project to docker container to server. The Dockerfile first line I wrote
# node version dependent on meteor version
FROM node:8.9.4
After successfully deployed, the docker logs shows error siad.
Waiting for mongodb server to start - sleeping
warn: --minUptime not set. Defaulting to: 1000ms
warn: --spinSleepTime not set. Your script will exit if it does not stay up for at least 1000ms
info: Forever processing file: /app/bundle/main.js
error: undefined
data: /app/bundle/main.js:34 - Meteor requires Node v12.0.0 or later.
data: /app/bundle/main.js:34 - error: Forever detected script exited with code: 1
I check inside docker, the node version is 8.9.4
(base) [xxx]$ docker exec -it -u root tblbuilder_meteor_1 /bin/bash -c 'node --version'
v8.9.4
So I assume it is meteor version. But first I dont know how to check meteor version inside the docker. And second why this happens? I am sure the release file is updated to push project folder.
With some great man help, I kinda understand it. In local I use meteor 2.2, in docker file I use node.js 8.9.4 work with meteor2.2. So the thing I left is to modify DOCKERFILE, change it from node 8.9.4 to node 12. Below is my Dockerfile file, I try to change it to node 12.22.2, but it keep give me error, I spent one day to solve them. Currently, I stack at install r-base part.
Is there some guide for change node 8 to node 12.
# node version dependent on meteor version
FROM node:8.9.4
# I am going to use 12.22.2
#FROM node:12.22.2
# (even if copied as root you still need to change)
# https://github.com/moby/moby/issues/6119
COPY ./compose/meteor/entrypoint.sh /entrypoint.sh
COPY ./compose/meteor/run_app.sh /run_app.sh
COPY ./compose/meteor/r-cran.pgp /r-cran.pgp
COPY ./settings/settings.json /app/settings.json
COPY ./requirements.txt /requirements.txt
COPY ./r_requirements.sh /r_requirements.sh
# set locale to utf8: https://github.com/docker-library/docs/pull/703/files
# added [check-valid-until=no] & Acquire::Check-Valid-Until "false"; https://unix.stackexchange.com/questions/508724/failed-to-fetch-jessie-backports-repository
# Needs work to bring it up-to-date
RUN \
echo "deb [check-valid-until=no] http://archive.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/jessie-backports.list && \
sed -i '/deb http:\/\/deb.debian.org\/debian jessie-updates main/d' /etc/apt/sources.list && \
apt-get -o Acquire::Check-Valid-Until=false update && \
\
sh -c 'echo "deb [check-valid-until=no] http://cran.rstudio.com/bin/linux/debian jessie-cran35/" >> /etc/apt/sources.list' && \
apt-key add /r-cran.pgp && \
\
apt-get -o Acquire::Check-Valid-Until=false update && \
apt-get -o Acquire::Check-Valid-Until=false install -y locales && \
\
localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 && \
export LC_ALL=en_US.UTF-8 && \
export LANG=en_US.UTF-8 && \
export LANGUAGE=en_US.UTF-8
ENV LANG en_US.utf8
ENV LC_ALL en_US.UTF-8
# add rstudio debian install for R (requires version >3.3)
# https://cran.r-project.org/bin/linux/debian/
# install R from apt-get
# install python 3.6 from source :/
RUN apt install -y --force-yes r-base-core r-recommended r-base-html r-base-core
RUN apt-get install -y --force-yes wget bsdtar r-base r-base-dev && \
apt-get clean && \
\
wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz && \
tar zxf Python-3.6.5.tgz && \
cd ./Python-3.6.5 && \
./configure && \
make && \
make altinstall && \
cd .. && \
rm Python-3.6.5.tgz && \
rm -rf ./Python-3.6.5
# create paths and users
# change executable permissions
RUN npm install forever -g && \
\
mkdir -p /app/production && \
mkdir -p /app/logs && \
mkdir -p /app/crons && \
\
groupadd -r app && \
useradd -m -d /home/app -g app app && \
\
chmod +x /entrypoint.sh && \
chmod +x /run_app.sh && \
chmod +x /r_requirements.sh && \
chmod +x /requirements.txt && \
\
chown -R app:app /app && \
chown app:app /entrypoint.sh && \
chown app:app /run_app.sh && \
chown app:app /r_requirements.sh &&\
chown app:app /requirements.txt
USER app
# 1) install R packages
# 2) install python packages
RUN export "R_LIBS=/home/app/R_libs" && \
mkdir /home/app/R_libs && \
bash /r_requirements.sh && \
\
/usr/local/bin/pip3.6 install --user -r /requirements.txt
USER root
COPY ./compose/meteor/src/src.tar.gz /app/src.tar.gz
COPY ./src/private /app/src/private
RUN chown -R app:app /app
USER app
RUN cd /app && \
bsdtar -xzvf src.tar.gz && \
npm install --prefix /app/bundle/programs/server --production
ENTRYPOINT ["/entrypoint.sh"]
There are many wrong understanding in your tests:
Your Meteor version is 2.2, because is the version inside your project;
To you see the Node version of this Meteor project, see this answers that many guys send to you in Meteor Docker Node.js version is not match
Usually, we build the Meteor, that mean transform it in a NodeJS package build, then, inside of the Docker you don't need Meteor.
We need see your Dockerfile and understand what process you do to build do Docker image.

Reset a docker image to initial state

I'm new to docker and recently I tried to use setup openstreetmap-tileserver. I tried a manual installation by cloning the project and run docker build -t SampleMap and docker run -v openstreetmap-data:/var/lib/postgresql/10/main SampleMap import and then run the proper command to run the container. I got three images using docker image ls:
ubuntu
none
SampleMap
Everything worked fined. Next, I tried to erase the DB and do the whole process for a new map (a different .osm.pbf file). I removed the image SampleMap (with docker image rm) and tried to do the whole process again but the problem is all the DB tables still exist. It seems that all the changes are written into the Ubuntu image rather than the SampleMap. I'm asking generally is there any way that I can reset the whole Ubuntu image to its initial state? It seems that all the changes are permanent in the Ubuntu image.
Here is the Dockerfile:
FROM ubuntu:18.04
# Based on
# https://switch2osm.org/manually-building-a-tile-server-18-04-lts/
# Set up environment
ENV TZ=UTC
ENV AUTOVACUUM=on
ENV UPDATES=disabled
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
# Install dependencies
RUN echo "deb [ allow-insecure=yes ] http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" >> /etc/apt/sources.list.d/pgdg.list \
&& apt-get update \
&& apt-get install -y apt-transport-https ca-certificates \
&& apt-get install -y --no-install-recommends --allow-unauthenticated \
apache2 \
apache2-dev \
autoconf \
build-essential \
bzip2 \
cmake \
fonts-noto-cjk \
fonts-noto-hinted \
fonts-noto-unhinted \
clang \
gdal-bin \
git-core \
libagg-dev \
libboost-all-dev \
libbz2-dev \
libcairo-dev \
libcairomm-1.0-dev \
libexpat1-dev \
libfreetype6-dev \
libgdal-dev \
libgeos++-dev \
libgeos-dev \
libgeotiff-epsg \
libicu-dev \
liblua5.3-dev \
libmapnik-dev \
libpq-dev \
libproj-dev \
libprotobuf-c0-dev \
libtiff5-dev \
libtool \
libxml2-dev \
lua5.3 \
make \
mapnik-utils \
nodejs \
npm \
postgis \
postgresql-10 \
postgresql-10-postgis-2.5 \
postgresql-10-postgis-2.5-scripts \
postgresql-contrib-10 \
protobuf-c-compiler \
python-mapnik \
sudo \
tar \
ttf-unifont \
unzip \
wget \
zlib1g-dev \
osmosis \
osmium-tool \
cron \
python3-psycopg2 python3-shapely python3-lxml \
&& apt-get clean autoclean \
&& apt-get autoremove --yes \
&& rm -rf /var/lib/{apt,dpkg,cache,log}/
# Set up renderer user
RUN adduser --disabled-password --gecos "" renderer
USER renderer
# Install latest osm2pgsql
RUN mkdir /home/renderer/src
WORKDIR /home/renderer/src
RUN git clone https://github.com/openstreetmap/osm2pgsql.git
WORKDIR /home/renderer/src/osm2pgsql
RUN mkdir build
WORKDIR /home/renderer/src/osm2pgsql/build
RUN cmake .. \
&& make -j $(nproc)
USER root
RUN make install
USER renderer
# Install and test Mapnik
RUN python -c 'import mapnik'
# Install mod_tile and renderd
WORKDIR /home/renderer/src
RUN git clone -b switch2osm https://github.com/SomeoneElseOSM/mod_tile.git
WORKDIR /home/renderer/src/mod_tile
RUN ./autogen.sh \
&& ./configure \
&& make -j $(nproc)
USER root
RUN make -j $(nproc) install \
&& make -j $(nproc) install-mod_tile \
&& ldconfig
USER renderer
# Configure stylesheet
WORKDIR /home/renderer/src
RUN git clone https://github.com/gravitystorm/openstreetmap-carto.git
WORKDIR /home/renderer/src/openstreetmap-carto
USER root
RUN npm install -g carto
USER renderer
RUN carto project.mml > mapnik.xml
# Load shapefiles
WORKDIR /home/renderer/src/openstreetmap-carto
RUN scripts/get-shapefiles.py
# Configure renderd
USER root
RUN sed -i 's/renderaccount/renderer/g' /usr/local/etc/renderd.conf \
&& sed -i 's/hot/tile/g' /usr/local/etc/renderd.conf
USER renderer
# Configure Apache
USER root
RUN mkdir /var/lib/mod_tile \
&& chown renderer /var/lib/mod_tile \
&& mkdir /var/run/renderd \
&& chown renderer /var/run/renderd
RUN echo "LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so" >> /etc/apache2/conf-available/mod_tile.conf \
&& a2enconf mod_tile
COPY apache.conf /etc/apache2/sites-available/000-default.conf
COPY leaflet-demo.html /var/www/html/index.html
RUN ln -sf /proc/1/fd/1 /var/log/apache2/access.log \
&& ln -sf /proc/1/fd/2 /var/log/apache2/error.log
# Configure PosgtreSQL
COPY postgresql.custom.conf.tmpl /etc/postgresql/10/main/
RUN chown -R postgres:postgres /var/lib/postgresql \
&& chown postgres:postgres /etc/postgresql/10/main/postgresql.custom.conf.tmpl \
&& echo "\ninclude 'postgresql.custom.conf'" >> /etc/postgresql/10/main/postgresql.conf
# copy update scripts
COPY openstreetmap-tiles-update-expire /usr/bin/
RUN chmod +x /usr/bin/openstreetmap-tiles-update-expire \
&& mkdir /var/log/tiles \
&& chmod a+rw /var/log/tiles \
&& ln -s /home/renderer/src/mod_tile/osmosis-db_replag /usr/bin/osmosis-db_replag \
&& echo "* * * * * renderer openstreetmap-tiles-update-expire\n" >> /etc/crontab
# install trim_osc.py helper script
USER renderer
RUN cd ~/src \
&& git clone https://github.com/zverik/regional \
&& cd regional \
&& git checkout 612fe3e040d8bb70d2ab3b133f3b2cfc6c940520 \
&& chmod u+x ~/src/regional/trim_osc.py
# Start running
USER root
COPY run.sh /
COPY indexes.sql /
ENTRYPOINT ["/run.sh"]
CMD []
EXPOSE 80 5432
And here is my run.sh file:
#!/bin/bash
set -x
function CreatePostgressqlConfig()
{
cp /etc/postgresql/10/main/postgresql.custom.conf.tmpl /etc/postgresql/10/main/postgresql.custom.conf
sudo -u postgres echo "autovacuum = $AUTOVACUUM" >> /etc/postgresql/10/main/postgresql.custom.conf
cat /etc/postgresql/10/main/postgresql.custom.conf
}
if [ "$#" -ne 1 ]; then
ls /home/renderer
echo "usage: <import|run>"
echo "commands:"
echo " import: Set up the database and import /data.osm.pbf"
echo " run: Runs Apache and renderd to serve tiles at /tile/{z}/{x}/{y}.png"
echo "environment variables:"
echo " THREADS: defines number of threads used for importing / tile rendering"
echo " UPDATES: consecutive updates (enabled/disabled)"
exit 1
fi
if [ "$1" = "import" ]; then
# Initialize PostgreSQL
CreatePostgressqlConfig
service postgresql start
sudo -u postgres createuser renderer
sudo -u postgres createdb -E UTF8 -O renderer gis
sudo -u postgres psql -d gis -c "CREATE EXTENSION postgis;"
sudo -u postgres psql -d gis -c "CREATE EXTENSION hstore;"
sudo -u postgres psql -d gis -c "ALTER TABLE geometry_columns OWNER TO renderer;"
sudo -u postgres psql -d gis -c "ALTER TABLE spatial_ref_sys OWNER TO renderer;"
# Download Luxembourg as sample if no data is provided
if [ ! -f /data.osm.pbf ]; then
echo "WARNING: No import file at /data.osm.pbf, so importing iran-latest as example..."
wget -nv http://download.geofabrik.de/north-america/canada-latest.osm.pbf -O /data.osm.pbf
# wget -nv http://download.geofabrik.de/europe/luxembourg.poly -O /data.poly
fi
# determine and set osmosis_replication_timestamp (for consecutive updates)
osmium fileinfo /data.osm.pbf > /var/lib/mod_tile/data.osm.pbf.info
osmium fileinfo /data.osm.pbf | grep 'osmosis_replication_timestamp=' | cut -b35-44 > /var/lib/mod_tile/replication_timestamp.txt
REPLICATION_TIMESTAMP=$(cat /var/lib/mod_tile/replication_timestamp.txt)
# initial setup of osmosis workspace (for consecutive updates)
sudo -u renderer openstreetmap-tiles-update-expire $REPLICATION_TIMESTAMP
# copy polygon file if available
if [ -f /data.poly ]; then
sudo -u renderer cp /data.poly /var/lib/mod_tile/data.poly
fi
# Import data
sudo -u renderer osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua -C 2048 --number-processes ${THREADS:-4} -S /home/renderer/src/openstreetmap-carto/openstreetmap-carto.style /data.osm.pbf
# Create indexes
sudo -u postgres psql -d gis -f indexes.sql
service postgresql stop
exit 0
fi
if [ "$1" = "run" ]; then
# Clean /tmp
rm -rf /tmp/*
# Fix postgres data privileges
chown postgres:postgres /var/lib/postgresql -R
# Initialize PostgreSQL and Apache
CreatePostgressqlConfig
service postgresql start
service apache2 restart
# Configure renderd threads
sed -i -E "s/num_threads=[0-9]+/num_threads=${THREADS:-4}/g" /usr/local/etc/renderd.conf
# start cron job to trigger consecutive updates
if [ "$UPDATES" = "enabled" ]; then
/etc/init.d/cron start
fi
# Run
sudo -u renderer renderd -f -c /usr/local/etc/renderd.conf
service postgresql stop
exit 0
fi
echo "invalid command"
exit 1
When you create a container from your image, you mount a volume, using the -v option:
docker run -v openstreetmap-data:/var/lib/postgresql/10/main SampleMap import
Your persistent data is stored in openstreetmap-data. That file/folder is not in your container (that is created every time), it is mounted from your host's filesystem. That's why it persists

Docker multistage build doesn't recognise installed application

FROM some-build:latest as build
COPY / /var/www/html
WORKDIR /var/www/html
RUN cd /var/www/html && composer install
FROM some-build2:latest as run
COPY --from=build /var/www/html /var/www/html
ENV PATH ${HOME}/local/bin:${PATH}:/home/site/wwwroot
RUN cd /var/www/html && \
npm install && \
npm run production
ENTRYPOINT ["/bin/init_container.sh"]
The image run contains an installed npm. Despite this fact, the npm install return the error: /bin/sh: 1: npm: not found
How is this possible? What am I doing wrong?
Edit:
As answer to #BMitch 's comment, when I run the RUN image, in the container the node is on the PATH and I can use it. The path is /root/local/bin. I've attached all the Dockerfiles.
I have 3 docker files:
APP
The one you've already seen before.
RUN
FROM php:7.2.5-apache
MAINTAINER Azure App Services Container Images <appsvc-images#microsoft.com>
COPY apache2.conf /bin/
COPY init_container.sh /bin/
RUN a2enmod rewrite expires include deflate
# install the PHP extensions we need
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
libpng-dev \
libjpeg-dev \
libpq-dev \
libldap2-dev \
libldb-dev \
libicu-dev \
libgmp-dev \
mysql-client \
libmagickwand-dev \
openssh-server vim curl wget tcptraceroute \
&& chmod 755 /bin/init_container.sh \
&& echo "root:Docker!" | chpasswd \
&& echo "cd /home" >> /etc/bash.bashrc \
&& ln -s /usr/lib/x86_64-linux-gnu/libldap.so /usr/lib/libldap.so \
&& ln -s /usr/lib/x86_64-linux-gnu/liblber.so /usr/lib/liblber.so \
&& ln -s /usr/include/x86_64-linux-gnu/gmp.h /usr/include/gmp.h \
&& rm -rf /var/lib/apt/lists/* \
&& pecl install imagick-beta \
&& docker-php-ext-configure gd --with-png-dir=/usr --with-jpeg-dir=/usr \
&& docker-php-ext-configure pdo_mysql --with-pdo-mysql=mysqlnd \
&& docker-php-ext-configure mysqli --with-mysqli=mysqlnd \
&& docker-php-ext-install gd \
mysqli \
opcache \
pdo \
pdo_mysql \
pdo_pgsql \
pgsql \
ldap \
intl \
gmp \
zip \
bcmath \
mbstring \
pcntl \
xml \
xmlrpc \
&& docker-php-ext-enable imagick
###################
# Installing node #
###################
RUN apt-get update -yq && apt-get upgrade -yq && \
apt-get install -yq g++ libssl-dev apache2-utils curl git python make nano
# setting up npm for global installation without sudo
# http://stackoverflow.com/a/19379795/580268
RUN MODULES="local" && \
echo prefix = ~/$MODULES >> ~/.npmrc && \
echo "export PATH=\$HOME/$MODULES/bin:\$PATH" >> ~/.bashrc && \
. ~/.bashrc && \
mkdir ~/$MODULES && \
\
# install Node.js and npm
# https://gist.github.com/isaacs/579814#file-node-and-npm-in-30-seconds-sh
mkdir ~/node-latest-install && cd $_ && \
curl http://nodejs.org/dist/v8.11.3/node-v8.11.3.tar.gz | tar xz --strip-components=1 && \
./configure --prefix=~/$MODULES && \
make install && \
curl -L https://www.npmjs.org/install.sh | sh
# optional, check locations and packages are correct
# RUN which node; node -v; which npm; npm -v; \
# npm ls -g --depth=0
# Remove unnecessary packages
# RUN apt-get -yq purge g++ libssl-dev curl git python make nano
# RUN apt-get -yq autoremove
###################
RUN \
rm -f /var/log/apache2/* \
&& rmdir /var/lock/apache2 \
&& rmdir /var/run/apache2 \
&& rmdir /var/log/apache2 \
&& chmod 777 /var/log \
&& chmod 777 /var/run \
&& chmod 777 /var/lock \
&& chmod 777 /bin/init_container.sh \
&& cp /bin/apache2.conf /etc/apache2/apache2.conf \
&& rm -rf /var/www/html \
&& rm -rf /var/log/apache2 \
&& mkdir -p /home/LogFiles \
&& ln -s /home/LogFiles /var/log/apache2
RUN { \
echo 'opcache.memory_consumption=128'; \
echo 'opcache.interned_strings_buffer=8'; \
echo 'opcache.max_accelerated_files=4000'; \
echo 'opcache.revalidate_freq=60'; \
echo 'opcache.fast_shutdown=1'; \
echo 'opcache.enable_cli=1'; \
} > /usr/local/etc/php/conf.d/opcache-recommended.ini
RUN { \
echo 'error_log=/var/log/apache2/php-error.log'; \
echo 'display_errors=Off'; \
echo 'log_errors=On'; \
echo 'display_startup_errors=Off'; \
echo 'date.timezone=UTC'; \
} > /usr/local/etc/php/conf.d/php.ini
COPY sshd_config /etc/ssh/
EXPOSE 2222 8080
ENV APACHE_RUN_USER www-data
ENV PHP_VERSION 7.2.5
ENV PORT 8080
ENV WEBSITE_ROLE_INSTANCE_ID localRoleInstance
ENV WEBSITE_INSTANCE_ID localInstance
ENV PATH ${PATH}:/home/site/wwwroot
ENTRYPOINT ["/bin/init_container.sh"]
BUILD
FROM composer:latest as composer
FROM php:7.2.5-apache as apache
COPY --from=composer /usr/bin/composer /usr/bin/composer
RUN apt-get update && \
apt-get install git zip unzip -y
Edit 2:
It is important that if I remove the RUN npm... commands, then the whole build is a success and the result image contains the npm and I can use it (I've verified by using a container in interactive mode).
Edit 3:
Here's a lot lot simpler solution that can be tried out instantly:
FROM alpine as img1
RUN echo "$HOME" > $HOME/test.txt
FROM alpine as img2
RUN cat $HOME/test.txt
The result is: cat: can't open '/root/test.txt': No such file or directory
Two issues going on here. The "php:7.2.5-apache" image won't have /root/local/bin in the path, and you did not add it to the path during your build. The npm commands will work when you login interactively likely because of some changes to the shell login scripts that setup the environment. You'll need to run these environment setup scripts before running any npm commands, and that must be done within the same RUN command. To verify for yourself, you can check your .bashrc for variables or commands run to setup the npm environment. And you can verify the environment is different by comparing the PATH value with an env command in the interactive shell and in your build, you should see two different outputs if this is your issue. When I ran part of your run image, I saw the following in the .bashrc:
export PATH=$HOME/local/bin:$PATH
So you'll want to update the line in your Dockerfile for the run image:
ENV PATH /root/local/bin:${PATH}:/home/site/wwwroot
Per your edit 3, that's an entirely different issue. You created a file in one new image, and then went back to the base image where the file doesn't exist. If you wanted to see the file in a multi-stage build, then you either need to copy it between the stages, or use the previous image as your "from".
FROM alpine as img1
RUN echo "$HOME" > $HOME/test.txt
FROM alpine as img2
COPY --from=img1 /root/test.txt /root/test.txt
RUN cat $HOME/test.txt
or
FROM alpine as img1
RUN echo "$HOME" > $HOME/test.txt
FROM img1 as img2
RUN cat $HOME/test.txt

Unknown Instruction ln Docker File in RUN

Hi Here is my Docker File. I getting an Error while Installing Maven it's not able to find "ln -s" Soft Link Command. Please Help me with this.
# We first will give the base image details for the tag "FROM" OS:Version
FROM centos:latest
#who will maintain the image
MAINTAINER XYZ <XYZ#gmail.com>
RUN yum install -y httpd curl sed grep egrep fgrep wget git net-tools zip unzip which source openssh-server
#JAVA INSTALLATION
ADD http://www.mediafire.com/file/177sevky311fbdh/jdk-8u144-linux-x64.rpm /
RUN rpm -ivh jdk-8u144-linux-x64.rpm
ENV JAVA_HOME="/usr/java/jdk1.8.0_144"
ENV JRE_HOME="/usr/java/jdk1.8.0_144/jre"
#MAVEN INSTALLATION
ADD https://www.mediafire.com/folder/6b0t6el85gtof/maven /
RUN mv /maven /opt/maven
ln -s /opt/maven/bin/mvn /usr/bin/mvn && \
wget http://www.mediafire.com/file/gpg2arhygj0a0wy/maven.sh && \
mv /maven.sh /etc/profile.d && \
chmod 755 /etc/profile.d/maven.sh
RUN cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original && \
chmod a-w /etc/ssh/sshd_config.original && \
mkdir /var/run/sshd && \
echo 'root:screencast' | chpasswd && \
sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
CMD ["/usr/sbin/sshd, "-D"]
EXPOSE 22
You need to add an &&\ to link RUN commands togethers
Otherwise Dockerfile won't be able to interpret ln as a Dockerfile command (like RUN, COPY, ADD, ...)
RUN mv /maven /opt/maven &&\ <============== missing
ln -s /opt/maven/bin/mvn /usr/bin/mvn && \
Or at least add a second run
RUN mv /maven /opt/maven
RUN ln -s /opt/maven/bin/mvn /usr/bin/mvn && \
^ ...
|
--- missing

Resources