composer install in Dockerfile not saving dependencies - docker

I have some trouble to dockerize a Symfony project. At the first start from cloning from git repo the dependencies have to installed through composer.
I have read many questions with the same background but i cant get it working.
i show u first my Dockerfile:
ARG PHP_VERSION=8.1
ARG APP_ENV=dev
# Prod image
FROM php:${PHP_VERSION}-fpm-alpine AS app_php
# Update
RUN apk --no-cache update
RUN apk --no-cache add bash git
# Install Node
RUN apk --no-cache add --update nodejs npm
RUN apk --no-cache add --update python3
RUN apk --no-cache add --update make
RUN apk --no-cache add --update g++
# Install pdo
RUN docker-php-ext-install pdo_mysql
# Install Composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
# Symfony CLI
RUN curl -sS https://get.symfony.com/cli/installer | bash && mv /root/.symfony/bin/symfony /usr/local/bin/symfony
# WORK DIR
WORKDIR /var/www/html
# https://getcomposer.org/doc/03-cli.md#composer-allow-superuser
ENV COMPOSER_ALLOW_SUPERUSER=1
ENV PATH="${PATH}:/root/.composer/vendor/bin"
COPY --from=composer:2 /usr/bin/composer /usr/bin/composer
# prevent the reinstallation of vendors at every changes in the source code
COPY composer.* symfony.* ./
RUN set -eux; \
if [ -f composer.json ]; then \
composer install --prefer-dist --no-dev --no-autoloader --no-scripts --no-progress; \
composer clear-cache; \
fi
RUN set -eux; \
mkdir -p var/cache var/log; \
if [ -f composer.json ]; then \
composer dump-autoload --classmap-authoritative --no-dev; \
composer dump-env prod; \
composer run-script --no-dev post-install-cmd; \
chmod +x bin/console; sync; \
fi
# copy sources
COPY . /var/www/html
RUN rm -Rf docker/
# Start Symfony server on Port 8000
EXPOSE 8000
#RUN symfony console doctrine:migrations:migrate
i can see that the packages were installed through the build process, but after docker-compose up the vendor folder isnt set.
Do u have an idea to solve this?

Running it for you, indeed there is no vendors files where you woudl expect them.
if you run a shell on your container you would see what's really happening:
Get your created container id or tag with docker image ls
And run it:
docker run --rm -it <CONTAINER_ID> /bin/bash
bash-5.1# ls /var/www/html/
Dockerfile composer.1 composer.2 symfony.1 var
bash-5.1# ls -al /root/.composer/
total 8
drwxr-xr-x 1 root root 50 Aug 14 15:47 .
drwx------ 1 root root 16 Aug 14 15:48 ..
-rw-r--r-- 1 root root 799 Aug 14 15:47 keys.dev.pub
-rw-r--r-- 1 root root 799 Aug 14 15:47 keys.tags.pub
bash-5.1# ls /usr/bin/composer
/usr/bin/composer
bash-5.1# ls /usr/local/bin/composer
/usr/local/bin/composer
bash-5.1# which composer
/usr/local/bin/composer
bash-5.1# which symfony
/usr/local/bin/symfony
bash-5.1#
The which command would make you realize:
you don't need to copy composer when you already install it
/usr/local/bin is already part of the PATH
The current ENV command is not necessary plus pointing to a non existing folder.
FYI to keep it slim I have created fake symfony.* and composer.* files and have no composer.json (not shared here).
I hope this helps you solve it.

Related

Downloaded file inside dockerfile missing after build

I'm trying to dockerize my php application, this is my very first attempt.
Dockerfile.
FROM ubuntu:18.04
WORKDIR /php55
ARG GIT_TOKEN
ARG DEBIAN_FRONTEND=noninteractive
# Install apache2
RUN set -x; \
perl -pe 's/(\S+\.)?archive\.ubuntu\.com/mirror.sg.gs/g' /etc/apt/sources.list > temp-sc && mv temp-sc /etc/apt/sources.list \
&& sed -i 's#security.ubuntu.com#mirror.sg.gs#g' /etc/apt/sources.list \
&& apt-get update && apt-get install --yes apache2 curl wget nano \
&& a2enmod rewrite headers
# Configure apache2
RUN set -x; \
sed -i.backup 's#/var/www/html#/var/www#g' "/etc/apache2/sites-available/000-default.conf" \
&& echo "ServerName localhost" > "/etc/apache2/conf-available/fqdn.conf" && a2enconf fqdn \
&& cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/000-default.conf.backup
# Copy configuration
COPY ujian.conf /etc/apache2/sites-available/000-default.conf
RUN set -x; \
curl -H "Authorization: token ${GIT_TOKEN}" -O https://git.mydomain.com/api/v1/repos/liso/ujian/archive/main.tar.gz \
&& mkdir -p /var/www/ujian \
&& tar -xvzf main.tar.gz -C /var/www/ujian --strip-components=1 \
&& rm main.tar.gz
# Install PHP
COPY install-php5 .
RUN chmod +x install-php5 && ./install-php5
EXPOSE 80 7825
CMD ["apachectl", "-D", "FOREGROUND"]
docker-compose.yml
version: '3'
services:
ujian:
image: liso/ujian-dockerize
container_name: docker-ujian
build:
context: .
args:
GIT_TOKEN: ${GIT_TOKEN} # from .env file
dockerfile: ./Dockerfile
ports:
- 127.0.0.1:8080:80
volumes:
- ./www:/var/www
extra_hosts:
- "host.docker.internal:host-gateway"
.env contain my api token to git instance.
The problem is after building, I can't find the downloaded file located in `/var/www` on the container, it's empty.
root#6835554968db:/var/www# ls -al
total 12
drwxr-xr-x 2 root root 4096 Jan 30 11:07 .
drwxr-xr-x 1 root root 4096 Jan 30 11:11 ..
I have rebuild several times but still empty /var/www, I never touch docker before so I'm really lost. Can you help me debugging this problem ?
Yeah it seems the volume was overwriting my previous downloaded files, that's why it keep missing after I launched the container. Ultimately I had to create docker-entrypoint.sh script which run after the contain has been provisioned. Then all is well.

Permission denied when trying to run an executable in a Docker container

I'm trying to make an Express server with access to sqlPackage for DACPAC-deployments.
This is the final stage in my Docker build:
FROM node:16-alpine
WORKDIR /usr/app
ARG SQLPACKAGE_URL=https://download.microsoft.com/download/f/0/9/f091c731-45be-48fa-ae84-bc28388e3ef8/sqlpackage-linux-x64-en-16.0.6161.0.zip
# Install sqlPackage
RUN apk add --no-cache wget unzip
RUN wget -progress=bar:force -q -O sqlpackage.zip $SQLPACKAGE_URL \
&& unzip -qq sqlpackage.zip -d /usr/app/sqlpackage \
&& chmod 777 /usr/app/sqlpackage \
&& chown -R node.node /usr/app/sqlpackage
COPY --from=ts-remover /usr/app ./
USER node
CMD ["main.js"]
ARG PORT=8080
EXPOSE $PORT
My node project cannot run the executable though. I've tried running it manually from within the container, but I get permission denied there too.
/usr/app $ whoami
node
/usr/app $ ls -l
total 41272
drwxrwxrwx 2 node node 20480 Sep 12 14:12 sqlpackage
/usr/app $ ./sqlpackage
/bin/sh: ./sqlpackage: Permission denied
Why can I not execute this file, even though the permissions seem to be correct?

Docker compose throws - adduser: group 'www-data' in use

I have a docker-composer.yml file which used to work just fine a couple of months ago, but now when I run it throws an error.
First, this is my file structure.
.data/db
logs
mariadb
nginx
php7-fpm
src/public
.env
.gitignore
README
docker-compose.yml
The only mention of the error i.e www-data is in two of the files. php7-fpm/Dockerfile and nginx/Dockerfile
Here is the content of these files:
php-fpm/Dockerfile
....
RUN apt-get update && apt-get install -y procps
RUN usermod -u 1000 www-data
USER www-data
WORKDIR /var/www
nginx/Dockerfile
FROM nginx:alpine
COPY ./config/nginx.conf /etc/nginx/
COPY ./sites /etc/nginx/sites-available
RUN apk update \
&& apk upgrade \
&& apk add --no-cache bash \
&& adduser -D -H -u 1000 -s /bin/bash www-data
ARG PHP_UPSTREAM_CONTAINER=php-fpm
ARG PHP_UPSTREAM_PORT=9000
# Set upstream conf and remove the default conf
RUN echo "upstream php-upstream { server ${PHP_UPSTREAM_CONTAINER}:${PHP_UPSTREAM_PORT}; }" > /etc/nginx/conf.d/upstream.conf \
&& rm /etc/nginx/conf.d/default.conf
CMD ["nginx"]
The docker-compose.yml file is a generic one, there is no tampering with user groups, but here is a pastebin for anyone who wants to take a look.
https://pastebin.com/ivRfPvZz
This is the partial output from the docker-compose up -d command.
Image for service php-fpm was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Building nginx
Step 1/8 : FROM nginx:alpine
alpine: Pulling from library/nginx
Digest: sha256:17bd1698318e9c0f9ba2c5ed49f53d690684dab7fe3e8019b855c352528d57be
Status: Downloaded newer image for nginx:alpine
---> ea1193fd3dde
Step 2/8 : COPY ./config/nginx.conf /etc/nginx/
---> 65c115482d37
Step 3/8 : COPY ./sites /etc/nginx/sites-available
---> 1fbe81620355
Step 4/8 : RUN apk update && apk upgrade && apk add --no-cache bash && adduser -D -H -u 1000 -s /bin/bash www-data
---> Running in c631ccdf63f2
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.4-61-g22a1991b6a [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.4-57-gb40ea6190b [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9776 distinct packages available
(1/1) Upgrading libbz2 (1.0.6-r6 -> 1.0.6-r7)
OK: 27 MiB in 37 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/2) Installing readline (7.0.003-r1)
(2/2) Installing bash (4.4.19-r1)
Executing bash-4.4.19-r1.post-install
Executing busybox-1.29.3-r10.trigger
OK: 29 MiB in 39 packages
adduser: group 'www-data' in use
Service 'nginx' failed to build: The command '/bin/sh -c apk update && apk upgrade && apk add --no-cache bash && adduser -D -H -u 1000 -s /bin/bash www-data' returned a non-zero code: 1
You can see the error is:
adduser: group 'www-data' in use
Service 'nginx' failed to build: The command '/bin/sh -c apk update && apk upgrade && apk add --no-cache bash && adduser -D -H -u 1000 -s /bin/bash www-data' returned a non-zero code: 1
but I don't know how to fix this.
See this, when you use FROM nginx:alpine, in fact it sames with using nginx:1.17.1-alpine because they are just different tags for same image id.
But several month ago, when you use nginx:alpine, latest maybe others, E.g. nginx:1.14.2-alpine, so when rebuild using the same dockerfile, the base image indeed changed. I strongly suggest you use an explicit tag not latest as base image to assure definiteness.
Finally, what happened for several month ago?
Use nginx:1.14.2-alpine, maybe not this version, just an example:
$ docker run --rm -it nginx:1.14.2-alpine cat /etc/group | grep www-data
You can see there is no www-data group in the image, so you can use next to add a new user also a new group with the name www-data:
adduser -D -H -u 1000 -s /bin/bash www-data
Use nginx:1.17.1-alpine, which currently same as nginx:alpine:
$ docker run --rm -it nginx:1.17.1-alpine cat /etc/group | grep www-data
www-data:x:82:
You can see there is a default www-data group in this image, don't know how it generates, in a word, the image update bring something difference.
So, as already a www-data group there, what you have to do is change your command to next to join a existed group:
adduser -D -H -u 1000 -s /bin/bash www-data -G www-data
You can find the DockerFile inside laradock/nginx folder. Just change the line
&& adduser -D -H -u 1000 -s /bin/bash www-data
to
&& adduser -D -H -u 1000 -s /bin/bash www-data -G www-data
This specifies the group that the user is a member of. Once done, build and bring your containers up with
docker-compose build --no-cache nginx
docker-compose up -d
I'm having this issue with Alpine 3.14, where the www-data group already exists in the image.
Adding (delgroup www-data || true) before the line with adduser in it will fix the problem.
&& apk upgrade \
&& apk add --no-cache bash \
&& (delgroup www-data || true) \
&& adduser -D -H -u 1000 -s /bin/bash www-data
The parentheses with the || true will ensure that the command won't fail if the group does not exist, so it is backward compatible.

docker gulp build failing with nginx

I am using this sample repository -
https://github.com/umputun/nginx-le
To create a docker image from Nginx with letsencrypt.
Now, I am getting the following error -
PEM_read_bio_X509_AUX("/etc/nginx/ssl/") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line:Expecting: TRUSTED CERTIFICATE)
nginx: [emerg] PEM_read_bio_X509_AUX("/etc/nginx/ssl/") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line:Expecting: TRUSTED CERTIFICATE)
When I try to test my nginx configuration.
Now, as stated in the above github repo I have this in my Dockerfile -
FROM nginx:stable-alpine
ADD conf/nginx.conf /etc/nginx/nginx.conf
ADD conf/service.conf /etc/nginx/conf.d/service.conf
RUN rm -rf /usr/share/nginx/html/*
COPY --from=builder /usr/build/app/dist /usr/share/nginx/html
ADD script/entrypoint.sh /entrypoint.sh
ADD script/le.sh /le.sh
RUN \
rm /etc/nginx/conf.d/default.conf && \
chmod +x /entrypoint.sh && \
chmod +x /le.sh && \
apk add --update certbot tzdata openssl && \
rm -rf /var/cache/apk/*
CMD ["/entrypoint.sh"]
-----Updated Dockerfile---------
FROM nginx:latest
RUN curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
RUN apt-get install -y nodejs
RUN apt-get install -y build-essential
RUN curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update -y && sudo apt-get install -y yarn
RUN mkdir -p /usr/build
WORKDIR /usr/build
COPY package.json .
#COPY package-lock.json .
COPY bower.json .
COPY .bowerrc .
RUN npm install --quite
RUN npm install -g gulp bower --quite
RUN bower install --allow-root
RUN mkdir /usr/build/app
RUN cp -R /usr/build/node_modules /usr/build/app
RUN cp -R /usr/build/bower_components /usr/build/app
RUN cp -R /usr/build/*.json /usr/build/app/
RUN cp /usr/build/.bowerrc /usr/build/app/
COPY src /usr/build/app
RUN mkdir /usr/build/app/gulp
ADD gulp/* /usr/build/app/gulp/
ADD gulpfile.js /usr/build/app
WORKDIR /usr/build/app
RUN ls -al .
RUN rm -rf /usr/build/app/dist
RUN mkdir /usr/build/app/dist
RUN gulp build
RUN ls -al /usr/build/app
#RUN yum -y install nodejs
#RUN yum install gcc-c++ make
ADD conf/nginx.conf /etc/nginx/nginx.conf
#ADD conf/service.conf /etc/nginx/conf.d/service.conf
RUN rm -rf /usr/share/nginx/html/*
RUN ls -al /usr/share/nginx/ && ls -al /usr/share/nginx/html/ && ls -al /usr/build/app/dist/
RUN mv /usr/build/app/dist/* /usr/share/nginx/html/
#ADD script/entrypoint.sh /entrypoint.sh
#ADD script/le.sh /le.sh
RUN rm /etc/nginx/conf.d/default.conf && \
chmod +x /entrypoint.sh
CMD ["/entrypoint.sh"]
Now, it reaches entrypoint.sh successfully & I have checked the files in my nginx webroot are getting copied along with the conf.
-------updated issue---------
So, I figured it was not creating the ssh keys because it was not able to generate any html files as part of "gulp build:dev" command & therefore
throwing an error. So I updated my entrypoint to remove lets encrypt for now & only run nginx conf like this -
#!/bin/sh
echo "start nginx"
export TZ="America/Chicago"
cp /usr/share/zoneinfo/${TZ} /etc/localtime && echo ${TZ} > /etc/timezone
echo "ssl_key=${SSL_KEY:=le-key.pem}, ssl_cert=${SSL_CERT:=le-crt.pem}, ssl_chain_cert=${SSL_CHAIN_CERT:=le-chain-crt.pem}"
SSL_KEY=/etc/nginx/ssl/${SSL_KEY}
SSL_CERT=/etc/nginx/ssl/${SSL_CERT}
SSL_CHAIN_CERT=/etc/nginx/ssl/${SSL_CHAIN_CERT}
mkdir -p /etc/nginx/conf.d
mkdir -p /etc/nginx/ssl
#copy /etc/nginx/service*.conf if any of servcie*.conf mounted
if [ -f /etc/nginx/nginx*.conf ]; then
cp -fv /etc/nginx/nginx*.conf /etc/nginx/conf.d/
fi
#replace SSL_KEY, SSL_CERT and SSL_CHAIN_CERT by actual keys
ls -al /etc/nginx/conf.d
#sed -i "s|SSL_KEY|${SSL_KEY}|g" /etc/nginx/conf.d/*.conf
#sed -i "s|SSL_CERT|${SSL_CERT}|g" /etc/nginx/conf.d/*.conf
#sed -i "s|SSL_CHAIN_CERT|${SSL_CHAIN_CERT}|g" /etc/nginx/conf.d/*.conf
#generate dhparams.pem
if [ ! -f /etc/nginx/ssl/dhparams.pem ]; then
echo "make dhparams"
cd /etc/nginx/ssl
openssl dhparam -out dhparams.pem 2048
chmod 600 dhparams.pem
fi
#disable ssl configuration and let it run without SSL
mv -v /etc/nginx/conf.d /etc/nginx/conf.d.disabled
(
sleep 5 #give nginx time to start
echo "start letsencrypt updater"
while :
do
echo "trying to update letsencrypt ..."
# /le.sh
rm -f /etc/nginx/conf.d/default.conf 2>/dev/null #remove default config, conflicting on 80
mv -v /etc/nginx/conf.d.disabled /etc/nginx/conf.d #enable
echo "reload nginx with ssl"
ls -al /etc/nginx/ssl
echo "key contents are - "
cat /etc/nginx/ssl/dhparams.pem
nginx -t
nginx -s reload
sleep 60d
done
) &
nginx -g "daemon off;"
So, here in the script in the end when I test the nginx configuration it gives the following error -
-----Update-----
So, now I am getting this error -
Step 38/40 : RUN mv /usr/build/app/dist/* /usr/share/nginx/html/
---> Running in 9f59c1d5cb90
mv: cannot stat '/usr/build/app/dist/*': No such file or directory
The command '/bin/sh -c mv /usr/build/app/dist/* /usr/share/nginx/html/' returned a non-zero code: 1
Logs for the gulp build:dev command are -
---> Running in 1acca8373940
[14:40:09] Using gulpfile /usr/build/app/gulpfile.js
[14:40:09] Starting 'scripts'...
[14:40:09] Starting 'styles'...
[14:40:09] Starting 'fonts-dev'...
[14:40:10] Starting 'other-dev'...
[14:40:10] Finished 'scripts' after 1.14 s
[14:40:10] Finished 'styles' after 1.13 s
[14:40:10] Starting 'inject'...
[14:40:10] Finished 'other-dev' after 39 ms
[14:40:10] Finished 'inject' after 29 ms
[14:40:10] Starting 'html-dev'...
[14:40:10] Finished 'html-dev' after 288 ms
[14:40:11] Finished 'fonts-dev' after 2.44 s
[14:40:11] Starting 'build:dev'...
[14:40:11] Finished 'build:dev' after 123 μs
Removing intermediate container 1acca8373940
which suggests that gulp build was successful but still in this step -
Step 34/40 : RUN ls -al /usr/build/app/dist
---> Running in c141120c29dc
total 8
drwxr-xr-x 2 root root 4096 Apr 27 14:35 .
drwxr-xr-x 1 root root 4096 Apr 27 14:35 ..
Removing intermediate container c141120c29dc
I am not getting anything in the dist directory. Any suggestions for debugging, solving this ?
Can anyone help me find / debug / solve this issue ?

Error on building Dockerfile to Image

I have the following Dockerfile. I'm trying to build it to an image, but somehow I receive the following error: ADD service /container/service
ADD failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder005872257/service: no such file or directory at Step 6/9. I don't know why... Can anyone help me?
FROM osixia/light-baseimage:1.1.1
ARG LDAP_OPENLDAP_GID
ARG LDAP_OPENLDAP_UID
RUN if [ -z "${LDAP_OPENLDAP_GID}" ]; then groupadd -r openldap; else groupadd -r -g ${LDAP_OPENLDAP_GID} openldap; fi && if [ -z "${LDAP_OPENLDAP_UID}" ]; then useradd -r -g openldap openldap; else useradd -r -g openldap -u ${LDAP_OPENLDAP_UID} openldap; fi
RUN echo "path-include /usr/share/doc/krb5*" >> /etc/dpkg/dpkg.cfg.d/docker && apt-get -y update && /container/tool/add-service-available :ssl-tools \
&& LC_ALL=C DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ldap-utils \
libsasl2-modules \
libsasl2-modules-db \
libsasl2-modules-gssapi-mit \
libsasl2-modules-ldap \
libsasl2-modules-otp \
libsasl2-modules-sql \
openssl \
slapd \
krb5-kdc-ldap \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD service /container/service
RUN /container/tool/install-service
ADD environment /container/environment/99-default
EXPOSE 389 636
EDIT
After adding some ls commands in the Dockerfile I've seen the following line in logs:
Step 6/11 : RUN ls /container/
---> Running in 623dca399324
environment
service
service-available
tool
Removing intermediate container 623dca399324
---> 5f7fcb8a1857
Step 7/11 : RUN ls
---> Running in 7f3bd8662113
bin
boot
container
dev
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
Removing intermediate container 7f3bd8662113
---> 99c17cefc572
Step 8/11 : ADD service /container/service
ADD failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder200387466/service: no such file or directory
Any idea how can I resolve this?
The error means it can't find the directory which mean it probably doesn't exist or you are doing it the wrong way.
One of the things you can do is to make directory and add service to it. Below is a snippet explanation that could teach or help you:
RUN mkdir /container/
Then ADD service to the directory you created. Thus
ADD service /container/service
This can only serve as what could help to put you to track. However I will advice #mohan08p answer above because that works for me.
it successfully build on my local machine.Can you delete the respective files or directories and try once. Also, check the permissions. Did you configure .dockerignore which will not allow to ADD those files. Or else try running with -f or --file command like,
$ docker build . -f Dockerfile
Hope this helps.

Resources