When copying files from host machine to container where a file already exists at the destination path, the copied file is empty.
I've attempted to copy the same files to a path with a different name and this works fine.
The two lines from my dockerfile that this issue happens on are:
COPY conf/policy.xml /etc/ImageMagick-6/
COPY conf/000-default.conf /etc/apache2/sites-available/
Full dockerfile:
FROM php:7.3-apache
RUN docker-php-ext-install pdo_mysql && docker-php-ext-enable pdo_mysql
RUN docker-php-ext-install mysqli && docker-php-ext-enable mysqli
RUN apt-get update && apt-get install -y \
git libmagick++-dev \
--no-install-recommends && \
git clone https://github.com/mkoppanen/imagick.git && \
cd imagick && git checkout master && phpize && ./configure && \
make && make install && \
docker-php-ext-enable imagick && \
cd ../ && rm -rf imagick && \
apt-get install -y ghostscript && rm -r /var/lib/apt/lists/*
RUN pecl install xdebug
RUN docker-php-ext-enable xdebug
COPY conf/php.ini /etc/php/7.3/fpm/conf.d/40-custom.ini
COPY conf/policy.xml /etc/ImageMagick-6/
COPY www/ /var/www/html/
COPY conf/000-default.conf /etc/apache2/sites-available/
COPY scripts/generate-ssl.sh /generate-ssl.sh
RUN chmod +x /generate-ssl.sh
RUN /bin/bash /generate-ssl.sh
EXPOSE 80 443
Is this intended behavior ?
From Docker Documentation - Docerfile Copy:
COPY src dest: If src is any kind of file, it is copied individually along with its metadata. In this case, if ends with a trailing slash /, it will be considered a directory and the contents of will be written at /base().
For your case try and specifiy the dest file:
COPY conf/policy.xml /etc/ImageMagick-6/policy.xml
COPY conf/000-default.conf /etc/apache2/sites-available/000-default.conf
Otherwise i dont see anything wrong with your dockerfile
Related
I'm quite new to Docker. I have a Laravel project I did using Laragon. I would like to create a Docker Compose project out of it. My problem is, when I install composer, even if the terminal is showing me no error, I can't see anything inside of the container.
What I have:
I have a docker-compose.yml with PHP and NGINX services
I have a Dockerfile for the laravel app
Here is the Dockerfile:
FROM php:7.3.19-fpm-stretch
ENV ACCEPT_EULA=Y
# Set working directory
WORKDIR /var/www
# Install dependencies
RUN apt-get update && apt-get install -y \
build-essential \
libpng-dev \
gnupg \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
libzip-dev \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
# Microsoft SQL Server Prerequisites
RUN apt-get update \
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
&& curl https://packages.microsoft.com/config/debian/9/prod.list \
> /etc/apt/sources.list.d/mssql-release.list \
&& apt-get install -y --no-install-recommends \
locales \
apt-transport-https \
&& echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
&& locale-gen \
&& apt-get update \
&& apt-get -y --no-install-recommends install \
unixodbc-dev \
msodbcsql17
# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Install extensions
RUN docker-php-ext-install pdo_mysql mbstring zip exif pcntl
RUN docker-php-ext-configure gd --with-gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ --with-png-dir=/usr/include/
RUN docker-php-ext-install gd
RUN pecl install sqlsrv pdo_sqlsrv xdebug \
&& docker-php-ext-enable sqlsrv pdo_sqlsrv xdebug
# Install composer
RUN curl -sS https://getcomposer.org/installer | \
php -- --install-dir=/usr/bin/ --filename=composer
# Copy composer.lock and composer.json
COPY composer.lock composer.json /var/www/
# Run composer install
RUN composer install --no-scripts --no-autoloader
# Add user for laravel application
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www
# Copy existing application directory contents
COPY . /var/www
RUN composer dump-autoload --optimize
# Copy existing application directory permissions
COPY --chown=www:www . /var/www
# Change current user to www
USER www
# Expose port 9000 and
EXPOSE 9000
# Start php-fpm server
CMD ["php-fpm"]
What I get when I run docker-compose build:
[...]
Step 11/21 : RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin/ --filename=composer
---> Running in 9ee162ed2895
All settings correct for using Composer
Downloading...
Composer (version 1.10.8) successfully installed to: /usr/bin/composer
Use it: php /usr/bin/composer
Removing intermediate container 9ee162ed2895
---> dc1df86b9a7b
Step 12/21 : COPY composer.lock composer.json /var/www/
---> 7bcf92a0a647
Step 13/21 : RUN composer install --no-scripts --no-autoloader
---> Running in f1695442d1db
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 113 installs, 0 updates, 0 removals
- Installing doctrine/inflector (2.0.3): Downloading (100%)
[...] (all composer packages seem to be installed correctly)
Step 17/21 : RUN composer dump-autoload --optimize
---> Running in 18b8c992ace9
Generating optimized autoload files
> Illuminate\Foundation\ComposerScripts::postAutoloadDump
> #php artisan package:discover --ansi
Discovered Package: facade/ignition
Discovered Package: fideloper/proxy
Discovered Package: fruitcake/laravel-cors
Discovered Package: laravel/socialite
Discovered Package: laravel/tinker
Discovered Package: nesbot/carbon
Discovered Package: nunomaduro/collision
Discovered Package: socialiteproviders/manager
Package manifest generated successfully.
Generated optimized autoload files containing 4925 classes
Removing intermediate container 18b8c992ace9
---> d5075486971d
Step 18/21 : COPY --chown=www:www . /var/www
---> 15fcdde49457
Step 19/21 : USER www
---> Running in be9ab541543c
Removing intermediate container be9ab541543c
---> e17d1ae7a052
Step 20/21 : EXPOSE 9000
---> Running in 3e09f6279930
Removing intermediate container 3e09f6279930
---> 7a3649b9df62
Step 21/21 : CMD ["php-fpm"]
---> Running in 0db51df5de3c
Removing intermediate container 0db51df5de3c
---> fbb8e77bb8d1
Successfully built fbb8e77bb8d1
Successfully tagged digitalocean.com/php:latest
The docker-compose build sounds like it's working.
When I run docker-composer up -d and I go to http://localhost, I have a PHP error :
Fatal error: require(): Failed opening required '/var/www/public/../vendor/autoload.php' (include_path='.:/usr/local/lib/php') in /var/www/public/index.php on line 24
What I did so far:
Delete all the containers/images, rebuild
Edit the Dockerfile according to some tutorials I found on the internet
Verified that there's a vendor directory in www. Result : no.
Tried to execute composer install directly into the container (within docker exec -it <container-name> /bin/sh) : Result : /bin/sh: composer: not found.
I can't see where I'm wrong, and I don't understand where exactly does composer had installed its packages, and how to run my project...
Thanks forehand.
I have a Dockerfile including e.g. apache, further installations and my code that is copied to /var/www/html to create a project. After I created the image locally, I export it as a .tar file and upload the image to portainer. Portainer is my productive environment. However, everytime when I want to update the Version and the services that are using my software, I have to update the whole new image which has a size of 800MB. Portainer has furthermore multiple managers, which causes that I have to upload it to each manager.
Because everything keeps the same, except my code that is inserted by the copy part COPY HRmAppBare/ /var/www/html, I thought about the idea, If it is possible to create two images. One Image for the whole Installations (let us say: 1.0-BaseInstall) and a second Image (let us say: 1.9-backend) that only stores my code. Then, for each Version update, I only have to upload the image with the new code and can maybe somehow refere to the 1.0-BaseInstall like e.g. From 1.0-BaseInstall. If the BaseInstall changes (which is really rarely), I could just create a new image for that.
Because I could not find anything about that, I want to know, if this approach is applicable and if yes, how I have to build this?
#start with base Image from php
FROM php:7.3-apache
#install system dependencies and enable PHP modules
RUN apt-get update && apt-get install -y \
libicu-dev \
vim \
cron \
libpq-dev \
libmcrypt-dev \
default-mysql-client \
zip \
unzip \
libzip-dev \
&& docker-php-ext-configure zip --with-libzip \
&& docker-php-ext-install zip \
&& rm -r /var/lib/apt/lists/* \
&& docker-php-ext-configure pdo_mysql --with-pdo-mysql=mysqlnd \
&& docker-php-ext-install \
intl \
mbstring \
pcntl \
pdo_mysql \
opcache \
gettext \
&& pecl install mcrypt-1.0.2 \
&& docker-php-ext-enable mcrypt \
&& rm -rf /var/lib/apt/lists/*
#configure imap for mails
RUN apt-get update && \
apt-get install -y \
libc-client-dev libkrb5-dev && \
rm -r /var/lib/apt/lists/*
RUN docker-php-ext-configure imap --with-kerberos --with-imap-ssl && \
docker-php-ext-install -j$(nproc) imap
#install composer
#RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin/ --filename=composer
#change uid and gid of apache to docker user uid/gid, enable apache module rewrite
RUN usermod -u 1000 www-data && groupmod -g 1000 www-data && a2enmod rewrite
#copy the source code (Can this be in a 2nd image?)
COPY HRmAppBare/ /var/www/html
#Update apache2.conf
RUN echo 'Alias ${COOKIE_PATH} "/var/www/html"' >> /etc/apache2/apache2.conf
RUN echo 'Alias ${COOKIE_PATH}/ "/var/www/html/"' >> /etc/apache2/apache2.conf
#change ownership of our applications
RUN chown -R www-data:www-data /var/www/html/
ENTRYPOINT [ "sh", "-c", "rm /var/www/html/app/tmp/cache/models/* && rm /var/www/html/app/tmp/cache/persistent/* && /var/www/html/app/Console/cake schema update -y && apache2-foreground"]
EXPOSE 80
You can break this into multiple docker files and it would be viable.
If you have no other consumers of the base image it may create confusion and just add more overhead to have to manage versions for your base image and your application but it is certainly viable.
It may be worth it to look into multi-stage builds.
https://github.com/docker/docker.github.io/blob/master/develop/develop-images/multistage-build.md
If the objects on the file system that Docker is about to produce are unchanged between builds, reusing a cache of a previous build on the host is a great time-saver. It makes building a new container really, really fast.
https://thenewstack.io/understanding-the-docker-cache-for-faster-builds/
I'm unsure when you say
Then, for each Version update, I only have to upload the code
I may be misunderstanding the phrase but instead I would suggest having all of the builds done on your build box and have the versioned image pushed to a docker repo for your production box to pull from when you're ready to grab the next version. You shouldn't need to upload your code anywhere. Just the built image to whatever docker repo you store your images on.
Edit: Add in link for creating your own docker repo
https://docs.docker.com/registry/deploying/
Edit 2: To better answer your question
FROM php:7.3-apache AS base
//...rest of your dockerfile until copy
FROM base
#copy the source code (Can this be in a 2nd image?)
COPY HRmAppBare/ /var/www/html
#Update apache2.conf
RUN echo 'Alias ${COOKIE_PATH} "/var/www/html"' >> /etc/apache2/apache2.conf
RUN echo 'Alias ${COOKIE_PATH}/ "/var/www/html/"' >> /etc/apache2/apache2.conf
#change ownership of our applications
RUN chown -R www-data:www-data /var/www/html/
ENTRYPOINT [ "sh", "-c", "rm /var/www/html/app/tmp/cache/models/* && rm /var/www/html/app/tmp/cache/persistent/* && /var/www/html/app/Console/cake schema update -y && apache2-foreground"]
EXPOSE
I have a problem with access rights in a Docker Container. I am copiyng a folder from the host to the docker image into the folder /var/www/html. This folder has a deeper folder structure. Then, I want www-data which is executing apache to have access to the complete /var/www/html folder. I create the container with the following dockerfile.
#start with base Image from php
FROM php:7.3-apache
#install system dependencies and enable PHP modules
RUN apt-get update && apt-get install -y \
libicu-dev \
libpq-dev \
libmcrypt-dev \
mysql-client \
git \
zip \
unzip \
&& rm -r /var/lib/apt/lists/* \
&& docker-php-ext-configure pdo_mysql --with-pdo-mysql=mysqlnd \
&& docker-php-ext-install \
intl \
mbstring \
pcntl \
pdo_mysql \
pdo_pgsql \
pgsql \
opcache
# zip \
# mcrypt \
#configure imap for mails
RUN apt-get update && \
apt-get install -y \
libc-client-dev libkrb5-dev && \
rm -r /var/lib/apt/lists/*
RUN docker-php-ext-configure imap --with-kerberos --with-imap-ssl && \
docker-php-ext-install -j$(nproc) imap
#install mcrypt
RUN apt-get update \
&& apt-get install -y libmcrypt-dev \
&& rm -rf /var/lib/apt/lists/* \
&& pecl install mcrypt-1.0.2 \
&& docker-php-ext-enable mcrypt
#install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin/ --filename=composer
#set our application folder as an environment variable
ENV APP_HOME /var/www/html
#change uid and gid of apache to docker user uid/gid
RUN usermod -u 1000 www-data && groupmod -g 1000 www-data
# enable apache module rewrite
RUN a2enmod rewrite
#COPY Data to html
COPY --chown=www-data:www-data AppBare/ /var/www/html
#change ownership of our applications
RUN chown -R www-data:www-data /var/www/html
#Copy file to start schema update on startup
ENTRYPOINT [ "sh", "-c", "/var/www/html/app/Console/cake schema update -y && /var/www/html/app/Console/cake migration && /usr/sbin/apachectl -D FOREGROUND"]
EXPOSE 80
After I create and start the container, I get the following error message accessing a website of the serving webserver. However it is also loading the website with images that were copied, so basically, the user has access to e.g. images, css and so on.
SplFileInfo::openFile(/var/www/html/app/tmp/cache/models/demo_backend_cake_model_default_backend_dockertest_list):
failed to open stream: Permission denied
When I go into the console of the container and reset the permissions with the chown command, the problem disappears. So the command itself must be right. Also when I create a volume and mount the folder from the host to /var/www/html, everything is working fine.
How can I give the user the full access to the folder? I also tried out to switch give the access before I copy the data, but that's not working also.
About your last comment
The two files are created by the Entrypoint of the code /var/www/html/app/Console/cake schema update -y. So this is executed by the root user. Is it possible to say to execute this as www-data not as root?
The answer is yes. You have to add the following line before your entrypoint :
USER www-data
This way, everything run after this line will be with this user.
I'm going crazy trying to ADD a directory from my host machine to my docker container. When building the container with docker-compose up --build, it seems to ADD just fine, but when I try to access module in my app.py file, I get the ModuleNotFoundError
My DockerFile contains the following:
FROM python:3.7-alpine
RUN apk update && \
apk add --virtual build-deps gcc musl-dev && \
apk add --no-cache postgresql-dev && \
apk add alsa-lib-dev && \
apk add pulseaudio-dev && \
apk add postgresql-dev && \
apk add ffmpeg-dev && \
apk add ffmpeg && \
rm -rf /var/cache/apk/*
COPY /scraper/requirements.txt requirements.txt
RUN pip install -r requirements.txt
ADD /common/testmodel /scraper/testmodel
WORKDIR home/scraper/
ENTRYPOINT ["python3", "-u", "app.py"]
CMD gunicorn -b 0.0.0.0:5000 --access-logfile - "app:app"
Then when building the image, the log shows:
Step 6/9 : ADD /common/testmodel home/scraper/testmodel
---> a7b27854d751
My project structure looks like the following:
-common
-testmodel
-test.py
-scraper
-DockerFile
-requirements
-docker-compose.yml
But in my app.py file, when I run from testmodel.test import TestClass I get ModuleNotFoundError: No module named 'testmodel'
Any help with this problem is greatly appreciated as this how now taken up a much larger chunk of my day that I ever thought it would. Thank you very much.
I may be missing some context but I think you've several issues:
You COPY /scraper... and ADD /common... -- are these directories hanging from root on your local machine?
You set WORKDIR after COPY and ADD but generally (although not required), you'd set this first as a default destination and then you could COPY something . and ADD something . and these destinations (.) would refer to your WORKDIR
You use /home/scraper as your WORKDIR but you don't copy and add your files into it. It will be empty at this point.
Your ENTRYPOINT references app.py but your file is called test.py
One useful debugging tool is to shell into containers to e.g. examine the directory structure to confirm it's as expected. Assuming your image is called scraper, you could:
docker build \
--tag=scraper \
--file=scraper/Dockerfile \
. # Don't forget the period ;-)
Then Alpine's shell is called ash:
docker run \
--interactive \
--tty \
scraper:latest ash
Or, if your Dockerfile has an ENTRYPOINT, then override it using:
docker run \
--interactive \
--tty \
--entrypoint=ash \
scraper:latest
and then you could browse the container's directory structure:
You'll default to /home/scraper (WORKDIR):
/home/scraper # ls -l
total 0
You may examine /scraper using:
/home/scraper # apk install tree
/home/scraper # tree /scraper
/scraper
└── testmodel
└── test.py
1 directory, 1 file
I'm not entirely clear as to what would be the correct solution for you but I hope this helps get you progressed:
FROM python:3.7-alpine
RUN apk update && \
apk add --virtual build-deps gcc musl-dev && \
apk add --no-cache postgresql-dev && \
apk add alsa-lib-dev && \
apk add pulseaudio-dev && \
apk add postgresql-dev && \
apk add ffmpeg-dev && \
apk add ffmpeg && \
rm -rf /var/cache/apk/*
WORKDIR home/scraper/
COPY scraper/requirements.txt .
RUN pip install -r requirements.txt
ADD common/testmodel .
ENTRYPOINT ["python3", "-u", "test.py"]
CMD gunicorn -b 0.0.0.0:5000 --access-logfile - "test:app"
I have some server and clients who are communicating using gRPC/golang. Now I want to containerize my application but the size of the docker image containing goland execution and grpc support is larger(more than 1GB). I would like to decrease the size of the docker image.
The required golang version is 1.9 and higher. Here is the Dockerfile script is given. If there is other way please suggest it.
FROM golang:1.11
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
CMD ["go", "run", "helloworld/greeter_server/main.go"]
try to make a multistage docker image like this
# Compile stage
FROM golang:1.11 as build-env
RUN apt-get update && \
apt-get -y install git unzip build-essential autoconf libtool
RUN git clone https://github.com/google/protobuf.git && \
cd protobuf && \
./autogen.sh && \
./configure && \
make && \
make install && \
ldconfig && \
make clean && \
cd .. && \
rm -r protobuf
RUN go get google.golang.org/grpc
RUN go get github.com/golang/protobuf/protoc-gen-go
RUN ls -la
WORKDIR /helloworld
COPY . /helloworld
RUN protoc -I helloworld/ helloworld/helloworld.proto --go_out=plugins=grpc:helloworld
RUN go build -o server helloworld/greeter_server/main.go
# Making image
FROM alpine:3.8 AS host
RUN apk add --no-cache \
ca-certificates
COPY --from=build-env /helloworld/server /
# copy any other files you need
WORKDIR /
EXPOSE 8000
CMD ["server"]
You can try to use distroless base images and multi-stage builds. That might help you.