Docker copy or add deletes existing files - docker

I have several files in a directory on the host machine which I am trying to copy to the Docker container.
The problem is that the files do get copied to the destination but all the existing files inside the destination directory get removed.
Before adding these new ADD lines to my Dockerfile, I had like 20 jar files in the lib directory, by adding these ADD lines the two crowd files below, all 20 existing jar files get deleted and the directory will now contains only two crowd files which were just copied from the host into the container!
I tried without the user ROOT but it would not copy the server.xml and tomcat.keystore
FROM guacamole/guacamole
RUN sed -i 's/redirectPort="8443"/redirectPort="8443" server="" secure="true"/g' /usr/local/tomcat/conf/server.xml \
&& sed -i 's/<Server port="8005" shutdown="SHUTDOWN">/<Server port="-1" shutdown="SHUTDOWN">/g' /usr/local/tomcat/conf/server.xml \
&& rm -rf /usr/local/tomcat/webapps/docs/* \
&& rm -rf /usr/local/tomcat/webapps/examples/* \
&& rm -rf /usr/local/tomcat/webapps/manager/* \
&& rm -rf /usr/local/tomcat/webapps/host-manager/*
WORKDIR /usr/local/tomcat
USER root
COPY server.xml conf/server.xml
RUN chmod 660 conf/server.xml
USER root
ADD tomcat.keystore /usr/local/tomcat/
RUN chmod 644 tomcat.keystore
RUN chown root:staff /usr/local/tomcat/tomcat.keystore
ADD ./lib/crowd-auth-filter-1.0.0.jar /usr/local/tomcat/webapps/guacamole/WEB-INF/lib/crowd-auth-filter-1.0.0.jar
ADD ./lib/crowd-filter.properties /usr/local/tomcat/webapps/guacamole/WEB-INF/lib/crowd-filter.properties
RUN chmod 644 /usr/local/tomcat/webapps/guacamole/WEB-INF/lib/crowd-filter.properties
ADD web.xml /usr/local/tomcat/webapps/guacamole/WEB-INF/web.xml
CMD /usr/local/tomcat/bin/shutdown.sh && /usr/local/tomcat/bin/startup.sh
docker-compose.yml:
version: '2'
services:
guacd:
hostname: guacd
image: guacamole/guacd
restart: always
container_name: guacd
mysql:
hostname: mysql
image: mysql:5.7
volumes:
- ./tmp/scripts:/docker-entrypoint-initdb.d
restart: always
container_name: mysql
ports:
- "3306:3306"
environment:
- MYSQL_DATABASE="guacamole"
- MYSQL_USER="guacamole"
- MYSQL_PASSWORD="password"
- MYSQL_ROOT_PASSWORD="password"
guacamole:
build: .
image: mine/guacamole
restart: always
ports:
- "8443:8443"
links:
- guacd
- mysql
container_name: guacamole
environment:
- GUACD_HOSTNAME=guacd
- GUACAMOLE_HOME=/opt/guacamole-home
volumes:
tmp-scripts:
To get things started:
1) I build the guacamole image with docker build . --no-cache -t mine/guacamole
2) Start the containers and create the services by running: docker-compose up --force-recreate -d
Can someone please help?
Thanks

Related

how can i maintain my data and public links in owncloud image after docker push&pull

I have a web server program which requires pdf files from owncloud server. I'm making installation code via docker-compose & docker hub.
I use Ubuntu 20.04LTS and Docker Compose v2.1.0.
Here is the process
store pdf files and create public links in owncloud docker container(under /var/www/owncloud/data)
create new images(both owncloud, mariadb) and tags from container by code below
docker commit 5cba8bf76904
docker tag 9315184e23f5 DOCKERID/docker-mariadb
docker push DOCKERID/docker-mariadb
pull those images in another new fresh Ubuntu server, using docker-compose up
After this process, when I connect to owncloud, running on a new fresh ubuntu server, there are no pdf files and all those configs are intialized (owncloud account, mariadb database configs)
and the owncloud start-up page(config admin account and database page) is opened.
My docker-compose, Dockerfiles are below(related parts only)
docker-compose.yml
owncloud:
#build: ./dockerfiles/owncloud/
image: "dockerhubid/docker-owncloud"
container_name: chatbot_owncloud
restart: always
networks:
- chatbot_network
depends_on:
- mariadb
volumes:
- 'owncloud_php:/var/www/owncloud'
command: php-fpm7.4 -F -R
mariadb:
# build: ./dockerfiles/mariadb/
image: dockerhubid/docker-mariadb
container_name: mariadb
restart: always
expose:
- '3306'
networks:
- chatbot_network
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_USER=owncloud
- MYSQL_PASSWORD=password
- MYSQL_DATABASE=owncloud
command: ["--max-allowed-packet=128M", "--innodb-log-file-size=64M"]
nginx:
#build: ./dockerfiles/nginx/
image: "dockerhubid/docker-nginx"
container_name: chatbot_nginx
restart: always
depends_on:
- owncloud
volumes:
- ./dockerfiles/certbot/conf:/etc/letsencrypt
- ./dockerfiles/certbot/www:/var/www/certbot
volumes_from:
- 'owncloud:ro'
networks:
- chatbot_network
ports:
- '80:80'
- '3000:3000'
- '8883:8883'
- '8884:8884'
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
certbot:
image: certbot/certbot
container_name: chatbot_certbot
networks:
- chatbot_network
volumes:
- ./dockerfiles/certbot/conf:/etc/letsencrypt
- ./dockerfiles/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
owncloud Dockerfile
FROM ubuntu:20.04
EXPOSE 9000
ARG DEBIAN_FRONTEND=noninteractive
# dependencies
RUN apt update && apt upgrade -y
RUN apt install -y php-bz2 php-curl php-gd php-imagick php-intl php-mbstring php-xml php-zip php-mysql php-fpm wget zip vim
# owncloud
RUN wget https://download.owncloud.org/community/owncloud-10.5.0.zip
RUN unzip owncloud-10.5.0.zip -d /var/www/
RUN rm /owncloud-10.5.0.zip
WORKDIR /var/www/owncloud
RUN chown www-data:www-data -R /usr/bin/php /var/www/owncloud/
RUN chmod -R 755 /var/www/owncloud/
# php-fpm setup
RUN sed -i 's+/run/php/php7.4-fpm.sock+9000+g' /etc/php/7.4/fpm/pool.d/www.conf
ADD init.sh /docker-entrypoint-initdb.d/
RUN chmod 755 /docker-entrypoint-initdb.d/init.sh
mariadb Dockerfile
from mariadb:10.5
EXPOSE 3306
ARG DEBIAN_FRONTEND=noninteractive
USER root
ADD init.sql /docker-entrypoint-initdb.d/
RUN chmod 755 /docker-entrypoint-initdb.d/init.sql
How can I maintain those files and public links?
Why are those things removed after docker hub push&pull?
I tried it with the owncloud official image first, but by my investigation official image stores data in external docker volume.
I thought that's why my data is gone after docker push&pull.
so I'm trying it by manual installation.

When does Dockerfile executes, and why groupadd and usereadd not working

I have two questions:
Dockerfile has two command, add group and user, both named www, but didn't create.
How to stop the container created by docker-compose up -d.
I followed this article:
https://www.digitalocean.com/community/tutorials/how-to-set-up-laravel-nginx-and-mysql-with-docker-compose
git clone https://github.com/laravel/laravel.git mylaravel9
Edit docker-compose.yml and Dockerfile, then
docker-compose up -d
The browser "http://localhost" shows, but with an error, a log file with permission problem. This was solved, but not really solved.
docker-compose.yml
version: '3'
services:
#PHP Service
app:
build:
context: .
dockerfile: Dockerfile
image: php:8.1.4-fpm
container_name: app
restart: unless-stopped
tty: true
environment:
SERVICE_NAME: app
SERVICE_TAGS: dev
working_dir: /var/www
volumes:
- ./:/var/www
- ./php/local.ini:/usr/local/etc/php/conf.d/local.ini
networks:
- app-network
#Nginx Service
webserver:
image: nginx:alpine
container_name: webserver
restart: unless-stopped
tty: true
ports:
- "80:80"
- "443:443"
volumes:
- ./:/var/www
- ./nginx/conf.d/:/etc/nginx/conf.d/
networks:
- app-network
#MySQL Service
db:
image: mysql:5.7.22
container_name: db
restart: unless-stopped
tty: true
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: laravel
MYSQL_ROOT_PASSWORD: 123456
SERVICE_TAGS: dev
SERVICE_NAME: mysql
volumes:
- dbdata:/var/lib/mysql/
- ./mysql/my.cnf:/etc/mysql/my.cnf
networks:
- app-network
#Docker Networks
networks:
app-network:
driver: bridge
#Volumes
volumes:
dbdata:
driver: local
Dockerfile:
FROM php:8.1.4-fpm
# Copy composer.lock and composer.json
COPY composer.lock composer.json /var/www/
# Set working directory
WORKDIR /var/www
# Install dependencies
RUN apt-get update && apt-get install -y \
build-essential \
libpng-dev \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Install extensions
RUN docker-php-ext-install pdo_mysql zip exif pcntl
RUN docker-php-ext-configure gd --with-gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ --with-png-dir=/usr/include/
RUN docker-php-ext-install gd
# Install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
# Add user for laravel application
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www
# Copy existing application directory contents
COPY . /var/www
# Copy existing application directory permissions
COPY --chown=www:www . /var/www
# Change current user to www
USER www
# Expose port 9000 and start php-fpm server
EXPOSE 9000
CMD ["php-fpm"]
The first question:
When first time do
docker-compose up -d
It pulls things, and run the commands in Dockerfile. There is a problem, which is also solved
failed to solve: executor failed running [/bin/sh -c docker-php-ext-install pdo_mysql mbstring zip exif pcntl]: exit code: 1
A post says the "mbstring" needs to be taken off. Ok, the Dockerfile really used. But there are two commands not working
# Add user for laravel application
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www
Because
docker exec -it app bash
and in the app container's shell
cd /var/www
ls -l
I saw the group and owner is number 1000, not www. Then
cat /etc/passwd
The user and group of "www" doesn't exist! Why?
I manually add the www group and user, and do chmod, the problem of log file permission is solved. But why www doesnt exist? The add commands are in the Dockerfile.
The second question
Exit the app shell, back to Ubuntu
docker ps
Shows three conatiners: php, nginx, mysql. But in docker interface(Windows 11), there is a container named by the folder mylaravel9.
docker stop mylaravel9
It says:
Error response from daemon: No such container: mylaravel9
So I can only stop the whole thing in the docker UI? If I want to use command, I have to stop the three containers? Is it?
There are two significant problems in the setup you show.
volumes:
- ./:/var/www
In the Dockerfile, you COPY --chown content to a different user, but then this volumes: mount hides everything the image setup does in the /var/www directory and replaces it with content from the host. Inside the container you'll see the host's numeric user ID and the possibly-unrelated code from the host. I'd recommend just deleting this line.
build: .
image: php:8.1.4-fpm
This combination tells Compose to build your application from its Dockerfile, then to label the result as the original php:8.1.4-fpm image. When you re-run docker-compose build it will start from the thing labeled as php:8.1.4-fpm, which means you're repeatedly reinstalling your application on top of itself.
Delete the image: line if you're not planning to push the built image to a registry (and if you are, use the name and tag for the built image, not the base image). docker pull php:8.1.4-fpm manually to make sure you have a "good" copy of this base image.
In the context of a Compose project, you don't usually need to use basic docker commands; there are docker-compose wrappers for most operations. If you want to update your application and restart its container it should be enough to
docker-compose build
docker-compose up -d
will will recreate the changed app container but leave the others alone. If you do need to stop an individual container for some reason ("stopped" is a somewhat unusual state) then docker-compose stop can do it.

dial tcp 127.0.0.1:8080: connect: connection refused. go docker app

I have two apps in go language. user_management app, which I run (docker-compose up --build) first, then I run(docker-compose up --build) sport_app. sport_app is dependent from user_management app.
sport_app Dockerfile file as below.
FROM golang:alpine
RUN apk update && apk upgrade && apk add --no-cache bash git openssh curl
WORKDIR /go-sports-entities-hierarchy
COPY . /go-sports-entities-hierarchy/
RUN rm -rf /go-sports-entities-hierarchy/.env
RUN go mod download
RUN chmod +x /go-sports-entities-hierarchy/scripts/*
RUN ./scripts/build.sh
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.2.1/wait /wait
RUN chmod +x /wait
ENV GIN_MODE="debug" \
GQL_SERVER_HOST="localhost" \
GQL_SERVER_PORT=7777 \
ALLOWED_ORIGINS=* \
USER_MANAGEMENT_SERVER_URL="http://localhost:8080/user/me" \
# GQLGen config
GQL_SERVER_GRAPHQL_PATH="graphql" \
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED=true \
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH="playground" \
# Export necessary port
EXPOSE 7777
CMD /wait && ./scripts/run.sh
sport_app docker-compose.yml file as below.
version: '3'
volumes:
postgres_data:
driver: local
services:
go-sports-entities-hierarchy:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
WAIT_HOSTS: postgres:5432
# Web framework config
GIN_MODE: debug
GQL_SERVER_HOST: go-sports-entities-hierarchy
GQL_SERVER_PORT: 7777
ALLOWED_ORIGINS: "*"
USER_MANAGEMENT_SERVER_URL: http://localhost:8080/user/me
# GQLGen config
GQL_SERVER_GRAPHQL_PATH: graphql
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED: "true"
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH: playground
ports:
- 7777:7777
depends_on:
- postgres
- redisearch
go-sports-events-workflow:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
WAIT_HOSTS: postgres:5432
# Web framework config
GIN_MODE: debug
GQL_SERVER_HOST: go-sports-events-workflow
GQL_SERVER_PORT: 7778
ALLOWED_ORIGINS: "*"
# GQLGen config
GQL_SERVER_GRAPHQL_PATH: graphql
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED: "true"
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH: playground
depends_on:
- postgres
- redisearch
- go-sports-entities-hierarchy
user_management app Dockerfile as below:
FROM golang:alpine
RUN apk update && apk add --no-cache git ca-certificates && update-ca-certificates
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
# Move to working directory /build
WORKDIR /build
# Copy and download dependency using go mod
COPY go.mod .
COPY go.sum .
RUN go mod download
# Copy the code into the container
COPY . .
# Build the application
RUN go build -o main .
# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist
# Copy binary from build to main folder
RUN cp -r /build/html .
RUN cp /build/main .
# Environment Variables
ENV DB_HOST="127.0.0.1" \
APP_PROTOCOL="http" \
APP_HOST="localhost" \
APP_PORT=8080 \
ALLOWED_ORIGINS="*"
# Export necessary port
EXPOSE 8080
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.2.1/wait /wait
RUN chmod +x /wait
# Command to run when starting the container
CMD /wait && /dist/main
user_management app docker-compose.yml file as below:
version: '3'
volumes:
postgres_data:
driver: local
services:
postgres:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- 5432:5432
go-user-management:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
# Postgres Details
DB_PORT: 5432
# APP details
APP_PROTOCOL: http
APP_HOST: localhost
APP_PORT: 8080
# System Configuration Details
ALLOWED_ORIGINS: "*"
ports:
- 8080:8080
depends_on:
- postgres
In sport_app I write below code and get error:
client := resty.New()
resp, err := client.R().SetHeader("Content-Type", "application/json").SetHeader("Authorization", "Bearer "+token).Get("http://localhost:8080/user/me")
Error is: Get "http://localhost:8080/user/me": dial tcp 127.0.0.1:8080: connect: connection refused:"
This API(http://localhost:8080/user/me) is written in the user_management app and this is working, I check with the postman.
I already read this question answers, but can not solve my problem.
I am new to docker, please help.
For communicating between multiple docker-compose clients, you need to make sure that the containers you want to talk to each other are on the same network.
For example, (edited for brevity) here you have one of the docker-compose.yml
# sport_app docker-compose.yml
version: '3'
services:
go-sports-entities-hierarchy:
...
networks:
- some-net
go-sports-events-workflow
...
networks:
- some-net
networks:
some-net:
driver: bridge
And the other docker-compose.yml
# user_management app docker-compose.yml
version: '3'
services:
postgres:
...
networks:
- some-net
go-user-management
...
networks:
- some-net
networks:
some-net:
external: true
Note: Your app’s network is given a name based on the project name, which is based on the name of the directory it lives in, in this case a prefix user_ was added.
They can then talk to each other using the service name, i.e. go-user-management, etc.
You can, after running the docker-compose up --build commands, run the docker network ls command to see it, then docker network inspect bridge, etc.

How to create docker compose for hybris commerce

I would like to run hybris with the docker.
I am trying to create a docker to run hybris.
Can anyone help me with this?
This is my code:
UPDATE Question:
When I enter the container (ubuntu) and try to build hybris, there is always an error.
I created the docker-compose and the dockerfile. However, when I run ant clean all in the container the build always fails.
[![![enter image description here][1]][1]
version: '3.3'
services:
db:
image: mysql:5.6
volumes:
- //C/dockerVolumes/db_local_hybris:/var/lib/mysql
container_name: mysql_hybris
hostname: mysql_hybris
ports:
- "3307:3306"
environment:
- MYSQL_ROOT_PASSWORD=***
- MYSQL_DATABASE=***
- MYSQL_USER=***
hybris:
image: miltex/jdk:hybrisDevs
build:
context: .
container_name: hybris_dev
hostname: hybris_dev
ports:
- "8009:8009"
- "8010:8010"
- "9001:9001"
- "9002:9002"
- "1099:1099"
tty: true
links:
- db
Dockerfile
FROM miltex/jdk:1.8
#update
RUN apt-get update
## Run Initial Ant ##
RUN mkdir -p /app/hybris_dev
COPY ./hybris/HYBRISCOMM6700P_10-80003492.ZIP /app/hybris_dev/
RUN cd /app/hybris_dev && unzip HYBRISCOMM6700P_10-80003492.ZIP
#COPY ./bin/hybris-wrapper.sh /app/hybris_dev/hybris/bin/platform/
RUN mkdir /app/hybris_dev/hybris/bin/custom
COPY ./src/custom /app/hybris_dev/hybris/bin/custom
RUN cd /app/hybris_dev/installer && rm -R recipes
RUN mkdir /app/hybris_dev/installer/recipes
COPY ./src/custom/recipes /app/hybris_dev/installer/recipes
#CMD /app/hybris_dev/intaller/install.sh -r local setup ; /app/hybris_dev/hybris/bin/custom/platform/setantenv.sh ; ant clean all
## Copy hybris-wrapper to configure template properties at runtime ##
COPY ./bin/hybris-wrapper.sh /hybris-wrapper.sh
## Expose AJP S-AJP HTTP HTTPS RMI ports ##
EXPOSE 8009 8010 9001 9002 1099
RUN chmod -R 777 /app
#RUN chmod +x /hybris-wrapper.sh
# Run Hybris server
#ENTRYPOINT ["/hybris-wrapper.sh"]
[1]: https://i.stack.imgur.com/uoK3E.png
The mistake was the image I was using before. Now this dockerhub miltex:jdk:1.8 image is working perfectly.

Docker volume is not fully sync directory for one container

I've created simple project for Symfony4 based on php7.3+mariadb via docker-compose. I used Docker for Windows 10 (x64)
It works correctly at one machine but at laptop it doesn't sync correctly with container.
In root folder I have standard Symfony structure with docker files like:
- /config
- /public
- /src
....
- /env
- /docker
- .env
- docker-compose.yaml
...
My actions in Git Bash to start app:
docker-compose build
it works correctly, all actions were finished successfully
docker-compose up -d
it works correctly, both containers run successfully
docker-compose exec app bash
works correctly, console starts
ls
result is docker env
it syncs only 2 directories - docker and env
docker dir was synced not in full mode - only subdirectories structure without files
I tried to detect what reason can be for problem with files sync but I haven't enough knowledge and experience with Docker. docker-compose logs have no errors.
Maybe somebody can help how to detect the reason? It starts once time but after reboot problem occurs again...
docker-compose.yaml:
version: '3'
services:
app:
restart: unless-stopped
build:
context: .
dockerfile: docker/webserver-apache/Dockerfile
image: php:7.3.1-apache-stretch
volumes:
- "./docker/webserver-apache/sites-enabled:/etc/apache2/sites-enabled:ro"
- "./:/var/www/html"
ports:
- 8080:80
networks:
- dphptrainnet
mariadb:
restart: unless-stopped
image: mariadb:10.4.1
networks:
- dphptrainnet
volumes:
- ./env/mariadb/data:/var/lib/mysql
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_PASSWORD}
networks:
dphptrainnet:
Dockerfile:
FROM php:7.3.1-apache-stretch
# Setting up constants for an environment
ENV PHP_MEMORY_LIMIT 512M
RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
php composer-setup.php && \
php -r "unlink('composer-setup.php');" && \
mv composer.phar /usr/local/bin/composer
RUN apt-get update && \
apt-get install -y curl vim git zip unzip
# Setting up httpd issues
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf
RUN a2enmod rewrite headers && /etc/init.d/apache2 restart
RUN echo "127.0.0.1 dockertrain.local" >> /etc/hosts
WORKDIR "/var/www/html"
RUN a2enmod rewrite
I've found only one working solution - reshare drive for Docker:
1. Disable shared disk, click Apply
2. Enable shared disk, click Apply
3. Restart application - files were synced
But how I should detect there any problems with drive access? No errors, no logs....

Resources