Docker compose throws permission denied on mac - docker

I am using this on Mac. Creating and running the docker image works fine. But with docker-compose I get the following error:
"exec: \"./boot.sh\": permission denied": unknown
Dockerfile:
FROM python:3.7-alpine
RUN adduser -D teamreacher
WORKDIR /home/teamreacher
# copy and install dependencies
COPY ./requirements.txt requirements.txt
RUN python -m venv venv
RUN venv/bin/pip install --upgrade pip
RUN venv/bin/pip install -r requirements.txt
# copy the app
COPY . .
RUN chmod +x boot.sh # Giving execution permissions here...
RUN chown -R teamreacher:teamreacher ./
USER teamreacher
# expose port and run server
EXPOSE 5000
CMD ["./boot.sh"] # ...so why do I get permission denied here?
Docker compose file:
version: '3'
services:
teamreacher-server:
build: ./server
volumes:
- ./server:/home/teamreacher
ports:
- 5000:5000
I've looked through similiar questions but none provided an answer for my specific problem.
boot.sh
#!/bin/sh
source venv/bin/activate
exec gunicorn -b :5000 --access-logfile - --error-logfile - wsgi:app

By bind-mounting your ./server directory into the same location where boot.sh is located on the conotainer, you're rewriting the permissions so that it is no longer executable.
Please see the below demonstration:
TJs-MacBook-Pro:stackoverflow tj$ ls -lah
total 8
drwxr-xr-x 4 tj wheel 128B Dec 23 14:58 .
drwxrwxrwt 15 root wheel 480B Dec 23 15:05 ..
-rw-r--r-- 1 tj wheel 143B Dec 23 14:57 docker-compose.yaml
drwxr-xr-x 6 tj wheel 192B Dec 23 15:03 server
TJs-MacBook-Pro:stackoverflow tj$ ls -lah server/
total 24
drwxr-xr-x 6 tj wheel 192B Dec 23 15:03 .
drwxr-xr-x 4 tj wheel 128B Dec 23 14:58 ..
-rw------- 1 tj wheel 177B Dec 23 15:06 .ash_history
-rw-r--r-- 1 tj wheel 508B Dec 23 14:56 Dockerfile
-rw-r--r-- 1 tj wheel 105B Dec 23 14:56 boot.sh
-rw-r--r-- 1 tj wheel 0B Dec 23 14:56 requirements.txt
TJs-MacBook-Pro:stackoverflow tj$ docker-compose build
Building teamreacher-server
Step 1/13 : FROM python:3.7-alpine
---> 020295c920c6
Step 2/13 : RUN adduser -D teamreacher
---> Using cache
---> 7dbd3131c941
Step 3/13 : WORKDIR /home/teamreacher
---> Using cache
---> d2754b1b8dc2
Step 4/13 : COPY ./requirements.txt requirements.txt
---> Using cache
---> 2d468491a297
Step 5/13 : RUN python -m venv venv
---> Using cache
---> 0c135fa6f980
Step 6/13 : RUN venv/bin/pip install --upgrade pip
---> Using cache
---> ea7df7153a5a
Step 7/13 : RUN venv/bin/pip install -r requirements.txt
---> Using cache
---> db540a631c19
Step 8/13 : COPY . .
---> 174a91493622
Step 9/13 : RUN chmod +x boot.sh # Giving execution permissions here...
---> Running in b9cedb0f163a
Removing intermediate container b9cedb0f163a
---> 6d5ce7df1969
Step 10/13 : RUN chown -R teamreacher:teamreacher ./
---> Running in 2e672e8ac6ef
Removing intermediate container 2e672e8ac6ef
---> 48342c4c31e3
Step 11/13 : USER teamreacher
---> Running in 4d3bc99fb515
Removing intermediate container 4d3bc99fb515
---> 11bdc7be7f84
Step 12/13 : EXPOSE 5000
---> Running in 0924db9c175b
Removing intermediate container 0924db9c175b
---> 0d96d773f42f
Step 13/13 : CMD ["./boot.sh"] # ...so why do I get permission denied here?
---> Running in 4a33ab4e342a
Removing intermediate container 4a33ab4e342a
---> a9acefb4eeac
Successfully built a9acefb4eeac
Successfully tagged stackoverflow_teamreacher-server:latest
TJs-MacBook-Pro:stackoverflow tj$ docker run stackoverflow_teamreacher-server ls -lah /home/teamreacher
total 24
drwxr-sr-x 1 teamreac teamreac 4.0K Dec 23 20:07 .
drwxr-xr-x 1 root root 4.0K Dec 23 19:56 ..
-rw------- 1 teamreac teamreac 177 Dec 23 20:06 .ash_history
-rw-r--r-- 1 teamreac teamreac 508 Dec 23 19:56 Dockerfile
-rwxr-xr-x 1 teamreac teamreac 105 Dec 23 19:56 boot.sh
-rw-r--r-- 1 teamreac teamreac 0 Dec 23 19:56 requirements.txt
drwxr-sr-x 1 teamreac teamreac 4.0K Dec 23 19:59 venv
TJs-MacBook-Pro:stackoverflow tj$ docker run -v $(pwd)/server:/home/teamreacher stackoverflow_teamreacher-server ls -lah /home/teamreacher
total 16
drwxr-xr-x 6 teamreac teamreac 192 Dec 23 20:03 .
drwxr-xr-x 1 root root 4.0K Dec 23 19:56 ..
-rw------- 1 teamreac teamreac 177 Dec 23 20:06 .ash_history
-rw-r--r-- 1 teamreac teamreac 508 Dec 23 19:56 Dockerfile
-rw-r--r-- 1 teamreac teamreac 105 Dec 23 19:56 boot.sh
-rw-r--r-- 1 teamreac teamreac 0 Dec 23 19:56 requirements.txt

It turns out I had to give the file boot.sh execution permissions on my mac as well. That means running:
chmod +x boot.sh
in my project directory.

Related

Why does my docker container have files from a different image?

My application requires two docker files. I am building them with docker compose. The compose file is like this
version: '3.4'
services:
cors:
container_name: cors
image: cors:latest
build:
context: ./compose/cors-anywhere
dockerfile: Dockerfile
user: "node"
ports:
- 8081:8081
command: node server.js
vite:
container_name: vite
image: vite:lts
build:
context: ./compose/vite
user: "node"
tty: true
working_dir: /app
ports:
- "5173:5173"
volumes:
- ./vite/:/app
command: /start
cors' Dockerfile is like this
ARG NODE_VERSION=lts
FROM node:${NODE_VERSION}
ENV NODE_ENV=production
ENV NODE_PATH=/usr/local/lib/node_modules
ARG version=latest
RUN npm install -g cors-anywhere#$version
COPY server.js .
CMD ["node", "server.js"]
EXPOSE 8081
and vite's docker file is like this
ARG NODE_VERSION=latest
FROM node:${NODE_VERSION}
RUN npm install -g npm#latest
COPY ./entrypoint /entrypoint
RUN sed -i 's/\r$//g' /entrypoint
RUN chmod +x /entrypoint
COPY ./start /start
RUN sed -i 's/\r$//g' /start
RUN chmod +x /start
ENTRYPOINT ["/entrypoint"]
vite's entrypoint script is like this
#!/bin/bash
set -o errexit
set -o pipefail
set -o nounset
until cd /app && npm install
do
echo "Retrying npm install"
done
exec "$#"
After the images have been built, when the cors container comes 'up' it logs lots of errors because it is trying to run vite's entrypoint script. I don't understand why entrypoint has been copied to the cors image. Here is the file structure for the cors container
node#d960251bf35e:/$ ls -l
total 80
drwxr-xr-x 1 root root 4096 Jan 11 03:25 bin
drwxr-xr-x 2 root root 4096 Dec 9 19:15 boot
drwxr-xr-x 5 root root 340 Jan 12 10:52 dev
-rwxr-xr-x 1 root root 139 Jan 11 21:59 entrypoint
drwxr-xr-x 1 root root 4096 Jan 12 10:52 etc
drwxr-xr-x 1 root root 4096 Jan 11 07:59 home
drwxr-xr-x 1 root root 4096 Jan 11 03:25 lib
drwxr-xr-x 2 root root 4096 Jan 9 00:00 media
drwxr-xr-x 2 root root 4096 Jan 9 00:00 mnt
drwxr-xr-x 1 root root 4096 Jan 11 07:59 opt
dr-xr-xr-x 211 root root 0 Jan 12 10:52 proc
drwx------ 1 root root 4096 Jan 11 07:59 root
drwxr-xr-x 3 root root 4096 Jan 9 00:00 run
drwxr-xr-x 1 root root 4096 Jan 11 03:24 sbin
-rw-r--r-- 1 root root 2023 Jan 11 15:24 server.js
drwxr-xr-x 2 root root 4096 Jan 9 00:00 srv
-rwxr-xr-x 1 root root 71 Jan 11 21:59 start
dr-xr-xr-x 13 root root 0 Jan 12 10:52 sys
drwxrwxrwt 1 root root 4096 Jan 11 07:59 tmp
drwxr-xr-x 1 root root 4096 Jan 9 00:00 usr
drwxr-xr-x 1 root root 4096 Jan 9 00:00 var
What am I not understanding?

`npm build` and `COPY --from` from multistage Dockerfiles fails with `COPY failed: stat /mnt/data/docker/overlay2/xxxxx/merged/app/build

I have a multistage Dockerfile which copies some parts from the first image to the second one, which always fails with this line
COPY --from=react-builder /app/build /var/www
with this error
Step 17/22 : COPY --from=react-builder /app/build /var/www
COPY failed: stat /mnt/data/docker/overlay2/901e2feb2323e7fc6d9de0f3e64e84d16d1e935e198bf19f15021af0eb6134d6/merged/app/build: no such file or directory
Here is the Dockerfile
FROM node:10-slim AS react-builder
ARG REACT_APP_ENV
ENV REACT_APP_ENV $REACT_APP_ENV
WORKDIR app
COPY . .
RUN cat /app/.env | grep = | sort | sed -e 's|REACT_APP_\([a-zA-Z_]*\)=\(.*\)|REACT_APP_\1=NGINX_REPLACE_\1|' > /app/.env.local
RUN npm config set registry=https://npm.company.com
RUN npm config set "strict-ssl" false
RUN npm install
RUN npm build
FROM nginx:1.15
COPY nginx.conf.template /etc/nginx/conf.d/nginx.conf.template
COPY gcp-deploy/htpasswd /etc/nginx/htpasswd
COPY --from=react-builder /app/.env.local /etc/nginx/conf.d/
RUN NGINX_SUB_FILTER=$(cat /etc/nginx/conf.d/.env.local | grep '=' | sort | sed -e 's/REACT_APP_\([a-zA-Z_]*\)=\(.*\)/sub_filter\ \"NGINX_REPLACE_\1\" \"$\{\1\}\";/') && cat /etc/nginx/conf.d/nginx.conf.template | sed -e "s|LOCATION_SUB_FILTER|$(echo $NGINX_SUB_FILTER)|" | sed -u 's|}";\ |}";\n\t\t|g' > /etc/nginx/conf.d/default.conf.template
COPY --from=react-builder /app/build /var/www
# This is a hack around the envsubst nginx config. Because we have `$uri` set
# up, it would replace this as well. Now we just reset it to its original value.
ENV uri \$uri
# Default config
ENV REST_URI " https://whatever.wherever.com/"
ENV PORT 80
ENV SERVER_NAME _
CMD ["sh", "-c", "envsubst < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"]
Expected behavior
Running docker build . simply builds the image without issues
Actual behavior
Build always fails with the same COPY --from statement. There is a second COPY --from command which always passes. If i switch the order I still get the same error:
Step 17/22 : COPY --from=react-builder /app/build /var/www
COPY failed: stat /mnt/data/docker/overlay2/901e2feb2323e7fc6d9de0f3e64e84d16d1e935e198bf19f15021af0eb6134d6/merged/app/build: no such file or directory
Output of docker version:
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.39 (downgraded from 1.40)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:25:41 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.0
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:19:08 2018
OS/Arch: linux/amd64
Experimental: false
Output of docker info:
Client:
Debug Mode: false
Server:
Containers: 17
Running: 1
Paused: 0
Stopped: 16
Images: 374
Server Version: 18.09.0
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.0-1062.9.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.701GiB
Name: devs0211
ID: 25FQ:7G3D:T46W:4DTJ:VV4A:V7JG:DZ53:C2E4:O7JN:WUP6:XWPA:3FX4
Docker Root Dir: /mnt/data/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://docker.company.com/
Live Restore Enabled: true
Product License: Community Engine
Some more details of the build process incl. an additional ls -ls /app
$ docker build . --no-cache
[sudo] password for sc146082:
Your password will expire in 4 day(s).
Sending build context to Docker daemon 3.561MB
Step 1/23 : FROM node:10-slim as react-builder
---> 92eff963766f
Step 2/23 : ARG REACT_APP_ENV
---> Running in 13e997164e2c
Removing intermediate container 13e997164e2c
---> 8edcfa0931a8
Step 3/23 : ENV REACT_APP_ENV $REACT_APP_ENV
---> Running in 17fa11f08e3e
Removing intermediate container 17fa11f08e3e
---> a906714bef73
Step 4/23 : WORKDIR /app
---> Running in cbc94fc1faa8
Removing intermediate container cbc94fc1faa8
---> 10eaff3f4774
Step 5/23 : COPY . .
---> 567cfa8797d9
Step 6/23 : RUN cat /app/.env | grep = | sort | sed -e 's|REACT_APP_\([a-zA-Z_]*\)=\(.*\)|REACT_APP_\1=NGINX_REPLACE_\1|' > /app/.env.local
---> Running in c28d9688f19e
Removing intermediate container c28d9688f19e
---> 65dd918e5162
Step 7/23 : RUN npm config set registry=https://npm.company.com
---> Running in f83b8b73b4e8
Removing intermediate container f83b8b73b4e8
---> d36d51e3cabb
Step 8/23 : RUN npm config set "strict-ssl" false
---> Running in 1f65e71e350b
Removing intermediate container 1f65e71e350b
---> d297ff074f44
Step 9/23 : RUN npm install
---> Running in 09a445362f73
....
added 1696 packages from 978 contributors in 42.078s
55 packages are looking for funding
run `npm fund` for details
Removing intermediate container 09a445362f73
---> c30a48bda924
Step 10/23 : RUN npm build
---> Running in be3f6a0a64ea
npm WARN build `npm build` called with no arguments. Did you mean to `npm run-script build`?
Removing intermediate container be3f6a0a64ea
---> 14a5a88aeeb1
Step 11/23 : RUN pwd
---> Running in 1b7603116844
/app
Removing intermediate container 1b7603116844
---> 0579c1f467a8
Step 12/23 : RUN ls -ls /app
---> Running in fd4479923203
total 1240
4 -rw-r--r--. 1 root root 1406 Feb 5 11:37 Dockerfile
4 -rw-r--r--. 1 root root 504 Feb 4 14:10 Jenkinsfile
4 -rw-r--r--. 1 root root 1057 Feb 4 14:09 README.md
0 drwxr-xr-x. 2 root root 83 Feb 4 14:09 gcp-deploy
4 -rw-r--r--. 1 root root 827 Feb 4 14:09 nginx.conf.template
60 drwxr-xr-x. 1150 root root 36864 Feb 5 11:37 node_modules
628 -rw-r--r--. 1 root root 639799 Feb 5 11:37 package-lock.json
4 -rw-r--r--. 1 root root 2821 Feb 4 14:09 package.json
12 -rw-r--r--. 1 root root 9106 Feb 4 14:18 pom.xml
0 drwxr-xr-x. 2 root root 64 Feb 4 14:09 public
4 -rw-r--r--. 1 root root 591 Feb 4 14:09 sonar-project.properties
4 drwxr-xr-x. 15 root root 4096 Feb 4 14:09 src
0 drwxr-xr-x. 2 root root 28 Feb 4 14:10 target
4 -rw-r--r--. 1 root root 486 Feb 4 14:09 tsconfig.json
508 -rw-r--r--. 1 root root 517508 Feb 4 14:09 yarn.lock
Removing intermediate container fd4479923203
---> e3329a5200cd
Step 13/23 : FROM nginx:1.15
---> 53f3fd8007f7
Step 14/23 : COPY nginx.conf.template /etc/nginx/conf.d/nginx.conf.template
---> c6ddc33a47e7
Step 15/23 : COPY gcp-deploy/htpasswd /etc/nginx/htpasswd
---> db3d7c99cea0
Step 16/23 : COPY --from=react-builder /app/.env.local /etc/nginx/conf.d/
---> 973f6987ab00
Step 17/23 : RUN NGINX_SUB_FILTER=$(cat /etc/nginx/conf.d/.env.local | grep '=' | sort | sed -e 's/REACT_APP_\([a-zA-Z_]*\)=\(.*\)/sub_filter\ \"NGINX_REPLACE_\1\" \"$\{\1\}\";/') && cat /etc/nginx/conf.d/nginx.conf.template | sed -e "s|LOCATION_SUB_FILTER|$(echo $NGINX_SUB_FILTER)|" | sed -u 's|}";\ |}";\n\t\t|g' > /etc/nginx/conf.d/default.conf.template
---> Running in a1eb5fdc7a02
Removing intermediate container a1eb5fdc7a02
---> ee3041f221df
Step 18/23 : COPY --from=react-builder /app/build /var/www
COPY failed: stat /mnt/data/docker/overlay2/7a85db99ff921b8588b566624634cc64c864f0e7db46c6540d09517662f7335a/merged/app/build: no such file or directory
and I can confirm that /mnt/data/docker/overlay2/7a85db99ff921b8588b566624634cc64c864f0e7db46c6540d09517662f7335a does not exists
The error is clear that the path /app/build does not exist in the react-builder layer.
What you can do to fix the problem is updating WORKDIR app to WORKDIR /app like this:
FROM node:10-slim AS react-builder
ARG REACT_APP_ENV
ENV REACT_APP_ENV $REACT_APP_ENV
WORKDIR /app
COPY . .
The problem is WORKDIR app is not the same as WORKDIR /app so when you are executing the RUN npm build, the generated build folder will not be saved in your expected /app/build and hence the error.
Apparently when running RUN npm build it does not happen what's expected - no compilation? - and thus there is no /app/build directory. However if running RUN npm run-script build the compilation works fine
Step 10/24 : RUN npm run-script build
---> Running in b61a1d5020d9
> opas-frontend#0.13.0 build /app
> react-scripts build
Creating an optimized production build...
Compiled successfully.
Therefore also COPY --from=react-builder /app/build /var/www works fine.

Docker: COPY can't find files in local directory where the `docker build` runs

in a directory with prefix /home/gitlab-runner/builds/, there is a example.jar file and a Dockerfile, in the Dockerfile, there are statements as below:
COPY example.jar /app
I run
docker build -t image_name ./
then I get the following error:
COPY failed: stat /var/lib/docker/tmp/docker-builder457658077/example.jar: no such file or directory
why can't COPY find the example.jar from within the directory with prefix /home/gitlab-runner/builds/? how does the strange /var/lib/docker.. path jumps in? how to deal with this? thanks!
[root#koala 53bdd1747e3590f90fcc84ef4963d4885711e25f]# pwd
/home/gitlab-runner/builds/pica/eureka/53bdd1747e3590f90fcc84ef4963d4885711e25f
[root#koala 53bdd1747e3590f90fcc84ef4963d4885711e25f]# ls -al
total 52068
drwxrwxr-x 5 gitlab-runner gitlab-runner 4096 Dec 11 15:23 .
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Dec 11 11:35 ..
-rw-rw-r-- 1 gitlab-runner gitlab-runner 17 Dec 11 11:35 APPLICATION_VERSION
-rw-rw-r-- 1 gitlab-runner gitlab-runner 644 Dec 11 11:35 docker-compose.yml
-rw-rw-r-- 1 gitlab-runner gitlab-runner 568 Dec 11 15:23 Dockerfile
drwxrwxr-x 8 gitlab-runner gitlab-runner 4096 Dec 11 11:35 .git
-rw-rw-r-- 1 gitlab-runner gitlab-runner 322 Dec 11 11:35 .gitignore
-rw-rw-r-- 1 gitlab-runner gitlab-runner 2438 Dec 11 11:35 .gitlab-ci.yml
-rw-rw-r-- 1 gitlab-runner gitlab-runner 53271183 Dec 11 11:35 example.jar
-rw-rw-r-- 1 gitlab-runner gitlab-runner 1043 Dec 11 11:35 pom.xml
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Dec 11 11:35 src
drwxrwxr-x 8 gitlab-runner gitlab-runner 4096 Dec 11 11:35 target
[ copying my answer from server fault, didn't realize this question was cross-posted ]
COPY example.jar /app
This command expects an example.jar in the root of your build context. The build context is the last argument to docker build, in this case ., or the current directory. From the ls -al output, you do not file this jar file in the directory and docker is telling you the COPY command cannot find the example.jar in the build context. If it is in one of the other sub directories, you'll need to update the COPY command with that location.
To debug issues with the build context, you can build and run the following Dockerfile:
FROM busybox
COPY . /build-context
WORKDIR /build-context
CMD find .
That will copy the entire build context into an image and list the contents out with a find command when you run the container.

File not found, but appears in directory

I am trying to run a docker build command to set up a container. When I run it, a file I need can't be found. I receive this error
/bin/sh: 1: ./downloadScript: not found
The command '/bin/sh -c ./downloadScript' returned a non-zero code: 127
I run ls -la before the script should run and it appears.
drwxr-xr-x 1 root root 4096 Apr 30 18:05 .
drwxr-xr-x 1 root root 4096 Apr 30 18:05 ..
-rwxr-xr-x 1 root root 714 Apr 29 18:20 downloadScript
drwxr-xr-x 4 root root 4096 Apr 30 18:05 gradleProject
I've tried a few things
chmod 777 the file and directory
cp the file, the chmod the new one (this new file can't be found either)
reset docker credentials
My current script:
...
WORKDIR /dir1
RUN ls -la # For manual verification
RUN ./downloadScript
...
Docker output
Sending build context to Docker daemon 120.3kB
Step 1/17 : FROM openjdk:8-slim as builder
---> e2581abdea18
Step 2/17 : WORKDIR /dir1
---> Using cache
---> 4ef289b22b45
Step 3/17 : ADD WCE_Docker/res /dir1
---> Using cache
---> 3d757ec6caba
Step 4/17 : ADD /GradleProject/ /dir1/gradleProject/
---> Using cache
---> 7e46cc77290d
Step 5/17 : WORKDIR /dir1
---> Using cache
---> fe7570221d31
Step 6/17 : RUN ls -la
---> Using cache
---> 019ef7d640da
Step 7/17 : RUN ./downloadScript
---> Running in 9870fd1e3af3
/bin/sh: 1: ./downloadScript: not found
The command '/bin/sh -c ./downloadScript' returned a non-zero code: 127
It runs correctly on Mac and Ubuntu, but not on Windows 10.
Update
I manually setup the container and attempted to run the script and received a slightly different error.
root#835516a24a7f:/dir# ./downloadScript
bash: ./downloadScript: /bin/bash^M: bad interpreter: No such file or directory
Which I've found is DOS based line ends. I am going to fix that then post results.
Update Results
Ran dos2unix downloadScript and it worked correctly. The problem was DOS line ends.

How to copy subfolders in docker

I have following folder structure:
nova-components
component1
dist
...
component2
dist
...
component3
dist
...
...
Is there any way to copy only dist folders in docker.
I am thinking about something like:
COPY --from=assets /nova-components/*/dist /var/www/nova-components/*/dist
The end goal is to include generated dist folders in the final image and keep the directory tree structure.
Currently multi-stage docker build do not respect .dockerignore, see this discussion, so you had to do it yourself, one way is to clean things in the first stage, like follows:
Dockerfile:
FROM ubuntu:16.04 AS assets
RUN mkdir -p /nova-components/component1/dist && \
mkdir -p /nova-components/component1/others && \
mkdir -p /nova-components/component2/dist && \
mkdir -p /nova-components/component2/others
RUN find /nova-components/*/* ! -name "dist" -maxdepth 0 | xargs rm -fr
FROM ubuntu:16.04
COPY --from=assets /nova-components /var/www/nova-components/
RUN ls -alh /var/www/nova-components
RUN ls -alh /var/www/nova-components/*
test:
# docker build --no-cache -t try:1 .
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM ubuntu:16.04 AS assets
---> b9e15a5d1e1a
Step 2/7 : RUN mkdir -p /nova-components/component1/dist && mkdir -p /nova-components/component1/others && mkdir -p /nova-components/component2/dist && mkdir -p /nova-components/component2/others
---> Running in d4c9c422d53a
Removing intermediate container d4c9c422d53a
---> d316032dd59d
Step 3/7 : RUN find /nova-components/*/* ! -name "dist" -maxdepth 0 | xargs rm -fr
---> Running in b6168b027f4c
Removing intermediate container b6168b027f4c
---> 9deb57cb5153
Step 4/7 : FROM ubuntu:16.04
---> b9e15a5d1e1a
Step 5/7 : COPY --from=assets /nova-components /var/www/nova-components/
---> 49301f701db2
Step 6/7 : RUN ls -alh /var/www/nova-components
---> Running in 9ed0cafff2fb
total 16K
drwxr-xr-x 4 root root 4.0K Nov 6 02:13 .
drwxr-xr-x 3 root root 4.0K Nov 6 02:13 ..
drwxr-xr-x 3 root root 4.0K Nov 6 02:13 component1
drwxr-xr-x 3 root root 4.0K Nov 6 02:13 component2
Removing intermediate container 9ed0cafff2fb
---> f1ee82cff972
Step 7/7 : RUN ls -alh /var/www/nova-components/*
---> Running in 23a27e5ce853
/var/www/nova-components/component1:
total 12K
drwxr-xr-x 3 root root 4.0K Nov 6 02:13 .
drwxr-xr-x 4 root root 4.0K Nov 6 02:13 ..
drwxr-xr-x 2 root root 4.0K Nov 6 02:13 dist
/var/www/nova-components/component2:
total 12K
drwxr-xr-x 3 root root 4.0K Nov 6 02:13 .
drwxr-xr-x 4 root root 4.0K Nov 6 02:13 ..
drwxr-xr-x 2 root root 4.0K Nov 6 02:13 dist
Removing intermediate container 23a27e5ce853
---> b9d5ab8f5157
Successfully built b9d5ab8f5157
Successfully tagged try:1
With the clean in first stage using RUN find /nova-components/*/* ! -name "dist" -maxdepth 0 | xargs rm -fr, then you can make it, let's wait for possible official feature support.
Add a .dockerignore file with your Dockerfile
nova-components/*/*
!nova-components/*/dist
And copy like
COPY nova-components/ /var/www/nova-components
EDIT
So with multi stage build, this is currently not working. A new solution is to run, on the last stage,
rsync -avz --include='dist/' nova-components/ nova-components-dist/.
then at the final stage,
COPY --from=assets /nova-components-dist/ /var/www/nova-components

Resources