Dockerfile and build context file tree, the core-site.xml file's relative path IS conf/etc/hadoop
.
├── Dockerfile
└── conf
├── etc
│ └── hadoop
│ └── core-site.xml
└── xxx.conf
Dockerfile(very simple) as below
FROM alpine:latest
RUN mkdir -p /data
WORKDIR /data
COPY conf/* /data/
After docker build -t any/any:any ., the COPY layer's file list as below. Intermediate path etc/ of file core-site.xml lost. Where the etc/ gone ?
data/
data/xxx.conf
data/hadoop/
data/hadoop/.wh..wh..opq
data/hadoop/core-site.xml
Remove the * so you just have COPY conf /data
The COPY command treats the copying of files and directories a bit differently. Your COPY statement expands to
COPY conf/etc /data/
COPY conf/xxx.conf /data/
The first statement actually copies the contents of conf/etc into the /data directory on the container.
Related
I'm trying to create a Dockerfile that copies all package.json files into the image but keeps the folder structure.
This what I have now:
FROM node:15.9.0-alpine as base
WORKDIR /app/
COPY ./**/package.json ./
CMD ls -laR /app
Running with: sudo docker run --rm -it $(sudo docker build -q .)
But it only copies 1 package.json and puts it in the base dir (/app)
Here is the directory I'm testings on:
├── Dockerfile
├── t1
│ └── package.json
└── t2
└── ttt
├── b.txt
└── package.json
And i would like it to look like this inside the container:
├── Dockerfile
├── t1
│ └── package.json
└── t2
└── ttt
└── package.json
The Dockerfile COPY directive is documented as using the Go filepath.Match function for glob expansion. That only supports the basic glob characters *, ?, [a-z], but not extensions like ** that some shells support.
Since COPY only takes a filename glob as input and it likes to flatten the file structure, I don't think there's a way to do the sort of selective copy you're describing in a single command.
Instead you need to list out the individual files you want to copy. COPY will create directories as needed, but that means you need to repeat paths on both sides of COPY.
COPY t1/package*.json t1/
COPY t2/ttt/package*.json t2/ttt/
I can imagine some hacky approaches using multi-stage builds; have an initial stage that copies in the entire source tree but then deletes all of the files except package*.json, then copies that into the actual build stage. I'd contemplate splitting my repository into smaller modules with separate Dockerfiles per module first.
Have a Dockerfile to build releases for an Elixir/Phoenix application...The tree directory structure is as follows, where the Dockerfile (which has a dependency on this other Dockerfile) is in the "infra" subfolder and needs access to all the files one level above "infra".
.
├── README.md
├── assets
│ ├── css
│ ├── js
│ ├── node_modules
│ ├── package-lock.json
│ ├── package.json
├── lib
├── infra
│ ├── Dockerfile
│ ├── config.yaml
│ ├── deployment.yaml
The Dockerfile looks like:
# https://github.com/bitwalker/alpine-elixir
FROM bitwalker/alpine-elixir:latest
# Set exposed ports
EXPOSE 4000
ENV PORT=4000
ENV MIX_ENV=prod
ENV APP_HOME /app
ENV APP_VERSION=0.0.1
COPY ./ ${HOME}
WORKDIR ${HOME}
RUN mix deps.get
RUN mix compile
RUN MIX_ENV=${MIX_ENV} mix distillery.release
RUN echo $HOME
COPY ${HOME}/_build/${MIX_ENV}/rel/my_app/releases/${APP_VERSION}/my_app.tar.gz .
RUN tar -xzvf my_app.tar.gz
USER default
CMD ./bin/my_app foreground
The command "mix distillery.release" is what builds the my_app.tar.gz file in the path indicated by the COPY command.
I invoke the docker build as follows in the top-level directory (the parent directory of "infra"):
docker build -t my_app:local -f infra/Dockerfile .
I basically then get an error with COPY:
Step 13/16 : COPY ${HOME}/_build/${MIX_ENV}/rel/my_app/releases/${APP_VERSION}/my_app.tar.gz .
COPY failed: stat /var/lib/docker/tmp/docker-builder246562111/opt/app/_build/prod/rel/my_app/releases/0.0.1/my_app.tar.gz: no such file or directory
I understand that the COPY command depends on the "build context" but I thought that by issuing the "docker build" in the parent directory of infra meant I had the appropriate context set for the COPY, but clearly that doesn't seem to be the case. Is there a way to have a Dockerfile one level below the parent directory that contains all the files needed to build an Elixir/Phoenix "release" (the my_app.tar.gz and associated files created via the command mix distillery.release)? What bits am I missing?
So I have a tree that looks like this:
.
├── README.md
├── dataloader
│ ├── Dockerfile
...
│ ├── log.py
│ ├── logo.py
│ ├── processors
...
│ └── tests
├── datastore
│ ├── datastore.py
and the Dockerfile inside the dataloader application looks like this:
FROM python:3.7
WORKDIR /var/dsys-2uid-dataloader
COPY assertions/ ./assertions/
COPY events/ ./events/
COPY processors/ ./processors/
COPY requirements.txt ./
<*>COPY datastore/ ./datastore/
COPY *.py ./
RUN pip3 install -r requirements.txt
ENTRYPOINT ["python", "dataloader.py"]
the line with the asterisk doesn't work since the datastore folder is in the parent of the Dockerfile. What can be done? I need this Dockerfile to be correct because it's going to be used as the image in the kubernetes deployment.
You can't access a file outside of your build context, but you can "trick" docker to be in a different build context.
Just run docker build -t foo -f dataloader/Dockerfile . from the root directory (where you have the README and the dirs)
$ tree
.
├── bar
│ └── wii
└── foo
└── Dockerfile
2 directories, 2 files
$ cat foo/Dockerfile
FROM ubuntu
COPY bar/wii .
$ docker build -t test -f foo/Dockerfile .
Sending build context to Docker daemon 3.584kB
Step 1/2 : FROM ubuntu
---> cf0f3ca922e0
Step 2/2 : COPY bar/wii .
---> c3ff3f652b4d
Successfully built c3ff3f652b4d
Successfully tagged test:latest
I have a repository with a directory structure like this
.
├── Dockerfile
├── README.md
├── frontend/
├── backend/
├── docs/
├── examples/
└── build/
The dockerfile is a simple ADD with no entrypoint:
FROM python:3.6-slim
WORKDIR /app
# Copy and install requirements.txt first for caching
ADD . /app
RUN pip install --no-cache-dir --trusted-host pypi.python.org -r backend/requirements.txt
EXPOSE 8200
WORKDIR /app/backend
My issue is that after docker build -t myimage ., the build folder is missing from the image.
I just ran an ls when verifying the image contents with docker run -it myimage /bin/bash, and the build folder is missing!
.
├── frontend/
├── backend/
├── docs/
├── examples/
Does anyone know why? How can I add modify my Dockerfile to add this folder into my image? All resources online say that ADD . <dest> should duplicate my current directory tree inside the image, but the build folder is missing...
Missed that there's a .dockerignore file in the repo that contains this folder. Whooooops, thank you #David Maze.
My docker file has following entry
ENV SCPATH /etc/supervisor/conf.d
RUN apt-get -y update
# The daemons
RUN apt-get -y install supervisor
RUN mkdir -p /var/log/supervisor
# Supervisor Configuration
ADD ./supervisord/conf.d/* $SCPATH/
The directory structure looks like this
├── .dockerignore
├── .gitignore
├── Dockerfile
├── Makefile
├── README.md
├── Vagrantfile
├── index.js
├── package.json
└── supervisord
└── conf.d
├── node.conf
└── supervisord.conf
As per my understanding this should work fine as
ADD ./supervisord/conf.d/* $SCPATH/
Points to a relative path in terms of dockerfile build context.
Still it fails with
./supervisord/conf.d : no such file or directory exists.
I am new to docker so might be a very basic thing I am missing. Really appreciate help
What are your .dockerignore file contents? Are you sure you did not accidentally exclude something below your supervisord directory that the docker daemon needs to build your image?
And: in which folder are you executing the docker build command? Make sure you execute it within the folder that holds the Dockerfile so that the relative paths match.
Update: I tried to reproduce your problem. What I did from within a temp folder:
mkdir -p a/b/c
echo "test" > a/b/c/test.txt
cat <<EOF > Dockerfile
FROM debian
ENV MYPATH /newdir
RUN mkdir $MYPATH
ADD ./a/b/c/* $MYPATH/
CMD cat $MYPATH/test.txt
EOF
docker build -t test .
docker run --rm -it test
That prints test as expected. The important part works: the ADD ./a/b/c* $MYPATH. The file at the end is found as its content test is displayed during runtime.
When I now change the path ./a/b/c/* to something else, I get the no such file or directory exists error. When I leave the path as is and invoke docker build from a different folder than the temp folder where I placed the files the error is shown, too.