First I would like to thank you for helping me.
Please be kind as it is my first post here.
I am trying to build a docker container but the COPY step seems not to work properly.
Here is my DockerFile
FROM maven:3.3-jdk-8
RUN apt-get update
RUN apt-get -y install vim tree unzip
WORKDIR /usr/src/app
COPY . /usr/src/app/
Here is my directory tree
├── Dockerfile
├── events
│ ├── backend drwxr-xr-x
│ │ ├── stuff drwxr-xr-x
│ ├── frontend drwxr-xr-x
│ │ ├── stuff drwxr-xr-x
I am part of the docker group.
The weird thing happening is that stuff from backend is copied but not stuff from frontend.
I remember using COPY this way and it worked.
I might have missed a behavior of COPY.
Thanks for helping me.
Cheers.
Did you check if you have .dockerignore in your project?
Docker COPY will silently ignore all patterns mentioned in .dockerignore.
Related
I try to build an image with Docker (10.20.13 on RH 7.9). But some of my resources are outside the Dockerfile directory. Below is my tree :
/dir1
├── dir2
│ ├── dir3
│ │ ├── dir4
│ │ │ ├── boost
│ │ │ │ └── lib
│ │ │ │ ├── linuxV2_6_18A32
│ │ │ │ │ ├── libboost_atomic-mt.a
│ │ │ │ │ ├── ....
/home/myproject/myDockerfile
I want to add in my image the resources that are in /dir1/dir2/dir3/dir4/boost which are not necessary my resources (but I do have at least read access).
My first try was to build an image from /home/myproject/myDockerfile with the following command :
/home/myproject/myDockerfile/docker build -t myimage:1.0 .
But it failed with, saying this:
ADD failed: file not found in build context or excluded by .dockerignore: stat dir1: file does not exist
Okay, the dir1 is not in the context. So I tried to make a link to dir1 in the Dockerfile directory, and again the same command, but different issue :
ADD failed: forbidden path outside the build context: netdata ()
Third try, I launch the command from the root directory (to get all the context as I understand), with the following command:
docker build -t myimage:1.0 -f /home/myproject/myDockerfile
This time I get this response:
error checking context: 'no permission to read from '/boot/System.map-3.10.0-1160.31.1.el7.x86_64''
So I image to add the last directory to my .dockerignore, but it should be in the context (root directory) which is impossible.
So is there a solution to my problem apart copying in project directory all the resources I need?
You have to copy all of the resources you need into the project directory. You can't really build a Docker image containing files from completely unrelated parts of the filesystem (you can't include a library from /usr/lib from a Dockerfile in your home directory).
Since what you're trying to include is a static library, you have a couple of options to get it. The easiest is to just install it via your base image's package manager:
FROM debian:stable
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
libboost-atomic1.74-dev
A second harder option is to use a multi-stage build or a similar technique to build the library from source.
Since the specific file you're referencing is a static library, another option could be to build the binary on the host system and COPY it into the image unmodified. This requires you to be on a native-Linux host with a similar base Linux distribution.
FROM debian:stable
COPY ./myapp /usr/local/bin
CMD ["myapp"]
gcc -o myapp ... -lboost_atomic-mt
docker build -t myapp .
If all else fails then you can make a copy locally. You might write a script to do this; this is also a place where Make works well since it's largely dealing with concrete files.
#!/bin/sh
mkdir ./docker
cp -a Dockerfile .dockerignore src ./docker
cp /dir1/dir2/.../libboost_atomic-mt.a ./docker
docker build -t myapp ./docker
I have this structure:
project
└── pipable
└── common
└── common
└── setup.py # setup.py for package "common"
└── utils
└── utils
└── setup.py # setup.py for package "utils"
└── platform
└── service_one
└── Dockerfile
└── requirements.txt
└── service_two
└── Dockerfile
└── requirements.txt
docker-compose.yml
The package common and utils get pip installed in service_one and service_two, this is done from a remote pypi repository.
I want to make these packages editable, if you change in common then the change should be reflected in the services. I've looked at editable from git but it seems like it would create a copy within the services meaning that editing common outside the services won't update.
Solved it by adding the pipable folder as a volume for each service, after which I run pip install -e location_of_mount_setup_file.
I have a project including multiple Dockerfiles.
The tree is like,
.
├── app1
│ ├── Dockerfile
│ ├── app.py
│ └── huge_modules/
├── app2
│ ├── Dockerfile
│ ├── app.py
│ └── huge_modules/
├── common
│ └── my_lib.py
└── deploy.sh
To build my application, common/ is necessary and we have to COPY it inside Dockerfile.
However, Dockerfile cannot afford to COPY files from its parent directory.
To be precise, it is possible if we run docker build with -f option in the project root.
But I would not like to do this because the build context will be unnecessarily large.
When building app1, I don't like to include app2/huge_modules/ in the build context (the same as when building app2).
So, I prepare a build script in each app directory.
Like this.
cd $(dirname $0)
cp ../common/* ./
docker build -t app1 .
But this solution seems ugly to me.
Is there a good solution for this case?
Build a base image containing your common library, and then build your two app images on top of that. You'll probably end up restructuring things slightly to provide a Dockerfile for your common files:
.
├── app1
│ ├── Dockerfile
│ ├── app.py
│ └── huge_modules/
├── app2
│ ├── Dockerfile
│ ├── app.py
│ └── huge_modules/
├── base
| ├── Dockerfile
| └── common
│ └── my_lib.py
└── deploy.sh
You start by building a base image:
docker build -t mybaseimage base/
And then your Dockerfile for app1 and app2 would start with:
FROM mybaseimage
One possible solution is to start the build process from the top directory, with the -f flag you mentioned, dynamically generating the .dockerignore file.
That is, lets say that you currently build app1. Then you would first create in the top directory a .dockerignore file with the content: app2, then run the build process. After finishing the build, remove the .dockerignore file.
Now you want to build app2? No problem! Similarly generate first dynamically a .dockerignore file with the content app1, build and remove the file. Voila!
So I have a tree that looks like this:
.
├── README.md
├── dataloader
│ ├── Dockerfile
...
│ ├── log.py
│ ├── logo.py
│ ├── processors
...
│ └── tests
├── datastore
│ ├── datastore.py
and the Dockerfile inside the dataloader application looks like this:
FROM python:3.7
WORKDIR /var/dsys-2uid-dataloader
COPY assertions/ ./assertions/
COPY events/ ./events/
COPY processors/ ./processors/
COPY requirements.txt ./
<*>COPY datastore/ ./datastore/
COPY *.py ./
RUN pip3 install -r requirements.txt
ENTRYPOINT ["python", "dataloader.py"]
the line with the asterisk doesn't work since the datastore folder is in the parent of the Dockerfile. What can be done? I need this Dockerfile to be correct because it's going to be used as the image in the kubernetes deployment.
You can't access a file outside of your build context, but you can "trick" docker to be in a different build context.
Just run docker build -t foo -f dataloader/Dockerfile . from the root directory (where you have the README and the dirs)
$ tree
.
├── bar
│ └── wii
└── foo
└── Dockerfile
2 directories, 2 files
$ cat foo/Dockerfile
FROM ubuntu
COPY bar/wii .
$ docker build -t test -f foo/Dockerfile .
Sending build context to Docker daemon 3.584kB
Step 1/2 : FROM ubuntu
---> cf0f3ca922e0
Step 2/2 : COPY bar/wii .
---> c3ff3f652b4d
Successfully built c3ff3f652b4d
Successfully tagged test:latest
Let's suppose I have an Elixir project tree organized like this:
$ tree .
.
├── apps
│ ├── a
│ │ ├── ...
│ │ └── mix.exs
│ ├── b
│ │ ├── ...
│ │ └── mix.exs
│ └── c
│ ├── ...
│ └── mix.exs
├── mix.exs
└── mix.lock
(unrelated files dropped)
The Dockerfile I wrote for this project looks roughly like this:
FROM bitwalker/alpine-elixir-phoenix:latest
ENV MIX_ENV=prod
# Elixir deps
# FIXME: do not hardcode apps/ subdirectories, use wildcards somehow
# FIXME: copy everything in one step somehow
ADD apps/a/mix.exs apps/a/
ADD apps/b/mix.exs apps/b/
ADD apps/c/mix.exs apps/c/
ADD mix.exs mix.lock ./
RUN mix local.hex --force && mix deps.get && mix deps.compile
<skipped>
ADD . .
RUN mix compile && mix phx.digest
CMD [ "mix", "phx.server" ]
Is there a way to ADD or COPY all the apps/*/mix.exs in a single step, preserving their directory structure (so that each apps/foo/mix.exs ends up in $WORKDIR/apps/foo/mix.exs)?
To the best of my knowledge, there is no obvious way to do what I want. E. g. saying something like ADD apps/*/mix.exs mix.exs mix.lock ./ just copies everything into the WORKDIR (and the files overwrite each other).
Maybe there is some non-obvious way?
A workaround I could think of is to add the following in the .dockerignore file
apps/**
!apps/*/mix.exs
and in the dockerfile,
COPY apps/ /apps
COPY mix.exs mix.lock /
or even
COPY . /
NOTE
If you don't have to stick to pure dockerfile solution, wrap your docker build in a shell script or makefile and use cp is also a good way to do it.