I'm trying to build a stack with one docker-compose that should contain another containers inside. This is to run a development environment with all my projects inside.
So the problem is the volume with application source isn't appearing on built image.
MacOS Sierra
Docker version 17.03.0-ce, build 60ccb22
Boot2Docker-cli version: v1.8.0
my directory tree
/dockers <======= one directory with all docker files for each project
docker-compose.yml <======= The main image
/project1 <======= dockerfile for each project
Dockerfile
/project2
Dockerfile
/project3
Dockerfile
/project1 <======= project1 source folder
test.txt
/project2
/project3
my docker-compose.yml
project1:
build: ./project1
volumes:
- ../project1/:/src
my dockerfile for project1
FROM python:2.7
RUN mkdir -p /src
WORKDIR /src
RUN echo "---------------------"
RUN ls -la
RUN echo "---------------------"
So I try to build the docker-compose file
$ sudo docker-compose build --no-cache
And then it shows an empty folder when I expect test.txt file
Building express
ERROR: Cannot locate specified Dockerfile: Dockerfile
➜ docker git:(master) ✗ sudo docker-compose build --no-cache
Building project1
Step 1/7 : FROM python:2.7
---> ca388cdb5ac1
Step 2/7 : RUN mkdir -p /src
---> Running in 393a462f7a44
---> 4fbeb32d88b3
Removing intermediate container 393a462f7a44
Step 3/7 : WORKDIR /src
---> 03ce193577ab
Removing intermediate container b1cd746b699a
Step 4/7 : RUN echo "--------------------------"
---> Running in 82df8a512c90
----------------------------
---> 6dea58ba5051
Removing intermediate container 82df8a512c90
Step 5/7 : RUN ls -la
---> Running in 905417d0cd19
total 8
drwxr-xr-x 2 root root 4096 Mar 23 17:12 . <====== EMPTY :(
drwxr-xr-x 1 root root 4096 Mar 23 17:12 .. <====== EMPTY :(
---> 53764caffb1a
Removing intermediate container 905417d0cd19
Step 6/7 : RUN echo "-----------------------------"
---> Running in 110e765d102a
----------------------------
---> b752230fd6dc
Removing intermediate container 110e765d102a
Step 7/7 : EXPOSE 3000
---> Running in 1cfe2e80d282
---> 5e3e740d5a9a
Removing intermediate container 1cfe2e80d282
Successfully built 5e3e740d5a9a
Volumes are runtime configurations in Docker. Because they are configurable, if you were to reference volumes during the build phase you would essentially be creating a potentially uncheckable broken dependency.
I'm sure there is a more technical reason - but it really shouldn't be done. Move all that stuff to the runtime setup command and you should be OK.
Related
I have a working build that I am trying to dockerise. This is my dockerfile, I added the "pwd" and "ls -l" to see if the build is copied correctly and it is. However when I try to run "docker run " I get the error "No such file or directory. Please let me know what I might be doing wrong. Appreciate your help.
Dockerfile
FROM <base image>
WORKDIR /app
RUN echo 'List all files'
RUN pwd
RUN ls -l
COPY src/mysolution-linux-amd64 /app/
RUN ls -l
ENTRYPOINT ["/app/mysolution-linux-amd64"]
I have tried ENTRYPOINT with both "./mysolution-linux-amd64" and "/app/mysolution-linux-amd64" but both fail when I run.
Output during Docker build
Sending build context to Docker daemon 1.014GB
Step 1/8 : FROM <base image>
---> 3ed27f7c19ce
Step 2/8 : WORKDIR /app
---> Running in 1b273ccccd22
Removing intermediate container 1b273ccccd22
---> 92560bbb67eb
Step 3/8 : RUN echo 'List all files'
---> Running in faddc1b6adfd
List all files
Removing intermediate container faddc1b6adfd
---> b7b2f657012e
Step 4/8 : RUN pwd
---> Running in 8354a5a476ac
/app
Removing intermediate container 8354a5a476ac
---> 204a625b730b
Step 5/8 : RUN ls -l
---> Running in 0d45cf1339d9
total 0
Removing intermediate container 0d45cf1339d9
---> 6df6451aef44
Step 6/8 : COPY src/mysolution-linux-amd64 /app/
---> 44ac2f066340
Step 7/8 : RUN ls -l
---> Running in d17ec6b0c7af
total 11460
-rwxrwxr-x 1 root root 11734780 Nov 26 04:25 mysolution-linux-amd64
Removing intermediate container d17ec6b0c7af
---> 56a879ef9440
Step 8/8 : ENTRYPOINT ["/app/mysolution-linux-amd64"]
---> Running in 33bea73f14dc
Removing intermediate container 33bea73f14dc
---> ef794fe310bc
Successfully built ef794fe310bc
Successfully tagged newtech/mysolution:latest
I have a dockerfile that's copying a binary from a previous step. The entrypoint/command does not work however. I can see the file exists though.
FROM rust as rust-builder
WORKDIR /app
COPY Cargo.lock Cargo.lock
COPY Cargo.toml Cargo.toml
COPY src/ src
RUN cargo build --release
FROM alpine:latest
COPY --from=rust-builder /app/target/release/kube-logs-generator /app/kube-logs-generator
RUN ls -la /app
CMD ["/app/kube-logs-generator"]
This is the output:
Sending build context to Docker daemon 2.159GB
Step 1/10 : FROM rust as rust-builder
---> dd3f19acb681
Step 2/10 : WORKDIR /app
---> Using cache
---> 5e0281f74323
Step 3/10 : COPY Cargo.lock Cargo.lock
---> Using cache
---> 060b7d6f8349
Step 4/10 : COPY Cargo.toml Cargo.toml
---> Using cache
---> 56b90bed67d5
Step 5/10 : COPY src/ src
---> Using cache
---> cdfa52607837
Step 6/10 : RUN cargo build --release
---> Using cache
---> 2f13c20bbebe
Step 7/10 : FROM alpine:latest
---> 9c6f07244728
Step 8/10 : COPY --from=rust-builder /app/target/release/kube-logs-generator /app/kube-logs-generator
---> Using cache
---> b2158ebfac6f
Step 9/10 : RUN ls -la /app
---> Running in cda38e0f4ff0
total 8056
drwxr-xr-x 1 root root 38 Oct 28 14:07 .
drwxr-xr-x 1 root root 140 Oct 28 14:14 ..
-rwxr-xr-x 1 root root 8245984 Oct 28 14:03 kube-logs-generator
Removing intermediate container cda38e0f4ff0
---> 6bf0803d98d6
Step 10/10 : CMD ["/app/kube-logs-generator"]
---> Running in 99ba34dd6afa
Removing intermediate container 99ba34dd6afa
---> c616fa4a1d55
Successfully built c616fa4a1d55
Successfully tagged bkauffman7/kube-logs-generator:latest
Then running
docker run bkauffman7/kube-logs-generator
exec /app/kube-logs-generator: no such file or directory
I'm not sure why it thinks the file isn't there.
Your final application stage is using an Alpine-based image, but your builder isn't using an Alpine base. There are some incompatibilities in the base C library stack that can cause "no such file or directory" errors, even when the file plainly exists.
The easiest way to work around this is to use a Debian- or Ubuntu-based image for the final stage. This is somewhat larger, but it includes the standard GNU libc.
FROM rust:bullseye as rust-builder
...
FROM debian:bullseye
COPY --from=rust-builder /app/target/release/kube-logs-generator /app/kube-logs-generator
The Docker Hub rust image includes an Alpine variant, and it also may work to use an Alpine-based image as the builder
FROM rust:alpine3.16 AS rust-builder
...
FROM alpine:3.16
In both cases, I've made sure that both the Linux distribution and their major versions match between the builder and runtime stages.
I have what I believe is a pretty simple setup.
I build a binary file outside of docker and then try to add it using this Dockerfile
FROM alpine
COPY apps/dist/apps /bin/
RUN chmod +x /bin/apps
RUN ls -al /bin | grep apps
CMD /bin/apps
And I think this should work.
The binary on its own seems to work on my host machine and I don't understand why it wouldn't on the docker image.
Anyways, the output I get is this:
docker build -t apps -f app.Dockerfile . && docker run apps
Sending build context to Docker daemon 287.5MB
Step 1/5 : alpine
---> d05cf6536f67
Step 2/5 : COPY apps/dist/apps /bin/
---> Using cache
---> c54d6d57154e
Step 3/5 : RUN chmod +x /bin/apps
---> Using cache
---> aa7e6adb0981
Step 4/5 : RUN ls -al /bin | grep apps
---> Running in 868c5e235d68
-rwxr-xr-x 1 root root 68395166 Dec 20 13:35 apps
Removing intermediate container 868c5e235d68
---> f052c06269b0
Step 5/5 : CMD /bin/apps
---> Running in 056fd02733e1
Removing intermediate container 056fd02733e1
---> 331600154cbe
Successfully built 331600154cbe
Successfully tagged apps:latest
/bin/sh: /bin/apps: not found
does this make sense, and am I just missing something obvious?
Your binary likely has dynamic links to libraries that don't exist inside the image filesystem. You can check those dynamic links with the ldd apps/dist/apps command.
I try to add to docker file, i simplified to very short one, that show the issue.
Simple Dockerfile - ddtest
FROM maven AS testdocker
COPY . /server
WORKDIR /server/admin
RUN mkdir target; echo "hello world" > ./target/test.txt
RUN pwd
RUN ls -la ./target/
COPY ./target/test.txt /test.txt
CMD ["/usr/bin/java", "-jar", "/server.jar"]
Building with command docker build . -f ddtest
Execution log:
docker build . -f ddtest
Sending build context to Docker daemon 245.8kB
Step 1/8 : FROM maven AS testdocker
---> e85864b4079a
Step 2/8 : COPY . /server
---> e6c9c6d55be1
Step 3/8 : WORKDIR /server/admin
---> Running in d30bab5d6b6b
Removing intermediate container d30bab5d6b6b
---> 7409cbc70fac
Step 4/8 : RUN mkdir target; echo "hello world" > ./target/test.txt
---> Running in ad507dfc604b
Removing intermediate container ad507dfc604b
---> 0d69df30d041
Step 5/8 : RUN pwd
---> Running in 72becb9ae3ba
/server/admin
Removing intermediate container 72becb9ae3ba
---> 7bde9ccae4c6
Step 6/8 : RUN ls -la ./target/
---> Running in ceb5c222f3c0
total 12
drwxr-xr-x 2 root root 4096 Aug 9 05:50 .
drwxr-xr-x 1 root root 4096 Aug 9 05:50 ..
-rw-r--r-- 1 root root 12 Aug 9 05:50 test.txt
Removing intermediate container ceb5c222f3c0
---> 3b4dbcb794ad
Step 7/8 : COPY ./target/test.txt /test.txt
COPY failed: stat /var/lib/docker/tmp/docker-builder015566471/target/test.txt: no such file or directory
copy to destination docker test.txt failed, why?
That ls command you invoked is running inside an intermediate container, which is not your host. So according to your WORKDIR you're listing files inside /server/admin/target of your container.
As the output of ls shows, this test.txt file is already inside your image
COPY src dst command copies a file/directory from your host to your image, and it seems that you don't have any file named test.txt inside ./target directory of your host.
I could not find that spoecifics via web search for site:stackoverflow.com dockerfile RUN fs changes not persisted.
I made Dockerfile and wanted to make some changes in image via RUN. Firstly I wanted to change file attributes, but changes were not there in started container. I've found this post taking about docker bug for chmod and workarounds: https://serverfault.com/questions/772227/chmod-not-working-correctly-in-docker. However, now I just try to create a file via RUN touch /path/file in Dockerfile and already on next command (next layer for docker as I understood the docs) changes (that file) are gone. As far as I see it is strange, otherwise how apt install works then because it will make changes to file system in the image... Why results of that particular RUN do not persist? Docs below tell opposite (or I misunderstand meaning of commit here):
https://docs.docker.com/engine/reference/builder/ :
The RUN instruction will execute any commands in a new layer on top of
the current image and commit the results. The resulting committed
image will be used for the next step in the Dockerfile.
Here is output of docker build --no-cache -t yandex:user5 yandex2/ (see steps 6 and 7; dockeruser is created, btw):
Step 1/8 : FROM artifactory.dev.foo.org:5000/yandex-tank:jmeter
---> b8286a9220ca
Step 2/8 : LABEL maintainer="foo#foo.org"
---> Running in 7cfde9a90bf2
Removing intermediate container 7cfde9a90bf2
---> b5acd9d55f9c
Step 3/8 : WORKDIR /var/loadteest
---> Running in 47e9adc401bb
Removing intermediate container 47e9adc401bb
---> 103f158e0be3
Step 4/8 : USER root
---> Running in 9923d71f7b08
Removing intermediate container 9923d71f7b08
---> bb3aa8672bc6
Step 5/8 : RUN groupadd -r dockeruser && useradd -r -g dockeruser -d /var/loadtest -s /sbin/nologin -c "Docker image user" dockeruser
---> Running in 48c89f33d750
Removing intermediate container 48c89f33d750
---> 5000afa7698d
Step 6/8 : RUN touch /var/loadtest/chmodtest.txt
---> Running in 00b2d1ccad75
Removing intermediate container 00b2d1ccad75
---> c35808f13344
Step 7/8 : RUN ls -al /var/loadtest
---> Running in cc08d129eeb3
total 8
drwxr-xr-x 2 root root 4096 Nov 20 12:19 .
drwxr-xr-x 1 root root 4096 Nov 29 06:15 ..
Removing intermediate container cc08d129eeb3
---> 842678ca5d49
Step 8/8 : ENTRYPOINT /bin/bash
---> Running in 20b92a97c8a8
Removing intermediate container 20b92a97c8a8
---> fefd0d665677
Successfully built fefd0d665677
Successfully tagged yandex:user5
The Problem is using the volumes in the base image:
Changing the volume from within the Dockerfile: If any build steps
change the data within the volume after it has been declared, those
changes will be discarded.
see This
Workaround is to use COPY docker doc
COPY path/to/local/file /var/loadtest/chmodtest.txt