I could not find that spoecifics via web search for site:stackoverflow.com dockerfile RUN fs changes not persisted.
I made Dockerfile and wanted to make some changes in image via RUN. Firstly I wanted to change file attributes, but changes were not there in started container. I've found this post taking about docker bug for chmod and workarounds: https://serverfault.com/questions/772227/chmod-not-working-correctly-in-docker. However, now I just try to create a file via RUN touch /path/file in Dockerfile and already on next command (next layer for docker as I understood the docs) changes (that file) are gone. As far as I see it is strange, otherwise how apt install works then because it will make changes to file system in the image... Why results of that particular RUN do not persist? Docs below tell opposite (or I misunderstand meaning of commit here):
https://docs.docker.com/engine/reference/builder/ :
The RUN instruction will execute any commands in a new layer on top of
the current image and commit the results. The resulting committed
image will be used for the next step in the Dockerfile.
Here is output of docker build --no-cache -t yandex:user5 yandex2/ (see steps 6 and 7; dockeruser is created, btw):
Step 1/8 : FROM artifactory.dev.foo.org:5000/yandex-tank:jmeter
---> b8286a9220ca
Step 2/8 : LABEL maintainer="foo#foo.org"
---> Running in 7cfde9a90bf2
Removing intermediate container 7cfde9a90bf2
---> b5acd9d55f9c
Step 3/8 : WORKDIR /var/loadteest
---> Running in 47e9adc401bb
Removing intermediate container 47e9adc401bb
---> 103f158e0be3
Step 4/8 : USER root
---> Running in 9923d71f7b08
Removing intermediate container 9923d71f7b08
---> bb3aa8672bc6
Step 5/8 : RUN groupadd -r dockeruser && useradd -r -g dockeruser -d /var/loadtest -s /sbin/nologin -c "Docker image user" dockeruser
---> Running in 48c89f33d750
Removing intermediate container 48c89f33d750
---> 5000afa7698d
Step 6/8 : RUN touch /var/loadtest/chmodtest.txt
---> Running in 00b2d1ccad75
Removing intermediate container 00b2d1ccad75
---> c35808f13344
Step 7/8 : RUN ls -al /var/loadtest
---> Running in cc08d129eeb3
total 8
drwxr-xr-x 2 root root 4096 Nov 20 12:19 .
drwxr-xr-x 1 root root 4096 Nov 29 06:15 ..
Removing intermediate container cc08d129eeb3
---> 842678ca5d49
Step 8/8 : ENTRYPOINT /bin/bash
---> Running in 20b92a97c8a8
Removing intermediate container 20b92a97c8a8
---> fefd0d665677
Successfully built fefd0d665677
Successfully tagged yandex:user5
The Problem is using the volumes in the base image:
Changing the volume from within the Dockerfile: If any build steps
change the data within the volume after it has been declared, those
changes will be discarded.
see This
Workaround is to use COPY docker doc
COPY path/to/local/file /var/loadtest/chmodtest.txt
Related
I have a working build that I am trying to dockerise. This is my dockerfile, I added the "pwd" and "ls -l" to see if the build is copied correctly and it is. However when I try to run "docker run " I get the error "No such file or directory. Please let me know what I might be doing wrong. Appreciate your help.
Dockerfile
FROM <base image>
WORKDIR /app
RUN echo 'List all files'
RUN pwd
RUN ls -l
COPY src/mysolution-linux-amd64 /app/
RUN ls -l
ENTRYPOINT ["/app/mysolution-linux-amd64"]
I have tried ENTRYPOINT with both "./mysolution-linux-amd64" and "/app/mysolution-linux-amd64" but both fail when I run.
Output during Docker build
Sending build context to Docker daemon 1.014GB
Step 1/8 : FROM <base image>
---> 3ed27f7c19ce
Step 2/8 : WORKDIR /app
---> Running in 1b273ccccd22
Removing intermediate container 1b273ccccd22
---> 92560bbb67eb
Step 3/8 : RUN echo 'List all files'
---> Running in faddc1b6adfd
List all files
Removing intermediate container faddc1b6adfd
---> b7b2f657012e
Step 4/8 : RUN pwd
---> Running in 8354a5a476ac
/app
Removing intermediate container 8354a5a476ac
---> 204a625b730b
Step 5/8 : RUN ls -l
---> Running in 0d45cf1339d9
total 0
Removing intermediate container 0d45cf1339d9
---> 6df6451aef44
Step 6/8 : COPY src/mysolution-linux-amd64 /app/
---> 44ac2f066340
Step 7/8 : RUN ls -l
---> Running in d17ec6b0c7af
total 11460
-rwxrwxr-x 1 root root 11734780 Nov 26 04:25 mysolution-linux-amd64
Removing intermediate container d17ec6b0c7af
---> 56a879ef9440
Step 8/8 : ENTRYPOINT ["/app/mysolution-linux-amd64"]
---> Running in 33bea73f14dc
Removing intermediate container 33bea73f14dc
---> ef794fe310bc
Successfully built ef794fe310bc
Successfully tagged newtech/mysolution:latest
I have what I believe is a pretty simple setup.
I build a binary file outside of docker and then try to add it using this Dockerfile
FROM alpine
COPY apps/dist/apps /bin/
RUN chmod +x /bin/apps
RUN ls -al /bin | grep apps
CMD /bin/apps
And I think this should work.
The binary on its own seems to work on my host machine and I don't understand why it wouldn't on the docker image.
Anyways, the output I get is this:
docker build -t apps -f app.Dockerfile . && docker run apps
Sending build context to Docker daemon 287.5MB
Step 1/5 : alpine
---> d05cf6536f67
Step 2/5 : COPY apps/dist/apps /bin/
---> Using cache
---> c54d6d57154e
Step 3/5 : RUN chmod +x /bin/apps
---> Using cache
---> aa7e6adb0981
Step 4/5 : RUN ls -al /bin | grep apps
---> Running in 868c5e235d68
-rwxr-xr-x 1 root root 68395166 Dec 20 13:35 apps
Removing intermediate container 868c5e235d68
---> f052c06269b0
Step 5/5 : CMD /bin/apps
---> Running in 056fd02733e1
Removing intermediate container 056fd02733e1
---> 331600154cbe
Successfully built 331600154cbe
Successfully tagged apps:latest
/bin/sh: /bin/apps: not found
does this make sense, and am I just missing something obvious?
Your binary likely has dynamic links to libraries that don't exist inside the image filesystem. You can check those dynamic links with the ldd apps/dist/apps command.
I am building a docker image and need to include some contents from particular url.
So I make a curl request and get the contents as a zip. I am able to extract the zip into a folder as well.
However in the very next line if I try to copy the folder I get error that such file or folder does not exist.
Can someone help me?
Below is the docker file. If I do a ls -a after the unzip I can see 2 items drop and amui which is correct. But very next line if I try to copy it fails.
# escape=`
# Setup arguments
ARG BuildConfiguration=Release
ARG ASSEMBLY_VERSION=0.0
ARG FILE_VERSION=0.0.0.0
ARG NUGET_CONFIG=NuGet.config
ARG SYSTEM_ACCESSTOKEN
# base image stage
FROM abc.io/aspnet-runtime:3.1.6 AS base
WORKDIR /app
EXPOSE 80
FROM abc.io/dotnet-build:3.1.6 AS build
ARG BuildConfiguration
ARG ASSEMBLY_VERSION
ARG FILE_VERSION
ARG NUGET_CONFIG
ARG SYSTEM_ACCESSTOKEN
ENV BuildConfiguration=$BuildConfiguration `
ASSEMBLY_VERSION=$ASSEMBLY_VERSION `
FILE_VERSION=$FILE_VERSION `
NUGET_CONFIG=$NUGET_CONFIG `
SYSTEM_ACCESSTOKEN=$SYSTEM_ACCESSTOKEN
WORKDIR /build
RUN Write-Host "BuildConfiguration: ${env:BuildConfiguration}"; `
Write-Host "ASSEMBLY_VERSION: ${env:ASSEMBLY_VERSION}"; `
Write-Host "FILE_VERSION: ${env:FILE_VERSION}"; `
Write-Host "NUGET_CONFIG: ${env:NUGET_CONFIG}"; `
Write-Host "SYSTEM_ACCESSTOKEN: ${env:SYSTEM_ACCESSTOKEN}";
RUN apt-get install -y apt-transport-https
RUN apt-get update
RUN apt-get install -y aspnetcore-runtime-3.1
RUN apt-get install -y curl
RUN apt-get install -y unzip
FROM build as drop
RUN curl -o drop -u sdf#dise.com:ydfesdfasdfdf56pxvwnd5z6445l7kgezpb2lky6ja2g6xvtina4a --request GET 'https://abe.com/ded/c8002d3e-f4a9-4aea-9ab8-230027996c53/_apis/build/builds/1133376/artifacts?artifactName=drop&api-version=6.0&%24format=zip' -L
RUN unzip drop -d amui
COPY amui .
Here is the output of the docker build command. Entire output is not shown for sake of brevity.
100 18.8M 0 18.8M 0 0 757k 0 --:--:-- 0:00:25 --:--:-- 450k
100 20.4M 0 20.4M 0 0 784k 0 --:--:-- 0:00:26 --:--:-- 573k
100 20.7M 0 20.7M 0 0 774k 0 --:--:-- 0:00:27 --:--:-- 543k
100 21.1M 0 21.1M 0 0 759k 0 --:--:-- 0:00:28 --:--:-- 582k
100 21.7M 0 21.7M 0 0 764k 0 --:--:-- 0:00:29 --:--:-- 795k
Removing intermediate container 5cf617b55a61
---> 79d3a513aaf3
Step 25/45 : RUN unzip drop -d amui
---> Running in d1c070949cf9
Archive: drop
creating: amui/drop/
creating: amui/drop/amui/
inflating: amui/drop/SQL/Sql.Template.json
Removing intermediate container d1c070949cf9
---> 73959bf35eca
Step 26/45 : COPY amui src/I.C.AM.C.P
Service 'p.am.ui' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder955237576/amui: no such file or directory
Make sure the file or folder is present in the location you expect it to be. The WORKDIR /app from the FROM abc.io/aspnet-runtime:3.1.6 AS base sets the working directory for all contexts until there's another WORKDIR encountered, thus even for the ones that "inherit" from the previous one (FROM previous as new) or for the RUN context. Once that is out of the way, for the remaining part it's about contexts created by FROM and its order.
Sending build context to Docker daemon 3.072kB
Step 1/7 : FROM alpine as base
---> 6dbb9cc54074
Step 2/7 : WORKDIR /hello
---> Running in 5b7f95f0d20b
Removing intermediate container 5b7f95f0d20b
---> 0f139883db01
Step 3/7 : RUN pwd
---> Running in 1a6d896f621b
/hello
Removing intermediate container 1a6d896f621b
---> a09834ecd175
Step 4/7 : RUN cd /
---> Running in 3e6f35bf70e1
Removing intermediate container 3e6f35bf70e1
---> 9636932e3f16
Step 5/7 : RUN pwd
---> Running in fa5dc0c60493
/hello
Removing intermediate container fa5dc0c60493
---> 1d2630c5a098
Step 6/7 : FROM base as test
---> 1d2630c5a098
Step 7/7 : RUN pwd
---> Running in fabd6c1e82f8
/hello
Removing intermediate container fabd6c1e82f8
---> b39b67911581
Successfully built b39b67911581
COPY is for external files i.e. OUT of the image filesystem/context, on your host system and relative to the context path you specify for the docker build [...] <context> (or relative to the WORKDIR of a previous stage IIRC, use absolute paths if unsure).
So the case is one of this:
you already have the contents in your image and you want to use the last context created by FROM base as drop (including the curl+unzip trash) for the final image -> use RUN cp
you want to drop the trash created by curl+unzip -> use new FROM (probably FROM base) to create a new context and finalize the previous one, then copy from the previous image with COPY --from=drop amui <dest>
Optionally COPY accepts a flag --from=<name> that can be used to set the source location to a previous build stage (created with FROM .. AS <name>) that will be used instead of a build context sent by the user. In case a build stage with a specified name can't be found an image with the same name is attempted to be used instead. (source)
Order of the FROM directives matters. If you want to copy to base image from drop, then FROM ... AS drop needs to be built prior to the FROM ... AS base. You are basically using one image for building and then you want to continue building upon it:
FROM [...] AS base # creates context for "base"
...
<base image created>
FROM base AS drop # creates context for "drop"
<download>
COPY ...
<drop image created>
but you haven't started a new context (as you want to copy from the previous context - and you're still either building it or simply misusing COPY for RUN cp) for the final image, therefore even if you use COPY --from=... correctly, you're still copying to the wrong context.
Try this:
FROM [...] AS base # creates context for "base"
...
<base image created>
FROM base AS drop # creates context for "drop"
<download>
<drop image created>
FROM base # creates a new, unnamed context
COPY --from=drop ...
<final image created>
The last FROM context is used for the final image built with:
docker build --tag <name> [other options] <context path>
Example:
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM alpine as base
---> 6dbb9cc54074
Step 2/7 : RUN mkdir content;echo "hello" > content/file.txt
---> Running in 1f1a82201436
Removing intermediate container 1f1a82201436
---> 35b7a54e6952
Step 3/7 : FROM base as drop
---> 35b7a54e6952
Step 4/7 : RUN echo "world" > content/file.txt
---> Running in e5b214cad371
Removing intermediate container e5b214cad371
---> 51a3a090f43a
Step 5/7 : FROM base
---> 35b7a54e6952
Step 6/7 : COPY --from=drop content content
---> e4280a26c86f
Step 7/7 : RUN cat content/file.txt
---> Running in d71a121a7ff9
world
Removing intermediate container d71a121a7ff9
---> 760e756c4c11
Successfully built 760e756c4c11
To change files that are inside the image you should use:
RUN cp filename path
you can "debug" using RUN echo "your path:" pwd and RUN echo "folders:" ls, and iteratively navigate and check your path to see if it is correct. Just remember to erase these lines from your Dockerfile and build again.
You also can make it after build with:
docker images
get your image name and tag
docker run --privileged -i -t image_name:image_tag bash
navigate trough the image, copy, modify, git clone etc ...
exit container and run:
docker ps -a
to get the image id from the container you just leaved. Then run:
docker commit [image id] image_name_you_choose:tag
to save the images changes.
I am using jenkins image to create a docker container. For now I am just trying to create a new directory and copy a couple of files. The image build process runs fine but when I start the container I cannot see the files and the directory.
Here is my dockerfile
FROM jenkins:2.46.1
MAINTAINER MandeepSinghGulati
USER jenkins
RUN mkdir /var/jenkins_home/aws
COPY aws/config /var/jenkins_home/aws/
COPY aws/credentials /var/jenkins_home/aws/
I found a similar question here but it seems different because I am not creating the jenkins user. It already exists with home directory /var/jenkins_home/. Not sure what I am doing wrong
Here is how I am building my image and starting the container:
➜ jenkins_test docker build -t "test" .
Sending build context to Docker daemon 5.632 kB
Step 1/6 : FROM jenkins:2.46.1
---> 04c1dd56a3d8
Step 2/6 : MAINTAINER MandeepSinghGulati
---> Using cache
---> 7f76c0f7fc2d
Step 3/6 : USER jenkins
---> Running in 5dcbf4ef9f82
---> 6a64edc2d2cb
Removing intermediate container 5dcbf4ef9f82
Step 4/6 : RUN mkdir /var/jenkins_home/aws
---> Running in 1eb86a351beb
---> b42587697aec
Removing intermediate container 1eb86a351beb
Step 5/6 : COPY aws/config /var/jenkins_home/aws/
---> a9d9a28fd777
Removing intermediate container ca4a708edc6e
Step 6/6 : COPY aws/credentials /var/jenkins_home/aws/
---> 9f9ee5a603a1
Removing intermediate container 592ad0031f49
Successfully built 9f9ee5a603a1
➜ jenkins_test docker run -it -v $HOME/jenkins:/var/jenkins_home -p 8080:8080 --name=test-container test
If I run the command without the volume mount, I can see the copied files and the directory. However with the volume mount I cannot see the same. Even if I empty the directory on the host machine. Is this the expected behaviour? How can I copy over files to the directory being used as a volume ?
Existing volumes can be mounted with
docker container run -v MY-VOLUME:/var/jenkins_home ...
Furthermore, the documentation of COPY states:
All new files and directories are created with a UID and GID of 0.
So COPY does not reflect your USER directive. This seems to be the second part of your problem.
I'm trying to build a stack with one docker-compose that should contain another containers inside. This is to run a development environment with all my projects inside.
So the problem is the volume with application source isn't appearing on built image.
MacOS Sierra
Docker version 17.03.0-ce, build 60ccb22
Boot2Docker-cli version: v1.8.0
my directory tree
/dockers <======= one directory with all docker files for each project
docker-compose.yml <======= The main image
/project1 <======= dockerfile for each project
Dockerfile
/project2
Dockerfile
/project3
Dockerfile
/project1 <======= project1 source folder
test.txt
/project2
/project3
my docker-compose.yml
project1:
build: ./project1
volumes:
- ../project1/:/src
my dockerfile for project1
FROM python:2.7
RUN mkdir -p /src
WORKDIR /src
RUN echo "---------------------"
RUN ls -la
RUN echo "---------------------"
So I try to build the docker-compose file
$ sudo docker-compose build --no-cache
And then it shows an empty folder when I expect test.txt file
Building express
ERROR: Cannot locate specified Dockerfile: Dockerfile
➜ docker git:(master) ✗ sudo docker-compose build --no-cache
Building project1
Step 1/7 : FROM python:2.7
---> ca388cdb5ac1
Step 2/7 : RUN mkdir -p /src
---> Running in 393a462f7a44
---> 4fbeb32d88b3
Removing intermediate container 393a462f7a44
Step 3/7 : WORKDIR /src
---> 03ce193577ab
Removing intermediate container b1cd746b699a
Step 4/7 : RUN echo "--------------------------"
---> Running in 82df8a512c90
----------------------------
---> 6dea58ba5051
Removing intermediate container 82df8a512c90
Step 5/7 : RUN ls -la
---> Running in 905417d0cd19
total 8
drwxr-xr-x 2 root root 4096 Mar 23 17:12 . <====== EMPTY :(
drwxr-xr-x 1 root root 4096 Mar 23 17:12 .. <====== EMPTY :(
---> 53764caffb1a
Removing intermediate container 905417d0cd19
Step 6/7 : RUN echo "-----------------------------"
---> Running in 110e765d102a
----------------------------
---> b752230fd6dc
Removing intermediate container 110e765d102a
Step 7/7 : EXPOSE 3000
---> Running in 1cfe2e80d282
---> 5e3e740d5a9a
Removing intermediate container 1cfe2e80d282
Successfully built 5e3e740d5a9a
Volumes are runtime configurations in Docker. Because they are configurable, if you were to reference volumes during the build phase you would essentially be creating a potentially uncheckable broken dependency.
I'm sure there is a more technical reason - but it really shouldn't be done. Move all that stuff to the runtime setup command and you should be OK.