Docker image composition COPY command - docker

Question

I am building a docker image and need to include some contents from particular url.
So I make a curl request and get the contents as a zip. I am able to extract the zip into a folder as well.
However in the very next line if I try to copy the folder I get error that such file or folder does not exist.
Can someone help me?
Below is the docker file. If I do a ls -a after the unzip I can see 2 items drop and amui which is correct. But very next line if I try to copy it fails.
# escape=`
# Setup arguments
ARG BuildConfiguration=Release
ARG ASSEMBLY_VERSION=0.0
ARG FILE_VERSION=0.0.0.0
ARG NUGET_CONFIG=NuGet.config
ARG SYSTEM_ACCESSTOKEN
# base image stage
FROM abc.io/aspnet-runtime:3.1.6 AS base
WORKDIR /app
EXPOSE 80
FROM abc.io/dotnet-build:3.1.6 AS build
ARG BuildConfiguration
ARG ASSEMBLY_VERSION
ARG FILE_VERSION
ARG NUGET_CONFIG
ARG SYSTEM_ACCESSTOKEN
ENV BuildConfiguration=$BuildConfiguration `
ASSEMBLY_VERSION=$ASSEMBLY_VERSION `
FILE_VERSION=$FILE_VERSION `
NUGET_CONFIG=$NUGET_CONFIG `
SYSTEM_ACCESSTOKEN=$SYSTEM_ACCESSTOKEN
WORKDIR /build
RUN Write-Host "BuildConfiguration: ${env:BuildConfiguration}"; `
Write-Host "ASSEMBLY_VERSION: ${env:ASSEMBLY_VERSION}"; `
Write-Host "FILE_VERSION: ${env:FILE_VERSION}"; `
Write-Host "NUGET_CONFIG: ${env:NUGET_CONFIG}"; `
Write-Host "SYSTEM_ACCESSTOKEN: ${env:SYSTEM_ACCESSTOKEN}";
RUN apt-get install -y apt-transport-https
RUN apt-get update
RUN apt-get install -y aspnetcore-runtime-3.1
RUN apt-get install -y curl
RUN apt-get install -y unzip
FROM build as drop
RUN curl -o drop -u sdf#dise.com:ydfesdfasdfdf56pxvwnd5z6445l7kgezpb2lky6ja2g6xvtina4a --request GET 'https://abe.com/ded/c8002d3e-f4a9-4aea-9ab8-230027996c53/_apis/build/builds/1133376/artifacts?artifactName=drop&api-version=6.0&%24format=zip' -L
RUN unzip drop -d amui
COPY amui .
Here is the output of the docker build command. Entire output is not shown for sake of brevity.
100 18.8M 0 18.8M 0 0 757k 0 --:--:-- 0:00:25 --:--:-- 450k
100 20.4M 0 20.4M 0 0 784k 0 --:--:-- 0:00:26 --:--:-- 573k
100 20.7M 0 20.7M 0 0 774k 0 --:--:-- 0:00:27 --:--:-- 543k
100 21.1M 0 21.1M 0 0 759k 0 --:--:-- 0:00:28 --:--:-- 582k
100 21.7M 0 21.7M 0 0 764k 0 --:--:-- 0:00:29 --:--:-- 795k
Removing intermediate container 5cf617b55a61
---> 79d3a513aaf3
Step 25/45 : RUN unzip drop -d amui
---> Running in d1c070949cf9
Archive: drop
creating: amui/drop/
creating: amui/drop/amui/
inflating: amui/drop/SQL/Sql.Template.json
Removing intermediate container d1c070949cf9
---> 73959bf35eca
Step 26/45 : COPY amui src/I.C.AM.C.P
Service 'p.am.ui' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder955237576/amui: no such file or directory

Answer 1

Make sure the file or folder is present in the location you expect it to be. The WORKDIR /app from the FROM abc.io/aspnet-runtime:3.1.6 AS base sets the working directory for all contexts until there's another WORKDIR encountered, thus even for the ones that "inherit" from the previous one (FROM previous as new) or for the RUN context. Once that is out of the way, for the remaining part it's about contexts created by FROM and its order.
Sending build context to Docker daemon 3.072kB
Step 1/7 : FROM alpine as base
---> 6dbb9cc54074
Step 2/7 : WORKDIR /hello
---> Running in 5b7f95f0d20b
Removing intermediate container 5b7f95f0d20b
---> 0f139883db01
Step 3/7 : RUN pwd
---> Running in 1a6d896f621b
/hello
Removing intermediate container 1a6d896f621b
---> a09834ecd175
Step 4/7 : RUN cd /
---> Running in 3e6f35bf70e1
Removing intermediate container 3e6f35bf70e1
---> 9636932e3f16
Step 5/7 : RUN pwd
---> Running in fa5dc0c60493
/hello
Removing intermediate container fa5dc0c60493
---> 1d2630c5a098
Step 6/7 : FROM base as test
---> 1d2630c5a098
Step 7/7 : RUN pwd
---> Running in fabd6c1e82f8
/hello
Removing intermediate container fabd6c1e82f8
---> b39b67911581
Successfully built b39b67911581
COPY is for external files i.e. OUT of the image filesystem/context, on your host system and relative to the context path you specify for the docker build [...] <context> (or relative to the WORKDIR of a previous stage IIRC, use absolute paths if unsure).
So the case is one of this:
you already have the contents in your image and you want to use the last context created by FROM base as drop (including the curl+unzip trash) for the final image -> use RUN cp
you want to drop the trash created by curl+unzip -> use new FROM (probably FROM base) to create a new context and finalize the previous one, then copy from the previous image with COPY --from=drop amui <dest>
Optionally COPY accepts a flag --from=<name> that can be used to set the source location to a previous build stage (created with FROM .. AS <name>) that will be used instead of a build context sent by the user. In case a build stage with a specified name can't be found an image with the same name is attempted to be used instead. (source)
Order of the FROM directives matters. If you want to copy to base image from drop, then FROM ... AS drop needs to be built prior to the FROM ... AS base. You are basically using one image for building and then you want to continue building upon it:
FROM [...] AS base # creates context for "base"
...
<base image created>
FROM base AS drop # creates context for "drop"
<download>
COPY ...
<drop image created>
but you haven't started a new context (as you want to copy from the previous context - and you're still either building it or simply misusing COPY for RUN cp) for the final image, therefore even if you use COPY --from=... correctly, you're still copying to the wrong context.
Try this:
FROM [...] AS base # creates context for "base"
...
<base image created>
FROM base AS drop # creates context for "drop"
<download>
<drop image created>
FROM base # creates a new, unnamed context
COPY --from=drop ...
<final image created>
The last FROM context is used for the final image built with:
docker build --tag <name> [other options] <context path>
Example:
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM alpine as base
---> 6dbb9cc54074
Step 2/7 : RUN mkdir content;echo "hello" > content/file.txt
---> Running in 1f1a82201436
Removing intermediate container 1f1a82201436
---> 35b7a54e6952
Step 3/7 : FROM base as drop
---> 35b7a54e6952
Step 4/7 : RUN echo "world" > content/file.txt
---> Running in e5b214cad371
Removing intermediate container e5b214cad371
---> 51a3a090f43a
Step 5/7 : FROM base
---> 35b7a54e6952
Step 6/7 : COPY --from=drop content content
---> e4280a26c86f
Step 7/7 : RUN cat content/file.txt
---> Running in d71a121a7ff9
world
Removing intermediate container d71a121a7ff9
---> 760e756c4c11
Successfully built 760e756c4c11

Answer 2

To change files that are inside the image you should use:
RUN cp filename path
you can "debug" using RUN echo "your path:" pwd and RUN echo "folders:" ls, and iteratively navigate and check your path to see if it is correct. Just remember to erase these lines from your Dockerfile and build again.
You also can make it after build with:
docker images
get your image name and tag
docker run --privileged -i -t image_name:image_tag bash
navigate trough the image, copy, modify, git clone etc ...
exit container and run:
docker ps -a
to get the image id from the container you just leaved. Then run:
docker commit [image id] image_name_you_choose:tag
to save the images changes.