Understanding multi stage DockerFile builds - docker

Question

I am new to Docker. I have the following directory structure for my project
docker-compose.yml
|-services
|-orders
|-DockerFile
I am using standard ASP.Net Core DockerFile that has the following content:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
My docker-compose.yml file has
# Please refer https://aka.ms/HTTPSinContainer on how to setup an https developer certificate for your ASP .NET Core service.
version: "3.4"
services:
orders-api:
image: orders-api
build:
context: .
dockerfile: Services/Orders/Dockerfile
ports:
- "2222:80"
I have some confusion with these two files
Question 1: What is the use of WORKDIR /app on line number 2?
My understanding is that we are using a base image that we can extend so
when we import the base image in line number 1 and then set the
WORKDIR and port in line number 2 and 3, will they be used by following commands
that use this image?
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
In the line COPY ["Services/Orders/Orders.csproj", "Services/Orders/"], the path that we are specifying seems to be relevant to the docker-compose.yml file and not the DockerFile which is nested down further in folders. Does this mean that paths in Dockerfile need to be relevant to docker-compose.yml? I am asking this because I am thinking that if I run the docker build command using this Dockerfile then I will get an error since the path in the copy command will not be valid.

Answer 1

For anyone coming to this thread later and facing a similar issue, I am going to share what I have learned so far based on Leo’s answer and other sources.
Docker has a feature called multi-stage builds. This allows us to create multiple layers and use them.
ASP.Net Core app can be built using different images. The SDK image is larger in size but gives us additional tools to build and debug our code in a development environment.
In production, however, we do not need the SDK. We only want to leverage the lightweight run time in PROD. We can leverage the multi-stage builds feature of Docker to create our final container and keep it lightweight.
Coming to the Dockerfile…
Each section that starts with the “From” keyword is defining a layer. In my original Dockerfile, you can see I am creating 4 layers base, build, publish and final. We can name a layer and then use that name to create a new layer based on the first one.
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
The above code will create a layer called “base” based on the aspnet:3.1 image which contains the lightweight runtime and will be used to host our final application.
We then set the working directory to /app folder so that when we use this layer later, our commands such as COPY will run in this folder.
Next, we just expose port 80 which is the default port for the web-server.
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
Next, we download the SDK image which we will use to build the app and call this layer "build".
We set the working directory that will be used by the following "COPY" commands.
The Copy command copies files from local file system to the specified location in the container. So essentially I am copying my Orders.csproj file into the container and then running "dotnet restore" to restore all Nuget packages that are required for this project.
Copying only the .csproj or .sln file and then restoring NuGet packages without copying the entire code is more efficient as it utilizes caching as mentioned here and is a widely adopted practice which I didn’t know and was wondering why can’t we simply copy everything and just run "dotnet restore" command?
Once we have restored NuGet packages we can copy all the files and then run the "dotnet build" command to build our project.
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
Nest, we create a new layer called "publish" based on our previous layer "build" and simply publish our project with release configuration in the "app/publish" directory.
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
The final layer uses "base" (our lightweight runtime image). Copies files from publish layer’s /app/publish location to the set working directory and then set’s the entry point of the application.
So we are using 2 separate images. SDK to build our app, and aspnet image to host our app in the container but the important thing to note here is that the final container that will be generated will only contain the aspnet runtime and will be smaller in size.
All these things are documented and can be searched but I struggled because the information was scattered. Hopefully, it will save some time to anyone that is new to ASP.Net Core and Docker in the future.

Answer 2

Question 1: What is the use of WORKDIR /app on line number 2?
This defines your current working directory. Following commands will by default be run from that location and relative paths will start there.
Example with the following file structure:
/app/
- README.md
- folder1/
- some.txt
WORKDIR /app
RUN ls
will print
folder1
README.md
and
WORKDIR /app/folder1
RUN ls
will print
some.txt
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
See the answer to Question 1. The following COPY and dotnet restore command are executed in the /src source directory. (The build is done from the /src directory and the created artifact in /app/publish is later copied to the /app directory in the last stage, to be executed from that /app directory.)
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
Copy takes two paths. The first one references the source (a file or folder from the context the docker image is build from) and the second one references the destination (a file order folder in your resulting docker image.) Hence these paths are usually only specific to the Dockerfile and independent from your docker-compose project.
However in your case the context of your docker image build is defined in the docker-compose.yml here:
build:
context: .
dockerfile: Services/Orders/Dockerfile
and therefore the context of your docker image image build seems to be the directory where your docker-compose.yml is located. You could build the same image though if you just run docker build -f Services/Orders/Dockerfile . in that folder. (It is not docker-compose specific)
Therefore you should find Orders.csproj in ./Services/Orders/ starting from the directory your docker-compose.yml is located in. This file is than copied to /src/Services/Orders/Orders.csproj in your second build stage. (The /src can be commited in the COPY statement as it is a relative path starting from your current working directory, which is defined in the line above. - See Question 1)

Answer 3

The reason we use "multi-stage" builds and use several images and copy files to continue instead of just sequentially carrying out steps is mainly due to trying to keep the image size small. Although disk space may be adequate, the following factors can also be relevant:
Build/deploy times. Larger images mean longer continuous integration, the more time will be spent waiting for these operations to complete.
Start-up time. When running your application in production the longer the download takes, the longer it will be before the new container can be up and running.
Therefore using a multi-stage approach as below we are able to omit the sdk which is much bigger and is only needed for building not running the application:
Notice that we use ENTRYPOINT in the second image as the "dotnet run" is only available in the sdk so we make a minor adjustment to just directly run the .dll to get the same outcome.