I'm using Docker to build an ASP site, and I'm confused about where my files are going. Here's my dockerfile (the application is called AspCore)
FROM microsoft/dotnet:2.2-sdk as build
ARG BUILDCONFIG=RELEASE
ARG VERSION=1.0.0
COPY AspCore.csproj /build/
RUN dotnet restore ./build/AspCore.csproj
COPY . ./build/
WORKDIR /build/
RUN dotnet publish ./AspCore.csproj -c $BUILDCONFIG -o out /p:Version=$VERSION
FROM microsoft/dotnet:2.2-aspnetcore-runtime
WORKDIR /app
COPY --from=build /build/out .
ENTRYPOINT ["dotnet", "AspCore.dll"]
This works correctly and I can access the site when I run the image, but I don't quite understand how this is working. When I open a shell to my container, I don't see a build directory either in the app directory or in the base directory. I also can't find the AspCore.csproj file.
Isn't the dockerfile copying AspCore.csproj into the build directory, so shouldn't there be a build directory with a bunch of files in it on my container? What am I misunderstanding?
That's just because you're using 2 stage on your Dockerfile to build your image.
Stage 1: (BUILD) Based on microsoft/dotnet:2.2-sdk as build image to build your source code into dll files
Stage 2: (SERVE) Based on microsoft/dotnet:2.2-aspnetcore-runtime to create a runtime for your dlls, in this stage you've already copy files from previous stage into a folder called app by this line COPY --from=build /build/out ..
After stage 2 copied files from stage1, nothing else you can see from stage1 but copied files, that's why when you start your container you didn't see /build folder
This pattern is good for production build when you want to minimize your image, because actually we don't need sdk in production environment, we just need runtime for the compiled code.
Hope that clear enough, for more information, you can take a look at this article
Related
I've been building nodejs applications and wanted to investigate projects inside a Docker container.
But when I check the running container via bin/bash I cannot find /app directory in a container
While building image no errors are being displayed
This is my docker image file
FROM node:15.7.0 as build-stage
WORKDIR /app
COPY package*.json /app/
RUN npm install
COPY ./ /app/
ARG configuration=production
RUN npm run build -- --output-path=./dist/out --configuration $configuration
FROM nginx:1.18
COPY --from=build-stage /app/dist/out/ /usr/share/nginx/html
COPY ./nginx-custom.conf /etc/nginx/conf.d/default.conf
Sèdjro has the correct answer, but I wanted to expand on it briefly.
You're using what is called a multi-stage build. If you haven't seen them already, the linked docs are a good read.
Each FROM ... line in your Dockerfile starts a new build stage, and only the contents of the final stage are relevant to the generated Docker image.
Other stages in the file are available as sources of the COPY --from=... directive.
In other words, as soon as you add a second FROM ... line, as in...
FROM node:15.7.0 as build-stage
.
.
.
FROM nginx:1.18
...then the contents of the first stage are effectively discarded,
unless you explicitly copy them into the final image using COPY --from=.... Your /app directory only exists in the first stage, since you're copying it into a different location in the final stage.
The /app directory is not inside of your docker image.
It's in your build stage image.
Check the /usr/share/nginx/html directory or don't use build stage.
I am new to Docker. I have the following directory structure for my project
docker-compose.yml
|-services
|-orders
|-DockerFile
I am using standard ASP.Net Core DockerFile that has the following content:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
My docker-compose.yml file has
# Please refer https://aka.ms/HTTPSinContainer on how to setup an https developer certificate for your ASP .NET Core service.
version: "3.4"
services:
orders-api:
image: orders-api
build:
context: .
dockerfile: Services/Orders/Dockerfile
ports:
- "2222:80"
I have some confusion with these two files
Question 1: What is the use of WORKDIR /app on line number 2?
My understanding is that we are using a base image that we can extend so
when we import the base image in line number 1 and then set the
WORKDIR and port in line number 2 and 3, will they be used by following commands
that use this image?
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
In the line COPY ["Services/Orders/Orders.csproj", "Services/Orders/"], the path that we are specifying seems to be relevant to the docker-compose.yml file and not the DockerFile which is nested down further in folders. Does this mean that paths in Dockerfile need to be relevant to docker-compose.yml? I am asking this because I am thinking that if I run the docker build command using this Dockerfile then I will get an error since the path in the copy command will not be valid.
For anyone coming to this thread later and facing a similar issue, I am going to share what I have learned so far based on Leo’s answer and other sources.
Docker has a feature called multi-stage builds. This allows us to create multiple layers and use them.
ASP.Net Core app can be built using different images. The SDK image is larger in size but gives us additional tools to build and debug our code in a development environment.
In production, however, we do not need the SDK. We only want to leverage the lightweight run time in PROD. We can leverage the multi-stage builds feature of Docker to create our final container and keep it lightweight.
Coming to the Dockerfile…
Each section that starts with the “From” keyword is defining a layer. In my original Dockerfile, you can see I am creating 4 layers base, build, publish and final. We can name a layer and then use that name to create a new layer based on the first one.
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
The above code will create a layer called “base” based on the aspnet:3.1 image which contains the lightweight runtime and will be used to host our final application.
We then set the working directory to /app folder so that when we use this layer later, our commands such as COPY will run in this folder.
Next, we just expose port 80 which is the default port for the web-server.
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
Next, we download the SDK image which we will use to build the app and call this layer "build".
We set the working directory that will be used by the following "COPY" commands.
The Copy command copies files from local file system to the specified location in the container. So essentially I am copying my Orders.csproj file into the container and then running "dotnet restore" to restore all Nuget packages that are required for this project.
Copying only the .csproj or .sln file and then restoring NuGet packages without copying the entire code is more efficient as it utilizes caching as mentioned here and is a widely adopted practice which I didn’t know and was wondering why can’t we simply copy everything and just run "dotnet restore" command?
Once we have restored NuGet packages we can copy all the files and then run the "dotnet build" command to build our project.
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
Nest, we create a new layer called "publish" based on our previous layer "build" and simply publish our project with release configuration in the "app/publish" directory.
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
The final layer uses "base" (our lightweight runtime image). Copies files from publish layer’s /app/publish location to the set working directory and then set’s the entry point of the application.
So we are using 2 separate images. SDK to build our app, and aspnet image to host our app in the container but the important thing to note here is that the final container that will be generated will only contain the aspnet runtime and will be smaller in size.
All these things are documented and can be searched but I struggled because the information was scattered. Hopefully, it will save some time to anyone that is new to ASP.Net Core and Docker in the future.
Question 1: What is the use of WORKDIR /app on line number 2?
This defines your current working directory. Following commands will by default be run from that location and relative paths will start there.
Example with the following file structure:
/app/
- README.md
- folder1/
- some.txt
WORKDIR /app
RUN ls
will print
folder1
README.md
and
WORKDIR /app/folder1
RUN ls
will print
some.txt
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
See the answer to Question 1. The following COPY and dotnet restore command are executed in the /src source directory. (The build is done from the /src directory and the created artifact in /app/publish is later copied to the /app directory in the last stage, to be executed from that /app directory.)
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
Copy takes two paths. The first one references the source (a file or folder from the context the docker image is build from) and the second one references the destination (a file order folder in your resulting docker image.) Hence these paths are usually only specific to the Dockerfile and independent from your docker-compose project.
However in your case the context of your docker image build is defined in the docker-compose.yml here:
build:
context: .
dockerfile: Services/Orders/Dockerfile
and therefore the context of your docker image image build seems to be the directory where your docker-compose.yml is located. You could build the same image though if you just run docker build -f Services/Orders/Dockerfile . in that folder. (It is not docker-compose specific)
Therefore you should find Orders.csproj in ./Services/Orders/ starting from the directory your docker-compose.yml is located in. This file is than copied to /src/Services/Orders/Orders.csproj in your second build stage. (The /src can be commited in the COPY statement as it is a relative path starting from your current working directory, which is defined in the line above. - See Question 1)
The reason we use "multi-stage" builds and use several images and copy files to continue instead of just sequentially carrying out steps is mainly due to trying to keep the image size small. Although disk space may be adequate, the following factors can also be relevant:
Build/deploy times. Larger images mean longer continuous integration, the more time will be spent waiting for these operations to complete.
Start-up time. When running your application in production the longer the download takes, the longer it will be before the new container can be up and running.
Therefore using a multi-stage approach as below we are able to omit the sdk which is much bigger and is only needed for building not running the application:
Notice that we use ENTRYPOINT in the second image as the "dotnet run" is only available in the sdk so we make a minor adjustment to just directly run the .dll to get the same outcome.
I am very much new on docker technology, I am getting the build error while creating the .Net Core 3.1 on Azure DevOps CI pipeline on Docker image tasks:
Step 7/17 : COPY ["API2/API2.csproj", "API2/"] COPY failed: CreateFile
\?\C:\ProgramData\docker\tmp\docker-builder021493529\API2\API2.csproj:
The system cannot find the path specified.
My default docker file is
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1809 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1809 AS build
WORKDIR /src
COPY ["API1/API1.csproj", "API1/"]
RUN dotnet restore "API1/API1.csproj"
COPY . .
WORKDIR "/src/API1"
RUN dotnet build "API1.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "API1.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "API1.dll"]
Please do let me know from where I am doing mistake.
Here are the docker tasks:
Here are the folder and files structure on azure DevOps as well:
COPY ["API1/API1.csproj", "API1/"]
Based on the error message, this should the line that caused the error message.
Step1:
Please ensure you did not configure the .dockerignore file to exclude this file: API1/API1.csproj, which must exists in the directory where you run your build from.
Step2:
After above confirmed, now we can consider the error is caused about the server could not find the csproj file correctly by following the context and the path you provided.
According to the your original definition: API1/API1.csproj, I guess the actual path of API1.csproj in your repository should be src/API1/API1.csproj, right?
If yes, here has 2 method you can try:
1). Change the COPY definition as:
COPY ["API1.csproj", "API1/"]
Updated:
When you apply this method, you may succeed to COPY, but failed with Program does not contain a static 'Main' method suitable for an entry point *****.
Here it means that the COPY . . does not copy the files correctly.
At this time, please also change the COPY . . to COPY . API1/. This will add folder to dest path.
2). Another way is you could specify API1 to Build context in task:
Below is what I am using, and I do not need make any changes into my dockerfile:
you can input $(Build.Repository.LocalPath) by replacing hard code the context:
Updated:
In Docker 2.*, you can also leave Build context to **:
You can refer to my previous answer on such question: #1.
Based on my opinions, I am not recommend the first method I mentioned above, because it let your dockerfile different with the one which you can run successfully in Visual studio.
I am really curious to how to interpret and debug with the following error:-
C:\users\project>docker-compose build
Step 6/15 : COPY *.csproj ./
ERROR: Service 'invoiceservice' failed to build: COPY failed: no source files were specified
This is particular micro-service as i have few more such services.
docker file :-
FROM mcr.microsoft.com/dotnet/core/runtime:2.2 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build
WORKDIR /src
COPY *.csproj ./
RUN dotnet restore -s https://api.nuget.org/v3/index.json -s https://www.myget.org/F/autoweb/api/v3/index.json
COPY . .
WORKDIR /src/src/InvoiceManagement/InvoiceService
RUN dotnet publish -c Release -o out
FROM build AS publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/out .
ENTRYPOINT ["dotnet", "InvoiceService.dll"]
Interesting part is when I build this from Visual Studio IDE, its being built fine but it does not build on CLI.
docker compose file:-
invoiceservice:
image: ${DOCKER_REGISTRY-}invoiceservice
build:
context: .
dockerfile: src/InvoiceManagement/InvoiceService/Dockerfile
I don't understand why CLI could not find the source location and copy where as VS works fine. Any clue???
It's likely an issue with your Docker context. Commands in Dockerfiles are relative to where the docker/docker-compose CLI tool is run, not relative to the Dockerfile location.
In other words, if you run the command from the solution root, then your csproj file is actually under ./src/InvoiceManagement/InvoiceService. As a result, *.csproj finds no files, because there's no project files literally in your solution root.
I tried replicating your problem and I was able to copy all the files successfully (instead of .csproj, I used .txt). The problem occurred when there was no txt file to copy. My COPY command failed with exactly the same message as yours.
Now, why is VS able to build the image successfully? That is because VS build the project first! When the project's build procedure is completed, a .csproj file is generated and that gets copied to the image.
To confirm the results, ls your current directory (when the build fails from command line) and see if there is any .csproj file in that directory.
I'm trying to put a dockerfile in a subdirectory of my main startup project AppMain. AppMain has a dependency project called AppDependency. When the dockerfile is in the root directory of AppMain, all works correctly, but when it's nested in a subdirectory of the AppMain, it fails with
4>Step 5/20 : COPY ["AppMain/AppMain.csproj", "AppMain/"]
4>COPY failed: stat /var/lib/docker/tmp/docker-builder453314675/AppMain/AppMain.csproj: no such file or directory
So that makes sense to me since it's nested in a subdirectory, so I simply added a "../" in front of the COPY commands but that results in the following context exception:
4>COPY failed: Forbidden path outside the build context: ../AppMain/AppMain.csproj ()
Alright so that makes sense as well because the docker documentation states that the context starts where the dockerfile is located, but this leads me to ask the real question.
If it cannot copy outside of the where the dockerfile is located and it works when its located in the root of AppMain, then why doesn't it fail when copying the AppReference project which obviously is above where the docker file is located and outside the context since it would have to go up one directory and then down to the AppReference project?
Is there a way to achieve having a dockerfile nested in a subdirectory?
The entire docker file is listed below.
FROM mcr.microsoft.com/dotnet/core/runtime:2.2-stretch-slim AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["AppMain/AppMain.csproj", "AppMain/"]
COPY ["AppReference/AppReference.csproj", "AppReference/"]
RUN dotnet restore "AppMain/AppMain.csproj"
COPY . .
WORKDIR "/src/AppMain"
RUN dotnet build "AppMain.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "AppMain.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "AppMain.dll"]
When use docker build -t xxx . to build in the subdirectory, only the contents in . will submit to docker daemon for docker build. The things in parent folder will not be able to submit to docker engine to build.
So, you need to move the execution directory back to the parent folder with cd .., then use next to build, it will then send the . which now is the contents of parent folder to engine(But you need now specify dockerfile, like this)
docker build -t xxx -f ./YOUR_Subdirectory/Dockerfile .
It is really working.
What it is ?
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.