My goal
I want to publish a docker image to Dockerhub from the Azure DevOps Pipeline.
The pipeline runs after a commit on the master branch in Gitlab.
The application I want to build an image for is a .Net Core Web API project.
I'm using the automatically created Dockerfile by dotnet.
Error
This is the error I get:
Information
The folder structure of my application looks like this:
- APIGateway [Folder]
- APIGateway [Folder]
- Controller/Models/Services etc... [Folders]
- APIGateway.csproj
- APIGateway.csproj.user
- **Dockerfile**
- Program.cs
- Startup.cs
- Appsettings,json
- Appsettings.Development.json
- APIGateway.UnitTests [Folder]
- .dockerignore
- APIGateway.sln
- Readme.md
My Dockerfile looks like this:
For the Azure Pipeline, I added the .Net Core template task (restore, build, test, publish, publish artifect) to the agent job. The pipeline executes these tasks successfully.
I also added the Docker BuildAndPush task and configured my Dockerhub as a container registry.
The commands for the BuildAndPush task:
I already looked at this similar post: Azure Pipeline to build docker images fails using same docker file in Visual Studio. People mainly suggested to add 'Build.Repository.LocalPath' to the build context. But I still get the same error.
Does anyone know how to solve this issue?
Others have alluded to this in the comments.
Given you have a folder structure containing multiple projects within a solution similar to this:
AND your Dockerfile was created in one of the projects within the solution (e.g., the Server project) similar to this (e.g., generated by VS-right click, Add Docker support):
FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /src
COPY ["Server/QuizManagerClientHosted.Server.csproj", "Server/"]
COPY ["Client/QuizManagerClientHosted.Client.csproj", "Client/"]
COPY ["Shared/QuizManagerClientHosted.Shared.csproj", "Shared/"]
RUN dotnet restore "Server/QuizManagerClientHosted.Server.csproj"
COPY . .
WORKDIR "/src/Server"
RUN dotnet build "QuizManagerClientHosted.Server.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "QuizManagerClientHosted.Server.csproj" -c Release -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "QuizManagerClientHosted.Server.dll"]
Problem: It works locally but not via ADO (or perhaps GitHub Actions or other CI/CD solutions).
Solution: Move the Dockerfile up one level (e.g., from the Server directory to the QuizManagerClientHosted directory).
Related
I have a simple dockerfile which I am trying to inject vc_redist into the container, when I build the image using docker desktop my appliaction runs correctly, however when I build the same image in Azure Pipeline and than pull that image from Azure Container Registry I am missing the vc_redist layers. Why is there a difference between building the image using docker desktop and Azure Pipeline Tasks? Here is a sample of my dockerfile
FROM mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019 AS base
WORKDIR /app
USER ContainerAdministrator
ADD https://download.visualstudio.microsoft.com/download/pr/9fbed7c7-7012-4cc0-a0a3-a541f51981b5/e7eec15278b4473e26d7e32cef53a34c/vc_redist.x64.exe /vc_redist.x64.exe
RUN /vc_redist.x64.exe /norestart /install /quiet
RUN del /vc_redist.x64.exe
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8 AS build
WORKDIR /src
RUN dotnet restore .....
COPY . .
RUN dotnet build ....
FROM build AS publish
RUN dotnet publish ...
FROM base AS runtime
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["myapp.exe"]
your URL is not working here. (it's probably a private URL only for your IP or time based)
see https://github.com/microsoft/dotnet-framework-docker/issues/15
he uses a Microsoft download link that is working
https://download.microsoft.com
while you use a visual studio link https://download.visualstudio.microsoft.com/
I would suggest to download the installer and re-upload it to a gist or GitHub, or just add the file itself to the docker
I am new to Docker. I have the following directory structure for my project
docker-compose.yml
|-services
|-orders
|-DockerFile
I am using standard ASP.Net Core DockerFile that has the following content:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
My docker-compose.yml file has
# Please refer https://aka.ms/HTTPSinContainer on how to setup an https developer certificate for your ASP .NET Core service.
version: "3.4"
services:
orders-api:
image: orders-api
build:
context: .
dockerfile: Services/Orders/Dockerfile
ports:
- "2222:80"
I have some confusion with these two files
Question 1: What is the use of WORKDIR /app on line number 2?
My understanding is that we are using a base image that we can extend so
when we import the base image in line number 1 and then set the
WORKDIR and port in line number 2 and 3, will they be used by following commands
that use this image?
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
In the line COPY ["Services/Orders/Orders.csproj", "Services/Orders/"], the path that we are specifying seems to be relevant to the docker-compose.yml file and not the DockerFile which is nested down further in folders. Does this mean that paths in Dockerfile need to be relevant to docker-compose.yml? I am asking this because I am thinking that if I run the docker build command using this Dockerfile then I will get an error since the path in the copy command will not be valid.
For anyone coming to this thread later and facing a similar issue, I am going to share what I have learned so far based on Leo’s answer and other sources.
Docker has a feature called multi-stage builds. This allows us to create multiple layers and use them.
ASP.Net Core app can be built using different images. The SDK image is larger in size but gives us additional tools to build and debug our code in a development environment.
In production, however, we do not need the SDK. We only want to leverage the lightweight run time in PROD. We can leverage the multi-stage builds feature of Docker to create our final container and keep it lightweight.
Coming to the Dockerfile…
Each section that starts with the “From” keyword is defining a layer. In my original Dockerfile, you can see I am creating 4 layers base, build, publish and final. We can name a layer and then use that name to create a new layer based on the first one.
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
The above code will create a layer called “base” based on the aspnet:3.1 image which contains the lightweight runtime and will be used to host our final application.
We then set the working directory to /app folder so that when we use this layer later, our commands such as COPY will run in this folder.
Next, we just expose port 80 which is the default port for the web-server.
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY ["Services/Orders/Orders.csproj", "Services/Orders/"]
RUN dotnet restore "Services/Orders/Orders.csproj"
COPY . .
WORKDIR "/src/Services/Orders"
RUN dotnet build "Orders.csproj" -c Release -o /app/build
Next, we download the SDK image which we will use to build the app and call this layer "build".
We set the working directory that will be used by the following "COPY" commands.
The Copy command copies files from local file system to the specified location in the container. So essentially I am copying my Orders.csproj file into the container and then running "dotnet restore" to restore all Nuget packages that are required for this project.
Copying only the .csproj or .sln file and then restoring NuGet packages without copying the entire code is more efficient as it utilizes caching as mentioned here and is a widely adopted practice which I didn’t know and was wondering why can’t we simply copy everything and just run "dotnet restore" command?
Once we have restored NuGet packages we can copy all the files and then run the "dotnet build" command to build our project.
FROM build AS publish
RUN dotnet publish "Orders.csproj" -c Release -o /app/publish
Nest, we create a new layer called "publish" based on our previous layer "build" and simply publish our project with release configuration in the "app/publish" directory.
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Orders.dll"]
The final layer uses "base" (our lightweight runtime image). Copies files from publish layer’s /app/publish location to the set working directory and then set’s the entry point of the application.
So we are using 2 separate images. SDK to build our app, and aspnet image to host our app in the container but the important thing to note here is that the final container that will be generated will only contain the aspnet runtime and will be smaller in size.
All these things are documented and can be searched but I struggled because the information was scattered. Hopefully, it will save some time to anyone that is new to ASP.Net Core and Docker in the future.
Question 1: What is the use of WORKDIR /app on line number 2?
This defines your current working directory. Following commands will by default be run from that location and relative paths will start there.
Example with the following file structure:
/app/
- README.md
- folder1/
- some.txt
WORKDIR /app
RUN ls
will print
folder1
README.md
and
WORKDIR /app/folder1
RUN ls
will print
some.txt
Question 2: Why are we setting WORKDIR /src for the SDK image and not WORKDIR /app?
See the answer to Question 1. The following COPY and dotnet restore command are executed in the /src source directory. (The build is done from the /src directory and the created artifact in /app/publish is later copied to the /app directory in the last stage, to be executed from that /app directory.)
Question 3: Are paths in copy commands relevant to Dockerfile or Docker-compose.yml file?
Copy takes two paths. The first one references the source (a file or folder from the context the docker image is build from) and the second one references the destination (a file order folder in your resulting docker image.) Hence these paths are usually only specific to the Dockerfile and independent from your docker-compose project.
However in your case the context of your docker image build is defined in the docker-compose.yml here:
build:
context: .
dockerfile: Services/Orders/Dockerfile
and therefore the context of your docker image image build seems to be the directory where your docker-compose.yml is located. You could build the same image though if you just run docker build -f Services/Orders/Dockerfile . in that folder. (It is not docker-compose specific)
Therefore you should find Orders.csproj in ./Services/Orders/ starting from the directory your docker-compose.yml is located in. This file is than copied to /src/Services/Orders/Orders.csproj in your second build stage. (The /src can be commited in the COPY statement as it is a relative path starting from your current working directory, which is defined in the line above. - See Question 1)
The reason we use "multi-stage" builds and use several images and copy files to continue instead of just sequentially carrying out steps is mainly due to trying to keep the image size small. Although disk space may be adequate, the following factors can also be relevant:
Build/deploy times. Larger images mean longer continuous integration, the more time will be spent waiting for these operations to complete.
Start-up time. When running your application in production the longer the download takes, the longer it will be before the new container can be up and running.
Therefore using a multi-stage approach as below we are able to omit the sdk which is much bigger and is only needed for building not running the application:
Notice that we use ENTRYPOINT in the second image as the "dotnet run" is only available in the sdk so we make a minor adjustment to just directly run the .dll to get the same outcome.
I am very much new on docker technology, I am getting the build error while creating the .Net Core 3.1 on Azure DevOps CI pipeline on Docker image tasks:
Step 7/17 : COPY ["API2/API2.csproj", "API2/"] COPY failed: CreateFile
\?\C:\ProgramData\docker\tmp\docker-builder021493529\API2\API2.csproj:
The system cannot find the path specified.
My default docker file is
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1809 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-nanoserver-1809 AS build
WORKDIR /src
COPY ["API1/API1.csproj", "API1/"]
RUN dotnet restore "API1/API1.csproj"
COPY . .
WORKDIR "/src/API1"
RUN dotnet build "API1.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "API1.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "API1.dll"]
Please do let me know from where I am doing mistake.
Here are the docker tasks:
Here are the folder and files structure on azure DevOps as well:
COPY ["API1/API1.csproj", "API1/"]
Based on the error message, this should the line that caused the error message.
Step1:
Please ensure you did not configure the .dockerignore file to exclude this file: API1/API1.csproj, which must exists in the directory where you run your build from.
Step2:
After above confirmed, now we can consider the error is caused about the server could not find the csproj file correctly by following the context and the path you provided.
According to the your original definition: API1/API1.csproj, I guess the actual path of API1.csproj in your repository should be src/API1/API1.csproj, right?
If yes, here has 2 method you can try:
1). Change the COPY definition as:
COPY ["API1.csproj", "API1/"]
Updated:
When you apply this method, you may succeed to COPY, but failed with Program does not contain a static 'Main' method suitable for an entry point *****.
Here it means that the COPY . . does not copy the files correctly.
At this time, please also change the COPY . . to COPY . API1/. This will add folder to dest path.
2). Another way is you could specify API1 to Build context in task:
Below is what I am using, and I do not need make any changes into my dockerfile:
you can input $(Build.Repository.LocalPath) by replacing hard code the context:
Updated:
In Docker 2.*, you can also leave Build context to **:
You can refer to my previous answer on such question: #1.
Based on my opinions, I am not recommend the first method I mentioned above, because it let your dockerfile different with the one which you can run successfully in Visual studio.
I'm attempting to write a Dockerfile that builds my .NET Core 2.2 console app project-microservice-logger.csproj. I'm getting some build errors, and I beleive this is because the Dockerfile I'm trying to build is residing in the directory that holds the project-microservice-logger.csproj
But this project has some references to projects that are in a different directory.
/company/project-common/project-common-database/project-common-database.csproj
/company/project-microservices/project-microservice-logger/project-microservice-logger.csproj
When I issue the command, while in the /company/project-microservices/project-microservice-logger directory, I'm getting errors like
The type or namespace name 'Common' does not exist in the namespace
Since I can build it just fine, I know it has something due to Dockers build context, and that it can't find the file.
My Docker file is nothing special, essentially just taken from some of the Docker templates for .NET Core that are readily available online.
FROM microsoft/dotnet:2.2-sdk AS build-env
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out
# Build runtime image
FROM microsoft/dotnet:2.2-runtime
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "logging.dll"]
I'm open to making changes to how we make project references, if it's a better solution for future Continuous Deployment situations.
All the code is in one repository, but separated into different solutions at the moment (core, micro-services, and web)
Versions
.NET Core 2.2
Docker for Mac / Version 2.0.0.2
I put the publish logic in a shell script, and the docker image uses the files built from that as its code base.
This script assumes the following folder structure:
Solution-Folder/build/this-build-script.sh
Solution-Folder/MyProject/MyProjectName.csproj
BUILD_MODE="Release"
DOCKER_IMAGE_TAG="latest"
readonly BUILD_MODE
readonly DOCKER_IMAGE_TAG
publish_and_build_image()
{
PROJECT_DIR=$1
PROJECT_FILE_NAME=$2
DOCKER_IMAGE_NAME=$3
echo "Publishing project $PROJECT_FILE_NAME ..."
echo
dotnet publish "..\\$PROJECT_DIR\\$PROJECT_FILE_NAME" --configuration $BUILD_MODE --verbosity quiet
echo "Building docker image $DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG ..."
docker build --tag=$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG "../$PROJECT_DIR" --quiet
#echo
#echo -e "\033[32m $DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG built \033[0m"
}
publish_and_build_image "MyProject" "MyProjectName.csproj" "my-docker-img-name"
I have a .NET Core 2.1 console app. I want to run this console app in a Docker image. I'm new to Docker and just trying to figure it out.
At this time, I have a Dockerfile, which was inspired from Microsoft's Example. My file actually looks like this:
FROM microsoft/dotnet:2.1-runtime-nanoserver-1709 AS base
WORKDIR /app
FROM microsoft/dotnet:2.1-sdk-nanoserver-1709 AS build
WORKDIR /src
COPY MyConsoleApp/MyConsoleApp.csproj MyConsoleApp/
RUN dotnet restore MyConsoleApp/MyConsoleApp.csproj
COPY . .
WORKDIR /src/MyConsoleApp
RUN dotnet build MyConsoleApp.csproj -c Release -o /app
FROM build AS publish
RUN dotnet publish MyConsoleApp.csproj -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "MyConsoleApp.exe"]
My question is, what is the difference between: microsoft/dotnet:2.1-runtime-nanoserver-1709, microsoft/dotnet:2.1-sdk, and microsoft/dotnet:2.1-runtime? Which one should I use for my .NET 2.1 console app? I'm confused as to if I a) build my console app then deploy it to a docker image or b) build a docker image, get the code, and build the console app on the image itself. Any help is appreciated.
For anyone interested in actual Dockerfile reference:
FROM microsoft/dotnet:2.2-sdk AS build
WORKDIR /app
COPY <your app>.csproj .
RUN dotnet restore <your app>.csproj
COPY . .
RUN dotnet publish -c Release -o out
FROM microsoft/dotnet:2.2-aspnetcore-runtime AS runtime
WORKDIR /app
COPY --from=build /app/out ./
ENTRYPOINT ["dotnet", "<your app>.dll"]
It's a question more not about docker itself, but .net docker images
You can go to the https://hub.docker.com/r/microsoft/dotnet/ click in particular image find Dockerfile and understand what exactly software builds in this image and find the difference between different Dockerfile's
Regarding second question better to "build a docker image, get the code, and build the console app on the image itself." So you build the image, expose port, mount your code as volume, and your code executes inside the container with help of software installed in the container
microsoft/dotnet:2.1-runtime-nanoserver-1709 - This image is used to setup .net runtime, under which any .net app runs.
microsoft/dotnet:2.1-sdk-nanoserver-1709 - This image is of SDK used to compile your .net core app
Refer this blog for how to create & run .net core console app in docker:
My answer for the first question is:
microsoft/dotnet:2.1-runtime-nanoserver-1709 is a docker image you can run .net core apps,
And microsoft/dotnet:2.1-sdk-nanoserver-1709 is a docker image you build apps , you can also run app with it.
And my answer for the second question is:
If you just want to run app in docker, build your app with visual studio on you local machine (this will be the most easy way to build it), use the microsoft/dotnet:2.1-runtime-nanoserver-1709 , build a docker image.
I put an example on Github on how to do this for a .net core webapi application, so should be pretty much the same for a console application.
My docker file looks like:
https://github.com/MikeyFriedChicken/MikeyFriedChicken.DockerWebAPI.Example
This is doing pretty much the same as yours.
Just to clarify previous answers what your dockerfile is doing is creating 'temporary' images which have the ability to build your software. Once it is all built these images just get scrapped.
If you look at your last lines in your dockerfile you can see it copies the output from the build images to your final image which is based on 'microsoft/dotnet:2.1-runtime-nanoserver-1709' which has just enough libraries and dependencies for running your .net core application.