Dockerfile copy folder to restore local nuget packages - docker

I would like to start a .NET Core WebApi in a docker container from a Unit test project. These tests will be ran during continuous integration using GitLab.
The WebApi project uses local nuget packages stored in a folder with path C:/Temp/nuget packages. This folder exists on the server which run the GitLab runner.
C:/Temp/nuget packages
C:/Users/myUser/project/MyProject with the following tree
- sln
- MyProjectWebAPI
-- dockerfile
- MyProject.Tests
-- link to the MyProjectWebAPI's dockerfile\
My dockerfile looks like this
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
#copying some csproj files
RUN mkdir Oncolin.DataAccess.WebAPI/nugetPackages
#VOLUME "C:/Temp/nuget packages":"Oncolin.DataAccess.WebAPI/nugetPackages" -> failed, unable to name a volume in a dockerfile so my folder is empty and the dotnet restore failed
#COPY ["C:/Temp/nuget packages", "Oncolin.DataAccess.WebAPI/nugetPackages"]-> failed, can't reach Temp folder (using ../.. and/or finished with '/' gave the same result)
#ADD["C:/Temp/nuget packages", "Oncolin.DataAccess.WebAPI/nugetPackages"]-> failed, can't reach Temp folder (using ../.. and/or finished with '/' gave the same result)
#RUN cp -r "C:/Temp/nuget packages" Oncolin.DataAccess.WebAPI/nugetPackages -> failed, can't reach Temp folder
RUN find Oncolin.DataAccess.WebAPI/nugetPackages
RUN dotnet restore "Oncolin.DataAccess.WebAPI/Oncolin.DataAccess.WebAPI.csproj" -s https://api.nuget.org/v3/index.json -s nugetPackages --verbosity n
COPY . .
WORKDIR "/src/Oncolin.DataAccess.WebAPI"
RUN dotnet build "Oncolin.DataAccess.WebAPI.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Oncolin.DataAccess.WebAPI.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Oncolin.DataAccess.WebAPI.dll"]
I have to copy C:/Temp/nuget packages in my container during the build to successfully restore nuget packages. I try several things as you can see in the dockerfile.
I run the docker build command like this
C:\Users\myUser\project\MyProject> docker build -t 'oncolin-webapi' --progress plain -f '.\Oncolin.DataAccess.WebAPI\Dockerfile' .
How can I achieve this ?
Is it possible to retrieve the name of the volume ?
Thanks in advance

I found a workaround to copy my local nuget packages :
access to the nuget packages folder of the current user (i.e. c:\users\myUser.nuget\packages) and copy my known local nuget packages (all started by "rnc" by convention) in a new folder in my solution
var localPackagesDirPath = Path.Combine(solutionDirPath, "localNugetPackages");
var localPackagesDir = Directory.CreateDirectory(localPackagesDirPath);
var directories = Directory.GetDirectories(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), ".nuget", "packages"), "rnc*");
foreach (var item in directories)
{
var rncDir = new DirectoryInfo(item);
var srcRncDir = localPackagesDir.CreateSubdirectory(rncDir.Name);
CopyAll(rncDir, srcRncDir);
}
copy it during build of the image
use the local nuget packages folder to restore them
COPY ["localNugetPackages/", "Oncolin.DataAccess.WebAPI/localNugetPackages/"]
RUN dotnet restore "Oncolin.DataAccess.WebAPI/Oncolin.DataAccess.WebAPI.csproj" -s https://api.nuget.org/v3/index.json -s localNugetPackages --verbosity n
It's clearly not pretty but it's the only solution I found to achieve my goal. Luckily I don't have a lot of local nuget packages.

Related

dotnet restore not finding folder while running dockerfile

This is a very simple API I've created with .NET 6 and ASP.NET Core that I'm trying to dockerize.
I'm always getting the same error while running docker build command.
this is my docker file:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore "MyAPI/MyAPI.csproj" //this is where the build stops
# Copy everything else and build
COPY .. ./
RUN dotnet publish -c Release -o out
# Build runtime image
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
EXPOSE 80
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "MyAPI.dll"]
this is the error I'm getting :
=> ERROR [build-env 4/6] RUN dotnet restore 0.7s
------
> [build-env 4/6] RUN dotnet restore:
#11 0.676 MSBUILD : error MSB1003: Specify a project or solution file. The current working directory does not contain a project or solution file.
------
executor failed running [/bin/sh -c dotnet restore]: exit code: 1
I guess the problem is a path problem and I've tried my ways to pass in the path to my folder yet it kept giving me the same error.
If I try to run the dotnet restore command individually in the terminal, it works which tells me that the path is correct.
What could be the problem?
COPY *.csproj ./ doesn't find anything to copy, since your .csproj file is in the MyAPI/ directory.
Change the statement to
COPY MyAPI/MyAPI.csproj MyAPI/
Where you copy the rest of the files and publish the project should also be changed to
COPY . ./
RUN dotnet publish -c Release -o out MyAPI/MyAPI.csproj
Please check if the csproj file really exists in your PC file system on the path yourSolutionRootFolder/MyAPI/MyAPI.csproj
You may have a typo in the path? Upper/lower case typo?

Dockerfile "COPY" command confusion

I have 3 .NET6 projects in my local 'src' folder as below:
src/A
src/B
src/C
A: web api project, has many nuget dependencies
B: class lib, dependent of A
C: class lib, dependent of A
(Note: of course, project related files including .csproj are present in the local project folders)
My Dockerfile is in project folder A.
I am running the "docker build" command from 'src' folder (i.e. this is the build context) as below:
docker build -f A/Dockerfile -t myapiimage:v1 .
below section of code is from my Dockerfile, where i try to restore nuget dependencies:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ./A/*.csproj A/
RUN dotnet restore A/A.csproj
this works fine and i understand that .csproj in the image is stored inside /src/A/A.csproj
Am i right?
I want to know whether COPY preserves the source folder structure in the image being built or not?
because after the below 3 lines:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY *.csproj ./
None of the line works:
RUN dotnet restore A/A.csproj
OR,
RUN dotnet restore A.csproj
OR,
RUN dotnet restore *A.csproj
OR,
RUN dotnet restore src/A/A.csproj
So, where this .csproj files (of all 3 projects) are getting stored in the image?
Your line COPY *.csproj ./ doesn't copy anything, because there are no .csproj files in your src directory on the host machine.
The way I usually do it is a bit clunky. First I do
COPY */*.csproj ./
which copies all the .csproj files into the workdir in the image. Then I do
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*}/ && mv $file ${file%.*}/; done
which looks at all the .csproj files and creates a directory for them and moves them into that directory.
With that approach, your Dockerfile would look like this
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY */*.csproj ./
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*}/ && mv $file ${file%.*}/; done
RUN dotnet restore A/A.csproj

error NETSDK1064: Package DnsClient, 1.2.0 was not found

I have an Asp.Net core docker image. The last time I tried building it was 2 months ago. Now, I'm getting an error while trying to build it.
Any ideas? Did something break the Microsoft docker image? This is also breaking when trying to publish and run on an Elasticbeanstalk instance.
Complete Error Log:
/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1064: Package DnsClient, version 1.2.0 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions. [/app/BacktraderDataApi.csproj]
The command '/bin/sh -c dotnet publish -c Release -o out' returned a non-zero code: 1
Dockerfile:
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o out
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build-env /app/out .
EXPOSE 5000
ENTRYPOINT ["dotnet", "myApp.dll"]
When solutions like using --no-cache are not restoring at all work, it usually indicates that artifact from the host machine are getting copied into the build container. This can cause problems when the host machine and build container have different RIDs. This can commonly occur on Windows machines when running linux containers. The .NET Core Docker Samples suggest using a .dockerignore file to avoid coping the bin and obj directories with your Dockerfiles.
For me it helped adding a --no-cache flag
RUN dotnet publish -c Release -o out --no-cache
Based on the information posted at https://github.com/microsoft/containerregistry/issues/20 for this same issue as well as the information here I tried the following:
dotnet new web
Edit the csproj and added the following:
<ItemGroup>
<PackageReference Include="MongoDB.Driver" Version="2.10.1" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
</ItemGroup>
Created the following Dockerfile
# https://hub.docker.com/_/microsoft-dotnet-core
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build /app ./
ENTRYPOINT ["dotnet", "aspnetapp.dll"]
Created the following .dockerignore
# directories
**/bin/
**/obj/
**/out/
# files
Dockerfile*
**/*.trx
**/*.md
**/*.ps1
**/*.cmd
**/*.sh
docker build --pull -t test .
The Dockerfile built without issue. Can you try these steps and report back the results?
I had a very similar problem
error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder '/usr/local/share/dotnet/sdk/NuGetFallbackFolder'.
I added RUN mkdir -p /usr/local/share/dotnet/sdk/NuGetFallbackFolder.
Then it got a lot farther but failed with a library resolution for something that I believe is just distributed with the SDK.
/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(234,5): error NETSDK1064: Package Microsoft.EntityFrameworkCore.Analyzers, version 3.1.1 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions.
I think the mcr.microsoft.com/dotnet/core/sdk:3.1 4/09/2010 Docker image with tags 3.1.201-buster, 3.1-buster, 3.1.201, 3.1, latest is broken.
I changed to the image based on Ubuntu bionic and it just worked again.
USE mcr.microsoft.com/dotnet/core/sdk:3.1-bionic
I was having the same issue, after i read this link https://github.com/dotnet/sdk/issues/3059
I just appended the /restore to build command and it worked.
just like this
RUN dotnet publish --no-restore -c Release -o /app --no-cache /restore
and now it is working just fine.

"An assembly specified in the application dependencies manifest was not found" using docker

I am trying to dockerize an an angular aspnet core 2.2 webapi using the following dockerfile:
FROM mcr.microsoft.com/dotnet/core/sdk:2.2.105 AS build
WORKDIR /src
# Copy csproj and restore as distinct layers
COPY ["Fightplan_v1/Fightplan_v1.csproj", "Fightplan_v1/"]
COPY ["Fightplan_v1.Autogenerate/Fightplan_v1.Autogenerate.csproj", "Fightplan_v1.Autogenerate/"]
COPY ["Fightplan_v1.Autogenerate.Test/Fightplan_v1.Autogenerate.Test.csproj", "Fightplan_v1.Autogenerate.Test/"]
COPY ["Fightplan_v1.Database/Fightplan_v1.Database.csproj", "Fightplan_v1.Database/"]
COPY ["Fightplan_v1.Helpers/Fightplan_v1.Helpers.csproj", "Fightplan_v1.Helpers/"]
COPY ["Fightplan_v1.Jobs/ConvertImagestoBlob/Fightplan_v1.ConvertImagestoBlob.csproj", "Fightplan_v1.Jobs/ConvertImagestoBlob/"]
COPY ["Fightplan_v1.Models/Fightplan_v1.Models.csproj", "Fightplan_v1.Models/"]
COPY ["Fightplan_v1.Shared/Fightplan_v1.Shared.csproj", "Fightplan_v1.Shared/"]
RUN dotnet restore "Fightplan_v1/Fightplan_v1.csproj"
# Copy everything else and build
COPY . .
WORKDIR "/src/Fightplan_v1"
RUN dotnet build "Fightplan_v1.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "Fightplan_v1.csproj" -c Release -o /app
# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2 AS runtime
WORKDIR /app
COPY --from=build /app .
EXPOSE 80
ENTRYPOINT ["dotnet", "Fightplan_v1.dll"]
The build command used is:
docker build -f .\fp.dockerfile -t test .
The build goes through fine but when I try to run it using:
docker run -p 5100:80 -it test
I get the following error:
I have tried:
Adding -r linux-x64 to the end of publish to define the runtime: RUN dotnet publish "Fightplan_v1.csproj" -c Release -o /app -r linux-x64
Adding <PublishWithAspNetCoreTargetManifest>false</PublishWithAspNetCoreTargetManifest> to my .csproj file
None of the above fixes work unfortunately.
Since this is combined poject of an angular application and a webapi I might be missing some installations/dependencies?
You are copying the runtime files from wrong layer
COPY --from=build /app .
Try to change it to
COPY --from=publish /app .
Also, you probably need to separate out path in build and publish steps:
RUN dotnet build "Fightplan_v1.csproj" -c Release -o /app/build
RUN dotnet publish "Fightplan_v1.csproj" -c Release -o /app/publish
and then copy runtime files from the publish folder
COPY --from=publish /app/publish .
Edit. Basically, you need to copy publish artifacts into runtime, not build. Build will produce deps.json, which will lists all external dependencies, and during runtime this libraries are resolving using NuGet cache, if it is empty you will see an error. To copy dependencies along with project files you need to run publish.
So I assume, that theoretically dotnet build can be omitted, and is using only to discover compilation errors on early stage of docker build.

Exception "error MSB3024: Could not copy the file..." is thrown when attempting to build in DevOps pipeline using .Net Core 3.0 SDK (preview5)

I am attempting to build a .Net Core 3.0 (preview) project in a DevOps build pipeline.
The steps in my azure-pipelines.yml executes up to the "docker build" step, successfully initiating the build process.
The Docker file is read and executed up to the "dotnet build" step after which the following error is thrown.
error MSB3024: Could not copy the file "/src/obj/Release/netcoreapp3.0/" to the destination file "/app/", because the destination is a folder instead of a file. To copy the source file into a folder, consider using the DestinationFolder parameter instead of DestinationFiles. [/src/.csproj]
I have attempted to build locally by executing dotnet build ".csproj" -c Release -o /app
The build then succeeds with 0 errors and 0 warnings. Could this be related to an SDK issue in DevOps?
Any advice would be greatly appreciated.
The build command from my Docker file.
RUN dotnet build "<project>.csproj" -c Release -o /app
--- {OMITED}
---1ff83de4bdba
Step 5/16 : WORKDIR /src
---Running in 972629766fad
Removing intermediate container 972629766fad
---1325ecd8e7c3
Step 6/16 : COPY ["<projectname>.csproj", "<projectname>/"]
---a4ba463683dc
Step 7/16 : RUN dotnet restore "<projectname>/<projectname>.csproj"
---Running in 82bb9095d412
Restore completed in 14.08 sec for /src/<projectname>/<projectname>.csproj.
Removing intermediate container 82bb9095d412
---7b0cc236782e
Step 8/16 : COPY . .
---884bb695bfb3
Step 9/16 : WORKDIR /src
---Running in 817b001e2060
Removing intermediate container 817b001e2060
---40b8690ecb63
Step 10/16 : RUN dotnet build "<projectname>.csproj" -c Release -o /app
---Running in 48d79b81c3cb
Microsoft (R) Build Engine version 16.0.462+g62fb89029d for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
Restore completed in 531.02 ms for /src/<projectname>.csproj.
/usr/share/dotnet/sdk/3.0.100-preview5-011568/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(157,5): message NETSDK1057: You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview [/src/<projectname>.csproj]
/usr/share/dotnet/sdk/3.0.100-preview5-011568/Microsoft.Common.CurrentVersion.targets(4560,5): error MSB3024: Could not copy the file "/src/obj/Release/netcoreapp3.0/<projectname>" to the destination file "/app/<projectname>", because the destination is a folder instead of a file. To copy the source file into a folder, consider using the DestinationFolder parameter instead of DestinationFiles. [/src/<projectname>.csproj]
Build FAILED.
/usr/share/dotnet/sdk/3.0.100-preview5-011568/Microsoft.Common.CurrentVersion.targets(4560,5): **error MSB3024: Could not copy the file "/src/obj/Release/netcoreapp3.0/<projectname>" to the destination file "/app/<projectname>", because the destination is a folder instead of a file. To copy the source file into a folder, consider using the DestinationFolder parameter instead of DestinationFiles. **[/src/<projectname>.csproj]
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:11.43
The command '/bin/sh -c dotnet build "<projectname>.csproj" -c Release -o /app' returned a non-zero code: 1
##[error]Bash exited with code '1'.
##[section]Finishing: docker build
In my case the Assembly name of my project was the same as the name of the project and also the same as the directory name of that project. Once I changed the assembly name in the project properties to something else, this error was gone.
in my case i had to rename the project folder to be different than the project itself in dockerfile:
FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build
WORKDIR /src
COPY ["<project-name>.csproj", "<project-name>_build/"]
RUN dotnet restore "<project-name>_build/<project-name>.csproj"
I am assuming this is an issue with the .Net Core 3.0 preview5 SDK. I have created the project in .Net Core 2.2 and it build successfully.
When adding the following parameters on the dotnet build command, the build succeeds in .Net Core 3.0
RUN dotnet build ".csproj" -c Release -r linux-musl-x64 -o /app --no-restore
The DockerFile below built successfully in DevOps
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build
WORKDIR /src
COPY ["<project-name>.csproj", "<project-name>/"]
RUN dotnet restore "<project-name>/<project-name>.csproj" -r linux-musl-x64
COPY . "<project-name>/"
WORKDIR "/src/<project-name>"
RUN dotnet build "<project-name>.csproj" -c Release -r linux-musl-x64 -o /app --no-restore
FROM build AS publish
RUN dotnet publish "<project-name>.csproj" -c Release -r linux-musl-x64 -o /app --no-restore
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "<project-name>.dll"]

Resources