I am new to docker and I developed a simple asp.net core webapi solution which uses a NuGet package from a private repository.
dotnet restore command returns an error for a missing NuGet package that is located inside a private repository which introduced in the nuget.config file inside the solution.
does anyone knows what's the problem with my configs and dockerfile ?
Dockerfile
FROM microsoft/aspnetcore-build:2.0 AS build-env
WORKDIR /app
# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out
# build runtime image
FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "aspnetcoretssl.dll"]
.dockerignore
bin\
obj\
Also I have a nuget.config file in the solution's root directory as below
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="myrepo" value="\\path\to\the\nuget_packages_folder" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
But I receive below messages from
docker build -t aspnetcoretssl .
error NU1101: Unable to find package TestContracts. No packages exist
with this id in source(s): nuget.org Generating MSBuild file
/app/obj/aspnetcoretssl.csproj.nuget.g.props.
The command '/bin/sh -c dotnet restore' returned a non-zero code: 1
It looks like there are two problems
The NuGet.config file is in your solution directory, but the Dockerfile is in the project folder. This means COPY . ./ won't copy NuGet.config into the Docker container
Even if you had the NuGet.config file, the "myrepo" source is an invalid filepath inside a Docker container.
This is invalid because this is a Windows network file path, but your container is running on Linux.
To solve this, I would recommend the following.
Either move your Dockerfile to the solution directory, or move NuGet.config into the project directory.
Add a second file named NuGet.linux.config, and point to this file when building on non-windows. Use the RestoreConfigFile property to point to this file when building on Linux. If you have moved NuGet.config into the project directory, adding these lines to your aspnetcoretssl.csproj files would work:
<PropertyGroup>
<RestoreConfigFile Condition="'$(OS)' != 'Windows_NT'">NuGet.linux.config</RestoreConfigFile>
<RestoreConfigFile Condition="'$(OS)' == 'Windows_NT'">NuGet.config</RestoreConfigFile>
</PropertyGroup>
Creating a network mount from \\path\to\the\nuget_packages_folder to Z:\
In NuGet.linux.config, change "myrepo" to <add key="myrepo" value="/nuget/myrepo" />
Mounting this drive into the container. This is done with the --volume parameter on docker-run. docker run --volume Z:/:/nuget/myrepo
Related
I have a .NET 6 Console project that includes 2 binaries as linked files:
<ItemGroup>
<None Include="..\HFMath\bin\HFMath.dll" Link="HFMath.dll">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Include="..\HFMath\bin\libHFMath.so" Link="libHFMath.so">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>
I need to deploy it as a docker container. The Dockerfile scaffolded by Visual Studio is
FROM mcr.microsoft.com/dotnet/runtime:6.0 AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["ConsoleApp1/ConsoleApp1.csproj", "ConsoleApp1/"]
RUN dotnet restore "ConsoleApp1/ConsoleApp1.csproj"
COPY . .
WORKDIR "/src/ConsoleApp1"
RUN dotnet build "ConsoleApp1.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "ConsoleApp1.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ConsoleApp1.dll"]
when I run docker build, I get the following error:
#15 1.054 Determining projects to restore...
#15 1.322 All projects are up-to-date for restore.
#15 2.552 /usr/share/dotnet/sdk/6.0.100/Microsoft.Common.CurrentVersion.targets(5100,5): error MSB3030: Could not copy the file "/src/HFMath/bin/libHFMath.so" because it was not found. [/src/ConsoleApp1/ConsoleApp1.csproj]
#15 2.552 /usr/share/dotnet/sdk/6.0.100/Microsoft.Common.CurrentVersion.targets(5100,5): error MSB3030: Could not copy the file "/src/HFMath/bin/HFMath.dll" because it was not found. [/src/ConsoleApp1/ConsoleApp1.csproj]
#15 2.556
#15 2.556 Build FAILED.
I know that docker doesn't support symlinks, but I still need to build the image.
How do I fix this (without changing the .NET project)?
Problem
If you have auto generated the Dockerfile using Visual Studio, it will also have created a .dockerignore file (usually in the .sln folder). That file will contain an entry to exclude **/bin from the build context.
This means when the COPY . . executes, it is ignoring the files in any bin folder
Solution
Find and edit the .dockerignore file.
Add the line !HFMath/bin/ - this will tell Docker to NOT ignore the HFMath/bin folder
Run your docker build command again
I have got a problem while creating docker image on my .Net core web api application. Thats the docker image generated by VS docker support:
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["src/appname.Api/appname.Api.csproj", "src/appname.Api/"]
COPY ["src/appname.Data/appname.Data.csproj", "src/appname.Data/"]
COPY ["src/Eappname.Models/appname.Models.csproj", "src/appname.Models/"]
RUN dotnet restore "src/appname.Api/appname.Api.csproj"
COPY . .
WORKDIR "/src/src/appname.Api"
RUN dotnet build "appname.Api.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "appname.Api.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "appname.Api.dll"]
Unfortunettly I use Azure Artifacts Feed package inside my app and this image does not work. I made a little research and try to use credential provider with nuget config file. Just like below:
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY NuGet.Config ./
COPY ["src/appname.Api/appname.Api.csproj", "src/appname.Api/"]
COPY ["src/appname.Data/appname.Data.csproj", "src/appname.Data/"]
COPY ["src/appname.Models/appname.Models.csproj", "src/appname.Models/"]
RUN dotnet restore --configfile NuGet.Config "src/appname.Api/appname.Api.csproj"
COPY . .
WORKDIR "/src/src/appname.Api"
RUN dotnet build "appname.Api.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "appname.Api.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "appname.Api.dll"]
and also added Nuget.Config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="BredasVSTS" value="https://pkgs.dev.azure.com/feed/nuget/v3/index.json" />
</packageSources>
<packageSourceCredentials>
<BredasVSTS>
<add key="Username" value="<usernameWithEmail>" />
<add key="ClearTextPassword" value="<token>" />
</BredasVSTS>
</packageSourceCredentials>
</configuration>
Now I made a little progress because I am able to pass step 11 - dotnet restore. Unfortunettly in step 14- dotnet build I get the following error:
/bin/sh: 2: cd: can't cd to *Undefined*
Any ideas how can I solve this problem?
error: /bin/sh: 2: cd: can't cd to Undefined
According to the error MSB3073: The command "cd "undefined"", it seems you are using the command line or build event cd $(SolutionDir) in your xxx.Api project.
When you build the project file .csproj without solution file .sln with dockerfile, it could not get the value of $(SolutionDir), which is based on the .sln file.
So, you will get the error "cd "undefined", to resolve this issue you can try to replace all $(SolutionDir) with $(ProjectDir)..\ in your project file.
As test, I create a sample with build event:
CD $(SolutionDir)
Then I got the same error:
To resolve this issue, I have replaced $(SolutionDir) with $(ProjectDir)..\, then build it, it works fine:
Check the this thread for some more details.
Hope this helps.
I am really curious to how to interpret and debug with the following error:-
C:\users\project>docker-compose build
Step 6/15 : COPY *.csproj ./
ERROR: Service 'invoiceservice' failed to build: COPY failed: no source files were specified
This is particular micro-service as i have few more such services.
docker file :-
FROM mcr.microsoft.com/dotnet/core/runtime:2.2 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build
WORKDIR /src
COPY *.csproj ./
RUN dotnet restore -s https://api.nuget.org/v3/index.json -s https://www.myget.org/F/autoweb/api/v3/index.json
COPY . .
WORKDIR /src/src/InvoiceManagement/InvoiceService
RUN dotnet publish -c Release -o out
FROM build AS publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/out .
ENTRYPOINT ["dotnet", "InvoiceService.dll"]
Interesting part is when I build this from Visual Studio IDE, its being built fine but it does not build on CLI.
docker compose file:-
invoiceservice:
image: ${DOCKER_REGISTRY-}invoiceservice
build:
context: .
dockerfile: src/InvoiceManagement/InvoiceService/Dockerfile
I don't understand why CLI could not find the source location and copy where as VS works fine. Any clue???
It's likely an issue with your Docker context. Commands in Dockerfiles are relative to where the docker/docker-compose CLI tool is run, not relative to the Dockerfile location.
In other words, if you run the command from the solution root, then your csproj file is actually under ./src/InvoiceManagement/InvoiceService. As a result, *.csproj finds no files, because there's no project files literally in your solution root.
I tried replicating your problem and I was able to copy all the files successfully (instead of .csproj, I used .txt). The problem occurred when there was no txt file to copy. My COPY command failed with exactly the same message as yours.
Now, why is VS able to build the image successfully? That is because VS build the project first! When the project's build procedure is completed, a .csproj file is generated and that gets copied to the image.
To confirm the results, ls your current directory (when the build fails from command line) and see if there is any .csproj file in that directory.
The setup
In my solution folder, I have this file/folder-structure
Source
docker-compose.yml
.dockerignore
paths.json
Webproject
//All web-project files (including DockerFile)
AnotherWebproject
//All web-project files (including DockerFile)
In my Visual Studio project, I've added the paths.json as a linked file and to include it in my image, I've set my csproj file (don't know if this is necessary...)
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
<DockerDefaultTargetOS>Windows</DockerDefaultTargetOS>
<UserSecretsId>8bcf6be2-9d06-4667-89f6-38f25af5dfbc</UserSecretsId>
<DockerComposeProjectPath>..\docker-compose.dcproj</DockerComposeProjectPath>
</PropertyGroup>
<ItemGroup>
<Content Include="..\paths.json" Link="paths.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<CopyToPublishDirectory>Always</CopyToPublishDirectory>
</Content>
</ItemGroup>
...
</Project>
And my Dockerfile to:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-nanoserver-1803 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.0-nanoserver-1803 AS base
WORKDIR /src
COPY ["Webproject/Webproject.csproj", "Webproject/"]
COPY "paths.json" "paths.json"
RUN dotnet restore "Webproject/Webproject.csproj"
COPY . .
WORKDIR "/src/Webproject"
RUN dotnet build "Webproject.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "Webproject.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Webproject.dll"]
To use the file in my application, I use:
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((context, config) =>
{
var environment = context.HostingEnvironment;
var folder = Path.Combine(environment.ContentRootPath);
config.AddJsonFile(Path.Combine(folder, "paths.json"), false);
})
.UseStartup<Startup>();
I even tried to exclude it in my .dockerignore file:
!paths.json
FYI... I'm using also a docker-compose project
The error
When starting up the application, I have this error:
System.IO.FileNotFoundException: 'The configuration file 'paths.json' was not found and is not optional. The physical path is 'C:\app\paths.json'.'
The question
I tried a lot of different paths in the Dockerfile to include it, but it doesn't seem to work. Can anyone help me with setting the correct path?
Also, I searched if it is possible to list all the files (to check if it's copied to the correct path), but it doesn't seem possible?
Not sure your full dockerfile content and dockerfile path, try following steps below:
Folder Structure
Source
Dockerfile
paths.json
Webproject
Dockerfile
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["DockerFileCopy/DockerFileCopy.csproj", "DockerFileCopy/"]
RUN dotnet restore "DockerFileCopy/DockerFileCopy.csproj"
COPY . .
WORKDIR "/src/DockerFileCopy"
RUN dotnet build "DockerFileCopy.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "DockerFileCopy.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "DockerFileCopy.dll"]
There is no need to copy the paths.json in docker command, the RUN dotnet build and RUN dotnet publish will not copy this file to the published folder
We use link file to copy the paths.json, the .csproj content
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
<UserSecretsId>2594cc66-3d1a-487b-a93c-99e0cf9975ef</UserSecretsId>
<DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
<DockerfileContext>.</DockerfileContext>
</PropertyGroup>
<ItemGroup>
<Content Include="..\paths.json" Link="paths.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<CopyToPublishDirectory>Always</CopyToPublishDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="2.2.0" PrivateAssets="All" />
<PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.7.9" />
</ItemGroup>
</Project>
Build the image in the Source folder by command like docker build -t dockerfilecopy .
This question already has answers here:
Where is NuGet.Config file located in Visual Studio project?
(5 answers)
Closed 1 year ago.
I'm trying to make use of local nuget package for my dotnet restore, I tried to follow this tutorial: dotnet restore w/out internet
My problem:
It doesn't see the path even though it exist on that path..
The server I'm using is on a Corporate Network that is why I can't use dotnet restore, so I'm also experiencing the problem with nuget.org similar to this link.
Environment:
For the sample project, I used:
the basic .Net Core web app from Visual Studio 2017
Docker Enterprise Edition(no UI), Windows container
Windows Server 2016 as OS.
UPDATE 10/15/2018
While the answer of #omajid has been very helpful, I believe docker volume mount is only possible when using docker run and can't be used in Dockerfile(which will be used for Build Pipeline). Got this link which is similar to what I want to achieve. How to mount a host directory in a Docker container
To have all packages ready you need restore before building.
To have all packages during the build you need to copy the packages.
Here is an example in form of an experiment:
Preparation:
Have the sdk ready: docker pull microsoft/dotnet:2.2-sdk.
Have src/src.csproj ready:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="12.0.2" />
</ItemGroup>
</Project>
Have src/Dockerfile ready:
FROM microsoft/dotnet:2.2-sdk AS byse
COPY packages /root/.nuget/packages
COPY src src
RUN ls /root/.nuget/packages
WORKDIR /src
RUN dotnet restore
RUN ls /root/.nuget/packages
Execution:
Restore the Packages:
docker run --rm -v $(pwd)/src:/src -v $(pwd)/packages:/root/.nuget/packages -w /src microsoft/dotnet:2.2-sdk dotnet restore
Build the Image:
docker build -t test -f src/Dockerfile .
Expectation:
Sending build context to Docker daemon 13.77MB
Step 1/7 : FROM microsoft/dotnet:2.2-sdk AS byse
---> e4747ec2aaff
Step 2/7 : COPY packages /root/.nuget/packages
---> 76c3e9869bb4
Step 3/7 : COPY src src
---> f0d3f8d9af0a
Step 4/7 : RUN ls /root/.nuget/packages
---> Running in 8323a9ba8cc6
newtonsoft.json
Removing intermediate container 8323a9ba8cc6
---> d90056004474
Step 5/7 : WORKDIR /src
---> Running in f879d52f81a7
Removing intermediate container f879d52f81a7
---> 4020c789c338
Step 6/7 : RUN dotnet restore
---> Running in ab62a031ce8a
Restore completed in 44.28 ms for /src/src.csproj.
Removing intermediate container ab62a031ce8a
---> 2cd0c01fc25d
Step 7/7 : RUN ls /root/.nuget/packages
---> Running in 1ab3310e2f4c
newtonsoft.json
Removing intermediate container 1ab3310e2f4c
---> 977e59f0eb10
Successfully built 977e59f0eb10
Successfully tagged test:latest
Note that the ls steps are cached and would not print on a subsequent call. Run docker rmi test to reset.
Step 4/7 runs before the restore and the packages are already cached.
Step 4/7 : RUN ls /root/.nuget/packages
---> Running in 8323a9ba8cc6
newtonsoft.json
This can solves excessive restore times for example during automated builds.
To solve your network issue you can try to mount the network patch instead of the local path during the resolve step or robocopy files from your corp network into a local cache first.
The entire selling point of docker and container technologies is isolation. So within a docker container, your user disk is not visible. If it was, there would be much less of an isolation. You need to mount your local nuget directory inside the container to be able to access it. For detailed steps, see https://rominirani.com/docker-on-windows-mounting-host-directories-d96f3f056a2c.
In particular:
Share your C: drive
Within your Dockerfile, have a dotnet restore --source /packages
Use a volume mount to mount your local packages into /packages inside the container: docker build -t webapp4 . -v c:/users/cnaling/.nuget/packages:/packages
One simpler alternative is to cache the NuGet packages in a separate layer by having them being restored from the csproj project file before copying the source files. Example:
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env
WORKDIR /app
# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out
This solution should speed up local development which typically requires quite a few stop -> build -> up cycles.
The main downside is that changing csproj triggers the layer being rebuilt, but this should happen less frequently than rebuilding of the container.
I am a little late to the game but I believe I found a simple solution to this problem...
Create a "NuGet.Config" file in the same directory as your .sln
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="{{CUSTOM NAME}}" value="{{CUSTOM SOURCE}}" />
</packageSources>
<packageRestore>
<add key="enabled" value="True" />
<add key="automatic" value="True" />
</packageRestore>
<bindingRedirects>
<add key="skip" value="False" />
</bindingRedirects>
<packageManagement>
<add key="format" value="0" />
<add key="disabled" value="False" />
</packageManagement>
</configuration>
That is it! Create your "Dockerfile" here as well
Run docker build with your Dockerfile and all will get resolved