Azure DevOps private nuget repo Unauthorized when building with docker - docker

I am trying to create a docker image for my project that has a private Azure DevOps nuget repo. I am able to connect to the repo in visual studio with my credentials but when I try to build a docker image I get an error.
Unable to load the service index for source
https://{my_url}/nuget/v3/index.json Response status code does not
indicate success: 401 (Unauthorized)
I copied my Nuget.config file to the root of my project
nuget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<disabledPackageSources>
<add key="Microsoft and .NET" value="true" />
</disabledPackageSources>
<activePackageSource>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</activePackageSource>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="libraries" value="https://{my_url}/nuget/v3/index.json" />
</packageSources>
<packageSourceCredentials>
<libraries>
<add key="Username" value="{my_username}" />
<add key="ClearTextPassword" value="{my_pass}" />
</libraries>
</packageSourceCredentials>
</configuration>
and this is my dockerfile
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0.0-preview8-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview8-nanoserver-1903 AS build
WORKDIR /src
COPY NuGet.config "MyProject/"
COPY ["MyProject/MyProject.csproj", "MyProject/"]
COPY . .
RUN dotnet restore "MyProject/MyProject.csproj"
WORKDIR "/src/MyProject"
RUN dotnet build "MyProject.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "MyProject.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "MyProject.dll"]
I am not sure what else to do for it to be able to connect to my repo

I stumbled across this post.
(ref: solution # the bottom)
Basically, change the PAT Organization to "All accessible organizations" AND scroll through the permission\security sections and set the following:
Build (Artifacts, definitions, requests, queue a build, and updated build properties): Read
Connected server (Access endpoints): Connected server
Packaging (Create, read, update, and delete feeds and packages): Read
(Not sure if all of these are required.)
Docker File:
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
# The Personal Access Token arg
ARG PAT
# Set environment variables
ENV NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED true
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS '{"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/ORG/_packaging/FEEDNAME/nuget/v3/index.json","username":"USERNAME","password":"'${PAT}'"}]}'
# Get and install the Artifact Credential provider
RUN wget -O - https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
COPY ["PROJECTNAME.csproj", "./"]
RUN dotnet restore -s "https://pkgs.dev.azure.com/ ORG /_packaging/ FEEDNAME /nuget/v3/index.json" -s "https://api.nuget.org/v3/index.json"
COPY . ./
WORKDIR /src
RUN dotnet build " PROJECTNAME.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish --no-restore " PROJECTNAME.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", " PROJECTNAME.dll"]
docker command:
docker build -t BUILDNAME:local . --build-arg PAT=xxxxxxxxxxxxxxxxxxxxxxxxx
Previously we were using a nuget.config file with credentials\PAT and this was working. I have no idea what changed to cause it to stop. If anyone had an idea pls post.

you need to force dotnet restore to use your config file:
RUN dotnet restore "MyProject/MyProject.csproj" --configfile Nuget.Config
alternatively you can do something like this in your dockerfile:
# Add Environment Variables references for access private Azure Artifacts Repository
# Use --build-arg <ARG> to pass this in.
ARG TOKENPASS
ARG TOKENUSER
ARG NUGET_ENDPOINT
# Fetch Azure Artifacts "Magic" credentials providers
RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
# Needs to be enabled to fetch the private repository credentials for NuGet restore.
ENV NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED true
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"$NUGET_ENDPOINT\", \"username\":\"$TOKENUSER\", \"password\":\"$TOKENPASS\"}]}"
# Workaround of some kind to help Azure Artifact Private repository image collection.
ENV DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER 0
and use docker build with --build-arg TOKENPASS=$(TOKENPASS) --build-arg TOKENUSER=$(TOKENUSER) --build-arg NUGET_ENDPOINT=$(NUGET_ENDPOINT)

Related

linux docker restore stuck

i have some pipelines in azure devops 2020 that were ok but now suddenly doesnt work. i found that docker cant restore and stuck. i clone my project in docker server and check with docker build command. i get errors like azure devops log.
environment:
os: rocky linux 8.6
docker 20.10.18
errors:
Build FAILED
"/src/RLData_API/RLData_API.csproj" (Restore target) (1) ->
(Restore target) ->
/src/RLData_API/RLData_API.csproj : error NU1301: Unable to load the service index for source http://api.nuget.org/v3/index.json.
/src/RLData_API/RLData_API.csproj : error NU1301: Unable to load the service index for source http://api.nuget.org/v3/index.json.
Dockerfile:
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["RLData_API/RLData_API.csproj", "RLData_API/"]
COPY ["RLData_API/nuget.config", ""]
#RUN dotnet restore "RLData_API/RLData_API.csproj"
RUN dotnet restore -v diag "RLData_API/RLData_API.csproj" --configfile nuget.config
COPY . .
WORKDIR "/src/RLData_API"
RUN dotnet build "RLData_API.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "RLData_API.csproj" -c Release -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "RLData_API.dll"]
nuget.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="Feed" value="http://dev.to.com/Iro/IQ/_packaging/Feed/nuget/v3/index.json" />
<add key="nuget.org" value="http://api.nuget.org/v3/index.json" protocolVersion="3" />
</packageSources>
<activePackageSource>
<add key="All" value="(Aggregate source)" />
</activePackageSource>
<packageSourceCredentials>
<Feed>
<add key="Username" value="iro" />
<add key="ClearTextPassword" value="my PAT Token" />
<add key="ValidAuthenticationTypes" value="basic" />
</Feed>
</packageSourceCredentials>
</configuration>
problem solved :)
replace sdk:6.0 with sdk:6.0-alpine

ASP.NET Core build docker image having shared libraries

I'm working in a microservice environemnt where there are shared libraries between these microservices.
These libraries are stored in a different location on the disk, a "shared" location, and are directly referenced by these microservices.
In a .csproj of such a microservice you would see something like this.
<ItemGroup>
<ProjectReference Include="..\..\..\Shared1.csproj" />
<ItemGropu>
Now in each of these microservices I have a dockerfile
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY . .
COPY ../../../Users/boris/Desktop/shared/Shared/Shared/Shared.csproj .
RUN dotnet restore "WebUi.csproj"
WORKDIR /src/.
RUN dotnet build "WebUi.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "WebUi.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
RUN ls
ENTRYPOINT ["dotnet", "WebUi.dll"]
when building a image
docker build -f dockerfile .
I'm getting
ERROR [build 4/7] COPY ..\..\..\Users\boris\Desktop\shared\Shared\Shared\Shared.csproj .
[build 4/7] COPY
......\Users\boris\Desktop\shared\Shared\Shared\Shared.csproj .:
failed to solve with frontend dockerfile.v0: failed to build LLB:
failed to compute cache key:
"/..\..\..\Users\boris\Desktop\shared\Shared\Shared\Shared.csproj"
not found: not found PS C:.demos\hangfire-dashboard\WebUi> docker
build -f .\dockerfile -t web . [+] Building 0.2s (10/18)
I'm thinking it's because of the build context, but not sure.
Any ideas on how can I fix this ?
Ok first thing first rename the dockerfile to Dockerfile. Next up is the path correct in the first place? I assume your csproj exists in the same folder as your Dockerfile and in your csproj you use <ProjectReference Include="..\..\..\Shared1.csproj" /> whereas in the docker file you use ../../../Users/boris/Desktop/shared/Shared/Shared/Shared.csproj so if the csproj AND the Dockerfile exist in the same directory the path is simply incorrect.

How to create docker image with Azure Artifacts feed in .Net core project

I have got a problem while creating docker image on my .Net core web api application. Thats the docker image generated by VS docker support:
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["src/appname.Api/appname.Api.csproj", "src/appname.Api/"]
COPY ["src/appname.Data/appname.Data.csproj", "src/appname.Data/"]
COPY ["src/Eappname.Models/appname.Models.csproj", "src/appname.Models/"]
RUN dotnet restore "src/appname.Api/appname.Api.csproj"
COPY . .
WORKDIR "/src/src/appname.Api"
RUN dotnet build "appname.Api.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "appname.Api.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "appname.Api.dll"]
Unfortunettly I use Azure Artifacts Feed package inside my app and this image does not work. I made a little research and try to use credential provider with nuget config file. Just like below:
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY NuGet.Config ./
COPY ["src/appname.Api/appname.Api.csproj", "src/appname.Api/"]
COPY ["src/appname.Data/appname.Data.csproj", "src/appname.Data/"]
COPY ["src/appname.Models/appname.Models.csproj", "src/appname.Models/"]
RUN dotnet restore --configfile NuGet.Config "src/appname.Api/appname.Api.csproj"
COPY . .
WORKDIR "/src/src/appname.Api"
RUN dotnet build "appname.Api.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "appname.Api.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "appname.Api.dll"]
and also added Nuget.Config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="BredasVSTS" value="https://pkgs.dev.azure.com/feed/nuget/v3/index.json" />
</packageSources>
<packageSourceCredentials>
<BredasVSTS>
<add key="Username" value="<usernameWithEmail>" />
<add key="ClearTextPassword" value="<token>" />
</BredasVSTS>
</packageSourceCredentials>
</configuration>
Now I made a little progress because I am able to pass step 11 - dotnet restore. Unfortunettly in step 14- dotnet build I get the following error:
/bin/sh: 2: cd: can't cd to *Undefined*
Any ideas how can I solve this problem?
error: /bin/sh: 2: cd: can't cd to Undefined
According to the error MSB3073: The command "cd "undefined"", it seems you are using the command line or build event cd $(SolutionDir) in your xxx.Api project.
When you build the project file .csproj without solution file .sln with dockerfile, it could not get the value of $(SolutionDir), which is based on the .sln file.
So, you will get the error "cd "undefined", to resolve this issue you can try to replace all $(SolutionDir) with $(ProjectDir)..\ in your project file.
As test, I create a sample with build event:
CD $(SolutionDir)
Then I got the same error:
To resolve this issue, I have replaced $(SolutionDir) with $(ProjectDir)..\, then build it, it works fine:
Check the this thread for some more details.
Hope this helps.

Copy and use json-config file in Docker ASP.NET Core webapplication

The setup
In my solution folder, I have this file/folder-structure
Source
docker-compose.yml
.dockerignore
paths.json
Webproject
//All web-project files (including DockerFile)
AnotherWebproject
//All web-project files (including DockerFile)
In my Visual Studio project, I've added the paths.json as a linked file and to include it in my image, I've set my csproj file (don't know if this is necessary...)
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
<DockerDefaultTargetOS>Windows</DockerDefaultTargetOS>
<UserSecretsId>8bcf6be2-9d06-4667-89f6-38f25af5dfbc</UserSecretsId>
<DockerComposeProjectPath>..\docker-compose.dcproj</DockerComposeProjectPath>
</PropertyGroup>
<ItemGroup>
<Content Include="..\paths.json" Link="paths.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<CopyToPublishDirectory>Always</CopyToPublishDirectory>
</Content>
</ItemGroup>
...
</Project>
And my Dockerfile to:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-nanoserver-1803 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.0-nanoserver-1803 AS base
WORKDIR /src
COPY ["Webproject/Webproject.csproj", "Webproject/"]
COPY "paths.json" "paths.json"
RUN dotnet restore "Webproject/Webproject.csproj"
COPY . .
WORKDIR "/src/Webproject"
RUN dotnet build "Webproject.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "Webproject.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Webproject.dll"]
To use the file in my application, I use:
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((context, config) =>
{
var environment = context.HostingEnvironment;
var folder = Path.Combine(environment.ContentRootPath);
config.AddJsonFile(Path.Combine(folder, "paths.json"), false);
})
.UseStartup<Startup>();
I even tried to exclude it in my .dockerignore file:
!paths.json
FYI... I'm using also a docker-compose project
The error
When starting up the application, I have this error:
System.IO.FileNotFoundException: 'The configuration file 'paths.json' was not found and is not optional. The physical path is 'C:\app\paths.json'.'
The question
I tried a lot of different paths in the Dockerfile to include it, but it doesn't seem to work. Can anyone help me with setting the correct path?
Also, I searched if it is possible to list all the files (to check if it's copied to the correct path), but it doesn't seem possible?
Not sure your full dockerfile content and dockerfile path, try following steps below:
Folder Structure
Source
Dockerfile
paths.json
Webproject
Dockerfile
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-stretch-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:2.2-stretch AS build
WORKDIR /src
COPY ["DockerFileCopy/DockerFileCopy.csproj", "DockerFileCopy/"]
RUN dotnet restore "DockerFileCopy/DockerFileCopy.csproj"
COPY . .
WORKDIR "/src/DockerFileCopy"
RUN dotnet build "DockerFileCopy.csproj" -c Release -o /app
FROM build AS publish
RUN dotnet publish "DockerFileCopy.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "DockerFileCopy.dll"]
There is no need to copy the paths.json in docker command, the RUN dotnet build and RUN dotnet publish will not copy this file to the published folder
We use link file to copy the paths.json, the .csproj content
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
<UserSecretsId>2594cc66-3d1a-487b-a93c-99e0cf9975ef</UserSecretsId>
<DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
<DockerfileContext>.</DockerfileContext>
</PropertyGroup>
<ItemGroup>
<Content Include="..\paths.json" Link="paths.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<CopyToPublishDirectory>Always</CopyToPublishDirectory>
</Content>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="2.2.0" PrivateAssets="All" />
<PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.7.9" />
</ItemGroup>
</Project>
Build the image in the Source folder by command like docker build -t dockerfilecopy .

asp.net core docker issue

I am new to docker and I developed a simple asp.net core webapi solution which uses a NuGet package from a private repository.
dotnet restore command returns an error for a missing NuGet package that is located inside a private repository which introduced in the nuget.config file inside the solution.
does anyone knows what's the problem with my configs and dockerfile ?
Dockerfile
FROM microsoft/aspnetcore-build:2.0 AS build-env
WORKDIR /app
# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out
# build runtime image
FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "aspnetcoretssl.dll"]
.dockerignore
bin\
obj\
Also I have a nuget.config file in the solution's root directory as below
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="myrepo" value="\\path\to\the\nuget_packages_folder" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
</configuration>
But I receive below messages from
docker build -t aspnetcoretssl .
error NU1101: Unable to find package TestContracts. No packages exist
with this id in source(s): nuget.org Generating MSBuild file
/app/obj/aspnetcoretssl.csproj.nuget.g.props.
The command '/bin/sh -c dotnet restore' returned a non-zero code: 1
It looks like there are two problems
The NuGet.config file is in your solution directory, but the Dockerfile is in the project folder. This means COPY . ./ won't copy NuGet.config into the Docker container
Even if you had the NuGet.config file, the "myrepo" source is an invalid filepath inside a Docker container.
This is invalid because this is a Windows network file path, but your container is running on Linux.
To solve this, I would recommend the following.
Either move your Dockerfile to the solution directory, or move NuGet.config into the project directory.
Add a second file named NuGet.linux.config, and point to this file when building on non-windows. Use the RestoreConfigFile property to point to this file when building on Linux. If you have moved NuGet.config into the project directory, adding these lines to your aspnetcoretssl.csproj files would work:
<PropertyGroup>
<RestoreConfigFile Condition="'$(OS)' != 'Windows_NT'">NuGet.linux.config</RestoreConfigFile>
<RestoreConfigFile Condition="'$(OS)' == 'Windows_NT'">NuGet.config</RestoreConfigFile>
</PropertyGroup>
Creating a network mount from \\path\to\the\nuget_packages_folder to Z:\
In NuGet.linux.config, change "myrepo" to <add key="myrepo" value="/nuget/myrepo" />
Mounting this drive into the container. This is done with the --volume parameter on docker-run. docker run --volume Z:/:/nuget/myrepo

Resources