Dockerfile "COPY" command confusion - docker

I have 3 .NET6 projects in my local 'src' folder as below:
src/A
src/B
src/C
A: web api project, has many nuget dependencies
B: class lib, dependent of A
C: class lib, dependent of A
(Note: of course, project related files including .csproj are present in the local project folders)
My Dockerfile is in project folder A.
I am running the "docker build" command from 'src' folder (i.e. this is the build context) as below:
docker build -f A/Dockerfile -t myapiimage:v1 .
below section of code is from my Dockerfile, where i try to restore nuget dependencies:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ./A/*.csproj A/
RUN dotnet restore A/A.csproj
this works fine and i understand that .csproj in the image is stored inside /src/A/A.csproj
Am i right?
I want to know whether COPY preserves the source folder structure in the image being built or not?
because after the below 3 lines:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY *.csproj ./
None of the line works:
RUN dotnet restore A/A.csproj
OR,
RUN dotnet restore A.csproj
OR,
RUN dotnet restore *A.csproj
OR,
RUN dotnet restore src/A/A.csproj
So, where this .csproj files (of all 3 projects) are getting stored in the image?

Your line COPY *.csproj ./ doesn't copy anything, because there are no .csproj files in your src directory on the host machine.
The way I usually do it is a bit clunky. First I do
COPY */*.csproj ./
which copies all the .csproj files into the workdir in the image. Then I do
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*}/ && mv $file ${file%.*}/; done
which looks at all the .csproj files and creates a directory for them and moves them into that directory.
With that approach, your Dockerfile would look like this
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY */*.csproj ./
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*}/ && mv $file ${file%.*}/; done
RUN dotnet restore A/A.csproj

Related

'No required module provides package' when building Go docker image

My Dockerfile is below:
# syntax=docker/dockerfile:1
FROM golang:1.18-alpine
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY *.go ./
RUN go build -o /datapuller
EXPOSE 8080
CMD [ "/datapuller" ]
I tried to build with $ docker build --tag datapuller .
But got error:
main.go:13:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/dbutil; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/dbutil
main.go:14:2: no required module provides package gitlab.com/mycorp/mycompany/data/datapuller/models; to add it:
go get gitlab.com/mycorp/mycompany/data/datapuller/models
How to solve this, I can run directly with go run main.go just fine.
My main.go's import is below. I think the imports caused this problem:
package main
import (
"encoding/json"
client "github.com/bozd4g/go-http-client"
"github.com/robfig/cron/v3"
"github.com/xuri/excelize/v2"
"gitlab.com/mycorp/mycompany/data/datapuller/dbutil"
"gitlab.com/mycorp/mycompany/data/datapuller/models"
"gorm.io/gorm"
)
func main() {
...
Because the associated package needs to be pulled when building.
Docker may be missing the necessary environment variables to pull these packages.
It is recommended that you use the go mod vendor command,then build image
FROM golang:1.18-alpine
ADD . /go/src/<project name>
WORKDIR /go/src/<project name>
RUN go build -mod=vendor -v -o /go/src/bin/main main.go
RUN rm -rf /go/src/<project name>
WORKDIR /go/src/bin
CMD ["/go/src/bin/main"]
When you copy your source code into the image, you only copy files in the current directory
COPY *.go ./ # just the current directory's *.go, not any subdirectories
It's usually more common to copy in the entire host source tree, maybe using a .dockerignore file to cause some of the source tree to be ignored
COPY ./ ./
Otherwise you need to copy the specific subdirectories you need into the image (each directory needs a separate COPY command)
COPY *.go ./
COPY dbutil/ dbutil/
COPY models/ models/

Dockerfile copy folder to restore local nuget packages

I would like to start a .NET Core WebApi in a docker container from a Unit test project. These tests will be ran during continuous integration using GitLab.
The WebApi project uses local nuget packages stored in a folder with path C:/Temp/nuget packages. This folder exists on the server which run the GitLab runner.
C:/Temp/nuget packages
C:/Users/myUser/project/MyProject with the following tree
- sln
- MyProjectWebAPI
-- dockerfile
- MyProject.Tests
-- link to the MyProjectWebAPI's dockerfile\
My dockerfile looks like this
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
#copying some csproj files
RUN mkdir Oncolin.DataAccess.WebAPI/nugetPackages
#VOLUME "C:/Temp/nuget packages":"Oncolin.DataAccess.WebAPI/nugetPackages" -> failed, unable to name a volume in a dockerfile so my folder is empty and the dotnet restore failed
#COPY ["C:/Temp/nuget packages", "Oncolin.DataAccess.WebAPI/nugetPackages"]-> failed, can't reach Temp folder (using ../.. and/or finished with '/' gave the same result)
#ADD["C:/Temp/nuget packages", "Oncolin.DataAccess.WebAPI/nugetPackages"]-> failed, can't reach Temp folder (using ../.. and/or finished with '/' gave the same result)
#RUN cp -r "C:/Temp/nuget packages" Oncolin.DataAccess.WebAPI/nugetPackages -> failed, can't reach Temp folder
RUN find Oncolin.DataAccess.WebAPI/nugetPackages
RUN dotnet restore "Oncolin.DataAccess.WebAPI/Oncolin.DataAccess.WebAPI.csproj" -s https://api.nuget.org/v3/index.json -s nugetPackages --verbosity n
COPY . .
WORKDIR "/src/Oncolin.DataAccess.WebAPI"
RUN dotnet build "Oncolin.DataAccess.WebAPI.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Oncolin.DataAccess.WebAPI.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Oncolin.DataAccess.WebAPI.dll"]
I have to copy C:/Temp/nuget packages in my container during the build to successfully restore nuget packages. I try several things as you can see in the dockerfile.
I run the docker build command like this
C:\Users\myUser\project\MyProject> docker build -t 'oncolin-webapi' --progress plain -f '.\Oncolin.DataAccess.WebAPI\Dockerfile' .
How can I achieve this ?
Is it possible to retrieve the name of the volume ?
Thanks in advance
I found a workaround to copy my local nuget packages :
access to the nuget packages folder of the current user (i.e. c:\users\myUser.nuget\packages) and copy my known local nuget packages (all started by "rnc" by convention) in a new folder in my solution
var localPackagesDirPath = Path.Combine(solutionDirPath, "localNugetPackages");
var localPackagesDir = Directory.CreateDirectory(localPackagesDirPath);
var directories = Directory.GetDirectories(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), ".nuget", "packages"), "rnc*");
foreach (var item in directories)
{
var rncDir = new DirectoryInfo(item);
var srcRncDir = localPackagesDir.CreateSubdirectory(rncDir.Name);
CopyAll(rncDir, srcRncDir);
}
copy it during build of the image
use the local nuget packages folder to restore them
COPY ["localNugetPackages/", "Oncolin.DataAccess.WebAPI/localNugetPackages/"]
RUN dotnet restore "Oncolin.DataAccess.WebAPI/Oncolin.DataAccess.WebAPI.csproj" -s https://api.nuget.org/v3/index.json -s localNugetPackages --verbosity n
It's clearly not pretty but it's the only solution I found to achieve my goal. Luckily I don't have a lot of local nuget packages.

Where is this dockerfile putting files in the container?

I'm using Docker to build an ASP site, and I'm confused about where my files are going. Here's my dockerfile (the application is called AspCore)
FROM microsoft/dotnet:2.2-sdk as build
ARG BUILDCONFIG=RELEASE
ARG VERSION=1.0.0
COPY AspCore.csproj /build/
RUN dotnet restore ./build/AspCore.csproj
COPY . ./build/
WORKDIR /build/
RUN dotnet publish ./AspCore.csproj -c $BUILDCONFIG -o out /p:Version=$VERSION
FROM microsoft/dotnet:2.2-aspnetcore-runtime
WORKDIR /app
COPY --from=build /build/out .
ENTRYPOINT ["dotnet", "AspCore.dll"]
This works correctly and I can access the site when I run the image, but I don't quite understand how this is working. When I open a shell to my container, I don't see a build directory either in the app directory or in the base directory. I also can't find the AspCore.csproj file.
Isn't the dockerfile copying AspCore.csproj into the build directory, so shouldn't there be a build directory with a bunch of files in it on my container? What am I misunderstanding?
That's just because you're using 2 stage on your Dockerfile to build your image.
Stage 1: (BUILD) Based on microsoft/dotnet:2.2-sdk as build image to build your source code into dll files
Stage 2: (SERVE) Based on microsoft/dotnet:2.2-aspnetcore-runtime to create a runtime for your dlls, in this stage you've already copy files from previous stage into a folder called app by this line COPY --from=build /build/out ..
After stage 2 copied files from stage1, nothing else you can see from stage1 but copied files, that's why when you start your container you didn't see /build folder
This pattern is good for production build when you want to minimize your image, because actually we don't need sdk in production environment, we just need runtime for the compiled code.
Hope that clear enough, for more information, you can take a look at this article

Docker compose build copy failed

I am really curious to how to interpret and debug with the following error:-
C:\users\project>docker-compose build
Step 6/15 : COPY *.csproj ./
ERROR: Service 'invoiceservice' failed to build: COPY failed: no source files were specified
This is particular micro-service as i have few more such services.
docker file :-
FROM mcr.microsoft.com/dotnet/core/runtime:2.2 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build
WORKDIR /src
COPY *.csproj ./
RUN dotnet restore -s https://api.nuget.org/v3/index.json -s https://www.myget.org/F/autoweb/api/v3/index.json
COPY . .
WORKDIR /src/src/InvoiceManagement/InvoiceService
RUN dotnet publish -c Release -o out
FROM build AS publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/out .
ENTRYPOINT ["dotnet", "InvoiceService.dll"]
Interesting part is when I build this from Visual Studio IDE, its being built fine but it does not build on CLI.
docker compose file:-
invoiceservice:
image: ${DOCKER_REGISTRY-}invoiceservice
build:
context: .
dockerfile: src/InvoiceManagement/InvoiceService/Dockerfile
I don't understand why CLI could not find the source location and copy where as VS works fine. Any clue???
It's likely an issue with your Docker context. Commands in Dockerfiles are relative to where the docker/docker-compose CLI tool is run, not relative to the Dockerfile location.
In other words, if you run the command from the solution root, then your csproj file is actually under ./src/InvoiceManagement/InvoiceService. As a result, *.csproj finds no files, because there's no project files literally in your solution root.
I tried replicating your problem and I was able to copy all the files successfully (instead of .csproj, I used .txt). The problem occurred when there was no txt file to copy. My COPY command failed with exactly the same message as yours.
Now, why is VS able to build the image successfully? That is because VS build the project first! When the project's build procedure is completed, a .csproj file is generated and that gets copied to the image.
To confirm the results, ls your current directory (when the build fails from command line) and see if there is any .csproj file in that directory.

How to point correct path in the Dockerfile

I have .sln file and Dockerfile at a different location on my local hard drive.
Till now I work on a source code where .sln file and Dockerfile are present at the same location.
I am facing difficulty to configure it correctly.
I have .sln file at the location CompanyCarsDocker\CompanyCarsDocker\CompanyCarsDocker.sln
And My Dockerfile is present inside CompanyCarsDocker\CompanyCarsDocker\CarApi\Dockerfile
-
Below is my Dockerfile
FROM microsoft/aspnetcore:2.0 AS base
WORKDIR /app
EXPOSE 54411
FROM microsoft/aspnetcore-build:2.0 AS builder
WORKDIR /src
COPY . ./*.sln ./
ADD ./site /inetpub/wwwroot
COPY ./CarApi.csproj CarApi/
RUN dotnet restore
COPY . .
WORKDIR /src/CarApi
RUN dotnet build -c Release -o /app
FROM builder AS publish
RUN dotnet publish -c Release -o /app
FROM base AS production
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "CarApi.dll"]
I have added ADD ./site /inetpub/wwwroot for testing purpose. I am getting
ADD failed: CreateFile \\?\C:\ProgramData\Docker\tmp\docker-builder782713904\site: The system cannot find the file specified.
Same error I am also Getting at COPY ./CarApi.csproj CarApi/
Please let me know if I configured something wrong in the Dockerfile.
You can use the -f flag of docker build to specify the path to the Dockerfile
Make sure you’re current working directory is the one which has the sln file and rest of the project. Run the command docker build -f <path to Dockerfile> .
your .sln file is two directories up so you should use
COPY ../../*.sln ./
instead of
COPY . ./*.sln ./

Resources