gitlab runner shell executer path finding problems - docker

I'm kinda stuck in how to deploy something.
As a dotnet developer I created a webapp (just the simple weatherforecastteamplte) and tried to deploy this.
My ci-cd has 3 stages: build, test and deploy.
My first approach was to use aa ssh key to ssh into my buildserver and transfer the files, because I had gitlab runner running on my gitlab server rather than my buildserver.
For security reasons and other reasons , I have installed gitlab-runner on my buildserver.
My buildserver includes all the tools I nedd: docker, docker-compose, dotnet-npm, ...
So i registered a new runner with shell.
It starts to build the image but when it comes in my dokcerfile it gets stuck because it can't find the cproj file. Currently I have been trying to figure out where the executer is looking for the project file but currently I can't figure it out.
Hope someone can help me.
my dockerfile (generated by visual studio':
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["Test/Test.csproj", "Test/"]
RUN dotnet restore "Test/Test.csproj"
COPY . .
WORKDIR "/src/Test"
RUN dotnet build "Test.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Test.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Test.dll"]
my gitlab ci cd file:
image: mcr.microsoft.com/dotnet/sdk:latest
stages:
- build
- test
- deploy
before_script:
- dotnet restore Test.sln
build:
stage: build
tags:
- dev
script:
- 'dotnet build --no-restore Test.sln'
tests:
stage: test
tags:
- dev
script:
- 'dotnet test --no-restore Test.sln'
deploy:
stage: deploy
environment: production
tags:
- dev
before_script:
- cd Test
script:
- 'docker build - < Dockerfile'

Related

Dotnet not found error when generating Dockerfile

I have a Gitlab-ci pipeline that should automatically generate a Dockerfile for a given .Net Core project if it does not already exist and after that execute a docker build.
I am getting an error when attempting to generate the Dockerfile :
section_end:1657538312:prepare_script
[0Ksection_start:1657538312:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Initialized empty Git repository in /builds/demo/devops/labs/.git/
[32;1mCreated fresh repository.[0;m
[32;1mChecking out c2e2d044 as ingress...[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1657538314:get_sources
[0Ksection_start:1657538314:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m[0;m
[32;1mDownloading artifacts for publish_job (36805)...[0;m
Downloading artifacts from coordinator... ok [0;m id[0;m=36805 responseStatus[0;m=200 OK token[0;m=-1ETTroN
section_end:1657538315:download_artifacts
[0Ksection_start:1657538315:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[32;1m$ dotnet restore --packages $NUGET_PACKAGES_DIRECTORY[0;m
/scripts-1175-36806/step_script: eval: line 118: dotnet: not found
section_end:1657538316:step_script
[0Ksection_start:1657538316:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1657538316:cleanup_file_variables
[0K[31;1mERROR: Job failed: command terminated with exit code 127
[0;m
However the dotnet command executes well in the publish stage. I am not sure why dotnet is not being recognized in the docker context.
The pipeline:
before_script:
- 'dotnet restore --packages $NUGET_PACKAGES_DIRECTORY'
build_job:
tags:
- labs
only:
- develop
- ingress
stage: build
script:
- dotnet build --configuration Release --no-restore
publish_job:
tags:
- labs
only:
- develop
- ingress
stage: publish
artifacts:
name: "$CI_COMMIT_SHA"
paths:
- ./$PUBLISH_DIR
script:
- dotnet publish ./src --configuration Release --output $(pwd)/$PUBLISH_DIR
docker_build_dev:
tags:
- labs
image:
name: gcr.io/kaniko-project/executor:v1.6.0-debug
entrypoint: [""]
only:
- develop
- ingress
stage: docker
script: |
echo "checking dockerfile existence"
if ! [ -e Dockerfile ]; then
echo "dockerfile doesn't exist. Trying to create a new dockerfile from csproj."
docker_entrypoint=$(grep -m 1 AssemblyName ./src/*.csproj | sed -r 's/\s*<[^>]*>//g' | sed -r 's/\r$//g').dll
cat > Dockerfile << EOF
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
RUN dotnet --version
WORKDIR /app
COPY ./publish .
ENTRYPOINT dotnet $docker_entrypoint
EOF
echo "dockerfile created"
else
echo "dockerfile exists"
fi
What am I missing ?
eval: line 118: dotnet: not found
It looks like you haven't defined an image: for this job and whatever default image you are using does not have dotnet installed (or if you're using a shell runner, your runner system does not have dotnet installed/on PATH).
To address this, you should either (1) specify an image: that has dotnet available (or install it on your runner if using a shell runner and ensure it is on PATH) or (2) install dotnet as part of your job.
For example:
myjob:
image: mcr.microsoft.com/dotnet/sdk
# ...
Alternatively, you can use a dotnet install script to install dotnet in your job.

gitlab - Fetch Dockerfile from script

We're using GITLAB CI/CD for deployment. This is the publish stage , Dockerfile is used here. If you check the script , I've integrated one environment variable(line no:10) , because we're using two jobs for publish itself like developer and stage. For that stage , I shown to you.
docker_build_stage:
stage: Publish
image: docker:19.03.11
services:
- docker:19.03.11-dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build --build-arg environment_name=stage -t $IMAGE_TAG .
- docker push $IMAGE_TAG
only:
- /^([A-Z]([0-9][-_])?)?SPRINT(([-_][A-Z][0-9])?)+/i
- stage
This is the docker file , we're using.
FROM maven:3.8.3-jdk-11 AS MAVEN_BUILD
COPY pom.xml /build/
COPY src /build/src/
WORKDIR /build/
RUN mvn clean install package -DskipTests=true
FROM openjdk:11
WORKDIR /app
COPY --from=MAVEN_BUILD /build/target/provider-service-*.jar /app/provider-service.jar
ENV PORT 8092
ENV env_var_name=$environment_name
EXPOSE $PORT
ENTRYPOINT ["java","-Dspring.profiles.active="$env_var_name,"-jar","/app/provider-service.jar"]
In the dockerfile , at last line(line no:12) , we add environment variable(line no:10), before ,the variable should be
active=stage"
Because , we're maintaining respective branch as per the environment. Now, we merged developer and stage environment into single script. We are facing some fetching issue. Pipeline was successful but it doesn't fetch.
I did not completely understand what is the issue you are mentioning, but I see you are missing ARG instruction in Dockerfile which is required to define the build-arg that you are passing if you want to use it in the build stage, in this case this arg "$environment_name"
You might change it like this
FROM maven:3.8.3-jdk-11 AS MAVEN_BUILD
COPY pom.xml /build/
COPY src /build/src/
WORKDIR /build/
RUN mvn clean install package -DskipTests=true
FROM openjdk:11
ARG environment_name
WORKDIR /app
COPY --from=MAVEN_BUILD /build/target/provider-service-*.jar /app/provider-service.jar
ENV PORT 8092
ENV env_var_name=$environment_name
EXPOSE $PORT
ENTRYPOINT ["java","-Dspring.profiles.active="$env_var_name,"-jar","/app/provider-service.jar"]

Gitlab Pipeline to build docker Image of dotnetcore application and deploy

I have Installed Gitlab in one of the Ubuntu machine. And I have dotnetcore project in the name of ABC in the Gitlab.
But, In that ABC repo have multiple small doetnetcore application with different different directory like abc1 abc2 abc3 abc4.
I want to write a single pipeline under ABC to create the docker Image whenever developer push the code in the respective directory. but that need to be create docker Image for that only directory.
eg: Developer push the code under abc3 directory, that time pipeline run and create the docker Image for only abc3 directory.
Please help me out with it.
Thanks in Advance...!!!
Below is my pipeline what I have written also Docker file:
stages:
- docker
- build
services:
- docker:dind
before_script:
- "echo $gitlab"
docker-job:
stage: docker
image: docker:dind
script:
- docker login -u username -p password $CI_REGISTRY
- docker build -t dotnetcore .
#- docker push $IMAGE_PUSH:latest
build:
stage: build
tags:
- shell
image: mcr.microsoft.com/dotnet/sdk
script:
- dotnet restore
- dotnet build
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 80
ENV ASPNETCORE_URLS=http://+:80
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["dotnetcore.csproj","./"]
RUN dotnet restore "dotnetcore.csproj"
COPY . .
WORKDIR "/src/"
RUN dotnet build "dotnetcore.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "dotnetcore.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "dotnetcore.dll"]
In this pipeline and dockerfile I am only able to build "dotnetcore" project. But I have dotnetcore1 doctnetcore2 dotnetcore3 projects under the same Repo.
gitlab ci has a only:changes feature. You can define multiple jobs, each for one docker.
job-a:
script: docker build a
only:
changes:
- a/**/*
job-b:
script: docker build b
only:
changes:
- b/**/*
Assuming you have a small number of directories. You can create a job for each directory. Use rules with changes to check if changes were made in the particular directory.

Dotnet publish on linux error in docker (Azure DevOps build)

I have self-hosted linux build agent in Azure DevOps. I try to build&push docker task and i have an error like this(locally dotnet build worked):
The specified task executable "node" could not be run. System.ComponentModel.Win32Exception (2): No such file or directory [/src/Cillian.csproj]
My log:
...
Step 11/26 : RUN dotnet build -c Release -o /app
---> Running in 9233fd642015
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
Restore completed in 1.51 sec for /src/Cillian.csproj.
/root/.nuget/packages/microsoft.typescript.msbuild/3.0.0/tools/Microsoft.TypeScript.targets(293,5): error MSB6003: The specified task executable "node" could not be run. System.ComponentModel.Win32Exception (2): No such file or directory [/src/Cillian.csproj]
...
Pipeline: AzureDevOpsImg
My Dockerfile:
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /src
COPY *.csproj ./
RUN dotnet restore
COPY . .
WORKDIR /src
RUN dotnet build -c Release -o /app
FROM build AS publish
RUN dotnet publish -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Cillian.dll"]
No such file or directory
Just focus on this error message, dotnet build could not find the csproj file to continue the build. This is the mostly prone error in Azure devops even if it can be built successfully in local.
That is because the docker runs at the Repos/solution level locally(e.g. VS). BUT, for Azure devops CI, it running the docker in the directory where the dockerfile lives, which is at project level.
The nutshell is the relative path which can succeed to used in the local to find csproj, not avilable in the Azure devops at all. Because the working directory has changed.
Please specify $(Build.Repository.LocalPath) to the Docker 2.* argument Build context.
For YAML:
- task: Docker#2
displayName: build
inputs:
containerRegistry: DockerHub
repository: xxx/xxx
Dockerfile: Docker/TestWebApi/Dockerfile
buildContext: '$(Build.Repository.LocalPath)' # Specify the context
tags: latest

Gitlab CI - Docker in Docker - How to maintain a pipeline?

I am trying to use gitlab CI for docker in docker builds and i have it up and running.
My issue is that the nice pipeline functionality of gitlab is somewhat lost since the whole build,test,etc... stages are defined in the DockerFile and not yaml.
I only have a single stage in the yaml right now.
Is there a way to use docker in docker and still leverage gitlab pipelines?
Relevant files.
Dockerfile
# Stage 1 - require and update image
FROM microsoft/aspnetcore-build:2.0.0 AS build
# Update the imgage OS
RUN apt-get update \
&& apt-get install -y
# set the container working directory
WORKDIR /code
# Stage 2 - copy data
# caches restore result by copying csproj file separately
COPY src/Nuget.Config ./src/
COPY src/Inflow.NewsMl.FileLoader/Inflow.NewsMl.FileLoader.csproj ./src/Inflow.NewsMl.FileLoader/
RUN dotnet restore src/Inflow.NewsMl.FileLoader/Inflow.NewsMl.FileLoader.csproj --configfile /code/src/Nuget.Config
COPY src/Inflow.NewsMl.FileLoader.Test/Inflow.NewsMl.FileLoader.Test.csproj ./src/Inflow.NewsMl.FileLoader.Test/
RUN dotnet restore src/Inflow.NewsMl.FileLoader.Test/Inflow.NewsMl.FileLoader.Test.csproj --configfile /code/src/Nuget.Config
# Copy the rest
COPY . .
# Stage 3 - Build and test
RUN dotnet test src/Inflow.NewsMl.FileLoader.Test/Inflow.NewsMl.FileLoader.Test.csproj
# Stage 4 - Deploym to image
RUN dotnet publish src/Inflow.NewsMl.FileLoader --output /code/output --configuration Release
FROM microsoft/aspnetcore:2.0.0
COPY --from=build /code/output /app
WORKDIR /app
ENTRYPOINT [ "dotnet", "Inflow.NewsMl.FileLoader.dll" ]
and the ci runner yaml
image: docker:latest
.before_script:
- docker info
variables:
DOCKER_DRIVER: overlay
CONTAINER_TEST_IMAGE: infomedia/newsml.fileloader:CI_COMMIT_SHA
#$CI_COMMIT_TAG
services:
- docker:dind
stages:
- build
build:
stage: build
#only:
# - tags
#except:
# - master
script:
- docker build -t $CONTAINER_TEST_IMAGE .

Resources