I have a github repository with a declarative pipeline Jenkinsfile.
The workingdirectory on my node contains subdirectories.
The project is a simple empty linkx docker project created with visual studio 2017 and .net core 2.1. It executes on my windows 7 machine normally and has a hello world web page.
I am unable to build the Dockerfile on jenkins.
I can start the Dockerfile build using dir(...){}.
The failing step is always
COPY ["MyProject/MyProject.csproj", "MyProject/"]
This step requires the relative path to be in MySolution.
The file Workspace/MySolution/MyProject/MyProject.csproj exists
The error Message is that Workspace/MyProject/MyProject.csproj does not exist.
I searched exhaustively using google and stackoverflow. Among the things I tries are combinations of sh commands, dir syntax, options on docker build like -f. Some of them were straigth up failures and the best results I had ran into the COPY step issue.
One example of a failing step in the Jenkinsfile would be:
dir("MySolution/MyProject")
{
script
{
docker.build("MyProject", ".")
}
}
This fails with the COPY issue from above.
I have seen questions on so that seem to not quite apply here and which solutions did not transfer to this issue.
It turns out I ended up really close to the solution of my issue.
This fixed my sub folder problem:
dir("MySolution")
{
script
{
docker.build("MyProject", "-f ./MyProject/Dockerfile .")
}
}
Related
Currently working to get a new app to deploy using Jenkins and stuck on an issue with package restore.
Please note...my focus is in dev, not DevOps so a lot of this is still pretty foreign to me. So, I apologize in advance if it takes some back and forth for clarification.
The project uses the following files: jenkinsfile, docker-compose.bi.build.yml, build.sh, and nuget.config. Those files are all listed below for reference.
Brief Overview:
I have 2 DLLs that are source controlled in 2 separate VS solutions (and seperate Git repos). Those DLLs are: AN.Infrastructure (infras) and AN.Domain.Core (domain). domain depends on infras. infras is some basic stuff...string extensions, etc. and doesn't have many depenedencies. domain has more dependencies on things like EF, Postgres EF, AutoMapper, etc., but it also pulls in infras during the package restore process. infras and domain are set up the same way as far as source control, jenkins, etc. are concerned (meaning I expect them to build nearly the same minus the extra dependencies).
Currently, infras is building and deploying as it should so i know the process I have is working.
The process goes like this:
The jenkinsfile spins up a container based on the ci.build.yml file.
This container is meant to build the source code and copy the output
to a staged directory so that the rest of the process can
package/deploy it.
The build.sh script is executed inside the running container that is
building the code and is doing the work to restore dependencies, kick
off msbuild, and then package the artifacts.
Problem:
I can't get the domain solution to build thru Jenkins due to what appears to be missing package references. The process throws the following error:
Package Microsoft.EntityFrameworkCore.Analyzers, version 3.1.4 was not
found. It might have been deleted since NuGet restore. Otherwise,
NuGet restore might have only partially completed, which might have
been due to maximum path length restrictions.
What I Know:
I have proven the infras pipeline works end-to-end as I can navigate
to and install the infras assembly via VS nuget UI to the domain
solution on my local machine. The domain solution builds and works
as expected in this manner.
I also know the code for domain is being pulled down to the jenkins
server when the pipeline job is kicked off...I remoted into the
workspace folder in /var/lib/jenkins to confirm.
Also, I see all my echo printouts in the output from the build.sh
script when the domain job runs. So, the process is running like I
expect, it just fails at package restore.
Since infras is hosted on a Linux file share and not directly at
nuget.org, I read that I have to use a nuget.config file next to the
solution in order to configure the package sources so dotnet build
knows where to restore the packages from. I created that file and
it is below. I have tried with and without that file though and it
fails both ways.
I noticed that the MS.EF.Analyzers DLL is being pulled into the
domain project indirectly via npgsql.entityframeworkcore.postgres
dependency. So, I decided to include npgsql in infras solution as
well as I thought this would break that build also. However, that
wasn't the case. infras built in Jenkins even after I pulled in
that depenedency.
What I Have Tried:
You name it....I'm at 45 builds in Jenkins right now...each time trying something new (eg...with nuget.config, without nuget.config, etc.). What I posted in the files below is the current state and what I have described in the question. I am more than happy to answer any question, provide any additional info, or try whatever to get this resolved...just not sure where to go at this point.
Jenkinsfile:
pipeline {
agent any
stages {
stage('Build The Codes') {
steps {
sh '''sudo AN_BUILD_NUMBER=${BUILD_NUMBER} /usr/local/bin/docker-compose -f docker-compose.ci.build.yml up'''
}
}
stage ('Deploy The Codes') {
when {
branch 'main'
}
steps {
sh '''sudo cp ./src/AN.Domain.Core/bin/Release/*${BUILD_NUMBER}.nupkg /opt/nuget/development/'''
}
}
stage ('Deploy The Codes 2.0') {
when {
branch 'release'
}
steps {
sh '''sudo cp ./src/AN.Domain.Core/bin/Release/*${BUILD_NUMBER}.nupkg /opt/nuget/production/'''
}
}
}
}
docker-compose.ci.build.yml:
version: '3.8'
volumes:
nugetdirectory:
services:
ci-build:
image: mcr.microsoft.com/dotnet/sdk:3.1
volumes:
- .:/src:z
- nugetdirectory:/opt/nuget/production
working_dir: /src
environment:
- "AN_BUILD_NUMBER=${AN_BUILD_NUMBER}"
command: /bin/bash -c "sh ./build.sh"
build.sh:
echo '*** Starting build process for AN.Domain project ***'
echo 'Generate New Assembly Version'
year=$(date '+%Y')
month=$(date '+%m')
day=$(date '+%d')
version="${year}.${month}.${day}.${AN_BUILD_NUMBER}"
echo 'New Assembly Version: ' + $version
echo 'Clean previous output directory for AN.Domain project'
echo ' - Remove Docker publish directory'
rm -rf obj/Docker/publish
echo ' - Remove previous nuget packages'
rm /src/src/AN.Domain.Core/bin/Release/*.nupkg
echo 'Clean output directory for AN.Domain project'
dotnet clean
echo 'Publish AN.Domain project'
dotnet msbuild /t:Restore -target:Publish -property:Configuration=Release -property:OutputPath=obj/Docker/publish -property:Version=$version
echo 'Copy NuGet packages to output directory'
cp /src/src/AN.Domain.Core/bin/Release/*.nupkg obj/Docker/publish
echo '*** Finishing build process for AN.Domain project ***'
nuget.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
<add key="privateRepo" value="/opt/nuget" />
</packageSources>
<activePackageSource>
<add key="All" value="(Aggregate source)" />
</activePackageSource>
</configuration>
UPDATES:
I removed the domain project from the solution and created a brand new console app to test with. This project built as expected when run via Jenkins. I even added the dependency to npgsql which also brought in the MS.EF.Analyzers DLL that is causing the issue.
I just added to the test console app the dependency on infras and
this breaks the build with the same error. So, it is definitely
bringing in that custom nuget package that is causing the build
error.
So, I replaced this line in build.sh:
dotnet msbuild /t:Restore -target:Publish -property:Configuration=Release -property:OutputPath=obj/Docker/publish -property:Version=$version
with this:
dotnet publish -c Release -o obj/Docker/publish /property:Version=$version
Initially I had "dotnet publish", but I had this same error. I got the former idea from the SO question below, but I believe I did this before I learned about nuget.config. So, I think what might have happened is that both were necessary but I didn't have nuget.config when I was using "dotnet publish" and then I switched it over to msbuild and never went back and tested with this combination of dotnet publish and nuget.config.
Jenkins not picking up the nuget restored packages
New to Jenkins so apologise in advance as I'm sure this answer is out there somewhere. Just not sure exactly how to search for what I'm after. I'm struggling a bit with the copyback process in Jenkins.
When I build, I'm running some unit tests that create some log files which I want to be stored as part of the Jenkins build. I'm running on Windows 10 and everything is running on my laptop (I'm purely trying to learn Jenkins so this is fine for me).
So my test results will always appear in C:\TestLogs\*.log. I want the results copied to my build directory which is URL: http://localhost:8080/job/loadrunner_test/1/ absolute: C:\Program Files (x86)\Jenkins\jobs\loadrunner_test\builds\1
I'm a bit confused with which plugin I should use in my post build step? Copy Artifact plugin looks as if it's meant to pass data between builds. For each build, I just want to copy C:\TestLogs*.* to the current build directory so I can see them when I click on the link for #1 in the Build History.
Many thanks!
Tim
WindowsDir
Jenkins Build
You can copy it with additional step.
Select Execute Windows batch command for that step and add this line:
xcopy C:\TestLogs C:\Program Files (x86)\Jenkins\jobs\jenkins_test\builds\%BUILD_NUMBER% /s /e
You can also check configuration for your test if you can set path location.
My spring-boot application was originally built with maven, but I'm trying to gradle-ify it. Part of my build process was making a docker image with the dockerfile-maven plugin. The most accepted gradle equivalent seems to be the com.palantir.docker, so that's what I'm using (at least according to this tutorial from pivotal.)
My setup is pretty simple. My dockerfile takes a build argument, which is the final jar, and adds it to the image.
FROM openjdk:8-jdk-alpine
MAINTAINER github.com/paul-nelson-baker
EXPOSE 5005 8080 35729
ENTRYPOINT ["java", "-Xdebug", "-Xrunjdwp:server=y,transport=dt_socket,address=5005,suspend=n", "-jar", "/var/paulbaker/pauls-blog.jar"]
ARG JAR_FILE
ADD ${JAR_FILE} "/var/paulbaker/pauls-blog.jar"
The part that's getting me hungup is the gradle setup. The build arg that I pass appears correct:
docker {
name = "paulbaker.io/my-blog"
tags 'latest'
dockerfile file('src/main/docker/Dockerfile')
files "${project.name}-${version}.jar"
buildArgs([JAR_FILE: "${project.name}-${version}.jar"])
}
However whenever I attempt to build the image, I get the following error even though the actual java build succeeds:
$ gradle build docker
> Task :docker FAILED
ADD failed: stat /var/lib/docker/tmp/docker-builder952857277/pauls-blog-0.0.1-SNAPSHOT.jar: no such file or directory
I've tried a myriad of combinations to try to specify the path (relative to my project) of ./build/libs/pauls-blog-0.0.1-SNAPSHOT.jar, which I can see exists while the printed directory above does not.
There is clearly something I'm not understanding about gradle builds, can someone clear up how to get the final jar file?
The following error:
> Task :docker FAILED
ADD failed: stat /var/lib/docker/tmp/docker-builder952857277/pauls-blog-0.0.1-SNAPSHOT.jar: no such file or directory
happens when the Palantir Docker plugin cannot find the file specified using the docker.files attribute with respect to your current context directory.
In your question, the plugin was searching for the jar file in the root directory of the project (likely where your build.gradle is) and fails because the jar file is not generated there. When you specify jar.archivePath, you are specifying the full location of the path to the jar file which will now obviously be found.
I've solved the problem, but I don't know why this is different, so I'm leaving the question opened so someone has a chance to give a detailed explanation as to why this works.
The correct values should be referencing jar.archivePath.
So the final result looks like this:
docker {
name = "paulbaker.io/my-blog"
tags 'latest'
dockerfile file('src/main/docker/Dockerfile')
files jar.archivePath
buildArgs([JAR_FILE: "${jar.archiveName}"])
}
The arguments to files are added to a gradle CopySpec of the docker task.
So files jar.archivePath adds that path where your jar build artifact was created to this CopySpec.
When your Dockerfile is processed, the jar.archiveName is inserted into the ADD template and because your jar.archivePath is in the CopySpec of the docker build context, it finds the jar.archiveName in the CopySpec and copies the file to the destination of ADD.
This is my take on what I'm seeing in my own testing.
Im trying to setup jenkins in my company and Ive got some problems.
Im using this commands to build the project:
SET MSBuild="C:\Windows\Microsoft.NET\Framework\v3.5\MSBuild.exe"
SET BUILDS=C:\Program Files (x86)\Jenkins\jobs\xxx\builds\
SET OUTPUT_PATH="%BUILDS%%BUILD_NUMBER%"
SET RSVARS="C:\Program Files (x86)\Embarcadero\RAD Studio\10.0\bin\rsvars.bat"
CALL %RSVARS%
SET PATH=%PATH%;D:\komponenty\DXE3\ADSI
%MSBuild% xxx.dproj /t:Build /p:Config=Debug;Platform=Win32;DCC_ExeOutput=%OUTPUT_PATH% /maxcpucount:4
It works fine when i type this in cmd but. I gave administrive privileges to jenkins service. When I try to build project with jenkins i receive error like this:
F1026: File not found: 'ADSI.dcu
this is a component for Delphi and i have this component on second partition. Jenkins has access for many components on this partition but not for this one.
The difference between the two will be your current directory.
Jenkins will start you off in a specific working folder for the job (possibly C:\Users\<User-ID>\.jenkins\jobs\<Job-Name>\workspace).
Add the following to your Jenkins commands to see where you're doing the build from:
echo Current Folder: %cd%
A simple "solution" would be to just add a command in Jenkins to change directory to the same folder you're in when you test from the command-line.
However, I suggest you rather do the following:
Ensure Jenkins gets the latest source from your source repository into its working folder. (There are various plugins depending on what particular tool you use.)
Ensure you cd (change directory) to the correct folder within the workspace folder.
I have set up a jenkins build and everything is working fine except the very last step.
The whole build creates a directory called: build
This directory contains a web-inf and all the files in it I would like to publish via SCP to a different location, so that all the content of the build/web-inf folder will become the content of the target folder.
The settings for jenkins scp plugin are (it is a post-build step):
source: build/web-inf/**
destination: public_html/
that results in:
public_html/build/web-inf/...
but should be:
public_html/...
(the keep hierarchy box is ticked)
how can I make that happen??
EDIT
I could solve the problem without any additional script. The solution is so simple that my question turned out to be stupid.
All I did was telling ant to copy all the webfiles to ./public_html instead of ./build/web-inf/ what made the jenkins scp copy all files from public_html to public_html exactly as it was intended to.
If your goal is just to SCP files generated during the build, and the plugin doesn't seem to be working (I couldn't see anything wrong in your configuration) you can use an "Execute shell" build step and type the scp command something like (try it in a shell first in your job's build directory to get the syntax right):
scp -r build/web-inf/* user#host:/destination-directory