How to use docker with gradle while having private docker registry server - docker

I am trying to setup https://spring.io/guides/gs/spring-boot-docker/#initial with my environment, which has a private registry server #
el-qa-docker.x.x.x:18445
my build.gradle has the following relevant content:
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.5.3.RELEASE")
classpath('se.transmode.gradle:gradle-docker:1.2')
classpath ('org.codehaus.groovy:groovy-backports-compat23:2.3.5')
}
group = 'james'
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'
apply plugin: 'docker'
task buildDocker(type: Docker, dependsOn: build) {
push = true
applicationName = jar.baseName
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}
when I run: $gradle build buildDock
I get build FAILURE, with:
$ gradle build buildDock
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Docker execution failed
Command line [docker build -t skahmed/gs-spring-boot-docker:latest /Users/skahmed/devops/TOOLS/DOCKER/gs-spring-boot-docker/initial/build/docker] returned:
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 16.243 secs
any ideas, on where do I specify docker's internal registry which can be used with Gradle.
I tried:
docker {
useApi true
hostUrl 'el-qa-docker.x.x.x:18445'
apiUsername 'james'
}
as provided by https://github.com/Transmode/gradle-docker
but that gives a different error:
$ gradle build buildDock
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Port is invalid: -1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1.389 secs
I tried : build.gradle
docker {
//useApi true
hostUrl 'el2-dt-docker.uscis.dhs.gov:18445'
apiUsername 'skahmed'
}
and
task buildDocker(type: Docker, dependsOn: build) {
push = true
applicationName = jar.baseName
//applicationName = 'el2-dt-docker.uscis.dhs.gov:18445/skahmed/gs-spring-boot-docker'
//applicationName = 'gs-spring-boot-docker'
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}
but get:
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Docker execution failed
Command line [docker push skahmed/gs-spring-boot-docker:latest] returned:
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED

You can specify your private Docker registry as part of your Docker image name e.g.:
el-qa-docker.x.x.x:18445/skahmed/gs-spring-boot-docker
Please make sure your Docker daemon is aware about private registry.

Related

Jenkins run a shell command when build is failed as a post build action to run failed TestNG plan

I want to rerun, below shell script in the same project, once my build is i completed with errors, so I can rerun my failed test cases in testng-failed.xml.
FILE=./target/surefire-reports/testng-failed.xml
if test -f "$FILE"; then
echo "$FILE exists."
mvn clean test -Dhttp.proxyHost=proxy-dev.aws.skynet.com -Dhttp.proxyPort=8080 -Dproxy-dev.aws.skynet.com -Dhttps.proxyPort=8080 -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dos_version="10" -Dos="Windows" -Dlocalrun="false" -Dtestxml=FILE
else
echo "$FILE is not there"
fi
But in Jenkins, post build section, I don't see an option to add Execute shell. Please help.
I installed post build plugin , but it's only show for maven projects and not for my free style project.
In free style project, No post steps option.
I managed a walkaround to do this without any plugins and still run in the Jenkins free style project. Add two execute shells and in first shell enter the below shell commands.
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
cd CommonPagesStorage
mvn clean install -DskipTests
cd ..
cd MultiWebdriverDemo
mvn clean test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}
echo "${TestPlanName}is ran"
#Dmaven.test.failure.ignore=true added this so even the script failed it doesn't mark as fail and mark it as unstable and continue.
In the second shell command ,conditionally checks the testng-failed.xml is exists if so run it to run the failed test cases.
cd MultiWebdriverDemo
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
#this to print current directory.
echo "the second Script executed from: ${PWD}"
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "ok file found pass"
mvn test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml
else
echo "file not found passed in first attempt ..."
exit 0
#exit 0 means forcefully mark it as passed
fi
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "Rerun also failed exiting with buil state as failure "
exit 1
#exit 1 means forcefully mark it as failed
else
echo "file not found rerun is passed marking the build as passed ..."
exit 0
fi
S0 even in the second time, my webdriver test case is failed, it's failed build mark as a failure.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.199 s
[INFO] Finished at: 2021-08-12T14:05:36Z
[INFO] ------------------------------------------------------------------------
+ [ -e ./target/surefire-reports/testng-failed.xml ]
+ echo Rerun also failed exiting with buil state as failure
Rerun also failed exiting with buil state as failure
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **/testng-results.xml
Saving reports...
Processing '/var/jenkins_home/jobs/August-FreeStyle/builds/37/testng/testng-results.xml'
100.000000% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE
TestNG Reports Processing: FINISH
Finished: FAILURE

Conan Artifactory Jenkins Integration fails

We have a problem regarding the Jenkins Artifactory Plug-in using Conan.
Basically, we created our scripted pipeline (following the example in https://github.com/jfrog/project-examples/blob/master/jenkins-examples/pipeline-examples/scripted-examples/conan-example/Jenkinsfile), which set-up the artifactory server, creates the conanClient, performs the creation of a Conan Package (by running through the conanClient the command "conan create", since our project is built via a conanfile.py recipe), uploads the package to our artifactory instance, and finally we want to clear the conan cache (through the "conan remove * -f" command).
But this last step always fails: the problem seems to be caused by the conanClient, which implicity invokes the "conan_build_info" command after the "conan remove * -f". The "conan_build_info" command fails since the "conan_build_info" apparently requires the "package" cache, which is, of course, cleared. Is this a conanClient bug (maybe because, as we read in the documentation, the conan_build_info command is not recommended to be used: https://docs.conan.io/en/latest/reference/commands/misc/conan_build_info.html), or are we missing something?
Is there a way to perform a package creation and upload via the ConanClient, and clear the conan cache without causing the pipeline to fail?
It seems to me this is a big bogus bug, since package creation and upload via CI - Jenkins is a fondumental aspect... and of course, at the end, the conan cache must be cleared...
Here our Jenkinsfile:
node()
{
// Obtain an Artifactory server instance, defined in Jenkins --> Manage:
def server = Artifactory.server "artifactory_server"
// Create a local build-info instance:
def buildInfo = Artifactory.newBuildInfo()
buildInfo.name = "our git master pipeline"
// Create a conan client instance:
def conanClient = Artifactory.newConanClient()
// Add a new repository named 'conan-local' to the conan client.
// The 'remote.add' method returns a 'serverName' string, which is used later in the script:
String serverName = conanClient.remote.add server: server, repo: "conan-local"
// We enable strict ABI dependency propagation
conanClient.run(command: "config set general.default_package_id_mode=package_revision_mode", buildInfo: buildInfo)
conanClient.run(command: "config set general.revisions_enabled=1", buildInfo: buildInfo)
conanClient.run(command: "config set general.full_transitive_package_id=1", buildInfo: buildInfo)
stage('Checkout')
{
// checkout from our repo...
}
stage('Build Release')
{
// Run a conan build. The 'buildInfo' instance is passed as an argument to the 'run' method:
conanClient.run(command: "create ./project_dir channel/channel", buildInfo: buildInfo)
}
stage('Upload')
{
// Create an upload command. The 'serverName' string is used as a conan 'remote', so that the artifacts are uploaded into it:
String command = "upload our_packet/*#*/master --all -r ${serverName} --confirm"
// Run the upload command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage("Clear Conan Cache")
{
// Clean all conan cache
String command = "remove * -f"
// Run the remove command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage('Publish build info')
{
// Publish the build-info to Artifactory:
server.publishBuildInfo buildInfo
}
}
The error from Jenkins log:
[out_project] $ cmd.exe /C "conan remove "*" -f && exit %%ERRORLEVEL%%"
[out_project] $ cmd.exe /C "conan_build_info D:\jenkins\workspace\out_project#tmp\artifactory\conan.tmp8862200414119894885\conan_log.log --output D:\jenkins\workspace\out_project#tmp\artifactory\conan1112119574977543529build-info && exit %%ERRORLEVEL%%"
[1m[31mERROR: [Errno 2] No such file or directory: 'D:\\.conan\\efa5c5\\1\\conaninfo.txt'[0m
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.RuntimeException: Conan build failed
at org.jfrog.hudson.pipeline.common.Utils.launch(Utils.java:256)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execute(ConanExecutor.java:132)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.collectConanBuildInfo(ConanExecutor.java:180)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execCommand(ConanExecutor.java:101)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:50)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:37)
at org.jfrog.hudson.pipeline.ArtifactorySynchronousNonBlockingStepExecution.run(ArtifactorySynchronousNonBlockingStepExecution.java:42)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Getting build failed as username is not readable

In my jenkins file failing at below command
Code :
stage('Release') {
steps {
sh '/opt/maven/bin/mvn --batch-mode release:clean release:prepare release:perform'
}
}
stage('Update Rel') {
steps {
sh 'git push https://xxxx:password#github.com/dxtrsd/maven-multi-module-example.git HEAD:master'
}
Build failure :
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:3.0.0-M1:prepare (default-cli) on project multi: Unable to commit files
[ERROR] Provider message:
[ERROR] The git-push command failed.
[ERROR] Command output:
[ERROR] fatal: could not read Username for 'https://github.com': No such device or address
[ERROR] -> [Help 1]
[ERROR]
Following "Bootstrap your CI with Jenkins and GitHub" from Michael Wanyoike, you need to have entered your GitHub credentials first:
Then you need to select that credential under your URL (the image shows a SSH URL, but in your case, use the HTTPS URL)
I fixed this issue with providing user and password in developer connection url in block in the POM.xml file

Test reports is missing in Jenkins

When my build failed - test report is missing in Jenkins. What do I do wrong?
This is part of my jenkinsfile:
node {
stage('Integration tests') {
git url: "https://$autotestsGitRepo", branch: 'develop',credentialsId: gitlabCredentialsId
sh 'chmod +x gradlew && ./gradlew clean test
step([$class: 'JUnitResultArchiver', testResults:'build/test-results/test/*.xml'])
}
}
This is my jenkins output:
5 tests completed, 5 failed
Task :test FAILED
FAILURE: Build failed with an exception.
What went wrong: Execution failed for task ':test'.
There were failing tests. See the report at: file:///var/jenkins_home/workspace/test2/build/reports/tests/test/index.html
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org
BUILD FAILED in 1m 7s 4 actionable tasks: 4 executed [Pipeline] }
[Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of
Pipeline ERROR: script returned exit code 1 Finished: FAILURE

Jenkins Post Build Task does not work when job is aborted

I have an issue about Post Build Task after an job abort (manual or timeout): the abort generates a java.lang.InterruptedException which seems to be normal. But as Jenkins java machine is stucked how can you run a post task script? I can't.
here is the java exception:
#174 aborted
java.lang.InterruptedException
at java.lang.ProcessImpl.waitFor(Native Method)
at hudson.Proc$LocalProc.join(Proc.java:319)
at hudson.Launcher$ProcStarter.join(Launcher.java:345)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at
hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:710)
at hudson.model.Build$RunnerImpl.build(Build.java:178)
at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:480)
at hudson.model.Run.run(Run.java:1438)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:239)
Here is my jenkins output:
Build was aborted
Aborted by admin
Performing Post build task...
Match found for :. : True
Logical operation result is TRUE
Running script : dir
[workspace] $ cmd /c call C:\Users\xxxx\AppData\Local
\Temp\2hudson5872719646331136420.bat
Exception when executing the batch command : null
Build step 'Post build task' marked build as failure
Finished: ABORTED
I ran a bat script on the server that copied all hudson****.bat from the folder (which is cleaned after the job) during the job execution/aborting so I've been able to run cmd /c call C:\xxxx\hudson5872719646331136420.bat manually from the windows run, it worked fine.
thanks for your help

Resources