What I am looking for is basically the same functionality that Timestamper provides, but in addition to showing the time for each line, I also want to see which node it was running on.
Instead of this:
13:35:12 > Task :compileJava NO-SOURCE
13:35:12 > Task :processResources NO-SOURCE
13:35:12 > Task :classes UP-TO-DATE
13:35:12 > Task :compileTestJava NO-SOURCE
13:35:12 > Task :processTestResources NO-SOURCE
13:35:12 > Task :testClasses UP-TO-DATE
13:36:39 fatalError
I want this (or similar):
13:35:12 [mac-01] > Task :compileJava NO-SOURCE
13:35:12 [mac-01] > Task :processResources NO-SOURCE
13:35:12 [mac-01] > Task :classes UP-TO-DATE
13:35:12 [mac-01] > Task :compileTestJava NO-SOURCE
13:35:12 [mac-01] > Task :processTestResources NO-SOURCE
13:35:12 [mac-01] > Task :testClasses UP-TO-DATE
13:36:39 [mac-01] fatalError
We have pipelines that run multiple builds on multipe nodes in parallel and sometimes they fail due to problems with a specific node. If each line in the log was prefixed with the node it was running on, it would be quite trivial to find out which node is having an issue. As it stands now, it can be rather challenging.
I've been trying to search for something like this on Google, but the terms are a bit too generic and I'm not getting any relevant results.
What you are looking for is label attribute.
This label attribute is available for all bat, sh, powershell and others.
https://www.jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
bat( script: ```Some Script ```,
label: "${env.NODE_NAME}"
)
Your execution will then be pre fixed with the NODE_NAME where the task is running.
Related
I want to rerun, below shell script in the same project, once my build is i completed with errors, so I can rerun my failed test cases in testng-failed.xml.
FILE=./target/surefire-reports/testng-failed.xml
if test -f "$FILE"; then
echo "$FILE exists."
mvn clean test -Dhttp.proxyHost=proxy-dev.aws.skynet.com -Dhttp.proxyPort=8080 -Dproxy-dev.aws.skynet.com -Dhttps.proxyPort=8080 -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dos_version="10" -Dos="Windows" -Dlocalrun="false" -Dtestxml=FILE
else
echo "$FILE is not there"
fi
But in Jenkins, post build section, I don't see an option to add Execute shell. Please help.
I installed post build plugin , but it's only show for maven projects and not for my free style project.
In free style project, No post steps option.
I managed a walkaround to do this without any plugins and still run in the Jenkins free style project. Add two execute shells and in first shell enter the below shell commands.
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
cd CommonPagesStorage
mvn clean install -DskipTests
cd ..
cd MultiWebdriverDemo
mvn clean test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}
echo "${TestPlanName}is ran"
#Dmaven.test.failure.ignore=true added this so even the script failed it doesn't mark as fail and mark it as unstable and continue.
In the second shell command ,conditionally checks the testng-failed.xml is exists if so run it to run the failed test cases.
cd MultiWebdriverDemo
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
#this to print current directory.
echo "the second Script executed from: ${PWD}"
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "ok file found pass"
mvn test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml
else
echo "file not found passed in first attempt ..."
exit 0
#exit 0 means forcefully mark it as passed
fi
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "Rerun also failed exiting with buil state as failure "
exit 1
#exit 1 means forcefully mark it as failed
else
echo "file not found rerun is passed marking the build as passed ..."
exit 0
fi
S0 even in the second time, my webdriver test case is failed, it's failed build mark as a failure.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.199 s
[INFO] Finished at: 2021-08-12T14:05:36Z
[INFO] ------------------------------------------------------------------------
+ [ -e ./target/surefire-reports/testng-failed.xml ]
+ echo Rerun also failed exiting with buil state as failure
Rerun also failed exiting with buil state as failure
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **/testng-results.xml
Saving reports...
Processing '/var/jenkins_home/jobs/August-FreeStyle/builds/37/testng/testng-results.xml'
100.000000% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE
TestNG Reports Processing: FINISH
Finished: FAILURE
I am trying to setup https://spring.io/guides/gs/spring-boot-docker/#initial with my environment, which has a private registry server #
el-qa-docker.x.x.x:18445
my build.gradle has the following relevant content:
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:1.5.3.RELEASE")
classpath('se.transmode.gradle:gradle-docker:1.2')
classpath ('org.codehaus.groovy:groovy-backports-compat23:2.3.5')
}
group = 'james'
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'
apply plugin: 'docker'
task buildDocker(type: Docker, dependsOn: build) {
push = true
applicationName = jar.baseName
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}
when I run: $gradle build buildDock
I get build FAILURE, with:
$ gradle build buildDock
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Docker execution failed
Command line [docker build -t skahmed/gs-spring-boot-docker:latest /Users/skahmed/devops/TOOLS/DOCKER/gs-spring-boot-docker/initial/build/docker] returned:
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 16.243 secs
any ideas, on where do I specify docker's internal registry which can be used with Gradle.
I tried:
docker {
useApi true
hostUrl 'el-qa-docker.x.x.x:18445'
apiUsername 'james'
}
as provided by https://github.com/Transmode/gradle-docker
but that gives a different error:
$ gradle build buildDock
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Port is invalid: -1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1.389 secs
I tried : build.gradle
docker {
//useApi true
hostUrl 'el2-dt-docker.uscis.dhs.gov:18445'
apiUsername 'skahmed'
}
and
task buildDocker(type: Docker, dependsOn: build) {
push = true
applicationName = jar.baseName
//applicationName = 'el2-dt-docker.uscis.dhs.gov:18445/skahmed/gs-spring-boot-docker'
//applicationName = 'gs-spring-boot-docker'
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}
but get:
:buildDocker FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildDocker'.
> Docker execution failed
Command line [docker push skahmed/gs-spring-boot-docker:latest] returned:
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
You can specify your private Docker registry as part of your Docker image name e.g.:
el-qa-docker.x.x.x:18445/skahmed/gs-spring-boot-docker
Please make sure your Docker daemon is aware about private registry.
I have followed instructions from here in order to add some global functions to my Jenkins Pipelines. I followed the directory structure provided and I load my code through an external git repository not the one provided by Jenkins itself. Although I can add some functions under src directory and are loaded correctly as explained, I cannot load the scripts added under vars directory. Is there any way to understand what is happening. My example groovy script is very simple:
// vars/hello.groovy
def call(name) {
echo "Hello world, ${name}"
}
and i try to invoke it as follows
hello "foo"
I get the following:
> java.lang.NoSuchMethodError: No such DSL method 'hello' found among
> [archive, bat, build, catchError, checkout, deleteDir, dir, echo,
> error, fileExists, git, input, isUnix, load, mail, node, parallel,
> properties, pwd, readFile, retry, sh, sleep, stage, stash, step, svn,
> timeout, tool, unarchive, unstash, waitUntil, withEnv, wrap,
> writeFile, ws] at
> org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:107) at
> org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
> at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source) at
> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
> at
> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151)
>
>
>
Has anyone faced any similar issue and how I can debug what is going on?
My Jenkins version is: 1.625.2
EDIT#1:
After as well reading this indeed after a restart the script works, but this is not efficient to restart on each change. Is there any other solution to load the vars/*.groovy scripts without a restart?
I'm puzzled why Gradle's uploadArchive is uploading artifacts from other configurations. My understanding is that when you declare a configuration it will get an upload task for you and when called it will upload the artifacts assigned to its configuration.
That's not the behavior I'm seeing:
apply plugin: 'base'
configurations {
foo
}
task fooIt(type: Zip) {
from 'blah.txt'
baseName 'foo'
}
task barIt(type: Zip) {
from 'blah.txt'
baseName 'bar'
}
artifacts {
foo fooIt
}
repositories {
flatDir {
name 'local'
dirs 'repo'
}
}
uploadArchives {
repositories {
add project.repositories.local
}
}
uploadFoo {
repositories {
add project.repositories.local
}
}
In this example there are no artifacts assigned to the archives configuration but when I call gradle uploadArchives it will upload foo's artifacts.
$ gradle -i uploadArchives
All projects evaluated.
Selected primary task 'uploadArchives' from project :
Tasks to be executed: [task ':fooIt', task ':uploadArchives']
:fooIt (Thread[Daemon Thread 17,5,main]) started.
:fooIt
Skipping task ':fooIt' as it is up-to-date (took 0.008 secs).
:fooIt UP-TO-DATE
:fooIt (Thread[Daemon Thread 17,5,main]) completed. Took 0.017 secs.
:uploadArchives (Thread[Daemon Thread 17,5,main]) started.
:uploadArchives
Executing task ':uploadArchives' (up-to-date check took 0.0 secs) due to:
Task has not declared any outputs.
Publishing configuration: configuration ':archives'
Publishing to Repository 'local'
Published :gradle:unspecified:foo.zip to file:/private/tmp/gradle/repo/foo-unspecified.zip
Published :gradle:unspecified:ivy.xml to file:/private/tmp/gradle/repo/ivy-unspecified.xml
:uploadArchives (Thread[Daemon Thread 17,5,main]) completed. Took 0.017 secs.
BUILD SUCCESSFUL
Two questions out of this:
Why is fooIt getting executed?
Why is uploadArchives uploading foo's artifacts?
Thanks
$ gradle --version
------------------------------------------------------------
Gradle 2.1
------------------------------------------------------------
Build time: 2014-09-08 10:40:39 UTC
Build number: none
Revision: e6cf70745ac11fa943e19294d19a2c527a669a53
Groovy: 2.3.6
Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013
JVM: 1.7.0_45 (Oracle Corporation 24.45-b08)
OS: Mac OS X 10.10.2 x86_64
The archives configuration contains all artifacts from all configurations. I believe the confusion comes from the fact that you can also add artifacts directly to the archives configuration if you want. That being the case, the uploadArchives task will always upload all the declared artifacts. If you want to upload a subset of your artifacts then you should call the upload<<ConfigurationName>> task.
I have a gradle task that has a dependsOn and then needs to execute an Ant task called runcukes. Gradle won't call the ant task, however:
ant.importBuild 'build.xml'
task runCukes(dependsOn: restoreSchema) {
runcukes
}
Running gradle with -d shows that gradle doesn't recognize the Ant target runcukes:
Finished executing task ':restoreSchema'
17:28:37.506 [LIFECYCLE] [org.gradle.TaskExecutionLogger] :runCukes
17:28:37.506 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter ]
Starting to execute task ':runCukes'
17:28:37.507 [INFO] [org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter ]
Skipping task ':runCukes' as it has no actions.
17:28:37.507 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter
Gradle doesn't recognize that it has to execute the ant target runcukes.
The gradle docs don't show anything as complicated as a task that has a dependsOn that then executes an ant task. (They only show "Hello World" examples as dependencies.)
What do I have to do to get Gradle to execute an Ant task after executing a dependsOn?
Thanks.
Just putting the Ant task name into the configuration block of a task won't execute it. This should do what you want if I understood your requirements correctly:
ant.importBuild 'build.xml'
runcukes.dependsOn restoreSchema
task runCukes(dependsOn: runcukes)