I successfully setup a casperjs test exporting a "result.xml" file.
In Jenkins, I execute the following shell command:
casperjs /home/testing-radu/generated-test.js
This produces "results.xml" in my build's workspace.
In post-build actions I added "Publish XUnit test result report". I don't know how and where to setup the path to the .xml file.
When my build console I see the following:
[37;42;1mPASS 2 tests executed in 9.96s, 2 passed, 0 failed. [0m
[32;1mResult log stored in results.xml [0m
[xUnit] [INFO] - Starting to record.
ERROR: Publisher org.jenkinsci.plugins.xunit.XUnitPublisher aborted due to exception
/var/lib/jenkins/jobs/17live2/workspace/generatedJUnitFiles does not exist.
I added full rights to the workspace directory. I tried creating "generatedJUnitFiles" folder and here is what I'm getting:
[37;42;1mPASS 2 tests executed in 5.296s, 2 passed, 0 failed. [0m
[32;1mResult log stored in results.xml [0m
[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Setting the build status to FAILURE
[xUnit] [INFO] - Stopping recording.
Build step 'Publish xUnit test result report' changed build result to FAILURE
Finished: FAILURE
Am I missing something?
Allright, here's the deal:
Under publish XUnit test results I chose JUnit (also setup the pattern to *.xml) and then
created a shell script to be executed by Jenkins:
# Auth tests
for f in auth/*.js ; do casperjs "$f"; done;
# Clean the old results and place the new ones
rm /var/lib/jenkins/jobs/17live2/workspace/*.xml
mv *.xml /var/lib/jenkins/jobs/17live2/workspace/
In Jenkins config I just do: ./run.sh
***You need read/write rights over your folders.
Related
I want to rerun, below shell script in the same project, once my build is i completed with errors, so I can rerun my failed test cases in testng-failed.xml.
FILE=./target/surefire-reports/testng-failed.xml
if test -f "$FILE"; then
echo "$FILE exists."
mvn clean test -Dhttp.proxyHost=proxy-dev.aws.skynet.com -Dhttp.proxyPort=8080 -Dproxy-dev.aws.skynet.com -Dhttps.proxyPort=8080 -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dos_version="10" -Dos="Windows" -Dlocalrun="false" -Dtestxml=FILE
else
echo "$FILE is not there"
fi
But in Jenkins, post build section, I don't see an option to add Execute shell. Please help.
I installed post build plugin , but it's only show for maven projects and not for my free style project.
In free style project, No post steps option.
I managed a walkaround to do this without any plugins and still run in the Jenkins free style project. Add two execute shells and in first shell enter the below shell commands.
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
cd CommonPagesStorage
mvn clean install -DskipTests
cd ..
cd MultiWebdriverDemo
mvn clean test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}
echo "${TestPlanName}is ran"
#Dmaven.test.failure.ignore=true added this so even the script failed it doesn't mark as fail and mark it as unstable and continue.
In the second shell command ,conditionally checks the testng-failed.xml is exists if so run it to run the failed test cases.
cd MultiWebdriverDemo
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
#this to print current directory.
echo "the second Script executed from: ${PWD}"
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "ok file found pass"
mvn test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml
else
echo "file not found passed in first attempt ..."
exit 0
#exit 0 means forcefully mark it as passed
fi
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "Rerun also failed exiting with buil state as failure "
exit 1
#exit 1 means forcefully mark it as failed
else
echo "file not found rerun is passed marking the build as passed ..."
exit 0
fi
S0 even in the second time, my webdriver test case is failed, it's failed build mark as a failure.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.199 s
[INFO] Finished at: 2021-08-12T14:05:36Z
[INFO] ------------------------------------------------------------------------
+ [ -e ./target/surefire-reports/testng-failed.xml ]
+ echo Rerun also failed exiting with buil state as failure
Rerun also failed exiting with buil state as failure
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **/testng-results.xml
Saving reports...
Processing '/var/jenkins_home/jobs/August-FreeStyle/builds/37/testng/testng-results.xml'
100.000000% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE
TestNG Reports Processing: FINISH
Finished: FAILURE
There is a similar issue here but with no answers and no progress in solving it.
I am running Selenium tests with Cucumber and with Gradle as the build tool. In Jenkins.
This is my build.gradle file:
task cucumber() {
dependsOn assemble, compileTestJava
doLast{
javaexec {
main = "cucumber.api.cli.Main"
classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
args = ['--plugin', 'pretty', '--glue', 'stepmethods', 'src/main/java/features', '--tags', 'not #proba and not #test and not #rucno and not #nedovrseno']
}
}
}
I wanted to exclude some scenarios from the feature I am testing so I added the '--tags', 'not #proba and not #test and not #rucno and not #nedovrseno' part to Gradle task arguments.
After that (and I'm not 100% sure this is the cause) tests run fine in a Jenkins job, but the reports with Cucumber reports plugin are not getting generated.
Instead, this is the Jenkins console output:
15:09:02 BUILD SUCCESSFUL in 1m 58s
15:09:02 3 actionable tasks: 3 executed
15:09:02 Build step 'Invoke Gradle script' changed build result to SUCCESS
15:09:02 [CucumberReport] Using Cucumber Reports version 4.6.0
15:09:02 [CucumberReport] JSON report directory is ""
15:09:02 [CucumberReport] Copied 1 json files from workspace "C:\Users\me\.jenkins\workspace\Project Name" to reports directory "C:\Users\me\.jenkins\jobs\Project Name\builds\12\cucumber-html-reports\.cache"
15:09:02 [CucumberReport] Copied 4 properties files from workspace "C:\Users\me\.jenkins\workspace\Project Name" to reports directory "C:\Users\me\.jenkins\jobs\Project Name\builds\12\cucumber-html-reports\.cache"
15:09:02 [CucumberReport] Processing 1 json files:
15:09:02 [CucumberReport] C:\Users\me\.jenkins\jobs\Project Name\builds\12\cucumber-html-reports\.cache\report.json
15:09:02 [CucumberReport] Missing report result - report was not successfully completed
15:09:02 [CucumberReport] Build status is left unchanged
15:09:02 Finished: SUCCESS
So, the files are processed, but the report is missing. What could be the problem?
EDIT
Added second '--plugin' to string to account for the OP's finding.
Removed additional examples.
Try
task cucumber() {
dependsOn assemble, compileTestJava
doLast{
javaexec {
main = "cucumber.api.cli.Main"
classpath = configurations.cucumberRuntime + sourceSets.main.output + sourceSets.test.output
args = ['--plugin', 'pretty', '--plugin', 'html:some/dir', '--glue', 'stepmethods', 'src/main/java/features', '--tags', 'not #proba and not #test and not #rucno and not #nedovrseno']
}
}
}
You didn't specify which report you wanted nor where you wanted it generated. This gives you pretty and html.
Yesterday, I wrote and ran a Katalon test suite, and today, I'm trying to integrate Katalon with Jenkins. I successfully setup Jenkins, created a new job for the Katalon testing, as per these instructions, but when I went to build it, I get failing builds.
In particular, this is the error message I keep getting :
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE
I went ahead and copied the Reports folder structure from the project directory that I specified to the Jenkins workspace. Upon later inspection, I found that, when Jenkins was running the Katalon tests, the JUnit_Report.xml file was actually getting created in the project's Reports folder, instead of at %JENKINS_HOME%\workspace\[project name]\Reports. I explicitly told it to generate test reports to : Reports/LoginSuite/*/JUnit_Report.xml.
NOTE: I'm on a Windows machine.
How can I fix this so that I can display test results from Jenkins?!
UPDATE : I have revised my Windows shell code to the following
C:
cd C:\Katalon
katalon -runMode=console -projectPath="C:\Users\mwarren\Katalon Studio\TestProject" -reportFolder="../../.jenkins/workspace/Katalon Studio Tests/Reports" -reportFileName="report" -retry=0 -testSuitePath="Test Suites/LoginSuite" -browserType="Chrome"
and it's still giving me the same error, even though now the tests are being created there.
Report folder is generated in the jenkins job folder.
Reports/**/JUnit_Report.xml
PEBKAC, apparently. I should have, from the getgo, listened to Jenkins and set my Test Report XMLs as */JUnit_Report.xml
Copy all reports to a temp folder, rename each xml with test case name and then copy it back to junit folder.
testCasesTxt = sh (
script: 'sudo find $WORKSPACE -name "*.ts*" -type f -printf "%f\n"',
returnStdout: true
).trim()
testCasesTxt = testCasesTxt.replace(".ts", "")
testCases = testCasesTxt.split("\n")
for (int i = 0; i < testCases.size(); i++) {
script {
try {
wrap([$class: 'Xvfb']) {
sh """
cd /opt/katalon
./katalon -noSplash -consoleLog -runMode=console -projectPath=$WORKSPACE/"katalon-project.prj" -reportFolder="Reports" -reportFileName="report" -retry=0 -testSuitePath="Test Suites/${testCases[i]}" -executionProfile="qa" -browserType="Chrome"
"""
}
} catch (any) {
currentBuild.result = 'FAILURE'
throw any //rethrow exception to prevent the build from proceeding
} finally {
sh """
cd /home/environment/tmp/
cd Reports
mkdir ${testCases[i]}
cd $WORKSPACE
cp -r Reports/ /home/environment/tmp/Reports/${testCases[i]}
cd /home/environment/tmp/Reports/${testCases[i]}/Reports
mv JUnit_Report.xml JUnit_Report_${testCases[i]}.xml
cd $WORKSPACE
cp -r "Data Files/" "/home/environment/tmp/"
"""
//
}
}
}
According to this thread, this can happen when trying to execute two Katalon instances at the same moment.
If that's the case, try changing Jenkins number of executors 1.
If your report is at \Reports\20200611_172240\TestSuite1\20200611_172240/JUnit_Report.xml location in your project folder, then configure the folder path as below - /Reports///*/JUnit_Report.xml
as the 3 folders after report folder name are always going to change after each execution.
Please use the Test report XMLs path as like this
**/target/surefire-reports/*.xml
We have a CI pipeline on Jenkins. Under test is a REST api running in a Docker container. It is tested with frisby.js which also uses jasmine.
Test are successful
Finished in 0.168 seconds
1 test, 4 assertions, 0 failures, 0 skipped
but xUnit sets the build status to FAILURE:
[xUnit] [INFO] - Converting '/path/to/TEST-FrisbyTestMyApiTest.xml' .
[xUnit] [INFO] - Check 'Failed Tests' threshold.
[xUnit] [INFO] - Check 'Skipped Tests' threshold.
[xUnit] [INFO] - Setting the build status to FAILURE
Why is that? How can I get it to be SUCCESS?
The paths are validated and correct. The tests run successful. I'm running the Jenkins xUnit plugin with JUnit pattern. All xUnit thresholds are set to 0.
I have an issue about Post Build Task after an job abort (manual or timeout): the abort generates a java.lang.InterruptedException which seems to be normal. But as Jenkins java machine is stucked how can you run a post task script? I can't.
here is the java exception:
#174 aborted
java.lang.InterruptedException
at java.lang.ProcessImpl.waitFor(Native Method)
at hudson.Proc$LocalProc.join(Proc.java:319)
at hudson.Launcher$ProcStarter.join(Launcher.java:345)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:82)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:58)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at
hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:710)
at hudson.model.Build$RunnerImpl.build(Build.java:178)
at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:480)
at hudson.model.Run.run(Run.java:1438)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:239)
Here is my jenkins output:
Build was aborted
Aborted by admin
Performing Post build task...
Match found for :. : True
Logical operation result is TRUE
Running script : dir
[workspace] $ cmd /c call C:\Users\xxxx\AppData\Local
\Temp\2hudson5872719646331136420.bat
Exception when executing the batch command : null
Build step 'Post build task' marked build as failure
Finished: ABORTED
I ran a bat script on the server that copied all hudson****.bat from the folder (which is cleaned after the job) during the job execution/aborting so I've been able to run cmd /c call C:\xxxx\hudson5872719646331136420.bat manually from the windows run, it worked fine.
thanks for your help