Allure Jenkins behave BDD report generating blank report - jenkins

I have integrated my Acceptance testing Python Behave BDD with Jenkins and wanted to generate HTML report using Allure report. I am not seeing correct results on Allure report, as all are blank/null.
I am getting the below message from jenkins console however the folders are empty and report has no data.
C:\Program Files (x86)\Jenkins\workspace\Scs Testing>cd C:\checkouts\scs\test
C:\checkouts\scs\test>"behave --tags=#defrost_007 -f allure_behave.formatter:AllureFormatter -o C:\checkouts\scs\test\TestResults scs_acceptance_test/ "
1 feature passed, 0 failed, 1 skipped
1 scenario passed, 0 failed, 98 skipped
5 steps passed, 0 failed, 356 skipped, 0 undefined
Took 0m9.663s
C:\checkouts\scs\test>exit 0
[Scs Testing] $ "C:\Program Files (x86)\Jenkins\tools\ru.yandex.qatools.allure.jenkins.tools.AllureCommandlineInstallation\allure_2\allure-2.5.0\bin\allure.bat" generate -o "C:\Program Files (x86)\Jenkins\workspace\Scs Testing\allure-report"
allure-results does not exists
Report successfully generated to C:\Program Files (x86)\Jenkins\workspace\Scs Testing\allure-report
Allure report was successfully generated.
Creating artifact for the build.
Artifact was added to the build.
Finished: SUCCESS
Could you please give me some input?
First image - there is a zip file creating for each build however it is not able to extract it

I am myself a newbie in allure. My OS is Ubuntu. So my answer is based on what I have seen and observed on my OS.
But since this question is still open I will put in my two bits:
You report is empty because the report format is incorrect. What
happens behind the scene is when you ask behave to use allure as a
reporter tool, it creates a JSON file for the test output. In your
case you can see the output on console. JSON should be generated on
path:
C:\checkouts\scs\test\TestResults scs_acceptance_test\
Then when you give command allure generate, allure will read the
JSON file generated in previous step and create a new folder called
reports on path:
C:\checkouts\scs\test\TestResults scs_acceptance_test\reports
in which it generates and places the HTML code and serves the folder
(reports) on a server that is starts.
Your command:
~allure.bat generate -o "C:\Program Files
(x86)\Jenkins\workspace\Scs Testing\allure-report"
is incorrect. The format is allure [path-to-directory-where-you-have-your-json-file] -o [path-to-directory-where-you-want-to-generate-your-reports-directory]

I have been facing the same issue for a while. I somehow figured out a solution.
My windows batch execution command is:
pytest -s -v Test_practice_file.py --alluredir=allure-results
After updating my allure reports folder n number of times, I updated the Post-build Actions for allure report as:
-o ./allure-results
In Allure Commandline, I have selected "From Maven Central">version 2.9.0
Select the check box "Install automatically"
Save all the configurations and run your script

Related

Jenkins TestNG plugin does not fetch results from the test-results.xml file

In my project workspace, the test-results.xml file exists inside the target\surefire-reports\testng-results.xml directory. But Jenkins fails to read the XML file and gives below error on console.
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **\target\surefire-reports\testng-results.xml
Did not find any matching files.
To ensure the file isn't too old, I had checked that the test-results.xml (and other files) belong to the latest test run. The Jenkins server is running on Ubuntu 14.04 LTS.
I'm running my tests in this manner: My project root directory has a run_tests.sh script which looks like this:
#!/bin/bash
if [ "$1" = "" ]; then
echo "Please provide a valid suite XML file name."
else
mvn clean
mvn compile
mvn clean test -Dsurefire.suiteXmlFiles="$1"
fi
I just pass the suite XML file name as a parameter to this script in Jenkins (execute shell).
Please help.
I found the solution for this.
Go to Configure of your Job
in General Tab, you may find
Advanced Button, Click on this
Check the check box of "Use custom
workspace", under this you see the Directory text box, here you copy
your Selenium Workspace Folder, for example mine is
"E:\eclipse\eclipse-workspace\WebDriveTest\"
Scroll down the page
under the Post-build Actions, Publish TestNG Results, TestNG XML
report pattern : give like this
"**/target/surefire-reports/testng-results.xml" (check this path in
the same workspace).
I hope this will help you!.
You should be using / instead of \ (since you mentioned that your Jenkins is running on a UNIX box)
Krishnan, in the testng-users Google Group, pointed out that it could be an issue with my Jenkins project workspace, and it was the same.
I changed the default workspace in my Jenkins project.
So I've added the path "$HOME/myWorkspace/myProject/" in my Jenkins project workspace, and "**/target/surefire-reports/testng-results.xml" in my TestNG setting in the same Jenkins project, and it works!
Thank you Krishan for your help.
Please see my answer in another post here, it should be very clear.
In short, it is caused by the current directory was changed to the default Jenkins workspace, you need set your custom workspace in the Job's Config.
I agree with Krishnan Mahadevan usage of '\' instead of '/' while providing the path for TestNG Report also solved my problem.
Extremely important thing to note here:
When providing path for Root POM in the build section '\' is used
C:\Users\harsh\eclipse-workspace\ProjTwo3\pom.xml
When providing path for TestNG XML report pattern in Publish TestNG Result section
'/' is used
C:/Users/harsh/eclipse-workspace/ProjTwo3/target/surefire-reports/testng-results.xml
Console Output:
channel stopped
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern:
C:/Users/harsh/eclipse-workspace/ProjTwo3/target/surefire-reports/testng-results.xml
Saving reports...
Processing 'C:\Users\harsh.jenkins\jobs\MyApplication\builds\12\testng\testng-results.xml'
11.688312% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE

Jenkins - failed: No test report files were found. Configuration error?

I'm beginner for "Jenkins" and following this tutorial.
At the Sixth step I got below error.
xcodebuild: error: Scheme JenkinsTest is not currently configured for the test action.
Build step 'Xcode' marked build as failure
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE
In the Test report XMLs I did set "test-reports/.*xml"
I tried to find my solution and also many questions are founded on SO like same issue I have too but did not get solution.
I have some confusion, Is .xml file automatically generated by "Jenkins" or First we manually need to add .xml file ?
In short guide me on right direction based on above error.
You can also enable the 'allowEmptyResults' option so that the junit plugin won't throw an exception when it doesn't find test results.
This way it doesn't matter if the 'test-results' directory exists.
junit allowEmptyResults: true, testResults: '**/test-results/*.xml'
First make sure junit.xml is getting generated after the test run.
Jenkins job at times cannot see past the current workspace. so it is always a good idea to copy the reports back to the current workspace before using it.
cd <path to report>
cp *.xml $WORKSPACE
Now the jenkins should pick-up the report.
Note: The config may show error first(since it cannot find the xml file in workspace) but after a build this should go away and also the result should get recorded
I'm using nosetest (python) to generate an xUnit compatible file and I was getting:
ERROR: No test report files were found. Configuration error?
I was using junit plugin as:
junit "test-results-unit.xml"
junit seems to add WORKSPACE directory by default so using the full PATH to the file wouldn't work either. I created symlink from the resulting file to the WORKSPACE directory to make it work:
sh 'ln -s tests/test-results-unit.xml $WORKSPACE'
junit "test-results-unit.xml"
Other answers suggest copying the files to the workspace directory, but for me simply changing the path to start with '**' seemed to solve the issue
junit '**/test-reports/*.xml'
The Jenkins junit plugin page says that you need to "specify the path to JUnit XML files in the Ant glob syntax". I've not dug into the full details of what this means, but starting the path with '**' was enough to get it working for me.
Thanks #Acid, it really helped me.
First, copy the module/build/test-results to workspace directory
cp -r app/build/test-results $WORKSPACE/test-results
And then I used this wildcard path
**/test-results/**/*.xml
I had the same problem and my test report file name got changed due to an upgrade in scala version,
Hence I have to change from,
junit '/myworkspace/target/myreport.xml'
to
junit '/myworkspace/target/TEST-myreport.xml'
How about:
junit allowEmptyResults: true, testResults: "${WORKSPACE}/test-results/*.xml"
or just:
junit "${WORKSPACE}/test-results/*.xml"
You have to find the path where your test reports are saved in your .jenkins workspace:
/Users/<USERNAME/>/.jenkins/workspace/<PROJECT/>/target/surefire-reports/*.xml
to find the folder type in your terminal:
.jenkins
then
open .

Soap UI jenkins report

I am trying to get a step by step soap tests report into jenkins. I have added the soapui testrunner in the jenkins command link, see below:
start "" "C:\Program Files (x86)\SmartBear\SoapUI-5.2.1\bin\testrunner.bat" -r -a -j -"C:\Users\cverma\Desktop\QA-soapui-project.xml"
I would like to know where does it save the junit report? I would like to publish the report in jenkins. I am adding the *.xml into the post build junit unit report but no luck with the report...
You can use Summary Display Plugin for publishing the test report, see https://wiki.jenkins-ci.org/display/JENKINS/Summary+Display+Plugin.
The report will be stored in the workspace or in the execution directory.

How to make cpplint work with Jenkins warnings plugin

I added an 'Execute shell' build step in Jenkins to run the cpplint.py
python /var/lib/jenkins/scripts/cpplint.py --counting=detailed `find path -name *.cpp
I also added 'Scan for compiler warnings' and added CppLint.
However it always gets 0 warnings even though it displayed in the Console output some warnings such as
filename.cpp:18: Missing space after , [whitespace/comma] [3]
If you run cpplint.py with --output=vs7 it will produce the format expected by the Jenkins warnings plugin.
I use the Cppcheck Plugin and cpplint_to_cppcheckxml.py to convert cpplint.py output to the XML format expected by Cppcheck Plugin. This works really well. I can click on the offending issue in Cppcheck Results displayed on the Jenkins job page and it will display the source code with offending line highlighted. Very cool.
You must provide cpplint an absolute path to your source code directory in order for the hyperlink generation to work on Cppcheck Results page. The only con I see is your Cppcheck and cpplint results are combined and not separated.
If using Linux bash scripts, here is how I turn a relative path into absolute in order for the cpplint to generate absolute paths in it's output:
# Build cpplint reports and transform to cppcheck compatible XML format
# Convert relative path to absolute path so that Jenkins job can easily display the source code errors
srcPathAbsolute=${PWD}/../dicegame/src/main
srcPathAbsolute=$(readlink -f ${srcPathAbsolute})
cpplint.py --counting=detailed ${srcPathAbsolute}/*.cpp 2>&1| cpplint_to_cppcheckxml.py &> cpplint-cppcheck-result.xml
In my Jenkins job configuration to locate both ccplint-cppcheck-result.xml files and my normal cppcheck-result.xml.
Publish cppcheck results
Cppcheck report XMLs **/*cppcheck-result.xml
Thanks to original developer of cpplint_to_cppcheckxml.py. This script serves as an example on how to connect output of other tools into existing Jenkins plugins. Very nice!

Getting output from remote hudson builds

I have recently added several Hudson agent machines to a working controller.
Some of the builds have result files (Doxygen output, Boost unit test results, cppcheck) which are published using the relevant plugins.
If these builds run on the controller, the output files are published successfully, but if they run on an agent, the build fails because the files cannot be found.
The plugins seem to be looking for the output files on the controller, when of course, they are on the agent.
Is there a way to fix this?
Plugins used:
- Jenkins Cppcheck Plug-in V1.2
- Doxygen Plugin V0.9
- HTML Publisher plugin V0.6
Edit:
Tried archiving suggestion, but I can't get it to work. cppcheck.xml is archived correctly and exists in the relative directory ../lastSuccessful/archive/cppcheck.xml
Hudson log:
Archiving artifacts
[Cppcheck] Starting the cppcheck analysis.
[Cppcheck] No cppcheck test report file(s) were found with the pattern
'../lastSuccessful/archive/cppcheck.xml' relative to
'/home/hudson/.hudson/jobs/1.11-IntegrationDebug/workspace'.
Did you enter a pattern relative to the correct directory?
Did you generate the XML report(s) for Cppcheck?
[Cppcheck] Parsing throws exceptions. No cppcheck test report file(s) were
found with the pattern '../lastSuccessful/archive/cppcheck.xml' relative to
'/home/hudson/.hudson/jobs/1.11-IntegrationDebug/workspace'.
Workaround: copy artifacts into your workspace and publish from there.

Resources