I am running sitespeed.io tests in Jenkins, and have configured it to output junit format test results.
I'd now like to publish those test results to SonarQube (I realise I can publish them in Jenkins, but I have a requirement to keep everything in one place as much as possible).
However, when I add the test results file into the Sonar analysis (using sonar.junit.reportsPath=/path/to/sitespeed-results, having named my results file TESTS-sitespeed.xml) SonarQube doesn't seem to show any results on its dashboard.
I understand that SonarQube also has a setting to configure the location of test files, and this is often cited as a reason for test results not being ingested correctly, and that leads me to wonder whether what I'm trying to do is possible.
Any help would be greatly appreciated.
Thanks,
Jez
the property sonar.junit.reportsPath will read and parse the report but will only save the information if the class name indicated by the report can be mapped back to a java resource of the project.
I have no idea about the output of your test in surefire format but I sense the classname won't match any resource of your project and so your report is parsed but data is ignored.
Related
I have a set of tests which run within Jenkins and at the end of the tests (in the post section) I generate an allure report. The tests are created with Webdriverio.
The last time I did anything like this, I was using standard selenium running with Junit and due to that combination, the Jenkins Rest API had access to things like failedCount, skipCount, passes and failed counts. This however seems to only be possible with JUnit.
Does allure expose this to Jenkins somehow - or am I going to have to try and scrape the statistics from a pre-generated report?
(as a side note, I have tried to configure a second reporter in webdriverio, but while that seems to create some xml files locally, the folder I configure is not appearing on Jenkins. Also, webdriverio doesn't support JUnit as a framework - currently it's using mocha.)
I eventually managed to figure this out.
There is url which you can go to to get a json file with the summary information.
So if you report is at /allure - then you navigate to /widgets/summary.json
I am using jenkins to run jobs that uses maven with cucumber . my jobs are running succesfully. I have also managed the plugin Cucumber report in jenkins. But when i view the report i am getting the message as shown in below picture.
I have tried to install various plugins like Jenkins plugin, standalone but with no joy. These are shown as options in the bottom right of the report.
Any clue is helpful. I have tried to check this error in stackover flow also. I didnt get any clue
I could see the reports file in xml which is created by surefire plugin. Any way i can achive reporting part in my project
I managed to get reports in jenkins now.
Previously Jenkins is unable to find the json file in my project.
I have set the path of json file as target and i could able to see the reports.
This might help for those who are looking for answer.
#CucumberOptions(
features= "C:\\Users\\dd pc\\workspace\\PracticeCucumber\\src\\main\\java\\com\\qa\\feature\\Test.feature",
glue= {"com\\qa\\stepDefinition"},
format= {"pretty","html:test-output_1",
"json:target/cucumber-reports/CucumberTestReport.json"},
plugin = ("json:target/cucumber-reports/CucumberTestReport.json"),
monochrome= true,
dryRun= false
)
I am able to solve by providing **/*.json
under Post-build Actions/Cucumber Report/JSON Report Location/ File Include Pattern = **/*.json
First of all in runner file we need to provide following plugin as follows: plugin= {"json:target/cucumber.json" }
Now in Jenkins job, in "post build" section during configuration, add cucumber reports, click on advanced option and provide 'target' text in "JSON Reports Path" text field. Save the job, it will run and will work fine.
Note: target text is the name of folder which we have provided in runner file for generating reports in Json, also cucumber report plugin works on json so need to generate report in json.
I my case the problem was in two jobs over the same test repository (one for Chrome one for Firefox), where one deletes report files after the job while second job was already running and vice versa.
I am building a Grails 3 application using Atlassian Bamboo. I've configured the Grails 3 task as explained in my answer to this question.
I am having trouble getting Bamboo to find my test reports. On the Grails task type, you can specify if the task produces test results, and if so, where they come from. So I have tried the following:
Look in the standard test results directory
Specify custom results directories:
**/*reports/*.xml (which is the default when you choose Specify...)
build/reports/tests/index.html
All three produce this error:
Could not find test result reports in the /home/bamboo/bamboo-agent-home/xml-data/build-dir/CSSO-MPA4-JOB1 directory.
The correct answer is build/test-results/*.xml, as that is where JUnit puts its test reports.
**/*-reports/*.xml fails because it is in -results.
build/reports/tests/index.html fails because, as it says right above the input box:
Test output must be in JUnit XML format.
I discovered (from somewhere in this thread) that JUnit is responsible for generating the test results independent of Gradle / Grails, which is how I found they were in an odd place.
My Jenkins job runs many tests that create log files. In case of failure, I want to look at the log of the failed test. I'd rather use Jenkins web-server to do it, even have a link in the email it sends me.
Is there any plugin that can do it? Or maybe another way?
You provide few details in your question, so it is impossible to give specific advice. In a general level: this is already possible. When your test framework creates JUnit XML files with test results, the test output can be included between the <failure> and </failure> tags. Usually test frameworks should take care of this automatically, so you are probably not using a test framework and are manually generating the XML files containing test results?
I recommend you adopt some test framework. It is usually well worth the effort.
My company is developing a web application that builds in ant. I've been tasked with getting CruiseControl.net to differentiate between a build failure and a unit test failure, which it can't do natively. ( It currently lumps both together but doesn't help developers understand what's broken )
I have CC.net call a script that returns specific exit codes depending on the nature of an ant task failure. I'd like these exit codes to be reflected in the CC.net failure report / dashboard but am having some trouble finding resources on how this might be done.
Any suggestions?
Not directly. All the reports and display works from information in the logs which are XML files. The display and reports work by applying XSLT to these XML files.
Take a look at your build logs and unit test logs, to see if each of those process write the failure information to their respective log files.
If they do, you should be able to write a custom XSLT or modify the existing XSLT to display that information.
Edit:
A different approach based on your comment. You could probably redirect the ANT error code to a file. Then you could have a seperate ccnet task that takes the error code from that file and re-format and display it (depending on how/where you want it displayed)