I am using kotlintest with gradle. When I run my tests in IntelliJ I get to see the whole Given/When/Then structure. This is also reflected in the report I can generate using the IntelliJ Export Test Results function. This works perfectly and does exactly what I want.
Sadly, when I run the tests using gradlew test it only picks up the Then blocks. That means that while all tests are run correctly, the generated report looks ugly.
The problem is that I want to generate an HTML report automatically in a Jenkins pipeline. At the moment the result is useless, because it does not contain the Given and When blocks.
Is it possible to somehow generate a complete HTML report, with all Given/When/Then blocks in a Jenkins pipeline?
Related
I have a set of tests which run within Jenkins and at the end of the tests (in the post section) I generate an allure report. The tests are created with Webdriverio.
The last time I did anything like this, I was using standard selenium running with Junit and due to that combination, the Jenkins Rest API had access to things like failedCount, skipCount, passes and failed counts. This however seems to only be possible with JUnit.
Does allure expose this to Jenkins somehow - or am I going to have to try and scrape the statistics from a pre-generated report?
(as a side note, I have tried to configure a second reporter in webdriverio, but while that seems to create some xml files locally, the folder I configure is not appearing on Jenkins. Also, webdriverio doesn't support JUnit as a framework - currently it's using mocha.)
I eventually managed to figure this out.
There is url which you can go to to get a json file with the summary information.
So if you report is at /allure - then you navigate to /widgets/summary.json
Jenkins test results screen shows only pass/failed results.
I would like to show quantitive (number/percentage/time duration etc') results, parsed from logs.
e.g. memory usage, run time of specific methods etc..
What is the best way to do so?
Thanks
good question.
i imagine you're probably looking to extend your Jenkins instance with some plugins that describe more info about the tests you've run. this plugin seems relevant, but requires some experience with JMeter (java-based performance measurment tool) to generate the output that this plugin can then read and display output from a JMeter task that you can run every time your build runs:
https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin
the 'readme' in the above plugin details page specifies how to set up a project to run JMeter( see the 'Configuring a project to run jmeter performance test:' near the bottom.
another way to do similar not so immediately tied to a specific jenkins build is to run resource monitors (like Cacti or collectd) on the machines running the tests and analyze those results post-build, but again, outside of the Jenkins context.
HTH.
I am building a Grails 3 application using Atlassian Bamboo. I've configured the Grails 3 task as explained in my answer to this question.
I am having trouble getting Bamboo to find my test reports. On the Grails task type, you can specify if the task produces test results, and if so, where they come from. So I have tried the following:
Look in the standard test results directory
Specify custom results directories:
**/*reports/*.xml (which is the default when you choose Specify...)
build/reports/tests/index.html
All three produce this error:
Could not find test result reports in the /home/bamboo/bamboo-agent-home/xml-data/build-dir/CSSO-MPA4-JOB1 directory.
The correct answer is build/test-results/*.xml, as that is where JUnit puts its test reports.
**/*-reports/*.xml fails because it is in -results.
build/reports/tests/index.html fails because, as it says right above the input box:
Test output must be in JUnit XML format.
I discovered (from somewhere in this thread) that JUnit is responsible for generating the test results independent of Gradle / Grails, which is how I found they were in an odd place.
I am running sitespeed.io tests in Jenkins, and have configured it to output junit format test results.
I'd now like to publish those test results to SonarQube (I realise I can publish them in Jenkins, but I have a requirement to keep everything in one place as much as possible).
However, when I add the test results file into the Sonar analysis (using sonar.junit.reportsPath=/path/to/sitespeed-results, having named my results file TESTS-sitespeed.xml) SonarQube doesn't seem to show any results on its dashboard.
I understand that SonarQube also has a setting to configure the location of test files, and this is often cited as a reason for test results not being ingested correctly, and that leads me to wonder whether what I'm trying to do is possible.
Any help would be greatly appreciated.
Thanks,
Jez
the property sonar.junit.reportsPath will read and parse the report but will only save the information if the class name indicated by the report can be mapped back to a java resource of the project.
I have no idea about the output of your test in surefire format but I sense the classname won't match any resource of your project and so your report is parsed but data is ignored.
When I run grails tests via jenkins I can check the console to see everything passed.
I'll see something like this:
Tests PASSED - view reports in /Users/me/developer/gradlemucks/grails_2/hello-world/target/test-reports
Is there anyway I could put in a hyperlink somewhere in Jenkins to the test reports so that I could just click thru to the report? Thanks
According to the testing guide, Grails outputs both .html and .xml formatted results from the tests by default. You can configure Jenkins to read the .xml files:
Install the JUnit or xUnit plugin.
Add a Post-build Action to Publish JUnit test result report.
Set Test report XMLs path to build/test-results/*.xml (or whatever path your JUnit .xml output files are dumped to).
This will permit you to click through the tests run for any build from the left sidebar within Jenkins.
Have you looked at the Side Bar Link Plugin? It allows you to add arbitrary links to the left side menu of a job.