I want to post detailed results of my tests to slack using fastlane. When glancing at the tests running from fastlane in my terminal, I notice I have reports such as name of each test, time spent on each test, which ones failed, and with what error. Currently, I have my fastfile set up as described in this doc (at the bottom where it says "Posting Build Results".
https://docs.fastlane.tools/getting-started/ios/running-tests/
However, this only sends number of tests run and number of failures to my slack channel. Is there a way to get a more detailed "report card" of all the tests that ran?
Yes, you can use the Fastlane tool "scan" to get a generalized report of your test cases, it comes up with a HTML page of the test statistics.
https://docs.fastlane.tools/actions/run_tests/
If you want a test coverage report then you should use the other tool "xcov" which will give you coverage report of each file which is included in the test case.
https://docs.fastlane.tools/actions/xcov/
Related
I am new around here, I have done a lot of googling, searching on this site and asking around and have not found a satisfactory answer.
I develop automated tests, UI as well as API. These are then run by TFS and the results are put into a JUnit xml document, which is then read by TFS. But alas the formatting is atrocious and leaves one unable to use the output for anything.
Vis:
There is no information about the Test Suite (which is there in the XML), the actual request sent (which is in the log) or response received and so one is left with absolutely no context to understand what has actually taken place (which request was sent, what test group/suite it belongs to and what any potential error was).
As far as I have been able to uncover, TFS simply has little to no support of proper test result formatting when it comes to automated testing. I am very surprised by this in 2018. Not even any documentation that allows me to develop my report structure/format in some kind of script.
What alternatives do I have? Can I automatically attach a generated HTML report somehow in TFS? Can I output more info anywhere?
You can group by Test Suite, Owner, Priority etc...
And you can double click the specific test result and navigate to the test run Summary to see more information there, also you can attach files there.
More information please see Review continuous test results after a build
I was wondering if anyone knew if it's possible to write to a log so Jenkins will display it when you are viewing nunit test results.
The idea is to have steps of test printed in the white space, the reason for this is so when a test fails its obvious to where it failed due to the console messages. When the a test currently fails on a assert we get an obvious message but we cant always guarantee the test will fail on an assert it may fail to find an object.
I have tried using the following
Console.writeline("About to click button x")
and
TestContext.Out.WriteLine("Message to write to log");
but nothing is showing up, is this even possible?
When you publish the NUnit Test results with the NUnit Plugin it is converting NUnit XML to JUnit XML.
Publishing it as JUnit XML because when it comes to test result XML in Jenkins all roads lead back to JUnit.
Console.WriteLine will at best emit content to the job console log. Not the Test Result XML file.
I had successfully configured jenkins build jobs send emails out with basic information in. The content is defined as:
Check console output at $BUILD_URL to view the results.
${FAILED_TESTS}
However, the ${FAILED_TESTS} only return the top-level information, e.g.
1 tests failed. FAILED: XXXMainTest.test Error Message: expected: but was: Stack Trace: java.lang.AssertionError: expected: but was: at test.XXXXTest.test(MainTest.java:27)
If I want to see detailed information thrown from the failed test, I need to click the link and click through each test suite until seeing the test failure information at every end.
Do you have any advice to add detailed tests information to email? Thanks.
I think you will find some help with this plugin:
EMail-ext Plugin
New to version 2.15 is the ability to add attachments using the Ant pattern matching syntax used in many places in Jenkins. You can set a maximum total attachment size in the global configuration page, or it will be unlimited.
I want to ask if anybody has an idea about a ruby reporter , what I want is a framework for reporter HTML or XML report to give me a status for each step and the whole test cases in my test..
A little background of what we are doing:
We are using Ant and jUnit framework for carrying out UI testing of our websites using an open source Selenmium RC tool.
At the end of test execution, HTML reports are produced using the Ant task jUnitreport. The test pass, fail stats are represented there with intuitive HTML navigation from test suite to test case level status.
The logging and assert failure messages go into the console, which is available by following a link in HTML test report at the bottom of the report.
Question:
The above report of logs is a run down log of all test cases together. Is there a way we control the logging to be tied up to the test case level in the HTML table report of jUnit.. like the logs related to each test cases should just show within the HTML test summary table.
For example, when an unhandled exception is raised, jUnit displays this error in the Summary table against the test. Can we display the logs in that format as well?
I use slf4j logging.
Any help on this greatly appreciated..
Regards,
Prakash