I'm looking into SpecFlow and SpecRun considering changing away from our MbUnit setup. But i ran into a road block, how do I attach a screenshot to a SpecRun report?
Or alternativly customize the test report to contain a clickable url?
Thanks in advance.
You can create your own custom report and specify it while running SpecRun.
Related
My case is this: I refactor some test steps and I wanna make sure after my refactoring is done all old tests steps are changed to new ones.
So I need to make sure my project doesn't have unbound steps. Is there a way to do this?
Yes, there is. Specflow has a 'stepdefinitionreport' command which you can use to generate a HTML report which shows which steps are used / unused.
specflow.exe stepdefinitionreport MySpecflowProject.csproj /BinFolder:bin/debug
MySpecflowProject.csproj is your project, BinFolder: is the output directory where the report will be made.
I use a unit test coverage tool (such as dotcover) to highlight used steps (green).
This show unused steps as grey for easy removal.
We are using SpecFlow to apply BDD practices and Pickles to generate the documentation.
Our build looks like this:
Build
Deploy
Test (using VSTest)
Generate documentation (using Pickles)
The generate documentation step comes after the Test step, so we can include testresults to pickles. I want to use pickles as a build step, so I can manage the pickles version to use and additional options in each Solution.
The Visual Studio Build step in TFS2017 looks like this:
/t:DocumentationGeneration
/p:Pickles_Generate=True
/p:Pickles_DocumentationFormat=dhtml
/p:Pickles_ResultsFile="$(build.sourcesdirectory)/TestResults/*.trx"
/p:Pickles_OutputDirectory="$(build.artifactstagingdirectory)/PickledDocumentation"
[some more generic parameters that are not relevant here]
When I do the above, I get an "Illegal character in path" error during the build. It seems to be the * causing this error, when I replace it with /p:Pickles_ResultsFile="$(build.sourcesdirectory)/TestResults/testresults.trx" there is no problem at all, only the testresults are not found because the testresults are in the non predictive format:
USERNAME_SERVERNAME1234 2017-02-02 09_09_09.trx
I tried to look into generating a generic name for the .trx file, but VSTest does not support custom names. Did anyone else encounter this problem and is there a nice approach to solve it?
Pickles only supports the exact name of the test result file. There is an issue on the GitHub project site to allow wildcards. As the maintainer of Pickles, I will be happy to review any pull requests that address that issue.
At request: This is how I solved it. It is more a work around that works in my particular situation, but at the bottom I explain how I hope it will be fixed soon.
Create a Command Line build step right after test build step
as tool use Rename
as arguments use "$(build.sourcesdirectory)\TestResults\*.trx" TestResults.trx
Now you can use a Visual Studio Build step to generate pickles documentation (you need to have pickles as NuGet package in your project)
As Solution set $(Build.SourcesDirectory)/yourproject.csproj
As buildarguments set additional parameters:
/t:DocumentationGeneration
/p:Pickles_Generate=True
/p:Pickles_DocumentationFormat=dhtml
/p:Pickles_ResultsFile="$(build.sourcesdirectory)/TestResults/TestResults.trx"
/p:Pickles_ResultsFormat="vstest"
/p:Pickles_OutputDirectory="yourdestinationpath"
As I said, the renaming is a bit a work around, so I extended the code of pickles itself to enable wildcard support, and is now a pull request in the pickles repo, so hopefully it is implemented in the next release and you all can use it that way :).
I am creating a build definition in visual studio 2013. As you know, while creating build definition we can also set the assembly (in "Automated Tests" section) which contains the unit test methods to run. I am able to create the definition till this point. Now, what I want is once the test run is over, I want to call an exe that will parse the Trx file generated as a result of test run and send the run results in an email.
Can anyone have an idea about where I can set the path to the exe and pass some arguments to it while creating build definition? If this cannot be done while creating definition, what else is the other way to achieve it?
Thanks!
What you ask involves multiple steps, especially if you want to pull the tool from version control (which is IMHO the best option).
See the ALM Rangers' Team Foundation Build Customization Guide for help and guidance on this topic.
Or you can just setup an email notification when the TFS Build completes. That should include the test run results, and a clickable link to go to the full build report.
I want create sections in JUnit report (displaying in Jenkins) for specify files name or some custom title.
It is possible? and if yes, how?
thanks for feedback
You could maybe achieve this by creating specific style sheets, or post-process your junit results before the reports are compiled, but without knowing more what you are trying to achieve, why, and where you would configure this task, I find it a bit hard to guide you further.
I need to build a report in Report Builder 2.0
The report is finished, but we use the report to view open issues and bugs. The report works fine, but now we need to set the current project ourselves in the report. I would like to pass the current project, or read it out somehow, so the report can be used in all our projects without having to change the report everytime.
So when I open the TFS portal, and select a specific project, I want the report to read the project and build the report.. That way we can use the report in all our projects.
Hope somebody can help me with this! Thanks in advance!
Ok I've found it.. If somebody issues the same problem, read this