How to get a "test plan" from SpecFlow feature files? Is it possible to get similar "report" (HTML, pdf, word, etc.) as we get from executed tests?
It would be great to get table with:
test description
steps
expected result
Information is already inside feature files, just how to structure it as "test plan"
I'm not entirely sure what you mean with "testplan" but using SpecFlow.exe you can get two kind of reports:
A test execution report, that shows how your tests has executed. I've written about it here: http://www.marcusoft.net/2010/12/specflowexe-and-mstest.html
A step definition report, that shows how the step definitions are used within your specifications. I've written about that here: http://www.marcusoft.net/2010/12/know-where-you-stepgenerate-step.html
Combining these two might give you what your looking for, but (!) I think what your actually looking for is something like SpecLog that is a tool to manage stories and requirements in an agile project.
I written about SpecLog here:http://www.marcusoft.net/2011/02/specification-by-example-with.html and their main site is here: http://www.speclog.net/
I hope this answers you question
Related
Our Cucumber Feature files are designed to have Scenario Description with Title.
So when I import feature files into Xray via Jenkins, Feature description gets copied over into Xray Test description area rather I want Cucumber Scenario description to be copied. Is that possible?
I am running Jira DC v8.9.0 with Xray v3.6.6
The process of importing the cucumber .feature files is detailed here; it does not include all the details though.
Right now, the import process is non-configurable.
Therefore, unfortunately the behaviour you're looking for is not possible.
The import process has evolved throughout the years though, based on customer feedback. My recommendation would be to ask the team for an improvement related to this, by reaching out the support team.
Please note that the Gherkin steps (your "Scenario description") will be on the specific field for hosting the gherkin specification.
The scenario title will be on the Test issue' summary.
There are more than 500 Cucumber Feature files in our automation test suite and I am trying to import them into Xray. I am using Jenkins to do that with XrayImportFeatureBuilder class. As suggested in Importing Cucumber Tests - REST doc, Feature section will not be imported because it is assumed to be there already in Jira as a Requirement Issue. Now since there are those many Feature files it's really time consuming to manually add each Feature title and description into Jira to have it linked with Tests when I run the import job.
Would be great to know if there a way to only import Feature section?
I have Jira DC v3.9.0 installed with Xray v3.6.6
The process of importing the cucumber .feature files is detailed here and mentions precisely that it's not possible to import Feature section, as that will correspond to, usually, an existing Story issue.
This whole idea assumes that people are using Jira in the most common scenario. Stories usually follow a process, where someone is responsible for creating and reviewing them.
Nevertheless, if you want to provision them, the only option I see would be to make some code/script to parse the .feature files and then either build out a CSV that could be imported by Jira's CSV Importer or do REST API requests to the Jira API directly to create those issues.
However, you would end with 500 Story issues that you would need to link by hand to the corresponding scenarios/tests. To make that linkage more or less automatic, you would need to edit the .feature files (after importing them), and add a tag with the requirement/story issue created earlier before the "Feature: " section. This would be a challenging thing though as you have many .feature files.
My question is simple: I have used WITAdmin in the past to add custom fields to various work item types in TFS. Most recently - to add time estimates to the Test Case work item template in TFS. However, when I use WitAdmin listwitd command on my project, I don't see anything for Test Result.
Is there any way to customize the work item template for test results in TFS? I want to add an additional test result steps beyond the ones that are already out of the box, i.e. pass, fail, blocked, not applicable.
Thanks,
There is no work item type which called test result in TFS. You can customize the Test Plan & Test suite & Test Case and add custom fields or define custom workflows to it, just without test result.
The test result is associated with MTM. If you want to customize Test Result Failure Type & Resolution Type, please refer the link from MSDN: Customize and manage the test experience [tcm and Microsoft Test Manager]
For more info, please take a look at this uservoice: Provide customization for test plan, test results.
I have been working with Pex(IntelliTests) for some time now and I wondered if it is possible to create the tests via some sort of command(or .exe) and not through the IDE(VS2015) with right-clicking the function.
I have an automated process which builds my projects and further runs my tests. However if the IntelliTests are not generated anew for the new implementation they are rather useless.
This may seem like a basic question but unfortunately I could not find any information on the internet.
A command line for IntelliTest is not yet supported. If you would like to see it supported, kindly add your vote here: https://visualstudio.uservoice.com/forums/330519-team-services/suggestions/8623015-enable-intellitest-to-run-in-the-build-pipeline
i'm currently trying to learn bdd techniques and trying frameworks for it. the idea comes from test to code, but i want to use it in revers way if possible.
i'm currently using specflow, selenium and nunit combination. i want to open my work to anyone (non-developers) for adding new examples or test cases. let's say i have implemented all use cases but changing step orders or modifying/adding examples would enrich my test cases.
the problem is that i have to build the project for each change in feature files. is it possible to use this technique in that way?
Given I have scenarios
And I have step definitions
When I want to add new tests
Then I should be able to load .feature file as input
if not possible, i'm thinking about some parsing operation to map them with my methods, i guess there is a better way to implement this idea.
From what I understand, you want to avoid having to rebuild your test for changes in your feature file
The problem with what you are suggesting is that you are assuming your mapping is as simple as:
Feature file <---> StepDefs
But in reality whats happening is:
.Feature File<--->FeatureFile.feature.cs<--->StepDefs
The features are mapped to the step defs using a 3rd file that is auto generated when your test builds. This maps the two files together. So even if you were simply passing in a .feature file you would still have to do some kind of build in order to generate the .cs file and map the two files together.