Our Cucumber Feature files are designed to have Scenario Description with Title.
So when I import feature files into Xray via Jenkins, Feature description gets copied over into Xray Test description area rather I want Cucumber Scenario description to be copied. Is that possible?
I am running Jira DC v8.9.0 with Xray v3.6.6
The process of importing the cucumber .feature files is detailed here; it does not include all the details though.
Right now, the import process is non-configurable.
Therefore, unfortunately the behaviour you're looking for is not possible.
The import process has evolved throughout the years though, based on customer feedback. My recommendation would be to ask the team for an improvement related to this, by reaching out the support team.
Please note that the Gherkin steps (your "Scenario description") will be on the specific field for hosting the gherkin specification.
The scenario title will be on the Test issue' summary.
Related
While importing Cucumber feature files to Xray, Background Title and Description are mapped to Pre-condition Summary and Description fields but when exporting a pre-condition issue from Xray it's Summary and Description are not copied over to Cucumber Background Title and Description section. Is this not implemented yet or is this a bug?
Using Jira DC v8.9.0 and Xray v6.1.3
It's not a bug; it follows these rules.
The Background section doesn't have a title; AFAIK the Gherkin specification doesn't support that (see example here). The reason for it may be that a Feature can only have a Background, therefore it wouldn't make sense to assign it a name (my guess).
Having the description of the Precondition as a comment on the Background may be feasible to implement; it has to be analyzed by the Xray team. Please contact Xray support team and ask for an improvement request, with a concrete example.
There are more than 500 Cucumber Feature files in our automation test suite and I am trying to import them into Xray. I am using Jenkins to do that with XrayImportFeatureBuilder class. As suggested in Importing Cucumber Tests - REST doc, Feature section will not be imported because it is assumed to be there already in Jira as a Requirement Issue. Now since there are those many Feature files it's really time consuming to manually add each Feature title and description into Jira to have it linked with Tests when I run the import job.
Would be great to know if there a way to only import Feature section?
I have Jira DC v3.9.0 installed with Xray v3.6.6
The process of importing the cucumber .feature files is detailed here and mentions precisely that it's not possible to import Feature section, as that will correspond to, usually, an existing Story issue.
This whole idea assumes that people are using Jira in the most common scenario. Stories usually follow a process, where someone is responsible for creating and reviewing them.
Nevertheless, if you want to provision them, the only option I see would be to make some code/script to parse the .feature files and then either build out a CSV that could be imported by Jira's CSV Importer or do REST API requests to the Jira API directly to create those issues.
However, you would end with 500 Story issues that you would need to link by hand to the corresponding scenarios/tests. To make that linkage more or less automatic, you would need to edit the .feature files (after importing them), and add a tag with the requirement/story issue created earlier before the "Feature: " section. This would be a challenging thing though as you have many .feature files.
We are using JIRA, SpecFlow, Jenkins in our project.
Can anybody tell us a tool that can be used as JIRA add in to organize, maintain, and track the testing process, progress, and results of execution the test scenarios in the BDD Feature files?
I want to record and track both the manual tests my team will run and all the automated tests generated through BDD Feature files in JIRA.
We used BehavePro which worked excellent. The business could create the user stories in Gherkin and we would export it to Visual Studio and automate the tests.
Besides that I have no experience in other tools/plugins.
We are using Jenkins to run the selenium automation tests and my manager wants to see the list of failed builds and what percentage of the tests passed for the builds. We also have manual tests that get executed in JIRA. I need to combine both and derive the test metrics from them.
The way I think of proceeding is as follows:
Get the Jenkins data in JIRA first using the Jenkins plugin for JIRA.
Use the jira api to collect the testing results from Jenkins and manual tests run on jira.
Prepare a dashboard in JIRA to display all the metrics
Could you suggest if the above approach is correct and suggest something additional.
Thanks in advance!
Are you using cucumber? In that case you could use the cucumber reporting plugin for jenkins. If it doesn't suit your needs but you still use cucumber you can also generate reports in a format like JSON, which you could later parse and get your data.
I have the feeling what you want to do seems a bit complicated, and with not a big benefit. If the tests are failing it's likely you'll have to see what is happening. Having the percentage is sure nice, but I think you can spend some hours/days tailoring this just for having something cute that your manager wants but that has no specific purpose. I would opt for something simpler.
If the automated tests fail, create a jira issue automatically with jenkins. You could put the build number as a tag, or in the title. You can also create it always to indicate that build nr. ## was tested and everything went ok.
As a part of the manual testing process, report in jira what failed.
Create a dashboard and play a bit with tags and search to show which builds failed.
I would suggest AssertThat BDD & Test Management in Jira
Provides end-to-end integration - from features creation to manual and automated tests execution and reporting. Out of the box integration with test automation frameworks through plugins.
The plugin allows to download feature files stored in Jira before the run, execute the test in the usual way and then upload cucumber tests results back to Jira, which gives you a clear view on the testing progress in one place.
More info and usage examples on website https://www.assertthat.com/
How to get a "test plan" from SpecFlow feature files? Is it possible to get similar "report" (HTML, pdf, word, etc.) as we get from executed tests?
It would be great to get table with:
test description
steps
expected result
Information is already inside feature files, just how to structure it as "test plan"
I'm not entirely sure what you mean with "testplan" but using SpecFlow.exe you can get two kind of reports:
A test execution report, that shows how your tests has executed. I've written about it here: http://www.marcusoft.net/2010/12/specflowexe-and-mstest.html
A step definition report, that shows how the step definitions are used within your specifications. I've written about that here: http://www.marcusoft.net/2010/12/know-where-you-stepgenerate-step.html
Combining these two might give you what your looking for, but (!) I think what your actually looking for is something like SpecLog that is a tool to manage stories and requirements in an agile project.
I written about SpecLog here:http://www.marcusoft.net/2011/02/specification-by-example-with.html and their main site is here: http://www.speclog.net/
I hope this answers you question