Can you tag individual examples in a scenario outline in SpecFlow? - specflow

Scenario outlines are very handy for creating data driven tests, but the number of scenarios increases with the number of examples. I've gotten in the habit of tagging scenarios to make it easier to filter on major features of our application.
I would like to set up a "smoke test" that hits all the major use cases. Some of these use cases are captured in scenario outlines that perform boundary testing on dates or numbers, but I just want to hit that one prototypical case within the examples.
For instance say we've got a feature allowing us to add job openings to a job (basically a "hiring opportunity" versus "we have warm bodies filling this position").
On screen we have two form fields for the minimum experience: years and months. The user should not enter more than 11 months in the months field, otherwise they should put something in the years field (e.g. 18 months should actually be 1 year and 6 months).
#job-openings
Scenario Outline: Adding a job opening with experience
Given a job exists
When I add a job opening requiring <years> years and <months> months experience
Then a job opening should exist requiring <years> years and <months> months experience
Examples:
| years | months |
| 0 | 1 |
| 0 | 11 |
| 1 | 0 |
| 2 | 6 | # <-- the "prototypical" example I want to tag
| 99 | 0 |
| 99 | 11 |
| 100 | 0 |
Having those examples hitting the boundaries of the acceptable values for years and months is definitely useful from a regression testing standpoint, but not when performing a "smoke test" of the system. It would be nice to run a single example in the scenario outline that represents a typical use case. As some background info, we have a PowerShell script that developers use to run automated tests of all sorts, and a general "smoke test" hitting all the major features would be useful.
Is there a way to tag an individual example in a scenario outline?

This is the way I do it:
#job-openings
Scenario Outline: Adding a job opening with experience
Given a job exists
When I add a job opening requiring <years> years and <months> months experience
Then a job opening should exist requiring <years> years and <months> months experience
#smoketest #regression
Examples:
| years | months |
| 2 | 6 | # <-- the "prototypical" example I want to tag
#regression
Examples:
| years | months |
| 0 | 1 |
| 0 | 11 |
| 1 | 0 |
| 99 | 0 |
| 99 | 11 |
| 100 | 0 |
There are two example sections that both belong to the scenario. The smoketest has its own example section. When running
dotnet test --filter "TestCategory=job-opening&TestCategory=smoketest"
it will only run the example with the smoketest tag. When running
dotnet test --filter "TestCategory=job-opening&TestCategory=regression"
it will run all the examples. It will also run the smoketest because it has the regression tag too.
user1207289's method also works. I sometimes do it that way when a test breaks and I want to retest it later. When the tests are generated the specific example you want to run will get a name (e.g. AddingAJob_ExampleYears2Months6). You can find the name of the generated unit tests in the scenario with the -t flag, which lists all the tests:
dotnet test --filter "TestCategory=job-opening" -t
To run one specific test (technically all tests with AddingAJob_ExampleYears2Months6 in the name):
dotnet test --filter AddingAJob_ExampleYears2Months6
I used the official dotnet cli tool in the examples above, but it's pretty similar for the other test runners.

I am able to run single example from scenario outline by below command
C:\Users\..\..\bin\Debug>"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" yourTests.exe /Tests:yourTestName
where yourTestName is the name of the test that is generated in test explorer upon build and yourTests.exe is the generated exe in /bin/debug. I am using MSTest
For more info on names generated look here

Related

Is there a Feature Outline?

In my application, the set of tests for an Estimate and Invoice are very similar. I can use the Scenario Outline and Examples to repeat a test with these types. But how do I repeat all the tests within a feature with examples and not repeat the examples at every scenario outline?
For example, is there a way I can rewrite the tests below without having the state the examples twice?
Scenario Outline: Adding a sales line item
Given I have a <Transaction>
And Add Hours of quantity 2 and rate 3
When I save
Then the total is 6
Examples:
| Transaction |
| Invoice |
| Estimate |
Scenario Outline: Adding two sales line item
Given I have a <Transaction>
And Add Hours of quantity 2 and rate 3
And Add Hours of quantity 5 and rate 2
When I save
Then the total is 16
Examples:
| Transaction |
| Invoice |
| Estimate |
In other words, is there such a thing called, for a lack of a better, Feature Outline?
Unfortunatelly Gherkin language does not support anything like this

Wait for a task to finish in cucumber feature file

I have been writing the cucumber tests for the application developed in Ruby on Rails. The application provisions a server (vm) and wait for the result (like success or fail) And displays the result. There are 7-8 different cases (or scenarios) for provisioning a server.
I have 2 questions
How can I club these 2 different scenario outlines in to one. So that I don't need to repeat the table in Examples section. I wanted to create a provsioning requests for all scenarios (Examples) and then wait for 15 min till all requests gets provisioned and after 15 min check if all scenarios are passed or failed.
As these tests take more time to complete, how can I skip them when I run "bundle cucumber" which runs all the feature files. I
don't want to run the following test every time.
I have written cucumber tests by using scenario outline and examples as below .
Scenario Outline: Create A New Server Request
Given user is on create new server request page
When user enters follwing data
| Datacenter | <Datacenter> |
| Cores |<Cores> | |
| Memory | <Memory> |
Then he should see request submitted
Examples:
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
Scenario: Wait till provisioning happens
Given user waits for 15 minutes
Scenario Outline: Verify a New Server Request
Given user is on provision server page
When user check the request with follwing data
| Datacenter | <Datacenter> |
| Cores |<Cores> | |
| Memory | <Memory> |
Then he should see servers are provisioned
Examples:
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
Anupam, I believe the Background holds the key to what you need. Set up your set of servers to work with as a variable using background and then simply refer to those during your scenario. I would also combine the "given the user is on ____ page" since you could bury that implementation detail in your stepdef to provision, check the queue and check the result.
Background: The set of servers to provision
Given the following set of servers are to be provisioned
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
#slow
Scenario: Provision New Servers
When a user provisions the set of servers
Then the user should see the set of servers submitted
When the user waits 15 minutes
Then the user should see the set of servers have been provisioned
To persist the background data across your steps, simply set it to a instance variable on your World object. I would probably use Cucumber::Ast::Table.hashes to get the data into an easier to use data format.
#server_set = table.hashes
As for not running these tests every time, I would set up a cucumber #tag which you use to exclude scenarios that take too long. You can also set up a yaml configuration file to handle this and then just call the appropriate profile in your tests. See the cucumber documentation for examples. https://github.com/cucumber/cucumber/wiki/cucumber.yml
Edit: You could also set up the varioable on your first step def, but I like using background for "data" that you don't want to repeat.

Specflow Feature-level Templates

I'm trying to execute an entire SpecFlow Feature using three different UserID/Password combinations. I'm struggling to find a way to do this in the .feature file without having to introduce any loops in the MSTest.
On the Scenario level I'm doing this:
Scenario Template: Verify the addition functionality
Given the value <x>
And the value <y>
When I add the values together
Then the result should be <z>
Examples:
|x|y|z|
|1|2|3|
|2|2|4|
|2|3|5|
Is there a way to do a similar table at the feature level that will cause the entire feature to be executed for each row in the table?
Is there other functionality available to do the same thing?
I don't think the snippet you have is working is it? I've updated the below with the corrections I think you need (as Fresh also points out) and a couple of possible improvements.
With this snippet, you'll see that the scenario is run for each line in the table of examples. So, the first test will connect with 'Bob' and 'password', ask your tool to add 1 and 2 and check that the answer is 3.
I've also added an ID column - that is optional but I find it much easier to read the results with an ID number.
Scenario Outline: Verify the addition functionality
Given I am connecting with <username> and <password>
When I add <x> and <y> together
Then the result should be <total>
Examples:
| ID | username | password | x | y | total |
| 1 | Bob | password | 1 | 2 | 3 |
| 2 | Helen | Hello123 | 1 | 2 | 3 |
| 3 | Dave | pa£sword | 1 | 2 | 3 |
| 4 | Bob | password | 2 | 3 | 5 |
| 5 | Helen | Hello123 | 2 | 3 | 5 |
| 6 | Dave | pa£sword | 2 | 3 | 5 |
| 7 | Bob | password | 2 | 2 | 4 |
| 8 | Helen | Hello123 | 2 | 2 | 4 |
| 9 | Dave | pa£sword | 2 | 2 | 4 |
"Is there a way to do a similar table at the feature level that will
cause the entire feature to be executed for each row in the table?"
No, Specflow (and indeed the Gherkin language) doesn't have a concept of a "Feature Outline" i.e. a way of specifying a collection of features which should be run in their entirety.
You could possibly achiever what you are looking for by making use of Specflow tags to tag related scenarios. You could then use your test runner to trigger the testing of all the scenarios with that tag e.g.
#related
Scenario: A
Given ...etc...
#related
Scenario: B
Given ...etc.
SpecFlow+ Runner (aka SpecRun, http://www.specflow.org/plus/), provides infrastructure (called test targets) to be able to run the same test suite (or selected scenarios) with different settings. With this you can solve problems like the one you have mentioned. It can be also used to run the same web tests with different browsers, etc. Check this screencast for details: http://www.youtube.com/watch?v=RZYV4Dvhw3w

Cucumber and DELETE with Mongoid

So, I am currently trying to test a Rails app wired up with Mongoid with Cucumber. I have everything setup (or so I believe) so that the following test will run:
Feature: Create and manage about entries
In order to use the about data effectively
As an application consumer
I want to create and manage some about entries
Scenario: Create about entries
Given the following abouts exist
| title | body_copy |
| "About entry #1" | "hello body!" |
When I go to the list of about entries
Then I should see "About entry #1"
Scenario: Create and retreive specific about entry
Given the following abouts exist
| id | title | body_copy |
| 4e4d37756ea257f031000003 | "About entry #1" | "hello body!" |
When I go to about entry with id 4e4d37756ea257f031000003
Then I should see "About entry #1"
In my paths file, I have the following entries to support the above tests:
when /^the list of about entries$/i
'/abouts'
when /^about entry with id (.+)$/i
"/abouts/#{$1}"
These tests work great. However, I need to test the delete action. I did some research online but everything seems to be going through the UI to delete these items and the problem I have is that my Rails app only serves JSON files and JSON files and I need a better (more programatic) way of testing things without the UI being involved. As far as mocks goes, I am using the default mocks built into Pickle. I am open to using other mocking software if necessary such as factory-girl, but you'll have to give me some detailed feedback how I can wire that up. What I have currently for my delete test (that DOESN'T work) is:
Scenario: Delete about
Given the following abouts exist
| title | body_copy |
| title 1 | body_copy 1 |
| title 2 | body_copy 2 |
| title 3 | body_copy 3 |
| title 4 | body_copy 4 |
When I delete the 3rd about
Then I should see the following abouts:
| Title | body_copy |
| title 1 | body_copy 1 |
| title 2 | body_copy 2 |
| title 4 | body_copy 4 |
The problem is that the auto-generated test (seen above) uses the click_link "Destroy" method call but that will not work.
You'll need to change the implementation of the delete step (in your web_steps.rb if your using the Cucumber generated defaults) to send a DELETE HTTP request. I'd recommend the RestClient gem for this, but there are plenty other choices.
The Cucumber Book currently in beta from PragProg has a chapter about using Cucumber to test REST APIs like this.

Testing Pagination Features with Cucumber

I am learning and liking cucumber, and now have a feature I'm not sure of the best way to proceed on via BDD: pagination. I have scenarios (in my mind) where there are zero pages, one page, several pages, etc. and where I want to make sure certain records are on certain pages, make sure the "next" button is not a link when on the last page, etc.
I will be using will_paginate, so essentially I want to figure out how to BDD its features for a specific list of items, for example, books.
I'm sure I can muddle through it but I feel this should be common and would like to see what others have done. Can anyone recommend an article, or point me to some example code, or even take a shot at an example themselves?
You could probably get away with using scenario outlines to keep the repetition down in your feature file, but be aware that it will expand to a very large number of actually-run scenarios, so it'll be slower than you'd expect. Something like this should probably work, assuming 5 books per page. I'll leave the step definitions as an exercise, but they should be pretty straightforward.
I should also mention that I haven't actually run this, so take any syntax errors with a grain of salt.
Feature: Book Browsing Pagination
Scenario: No results
Given I have 0 books
When I view all books
Then I should see "No results" on the page
Scenario: Some results
Given I have 3 books
When I view all books
Then I should see "Book 1"
And I should see "Book 2"
And I should see "Book 3"
Scenario: Page links
Given I have <count> books
When I view all books from page <page>
Then I should see a link to page <target page>
Examples:
| count | page | target page |
| 8 | 1 | 2 |
| 8 | 2 | 1 |
| 13 | 1 | 2 |
| 13 | 1 | 3 |
| 13 | 2 | 1 |
| 13 | 2 | 3 |
| 13 | 3 | 1 |
| 13 | 3 | 2 |
Scenario: Page links for current page
Given I have <count> books
When I view all books from page <page>
Then I should see a disabled pagination link to page <page>
Examples:
| count | page |
| 8 | 1 |
| 8 | 2 |
| 13 | 1 |
| 13 | 2 |
| 13 | 3 |
Scenario: Next Page links
Given I have <count> books
When I view all books from page <page>
Then I should see a link "Next Page"
When I click "Next Page"
Then I should be on page <next page> # assert against query params maybe?
# alternately, to keep page requests down, one could use something like:
# Then I should see a link "Next Page" going to "?p=<next page>"
Examples:
| count | page | next page |
| 8 | 1 | 2 |
| 13 | 1 | 2 |
| 13 | 2 | 3 |
Scenario: Next Page links on last page
Given I have <count> books
When I view all books from page <page>
Then I should see a disabled pagination link "Next Page"
Examples:
| count | page |
| 8 | 2 |
| 13 | 3 |
Similar scenarios could be used for checking the link state of Previous/First/Last as for Next, and you could add some followup clicking to the Page Links scenario similar to what the Next Page scenario is doing if you wanted. You might also want to add extra examples to check that your pagination is at exactly 5, since the examples I've picked would also pass if pagination was 6 per page. Depends on what exactly your goals are for checking the pagination functionality.
If you eventually decide on something other than will_paginate, then the only things you'd need to look at changing might be a few of the steps (like the disabled pagination ones).
And as you mention asking for links, this might make a good blog post too ;)

Resources