I am creating a Cucumber test for a multistep registration process and am a little unsure about the best practice for the Scenario Steps...
There are 4 forms / pages in the registration. Should I loop through the Given, When & Then 4 times in the one scenario or is there a better way to organise it?
So far, I have got...
Scenario: Company User
Given I am on the registration page
When I follow "Register as a supplier"
When I fill in the following:
| user_email | test#test.com |
| user_password | secret |
| user_password_confirmation | secret |
And I press "Create login - Proceed to step 2"
Then I should see "Create Company Profile"
When I fill in the following:
| company_name | Test Company |
| company_description | Lorem |
| company_telephone | 01928740436 |
| company_email | info#agency.com |
And I press "Create company - Proceed to step 3"
Then I should see "Test Company office(s)"
I think Andy Waite has given good advice, but rather than generic names like step 1, step2, etc. I would be more descriptive:
When I register as a supplier with valid information
And I create company profile with valid information
And I ... with valid information
And I ... with valid information
Then I should see "Thank you for registering"
I would recommend having 4 scenarios covering the detail of each step, e.g:
Given I am on step 2
When I fill in the following:
| company_name | Test Company |
| company_description | Lorem |
| company_telephone | 01928740436 |
| company_email | info#agency.com |
And I press "Create company - Proceed to step 3"
Then I should see "Test Company office(s)"
You can hide away any necessary but irrelevant form-filling within the definition of "Given I am on step X".
You should probably also have a scenario which covers how everything fits together, e.g.:
When I complete step 1 with valid information
And I complete step 2 with valid information
And I complete step 3 with valid information
And I complete step 4 with valid information
Then I should see "Thank you for registering"
I like Mark Irvine`s suggestion - one of the main concepts of Cucumber automation is to write steps that are as clear to the reader as possible. It should be absolutely clear what the test is doing even to a person that is not programmer at all.
If you are interested - you can also read "The Cucumber Book - Behaviour-Driven-Development for testers and Developers" by Matt Wynne for more good practices.
Regards,
Alex
Related
Am writing a sceanarios to verify a card in mobile app in BDD. The card contains 7 elements in it and each has a value or a copy. These need to be verified with predefined values / calculated values. so wanted to know, can i write the Assertions for all the 7 elements in single scenario or split it with 2/3 ?
There are no much details, so I can't be specific with the answer. Let's say you are using Python client for Appium tests. In such case it would be nice to use some unit testing framework (it could be Python built-in unittest module).
I'll recommend you to verify each element in a separate test case. This approach will make your life easier - you'll get separate status for each element verification.
Speaking about "How many assertions can take place for a scenario" question - I believe it depends on tools your are using. With Python unittest you may have a lot of assertions in a single test case, but this is bad practise. Please read the following:
https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test
If I knew what you were meaning by "card", this would be a help, but lets assume for this that it's a debit/credit card.
What we could do here, is simply have one assertion:
Scenario: Adding a new payment method
Given I have a card with the following details:
| Name | Mr Test McTestington |
| Card Number | 4567 8901 2345 6789 |
| Card Type | Credit |
| Issuer | MasterCard |
| Valid From Date | 01/23 |
| Expiry Date | 12/34 |
| Security Code | 123 |
And the card details are valid
When I add the card as a new payment method
Then I should be able to checkout the items in my basket with the card
And I should see the order confirmation screen
Inside the And the card details are valid step, you would have the validation code for all of the items. This may involve breaking down each of these into functions that can be used elsewhere:
public boolean validName(string name){
bool valid = false;
// validate name - set valid to true if it meets validation criteria
return valid;
}
As an example (Java is not my strong point, but this is just an outline on my suggestion).
In essence, making it readable is what cucumber does best, to describe functionality in language that the Development Team and the Business have agreed on, so that everyone can understand exactly what is being described in your Scenario. It's more about conversations than testing things.
Is it like I have done in my example, where you don't necessarily need to validate each card detail individually?
It all comes down to a judgement call.
And if it does come down to this judgement call, where you believe each thing needs to be validated individually, why not use a Scenario Outline to help you out?
Scenario Outline: Valid card details
Given I have a card with the "<detail>" of "<value>"
Then the card detail "<detail>" should be valid
Examples:
| detail | value |
| Name | Mr Test McTestington |
| Card Number | 4567 8901 2345 6789 |
| Card Type | Credit |
| Issuer | MasterCard |
| Valid From Date | 01/23 |
| Expiry Date | 12/34 |
| Security Code | 123 |
I have a feature that logs into a trading system and keys a number of trades. Theres a lot of reusable steps at the beginning of each trade (initial trade set up) But each trade has different arguments.
Here is an example
Scenario: Trade 1
Given I have selected my test data: "20003"
And I have connected to VMS with the following details:
| Field | Value |
| Username | user |
| Password | password |
| Session | myServer |
When I run the DCL command to set my privileges to full
Then I expect to see the following:
| Pass Criteria | Timeout |
| Privileges Set | 00:00:30 |
When I ICE to the test account: "System Test"
Then I expect to be ICED see the following:
| Pass Criteria | Timeout |
| "ICED to System Test" | "00:00:10" |
When I run a dcl to delete the company: "Test_Company"
Then I expect to see a confirmation that company: "Test_Company" has been deleted or doesnt exist
So within those steps the 2 things that could change is the "Given" argument so the test data ID and also the Test company at the end.
What I wanted was some way to run a background step so that its being able to know what parameters to enter. So if it was Trade 1 for example it would enter 20003, if it was Trade 2 enter 20004 etc.
Can I do this? I was thinking using the "Example" table that Scenario Outline uses. Or is there a better way to do this? I dont want these repeatable steps in all of my scenarios as it takes up lots of room and doesnt look too readable.
So I did some searching and couldn't find a solution that didn't require a lot of coding so I made this up:
this is what the background looks like
Background:
Given I have selected my test data:
| Scenario | ID |
| DirectCredit_GBP | 20003 |
| Cheque_GBP | 20004 |
| ForeignCheque_GBP | 20005 |
And in order to find which row it should use the method behind it uses ScenarioContext. Here is the method:
[Given(#"I have selected my test data:")]
[When(#"I have selected my test data:")]
public static void setTestDataID(Table data)
{
string scenario = ScenarioContext.Current.ScenarioInfo.Title;
string testDataId = data.ReadTable("Scenario", scenario, "ID"));
TestDriver.LoadTestData(testDataId);
}
What the method does is search the table for the scenario name (using an extension method I wrote) and get the ID, once its got the ID it passes it into my TestDriver method.
It seems to work fine and keeps the test readable.
I used to create scenarios where in scenario name I explain what is scenario.
For example:
Scenario: When during context switch, context doesn't match and list of facts for delete are shown to user, facts should be deleted if user has selected them in the list.
But problem is that scenarios getting more and more complicated and scenario names longer and longer. Should I keep writing long names or do you have some better suggestion?
The scenario outlined in the question sounds highly coupled with the system. What is the behaviour you're specifying?
However, to roll with what you've got, I think this is just a language issue.
I Think personally I would just rename it to something like:
Scenario: Should be able to delete non-matching facts
It's more generic but still tells you what's going on when somebody reads the scenario (given the context of the feature and other associated scenarios).
At the end of the day, the length of the scenario name shouldn't matter - just as long as those involved in the development (think the 3 amigos, developer, tester and business stakeholder); all know what it means. But obviously, the easier it is for somebody else to understand, the better.
Well it sounds like you are repeating yourself.
The test below probably doesn't match up to what you mean. But pretend it does.
If your scenario looks like this:
Given the current context is Green
And the following list of facts for delete are selected
| Fact | Checkboxstate |
| A | checked |
| B | |
| C | checked |
| D | |
When I perform a context switch to Orange
Then the following facts should be deleted
| Fact |
| A |
| C |
And the following facts should not be deleted
| Fact |
| B |
| D |
Then the test is barely more complicated than the scenario title you suggested.
(if the test is much more complex than this, then that might be another problem)
Instead try and keep the feature and scenario titles brief and meaningful: e.g.
Feature: Context Switching
Scenario: New Context should be enabled
Scenario: Selected facts should be deleted
etc.
I am trying to create row tests using SpecFlow and the Microsoft built-in Test Framework, something along these lines:
Scenario Outline: Test Calculator
Given I have entered <x> into the calculator
And I have entered <y> into the calculator
When I press add
Then the result should be <result> on the screen
Examples:
| x | y | result|
| 1 | 2 | 3|
| 2 | 2 | 4|
The problem I am facing is that given any step in the Scenario Outline a separate step method is auto-generated for each value from the Examples table. I would like to be able to implement for each step a generic method receiving input values as parameters but it just does not seem to work.
In the end it looks like it works as expected, what I was missing were quotes around input parameters placeholders:
Scenario Outline: Test Calculator
Given I have entered "<x>" into the calculator
And I have entered "<y>" into the calculator
When I press add
Then the result should be "<result>" on the screen
Examples:
| x | y | result|
| 1 | 2 | 3|
| 2 | 2 | 4|
I had this same problem in VS 2012. I think it may be a bug with SpecFlow, because when I change the Scenario Outline to only be a Scenario, it generates everything correctly. All the documentation says you should not have to surround the placeholders in quotes.
In short, my solution is to change it to a Scenario to generate the steps. But don't forget, you have to change it back to a Scenario Outline to compile. This is what is working for me.
I am relatively new to BDD and I have a question regarding scenario outlines. When looking at samples over the internet I have the feeling that the placeholders can take any values. The number of elements in their domain is not restricted. Here is one example:
Scenario Outline: eating
Given there are <start> cucumbers
When I eat <eat> cucumbers
Then I should have <left> cucumbers
Examples:
| start | eat | left |
| 12 | 5 | 7 |
| 20 | 5 | 15 |
The placeholder <start> for example can be any number so the number of values is infinite.
In my specs I have to deal with contracts which can have one of four states (planned, ongoing, paused, and closed). My specs say that I can edit planned contracts but I am not allowed to edit contracts which have one of the remaining three states.
I think I would write a scenario named "Updating a planned contract" and one scenario outline where the status of a contract is a placeholder.
Scenario: Update a planned contract
Given the list of contracts as follows
| name | status | some value |
| c1 | planned | 123 |
And I have edited contract c1 as follows
| field | value |
| name | c1 |
| some value | 456 |
When I save contract c1
Then the list of contracts should be as follows
| name | status | some value |
| c1 | planned | 456 |
Scenario Outline: Update contract
Given there is a <status> contract
And I have edited that contract
When I save that contract
Then I an error 'only planned contracts are allowed to change' should be displayed
Examples:
| status |
| ongoing |
| paused |
| closed |
Is that the right way? One expicit scenario and one parameterized? Or should I write the scenario outline as explicit scenarios for each possibility? I am not sure because the status of a contract is restricted by four possible values as opposed to the examples on the internet.
One thing I find that helps is to remember that Gherkin is just a syntax for Specification by Example. You are trying to provide the examples that make most sense to you in the business domains language.
As such, what you are proposing is perfectly valid. You have one example of a scenario that uses tables to define what happens when a planned contract is edited, and another set of examples that produce errors when contracts in other states. You could also do it explicitly by expanding the outline for each state. Both are valid, and you can always refactor your feature sepcifications as you would the codebase.
What you are aiming to do here however is to provide a grammar, a framework, a language, call it what you will, that you can use to have conversations with your business analysts. You want to be able to pull out this document and say "This is how the system works now, how do we change this to make it support your new feature?".
Personally, I'm avoiding tabular and outline forms in my features right now as I want to make it look as friendly as possible to all I show it to, and as yet, my features are still easy to describe.