Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
Suppose I'm designing some SaaS service. And I need to have a function that allow users create sites. User can makes special settings for each site in the admin panel (e.g. design of widget) and gets unique code for install service to his own site.
User story could be:
As a logged user I want to add the new site in the admin panel so that I could configure each instance of widgets separately and could get a unique code for install widget to my own site.
Form
But if I will try to describe this functionality with BDD or GWT (Given When Then) or Gherkin-style, I will face with some trouble. I start from next description:
GIVEN I'm logged into admin panel
AND I'm on "Sites" page
WHEN I click "Add site" button
THEN Pop-up window "Add site" come up
As you can see above realization suppose that site adding will be in pop-up window (e.g. it very important for UX). Pop-up window contains Site URL input field, drop-down control with languages and "Add" and "Cancel" buttons.
And we got a strange scenario which responsible for just pop-up opening. Is it correct? And how can I name this scenario ("Add site's form opening" ??)? Also this scenario has only a one case (when I click - pop-up open). Maybe this scenario not needed at all? I'm confused...
In this case we need to create another scenario when describes:
GIVEN "Add site" pop-up form is opened
WHEN I fill the "Site URL" field
AND click on "Add" button
THEN New site will be create in system
AND I will transfer to my own site's list
How do you think, where do I need to apply a business rules such as:
1) When new site is created a unique code must be generated and consist of minimum 8 characters including numbers and Alphabetical symbols.
2) checks doesn't apply for Site URL input field and user can input a Cyrillic symbols
3) etc?
I have a lot of additional question and hope on the community help!
The thing with BDD is to stay away from implementation details as much as possible. This scenario has multiple implementation details:
GIVEN I'm logged into admin panel AND I'm on "Sites" page
WHEN I click "Add site" button
THEN Pop-up window "Add site" come up
What happens if the "Sites" page becomes "Awesome Site" page or is simply deleted?
What happens if "Add site" is not a button anymore?
What happens if it's not a popup but a redirect occuring
What happens after? Is the value simply in showing the popup? I guess not...
For this specific example, a better approach would be:
GIVEN I'm an authorised administrator
WHEN I enter all the required information for a new site and save it
THEN I should see that site in my own sites list
With this scenario, if your implementation changes, you will only have to change your step definitions, you won't have to change your gherkin. Don't forget that those tests are supposed to explain the Behaviour of the system, not the way it is implemented.
The other questions you have are more related to unit testing in my opinion:
When new site is created a unique code must be generated and
consist of minimum 8 characters including numbers and Alphabetical
symbols => I'd do it at the class level, gherkin would not be
appropriate unless the customer specifically asked for this, then
the condition is "THEN a code having the required characteristics is
generated for that site" and you would have to define "required
characteristics" in a glossary the customer can read and understand.
Checks doesn't apply for Site URL input field and user can input a Cyrillic symbols => again, would put that at the class level en same as for 1. unless the customer wants to be able to read something about it, it should be at the unit level.
I hope that answers your question. I recommend this article by Dan North if you want to have a better idea of how to write better gherkin features.
EDIT 11/13/14
Based on your comments, I suggest we take a step back and describe a way to deal with requirements in your case. I have to tell you that I'm not a BDD expert and am only sharing my own personal experience, for more info on the subject I suggest you get in touch with the guys behind BDD Kickstart and Cucumber.pro where you will find online BDD courses. They will be able to give you lots of information and books to read.
That being said, let's dive into the subject.
The first thing you get is a list of features or stories that, if you follow Mike Cohn's story template would look like:
As a <type of user> I want <to do something> in order to <serve a business purpose>
I personally like to put the business purpose first to make sure we don't skip it in the discussions. You might also not follow that template and that's fine, but remember that it's a good idea to make sure the features you are listing with your customers have a business purpose. If there is no business value behind a feature then what's the point of doing it anyway...
So you do have a list of features/stories described as above. Now for each of these features, there are different cases or scenarios, that's what Dan describes in his article. This is where the Given-When-Then is introduced.
Scenario: Title
Given <some context>
When <there is an event>
Then <something happens>
Each of those scenarios are examples on how this specific feature behaves in different contexts. They are the different acceptance criteria for a specific feature, things the customer described as the expected behaviour of the system. They should be ignorant of any implementation details. So stuff like:
Given I am on page "First page"
When I click "Hello world"
Then I should see "You clicked hello world"
Is wrong for the reasons described prior to this edit.
Let's assume the following feature:
In order to save time when answering clients requests, as a webmaster,
I want to be able to manage the list of websites I am responsible for
Scenarios for this story would be:
Scenario 1: Show a list of websites
GIVEN I am an authorised administrator
AND I am managing several websites
THEN I should see a list of all the sites I manage
Scenario 2: Add website to list
GIVEN I am an authorised administrator
WHEN I enter all the required information for a new site and save it
THEN I should see that site in my own sites list
Scenario 3: Edit website from list
GIVEN I am an authorised administrator
WHEN I edit the site informations
THEN I the changes should be visible in my sites list
...
Now what if you want to go into data validation stuff like "site should have a title" for instance. To me there are two different ways to approach this. You can test that from the user's perspective with a full-stack test or test that there is some validation at the object level.
Let's assume the following scenario:
Scenario: New site has no title
GIVEN I'm an authorised administrator
WHEN I forget to fill in the title for a new site and save it
THEN I should be warned the site is not valid
You can use cucumber or specflow to run this scenario from the UX, therefore using some kind of browse-based proxy to test your app. That is usually slow as it hits the whole system and simulates a real user. It's an option, but I don't think it's the best though. IMO not all tests should be run against the UX and having too many Gherkin features can be a pain to maintain, that's why I prefer focusing on having the happy or critical path (usually I ask myself, where does the money comes from) tested full-stack and put the rest at lower levels.
You can still use Gherkin for these unit tests if you'd like. But that is not mandatory. You only need a way to show your customer you actually have a test for all those specific format controls and validation checks.
That doesn't mean you are not doing BDD anymore, you can still use the given-when-then-should pattern in rspec if you're a rubyist, or any other testing framework you use.
Hope that clarifies all this, let me know if there are any confusing parts...
I think Marc simply deserves the big green tick on this one, thanks to his amazingly thorough answer!
I just wanted to add a few comments.
You don't need to automate all your scenarios.
If you want to capture business requirements in a form that everyone (i.e. including non-tech savvy folks) can understand and Gherkin's Given/When/Then work for you, just go for it. There's nothing forcing you to automate all of your scenarios.
You don't need to automate all your scenarios through the UI.
Your software is made of layers that often respond to similar behaviours, via different interfaces (UI, HTTP, API, ...). Should you want to describe fine-grained business rules (i.e. site name constraints) with automated gherkin scenarios, you could write step definitions that talk directly to your domain layer instead of going through the user interface. That would probably still give you a decent level of confidence.
As a side note, I would recommend not to use Given/When/Then in classic testing frameworks (i.e. those that only devs can see!) if your purpose is to share your tests/requirements with non-tech people.
Have conversations!
Above all, BDD is about better communication: try to talk more, involve your developers (or some of them) earlier in the process so that they gain more knowledge, sooner. Formalising Gherkin scenarios comes in a second phase. Automating them should even be further down your priority list!
Related
I may have this completely wrong, but I've been searching available documentation and googling for 2 weeks now, and have my head completely wrapped around the axle.
I am trying to use SpecFlow to write a regression test for our site. This means that I want to exercise all the features so that if we inadvertently broke something, it will catch it.
The site is basically an incident reporting portal. The home page has about 50 different buttons, each of which opens up the data entry pages for a different class of incident.
The data entry pages are arranged in a "wizard" fashion, where it starts with a page of general questions, then moves on to a page of more specific questions and so on. The questions are more or less grouped in the classic "who/what/when/where/why" grouping, with one wizard page for each group, so that we don't overwhelm the user with 100 questions presented all at once.
Exactly which pages are needed depends on the particular type of incident. Some incident types have as many as 8 pages, some as few as 3.
Our specifications for each page are framed in BDD style - Given/When/Then. So it is very natural to translate those specifications into SpecFlow features, and I have done that, at least for the first page of general information questions. But the Scenario had about 30+ steps in it.
I have also written another Feature for testing from the home page -
Given I'm logged in on the home page
When I clicked the button for XYZ ticket
Then it opens XYZ ticket
And the General Information page is displayed.
And I can drive that scenario from a table so that I can test as many different incident types as I want.
So far so good.
But now I want to add
And the General Information page requirements are verified
Where the step definition for that last clause would run the whole scenario for the general information page. In other words, I want to use that other scenario that I have written as a subroutine in this one.
(And then I want to go on and do the same for each of the other wizard pages. But let's get the first one first!)
I can't figure out a way to do that. I tried writing the step definition for the above clause to invoke the step definitions of the General Information scenario, e.g.
Given("I am on the General Information page")
When ("I click this checkbox")
Then ("This happens")
You used to be able to do that (although that would still be a lot of repetition). But now that's giving a warning message that function is deprecated and will be removed (and since I've now upgraded, it may already have been removed - I haven't tried it since I upgraded.) The github issues page (https://github.com/SpecFlowOSS/SpecFlow/issues/1733 has a lot of discussion on it, none of which sheds any light on how to do what I'm trying to do. The primary author (SabotageAndi) seemed to be saying "That's a bad thing; don't do that" without really giving any alternative, at least none that I was able to understand.
Can anyone give me a direction for how to accomplish what I'm trying to do?
I want to use that other scenario that I have written as a subroutine
in this one.
You can't reuse scenarios defined in feature files.
The best you can do is create a new Step that reuse already defined steps by calling them direcly (jameswtelfer comment on 31 Jan in github issue you provide).
Using the power tool I was able to add a new custom field but I want that it take automatically a specific value if present.
Each test case has a "user story" associated but to associate it we have to go in "Tested User Story Section" so I'd like that in the main page appear automatically the ID of the user story.
Sorry for the confusion maybe with the image will be more understandable:
Someone can advice me of what to introduce in the Visual Studio Power Tool please? because I'm finding different example but none explain how to auto-populate with another dynamic item.
If you are asking me why it's because we want to have the "User Story ID" present in the main page so we know directly at which user story the test case is associated:
There is no way to get the User Story ID as a field. Since a single Test Case can test multiple User Stories this makes little sense anyway.
Have you tried adding Requirement based Suits to Test Professional so that you can see all of the Test Cases under a Story? From a testing perspective it makes lots of sense to look from the Story down to Tests rather than Test up to Stories.
Although my screenshots are from the Web App the features at the same in MTM. You can add a Requirement Based Test Suite that relates back to the User Story.
As #MrHinsh mentioned, there is no direct way to get the User Story ID from a field.
From TFS query, it's easy to get the linked User Story, you can select the Work Item and Direct Links in work item query, check the screenshot below:
Reading online material (e.g. Fowler, Gerard), it seems that Specification By Example stories should not be complete specifications of functionality.
Question 1: How does one starting off with SBE's decide how comprehensive their stories need to be in terms of describing all of the functionality of a system? I.e. when can I stop writing stories because I have captured enough?
Question 2: In an organisation where test teams verify products against the product documentation, if the stores are not a complete specification, am I correct in thinking that 'other' product documentation needs to contain all the cases that are not covered by the SBE's?
Regarding question 1:
The most important part of developing any system is that the development team has a conversation with the product owner. First find out the crux of the feature which they require. I'll answer this question by working through an example; let us say that the product owner may want a facility to login to their new website. This requirement could be written as:
In order to gain access to the website's facilities
As a user
I want to be able to login to the website
(Note that I'm using the Gherkin domain specific language for writing the scenarios and features in this answer)
With the product owner's key requirement specified, you should now discuss with them how you think this feature should be implemeneted from a users perspective (keep it high-level, don't use technical jargon, discuss with the business to find out what they want). So the first "happy path" scenario you might identify could be:
Given a user is on the login screen
When they submit valid login credentials
Then they gain access to the main website
After further discussion with the product owner they tell you that as the website contains extremely sensitive information, and that any failed log-in attempts should be reported to a system administrator. This would result in another scenario:
Given a user is on the login screen
When they submit invalid login credentials
Then the system administrator is informed of the failed log-in attempt
And the user is informed that their login attempt failed
At this point the product owner might say that these are the only scenarios they want for logging into the system. So from the development teams perspective no more investigation would need to be done regarding this feature (so you wouldn't need to write any more user stories). Sure, at a later point in the projects development, the product owner might also tell you that they'd like to inform a user when they last logged into their site before reaching the main website, but you'd only need to worry about this when they ask for it.
Regarding question 2:
The organisation should be verifying the products against "living" documentation e.g. using Cucumber(for example) which generates tests from the scenarios detailed above.
Also as I said in the answer to question 1, you should identify "just enough" of the scenarios/use cases to satisfy the product owner. What the product owner asks for is the complete specification. Don't try and second guess what the product owner might want because this may result in be a classic case of YAGNI.
I have developed a jBehave story to test some work flow implemented in our system.
Let’s say this story is called customer_registration.story
That story is a starting point of some other more complex work flows that our system supports.
Those more complex work flows are also covered by different stories.
Let’s say we have one of our more complex work flows covered by a customer_login.story
So the customer_login.story will look somehow like below:
Story: Customer Login
Narrative:
In order to access ABC application
As a registered customer
I want to login into the application
Scenario: Successfully login into the application
GivenStories: customer_registration.story
Given I am at the login page
When I type a valid password
Then I am able to see the application main menu
All works perfectly and I am happy with that.
3.The story at point 1 above (customer registration) is something I need to run on different sets of data.
Let’s say our system supports i18n and we need to check that customer registration story runs OK for all supported languages, say we want to test our customer registration works OK with both en-gb and zh-tw
So I need to implement a multi_language_customer_registration.story that will look something like that:
Story: Multi language customer registration
Narrative:
In order to access ABC application
As a potential customer
I want to register for using the application
Scenario: Successfully customer registration using different supported languages
GivenStories: customer_registration.story
Then some clean up step so the customer registration story can run again
Examples:
|language|
|en-gb |
|zh-tw |
Any idea about how I could achieve this?
Note that something like below is not an option as I do need to run the clean up step between runs.
GivenStories: customer_registration.story#{0},customer_registration.story#{1}
Moving the clean up step inside the customer registration story is not an option too as then the login story will stop working.
Thanks in advance.
P.S. As you could guess in reality the stories we created are more complex and it is not an easy task to refactor them, but I am happy to do this for a real gain.
First off, BDD is not the same as testing. I wouldn't use it for every single i18n scenario. Instead, isolate the bit which deals with i18n and unit test that, manually test for a couple and call it done. If you really need more thorough then use it with a couple of languages, but don't do it with all of them - just enough examples to give you some safety.
Now for the bit with the customers. First of all, is logging in and registration really that interesting? Are you likely to change them once you've got them working? Is there anything special about logging in or registration that's particular to your business? If not, try to keep that stuff out of the scenarios - it'll be more of a pain to maintain than it's worth, and if it's never going to change you can just test it once, manually.
Scenarios which show what the user is logging in for are usually more enticing and interesting to the business (you are having conversations with the business, right?).
Otherwise, here are the three ways in which you can set up a context (Given):
By hacking the data (so accessing the database directly)
Through the UI (or controller if you're automating from that level)
By using existing data.
You can also look to see if data exists, and if it doesn't, set it up. So for instance, if your customer is registered and you don't want him to be registered, you can delete his registration as part of setting up the context (running the Given step); or if you need him to be registered and he isn't, you can go through the UI to register him.
Lastly, JBehave has an #AfterScenario annotation which you can use to denote a clean-up step for that scenario. Steps are reusable - you can call the steps of the scenario from within another step in code, rather than using JBehave's mechanism (this is more maintainable anyway IMO) and this will allow you to avoid clearing registration when you log in.
Hope one of these options works for you!
From a tactical standpoint, I would do this:
In your .story file
Given I set my language to {language}
When I type a valid password {pass}
Then I am able to see the application main menu
Examples:
|language|pass|
|en-gb |password1|
|zh-tw |kpassword2|
Then in your Java file,
#Given ("I set my language to $lang")
#Alias ("I set my language to {language}")
// method goes here
#When ("I type a valid password $pwrd")
#Alias ("I type a valid password {pass}")
// method goes here
#Then ("I am able to see the application main menu")
most unit testing frameworks support this.
Look how mstest you can specify DataSource, nunit is similar
https://github.com/leblancmeneses/RobustHaven.IntegrationTests
unfortunately some of the bdd frameworks i've seen try to replace existng unit test frameworks when it should instead work together to reuse the infrastructure.
https://github.com/leblancmeneses/BddIsTddDoneRight
is fluent bdd syntax that can be used with mstest/nunit and works with existing test runners.
I am implementing some smoke tests to our website.
I'm using a Given/When/Then format for existing automated acceptance tests/User stories. But now I want to do an initial smoke test of:
Given I'm on the homepage
Then I should see "Welcome To The Site"
Am I missing something? Is it "ok" to not have a When?
Tools Used: MVC3, SpecFlow, Nunit, Watin
It is completely valid syntax to omit any of Given, When or Then (and even to mix them in any order - specflow doesn't care.)
However, for the purpose of readability, rather than omit the When I often rephrase the Given, i.e.
When I view the homepage
Then I should see "Welcome To The Site"
I prefer to omit the Given section, because the When is supposed to indicate what the tested action is.
If the code for the step binding is the same and you want to re-use it, you can always bind your given and my when to the same method.
[Given(#"I'm on the homepage"]
[When(#"I view the homepage"]
public void NavigateToHomePage()
{
...
I think you are really missing the point here. You ALWAYS need a When. That is the thing you should be testing! What you can leave out are the Givens
What you should be saying is;
When I visit the homepage
Then I should see "Welcome To The Site"
Given When Then is really a nicer way of representing a state machine.
Given some initial state // in your case, non
When I perform some action // in your case, visiting the homepage
Then I have some final state // in your case, text displayed to a user
What I like to do is to think about all the things that must be present to allow the When to happen. In your case there doesn't seem to be any initial state. But consider if you had some web application. You would need to have an initial state before visiting the homepage (you'd need to make sure the user is logged in);
Given a user // user must be stored in the database
And the user is logged in // a logged in user must be in the session
When the user visits their homepage
Then the user should see "Welcome To Your Homepage"
An alternative scenario would be;
Given no logged in user // some people would leave this Given out, but I add it for completness
When a user visits their homepage
Then the user should be redirect to the login page
As someone correctly pointed out, most BDD tools don't actually differentiate between Given When Then but you should! The verbose nature of 'Given When Then' has been chosen as its easier for us humans to understand and helps our thought processes. A machine couldn't care less what you call the steps. With this being the case, you should make every effort to utilise the keywords correctly at all times.
Additional
BDD structure is no different to a well set-out test with arrange, act, assert.
The benefit of BDD though is that it gives a verbose structure to tests. This helps devs have domain appropriate conversations with product owners - Behaviour Driven Development.
If you aren't having these conversations, there's very little value in using BDD over normal tests practices.
I tend to see Given as the equivalent of traditional pre-conditions. When as the equivalent of the test action. And Then as the equivalent of the expected result.
Therefore, if there are no pre-conditions, I would leave out Given and simply focus on When and Then:
When I'm on the homepage
Then I should see "Welcome To The Site"
Specflow will allow you to use Given or When, but Visual Studio will also allow you to write a single class that is 1000's of lines long. Just because both are possible, does not mean either is 'right'.
Apologies for the thread resurrection...
I'd have probably gone with:
Given there is a homepage
When I view the homepage
Then I should see "Welcome To The Site"
I like to keep at least one Given, When and Then in each Scenario - don't forget you can also use And and But (not that they're particularly relevant to this scenario). You can even just make a bullet-point style list with *.
I would say:
Given I have requested the home page
When the home page loads
Then I should see 'Welcome To The Site'
You don't need a When. I like the think of the Given/When/Then keywords like
Given - This is a preparation step, do anything you need to be able to perform the test
When - This should be an action that your test will verify.
Then - This should be where you verify your test based on the action performed in the When steps.
As previously suggestion they only affect the execution order.