I know BDD and also read Specification by Example the great book by Gojko Adzic; As I understand they are almost the same and have many in common; But I was not able to understand their main difference. I mean Can we use them interchangeably?
General idea
BDD is the format of describing tests in a form of "this functionality does this and that". If requirements are written in the same format, then we end up with the whole team using BDD. Because these requirements can be reused for testing.
Specs by example is.. well, putting a particular example into the requirements.
Testing formats
BDD-for-testing can come in different forms:
Given When Then
Gherkin (a more strict dialect of G/W/T used by Cucumber family of frameworks)
Describe-it (Jasmine, Mocha, etc)
Or something custom. Could be same old JUnit with tests named as "adminHasAccessToProfileOfOthers" instead of older style "testAdminCanAccessProfileOfOthers"
Requirements
As for BDD-for-requirements, here's an example:
If user has sufficient amounts of money, they can buy the product
And here is Specs-by-example:
If user has 10$ and the product is 9$, user can buy that product
See, in the 1st example we used abstract ideas, while in the latter one we used particular values.
Oftentimes when people talk about BDD/Specs-by-example, they would write requirements using Given-When-Then. But as an abstract idea this isn't compulsory - it's just what most testing frameworks support it.
Overall, BDD is a very good idea for testing, especially if it's not a Given-When-Then :) But when it comes to requirements - I haven't seen any project in my experience that really benefited from it.
As for other common misconceptions: BDD and how it does NOT relate to TDD.
Related
We have been using waterfall to develop and enhance the system in our company and recently we are moving into Agile and the management is interested in BDD.
My understanding is that BDD is a tool for business and development team to refine the stories and at the same time these scenarios will become test cases for the feature. This sounds perfect to me but since we already have the features available, how can BDD work in this type of situation?
Should we just write up the stories and scenarios per our knowledge of the feature?
My only concern of the above is the coverage of the scenarios. Or we shouldn't worry and keep adding new scenarios and test it whenever the team came up with new ones?
Prompted by yet another person who mailed me with the same question today, I've written up an answer for this.
Short version, you can use BDD to help you understand what the system actually does, and why, but you'll be clarifying the requirements rather than exploring them.
Additionally, you asked, "Should we just write up the stories and scenarios per our knowledge of the feature?"
I'd speak with any stakeholders you can find, ask them what the system should do, then look to see if it actually does it. Systems designed before adopting a practice of conversations with examples often don't do what the originators intended. You can then differentiate between the behaviour you've actually got, while creating a new backlog from the behaviour you want.
I advise grabbing someone who's good at asking questions and spotting missing scenarios to have these discussions with (usually a tester). Because you already have some knowledge of the system, it's likely you'll be very good at describing what you think it does, while missing gaps.
If you don't have any automated testing yet and you want to start using BDD, I would suggest you to start by writing some Scenarios for some of your manual tests scripts, I find it to be a good way to train the writing of the BDD style, then as Lunivore said, you should work together with the business people and QA to find out better about the behaviour of your system and preferably write the scenarios with them.
I am starting on BDD. Was wondering which would be better to start with Cucumber or Spinach. My impression is that Spinach is new off the block. Look here
Which one should I start with. The criteria would be -
Support across the board.
Flexibility of use
Third party tool and APIs integration.
Again it might be ignorant question of the newbie: Where does capybara fit into the picture.
For some context, I've been a long time user of Cucumber, but always wished it was Spinach since day one. I'm switching all my projects to Spinach despite its shortcomings because it uses the new, hot-off-the-block PORO technique (Plain Old Ruby Objects ;). Now I can expand my steps however I want, because it's just Ruby.
To answer your question, as of this writing:
Support across the board.
Cucumber
Spinach is still developing some features, including 'Background' blocks, and I'm currently trying to get it to recognize tables.
Flexibility of use
Spinach
Cucumber encourages bad step design from the start, IMO. If you create feature-specific steps, you'll trip over them later, and if you create reusable global steps, your feature definitions will be long, generic, and boring to read. I've heard people claim they can walk a balance successfully and be just specific enough but still have reusable steps; I consider myself well versed enough that if I can't do it reliably, it's too hard.
Third party tool and APIs integration
Cucumber, assuming the bullet point could be interpreted as community.
If it's really "third party tools and API integration" you're after, Spinach supports capybara and rspec, which is most of what you're after. Cucumber has 3rd party reusable step libraries, but as noted in my earlier point, I think this is bad. In regard to 3rd party & integrations, even if it's not there yet, your really can't get any better than plain old ruby objects.
Where does capybara fit into the picture
Capybara is your test interface to your site, aka a testing mouse & keyboard. You could start it up in a console and drive your app, but that'd get repetitive. Cucumber/Spinach (or rspec/test-unit/minitest) all could use capybara to automate testing your app. People prefer Cucumber/Spinach because they help you step out of the code for a bit to think like a user.
Overall, you'd probably be best off getting an rspec/cucumber book and doing what it says. Just be aware that testing takes a while to get good at, so don't stop there. Maybe check out Spinach somewhere in the process; if you like Cucumber, you might find you'll really like Spinach.
DISCLAIMER: I'm a Spinach mantainer.
If you're starting with BDD I'd highly recommend two books:
The RSpec book
The Cucumber book
I think it's important to learn all the BDD and TDD process (outside-in etc..) and then choose the tool you feel more comfortable with.
Having said that, Cucumber has a huge community, but a lot of things are also aplicable to Spinach, since what they have in common is Gherkin.
As for flexibility of use I would say both are really flexible, but I (obviously) prefer Spinach as every feature it's just a Ruby class, where you can include modules, inherit from other classes and so on (this also applies to APIs integration).
I you want, you can take a look at the spinach-rails-demo and see how everything works.
If you're stuck with Cucumber and you don't want global steps, you can work around the problem by tagging the steps with some sort of scenario ID:
# features/1_greetings.feature
Scenario: Formal greeting
Given I have an empty array [#1]
And I append my first name and my last name to it [#1]
When I pass it to my super-duper method [#1]
Then the output should contain a formal greeting [#1]
The #1 scenario id can be any value. I like to use ticket numbers for future reference.
You can then place all the steps in one step definition file. It's close enough to the look of Spinach::FeatureSteps. No regex arguments too!
# features/step_definitions/1_greetings.rb
Given 'I have an empty array [#1]' do
#...
end
And 'I append my first name and my last name to it [#1]' do
#...
end
When 'I pass it to my super-duper method [#1]' do
#...
end
Then 'the output should contain a formal greeting [#1]' do
#...
end
I posted more about the workaround at github.
I can't really speak for Spinach, as I've never used it, but Cucumber definitely has a huge community support with loads of external libraries.
Capybara allows you to easily test web applications
When I fill in "username" with "foo"
And I click on "login"
Then I should see "enter your password"
I am trying to get started with BDD and found a view blog posts about MSpec and SpecFlow. I'm currently not quite sure when I would use which and what the advantages/disadvantages of either framework are.
Looking at the documentation it seems that MSpec uses the context specification style whereas SpecFlow uses Given/When/Then style. I don't really mind either but I would like to know if there are any pitfalls to watch out for further down the track when the project/test suite grows.
Basically some real world advice/feedback of someone who uses it in their every day work would be great.
So I've used both.
I like the mspec workflow in away because its an easier sell for me to speak to users and say.
"When logging in"
"I should return to the page I requested"
When I've worked for organisations that have bought more into active collaboration (read agile) I've used the Given When Then pattern. That organisation was used to user stories so they were used to a more rigid style of specification. Also we were using more than one tool to feed the specs into. so the 'text only' feature files could be reused between tools.
In my own projects I use SpecFlow for the 'outside' and 'mspec' for inside of tests.
If I was to give someone advice it would be to use specflow if non technical people are writing the outside specs and mspec if a developer is writing the.
Bad points:
Mspec is class explosion
SpecFlow is a slower workflow
Good points:
Mspec is a more natural language
Specflow is better for reusability for steps.
The bottom line is they work well together.
One disadvantage of mspec is you cannot run in parallel whereas with specflow runner you can. That is a big performance issue.
I am newbie to BDD trying to understand it.. My understanding about BDD was ..
"Its a Kind of test where the user specfications are used to generate the Ubquitious language from the business"
But the examples i am able to see only the example of UI.. Like when the button is pressed .. when the user enters the text ... this wont form a language which i can use in my code..
Am i wrong in understanding this concept
BDD (Behavior Driven Design) was a term coined by Dan North and the best source to understand his intent is this excellent blog post
Here you can read that Dan want to shift focus from testing details to describe behavior instead. Of course this can (and sure has been) interpreted in oh so many ways :). So where ever you'll turn to you'll get an opinionated view - here is mine.
The idea with Cucumber-based tools, like SpecFlow, is to write down the teams shared understanding of a feature in a language and tool that all involved can read and understand. This is done (again in Cucumber-based tools) by writing down a couple of scenarios or examples on how the feature can be used. Some people call this specfication by example.
Some goodness comes out of writing the specifications by the use of examples in this way:
you can discuss the behavior of the feature before it's implemented
the specifications gives you a great specification to code after, using the outside-in approach
the specifications over time becomes regression tests that verifies that the system behaves as specified
the specification also comes through on a old TDD promise and becomes a living documentation for the system. You can easily see what the current state of an feature is by looking at the executable specification that the feature has passed
So now finally to your question (which by the way is a great one that I often have asked myself and others). Excuse me for rephrasing it, I hope I catch your intent:
Do my scenarios have to (or should they) be run against the UI?
You sure don't have to run the scenarios against the UI - the principles of BDD and the tooling works great going against your domain in any layer.
But to get the most out of your specifications you should consider my (inconclusive) list above. If you don't include the GUI (or the database, or services etc.) then you cannot be sure that the whole application stack works correctly together. So very often the specifications are run End-to-end.
And this makes these "test" something very different than your unit-tests (which you want fast as lightning, mocking out external dependencies, not hitting the database etc.). They take longer time to execute, all of them should not be run on every check-in, the don't use mock etc.
Often you start out with a step of a scenario and as a driver for a behavior and then use ordinary TDD to drive out the details of the internal of the system. This is outside-in programming.
Finally to your example above. So I recommend you to run your specifications against the UI end-to-end all the way to the database; but I would advise describing the UI in technical terms as above (using buttons, links and textboxes for example). When I asked this question on the BDD Google Group I got a great tip from Elisabeth Keogh:
Don't describe the UI. Describe instead what you're trying to achieve with the UI.
So to describe a login feature don't write:
Scenario: Login (describing the UI)
Given I am on the Login-page
When I enter 'AUser' in the textbox 'UserName'
And I enter 'APassword' in the textbox 'Password'
And I click the 'Login' button
Then I should see the following text 'You are logged in'
rather write it something like this:
Scenario: Login (describing what we want to achieve)
Given I am not logged in
When I log in using 'AUser' and 'APassword'
Then I should be logged in
That leaves the complexity on how this is done (click buttons, filling out forms, checking messages etc.) is done in the step definitions that you write in code.
I hope this was helpful. Also I am bracing myself for some "bashing" that can come from other, more experienced BDD-people. But hey, this is my two cents :)
I'm still pretty new to BDD and Specflow too. We've been using it to establish behaviour and code out the controllers, which in turn drive out the views. I don't have a code example in front of me, but I'll try to find something to post later. Cheers!
edit - BTW, if you're looking for a good book on using Cucumber (it uses the same language as Specflow - Gherkin? I'm still getting all the pieces straight), I can highly recommend The RSpec Book from Pragmatic Programmers. The code is Ruby based, but there are some higher level chapters on project definition and running. Well worth the price.
I'd like to enable the business analysts to be able to write all of their specs for features, scenarios and steps that is Cucumber friendly using Gherkin.
I've read some of the basic info on the github site for Cucumber and from doing a quick Google search but wanted to know if there were recommended resources for getting non-technical folks to be able to write comprehensive BDD using Gherkin (I assume that's the preferred language for Cucumber tests to be created in).
Thanks.
What I did with the business analysts in our company was to teach them the structure by giving them the keywords: Given, When, Then, And for Scenarios and In order to, As a and I want to for Features.
Then I gave them a simple example and told them to write down their own features as they thought they should be written. Surprisingly enough the structure was self explanatory and the features they wrote became a great start.
The only big problem was that they had contained to much logic in each scenario step. I solved that by iteratively asking "why?" which in most cases revealed the core functionality they were after and we re-wrote the scenarios accordantly.
By giving them the guidelines and letting them write the features themselves they got their hands dirty and were forced to think about what they wrote. Today they have a much better understanding and the "why?" iterations are not that common anymore.
Ofcourse you need to have the business analysts and the developers to work closely together and the features the analysts write should only act as a start. Remember that the Cucumber features are just a common language between the analysts and the developers. They still need to sit together often to be able to speak with each other :)
http://cukes.info is a great resource for teaching people how to write them. Ben Mabey also made a great presentation on Cucumber at Mountain West Ruby Conference 2009.
Having just worked on an agile project using cucumber for the first time I think that the best way to learn Cucumber and Gherkin is to get your hands dirty.
I may be wrong but I get the impression from your question you are wanting to train your BAs to write Gherkin; then they will write a bunch of features and hand them to developers.
This is definitely not the way to go. It is much better to have BA's devs and users (if possible) working together to write your scenarios and build them as you go. Then you all learn together what works and what doesn't.
We tried having a BA write entire features and hand them over. We (the devs) ended up having to do major rewrites because the implementation ended up different to that originally envisioned by the BA. We also had to change the syntax of the steps and do find and replace through the whole file.
Do one scenario at a time, get it working then move on to the next. An iterative approach reduces wasted effort and makes sure you all understand how you want the app to behave.
In terms of how to write steps it is best to start with the ones that come with Cucumber and copy and adapt them as you work on your project to fit your particular application. There is no right or wrong, it is what works for you. The documentation on the cucumber sites is generally good and will be a valuable resource as you learn more.
We are teaching Gherkin (for SpecFlow) in a similar way, how mrD has described it.
I think it is very important though, that the audience is familiar with the main intention of "Specification by Example", agile requirement analysis and BDD, so we usually start discussing the background first. We also show a sample Gherkin scenario and explain the very basics (like Given/When/Then/But and tables).
Than we take a simple example story (that is quite familiar to everyone), like "add items to shopping cart" (with some orientation, of course) and let them formulate the acceptance criteria in small groups.
After that every team shows / explains their solutions and we discuss the good and bad practices that were present. After the second team, you can see almost all of the most important (good or bad) practices appearing.
I also type in the concluded solution, and show here alternative ways of describing the scenarios (background, scenario outline, etc.). If there is enough time, I also show how to automate & implement the imagined functionality based on that. This also help to understand some important rules to follow, that makes the automation much easier.
Although, I never know upfront what will happen, usually this exercise is the best part of our BDD training.
The RSpec book has a couple of chapters in it that are relevant to Business Analysts:
http://pragprog.com/book/achbd/the-rspec-book
I think the best way to learn is to start writing. Gherkin & Cucumber are easy to learn but difficult to master, so it’s important to get to practical examples as soon as possible.
While it’s important to get started by writing your first scenarios, you also need some resources to establish good habits and understand key practices. I wrote a book that could help. “Writing Great Specifications” is, I hope, a good way to learn Gherkin and Cucumber. It covers patterns and antipatterns as well as key techniques for writing great scenarios. :) If you have any questions, you can always hit me up on Twitter.
If you are interested in buying “Writing Great Specifications,” you can save 39% with the promo code 39nicieja2 :)
Other great resources:
“Specification by Example” by Gojko Adzic if you’re interested in software development processes and high-level engineering practices.
“BDD in Action” by John Smart if you don’t mind reading testing code in Java. It’s a comprehensive end-to-end view on defining and testing software requirements.
“Behaviour-Driven Development” by Liz Keogh if automated testing doesn’t ring a bell, but you want to understand how specifications with examples affect your business analysis processes.
“The Cucumber Book: Behaviour-Driven Development for Testers and Developers” by Matt Wynne and Aslak Hellesøy
“The RSpec Book: Behaviour-Driven Development with RSpec, Cucumber, and Friends” by David Chelimsky, Dave Astels, Zach Dennis, Aslak Hellesøy, Bryan Helmkamp, Dan North
Best way to learn Gherkin is to read the Behat documents first: http://behat.readthedocs.org
Then read official documents from cucumber site: https://cucumber.io/docs/reference
Also, you can read Guru99 article to write your first cucumber script: http://www.guru99.com/your-first-cucumber-script.html