I am newbie to BDD trying to understand it.. My understanding about BDD was ..
"Its a Kind of test where the user specfications are used to generate the Ubquitious language from the business"
But the examples i am able to see only the example of UI.. Like when the button is pressed .. when the user enters the text ... this wont form a language which i can use in my code..
Am i wrong in understanding this concept
BDD (Behavior Driven Design) was a term coined by Dan North and the best source to understand his intent is this excellent blog post
Here you can read that Dan want to shift focus from testing details to describe behavior instead. Of course this can (and sure has been) interpreted in oh so many ways :). So where ever you'll turn to you'll get an opinionated view - here is mine.
The idea with Cucumber-based tools, like SpecFlow, is to write down the teams shared understanding of a feature in a language and tool that all involved can read and understand. This is done (again in Cucumber-based tools) by writing down a couple of scenarios or examples on how the feature can be used. Some people call this specfication by example.
Some goodness comes out of writing the specifications by the use of examples in this way:
you can discuss the behavior of the feature before it's implemented
the specifications gives you a great specification to code after, using the outside-in approach
the specifications over time becomes regression tests that verifies that the system behaves as specified
the specification also comes through on a old TDD promise and becomes a living documentation for the system. You can easily see what the current state of an feature is by looking at the executable specification that the feature has passed
So now finally to your question (which by the way is a great one that I often have asked myself and others). Excuse me for rephrasing it, I hope I catch your intent:
Do my scenarios have to (or should they) be run against the UI?
You sure don't have to run the scenarios against the UI - the principles of BDD and the tooling works great going against your domain in any layer.
But to get the most out of your specifications you should consider my (inconclusive) list above. If you don't include the GUI (or the database, or services etc.) then you cannot be sure that the whole application stack works correctly together. So very often the specifications are run End-to-end.
And this makes these "test" something very different than your unit-tests (which you want fast as lightning, mocking out external dependencies, not hitting the database etc.). They take longer time to execute, all of them should not be run on every check-in, the don't use mock etc.
Often you start out with a step of a scenario and as a driver for a behavior and then use ordinary TDD to drive out the details of the internal of the system. This is outside-in programming.
Finally to your example above. So I recommend you to run your specifications against the UI end-to-end all the way to the database; but I would advise describing the UI in technical terms as above (using buttons, links and textboxes for example). When I asked this question on the BDD Google Group I got a great tip from Elisabeth Keogh:
Don't describe the UI. Describe instead what you're trying to achieve with the UI.
So to describe a login feature don't write:
Scenario: Login (describing the UI)
Given I am on the Login-page
When I enter 'AUser' in the textbox 'UserName'
And I enter 'APassword' in the textbox 'Password'
And I click the 'Login' button
Then I should see the following text 'You are logged in'
rather write it something like this:
Scenario: Login (describing what we want to achieve)
Given I am not logged in
When I log in using 'AUser' and 'APassword'
Then I should be logged in
That leaves the complexity on how this is done (click buttons, filling out forms, checking messages etc.) is done in the step definitions that you write in code.
I hope this was helpful. Also I am bracing myself for some "bashing" that can come from other, more experienced BDD-people. But hey, this is my two cents :)
I'm still pretty new to BDD and Specflow too. We've been using it to establish behaviour and code out the controllers, which in turn drive out the views. I don't have a code example in front of me, but I'll try to find something to post later. Cheers!
edit - BTW, if you're looking for a good book on using Cucumber (it uses the same language as Specflow - Gherkin? I'm still getting all the pieces straight), I can highly recommend The RSpec Book from Pragmatic Programmers. The code is Ruby based, but there are some higher level chapters on project definition and running. Well worth the price.
Related
I am currently working for a large organisation with about 2k developers working in our IT department. We maintain many things including our e-commerce platform and there are currently about 30 projects currently impacting that.
Recently all of our teams have been instructed to deliver a series of automated tests using Concordion and Selenium Webdriver. For a while this has been going fairly well and many tests have been created but lately maintaining the existing tests while our e-commerce platform constantly changes has been somewhat of a nightmare. We have thousands of test scripts covering many parts of our website but there does not seem to be any facility in Concordion to split scripts into reusable compartments which could then be maintained once, rather than having to make changes to hundreds of HTML files for one change.
How are other people approaching this?
The goal of Concordion is not to implement test scripts as HTML, but rather for the HTML to describe the behaviour that you are testing (what you are trying to achieve). The implementation details (how it is being tested) are implemented as Java code. This code can then be structured with an appropriate level of abstraction so that each change to the system under test only requires a change to one part of the code.
Your HTML specifications should only need to change on the rare occasions that the business rules change.
These concepts are described further on the Hints and Tips tab of the Concordion home page.
Thank you for sharing your experience with us. It’s great to hear / read about large scale application of behavior driven development / specification by example.
One approach that could help you is to focus on key examples (http://gojko.net/2014/05/05/focus-on-key-examples). During specification workshops the entire team is working to get a common understanding of the new user needs and requirements. Then you go on and write specification documents containing key examples. There you should not try to cover everything, but to write only as many examples as necessary to express the common understanding.
Additionally, you should try to identify concepts, on which the examples are based. Are there some examples related to a similar topic – this is probably an underlying concept. It is often easier to understand the examples, if they focus just on one concept (e.g. the validation of a card number). Each concept can be usually described with only a few examples.
Do you have any other types of automated tests (e.g. unit tests)? Are you experiencing the same maintainability challenges with these other tests? Could you use good practices from these other test types to improve your Concordion approach?
Could you tell us more about your setup? How many active specifications have you already created within your company?
I am starting on BDD. Was wondering which would be better to start with Cucumber or Spinach. My impression is that Spinach is new off the block. Look here
Which one should I start with. The criteria would be -
Support across the board.
Flexibility of use
Third party tool and APIs integration.
Again it might be ignorant question of the newbie: Where does capybara fit into the picture.
For some context, I've been a long time user of Cucumber, but always wished it was Spinach since day one. I'm switching all my projects to Spinach despite its shortcomings because it uses the new, hot-off-the-block PORO technique (Plain Old Ruby Objects ;). Now I can expand my steps however I want, because it's just Ruby.
To answer your question, as of this writing:
Support across the board.
Cucumber
Spinach is still developing some features, including 'Background' blocks, and I'm currently trying to get it to recognize tables.
Flexibility of use
Spinach
Cucumber encourages bad step design from the start, IMO. If you create feature-specific steps, you'll trip over them later, and if you create reusable global steps, your feature definitions will be long, generic, and boring to read. I've heard people claim they can walk a balance successfully and be just specific enough but still have reusable steps; I consider myself well versed enough that if I can't do it reliably, it's too hard.
Third party tool and APIs integration
Cucumber, assuming the bullet point could be interpreted as community.
If it's really "third party tools and API integration" you're after, Spinach supports capybara and rspec, which is most of what you're after. Cucumber has 3rd party reusable step libraries, but as noted in my earlier point, I think this is bad. In regard to 3rd party & integrations, even if it's not there yet, your really can't get any better than plain old ruby objects.
Where does capybara fit into the picture
Capybara is your test interface to your site, aka a testing mouse & keyboard. You could start it up in a console and drive your app, but that'd get repetitive. Cucumber/Spinach (or rspec/test-unit/minitest) all could use capybara to automate testing your app. People prefer Cucumber/Spinach because they help you step out of the code for a bit to think like a user.
Overall, you'd probably be best off getting an rspec/cucumber book and doing what it says. Just be aware that testing takes a while to get good at, so don't stop there. Maybe check out Spinach somewhere in the process; if you like Cucumber, you might find you'll really like Spinach.
DISCLAIMER: I'm a Spinach mantainer.
If you're starting with BDD I'd highly recommend two books:
The RSpec book
The Cucumber book
I think it's important to learn all the BDD and TDD process (outside-in etc..) and then choose the tool you feel more comfortable with.
Having said that, Cucumber has a huge community, but a lot of things are also aplicable to Spinach, since what they have in common is Gherkin.
As for flexibility of use I would say both are really flexible, but I (obviously) prefer Spinach as every feature it's just a Ruby class, where you can include modules, inherit from other classes and so on (this also applies to APIs integration).
I you want, you can take a look at the spinach-rails-demo and see how everything works.
If you're stuck with Cucumber and you don't want global steps, you can work around the problem by tagging the steps with some sort of scenario ID:
# features/1_greetings.feature
Scenario: Formal greeting
Given I have an empty array [#1]
And I append my first name and my last name to it [#1]
When I pass it to my super-duper method [#1]
Then the output should contain a formal greeting [#1]
The #1 scenario id can be any value. I like to use ticket numbers for future reference.
You can then place all the steps in one step definition file. It's close enough to the look of Spinach::FeatureSteps. No regex arguments too!
# features/step_definitions/1_greetings.rb
Given 'I have an empty array [#1]' do
#...
end
And 'I append my first name and my last name to it [#1]' do
#...
end
When 'I pass it to my super-duper method [#1]' do
#...
end
Then 'the output should contain a formal greeting [#1]' do
#...
end
I posted more about the workaround at github.
I can't really speak for Spinach, as I've never used it, but Cucumber definitely has a huge community support with loads of external libraries.
Capybara allows you to easily test web applications
When I fill in "username" with "foo"
And I click on "login"
Then I should see "enter your password"
I'm new to unit/func testing in Rails 3. So I'm starting now, better late than never.
I have a method in /lib/mailingjob.rb called find_reply(body)
Right now I'm using cucumber to test this but given that this is all backend, no web interface to this, should I not be using cucumber?
I'm finding when to use RPSEC vs cucumber confusing.
Thanks
In my opinion, you need to use both. Rspec is very good for unit testing, that is testing models, controllers, views. On the other hand, cucumber is a very nice tool to check full scenarios like a user logs in, clicks a link and he is supposed to view this.
I highly advise that you take a look at the cucumber railscast from railscasts.com. Also, make sure that you use webrat and maybe something to auto load your specs like watchr(which i prefer).
I'd advise you to never use Cucumber unless you are specifically employed as a business analyst. That makes your life much easier: all your tests will be in Rspec.
Cucumber takes a toll on your productivity, and your sanity. Running parallel test environments, adding layers of abstraction and breaking your editor with its odd syntax make for lost time. I've written about this extensively on my blog:
Why Bother With Cucumber Testing?
If I didn't already have a Stack Overflow account, I would have made one JUST to write this post. I urge you to re-read #Jack Kinsella's answer
I got in to BDD about 3 months ago, fell in love with it, and quickly started using Cucumber for everything. I forced it in to every project, no matter what
Recently, I've had to learn a yet another language, and my first step was to get the testing environment set up. I came across these 3 articles:
http://robots.thoughtbot.com/post/25650434584/writing-better-cucumber-scenarios-or-why-were
http://robots.thoughtbot.com/post/25653308407/describe-the-users-perspective-ddd-acceptance
http://37signals.com/svn/posts/3159-testing-like-the-tsa
Honestly, their content isn't that important; what's important is that they hinted at an overall idea. I began to suspect that I'd been using Cucumber wrong all along, and I went hunting for answers. This morning I found your SO question, and #Jack Kinsella's blog post
#Jack comes right out and says the idea I believe those articles were tentatively circling. He also gives me the language I was looking for. I now consider his article to be the final word on the subject :)
According to him, what we've ACTUALLY been using Cucumber for is "integration testing". I never understood what that meant before
Unit testing:
How the code works internally. Math.Add(1,1) should be 2, but the website user doesn't care. Just give me a web page!
-> Use RSpec or an equivalent
Integration testing:
How different branches of the code work together to make the website. I type my name and click Log In, and should be taken to the Home page
-> ALSO use RSpec!
(Inside of RSpec, add whatever you need to handle multiple-technologies-touching-eachother. For the code-to-webBrowsers example: Capybara, Watir, etc)
Acceptance testing:
Many of us don't need it. Someone signed a contract with you saying "I'm going to write 'I can add subpages to my micro-site.' in a text file". You write me code that makes it turn green
-> Use Cucumber. Only if you have to do this kind of testing. Which you almost certainly don't
What an elegant solution. No 2nd testing environment. No steps directory and swarm of extra files!
I had a moment of "But I love how Cucumber seperates English from the code". Regular bdd does that too. "rspec --format nested" or Jasmine test results
#Jack is right. Cucumber isn't adding anything; not the way we've been using it. And it's costing a lot. To quote him:
Why not admit to yourself that you don’t do acceptance testing and that you do not need it in your projects? Swap Cucumber for pure integration tests using Capybara, and you’ll be surprised by how much more productive you can be
To complete SpyrosP's answer, there is an awesome blog post from Sarah Mei which describes a scenario where you use both Rspec and Cucumber. It is called oustide-in Behavior Driven Development and you can find it here.
However, I think most of Jack's problem can be solved by a good cucumber editor. So the problem is not Cucumber. The problem is that there wasn't a capable editor when he wrote this two years ago. But today RubyMine has full cucumber editing capability. While it is not free, it can solve most of Jack's problem: Auto completing, pointing out errors, refactoring. So you probably want to go back and check the points he raised, and see if they are still valid today.
My take so this issue many years later is very simple: If you do not know what to use now, just start with RSpec, because it is the central stone for all your testing that you will always need.
With time you will discover that two big testing needs:
Is my code still working?
Can my users still perform the actions already implemented?
As a developer I tend to think that (1) is for every single code change and (2) when I finish a feature and/or deploy.
Having two test suites is nice because during a deployment, you can sit and run a very generic test suite: Are my user features still working? Can I finally deploy this?
Indeed, it should be so generic that if something breaks in (2) you need to do some heavy research (one row missing in your tables) while a fail in (1) should pin-point the exact class with the problem.
Normally I find myself in the situation where I want separate integration+acceptance testing without the Cucumber burden. First I use RSpec tags to filter out these tests. Whenever I need more I like going for Steak+Capybara since it enables Cucumber like objectives on top of something I already know quite well.
Definitely use both. Use cucumber for user experience tests(views), use rspec for testing internals(models, etc.). You may write controller rspec tests but I think they are not necessary(try to move the logic to models).
Whatever your system is, I strongly advise you to use Cucumber because scenarios will let you and your stakeholders know the functions of your system in a clear written fashion.
You will also have your documentation ready and you'll always know where you are and where you are heading. You'll also relate scenarios with support issues in the future.
Cuke them right by the way don't use variables in scenarios, they should be understandable.
And if you want them to run fast as rspecs use poltergeist(phantomjs).
I've been tempted not use cucumber because in many cases it's not simple as rspec. However every time I understood that I needed scenarios to take full control on the project.
I'd like to enable the business analysts to be able to write all of their specs for features, scenarios and steps that is Cucumber friendly using Gherkin.
I've read some of the basic info on the github site for Cucumber and from doing a quick Google search but wanted to know if there were recommended resources for getting non-technical folks to be able to write comprehensive BDD using Gherkin (I assume that's the preferred language for Cucumber tests to be created in).
Thanks.
What I did with the business analysts in our company was to teach them the structure by giving them the keywords: Given, When, Then, And for Scenarios and In order to, As a and I want to for Features.
Then I gave them a simple example and told them to write down their own features as they thought they should be written. Surprisingly enough the structure was self explanatory and the features they wrote became a great start.
The only big problem was that they had contained to much logic in each scenario step. I solved that by iteratively asking "why?" which in most cases revealed the core functionality they were after and we re-wrote the scenarios accordantly.
By giving them the guidelines and letting them write the features themselves they got their hands dirty and were forced to think about what they wrote. Today they have a much better understanding and the "why?" iterations are not that common anymore.
Ofcourse you need to have the business analysts and the developers to work closely together and the features the analysts write should only act as a start. Remember that the Cucumber features are just a common language between the analysts and the developers. They still need to sit together often to be able to speak with each other :)
http://cukes.info is a great resource for teaching people how to write them. Ben Mabey also made a great presentation on Cucumber at Mountain West Ruby Conference 2009.
Having just worked on an agile project using cucumber for the first time I think that the best way to learn Cucumber and Gherkin is to get your hands dirty.
I may be wrong but I get the impression from your question you are wanting to train your BAs to write Gherkin; then they will write a bunch of features and hand them to developers.
This is definitely not the way to go. It is much better to have BA's devs and users (if possible) working together to write your scenarios and build them as you go. Then you all learn together what works and what doesn't.
We tried having a BA write entire features and hand them over. We (the devs) ended up having to do major rewrites because the implementation ended up different to that originally envisioned by the BA. We also had to change the syntax of the steps and do find and replace through the whole file.
Do one scenario at a time, get it working then move on to the next. An iterative approach reduces wasted effort and makes sure you all understand how you want the app to behave.
In terms of how to write steps it is best to start with the ones that come with Cucumber and copy and adapt them as you work on your project to fit your particular application. There is no right or wrong, it is what works for you. The documentation on the cucumber sites is generally good and will be a valuable resource as you learn more.
We are teaching Gherkin (for SpecFlow) in a similar way, how mrD has described it.
I think it is very important though, that the audience is familiar with the main intention of "Specification by Example", agile requirement analysis and BDD, so we usually start discussing the background first. We also show a sample Gherkin scenario and explain the very basics (like Given/When/Then/But and tables).
Than we take a simple example story (that is quite familiar to everyone), like "add items to shopping cart" (with some orientation, of course) and let them formulate the acceptance criteria in small groups.
After that every team shows / explains their solutions and we discuss the good and bad practices that were present. After the second team, you can see almost all of the most important (good or bad) practices appearing.
I also type in the concluded solution, and show here alternative ways of describing the scenarios (background, scenario outline, etc.). If there is enough time, I also show how to automate & implement the imagined functionality based on that. This also help to understand some important rules to follow, that makes the automation much easier.
Although, I never know upfront what will happen, usually this exercise is the best part of our BDD training.
The RSpec book has a couple of chapters in it that are relevant to Business Analysts:
http://pragprog.com/book/achbd/the-rspec-book
I think the best way to learn is to start writing. Gherkin & Cucumber are easy to learn but difficult to master, so it’s important to get to practical examples as soon as possible.
While it’s important to get started by writing your first scenarios, you also need some resources to establish good habits and understand key practices. I wrote a book that could help. “Writing Great Specifications” is, I hope, a good way to learn Gherkin and Cucumber. It covers patterns and antipatterns as well as key techniques for writing great scenarios. :) If you have any questions, you can always hit me up on Twitter.
If you are interested in buying “Writing Great Specifications,” you can save 39% with the promo code 39nicieja2 :)
Other great resources:
“Specification by Example” by Gojko Adzic if you’re interested in software development processes and high-level engineering practices.
“BDD in Action” by John Smart if you don’t mind reading testing code in Java. It’s a comprehensive end-to-end view on defining and testing software requirements.
“Behaviour-Driven Development” by Liz Keogh if automated testing doesn’t ring a bell, but you want to understand how specifications with examples affect your business analysis processes.
“The Cucumber Book: Behaviour-Driven Development for Testers and Developers” by Matt Wynne and Aslak Hellesøy
“The RSpec Book: Behaviour-Driven Development with RSpec, Cucumber, and Friends” by David Chelimsky, Dave Astels, Zach Dennis, Aslak Hellesøy, Bryan Helmkamp, Dan North
Best way to learn Gherkin is to read the Behat documents first: http://behat.readthedocs.org
Then read official documents from cucumber site: https://cucumber.io/docs/reference
Also, you can read Guru99 article to write your first cucumber script: http://www.guru99.com/your-first-cucumber-script.html
I'm a Java developer trying to learn Grails, and I'd like to get exposure to as many parts of the Grails framework as possible. Preferably doing so by solving small real-world problems using the "Grails way to do it" (DRY, convention-over-configuration, and so on).
Three example could be:
Learn about GORM by creating a few classes (say Movie, Actor, etc.) and relations/mappings between them (1:1, 1:N, N:N, etc.).
Learn about the layout support (sitemesh) by using it to generate headers and footers common to all GSP:s on the site.
Learn about the filter support by using it to make sure all accesses to a certain controller comes from authenticated users.
My question goes to all Grails developers out there - what would you include in a "Grails curriculum" and in what order?
All input appreciated!
Here's some examples, but be warned that they're fairly trivial and don't really show you how the system works together. One of the strengths of Grails is that the different parts all combine to reduce your code complexity and speed development. I recommend doing a single project of moderate size (like blogging software or a photo gallery) that forces you to touch virtually everything. I'm currently working on a ticket management application, and I've had to learn basically everything in the framework. It's really not that much material, actually.
That being said, here's my list of "must study", along with some examples:
Groovy, especially closures, maps, and properties. If you're coming from Java, closures might seem a little strange at first. However, once you wrap your head around them, it'll be hard to go back to a language that doesn't use them. Maps and properties use ideas that might be familiar, but the syntax and usage is different enough that it's worth studying them closely. Grails uses these three things ALL THE TIME, all throughout the framework. For a good example, examine the "BeanBuilder" that instantiates the Spring beans defined in resources.groovy. Also, run through the Groovy documentation at groovy.codehaus.org. A couple of hours there will save you DAYS down the road.
MVC programming. The "MVC" model in Grails pretty closely matches the one used in Rails, but it's significantly different than the "MVC" model used in Java desktop applications. Basically, all incoming URL requests are a message to a controller, which returns a view. Domain objects are the data that you want to store, view, and manipulate through views and controllers. Do an input form that validates the user's input using constraints, and then manipulates it somehow using a controller. Something like a page that takes in your birthday, and returns your Zodiac sign and Chinese Zodiac animal. See if you can get it to return errors to the user when bad input is given.
GORM. GORM is super-important, but you'll be forced to learn it with virtually any project you pick. Give the documentation a once-over, just so you know what its' capabilities are.
Filters and Services. These are "the grails way" to do a lot of DRY programming. Authentication is a canonical example, and it's perfect for learning filters. For services, write something that will send out email. There's a great example of a simple emailer service on the Grails website.
Groovy Server Pages. If you've worked with a templating engine before, then this should seem familiar. Get to know the GSP tag library, it's a huge help. Practical examples include: virtually anything. Every application needs a front-end. Try and make it pretty. NOTE: This spills into a lot of stuff that isn't Grails-specific, like JavaScript, CSS, etc. Unless you have that knowledge already, prepare for a bit of a learning curve.
Your "conf" directory. Get to know every file in there, especially UrlMappings.groovy. Play with UrlMappings so that you have an app that takes meaningful information from the URL. Something like /myapp/calculate/36/times/145, where the app returns an answer.
I'd say those are the basics, but there's a lot of other topics like webflows, i18n, testing, session handling, and so on. The best way to learn those is by building a decent sized project. While you're doing that, you'll probably find yourself thinking, "Gosh, I wish that Grails did ____". Read through the excellent documentation on Grails.org, and you'll probably find a built-in capability or plugin that does what you want. The reference PDF lives on my desktop, and I've found it invaluable during my learning experience.
Oh, and look at the scaffolding code that Grails generates. You'll probably end up pitching it all out, but it'll give you a good idea of how the system works.
Have fun, and happy hacking!
Step 1 - Learn Groovy
If you already know Java, I highly recommend Programming Groovy. It's a lot more concise and up-to-date than the otherwise excellent Groovy in Action. Neither of these books cover the significant language changes in Groovy 1.6, so you should also read this page.
Step 2 - Learn Grails
The Definitive Guide to Grails is probably the standard Grails reference - make sure you get the second edition. Grails in Action is slightly more recent, but I haven't read it so can't comment further on it. I found TDGTG a little lighton GORM, so you might also consider checking out Grails Persistence with GORM and GSQL. It's a very short book, but worth it's weight in gold.
Step 3 - Dive In
Try and modify the sample app in the Grails book, or build your own from scratch. The Groovy console is a great way to experiment with snippets of Groovy code.
If the audience is not familiar with programming in Groovy there should be an introduction to that. Java will work but it gets the juices going when you see how less verbose the code is in Groovy. When discussing GORM include constraints and how they influence validation. Scaffolding is a real time saver when starting a new project so be sure to include it. One of the features of Grails that really has helped me is Plug-ins. Pick a few and show how they provide solutions while saving development time. A security plug-in would fit right into the filter topic you mention. Testing, can there ever be enough testing?
I would really recommend reading the Definite Guide to Grails, Second Edition. It covers everything you need to know about writing applications in Grails. It probably lacks the "what happens under the hood" knowledge, but you should get the hang of it. You can buy it as a PDF and start reading it immediately.
You should also have a list of plug-ins to use - Grails has some really nice ones that come in handy. I can tell you some of the ones I use, but that may be a good question here, too. :-)