I'm using SpecFlow to do some BDD-style testing. Some of my features are UI tests, so they use WatiN. Some aren't UI tests, so they don't.
At the moment, I have a single StepDefinitions.cs file, covering all of my features. I have a BeforeScenario step that initializes WatiN. This means that all of my tests start up Internet Explorer, whether they need it or not.
Is there any way in SpecFlow to have a particular feature file associated with a particular set of step definitions? Or am I approaching this from the wrong angle?
There is a simple solution to your problem if you use tags.
First tag you feature file to indicate that a particular feature needs WatiN like that:
Feature: Save Proportion Of Sample Pool Required
As an <User>
I want to <Configure size of the Sample required>
so that <I can advise the deployment team of resourcing requirments>.
#WatiN
Scenario: Save valid sample size mid range
Given the user enters 10 as sample size
When the user selects save
Then the value is stored
And then decorate the BeforeScenario binding with an attribute that indicates the tag:
[BeforeScenario("WatiN")]
public void BeforeScenario()
{
...
}
This BeforeScenario method will then only be called for the features that use WatiN.
Currently (in SpecFlow 1.3) step-definitions are global and cannot be scoped to particular features.
This is by design to have the same behavior as Cucumber.
I asked the same question on the cucumber group:
http://groups.google.com/group/cukes/browse_thread/thread/20cd7e1db0a4bdaf/fd668f7346984df9#fd668f7346984df9
The baseline is, that the language defined by all the feature files should also be global (one global behavior of the whole application). Therefore scoping definitions to features should be avoided. Personally I am not yet fully convinced about this ...
However your problem with starting WatiN only for scenarios that need UI-Integration can be solved in two different ways:
Tags and tagged hooks: You can tag your scenarios (i.e with #web) and define ina BeforeScenario-Hook that only should run for scenarios with a certain tag (i.e. [BeforeScenario("web")]). See the Selenium integration in our BookShop example: http://github.com/techtalk/SpecFlow-Examples/blob/master/ASP.NET-MVC/BookShop/BookShop.AcceptanceTests.Selenium/Support/SeleniumSupport.cs
We often completely separate scenarios that are bound to the UI and scenarios that are bound to a programmatic API (i.e controller, view-model ...) into different projects. We tried to illustrate this in our BookShop example: http://github.com/techtalk/SpecFlow-Examples/tree/master/ASP.NET-MVC/BookShop/ .
Check this out (new feature in SpecFlow 1.4): https://github.com/techtalk/SpecFlow/wiki/Scoped-Bindings
I originally assumed that a step file was associated with a particular feature file. Once I realized this was not true it helped me to improve all my SpecFlow code and feature files. The language of my feature files is now less context depended, which has resulted in more reusable step definitions and less code duplication. Now I organize my step files according to general similarities and not according to which feature they are for. As far as I know there is no way to associate a step with a particular feature, but I am not a SpecFlow expert so don't take my word for it.
If you still would like to associate your step files with a particular feature file, just give them similar names. There is no need for it to be forced to only work for that feature even if the step code only makes sense for that feature. This is because even if you happen to create a duplicate step for a different feature, it will detect this as an ambiguous match. The behavior for ambiguous matches can be specified in an App.config file. See
http://cloud.github.com/downloads/techtalk/SpecFlow/SpecFlow%20Guide.pdf
for more details the about App.config file. By default ambiguous matches are detected and reported as an error.
[edit]:
Actually there is a problem with working this way (having step files associated with feature files in your mind only). The problem comes when you add or modify a .feature file and use the same wording you have used before, and you forget to add a step for it, but you don't notice this because you already created a step for that wording once before, and it was written in a context sensitive manner. Also I am no longer convinced of the usefulness of not associating step files with feature files. I don't think most clients would be very good at writing the specification in a context independent manner. That is not how we normally write or talk or think.
Solution for this is to implement Tags & Scoped Binding with the test scenario which is related to Web or related to Controller / Core logic in code.
And drill down the scope for each scenario to any of the below mentioned Before / After execution
BeforeTestRunScenario
BeforeFeature
BeforeScenario
BeforeScenarioBlock
BeforeStep
AfterStep
AfterScenarioBlock
AfterScenario
AfterFeature
AfterTestRunScenario
Also consider using implementation-agnostic DSL along with implementation-specific step definitions. For example, use
When I search for 'Barbados'
instead of
`When I type 'Barbados' in the search field and push the Search button
By implementing multiple step definition assemblies, the same Scenario can execute through different interfaces. We use this approach to test UI's, API's, etc. using the same Scenario.
Related
When I am running my test solution, single browser is getting launched but it is running two feature file simultaneously due to which test cases are failing. One step it is taking from one feature file and other from other feature file.
Contrary to the comment left on your question, I think I may have enough context to answer you.
You describe feature files that are sharing steps and concerns about multiple browser instances. This tells me that your various step files might each be containing a browser instance.
What you're likely looking to do instead is to use a SpecFlow Context -- SpecFlow provides it own ScenarioContext object you can use, or you can create your own context and inject it.
Some links that might help:
SpecFlow docs on sharing data between bindings, which explains about ScenarioContext and FeatureContext:
https://docs.specflow.org/projects/specflow/en/latest/Bindings/Sharing-Data-between-Bindings.html
Here's an article on using SpecFlow with Selenium and the Page Object Model: https://docs.specflow.org/projects/specflow/en/latest/ui-automation/Selenium-with-Page-Object-Pattern.html
The SpecFlow YouTube channel will likely be helpful as it's full of experts walking through these sort of examples: https://www.youtube.com/c/SpecFlowBDD/videos
Here's the first video in a 5 part series on how to automate a web application with Selenium and SpecFlow: https://www.youtube.com/watch?v=y1dAogvWVh8
Based on your question, it's also possible that your issue could be that you want things to run in parallel, but have the problem of your tests being dependent on one another or running in a certain order. This will be a bit more complex to solve.
I strongly suggest you treat your tests so that they can be run in isolation. You may need to add separate data to a database, or operate your tests so that they're not touching the same thing. This takes more work, but is more than worth it, because it will enable better maintainability and reliability of your tests and also ensure they can run in parallel successfully.
I hope this helps!
I'm developing an application in rails with cucumber.
The application includes a workflow that have multiple steps.
Some steps are
A user import files (3 different files),
Other user make make some checks to date that was imported,
Other user input some parameter,
Other user apply the parameter to the data that was imported,
etc.
The steps must be executed in the correct order, and I is necessary to run all the previous steps in order to execute each one, for example to apply the parameter its necessary to have the data imported and the parameters defined.
My problem is how to build cucumber scenarios/features in this situation.
I know that a scenario is not suppose to call all the previous scenario.But the only other idea that I have is to create a very long scenario performing all this steps, and that make sense because it will be a scenario with more than 2 hundred steps.
Any thought on a pragmatical way of implementing cucumber in this kind of situation?
Many Tks
It sounds as if you have to perform every thing every time.
Will every usage of your system include importing three files? Are there any cases where the user may only need to import two files? If the case is that there will always be three files imported, then you might abstract that step as
given the files are imported
Things that always have to be done may be combined into some generic setup. As the setup never changes, the details may not be necessary to mention explicit.
My experience though, is that at the beginning it is hard to separate scenarios and try to do too much in a few scenarios with many steps. If you don't see any other way, start there. Look at your scenario and see if they possible to separate into perhaps two independent scenarios. It may be possible to separate it into two scenarios that are independent. Next step would be to see if these two new scenarios are possible to divide into two smaller, independent scenarios. It happens that it is possible.
It is obviously always possible that Cucumber is not the tool you need. It is possible that you would be better off with a unit test framework.
In BDD, how would you deal with the specification of cross cutting features?
Consider for example an application that allows working on a document. There are features like editing text or adding images to the document. Now there's an additional feature "Changelog" that should provide the ability to investigate any change that has been done to a document before.
Now here's my dilemma: Either the "Changelog" gets it's own spec but than it's kind of a never-ending feature. Whenever a new feature for editing the document is added I also need to add something to the "Changelog" feature. Or the "Changelog" is specified in all other features' specs by always sketching out which kind of entry should appear in the changelog after a certain operation. In this case I need to foresee the changelog feature when defining other features, and features that have already been defined and possibly implemented need refinement for the changelog feature.
Any practical advice how to solve this dilemma?
I handle things like this by adding an extra assert step on any scenario that is relevant.
Given some set up
When I use a feature
Then something happens
And it is reflected in the changelog as a 'something happened' entry
The reason I would do it this way, rather than having a separate spec is that it sounds like the two actions are part of the same feature. It wouldn't make sense to me to split them out into separate scenarios. Any existing scenarios that would be broken by a change will be using the same step definition so will be updated when you make this test pass.
The downside of doing it this way is that the relevant tests that you would want to run when making changes to the changelog functionality are distributed through your test suite. I would remedy this by tagging a relevant subset of the tests with #changelog and creating a test run to run only these tests.
we have set of feature file which have scenarios shared between mid-tier and UI using tags ( #services , #UI )
Now when i give this feature file to Specflow . it generates the test-runner wiring for whole feature file.. including #services scenarios
which is useless as we don;t implement them at UI level.
So my question ..is there a way to tell Specflow that only auto-generate test-runner wiring code for scenarios which are tagged with #UI ..?
Apart from the obvious answer of splitting your feature file into UI.feature and Services.feature, then no. There is no way of getting generating multiple .feature.cs files from a single .feature.
However can I suggest that this mixing of steps is indicative that your specifications are crossing domains, which usually suggests that they are might be very technical instead of behaviour driven.
We have a complex Fitnesse suite with tens of subsuites, includes and symbolic links. Sometimes we don't want to run the whole thing and would like to run
selected test cases and not run the others. We see two ways to do it:
By managing page properties (Suite - Test - Normal), we can turn on/off test cases.
But this is inconvenient. First of all, it's boring. And second, we can't see the suite current state (what test cases are turned on and gonna be run).
In Fitnesse, there are tags, and we can specify desired tags in suiteFilter or excludeSuiteFilter.
This is inconvenient too. You have to remember tag names and don't forget or misspell them in filters. Of course, we can store predefined links with carefully selected lists of tags, but in our case it's not the option, since the lists are subject to frequent changes.
Also, we don't want to divide our suite on several parts, since we benefit from having the general Scenario Library and variable lists.
The ideal solution for us would be having a Fitnesse suite configurator, which could display and change settings of Fitnesse pages. Say, it may be a Fitnesse plugin, which read Fitnesse folder structure and display current settings in an html page or with Windows Form, letting change these settings and save the changes. Or an external tool with similar functionality.
Have you heard of such tools? Did you bump into the same troubles? What would be your suggestions?
I would agree that the first option you listed, manipulating the page properties is a bad idea. It will cause pain in the log run.
I would note that tags area very reasonable approach. The thing to keep in mind about tags filets is that you can build links that will run all of the tests tagged a specific value and make that a part of the FrontPage
For example, you can put a link in your FrontPage that will run all tests marked "smoke".
[[Run Smoke Tests][.FrontPage.MonsterSuite?suite&suiteFilter=smoke]]
There is one other variation on selective execution you could do, but I've not been as successful with. Take a look at SuiteQuery: http://fitnesse.org/FitNesse.UserGuide.TestSuites.SuiteQuery.
SuiteQuery is a technique that lets you specify a suite by building a table that lists the pages or page name filters to run.
!|Suite|
|Page|FitNesse.SuiteAcceptanceTests|
|Content|[Bb]ug|
!|Suite|
|Page|FitNesse.SuiteAcceptanceTests|
|Title|Import|
There is an another way..
Create a new suite and add the following code to it..
!see .FrontPage.TestPage