we have set of feature file which have scenarios shared between mid-tier and UI using tags ( #services , #UI )
Now when i give this feature file to Specflow . it generates the test-runner wiring for whole feature file.. including #services scenarios
which is useless as we don;t implement them at UI level.
So my question ..is there a way to tell Specflow that only auto-generate test-runner wiring code for scenarios which are tagged with #UI ..?
Apart from the obvious answer of splitting your feature file into UI.feature and Services.feature, then no. There is no way of getting generating multiple .feature.cs files from a single .feature.
However can I suggest that this mixing of steps is indicative that your specifications are crossing domains, which usually suggests that they are might be very technical instead of behaviour driven.
Related
When I am running my test solution, single browser is getting launched but it is running two feature file simultaneously due to which test cases are failing. One step it is taking from one feature file and other from other feature file.
Contrary to the comment left on your question, I think I may have enough context to answer you.
You describe feature files that are sharing steps and concerns about multiple browser instances. This tells me that your various step files might each be containing a browser instance.
What you're likely looking to do instead is to use a SpecFlow Context -- SpecFlow provides it own ScenarioContext object you can use, or you can create your own context and inject it.
Some links that might help:
SpecFlow docs on sharing data between bindings, which explains about ScenarioContext and FeatureContext:
https://docs.specflow.org/projects/specflow/en/latest/Bindings/Sharing-Data-between-Bindings.html
Here's an article on using SpecFlow with Selenium and the Page Object Model: https://docs.specflow.org/projects/specflow/en/latest/ui-automation/Selenium-with-Page-Object-Pattern.html
The SpecFlow YouTube channel will likely be helpful as it's full of experts walking through these sort of examples: https://www.youtube.com/c/SpecFlowBDD/videos
Here's the first video in a 5 part series on how to automate a web application with Selenium and SpecFlow: https://www.youtube.com/watch?v=y1dAogvWVh8
Based on your question, it's also possible that your issue could be that you want things to run in parallel, but have the problem of your tests being dependent on one another or running in a certain order. This will be a bit more complex to solve.
I strongly suggest you treat your tests so that they can be run in isolation. You may need to add separate data to a database, or operate your tests so that they're not touching the same thing. This takes more work, but is more than worth it, because it will enable better maintainability and reliability of your tests and also ensure they can run in parallel successfully.
I hope this helps!
In BDD, how would you deal with the specification of cross cutting features?
Consider for example an application that allows working on a document. There are features like editing text or adding images to the document. Now there's an additional feature "Changelog" that should provide the ability to investigate any change that has been done to a document before.
Now here's my dilemma: Either the "Changelog" gets it's own spec but than it's kind of a never-ending feature. Whenever a new feature for editing the document is added I also need to add something to the "Changelog" feature. Or the "Changelog" is specified in all other features' specs by always sketching out which kind of entry should appear in the changelog after a certain operation. In this case I need to foresee the changelog feature when defining other features, and features that have already been defined and possibly implemented need refinement for the changelog feature.
Any practical advice how to solve this dilemma?
I handle things like this by adding an extra assert step on any scenario that is relevant.
Given some set up
When I use a feature
Then something happens
And it is reflected in the changelog as a 'something happened' entry
The reason I would do it this way, rather than having a separate spec is that it sounds like the two actions are part of the same feature. It wouldn't make sense to me to split them out into separate scenarios. Any existing scenarios that would be broken by a change will be using the same step definition so will be updated when you make this test pass.
The downside of doing it this way is that the relevant tests that you would want to run when making changes to the changelog functionality are distributed through your test suite. I would remedy this by tagging a relevant subset of the tests with #changelog and creating a test run to run only these tests.
We have utilized Specflow and WatIn for acceptance tests at my current project. The customer wants us to use Microsoft coded-ui instead. I have never tested coded ui, but from what I've seen so far it looks cumbersome. I want to specify my acceptance tests up front, before I have a ui, not as a result of some record/playback stuff. Anyway, can someone please tell me why we should throw away the Specflow/watin combo and replace it with coded ui?
I've also read that you can combine specflow with coded ui, but it looks like a lot of overhead for something which I am already doing fine in specflow.
I wrote a blog post on how to do this you might find useful
http://rburnham.wordpress.com/2011/03/15/bdd-ui-automation-with-specflow-and-coded-ui-tests/
The pro's and con's of Coded UI Test that i can think of is your testing the application exactly how the user will be using it. This is good for acceptance test but it also has its limitations. Its also really good for end to end testing. In the past UI Tests have been know to be fragile. For example when MS created the VS2010 UI almost all of the UI tests broke. The main reason being is the technology change. Coded UI tests do help to limit this from happening by the way it matches a control. It uses more of a probability based match. This mean it will try to find the best match based on the information it has such as control name. For us Coded UI tests was our choice because of technology limitations. Our Legacy app is VB and although CUIT does not work great, i'm in the progress of writing an extension to get better control information, it was still our only choice. Also keep in mind CUIT is new and has its own limitations. You should be prepared to be very structured in the way you lay out your project as maintaining your UIMaps can be a bit of manual work due to the current end to end behaviour in VS2010, for example creating a CUIT from an existing action recording always places the test in a UIMap called UIMap.uitest and there is no way to change that or transfer to another UIMap. If you use multiple ui maps this means you will need to record your steps first and then use them in your test. However being in .net it its still very flexible.
By far the best thing about specflow is its gerkin syntax for readability and living documentation. Normally your testing features or behaviours of your app which is where the value comes from It generally aims the test just below the UI. There is a little less chance of the test breaking when the UI changes here but there. Specflow to me is great when your application is under constant change and you want to ensure existing features remain working. It fits well in a Scrum environment as well where you can write your scenario's as a description about how it should work. One limitation to specflow i can see is its open for interpretation. Because of this it can be easy to write a test that is not very reusable and hard to maintain. I like to use more generic terms to describe my steps like "Log in as User1" instead of "Go to Login Page, Enter Username and Password, Click login". Describing it more granular makes it harder to reuse tightly couples it to the UI. How the login actually work should be up to the code behind not the specflow feature.
Combining the 2 however to us seems more beneficial than just using Coded UI Tests. If we decide to completely change the UI we would at least have the behaviours that are expected stored in our specflow features in a way anyone can understand. In the end you need to consider how the application will evolve and the type of application.
I'm using SpecFlow to do some BDD-style testing. Some of my features are UI tests, so they use WatiN. Some aren't UI tests, so they don't.
At the moment, I have a single StepDefinitions.cs file, covering all of my features. I have a BeforeScenario step that initializes WatiN. This means that all of my tests start up Internet Explorer, whether they need it or not.
Is there any way in SpecFlow to have a particular feature file associated with a particular set of step definitions? Or am I approaching this from the wrong angle?
There is a simple solution to your problem if you use tags.
First tag you feature file to indicate that a particular feature needs WatiN like that:
Feature: Save Proportion Of Sample Pool Required
As an <User>
I want to <Configure size of the Sample required>
so that <I can advise the deployment team of resourcing requirments>.
#WatiN
Scenario: Save valid sample size mid range
Given the user enters 10 as sample size
When the user selects save
Then the value is stored
And then decorate the BeforeScenario binding with an attribute that indicates the tag:
[BeforeScenario("WatiN")]
public void BeforeScenario()
{
...
}
This BeforeScenario method will then only be called for the features that use WatiN.
Currently (in SpecFlow 1.3) step-definitions are global and cannot be scoped to particular features.
This is by design to have the same behavior as Cucumber.
I asked the same question on the cucumber group:
http://groups.google.com/group/cukes/browse_thread/thread/20cd7e1db0a4bdaf/fd668f7346984df9#fd668f7346984df9
The baseline is, that the language defined by all the feature files should also be global (one global behavior of the whole application). Therefore scoping definitions to features should be avoided. Personally I am not yet fully convinced about this ...
However your problem with starting WatiN only for scenarios that need UI-Integration can be solved in two different ways:
Tags and tagged hooks: You can tag your scenarios (i.e with #web) and define ina BeforeScenario-Hook that only should run for scenarios with a certain tag (i.e. [BeforeScenario("web")]). See the Selenium integration in our BookShop example: http://github.com/techtalk/SpecFlow-Examples/blob/master/ASP.NET-MVC/BookShop/BookShop.AcceptanceTests.Selenium/Support/SeleniumSupport.cs
We often completely separate scenarios that are bound to the UI and scenarios that are bound to a programmatic API (i.e controller, view-model ...) into different projects. We tried to illustrate this in our BookShop example: http://github.com/techtalk/SpecFlow-Examples/tree/master/ASP.NET-MVC/BookShop/ .
Check this out (new feature in SpecFlow 1.4): https://github.com/techtalk/SpecFlow/wiki/Scoped-Bindings
I originally assumed that a step file was associated with a particular feature file. Once I realized this was not true it helped me to improve all my SpecFlow code and feature files. The language of my feature files is now less context depended, which has resulted in more reusable step definitions and less code duplication. Now I organize my step files according to general similarities and not according to which feature they are for. As far as I know there is no way to associate a step with a particular feature, but I am not a SpecFlow expert so don't take my word for it.
If you still would like to associate your step files with a particular feature file, just give them similar names. There is no need for it to be forced to only work for that feature even if the step code only makes sense for that feature. This is because even if you happen to create a duplicate step for a different feature, it will detect this as an ambiguous match. The behavior for ambiguous matches can be specified in an App.config file. See
http://cloud.github.com/downloads/techtalk/SpecFlow/SpecFlow%20Guide.pdf
for more details the about App.config file. By default ambiguous matches are detected and reported as an error.
[edit]:
Actually there is a problem with working this way (having step files associated with feature files in your mind only). The problem comes when you add or modify a .feature file and use the same wording you have used before, and you forget to add a step for it, but you don't notice this because you already created a step for that wording once before, and it was written in a context sensitive manner. Also I am no longer convinced of the usefulness of not associating step files with feature files. I don't think most clients would be very good at writing the specification in a context independent manner. That is not how we normally write or talk or think.
Solution for this is to implement Tags & Scoped Binding with the test scenario which is related to Web or related to Controller / Core logic in code.
And drill down the scope for each scenario to any of the below mentioned Before / After execution
BeforeTestRunScenario
BeforeFeature
BeforeScenario
BeforeScenarioBlock
BeforeStep
AfterStep
AfterScenarioBlock
AfterScenario
AfterFeature
AfterTestRunScenario
Also consider using implementation-agnostic DSL along with implementation-specific step definitions. For example, use
When I search for 'Barbados'
instead of
`When I type 'Barbados' in the search field and push the Search button
By implementing multiple step definition assemblies, the same Scenario can execute through different interfaces. We use this approach to test UI's, API's, etc. using the same Scenario.
The Grails framework has a lot of constructs/features that allows for adhering to the DRY principle ("don't repeat yourself") within a project. That is: within a specific project you're seldom required to repeat identical blocks of settings or code. So far so good.
However, the more I've worked with Grails the more of I've observed that I repeat code not within the same project but between projects. That is project A has controllers, GSP:s and images that overlaps with project B. This is a maintenance nightmare since bug fixes in project A must also be fixed in project B, etc.
I'd like to take DRY to the next level by not duplicating code between my projects.
My question: How do you tackle this problem (violated "inter-projects DRY") in your own internal Grails projects?
Please be very specific/concrete. If possible try to include specific code examples on how you solve it in practice.
Writing a custom plugin is the best way. You don't need to release it to the public repository, as you can use a private repository somewhere within your own network.
I haven't had enough duplicated code yet to pull out a plugin (most of the code repeated in my projects seem to be covered by the various public plugins), but a plugin can be as simple as a few common domain classes or services.
I agree with Lee, Using common/shared plugins is probably the best way to go. At one place that I worked we had quite a few internal plugins for this very reason.
The most common pattern is to put your common domain objects into their own plugin. This works really well for domain classes or services. We didn't end up refactoring the controllers, views, and static resources into a plugin, but the same principle should apply.
Long story short: Reuse of Grails artifacts = use a plugin.
To add to Lee and Colin's points, which are both valid, I think thinking in terms of multiple plugins can yield other benefits.
For example, you can split up your application functinality into multiple pieces, and have different people work on them. Or it can yield results during deployment, if, say, you need to have two layers of access to an app - user-level and admin - if your domain model is in a separate plugin, as Colin suggested, you can easily build two applications and deploy them separately.
For my app, I have several plugins specific to my project - domain classes plugin, one that is a bunch of code for importing data (which I can run easily against my site), some other plugins for graphing and customization of scaffolding. It takes a bit more thinking, but I expect this factoring will yield dividends in the future as we bring on more people to the team.