I have been writing the cucumber tests for the application developed in Ruby on Rails. The application provisions a server (vm) and wait for the result (like success or fail) And displays the result. There are 7-8 different cases (or scenarios) for provisioning a server.
I have 2 questions
How can I club these 2 different scenario outlines in to one. So that I don't need to repeat the table in Examples section. I wanted to create a provsioning requests for all scenarios (Examples) and then wait for 15 min till all requests gets provisioned and after 15 min check if all scenarios are passed or failed.
As these tests take more time to complete, how can I skip them when I run "bundle cucumber" which runs all the feature files. I
don't want to run the following test every time.
I have written cucumber tests by using scenario outline and examples as below .
Scenario Outline: Create A New Server Request
Given user is on create new server request page
When user enters follwing data
| Datacenter | <Datacenter> |
| Cores |<Cores> | |
| Memory | <Memory> |
Then he should see request submitted
Examples:
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
Scenario: Wait till provisioning happens
Given user waits for 15 minutes
Scenario Outline: Verify a New Server Request
Given user is on provision server page
When user check the request with follwing data
| Datacenter | <Datacenter> |
| Cores |<Cores> | |
| Memory | <Memory> |
Then he should see servers are provisioned
Examples:
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
Anupam, I believe the Background holds the key to what you need. Set up your set of servers to work with as a variable using background and then simply refer to those during your scenario. I would also combine the "given the user is on ____ page" since you could bury that implementation detail in your stepdef to provision, check the queue and check the result.
Background: The set of servers to provision
Given the following set of servers are to be provisioned
|DataCenter |Cores| Memory|
| ABC | 2 | 1 GB |
| DEF | 4 | 2 GB |
| GHI | 8 | 4 GB |
| JKL | 4 | 2 GB |
#slow
Scenario: Provision New Servers
When a user provisions the set of servers
Then the user should see the set of servers submitted
When the user waits 15 minutes
Then the user should see the set of servers have been provisioned
To persist the background data across your steps, simply set it to a instance variable on your World object. I would probably use Cucumber::Ast::Table.hashes to get the data into an easier to use data format.
#server_set = table.hashes
As for not running these tests every time, I would set up a cucumber #tag which you use to exclude scenarios that take too long. You can also set up a yaml configuration file to handle this and then just call the appropriate profile in your tests. See the cucumber documentation for examples. https://github.com/cucumber/cucumber/wiki/cucumber.yml
Edit: You could also set up the varioable on your first step def, but I like using background for "data" that you don't want to repeat.
Related
I have a Feature where the Scenarios in the Feature file is logically interconnected - hence my Scenarios cannot be run independently.
Nice: I created a CustomWorld to let Cucumber create and destroy the instance of my framework by itself.
Bad: Cucumber creates and destroys the instance for every scenario. But I want it to be created and destroyed for every feature instead of scenarios.
Here is my feature file
Feature: Table Headers
Scenario: Check the default headers ### My framework instance created here
Given I log in to the application ### A setup
When I navigate to the list page
Then the table should have the below headers
| Default Headers |
| First Name |
| Last Name |
| Age |
Scenario: Add columns ### want to reuse the instance created above and destroy automatically after this scenario
When I add the below columns to display
| Headers |
| City |
| Country |
Then the table should have the below headers
| Default Headers |
| First Name |
| Last Name |
| Age |
| City |
| Country |
And I log out from the application ### A teardown
A bite of my framework:
When I create an instance of my class, a selenium webdriver instance would be created, opens a browser and launches URL.
The real problem: I want to open the browser and launch URL only once per feature and not for every scenario.
This is breaking the rules of BDD and cucumber. You should not have dependencies across Scenario. What I would suggest is that your Given or Background do the setup and that steps are NOT explicit.
Given I am on the list page
|Application|
|###|
Then the table should have the below headers
| Default Headers |
| First Name |
| Last Name |
| Age |
Given I am on the list page
|Application|
|###|
When I add the below columns to display
| Headers |
| City |
| Country |
Then the table should have the below headers
| Default Headers |
| First Name |
| Last Name |
| Age |
| City |
| Country |
Also your final And is not a step and should not be included in your Scenario it should be part of AfterHook
I'm looking my most time consuming transaction on New Relic right now. And I see that "Excon[api.heroku.com]: GET" takes up the majority of the time required. What is it? And how can I reduce it?
Category | Segment % | Time | Avg calls(per txn) | Avg time (ms)
External Excon[api.heroku.com]: GET | 34.3 | 0.653 | 133
View | projects/show.html.erb Template | 9.9 | 0.997 | 38.5
Controller | ProjectsController#show | 9.3 | 1.0 | 36.3
That's an external HTTP request to api.heroku.com. External HTTP calls generally introduce additional execution time because of the IO.
If you want to reduce it you need to determine why your action connects to Heroku and if you can either cache or replace the request.
I have many applications which I want to test, which have a largely overlapping set of features. Here is an oversimplified example of a scenario I might have:
Given <name> is playing a game,
When they shoot at a <color> target
Then they should <event>
Examples:
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
It's a silly example, but suppose really I have a lot of players with different hit/miss conditions, and I want to run just the scenarios for a given name? Say, I only want to run the tests for Alice. There's still advantage to having all the hit/miss tests in a single Scenario Outline (since, after all, they're all closely related).
One approach would be to just duplicate the test for every name and tag them, so something like:
#Alice
Given Alice is playing a game
When she shoots at a <color> target
Then she should <event>
Examples:
| color | event |
| red | hit |
| blue | miss |
This way I can run behave --tags #Alice, But then I'm repeated the same scenario for every user, and that's a lot of duplication. Is there a good way to still compress all the examples into one scenario - but only selectively run some of them? What's the right approach here?
Version 1.2.5 introduced better ways to distinguish scenario outlines. It is now possible to uniquely distinguish them and thus select a unique scenario generated from an outline with --name= at the command line. For instance, suppose the following feature file:
Feature: test
Scenario Outline: test
Given <name> is playing a game,
When they shoot at a <color> target
Then they should <event>
Examples:
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
Let's say I want to run only the test for Bob, red, miss. It is in the first table, 3rd row. So:
behave --name="#1.3"
will select this test. In version 1.2.5 and subsequent versions. A generated scenario gets a name which includes "#<table number>.<row number>" where <table number> is the number of the table (starting from 1) and <row number> is the number of the row.
This won't easily allow you to select all scenarios that pertain to a single user. However, you can achieve it in another way. You can split your examples in two:
Examples: Alice
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
Examples: Bob
| name | color | event |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
The table names will appear in the generated scenario names and you could ask behave to run all the tests associated with one table:
behave --name="Alice"
I do not know of a way to access the example name in steps and thus get rid of the first column.
The full set of details is in the release notes for 1.2.5.
I'm trying to execute an entire SpecFlow Feature using three different UserID/Password combinations. I'm struggling to find a way to do this in the .feature file without having to introduce any loops in the MSTest.
On the Scenario level I'm doing this:
Scenario Template: Verify the addition functionality
Given the value <x>
And the value <y>
When I add the values together
Then the result should be <z>
Examples:
|x|y|z|
|1|2|3|
|2|2|4|
|2|3|5|
Is there a way to do a similar table at the feature level that will cause the entire feature to be executed for each row in the table?
Is there other functionality available to do the same thing?
I don't think the snippet you have is working is it? I've updated the below with the corrections I think you need (as Fresh also points out) and a couple of possible improvements.
With this snippet, you'll see that the scenario is run for each line in the table of examples. So, the first test will connect with 'Bob' and 'password', ask your tool to add 1 and 2 and check that the answer is 3.
I've also added an ID column - that is optional but I find it much easier to read the results with an ID number.
Scenario Outline: Verify the addition functionality
Given I am connecting with <username> and <password>
When I add <x> and <y> together
Then the result should be <total>
Examples:
| ID | username | password | x | y | total |
| 1 | Bob | password | 1 | 2 | 3 |
| 2 | Helen | Hello123 | 1 | 2 | 3 |
| 3 | Dave | pa£sword | 1 | 2 | 3 |
| 4 | Bob | password | 2 | 3 | 5 |
| 5 | Helen | Hello123 | 2 | 3 | 5 |
| 6 | Dave | pa£sword | 2 | 3 | 5 |
| 7 | Bob | password | 2 | 2 | 4 |
| 8 | Helen | Hello123 | 2 | 2 | 4 |
| 9 | Dave | pa£sword | 2 | 2 | 4 |
"Is there a way to do a similar table at the feature level that will
cause the entire feature to be executed for each row in the table?"
No, Specflow (and indeed the Gherkin language) doesn't have a concept of a "Feature Outline" i.e. a way of specifying a collection of features which should be run in their entirety.
You could possibly achiever what you are looking for by making use of Specflow tags to tag related scenarios. You could then use your test runner to trigger the testing of all the scenarios with that tag e.g.
#related
Scenario: A
Given ...etc...
#related
Scenario: B
Given ...etc.
SpecFlow+ Runner (aka SpecRun, http://www.specflow.org/plus/), provides infrastructure (called test targets) to be able to run the same test suite (or selected scenarios) with different settings. With this you can solve problems like the one you have mentioned. It can be also used to run the same web tests with different browsers, etc. Check this screencast for details: http://www.youtube.com/watch?v=RZYV4Dvhw3w
is there any way to tell a cucumber table's diff! method, that I don't care about the row order?
Example:
The feature says:
| start | eat | left |
| 12 | 5 | 7 |
| 20 | 5 | 15 |
The code outputs
| start | eat | left |
| 20 | 5 | 15 |
| 12 | 5 | 7 |
which is ok for me. Cucumber would fail nonetheless, because it also checks for the order (which is nice in most cases).
Couldn't find a solution for it :(
Maybe you can sort the rows in both tables (the test value and the tested value) in a way ensuring a unique order.
I would have expected that there is a corresponding ruby version of the following Java method:
cucumber.api.DataTable#unorderedDiff(cucumber.api.DataTable)
This is under cucumber-core artifact.