In Ruby on Rails, if all of the validation rules for a given model are being tested within that model's spec (or unit tests), is it still considered necessary to write a Cucumber scenario for each validation?
Would it suffice to instead write two scenarios: One for when valid data is entered, and one for when invalid data is entered?
This is a good question, and the answer is: It depends.
You can think of Cucumber as a way of communicating between the product owner, developers and testers.
If you feel that having the validations in Cucumber adds to the shared understanding of what the product does, then keep them there.
One approach is to combine the validations into a scenario outline:
Scenario Outline: User tries to register but skips a mandatory field
Given I am registering
And I leave the "<field>" blank
When I click "Submit"
Then I should see "<message>"
And I should not be registered
| field | message |
| Forename | Please enter your forename |
| Surname | Please enter your surname |
| Date of Birth | Please enter your date of birth |
This is a great question, and one that I have been dealing with recently.
It may be different for your organization, but in my organization we try to leave the field validation tests to the unit tests or some other framework that handles these situations better. Aside from that, I am of the school of thought that Cucumber is meant purely for automated acceptance testing (AKA a communications tool between you/your team and the PO) -- and that other methods should be used for anything outside that scope.
Why leave anything open to suggestion? Allowing for assumptions made by developers is inherently risky in terms of generating rework.
How each field a form behaves under different circumstances (create, display, edit, persona) is clearly acceptance criteria.
If you hide it away in a unit test then you are disconnecting it from the living documentation
You really shouldn't need unit tests if you are really bought into the BDD way.
Feature: Edit staff personal
Scenario Outline: Form validation
Given I am editing a staff personal details
And the form contains a "<Mandatory?>" field with a label "<Label>"
And text fields have a input length of between "<Min Length>" and "<Max Length>"
And select fields have these "<Options>"
When I submit the form by clicking the save button
Then an error displays if validation fails
But commits my changes if validation is successful and returns the form back to display mode
Examples:
| Label | Mandatory? | Type | Min length | Max length | Options
| Title | true | select | 0 | 0 | Mr, Mrs, Miss, Ms, Dr, Prof |
| Surname | true | text | 2 | 50 | null |
| Forename | true | text | 2 | 50 | null |
| Known as / Other Surname | false | text | 2 | 50 | null |
| Known as forename | false | text | 2 | 50 | null |
| Date of birth | true | date-picker | 0 | 0 | null |
| NI number | true | ni-number | 0 | 0 | null |
Related
I have a Google Sheet with one page for team leaders to list their desired team members (first names, last names, emails) by row--each TL fills in one row--, and a second page where team members are listed who have actually registered with my program.
Page 1
+------------------------+------------+---------------+------------+--------------------+
| Team Leader First Name | First Name | Email Address | First Name | Email Address |
+------------------------+------------+---------------+------------+--------------------+
| Danielle | Elizabeth | XXX#tamu.edu | Matthew | XXX#tamu.edu |
| Stoian | William | XXX#tamu.edu | Victoria | XXX#email.tamu.edu |
| Christa | Olivia | XXX#tamu.edu | | |
+------------------------+------------+---------------+------------+--------------------+
Page 2
+--------------------+-------------------------+
| Scholar First Name | Scholar Preferred Email |
+--------------------+-------------------------+
| elizabeth | xxx#gmail.com |
| william | xxx#tamu.edu |
+--------------------+-------------------------+
I want to be able to see at a glance which of the names listed by the TL on pg 1 have not registered and thus don't appear on pg 2.
In the example above, I want Olivia, Matthew, and Victoria's names to appear red because she does not show up on pg2 (which means they still need to register). Everyone else should appear normally.
I tried at first to importrange from pg1 to get a clean list of the team members, then conditional formatting to match against pg2, the idea I had being it shows up red if a name is not found.
Import range from page 2 to page 1 the scholar first name to F12:F14
Conditional Formatting: Apply to range B2:B999(first name list in page 1)
=NOT(OR(ISNUMBER(MATCH(TRIM(B2),$F$12:$F$13,0)),ISBLANK(B2)))
Conditional Formatting2: Apply to range D2:D999(Second First name list)
=NOT(OR(ISNUMBER(MATCH(TRIM(D2),$F$12:$F$13,0)),ISBLANK(D2)))
Note: Instead of importing, You could also reference the second sheet using INDIRECT.
I have many applications which I want to test, which have a largely overlapping set of features. Here is an oversimplified example of a scenario I might have:
Given <name> is playing a game,
When they shoot at a <color> target
Then they should <event>
Examples:
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
It's a silly example, but suppose really I have a lot of players with different hit/miss conditions, and I want to run just the scenarios for a given name? Say, I only want to run the tests for Alice. There's still advantage to having all the hit/miss tests in a single Scenario Outline (since, after all, they're all closely related).
One approach would be to just duplicate the test for every name and tag them, so something like:
#Alice
Given Alice is playing a game
When she shoots at a <color> target
Then she should <event>
Examples:
| color | event |
| red | hit |
| blue | miss |
This way I can run behave --tags #Alice, But then I'm repeated the same scenario for every user, and that's a lot of duplication. Is there a good way to still compress all the examples into one scenario - but only selectively run some of them? What's the right approach here?
Version 1.2.5 introduced better ways to distinguish scenario outlines. It is now possible to uniquely distinguish them and thus select a unique scenario generated from an outline with --name= at the command line. For instance, suppose the following feature file:
Feature: test
Scenario Outline: test
Given <name> is playing a game,
When they shoot at a <color> target
Then they should <event>
Examples:
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
Let's say I want to run only the test for Bob, red, miss. It is in the first table, 3rd row. So:
behave --name="#1.3"
will select this test. In version 1.2.5 and subsequent versions. A generated scenario gets a name which includes "#<table number>.<row number>" where <table number> is the number of the table (starting from 1) and <row number> is the number of the row.
This won't easily allow you to select all scenarios that pertain to a single user. However, you can achieve it in another way. You can split your examples in two:
Examples: Alice
| name | color | event |
| Alice | red | hit |
| Alice | blue | miss |
Examples: Bob
| name | color | event |
| Bob | red | miss |
| Bob | blue | hit |
| Bob | green | hit |
The table names will appear in the generated scenario names and you could ask behave to run all the tests associated with one table:
behave --name="Alice"
I do not know of a way to access the example name in steps and thus get rid of the first column.
The full set of details is in the release notes for 1.2.5.
I'm trying to execute an entire SpecFlow Feature using three different UserID/Password combinations. I'm struggling to find a way to do this in the .feature file without having to introduce any loops in the MSTest.
On the Scenario level I'm doing this:
Scenario Template: Verify the addition functionality
Given the value <x>
And the value <y>
When I add the values together
Then the result should be <z>
Examples:
|x|y|z|
|1|2|3|
|2|2|4|
|2|3|5|
Is there a way to do a similar table at the feature level that will cause the entire feature to be executed for each row in the table?
Is there other functionality available to do the same thing?
I don't think the snippet you have is working is it? I've updated the below with the corrections I think you need (as Fresh also points out) and a couple of possible improvements.
With this snippet, you'll see that the scenario is run for each line in the table of examples. So, the first test will connect with 'Bob' and 'password', ask your tool to add 1 and 2 and check that the answer is 3.
I've also added an ID column - that is optional but I find it much easier to read the results with an ID number.
Scenario Outline: Verify the addition functionality
Given I am connecting with <username> and <password>
When I add <x> and <y> together
Then the result should be <total>
Examples:
| ID | username | password | x | y | total |
| 1 | Bob | password | 1 | 2 | 3 |
| 2 | Helen | Hello123 | 1 | 2 | 3 |
| 3 | Dave | pa£sword | 1 | 2 | 3 |
| 4 | Bob | password | 2 | 3 | 5 |
| 5 | Helen | Hello123 | 2 | 3 | 5 |
| 6 | Dave | pa£sword | 2 | 3 | 5 |
| 7 | Bob | password | 2 | 2 | 4 |
| 8 | Helen | Hello123 | 2 | 2 | 4 |
| 9 | Dave | pa£sword | 2 | 2 | 4 |
"Is there a way to do a similar table at the feature level that will
cause the entire feature to be executed for each row in the table?"
No, Specflow (and indeed the Gherkin language) doesn't have a concept of a "Feature Outline" i.e. a way of specifying a collection of features which should be run in their entirety.
You could possibly achiever what you are looking for by making use of Specflow tags to tag related scenarios. You could then use your test runner to trigger the testing of all the scenarios with that tag e.g.
#related
Scenario: A
Given ...etc...
#related
Scenario: B
Given ...etc.
SpecFlow+ Runner (aka SpecRun, http://www.specflow.org/plus/), provides infrastructure (called test targets) to be able to run the same test suite (or selected scenarios) with different settings. With this you can solve problems like the one you have mentioned. It can be also used to run the same web tests with different browsers, etc. Check this screencast for details: http://www.youtube.com/watch?v=RZYV4Dvhw3w
I've got a Rails application in which we want to change the wording depending on a profile the user picks so that it's suited to them. It's all in english but just different wording. For instance some people call 'Suppliers', 'creditors' or 'vendors' depending on if they are accountants or not.
I was thinking of something like this
User has one profile
Profile has many users
Profile has many words
Problem with this is I would have to make sure that in the words table I have the same set of words for each profile. eg:
Words table
| profile_id | Name | Value |
|------------+----------+----------|
| 1 | creditor | customer |
| 2 | creditor | creditor |
| 3 | creditor | supplier |
Where the profile table maybe
Profiles
| ID | Name |
|----+------------------|
| 1 | Service Industry |
| 2 | Accountant |
| 3 | Basic |
Update:
Later down the track I also plan to use i18n in this project.
Assuming you're not going to make this a multilingual app, you may want to just leverage the existing i18n system (which supports fallback to a default locale for each term). You could define a 'Service Industry' locale, an 'Accountant' locale, etc. and have the default locale ('en') contain all of the terms.
This also wouldn't require database hits. Assuming the terms are reasonably static, that's an advantage. If you're frequently changing the custom terms, then this might not be a good approach.
If I have the following steps defined, is this valid scenario? I feel like it is some kind of smell.
Scenario: Change users status
Given I have the following users exist:
| code | status |
| u1 | active |
| u2 | inactive |
| u3 | active |
And the status filter is "active"
When I update "u1" to "inactive"
Then I should see the following users:
| code |
| u3 |
When I change status filter to "inactive"
Then I should see the following users:
| code |
| u1 |
| u2 |
Liz is right about describing capabilities of system. I would also suggest to have only one When in scenario. Because each scenario (I think) should verify that system goes from Given state to Then state When something happens.
In this case I also suggest to have two different features. One feature is about your application can change user status (activate/deactivate):
Feature: Changing user status
Scenario: Deactivating user
Given following users exist:
| code | status |
| u1 | active |
| u2 | inactive |
| u3 | active |
When user u1 is deactivated
Then following users exist:
| code | status |
| u1 | inactive |
| u2 | inactive |
| u3 | active |
Another feature is about you can filter users by status:
Feature: Filtering users
Scenario: Filtering active users
Given following users exist:
| code | status |
| u1 | active |
| u2 | inactive |
| u3 | active |
When I filter for active users
Then I should see the following users:
| code |
| u1 |
| u3 |
Scenario: Filtering inactive users
Given following users exist:
| code | status |
| u1 | active |
| u2 | inactive |
| u3 | active |
When I filter for inactive users
Then I should see the following users:
| code |
| u2 |
In this case when one of the scenarios is broken, you will know the reason. It's either not changing status of user, or not filtering them properly. You know which feature is broken in your application.
You're describing it in quite code-driven terms, but otherwise it's a valid scenario.
If you wanted to make it better, you could describe it in terms of the capabilities of the system, in the language that users might use to describe what they're doing:
Scenario: Change users status
Given these users exist:
| code | status |
| u1 | active |
| u2 | inactive |
| u3 | active |
And we filter for active users
When I disable user u1
Then I should see the following users:
| code |
| u3 |
When we filter for inactive users
Then I should see the following users:
| code |
| u1 |
| u2 |
You could also use more typical usernames so that people reading it understand what those names represent at a glance (Lunivore, Soe, Jon instead of u1 and u2).
Not so much difference. What did you identify as a bad smell? Was it just the language?
Typically it is good to just have one 'when' step, because that is what you actually test. However, sometimes I find it also useful to specify whole use cases that might include several then and when steps that depend on each other. For example:
when a new user registers
then the user receives an email confirmation
when the email confirmation is confirmed by the user
then the user is registered
In your example, however, you really should write two tests, because you actually test two different features that also do not directly depend on each other.
Overall
BDD is really Design-by-Contract using different terms. Generally speaking, BDD is in the form of Given-When-Then, which is roughly analogous to Preconditions (Given), Check-conditions/Loop-invariants (When), and Post-conditions/Invariants (Then).
Notice
Note that BDD is very much Hoare-logic (i.e. {P}C{Q} or {P}recondition-[C]ommand-{Q}Post-condition). Therefore:
Preconditions (Given) must hold true for the command (method/function) to compute correctly. Any violation of the Given (precondition) signals a fault in the calling Client code.
Command(s) (When) are what happens after the precondition(s) are met. In Eiffel, they can be punctuated within the method or function code with other contracts. Think about these as though they are QA/QC checks along a process assembly line.
Post-conditions (Then) must hold true once the Command (When) is finished.
Moral of the Story
Because BDD is just DbC (Hoare-logic) repackaged in different words, this means it is not TDD. Why? Because TDD is not about preconditions/checks/post-condition contracts tied directly to methods, functions, properties, and class-state. TDD is the next step up the ladder in testing methods, functions, properties, and classes with their discrete states. Once you see this and fully appreciate that TDD is not BDD and BDD is not TDD, but that they are separate and complementary technologies for software correctness proofs—THEN—you will finally understand these topics correctly. You will also use and apply them correctly.
Conclusion
Eiffel is the only language I am aware of where BDD (Design-by-Contract) is baked raw into both the language specification and compiler. It is not a Frankenstein bolt-on monster with limitations. In Eiffel, BDD (aka DbC) is an elegant, helpful, useful, and direct participant in the software correctness toolbox.
See Also
Wikipedia helps defined Hoare-logic. See: https://en.wikipedia.org/wiki/Hoare_logic
I have created an example in Eiffel that you can look at. See:
Primary class: https://github.com/ljr1981/stack_overflow_answers/blob/main/src/so_73347395/so_73347395.e
Test class: https://github.com/ljr1981/stack_overflow_answers/blob/main/testing/so_73347395/so_73347395_test_set.e