I am exploring using behave in testing a chatbot.
I can define simple scenarios like
Scenario: Asked a widget function question
Given a new dialog
When the visitor asks
"""
What do your widgets do?
"""
Then bot should explain widget function
Scenario: Asked a widget cost question
Given a new dialog
When the visitor asks
"""
How much does your widget cost?
"""
Then bot should provide widget cost
The trouble is that a good chat bot needs to preserve context. If it successfully preserves context, then sequence matters. If both of the above scenarios pass, it would not mean that a dialog where those questions were asked one after the other wouldn't throw an error.
For that you'd need a scenario like this
Scenario: Asked a widget function question and price question
Given a new dialog
When the visitor asks
"""
What do your widgets do?
"""
And the bot explains widget function
When the visitor asks
"""
How much does your widget cost?
"""
Then bot should provide widget cost
Of course, writing a scenario that has a dialog preamble in the Given block for each dialog one wants to test would be extremely repetitive.
I would rather "nest" scenarios somehow. Something like this (I know it's not valid).
Scenario: Asked a widget cost question
Given user Asked a widget function question
When the visitor asks
"""
How much does your widget cost?
"""
Then bot should provide widget cost
How could I do something something like this in behave? I was thinking of perhaps using the given, when, and then decorators inside step_impl functions...
Make each of the previous messages a Given.
Scenario: Asked a widget function question and price question
Given a new dialog
And the visitor already asked
"""
What do your widgets do?
"""
When the visitor asks
"""
How much does your widget cost?
"""
Then bot should provide widget cost
The Given statement for previous questions can explicitly wait for the correct answer, or it could just wait for any answer before proceeding to the next step. This way the prerequisites for getting a widget price are clearly indicated as Given's, fitting nicely into the Given-When-Then style of a BDD test. You also aren't cluttering up your scenario with a bunch of Then's that are not directly related to the scenario.
Related
I have developed a Covid-19 Symptom Whatsapp Chatbot using Twilio. The bot basically asks users questions about key symptoms related to Covid-19.
Based on their answers, the bot has to give the users a summary/conclusion i.e. visit the doctor ASAP or call authorities.
My challenge is how to keep the answers to the questions and use them at the end of the survey in Studio. In normal programming i would keep the answers in variables then at the end check the variables and offer a conclusion. How can i implement this in Twilio Studio?
Twilio maintains the variable state throughput the flow. You can use the Set Variables Widget to consolidate all the variable in one place when you reference them or reference them from the individual widgets where the variables are collected (not so ideal).
Set Variables
You can use the Studio Execution logs, to see the JSON path and how to access the variable using {{liquid syntax}}.
you could easily retrieve the variables by referring to them with the syntax {{widgets.name_of_widget.inbound.body}}
you could send them out in a function widget and place them in parameters by inserting the above syntax as the VALUES, and make calculations there.
you call the parameters by event.KEY
Then you can refer back to them by {{widget.function_name.body}}
Just to give you a brief, I'm working on a Chatbot, that will suggest the user some things, based on the answers of a few quetions. Now, most of the questions that the agent asks the user are in Yes/No and with a very little variation.
So here's the usercase:
The agent asks the user these two questions:
Did you have your meal(s) on time?
Did you work out today?
The answers to both these questions can be in a YES/NO.
In that case, if the user is asked the meal question and the user responds with a NO, the agent updates the parameter values for both the meal question as well as the workout question.
But that's something that I don't want.
I'm using the slot-filling technique here. Is there a way around this problem.
Here's my User Info Intent and the parameters that I have:
PS - I don't really want to create multiple intents for all these parameters and want to handle all these answers with just one intent
If the agent updates the parameter values for both the meal question as well as the workout question that means you're using the same parameter to host both answers, you need to use different parameters.
If would be very helpful if you shared how you have handled the dialog so far. And can I ask why do you want to use only one intent?
You can try creating a generic intent and setting both parameters as required, defining your questions as prompts for each parameter. That way the answers will be hosted in two different parameters.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
Suppose I'm designing some SaaS service. And I need to have a function that allow users create sites. User can makes special settings for each site in the admin panel (e.g. design of widget) and gets unique code for install service to his own site.
User story could be:
As a logged user I want to add the new site in the admin panel so that I could configure each instance of widgets separately and could get a unique code for install widget to my own site.
Form
But if I will try to describe this functionality with BDD or GWT (Given When Then) or Gherkin-style, I will face with some trouble. I start from next description:
GIVEN I'm logged into admin panel
AND I'm on "Sites" page
WHEN I click "Add site" button
THEN Pop-up window "Add site" come up
As you can see above realization suppose that site adding will be in pop-up window (e.g. it very important for UX). Pop-up window contains Site URL input field, drop-down control with languages and "Add" and "Cancel" buttons.
And we got a strange scenario which responsible for just pop-up opening. Is it correct? And how can I name this scenario ("Add site's form opening" ??)? Also this scenario has only a one case (when I click - pop-up open). Maybe this scenario not needed at all? I'm confused...
In this case we need to create another scenario when describes:
GIVEN "Add site" pop-up form is opened
WHEN I fill the "Site URL" field
AND click on "Add" button
THEN New site will be create in system
AND I will transfer to my own site's list
How do you think, where do I need to apply a business rules such as:
1) When new site is created a unique code must be generated and consist of minimum 8 characters including numbers and Alphabetical symbols.
2) checks doesn't apply for Site URL input field and user can input a Cyrillic symbols
3) etc?
I have a lot of additional question and hope on the community help!
The thing with BDD is to stay away from implementation details as much as possible. This scenario has multiple implementation details:
GIVEN I'm logged into admin panel AND I'm on "Sites" page
WHEN I click "Add site" button
THEN Pop-up window "Add site" come up
What happens if the "Sites" page becomes "Awesome Site" page or is simply deleted?
What happens if "Add site" is not a button anymore?
What happens if it's not a popup but a redirect occuring
What happens after? Is the value simply in showing the popup? I guess not...
For this specific example, a better approach would be:
GIVEN I'm an authorised administrator
WHEN I enter all the required information for a new site and save it
THEN I should see that site in my own sites list
With this scenario, if your implementation changes, you will only have to change your step definitions, you won't have to change your gherkin. Don't forget that those tests are supposed to explain the Behaviour of the system, not the way it is implemented.
The other questions you have are more related to unit testing in my opinion:
When new site is created a unique code must be generated and
consist of minimum 8 characters including numbers and Alphabetical
symbols => I'd do it at the class level, gherkin would not be
appropriate unless the customer specifically asked for this, then
the condition is "THEN a code having the required characteristics is
generated for that site" and you would have to define "required
characteristics" in a glossary the customer can read and understand.
Checks doesn't apply for Site URL input field and user can input a Cyrillic symbols => again, would put that at the class level en same as for 1. unless the customer wants to be able to read something about it, it should be at the unit level.
I hope that answers your question. I recommend this article by Dan North if you want to have a better idea of how to write better gherkin features.
EDIT 11/13/14
Based on your comments, I suggest we take a step back and describe a way to deal with requirements in your case. I have to tell you that I'm not a BDD expert and am only sharing my own personal experience, for more info on the subject I suggest you get in touch with the guys behind BDD Kickstart and Cucumber.pro where you will find online BDD courses. They will be able to give you lots of information and books to read.
That being said, let's dive into the subject.
The first thing you get is a list of features or stories that, if you follow Mike Cohn's story template would look like:
As a <type of user> I want <to do something> in order to <serve a business purpose>
I personally like to put the business purpose first to make sure we don't skip it in the discussions. You might also not follow that template and that's fine, but remember that it's a good idea to make sure the features you are listing with your customers have a business purpose. If there is no business value behind a feature then what's the point of doing it anyway...
So you do have a list of features/stories described as above. Now for each of these features, there are different cases or scenarios, that's what Dan describes in his article. This is where the Given-When-Then is introduced.
Scenario: Title
Given <some context>
When <there is an event>
Then <something happens>
Each of those scenarios are examples on how this specific feature behaves in different contexts. They are the different acceptance criteria for a specific feature, things the customer described as the expected behaviour of the system. They should be ignorant of any implementation details. So stuff like:
Given I am on page "First page"
When I click "Hello world"
Then I should see "You clicked hello world"
Is wrong for the reasons described prior to this edit.
Let's assume the following feature:
In order to save time when answering clients requests, as a webmaster,
I want to be able to manage the list of websites I am responsible for
Scenarios for this story would be:
Scenario 1: Show a list of websites
GIVEN I am an authorised administrator
AND I am managing several websites
THEN I should see a list of all the sites I manage
Scenario 2: Add website to list
GIVEN I am an authorised administrator
WHEN I enter all the required information for a new site and save it
THEN I should see that site in my own sites list
Scenario 3: Edit website from list
GIVEN I am an authorised administrator
WHEN I edit the site informations
THEN I the changes should be visible in my sites list
...
Now what if you want to go into data validation stuff like "site should have a title" for instance. To me there are two different ways to approach this. You can test that from the user's perspective with a full-stack test or test that there is some validation at the object level.
Let's assume the following scenario:
Scenario: New site has no title
GIVEN I'm an authorised administrator
WHEN I forget to fill in the title for a new site and save it
THEN I should be warned the site is not valid
You can use cucumber or specflow to run this scenario from the UX, therefore using some kind of browse-based proxy to test your app. That is usually slow as it hits the whole system and simulates a real user. It's an option, but I don't think it's the best though. IMO not all tests should be run against the UX and having too many Gherkin features can be a pain to maintain, that's why I prefer focusing on having the happy or critical path (usually I ask myself, where does the money comes from) tested full-stack and put the rest at lower levels.
You can still use Gherkin for these unit tests if you'd like. But that is not mandatory. You only need a way to show your customer you actually have a test for all those specific format controls and validation checks.
That doesn't mean you are not doing BDD anymore, you can still use the given-when-then-should pattern in rspec if you're a rubyist, or any other testing framework you use.
Hope that clarifies all this, let me know if there are any confusing parts...
I think Marc simply deserves the big green tick on this one, thanks to his amazingly thorough answer!
I just wanted to add a few comments.
You don't need to automate all your scenarios.
If you want to capture business requirements in a form that everyone (i.e. including non-tech savvy folks) can understand and Gherkin's Given/When/Then work for you, just go for it. There's nothing forcing you to automate all of your scenarios.
You don't need to automate all your scenarios through the UI.
Your software is made of layers that often respond to similar behaviours, via different interfaces (UI, HTTP, API, ...). Should you want to describe fine-grained business rules (i.e. site name constraints) with automated gherkin scenarios, you could write step definitions that talk directly to your domain layer instead of going through the user interface. That would probably still give you a decent level of confidence.
As a side note, I would recommend not to use Given/When/Then in classic testing frameworks (i.e. those that only devs can see!) if your purpose is to share your tests/requirements with non-tech people.
Have conversations!
Above all, BDD is about better communication: try to talk more, involve your developers (or some of them) earlier in the process so that they gain more knowledge, sooner. Formalising Gherkin scenarios comes in a second phase. Automating them should even be further down your priority list!
I am trying to integrate an external multimedia application with Valence. The application presents the user with quizzes along the way, with each question mapping to a learning objective. Using the API, how can I send the results of each quiz/objective to D2L so it's part of the overall course grade?
Thanks for your question, it's definitely a good one that I'd love to hear more about!
Right off the top, I'll say that unfortunately the short answer is that Valence currently doesn't support retrieving quiz information, nor accepting question responses to contribute to the user's quiz grade in the grade book, and competency-based evaluation. We recognize this is a big gaping hole in Valence, but the reason why we don't have it yet is due to an assessment tools re-architecture which we're embarking on now, which will put things in place to expose more assessments tools functionality via Valence.
That said, I'd love to capture more details about your use case so that we # D2L can be well-positioned to support it in the future.
Can I assume that you would want to perform the quiz create, update, delete operations, inclusive of learning objective question association, directly in D2L, and not via Valence?
Get Questions: Would you be looking to retrieve all quiz question information, providing a quiz identifier?
Submit Responses, and Trigger Evaluation: Would you be looking to submit all quiz question responses, in order to trigger automatic evaluation, to populate the quiz's score in the grade book, and contribute to competency-based evaluation? Would you ever have a case where a question response was provided which required manual evaluation (such as a short/long answer question type)? If so, how would you like to see that handled?
Many thanks again, I appreciate your time to communicate this request.
I am trying to write requirements for a multi-step business process (via a wizard). I have many scenarios where user interactions with one screen will change whether you are allowed to pick options on another screen.
For example (I've obscured the actual business, but the process and form of the steps is nearly identical):
Feature: Personal Diagnostic Search Filter
In order to select a Technician who offers Personal Diagnostics,
when I've asked for a Personal Diagnostic
As a Business Customer
I want to limit my search to Technicians who offer Personal Diagnostics
Background:
Given a Business named "Big Al's Auto Supply"
And a Customer named "Bob Test" at the "Big Al's Auto Supply" Business
And an Account named "bobtest#testbusiness.com" owned by "Bob Test"
And the "bobtest#testbusiness.com" Account has the "Repair Order Creator"
permission
And you log in as "bobtest#testbusiness.com"
And you start scheduling a new Repair Order
Scenario: Enter the Select Technician page when Use PD is selected
Given you select Use PD
When you enter the Select Technician page
Then the PD Filter should be visible
And the PD Filter should be selected
Scenario: Basic Search for Technicians when PD Filter is selected
Given a Technician named "PD Technician"
And the Technician named "PD Technician" supports PD
And a Technician named "Non-PD Technician"
And the Technician named "Non-PD Technician" does not support PD
And you select Use PD
And you enter the Select Technician page
And you select the PD Filter
And you select Basic Search
When you search for Technicians with the name "Technician"
Then your search results should contain "PD Technician"
And your search results should not contain "Non-PD Technician"
But on the Gherkin wiki, it is recommended that you:
Avoid talking about user interaction in givens
They go on to make an exception, though:
Log in a user (An exception to the no-interaction recommendation. Things that “happened earlier” are ok).
It also is said on that page that:
The purpose of When steps is to describe the key action the user performs
What belongs in a Given and what belongs in a When if you have a lot of UI interaction?
In my first scenario, select Use PD requires UI interaction, as it is part of the wizard for creating a new Repair Order. However it is a precondition for PD Filter to be visible, and is triggered when the user enters the Select Technician page.
The first scenario might not be so bad, but the second scenario exacerbates the problem. The search is triggered when you hit Search but there are a lot of UI interactions that must be done to navigate to that page. Some of those interactions can't be glossed over, either, as Use PD must be selected for the search filter to even appear. But those UI interactions aren't the key action of the scenario.
As a general rule, try as much as possible to phrase the scenario as if you're having a conversation about it, and exclude as much irrelevant information as possible.
For instance, I would love your scenarios above to read something like:
Given our customer Bob Test is scheduling a repair order
And we have two technicians: "Fred Technician" and "George Nontechnician"
When Bob Test decides he wants a Personal Diagnostic
And he selects a technician
Then the search results should only contain "Fred Technician"
Then do whatever is necessary to make those steps work - be they logging in or otherwise. Notice I haven't talked about "pages", or taken the actual steps - they should be intuitive for the user. BDD isn't about testing. It's about coming up with examples of how people are going to use the system, so that you can have conversations around those examples and explore them, find exceptions and different scenarios, etc.
Checking that the filter is visible is not valuable. The user doesn't care that the filter is visible. He cares that he can use the filter to get his results, so just do that.
In the code, I usually pass a "World" object between my steps. That can get a lot of the gubbins out of the way. I haven't used Gherkin much but I imagine it provides for a similar ability. You can store all the user details in there, which technicians you've created so that you can check it doesn't bring back "George Nontechnician" in the results, etc.
Using friendly names for roles is useful too, because people can then imagine what Fred and George look like.
Get rid of anything which isn't going to make a difference in the scenario and isn't going to help people imagine it happening. You know that Bob has permission to schedule an order because that's what he's doing - just add the necessary stuff to that step.
The "When" is the behavior you're interested in describing. In this case, you're interested in the ability to filter for Personal Diagnostics, so all the user interaction associated with behavior should be in the "When", and all previous interaction should be in the "Givens". I find it useful to try and think of a context in which the outcome is different - for instance, what happens if there are no PD technicians available? That tells me what the difference is; we'll be setting up a different context but performing the same event. Context goes in the Given, events go in the When. (This used to be much simpler before "Background" was introduced).
In general, if your eyes glaze over when you look at a scenario, you're doing something wrong.
Hope this helps.