I am new to specflow, want help with a specflow scenario like the one below.
Scenario Outline: Error messages validation for maximum allowed term rule
Given a <product>
When term exceeds the max allowed term
Then this <errormessage> is displayed
Examples:
| product | errormessage |
| ProductA | This is an error message 1 |
| ProductB | This is an error message 2 |
| ProductC | This is an error message 3 |
For the last step definition "*Then this errormessage is displayed " step, I want to reuse an existing binding method
"Then this (.) is displayed"
This existing binding method takes a string as a parameter (expected error message) and asserts it against the actual message picked form the app under test.
But when I use the method as is - its unable to pass the error message content as an array of strings . Would someone be able to help me to understand what do I need to do to make it work ?
Binding method example below . The step Then this is displayed is not able to recognize this binding, its asking me to write another method.
[Then(#"this ""(.*)"" is displayed")]
public void ThenErrorMessageIsDisplayed(string errorMessage)
{
var msg = uServiceSupport.GetMessages(responseData);
var found = new JObject();
// due to multiple error and warning messages
foreach (var elem in msg)
{
if (elem["message"].ToString().Contains(errorMessage))
found = (JObject)elem;
}
try
{
Assert.IsTrue(found.HasValues, "Check if response has warning/error message");
Assert.AreEqual(errorMessage, found["message"].ToString(), "Check if the error message is {0}", errorMessage);
}
catch (AssertionException)
{
Helper.LogInfo(string.Format("Response:\n {0}", JObject.Parse(responseData)));
throw;
}
}
The problem is in your step regex. you have this:
[Then(#"this ""(.*)"" is displayed")]
but you try and call it like this:
Then this <errormessage> is displayed
you do not have consistent usage of the ". you either need:
[Then(#"this (.*) is displayed")]
or
Then this "<errormessage>" is displayed
Related
This question concerns how to get error messages out of an ANTLR4 parser in C# in Visual Studio. I feed the ANTLR parser a known bad input string, but I am not seeing any errors or parse exceptions thrown during the (bad) parse. Thus, my exception handler does not get a chance to create and store any error messages during the parse.
I am working with an ANTLR4 grammar that I know to be correct because I can see correct parse operation outputs in graphical form with an ANTLR extension to Visual Studio Code. I know the generated parser code is correct because I can compile it correctly without errors, override the base visitor class, and print out various bits of information from the parse tree with my overwritten VisitXXX methods.
At this point, I am running a very simple test case that feeds in a bad input string and looks for a nonzero count on my list of stored parse errors. I am confident of the error-handling code because it works in a similar situation on another grammar. But the error-handling code must catch a parse exception to generate an error message. (Maybe that's not the right way to catch/detect parse errors such as unexpected tokens or other errors in the input stream.)
Here is the code that I used to replace the default lexer and parser error listeners.
// install the custom ErrorListener into the parser object
sendLexer.RemoveErrorListeners();
sendLexer.AddErrorListener(MyErrorListener.Instance);
Parser.RemoveErrorListeners();
Parser.AddErrorListener(MyErrorListener.Instance);
I have attached a screenshot of the graphical output showing the presence of unexpected tokens in the input string.
Q1. Why don't the unexpected tokens cause parse exceptions that I can catch with my exception handler? Are all parse errors supposed to throw exceptions?
Q2. If catching parse exceptions is not the right way, could someone please suggest a strategy for me to follow to detect the unexpected token errors (or other errors that do not throw parse exceptions)?
Q3. Is there a best practice way of catching or finding parse errors, such as generating errors from walking the parse tree, rather than hoping that ANTLR will throw a parse exception for every unexpected token? (I am wondering if unexpected tokens are supposed to generate parse exceptions, as opposed to producing and legitimate parse tree that happens to contain unexpected tokens? If so, do they just show up as unexpected children in the parse tree?)
Thank you.
Screenshot showing unexpected tokens in the (deliberate) bad input string to trigger errors:
UPDATE:
Currently, the parser and unit tests are working. If I feed a bad input string into the parser, the default parser error listener produces a suitable error message. However, when I install a custom error listener, it never gets called. I don't know why it doesn't get called when I see an error message when the custom error listener is not installed.
I have the parser and unit tests working now. When I inject a bad input string, the default parse error listener prints out a message. But when I install a custom error listener, it never gets called. 1) A breakpoint placed in the error listener never gets hit, and 2) (as a consequence) no error message is collected nor printed.
Here is my C# code for the unit test call to ParseText:
// the unit test
public void ModkeyComboThreeTest() {
SendKeysHelper.ParseText("this input causes a parse error);
Assert.AreEqual(0, ParseErrors.Count);
// the helper class that installs the custom error listener
public static class SendKeysHelper {
public static List<string> ParseErrorList = new List<string>();
public static MyErrorListener MyErrorListener;
public static SendKeysParser ParseText(string text) {
ParseErrors.Clear();
try {
var inputStream = new AntlrInputStream(text);
var sendLexer = new SendKeysLexer(inputStream);
var commonTokenStream = new CommonTokenStream(sendLexer);
var sendKeysParser = new SendKeysParser(commonTokenStream);
Parser = sendKeysParser;
MyErrorListener = new MyErrorListener(ParseErrorList);
Parser.RemoveErrorListeners();
Parser.AddErrorListener(MyErrorListener);
// parse the input from the starting rule
var ctx = Parser.toprule();
if (ParseErrorList.Count > 0) {
Dprint($"Parse error count: {ParseErrorList.Count}");
}
...
}
// the custom error listener class
public class MyErrorListener : BaseErrorListener, IAntlrErrorListener<int>{
public List<string> ErrorList { get; private set; }
// pass in the helper class error list to this constructor
public MyErrorListener(List<string> errorList) {
ErrorList = errorList;
}
public void SyntaxError(IRecognizer recognizer, int offendingSymbol,
int line, int offset, string msg, RecognitionException e) {
var errmsg = "Line " + line + ", 0-offset " + offset + ": " + msg;
ErrorList.Add(errmsg);
}
}
So, I'm still trying to answer my original question on how to get error information out of the failed parse. With no syntax errors on installation, 1) the default error message goes away (suggesting my custom error listener was installed), but 2) my custom error listener SyntaxError method does not get called to register an error.
Or, alternatively, I leave the default error listener in place and add my custom error listener as well. In the debugger, I can see both of them registered in the parser data structure. On an error, the default listener gets called, but my custom error listener does not get called (meaning that a breakpoint in the custom listener does not get hit). No syntax errors or operational errors in the unit tests, other than that my custom error listener does not appear to get called.
Maybe the reference to the custom listener is somehow corrupt or not working, even though I can see it in the parser data structure. Or maybe a base class version of my custom listener is being called instead. Very strange.
UPDATE
The helpful discussion/answer for this thread was deleted for some reason. It provided much useful information on writing custom error listeners and error strategies for ANTLR4.
I have opened a second question here ANTLR4 errors not being reported to custom lexer / parser error listeners that suggests an underlying cause for why I can't get error messages out of ANTLR4. But the second question does not address the main question of this post, which is about best practices. I hope the admin who deleted this thread undeletes it to make the best practice information visible again.
The parser ErrorListener SyntaxError method needs the override modifier to bypass the default method.
public class ParserErrorListener : BaseErrorListener
{
public override void SyntaxError(
TextWriter output, IRecognizer recognizer,
IToken offendingSymbol, int line,
int charPositionInLine, string msg,
RecognitionException e)
{
string sourceName = recognizer.InputStream.SourceName;
Console.WriteLine("line:{0} col:{1} src:{2} msg:{3}", line, charPositionInLine, sourceName, msg);
Console.WriteLine("--------------------");
Console.WriteLine(e);
Console.WriteLine("--------------------");
}
}
The lexer ErrorListener is a little different. While the parser BaseErrorListener implements IAntlrErrorListener of type IToken, the lexer requires an implementation of IAntlrErrorListener of type int. The SyntaxError method does not have an override modifier. Parameter offendingSymbol is an int instead of IToken.
public class LexerErrorListener : IAntlrErrorListener<int>
{
public void SyntaxError(
TextWriter output, IRecognizer recognizer,
int offendingSymbol, int line,
int charPositionInLine, string msg,
RecognitionException e)
{
string sourceName = recognizer.InputStream.SourceName;
Console.WriteLine("line:{0} col:{1} src:{2} msg:{3}", line, charPositionInLine, sourceName, msg);
Console.WriteLine("--------------------");
Console.WriteLine(e);
Console.WriteLine("--------------------");
}
}
If i tried to run a givwenzen script using FitNesse SLIM getting error that givwenzen class is not picked up.
May be some class path error.
If anyone could help me with an example of givwenzen
Even with the addition of two numbers.
Thanks in advance
The project https://github.com/weswilliams/GivWenZen has a number of examples. Here's one:
1) Start with an example fixture actual class found in the givwenzen_test.jar.
2) In a FitNesse table it could look like this.
import and start should go in SetUp or SuiteSetUp
|import|
|org.givwenzen|
|script|
|start|giv wen zen for slim|
this is your test
|script|
| given| a ToDo item is due tomorrow |
| when | the date changes to tomorrow |
| then | a notification exists indicating the ToDo is due |
3) The following is an example step class and test step method ===
package bdd.steps;
#DomainSteps
public class ExampleSteps {
#DomainStep( “a ToDo item is due (.*)” )
public void createToDoWithDueDateOf(CustomDate date) {
// do something
}
#DomainStep( “the date changes to (.*)” )
public void theDateIs(CustomDate date) {
// do something
}
#DomainStep( “a notification exists indicating the ToDo is due” )
public boolean verifyNotificationExistsForDueToDo() {
// do something
return false;
}
}
I would like to define a default value for a discriminated union, like this:
open System.Linq
type Result =
| Ok
| Error
let results : seq<Result> = [] |> Seq.ofList
let firstResult = results.FirstOrDefault()
// I want firstResult to be Error, currently it is null.
option<'a> works in this way (firstResult would be None), so it should be possible.
Thanks for your help.
Edit:
I'm using SQLDataProvider and would like to write code like
let userEmail =
query {
for user in dbContext.Public.Users do
where (user.Id = 42)
select (Ok user.Email)
headOrDefault
// should result in Error
// when no user with Id=42 exists
}
My actual result type looks like this:
type Result<'a> =
| Ok of 'a
| Failure of string // Expected, e. g. trying to log in with a wrong password
| Error // Unexpected
Returning an option, the caller would not be able to differentiate between failures and errors.
In general F# avoids the concept of defaults, instead making everything as explicit as possible. It's more idiomatic to return an option and for the caller to decide what to default to for a particular use case.
The FirstOrDefault method can only return .NET's default value for any type. So for any class it would return null, and for a number it would return zero.
I would recommend this approach instead assuming your desired default is Ok:
results |> Seq.tryHead |> Option.defaultValue Ok
You might be able to do this using the UseNullAsTrueValue compilation parameter. This tells the compiler to use null as an internal representation of one of the parameter-less cases. It is a bit of a hack (because this is meant mostly for performance optimization and here we are misusing it somewhat, but it might work).
I don't have SQL database setup to try that with the SQL provider, but the following works in memory and I think it should work with SQL too:
[<CompilationRepresentation(CompilationRepresentationFlags.UseNullAsTrueValue)>]
type Result<'a> =
| Ok of 'a
| Failure of string
| Error
let userEmail =
query {
for a in [1] do
where (a = 2)
select (Ok a)
headOrDefault }
If you run this, F# interactive prints that userEmail = null, but that's fine and the following is true:
userEmail = Error
If you want to represent issues with the data or the query parameters, such as not finding a particular record, as being distinct from other types of failures, such as not connecting to the database or hitting an unhandled exception, you can create a discriminated union to represent your expected data/query problems explicitly. You can then return that DU instead of string as the data for the Error case, and you won't really need the Failure case. Consider something like this:
type SqlError =
| NoMatchingRecordsFound of SqlParameter list
| CouldNotConnectToDatabase
| UserDoesNotHaveQueryPermissions
| UnhandledException of exn
I would suggest taking a look at Railway-Oriented Programming and following the pattern of defining a discriminated union for your different error cases. You can still include an error message in that DU, but I would suggest using explicit cases for all your different expected failures, and then having an "UnhandledException" or similar case for your unexpected errors, perhaps with an exn for the data.
If you're interested, I have a library on GitHub/NuGet that puts all the ROP stuff together and adds interoperability with the Task, Async, and Lazy types in one computation builder, so you don't have to include all that code in your own project.
EDIT
Here's a complete example of doing it ROP-style (using the linked framework):
open FSharp.Control
open System.Linq
// Simulate user table from type-provider
[<AllowNullLiteral>]
type User() =
member val Id = 0 with get,set
member val Name = "" with get,set
// Simulate a SQL DB
let users = [User(Id = 42, Name = "The User")].AsQueryable()
// Our possible events from the SQL query
type SqlEvent =
| FoundUser
| UserIdDoesNotExist of int
| CouldNotConnectToDatabase
| UnhandledException of exn
// Railway-Oriented function to find the user by id or return the correct error event(s)
let getUserById id =
operation {
let user =
query {
for user in users do
where (user.Id = id)
select user
headOrDefault
}
return!
if user |> isNull
then Result.failure [UserIdDoesNotExist id]
else Result.successWithEvents user [FoundUser]
}
Caling getUserById 42 would return a successfully completed operation with the user and the event FoundUser. Calling getUserById for any other number (e.g. 0) would return a failed operation with the error event UserIdDoesNotExist 0. You would add more events as necessary.
I am writing code to generate a JavaCC parser, which will read a user's input and check if it is in any one of a set of languages defined in my code.
One condition on allowable input is that it must not be empty - i.e., the user must enter some block of characters (with length greater than or equal to 1) other than white space " ".
I would like to be able to determine if the user's input is empty, so that an error message can be printed out on the screen in that case.
I have written a production (a.k.a rule) that gets the user's input; it's called Input() and is declared to be void.
In the main method, I have tried to write code which determines if the user's input is empty, by writing:
if parser.Input() == null {
// print error message onto the screen
}
However, I get an error message on the terminal when I try to compile, stating that a 'void' type is not allowed here (I am sure this is referring to Input).
Could I please have a hint/hints for getting around this issue?
Write the Input production like this
boolean Input() : {
} {
<EOF>
{return true;}
|
... // other possibilities here
{return false;}
}
Then, in the main method, you can write
if( parser.Input() ) {
... // report error
}
This solves the problem of reporting the error.
However you may also want to report the language found. For that you could make an enumeration type and have Input return a member of the enumeration. EMPTY could be one of the possibilities.
Language lang = parser.Input() ;
switch( lang ) {
case EMPTY:
... // report error
break ;
case LANGA:
...
break ;
... // etc.
}
Change your type method so this can return a value and you can validate the result, when you do this but change the comparison like this:
if null==parser.Input(){
//print error message on screen
}
Another option is to validate data inside your Input method so you keep it like void.
I am using Rest Fixture with Fitnesse to make a GET request to url which returns non-xml response. Is there a way I could verify text/string (without xpath) in the content returned?
I found this solution.
TEXT is a supported content handler along XML and JSON. It is possible to override the content handler as TEXT and expect the content. Regular expression can also be used to expect content.
| Table:smartrics.rest.fitnesse.fixture.RestFixtureConfig | overridesContentHandlerConfig|
| restfixture.content.handlers.map | application/smil=TEXT |
!| Table:smartrics.rest.fitnesse.fixture.RestFixture | ${host} ||overridesContentHandlerConfig |
| GET | www.someurl.com | 200 | | [\s\w\d<>/=\:'.]*stringtoverify[\s\w\d<>/=\:'.]* |
You can use fit mode + NetRunner plugin (for .Net).
See here example, how to parse input line to the object.
Another way is to use Custom Comparators. This gives you more flexibility on customizing validation on custom/complicated results.
To use custom comparators:
documented here (search for 'CustomComparators')
required property: CustomComparators = <prefix:classname>[,<prefix:class name>]
motivation: The Slim protocol is all String values. It means that
comparison of an expected and actual result for complex datatypes is
limited to String equality or Regular Expression matching. If that is
not sufficient, a Custom Comparator can do more sophisticated
comparisons. Once registered, a Custom Comparator is triggered by its
prefix, followed by a colon, in front of the expected value.
Example Comparator implementation:
public class JSONAssertComparator implements CustomComparator {
#Override
public boolean matches(String actual, String expected) {
try {
JSONAssert.assertEquals(expected, actual, false);
return true;
} catch (JSONException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
}
Example plugins.properties:
CustomComparators = json:com.acme.JSONAssertComparator
Example ScriptTable usage:
|script|Customer |
|check|get|cust1|json:{id:cust1,name:"John Doe"}|