Predictive parser for EBNF productions - parsing

Question

I am trying to write a recursive descent parser without backtracking for a kind of EBNF like this:
<a> ::= b [c] | d
where
<a> = non-terminal
lower-case-string = identifier
[term-in-brackets] = term-in-brackets is optional
a|b is the usual mutually exclusive choice between a and b.
For now, I care only about the right hand side.
Following the example at http://en.wikipedia.org/wiki/Recursive_descent_parser, I eventually ended up with the following procedures (rule in GNU bison syntax in comments above):
/* expression: term optional_or_term */
void expression()
{
term();
if (sym == OR_SYM)
optional_or_term();
}
/* optional_or_term: // empty
| OR_SYM term optional_or_term
*/
void optional_or_term()
{
while (sym == OR_SYM)
{
getsym();
term();
}
}
/* term: factor | factor term */
void term()
{
factor();
if (sym == EOF_SYM || sym == RIGHT_SQUAREB_SYM)
{
;
}
else if (sym == IDENTIFIER_SYM || sym == LEFT_SQUAREB_SYM)
term();
else if (sym == OR_SYM)
optional_or_term();
else
{
error("term: syntax error");
getsym();
}
}
/*
factor: IDENTIFIER_SYM
| LEFT_SQUAREB_SYM expression RIGHT_SQUAREB_SYM
*/
void factor()
{
if (accept(IDENTIFIER_SYM))
{
;
}
else if (accept(LEFT_SQUAREB_SYM))
{
expression();
expect(RIGHT_SQUAREB_SYM);
}
else
{
error("factor: syntax error");
getsym();
}
}
It seems to be working, but my expectation was that each procedure would correspond closely with the corresponding rule. You will notice that term() does not.
My question is: did the grammar need more transformation before the procedures were written?

Answer 1

I don't think your problem is the absence of operators for concatenation. I think it is not using Kleene star (and plus) for lists of things. The Kleene star lets you actually code a loop inside a procedure that implements the grammar rule.
I would have written your grammar as:
expression = term (OR_SYM term)*;
term = factor+;
factor = IDENTIFIER_SYM | LEFT_SQUAREB_SYM expression RIGHT_SQUAREB_SYM ;
(This is a pretty classic grammar for a grammar).
The parser code then looks like:
boolean function expression()
{ if term()
{ loop
{ if OR_SYM()
{ if term()
{}
else syntax_error();
}
else return true;
}
else return false;
}
boolean term()
{ if factor()
{ loop
{ if factor()
{}
else return true;
}
}
else return false;
}
boolean factor()
{ if IDENTIFIER(SYM)
return true;
else
{ if LEFT_SQUAREB_SYM()
{ if expression()
{ if RIGHT_SQUAREB_SYM()
return true;
else syntax_error();
}
else syntax_error();
else return false;
}
}
I tried to generate this in an absolutely mechanical way, and you can do pretty well like this. I did a lot of this earlier my career.
What you're not going to get is 150 working rules per day. First, for a big language, it is hard to get the grammar right; you'll be tweaking it repeatedly to get a grammar that works in the abstract, then you have to adjust the code you wrote. Next you'll discover that writing the lexer has its troubles too; just try writing a lexer for Java. Finally, you'll discover that parser rules isn't the whole game or even the biggest part of your effort; you need a lot to process real code. I call this "Life After Parsing"; see my bio
for more information.
If you want to get 150 working rules per day, switch to a GLR parser and stop coding parsers manually. That won't address the other issues, but it does make you incredibly productive at getting out a usable grammar. This is what I do now. Without exception. (Our DMS Software Reengineering Toolkit uses this, and we parse a lot of things that people claim are hard.)