I'm trying to parse statements to see whether they are valid or not according to these rules:
assignment:
id = exp ;
expression:
id op id {op id}
id is combination of digits and char, the first position contain a char.
I'm getting syntax error when I have something like this in my in.txt file: hellow = three3
but that shouldn't be a syntax error, and then when i put something like: hellow = =
that does not display a syntax error but it should. What am i doing wrong?
Lex:
%{
#include "y.tab.h"
#include <stdio.h>
%}
%%
[ \t\n]+ ;
[a-zA-Z][a-zA-Z0-9]* {
ECHO;
return ID;
}
%%
YACC:
%{
#include <stdio.h>
extern FILE * yyin;
%}
%token ID
%left '+' '-'
%left '*' '/' '%'
%right ';'
%%
assignment: expression
|
ID '=' expression
;
expression: ID
|
expression '*' expression
|
expression '/' expression
|
expression '%' expression
|
expression '+' expression
|
expression '-' expression
|
'(' expression ')'
;
%%
int main(void) {
yyin = fopen("in.txt", "r");
yyparse();
fclose(yyin);
return 0;
}
I can't say from just looking at this, but a good way to find out is running your parser when the environment variable YYDEBUG is 1, like in (unix):
YYDEBUG=1 ./a.out
You should get a detailed list of the steps the parser takes and the tokens it receives from the lexer (where the error is in most cases, like in your example: how does the parser ever get the '=' sign and the operators?). Compare this with the y.output file you get when you run yacc -v
Related
I am using Bison, together with Flex, to try and parse a simple grammar that was provided to me. In this grammar (almost) everything is considered an expression and has some kind of value; there are no statements. What's more, the EBNF definition of the grammar comes with certain ambiguities:
expression OP expression where op may be '+', '-' '&' etc. This can easily be solved using bison's associativity operators and setting %left, %right and %nonassoc according to common language standards.
IF expression THEN expression [ELSE expression] as well as DO expression WHILE expression, for which ignoring the common case dangling else problem I want the following behavior:
In if-then-else as well as while expressions, the embedded expressions are taken to be as long as possible (allowed by the grammar). E.g 5 + if cond_expr then then_expr else 10 + 12 is equivalent to 5 + (if cond_expr then then_expr else (10 + 12)) and not 5 + (if cond_expr then then_expr else 10) + 12
Given that everything in the language is considered an expression, I cannot find a way to re-write the production rules in a form that does not cause conflicts. One thing I tried, drawing inspiration from the dangling else example in the bison manual was:
expression: long_expression
| short_expression
;
long_expression: short_expression
| IF long_expression THEN long_expression
| IF long_expression long_expression ELSE long_expression
| WHILE long_expression DO long_expression
;
short_expression: short_expression '+' short_expression
| short_expression '-' short_expression
...
;
However this does not seem to work and I cannot figure out how I could tweak it into working. Note that I (assume I) have resolved the dangling ELSE problem using nonassoc for ELSE and THEN and the above construct as suggested in some book, but I am not sure this is even valid in the case where there are not statements but only expressions. Note as well as that associativity has been set for all other operators such as +, - etc. Any solutions or hints or examples that resolve this?
----------- EDIT: MINIMAL EXAMPLE ---------------
I tried to include all productions with tokens that have specific associativity, including some extra productions to show a bit of the grammar. Notice that I did not actually use my idea explained above. Notice as well that I have included a single binary and unary operator just to make the code a bit shorter, the rules for all operators are of the same form. Bison with -Wall flag finds no conflicts with these declarations (but I am pretty sure they are not 100% correct).
%token<int> INT32 LET IF WHILE INTEGER OBJECTID TYPEID NEW
%right <str> THEN ELSE STR
%right '^' UMINUS NOT ISNULL ASSIGN DO IN
%left '+' '-'
%left '*' '/'
%left <str> AND '.'
%nonassoc '<' '='
%nonassoc <str> LOWEREQ
%type<ast_expr> expression
%type ...
exprlist: expression { ; }
| exprlist ';' expression { ; };
block: '{' exprlist '}' { ; };
args: %empty { ; }
| expression { ; }
| args ',' expression { ; };
expression: IF expression THEN expression { ; }
| IF expression THEN expression ELSE expression { ; }
| WHILE expression DO expression { ; }
| LET OBJECTID ':' type IN expression { ; }
| NOT expression { /* UNARY OPERATORS */ ; }
| expression '=' expression { /* BINARY OPERATORS */ ; }
| OBJECTID '(' args ')' { ; }
| expression '.' OBJECTID '(' args ')' { ; }
| NEW TYPEID { ; }
| STR { ; }
| INTEGER { ; }
| '(' ')' { ; }
| '(' expression ')' { ; }
| block { ; }
;
The following associativity declarations resolved all shift/reduce conflicts and produced the expected output (in all tests I could think of at least):
...
%right <str> THEN ELSE
%right DO IN
%right ASSIGN
%left <str> AND
%right NOT
%nonassoc '<' '=' LOWEREQ
%left '+' '-'
%left '*' '/'
%right UMINUS ISNULL
%right '^'
%left '.'
...
%%
...
expression: IF expression THEN expression
| IF expression THEN expression ELSE expression
| WHILE expression DO expression
| LET OBJECTID ':' type IN expression
| LET OBJECTID ':' type ASSIGN expression IN expression
| OBJECTID ASSIGN expression
...
| '-' expression %prec UMINUS
| expression '=' expression
...
| expression LOWEREQ expression
| OBJECTID '(' args ')'
...
...
Notice that the order of declaration of associativity and precedence rules for all symbols matters! I have not included all the production rules but if-else-then, while-do, let in, unary and binary operands are the ones that produced conflicts or wrong results with different associativity declarations.
I am trying to enforce some parsing errors by defining a non-associative precedence.
Here is part of my grammar file:
Comparison :
Value ComparisonOp Value
{
$2->Left($1);
$2->Right($3);
$$ = $2;
}
;
Expressions like 1 = 2 should parse, but expressions like 1 = 2 = 3 are not allowed in the grammar. To accommodate this, I tried to make my operator non associative as follows:
%nonassoc NONASSOCIATIVE
.
.(rest of the grammar)
.
Comparison :
Value ComparisonOp Value %prec NONASSOCIATIVE
{
$2->Left($1);
$2->Right($3);
$$ = $2;
}
;
1 = 2 = 3 still passes, Could someone please tell me what I am doing wrong?
You need to set the associativity of the ComparisonOp token(s). Either
%nonassoc ComparisonOp NONASSOCIATIVE
if ComparisonOp is a token or something like
%nonassoc '=' '<' '>' NOT_EQUAL GREATOR_OR_EQUAL LESS_OR_EQUAL NONASSOCIATIVE
if you have mulitple tokens and ComparisonOp is a rule that expands to any of them
As a concrete example, the following works exactly like you are requesting:
%{
#include <stdio.h>
#include <ctype.h>
int yylex();
void yyerror(const char *);
%}
%nonassoc '=' '<' '>' CMPOP
%left '+' '-' ADDOP
%left '*' '/' '%' MULOP
%token VALUE
%%
expr: expr cmp_op expr %prec CMPOP
| expr add_op expr %prec ADDOP
| expr mul_op expr %prec MULOP
| VALUE
| '(' expr ')'
;
cmp_op: '=' | '<' | '>' | '<' '=' | '>' '=' | '<' '>' ;
add_op: '+' | '-' ;
mul_op: '*' | '/' | '%' ;
%%
int main() { return yyparse(); }
int yylex() {
int ch;
while(isspace(ch = getchar()));
if (isdigit(ch)) return VALUE;
return ch;
}
void yyerror(const char *err) { fprintf(stderr, "%s\n", err); }
so if you are having other problems, try posting an MVCE that shows the actual problem you are having...
i was trying to code a parser using yacc and lex that count the number of nested loops (while or for).I started the implementation for just while loops.But for some reason the parser gives me an error at the end of a closing brace.
Here is the code.
%{
#include<stdio.h>
/*parser for counting while loops*/
extern int yyerror(char* error);
int while_count=0;
extern int yylex();
%}
%token NUMBER
%token VAR
%token WHILE
%%
statement_list : statement'\n'
| statement_list statement'\n'
;
statement :
while_stmt '\n''{' statement_list '}'
| VAR '=' NUMBER ';'
;
while_stmt :
WHILE '('condition')' {while_count++;}
;
condition :
VAR cond_op VAR
;
cond_op : '>'
| '<'
| '=''='
| '!''='
;
%%
int main(void){
yyparse();
printf("while count:%d\n",while_count);
}
int yyerror(char *s){
printf("Error:%s\n",s);
return 1;
}
what is wrong with that code.And is there a way in yacc to mention optional arguments? like the "\n" after while?
here is the lexer code
%{
#include"y.tab.h"
/*lexer for scanning nested while loops*/
%}
%%
[\t ] ; /*ignore white spaces*/
"while" {return WHILE;}
[a-zA-Z]+ {return VAR;}
[0-9]+ {return NUMBER;}
'$' {return 0;}
'\n' {return '\n' ;}
. {return yytext[0];}
%%
VAR is a variable name with just ascii characters and WHILE is the keyword while.type is not taken into consideration on variable assignments
The problem you seem to be having is with empty loop bodies, not nested loops. As written, your grammar requires at least one statement in the while loop body. You can fix this by allowing empty statement lists:
statement_list: /* empty */
| statement_list statement '\n'
;
You also ask about making newlines optional. The easiest way is to make the lexer simply discard newlines (as whitespace) rather than returning them. Then just get rid of the newlines in the grammar, and newlines can appear between any two tokens and will be ignored.
If you really must have newlines in the grammar for some reason, you can add a rule like:
opt_newlines: /* empty */ | opt_newlines '\n' ;
and then use this rule wherever you want to allow for newlines (replace all the literal '\n' in your grammar.) You have to be careful not to use it redundantly, however. If you do something like:
statement_list: /* empty */
| statement_list statement opt_newlines
;
while_stmt opt_newlines '{' opt_newlines statement_list opt_newlines '}'
you'll get shift/reduce conflicts as newlines before the } in a loop could be either part of the opt_newlines in the while or the opt_newlines in the statement_list. Its pretty easy to deal such conflicts by just removing the redundant opt_newlines.
As a test file, I'm using this piece of code to see whether or not the parser is working well:
/* A test program */
void main(void){
int x;
int y;
int z;
x= 10;
y = 20;
z =x* (x+y);
}
However, I'm getting a syntax error right after the first void, and I don't quite understand why it won't even reach the parameter part. If there's any tips or see something that perhaps I'm not seeing, I'd appreciate it. I know that I have yet to solve the dangling else problem, but that shouldn't be of an issue for this test.
Here's my parser so far:
%{
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
/* external function prototypes */
extern int yylex();
extern int initLex(int , char **);
/* external global variables */
extern int yydebug;
extern int yylineno;
/* function prototypes */
void yyerror(const char *);
/* global variables */
%}
/* YYSTYPE */
/* terminals */
/* Start adding token names here */
/* Your token names must match Project 1 */
/* The file cmparser.tab.h was gets generated here */
%token TOK_ERROR
%token TOK_ELSE
%token TOK_IF
%token TOK_RETURN
%token TOK_VOID
%token TOK_INT
%token TOK_WHILE
%token TOK_PLUS
%token TOK_MINUS
%token TOK_MULT
%token TOK_DIV
%token TOK_LT
%token TOK_LE
%token TOK_GT
%token TOK_GE
%token TOK_EQ
%token TOK_NE
%token TOK_ASSIGN
%token TOK_SEMI
%token TOK_COMMA
%token TOK_LPAREN
%token TOK_RPAREN
%token TOK_LSQ
%token TOK_RSQ
%token TOK_LBRACE
%token TOK_RBRACE
%token TOK_NUM
%token TOK_ID
/* associativity and precedence */
/* specify operator precedence (taken care of by grammar) and associatity here -
-uncomment */
//%left
%left '*' '/'
%left '+' '-'
%nonassoc '<' '>' "<=" ">="
%left "==" "!="
%right '='
%nonassoc error
/* Begin your grammar specification here */
%%
Start : /* put your RHS for this rule here */
Declarations
{ printf ("Declaration.\n"); }
; /* note that the rule ends with a semicolon */
Declarations :
/* empty */
{}
| Vardeclaration Declarations
{ printf ("Var-declaration.\n");}
| Fundeclaration Declarations
{ printf ("Fun-declaration.\n");}
;
Vardeclaration :
Typespecifier TOK_ID ';'
{ printf ("Not an array.\n");}
| Typespecifier TOK_ID '[' TOK_NUM ']' ';'
{ printf ("An array.\n");}
;
Typespecifier :
TOK_INT
{ printf ("Type int.\n");}
| TOK_VOID
{ printf("Type void.\n");}
;
Fundeclaration :
Typespecifier TOK_ID '(' Params ')' Compoundstmt
{ printf ("Function Declaration.");}
;
Params :
Paramlist
| TOK_VOID
{ printf ("Void param.\n");}
;
Paramlist :
Paramlist ',' Param
| Param
;
Param :
Typespecifier TOK_ID
| Typespecifier TOK_ID '[' ']'
;
Compoundstmt :
'{' Localdeclaration Statement '}'
;
Localdeclaration :
Vardeclaration Localdeclaration
| /* empty */
;
Statement :
Expressionstmt Statement
| Compoundstmt Statement
| Selectionstmt Statement
| Iterationstmt Statement
| Returnstmt Statement
| /* empty */
;
Expressionstmt :
Expression ';'
| ';'
;
Selectionstmt :
TOK_IF '(' Expression ')' Statement
| TOK_IF '(' Expression ')' Statement TOK_ELSE Statement
;
Iterationstmt :
TOK_WHILE '(' Expression ')' Statement
;
Returnstmt :
TOK_RETURN ';'
| TOK_RETURN Expression ';'
;
Expression :
Var '=' Expression
| SimpleExpression
;
Var :
TOK_ID
| TOK_ID '[' Expression ']'
;
SimpleExpression :
AdditiveExpression Relop AdditiveExpression
| AdditiveExpression
;
Relop :
"<="
| '<'
| '>'
| ">="
| "=="
| "!="
;
AdditiveExpression :
AdditiveExpression Addop Term
| Term
;
Addop :
'+'
| '-'
;
Term :
Term Mulop Factor
| Factor
;
Mulop :
'*'
| '/'
;
Factor :
'(' Expression ')'
| Var
| Call
| TOK_NUM
;
Call :
TOK_ID '(' Args ')'
;
Args :
Arglist
| /* empty */
;
Arglist :
Arglist ',' Expression
| Expression
;
%%
void yyerror (char const *s) {
fprintf (stderr, "Line %d: %s\n", yylineno, s);
}
int main(int argc, char **argv){
initLex(argc,argv);
#ifdef YYLLEXER
while (gettok() !=0) ; //gettok returns 0 on EOF
return;
#else
yyparse();
#endif
}
Your grammar looks ok, so my best guess is that your lexer doesn't recognize the string void as TOK_VOID, but instead returns something else.
Try compiling your parser with -DYYDEBUG and setting yydebug = 1 before calling yyparse to see what its getting from the lexer.
i'm trying to make a very simple YACC parser on Pascal language which just includes integer declarations, some basic expressions and if-else statements. however, i cant find the error for hours and i'm going to be crazy soon. terminal says Error at line:0 but it is impossible!. i use flex and byacc for parser.i will be very glad if you can help me. this is my lex file as you can see;
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern int yylval;
int linenum=0;
%}
digit [0-9]
letter [A-Za-z]
%%
if return IF;
then return THEN;
else return ELSE;
for return FOR;
while return WHILE;
PROGRAM return PROGRAM_SYM;
BEGIN return BEGIN_SYM;
VAR return VAR_SYM;
END return END_SYM;
INTEGER return INTEGER_SYM;
{letter}({letter}|{digit})* return identifier;
[0-9]+ return NUMBER;
[\<][\=] return CON_LE;
[\>][\=] return CON_GE;
[\=] return CON_EQ;
[\:][\=] return ASSIGNOP;
; return semiColon;
, return comma;
\n {linenum++;}
. return (int) yytext[0];
%%
and this is my Yacc file
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern FILE *yyin;
extern int linenum;
%}
%token PROGRAM_SYM VAR_SYM BEGIN_SYM END_SYM INTEGER_SYM NUMBER
%token identifier INTEGER ASSIGNOP semiColon comma THEN
%token IF ELSE FOR WHILE
%token CON_EQ CON_LE CON_GE GE LE
%left '*' '/'
%left '+' '-'
%start program
%%
program: PROGRAM_SYM identifier semiColon VAR_SYM dec_block BEGIN_SYM statement_list END_SYM '.'
;
dec_block:
dec_list semiColon;
dec_list:
dec_list dec
|
dec
;
dec:
int_dec_list
;
int_dec_list:
int_dec_list int_dec ':' type
|
int_dec ':' type
;
int_dec:
int_dec comma identifier
|
identifier
;
type:
INTEGER_SYM
;
statement_list:
statement_list statement
|
statement
;
statement:
assignment_list
|
expression_list
|
selection_list
;
assignment_list:
assignment_list assignment
|
assignment
;
assignment:
identifier ASSIGNOP expression_list
;
expression_list:
expression_list expression semiColon
|
expression semiColon
;
expression:
'(' expression ')'
|
expression '*' expression
|
expression '/' expression
|
expression '+' expression
|
expression '-' expression
|
factor
;
factor:
identifier
|
NUMBER
;
selection_list:
selection_list selection
|
selection
;
selection:
IF '(' logical_expression ')' THEN statement_list ELSE statement_list
;
logical_expression:
logical_expression '=' expression
|
logical_expression '>' expression
|
logical_expression '<' expression
;
%%
void yyerror(char *s){
fprintf(stderr,"Error at line: %d\n",linenum);
}
int yywrap(){
return 1;
}
int main(int argc, char *argv[])
{
/* Call the lexer, then quit. */
yyin=fopen(argv[1],"r");
yyparse();
fclose(yyin);
return 0;
}
and finally i take an error at the first line when i give the input;
PROGRAM myprogram;
VAR
i:INTEGER;
i3:INTEGER;
j:INTEGER;
BEGIN
i := 3;
j := 5;
i3 := i+j*2;
i := j*20;
if(i>j)
then i3 := i+50+(45*i+(40*j));
else i3 := i+50+(45*i+(40*j))+i+50+(45*i+(30*j));
END.
For debugging grammars, YYDEBUG is your friend. Either stick #define YYDEBUG 1 in the %{..%} in the top of your .y file, or compile with -DYYDEBUG, and stick a yydebug = 1; in main before calling yyparse and you'll get a slew of info about what tokens the parser is seeing and what it is doing with them....
Your lexical analyzer returns blanks and tabs as tokens, but the grammar doesn't recognize them.
Add a parser rule:
[ \t\r] { }
This gets you to line 6 instead of line 0 before you run into an error. You get that error because you don't allow semicolons between declarations:
dec_block:
dec_list semiColon;
dec_list:
dec_list dec
|
dec
;
dec:
int_dec_list
;
That should probably be:
dec_block:
dec_block dec
|
dec
;
dec:
int_dec_list semiColon
;
Doing this gets you to line 14 in the input.
Incidentally, one of the first things I did was to ensure that the lexical analyzer tells me what it is doing, by modifying the rules like this:
if { printf("IF\n"); return IF; }
In long term code, I'd make that diagnostic output selectable at run-time.
You have a general problem with where you expect semicolons. It is also not clear that you should allow an expression_list in the rule for statement (or, maybe, 'not yet' — that might be appropriate when you have function calls, but allowing 3 + 2 / 4 as a 'statement' is not very helpful).
This grammar gets to the end of the input:
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern FILE *yyin;
extern int linenum;
%}
%token PROGRAM_SYM VAR_SYM BEGIN_SYM END_SYM INTEGER_SYM NUMBER
%token identifier INTEGER ASSIGNOP semiColon comma THEN
%token IF ELSE FOR WHILE
%token CON_EQ CON_LE CON_GE GE LE
%left '*' '/'
%left '+' '-'
%start program
%%
program: PROGRAM_SYM identifier semiColon VAR_SYM dec_block BEGIN_SYM statement_list END_SYM '.'
;
dec_block:
dec_block dec
|
dec
;
dec:
int_dec_list semiColon
;
int_dec_list:
int_dec_list int_dec ':' type
|
int_dec ':' type
;
int_dec:
int_dec comma identifier
|
identifier
;
type:
INTEGER_SYM
;
statement_list:
statement_list statement
|
statement
;
statement:
assignment
|
selection
;
assignment:
identifier ASSIGNOP expression semiColon
;
expression:
'(' expression ')'
|
expression '*' expression
|
expression '/' expression
|
expression '+' expression
|
expression '-' expression
|
factor
;
factor:
identifier
|
NUMBER
;
selection:
IF '(' logical_expression ')' THEN statement_list ELSE statement_list
;
logical_expression:
expression '=' expression
|
expression '>' expression
|
expression '<' expression
;
%%
void yyerror(char *s){
fprintf(stderr,"Error at line: %d\n",linenum);
}
int yywrap(){
return 1;
}
int main(int argc, char *argv[])
{
/* Call the lexer, then quit. */
yyin=fopen(argv[1],"r");
yyparse();
fclose(yyin);
return 0;
}
Key changes include removing assignment_list and expression_list, and modifying logical_expression so that the two sides of the expansion are expression, rather than the LHS being logical_expression (which then never had a primitive definition, leading to the problems with warnings).
There are still issues to resolve; the expression_list in the selection should be more restrictive to accurately reflect the grammar of Pascal. (You need a block where that could be a single statement or a BEGIN, statement list, END.)