I want to design a program using lex and yacc that will take input in Morse code form and the parser would generate their English meaning.
This is the lex file:
%{
#include <stdio.h>
int i=0;
char c;
%}
NewLine ".-.-"
EOF ".-.-"
NewPara "-...-"
Over "K"
SOS "...---..."
%%
".." { yylval=(char *)calloc(yyleng,sizeof(char));
strcpy(yylval,yytext);
return (I);}
".-""--" { yylval=(char *)calloc(yyleng,sizeof(char));
strcpy(yylval,yytext);
return (AM);}
".-""...-"".-""-."".." {yylval.value=(char *)calloc(yyleng,sizeof(char));
strcpy(yylval,yytext);
return (AVANI);}
. return yytext[0];
%%
The yacc file:
%{
#include <stdio.h>
#define YYSTYPE char*
char message [20];
%}
%token I AM AVANI
%start msg
%%
msg : Np Vp {printf("The sentence is I AM AVANI");}
;
Np : I
;
Vp : Aux N
;
Aux : AM
;
N : AVANI
;
%%
#include "lex.yy.c"
/*extern YYSTYPE yylval*/
void main(){
printf("Enter the message\n");
scanf("%s",message);
yyparse();
}
void yyerror(const char *s)
{
fprintf(stderr,"s\n",s);
}
yywrap ()
{
return 1;
}
This shows the following error on Compiling:
morse_code.l: In function ‘yylex’:
morse_code.l:13:10: warning: assignment makes integer from pointer
without a cast [enabled by default]
".." { yylval=(char *)calloc(yyleng,sizeof(char));
^
morse_code.l:14:2: warning: passing argument 1 of ‘strcpy’ makes
pointer from integer without a cast [enabled by default]
strcpy(yylval,yytext);
^
In file included from lex.yy.c:20:0:
/usr/include/string.h:125:14: note: expected ‘char * __restrict__’ but
argument is of type ‘YYSTYPE’
extern char *strcpy (char *__restrict __dest, const char *__restrict __src)
^
morse_code.l:17:9: warning: assignment makes integer from pointer
without a cast [enabled by default]
".-""--" { yylval=(char *)calloc(yyleng,sizeof(char));
^
morse_code.l:18:5: warning: passing argument 1 of ‘strcpy’ makes
pointer from integer without a cast [enabled by default]
strcpy(yylval,yytext);
^
In file included from lex.yy.c:20:0:
/usr/include/string.h:125:14: note: expected ‘char * __restrict__’ but
argument is of type ‘YYSTYPE’
extern char *strcpy (char *__restrict __dest, const char *__restrict __src)
^
morse_code.l:21:8: error: request for member ‘value’ in something not
a structure or union
".-""...-"".-""-."".." {yylval=(char *)calloc(yyleng,sizeof(char));
^
morse_code.l:22:4: warning: passing argument 1 of ‘strcpy’ makes
pointer from integer without a cast [enabled by default]
strcpy(yylval,yytext);
^
In file included from lex.yy.c:20:0:
/usr/include/string.h:125:14: note: expected ‘char * __restrict__’ but
argument is of type ‘YYSTYPE’
extern char *strcpy (char *__restrict __dest, const char *__restrict __src)
It'll be great if someone could help me with the correct way to pass the string value of a token to the yacc file.
Two obvious problems
You're missing a declaration for yylval in your .l file. If you include the "y.tab.h" generated by yacc, that will define it for you, but you don't have that either.
you don't allocate space for the terminating NUL character of your strings with calloc, so when you strcpy the string it will write one byte past the end of your allocation, causing undefined behavior.
Related
I have a handwritten parser in C++ which contains a next_token() method and I wanna use it inside yylex() (I already did that correctly with Bison's C API but wanna use dynamic types so moved to C++). I read these parts of documentation, read examples and tried both existing signatures but still can't do it correctly...
parser.yy:
%require "3.2"
%language "c++"
%define api.value.type variant
%define api.token.constructor
%define parse.assert
%code requires
{
#include <iostream>
#include <string>
}
%code{
yy::parser::symbol_type yy::parser::yylex();
void yy::parser::yyerror(const char *error);
}
%token VAR COL ITYPE
%token IDENTIFIER
%token INTEGER
%token EOL
%type <std::string> type PrimitiveType IDENTIFIER
%type <int> INTEGER
%%
program:
| program EOL
| program SimpleDeclaration { }
;
SimpleDeclaration: VariableDeclaration
;
VariableDeclaration: VAR IDENTIFIER COL type {std::cout<<"defined variable " << $2 << " with type " << $4 << std::endl; }
type: IDENTIFIER
| PrimitiveType
;
PrimitiveType: ITYPE { $$ = "int"; }
;
%%
void yy::parser::yyerror(const std::string& m)
{
std::cout << "syntax error" << std::endl;
}
// also 'yy::parser::symbol_type yy::parser::yylex(semantic_type* yylval)' does the same
int yy::parser::yylex(semantic_type* yylval)
{
return 0; // just returning a zero for now
}
int main()
{
yy::parser p;
return 0;
}
Errors :
bison -d parser.ypp
g++ -std=c++17 -o foobar parser.tab.cpp
parser.ypp:14:29: error: no declaration matches ‘yy::parser::symbol_type yy::parser::yylex()’
14 | yy::parser::symbol_type yy::parser::yylex();
| ^~
parser.ypp:14:29: note: no functions named ‘yy::parser::symbol_type yy::parser::yylex()’
In file included from parser.tab.cpp:41:
parser.tab.hpp:193:9: note: ‘class yy::parser’ defined here
193 | class parser
| ^~~~~~
parser.ypp:15:10: error: no declaration matches ‘void yy::parser::yyerror(const char*)’
15 | void yy::parser::yyerror(const char *error);
| ^~
parser.ypp:15:10: note: no functions named ‘void yy::parser::yyerror(const char*)’
In file included from parser.tab.cpp:41:
parser.tab.hpp:193:9: note: ‘class yy::parser’ defined here
193 | class parser
| ^~~~~~
parser.tab.cpp: In member function ‘virtual int yy::parser::parse()’:
parser.tab.cpp:453:38: error: ‘yylex’ was not declared in this scope; did you mean ‘yylen’?
453 | symbol_type yylookahead (yylex ());
| ^~~~~
| yylen
parser.ypp: At global scope:
parser.ypp:45:6: error: no declaration matches ‘void yy::parser::yyerror(const string&)’
45 | void yy::parser::yyerror(const std::string& m)
| ^~
parser.ypp:45:6: note: no functions named ‘void yy::parser::yyerror(const string&)’
In file included from parser.tab.cpp:41:
parser.tab.hpp:193:9: note: ‘class yy::parser’ defined here
193 | class parser
| ^~~~~~
parser.ypp:50:5: error: no declaration matches ‘int yy::parser::yylex(yy::parser::semantic_type*)’
50 | int yy::parser::yylex(semantic_type* yylval)
| ^~
parser.ypp:50:5: note: no functions named ‘int yy::parser::yylex(yy::parser::semantic_type*)’
In file included from parser.tab.cpp:41:
parser.tab.hpp:193:9: note: ‘class yy::parser’ defined here
193 | class parser
| ^~~~~~
I feel weird also that even without overriding, it seems to use yylex here (which causes the error to appear even though I don't try to override):
parser.tab.cpp: In member function ‘virtual int yy::parser::parse()’:
parser.tab.cpp:453:38: error: ‘yylex’ was not declared in this scope; did you mean ‘yylen’?
453 | symbol_type yylookahead (yylex ());
| ^~~~~
| yylen
What did I do wrong here ?
Thanks in advance
I was trying to write a simple clang-tidy checker that will check for constructor that is calling fopen() more than once. My indention is to find potential memory leak in case any exception happens in the second fopen() call.
class Dummy_file
{
FILE *f1_;
FILE *f2_;
public:
Dummy_file(const char* f1_name, const char* f2_name, const char * mode){
f1_ = fopen(f1_name, mode);
f2_ = fopen(f2_name, mode);
}
~Dummy_file(){
fclose(f1_);
fclose(f2_);
}
};
Using this
callExpr(callee(functionDecl(hasName("fopen")))).bind("fopencalls")
was able to find all the fopen() calls.
But I could not find cxxConstructorDeclusing this.
cxxConstructorDecl(has(callExpr(callee(functionDecl(hasName("fopen")))))).bind("ctr")
I am doubting since I am using cxxConstructorDecl my filter is not applied to the constructor body.
So how to find function body from a function declaration?
Short explanation
You should use hasDescendant matcher instead of has matcher. While has checks only immediate children of the tested node for the match, hasDescendant matches any descendant.
Here you can see that for your example:
|-CXXConstructorDecl <line:8:3, line:11:3> line:8:3 Dummy_file 'void (const char *, const char *, const char *)'
| |-ParmVarDecl <col:14, col:26> col:26 used f1_name 'const char *'
| |-ParmVarDecl <col:35, col:47> col:47 used f2_name 'const char *'
| |-ParmVarDecl <col:56, col:68> col:68 used mode 'const char *'
| `-CompoundStmt <col:74, line:11:3>
| |-BinaryOperator <line:9:5, col:30> 'FILE *' lvalue '='
| | |-MemberExpr <col:5> 'FILE *' lvalue ->f1_ 0x55d36491a230
| | | `-CXXThisExpr <col:5> 'Dummy_file *' this
| | `-CallExpr <col:11, col:30> 'FILE *'
| | |-ImplicitCastExpr <col:11> 'FILE *(*)(const char *__restrict, const char *__restrict)' <FunctionToPointerDecay>
| | | `-DeclRefExpr <col:11> 'FILE *(const char *__restrict, const char *__restrict)' lvalue Function 0x55d3648fa220 'fopen' 'FILE *(const char *__restrict, const char *__restrict)'
| | |-ImplicitCastExpr <col:17> 'const char *' <LValueToRValue>
| | | `-DeclRefExpr <col:17> 'const char *' lvalue ParmVar 0x55d36491a310 'f1_name' 'const char *'
| | `-ImplicitCastExpr <col:26> 'const char *' <LValueToRValue>
| | `-DeclRefExpr <col:26> 'const char *' lvalue ParmVar 0x55d36491a400 'mode' 'const char *'
CallExpr is a not a child of CXXConstructorDecl, but of BinaryOperator.
Solution
Below I finalized your matcher and checked it in clang-query.
clang-query> match cxxConstructorDecl(hasDescendant(callExpr(callee(functionDecl(hasName("fopen")))).bind("fopencall"))).bind("ctr")
Match #1:
$TEST_DIR/test.cpp:8:3: note: "ctr" binds here
Dummy_file(const char *f1_name, const char *f2_name, const char *mode) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$TEST_DIR/test.cpp:9:11: note: "fopencall" binds here
f1_ = fopen(f1_name, mode);
^~~~~~~~~~~~~~~~~~~~
$TEST_DIR/test.cpp:8:3: note: "root" binds here
Dummy_file(const char *f1_name, const char *f2_name, const char *mode) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 match.
I hope this answers your question!
I have the following yacc grammar:
%{
#include <stdio.h>
extern FILE* yyin;
extern char* yytext;
%}
%token VAR ID_NAME TYPE_STRING TYPE_BOOL TYPE_NUMBER CONST
%%
var_declaration: VAR ':' type ID_NAME ';' { printf("var\n"); }
;
const_declaration: CONST ':' type ID_NAME ';' {printf("const\n");}
;
type: TYPE_NUMBER
| TYPE_STRING
| TYPE_BOOL
;
%%
void yyerror (char const *s) {
fprintf (stderr, "%s\n", s);
}
int main(int argc, char** argv[])
{
yyparse();
return 0;
}
It should describe a little language that at this time should allow variable declarations of the form var:<type> <name>; and constants declarations of the form const:<type> <name>;.
When I run yacc -vd grammar.y I get:
yacc -vd grammar.y
grammar.y: warning: 1 nonterminal useless in grammar [-Wother]
grammar.y: warning: 1 rule useless in grammar [-Wother]
grammar.y:16.1-17: warning: nonterminal useless in grammar: const_declaration [-Wother]
const_declaration: CONST ':' type ID_NAME ';' {printf("const\n");}
^^^^^^^^^^^^^^^^^
grammar.y:16.20-67: warning: rule useless in grammar [-Wother]
const_declaration: CONST ':' type ID_NAME ';' {printf("const\n");}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Why does it say that const_declaration rule is useless?
The reason you get this error message is that the rule const_declaration does not appear in any other rule, and is thus not needed for the grammar.
The rule var_declaration is taken as the start rule (as you have not specified a start it uses the first one). This rule only uses the type rule and thus all other rules are redundant, which is what it is telling you.
Perhaps this is a subset of a larger grammar? When all the grammar is in the file and the const_declaration rule is used somewhere the error will go away.
I'm trying to parse statements to see whether they are valid or not according to these rules:
assignment:
id = exp ;
expression:
id op id {op id}
id is combination of digits and char, the first position contain a char.
I'm getting syntax error when I have something like this in my in.txt file: hellow = three3
but that shouldn't be a syntax error, and then when i put something like: hellow = =
that does not display a syntax error but it should. What am i doing wrong?
Lex:
%{
#include "y.tab.h"
#include <stdio.h>
%}
%%
[ \t\n]+ ;
[a-zA-Z][a-zA-Z0-9]* {
ECHO;
return ID;
}
%%
YACC:
%{
#include <stdio.h>
extern FILE * yyin;
%}
%token ID
%left '+' '-'
%left '*' '/' '%'
%right ';'
%%
assignment: expression
|
ID '=' expression
;
expression: ID
|
expression '*' expression
|
expression '/' expression
|
expression '%' expression
|
expression '+' expression
|
expression '-' expression
|
'(' expression ')'
;
%%
int main(void) {
yyin = fopen("in.txt", "r");
yyparse();
fclose(yyin);
return 0;
}
I can't say from just looking at this, but a good way to find out is running your parser when the environment variable YYDEBUG is 1, like in (unix):
YYDEBUG=1 ./a.out
You should get a detailed list of the steps the parser takes and the tokens it receives from the lexer (where the error is in most cases, like in your example: how does the parser ever get the '=' sign and the operators?). Compare this with the y.output file you get when you run yacc -v
i'm trying to make a very simple YACC parser on Pascal language which just includes integer declarations, some basic expressions and if-else statements. however, i cant find the error for hours and i'm going to be crazy soon. terminal says Error at line:0 but it is impossible!. i use flex and byacc for parser.i will be very glad if you can help me. this is my lex file as you can see;
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern int yylval;
int linenum=0;
%}
digit [0-9]
letter [A-Za-z]
%%
if return IF;
then return THEN;
else return ELSE;
for return FOR;
while return WHILE;
PROGRAM return PROGRAM_SYM;
BEGIN return BEGIN_SYM;
VAR return VAR_SYM;
END return END_SYM;
INTEGER return INTEGER_SYM;
{letter}({letter}|{digit})* return identifier;
[0-9]+ return NUMBER;
[\<][\=] return CON_LE;
[\>][\=] return CON_GE;
[\=] return CON_EQ;
[\:][\=] return ASSIGNOP;
; return semiColon;
, return comma;
\n {linenum++;}
. return (int) yytext[0];
%%
and this is my Yacc file
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern FILE *yyin;
extern int linenum;
%}
%token PROGRAM_SYM VAR_SYM BEGIN_SYM END_SYM INTEGER_SYM NUMBER
%token identifier INTEGER ASSIGNOP semiColon comma THEN
%token IF ELSE FOR WHILE
%token CON_EQ CON_LE CON_GE GE LE
%left '*' '/'
%left '+' '-'
%start program
%%
program: PROGRAM_SYM identifier semiColon VAR_SYM dec_block BEGIN_SYM statement_list END_SYM '.'
;
dec_block:
dec_list semiColon;
dec_list:
dec_list dec
|
dec
;
dec:
int_dec_list
;
int_dec_list:
int_dec_list int_dec ':' type
|
int_dec ':' type
;
int_dec:
int_dec comma identifier
|
identifier
;
type:
INTEGER_SYM
;
statement_list:
statement_list statement
|
statement
;
statement:
assignment_list
|
expression_list
|
selection_list
;
assignment_list:
assignment_list assignment
|
assignment
;
assignment:
identifier ASSIGNOP expression_list
;
expression_list:
expression_list expression semiColon
|
expression semiColon
;
expression:
'(' expression ')'
|
expression '*' expression
|
expression '/' expression
|
expression '+' expression
|
expression '-' expression
|
factor
;
factor:
identifier
|
NUMBER
;
selection_list:
selection_list selection
|
selection
;
selection:
IF '(' logical_expression ')' THEN statement_list ELSE statement_list
;
logical_expression:
logical_expression '=' expression
|
logical_expression '>' expression
|
logical_expression '<' expression
;
%%
void yyerror(char *s){
fprintf(stderr,"Error at line: %d\n",linenum);
}
int yywrap(){
return 1;
}
int main(int argc, char *argv[])
{
/* Call the lexer, then quit. */
yyin=fopen(argv[1],"r");
yyparse();
fclose(yyin);
return 0;
}
and finally i take an error at the first line when i give the input;
PROGRAM myprogram;
VAR
i:INTEGER;
i3:INTEGER;
j:INTEGER;
BEGIN
i := 3;
j := 5;
i3 := i+j*2;
i := j*20;
if(i>j)
then i3 := i+50+(45*i+(40*j));
else i3 := i+50+(45*i+(40*j))+i+50+(45*i+(30*j));
END.
For debugging grammars, YYDEBUG is your friend. Either stick #define YYDEBUG 1 in the %{..%} in the top of your .y file, or compile with -DYYDEBUG, and stick a yydebug = 1; in main before calling yyparse and you'll get a slew of info about what tokens the parser is seeing and what it is doing with them....
Your lexical analyzer returns blanks and tabs as tokens, but the grammar doesn't recognize them.
Add a parser rule:
[ \t\r] { }
This gets you to line 6 instead of line 0 before you run into an error. You get that error because you don't allow semicolons between declarations:
dec_block:
dec_list semiColon;
dec_list:
dec_list dec
|
dec
;
dec:
int_dec_list
;
That should probably be:
dec_block:
dec_block dec
|
dec
;
dec:
int_dec_list semiColon
;
Doing this gets you to line 14 in the input.
Incidentally, one of the first things I did was to ensure that the lexical analyzer tells me what it is doing, by modifying the rules like this:
if { printf("IF\n"); return IF; }
In long term code, I'd make that diagnostic output selectable at run-time.
You have a general problem with where you expect semicolons. It is also not clear that you should allow an expression_list in the rule for statement (or, maybe, 'not yet' — that might be appropriate when you have function calls, but allowing 3 + 2 / 4 as a 'statement' is not very helpful).
This grammar gets to the end of the input:
%{
#include <stdio.h>
#include <string.h>
#include "y.tab.h"
extern FILE *yyin;
extern int linenum;
%}
%token PROGRAM_SYM VAR_SYM BEGIN_SYM END_SYM INTEGER_SYM NUMBER
%token identifier INTEGER ASSIGNOP semiColon comma THEN
%token IF ELSE FOR WHILE
%token CON_EQ CON_LE CON_GE GE LE
%left '*' '/'
%left '+' '-'
%start program
%%
program: PROGRAM_SYM identifier semiColon VAR_SYM dec_block BEGIN_SYM statement_list END_SYM '.'
;
dec_block:
dec_block dec
|
dec
;
dec:
int_dec_list semiColon
;
int_dec_list:
int_dec_list int_dec ':' type
|
int_dec ':' type
;
int_dec:
int_dec comma identifier
|
identifier
;
type:
INTEGER_SYM
;
statement_list:
statement_list statement
|
statement
;
statement:
assignment
|
selection
;
assignment:
identifier ASSIGNOP expression semiColon
;
expression:
'(' expression ')'
|
expression '*' expression
|
expression '/' expression
|
expression '+' expression
|
expression '-' expression
|
factor
;
factor:
identifier
|
NUMBER
;
selection:
IF '(' logical_expression ')' THEN statement_list ELSE statement_list
;
logical_expression:
expression '=' expression
|
expression '>' expression
|
expression '<' expression
;
%%
void yyerror(char *s){
fprintf(stderr,"Error at line: %d\n",linenum);
}
int yywrap(){
return 1;
}
int main(int argc, char *argv[])
{
/* Call the lexer, then quit. */
yyin=fopen(argv[1],"r");
yyparse();
fclose(yyin);
return 0;
}
Key changes include removing assignment_list and expression_list, and modifying logical_expression so that the two sides of the expansion are expression, rather than the LHS being logical_expression (which then never had a primitive definition, leading to the problems with warnings).
There are still issues to resolve; the expression_list in the selection should be more restrictive to accurately reflect the grammar of Pascal. (You need a block where that could be a single statement or a BEGIN, statement list, END.)