ANTLR v4 grammar to recognize jQuery blocks in Java code - parsing

Question

I am having a hard time trying to implement a grammar to parse jQuery blocks in between java code.
I do not need to implement a java grammar. This is going to be a translator. I just need to output the java as it is and translate jQuery to java...
jQuery blocks are surrounded by the following tokens: /*#jQ ... */. There can be multiple blocks, but nesting is not allowed. Here is an example:
package test;
public class Test {
public static void main(String[] args) {
System.out.println("Hello world!");
/*#jQ
*/
System.out.println("Good bye world!");
}
}
The desired output of the translator, for this particular case, would be:
package test;
public class Test {
public static void main(String[] args) {
System.out.println("Hello world!");
System.out.println("Good bye world!");
}
}
The problem is I am not being able to read java until a /*#jQ is found. Here is an excerpt of what I have so far:
main
:
java
(
jQueryBlock+ java
)*
;
java
:
.*?
;
jQueryBlock
:
JQUERYBLOCKSTART
(
jQueryStatement SINGLE_LINE_COMMENT?
)* JQUERYBLOCKEND
;
and...
JQUERYBLOCKSTART
:
'/*#jQ'
;
Although the generated parse tree is somewhat acceptable (see below), I get several token recognition error...
JjQuery::main:3:22: token recognition error at: '{'
JjQuery::main:5:44: token recognition error at: '{'
JjQuery::main:6:12: token recognition error at: '.'
JjQuery::main:6:16: token recognition error at: '.'
JjQuery::main:6:37: token recognition error at: '!"'
JjQuery::main:12:12: token recognition error at: '.'
JjQuery::main:12:16: token recognition error at: '.'
JjQuery::main:12:40: token recognition error at: '!"'
JjQuery::main:13:5: token recognition error at: '}'
JjQuery::main:15:4: token recognition error at: '}'
Thanks in advance!
UPDATE
I have modified my grammar as suggested, but I'm still having some problems. Here is an example input, the generated parse tree, and below it the errors thrown.
warning(155): Lexer.g4:22:28: rule SINGLE_LINE_COMMENT contains a lexer command with an unrecognized constant value; lexer interpreters may produce incorrect output
warning(155): Lexer.g4:28:25: rule WS contains a lexer command with an unrecognized constant value; lexer interpreters may produce incorrect output
Parser::src:1:3: extraneous input '\n\n' expecting {<EOF>, '/*#jQ', JAVA}
Parser::src:3:5: token recognition error at: '\n'
Parser::src:4:0: token recognition error at: '\n'
Parser::src:5:2: token recognition error at: ' '
Parser::src:5:8: token recognition error at: '\n'
Parser::src:6:0: token recognition error at: '\n'
Parser::src:7:2: extraneous input '\n\n' expecting {<EOF>, '/*#jQ', JAVA}
Here is the current Lexer.g4:
lexer grammar Lexer;
#lexer::members {
public static final int WHITESPACE = 1;
public static final int COMMENTS = 2;
}
// Default mode rules (the SEA)
JQBegin
:
'/*#jQ' -> pushMode ( JQUERY )
;
JAVA
:
.
;
WS
:
[ \t\r\n]+ -> channel ( WHITESPACE ) // channel(1)
;
SINGLE_LINE_COMMENT
:
'//' .*? '\n' -> channel ( COMMENTS ) // channel(2)
;
mode JQUERY;
JQEnd
:
'*/' -> popMode
;
IN
:
'in'
;
OUT
:
'out'
;
ID
:
[a-zA-Z_] [a-zA-Z0-9_]*
;
SEMICOLON
:
';'
;
And the Parser.g4:
parser grammar Parser;
options {
tokenVocab = Lexer;
} // use tokens from ModeTagsLexer.g4
src
:
(
JAVA
| jQuery
)+ EOF
;
jQuery
:
JQBegin
(
in
| out
)* JQEnd
;
in
:
IN ID SEMICOLON
;
out
:
OUT ID SEMICOLON
;

Answer 1

Use lexical modes to separately handle JQuery and Java blocks (even though the Java blocks are trivial in your case). Note, lexer modes are only available in Lexer grammars and not in combined grammars.
Also, the Java catchall must match a single character at a time. Otherwise it can consume the JQuery begin sequence (this is likely the source of the errors you are seeing).
main: ( JAVA | jqBlock )+ EOF ;
jqBlock: JQBegin
( ... | ... | ... ) // your JQuery rules
JQEnd
;
JQBegin: '/*#jQ' -> pushMode(JQ) ;
JAVA : . ;
mode JQ;
... // your JQuery specific rules
BlockComment : '/*' .*? '*/' ; // handle any possibly ambiguous
// sequences that otherwise might
// cause early exits
JQEnd: '*/' -> popMode() ;