How to use antlr4 to analyze the grammar of .aidl files? - parsing

Question

Actually, I was assigned to analyze the grammar of .aidl files and extract the grammar elements using listener methods.
In order to finish this, I have thought for long and finally I worked out a .g4 file.
grammar aidl3;
file : pack* imp* parcelable? interfa? ;
pack : 'package' WS+ PAC_NAME WS* ';' WS* ;
imp : 'import' WS+ IMP_NAME WS* ';' WS* ;
parcelable : 'parcelable' WS+ PARCE_NAME WS* ';' WS* ;
interfa : INTER_TAG? WS* 'interface' WS+ INTER_NAME WS* '{' WS* methods+ WS* '}' WS*;
methods : RETURN_TYPE WS+ METHOD_NAME WS* '(' WS* argmentsa* WS* argmentsb* WS* ')' WS* ';' WS* ;
argmentsa : TAG? WS* ARG_TYPE WS+ ARG_NAME WS* ',' WS* ;
argmentsb : TAG? WS* ARG_TYPE WS+ ARG_NAME WS* ;
PAC_NAME : ~[; \n\r]+ ;
//PAC_NAME : [_a-zA-Z] [_.a-zA-Z0-9]* ;
IMP_NAME : ~[ ;\n\r]+ ;
PARCE_NAME : ~[ ;\n\r.]+ ;
INTER_TAG : 'oneway';
INTER_NAME : ~[ ;\n\r{.]+ ;
RETURN_TYPE : ~[ ;\n\r.]+ ;
METHOD_NAME : ~[ ;\n\r(]+ ;
TAG : 'in' | 'out' | 'inout' ;
//ARG_TYPE : ~[) ,\n\r]+ ;
ARG_TYPE : [a-zA-Z] ~' '* | [a-zA-Z] ~' '* ' ' '[' ']' ;
ARG_NAME : ~[ ,\n\r).]+ ;
WS: [ \t\n\r];
However, I've run into a weird problem: that is when I'm trying to analyze the .aidl files e.g.
package android.view.accessibility;
oneway interface IAccessibilityInteractionConnection {
void findAccessibilityNodeInfoByAccessibilityId(long accessibilityNodeId, in Region bounds,
int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags,
int interrogatingPid, long interrogatingTid, in MagnificationSpec spec);
void findAccessibilityNodeInfosByViewId(long accessibilityNodeId, String viewId,
in Region bounds, int interactionId, IAccessibilityInteractionConnectionCallback callback,
int flags, int interrogatingPid, long interrogatingTid, in MagnificationSpec spec);
void findAccessibilityNodeInfosByText(long accessibilityNodeId, String text, in Region bounds,
int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags,
int interrogatingPid, long interrogatingTid, in MagnificationSpec spec);
void findFocus(long accessibilityNodeId, int focusType, in Region bounds, int interactionId,
IAccessibilityInteractionConnectionCallback callback, int flags, int interrogatingPid,
long interrogatingTid, in MagnificationSpec spec);
void focusSearch(long accessibilityNodeId, int direction, in Region bounds, int interactionId,
IAccessibilityInteractionConnectionCallback callback, int flags, int interrogatingPid,
long interrogatingTid, in MagnificationSpec spec);
void performAccessibilityAction(long accessibilityNodeId, int action, in Bundle arguments,
int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags,
int interrogatingPid, long interrogatingTid);
}
it would give this output:
[#0,0:6='package',<'package'>,1:0]
[#1,7:7=' ',<WS>,1:7]
[#2,8:41='android.view.accessibility;\noneway',<ARG_TYPE>,1:8]
[#3,42:42=' ',<WS>,2:6]
[#4,43:51='interface',<'interface'>,2:7]
[#5,52:52=' ',<WS>,2:16]
[#6,53:87='IAccessibilityInteractionConnection',<PAC_NAME>,2:17]
[#7,88:88=' ',<WS>,2:52]
[#8,89:89='{',<'{'>,2:53]
[#9,90:90='\n',<WS>,2:54]
[#10,91:91=' ',<WS>,3:0]
[#11,92:92=' ',<WS>,3:1]
[#12,93:93=' ',<WS>,3:2]
[#13,94:94=' ',<WS>,3:3]
[#14,95:98='void',<PAC_NAME>,3:4]
[#15,99:99=' ',<WS>,3:8]
[#16,100:146='findAccessibilityNodeInfoByAccessibilityId(long',<PAC_NAME>,3:9]
[#17,147:147=' ',<WS>,3:56]
[#18,148:167='accessibilityNodeId,',<PAC_NAME>,3:57]
...
You can see in output line 3 '[#2,8:41='android.view.accessibility;\noneway',ARG_TYPE,1:8]' , in which the expression 'pack' uses 'ARG_TYPE' to match 'android.view.accessibility;\noneway'. But.. How could it be? 'ARG_TYPE' never appears in the expression 'pack' and it should have used 'PAC_NAME' to match 'android.view.accessibility'
It would be nice if someone could help me figure this thing out, because I'm facing a close deadline.
In fact, I'm just a new learner and I know that my g4 file doesn't look good, so if possible, could you please tell me how to program the g4 for .aidl in a better way? Or even show me the write answer?
I would be really grateful if you can help me! Thanks!

Answer 1

ANTLR's lexer tries to create tokens with as much characters as possible. And since ARG_TYPE is able to match android.view.accessibility;\noneway (and no other rule can match more characters), an ARG_TYPE token is created. Only when 2 or more rules match the same characters, ANTLR will choose the one defined first.
You must understand that the lexer does not create tokens based on what the parser is trying to match. Tokenisation is a process that is done independently from the parsing phase. Therefor, most of your rules that look like ~[ ;\n\r(]+ are way too broad.
I suggest you take a look at an existing Java grammar, and use that in order to work with AIDL files.
EDIT
If I take the grammar file posted above, and change:
formalParameter
: variableModifier* unannType variableDeclaratorId
;
into:
formalParameter
: 'in'? variableModifier* unannType variableDeclaratorId
;
and change:
interfaceModifier
: annotation
| 'public'
| 'protected'
| 'private'
| 'abstract'
| 'static'
| 'strictfp'
;
into:
interfaceModifier
: annotation
| 'public'
| 'protected'
| 'private'
| 'abstract'
| 'static'
| 'strictfp'
| 'oneway'
;
then your example file parses correctly.