How to parse a partial date in ANTLR? - parsing

Question

I am taking my first steps to use antlr4 and try to parse a partial date in european format DD.MM.YYYY.
I want to recognize a normal date like 15.05.2020 or 7.5.20 but also dates which only contains month and year like 05.2020 or 5.20 and in addition to that dates that only contain out of a year like 2020 or 20. In my application I want to have access to all parts of a date (day, month and year) at which some parts may be empty/null.
Here is my grammar so far.
grammar LogicalDateExpressions;
stmt : date EOF
;
date : (YEAR)
| (MONTH DOT YEAR)
| (DAY DOT MONTH DOT YEAR)
;
YEAR : ([12] [0-9] [0-9] [0-9])
| ([0-9] [0-9])
;
MONTH : ('0'? [1-9])
| ('1' [012])
;
DAY : ('0'? [1-9])
| ([12][0-9])
| ('3'[01])
;
DOT : '.';
WS : [ \t\r\n\u000C]+ -> skip;
This grammar works with a single year (2020) but fails to recognize a month-year combination (05.2020). grun -tokens told me the following.
[#0,0:1='05',<YEAR>,1:0]
[#1,2:2='.',<'.'>,1:2]
[#2,3:6='2020',<YEAR>,1:3]
[#3,9:8='<EOF>',<EOF>,2:0]
line 1:2 mismatched input '.' expecting <EOF>
So with my smattering I figured the parser rule date is the problem and I rewrote it to
date : (
(DAY DOT)?
MONTH DOT
)?
YEAR
;
But I still got the same error. Then I thought maybe I need to reorder the lexer rules. So instead of YEAR -> MONTH -> DAY, I wrote them DAY -> MONTH -> YEAR. But grun told me.
[#0,0:1='05',<DAY>,1:0]
[#1,2:2='.',<'.'>,1:2]
[#2,3:6='2020',<YEAR>,1:3]
[#3,9:8='<EOF>',<EOF>,2:0]
line 1:3 mismatched input '2020' expecting MONTH
I also tried to change the order of the or'ed alternatives in the parser rule date but that did not work out either. Then I tried to change the lexer rules DAY, MONTH, YEAR to make them parser rules (day, month, year). After getting some errors because apparently the [0-9] notation is not allowed in in parser rules I changed the grammar to this.
date : (year)
| (month DOT year)
| (day DOT month DOT year)
;
[...]
year : (('1'|'2') DIGIT DIGIT DIGIT)
| (DIGIT DIGIT)
;
month : ('0'? DIGIT_NO_ZERO)
| ('1' ('0'|'1'|'2'))
;
day : ('0'? DIGIT_NO_ZERO)
| (('1'|'2') DIGIT)
| ('3' ('0'|'1'))
;
[...]
DIGIT : [0-9];
DIGIT_NO_ZERO : [1-9];
That was a bummer too. grun told me.
[#0,0:0='0',<'0'>,1:0]
[#1,1:1='5',<DIGIT>,1:1]
[#2,2:2='.',<'.'>,1:2]
[#3,3:3='2',<'2'>,1:3]
[#4,4:4='0',<'0'>,1:4]
[#5,5:5='2',<'2'>,1:5]
[#6,6:6='0',<'0'>,1:6]
[#7,9:8='<EOF>',<EOF>,2:0]
line 1:1 no viable alternative at input '05'
As far as I understand the language I am looking for is a regular one. And every input is unambiguous. So I tried to get the whole "logic" into the lexer and I succeeded with the following grammar.
grammar LogicalDateExpressions;
stmt : date EOF
;
date : DT
;
DT : (
((('0'? [1-9])|([12][0-9])|('3'[01])) DOT)? // Day
(('0'? [1-9])|('1' [012])) DOT // Month
)?
((DIGIT DIGIT DIGIT DIGIT)|(DIGIT DIGIT)) // Year
;
DIGIT : [0-9];
DOT : '.';
WS : [ \t\r\n\u000C]+ -> skip;
It parses every input I give it. But the problem is that every input is just a DT.
[#0,0:6='05.2020',<DT>,1:0]
[#1,9:8='<EOF>',<EOF>,2:0]
I can not distinguish between the day, the month and the year in a visitor/listener because labels are not allowed in lexer rules.
So my question is where is the problem with the first given grammar and what do I need to change to make it work?
From a look at the token output from grun I think I might grasp the problem every input for a day, month and/or year might be ambiguous but as a whole input in conjunction with the dots it should not be. How can I tell antlr that?

Answer 1

So my question is where is the problem with the first given grammar and what do I need to change to make it work?
The problem is that the lexer is not driven by the parser. What this means is that when the parser tries to match the tokens DAY DOT MONTH and the input is 01.01, the lexer will not create a DAY and a MONTH for these two 01's, but two MONTH tokens. This is how ANTLR's lexer works: try to grab as much characters for a token, and when there are 2 or more tokens that match the same characters (like 01 can be matched by both DAY and MONTH), let the token defined first "win" (which is the MONTH token). There is no way around this.
What you could do is something like this (untested):
date
: year
| month DOT year
| day DOT month DOT year
;
day
: N_01_12
| N_13_31
;
month
: N_01_12
;
year
: N_01_12
| N_13_31
| N_32_99
| N_1000_2999
;
N_01_12
: '0'? D // 01-09
| '1' [0-2] // 10-12
;
N_13_31
: '1' [3-9] // 13-19
| '2' D // 20-29
| '3' [01] // 30-31
;
N_32_99
: '3' [2-9] // 32-39
| [4-9] D // 40-99
;
N_1000_2999
: [12] D D D // 1000-2999
;
fragment D : [0-9];