ANTLR3 : MissingTokenException when using already defined token in input in place of ID -
i stumbled upon problem while writing grammar rules create table. grammar failing when column names defined tokens in grammar (can't have column name 'create' matched 'create' keyword!
simple usecase :
grammar hello; start : 'hello' 'world' id ; id : 'a'..'z'+ ; ws : (' '|'\n'|'\r')+ {$channel=hidden;} ;
for grammar how make "hello world hello" valid input. failing missingtokenexception.
ast
root | start __________________________________ | | | hello world missingtokenexception
thanks in advance.
edit:
i have found inline-rule while definition rule "hello" & "world", still find how works.
grammar hello; stat: keyhello keyworld expr ; expr: id ; /** id text "hello" */ keyhello : {input.lt(1).gettext().equals("hello")}? id ; /** id text "world" */ keyworld : {input.lt(1).gettext().equals("world")}? id ; // end:rules id : 'a'..'z'+ ; ws : (' '|'\n'|'\r')+ {$channel=hidden;} ;
ast
root | start __________________________________ | | | keyhello keyworld expr | | | hello world world
hope might help.
when parsing language, have "reserved words". it's impossible work without them. in case, have 2 options:
define group of reserved words , make id extension of it. don't recommend possibility because terrible mess when work entire grammar, or when use lexer-parser-tree, , want make different tokens or reserved words (maybe skip them).
rw: 'hello' | 'world'; id: 'a'..'z'+ | rw;
in case able sure when parsing rw, not id, because id , rw go through same rule...
- choose other reserved words, better 'hello' or ordinary words it. me, best option.
taking case of second option can define 1 subgroup apart these news id's tell you. can establish difference between selected group of words vocabulary ('world','hello' etc.) , deal them different in tree.
i hope you!!
Comments
Post a Comment