How to remove entries from the token list at runtime / from being passed to yysyntax_error?

[GitMensch]

I have a rather complex grammar which handles different "dialects".
To support that there is an internal list of reserved words for each dialect and their matching tokens, which is used by the scanner to distinguish between "this literal means either TOKEN1, TOKEN2 or the token WORD" and the grammar uses all possible tokens.

This works quite good in general - until the parser is given input that does not match the "dialect".

Example: TOKEN2 is "disabled" in a given dialect, in this case the scanner returns the token WORD and the user gets an error message like

"unexpected WORD expecting TOKEN1 or TOKEN2"

This is of course confusing because "TOKEN2" is what is actually given as input (the scanner just pass it as different internal token).

I see two possible options to handle this:

1. Improve the parser's error handling and performance by dropping any "disabled" tokens / "clear" them from the internal parser list to never let it be an expected token any more (I have no idea if this is possible or could be achieved, but that seems to be the ideal solution)
2. get into the construction of the diagnostic and remove the "disabled" tokens from the list, before bison produces the message (= also not consider it in the amount of tokens available - too much will disable the verbosity), this is possibly the easiest option but I'm not sure how to actually do this - the definition of yyerror does not help as that is too late as the error message is already constructed

[akimd]

Hi,
I would go for option 2. Have a look at https://www.gnu.org/software/bison/manual/html_node/Syntax-Error-Reporting-Function.html. This should provide you with all you need to filter out the disabled tokens.
Cheers!