fix(typst): resolve all overlapping-token errors via @precedence
Build and Deploy Verso / deploy (push) Has been cancelled
Build and Deploy Verso / deploy (push) Has been cancelled
Lezer's buildTokenGroups rejects grammars with ambiguous token sets.
Eight overlaps existed:
EscapeChar vs spaces — EscapeChar { _ } matches \t; after '\'
it must win over the @skip spaces token.
"(" / "." vs text tokens — in the LALR-merged state after #CodeIdent,
callSuffix delimiters must beat
MarkupContent / StrongText / EmphText.
"]" vs LineCommentContent — inside #[...], the ContentBlock closer
must win even if it follows "//".
One extended @precedence declaration resolves all eight.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -207,8 +207,14 @@ Escape { "\\" EscapeChar }
|
||||
// Newline item — kept out of @skip so heading detection works.
|
||||
Newline { "\n" }
|
||||
|
||||
// Resolve ambiguities: longer/more-specific tokens win.
|
||||
@precedence { CodeKeyword CodeBool CodeIdent }
|
||||
// Resolve ambiguities: more-specific tokens win over broader catch-alls.
|
||||
// EscapeChar > spaces: after '\', EscapeChar must win over the skip token
|
||||
// (both match \t; without this, '\t' would be mis-tokenized).
|
||||
// "(" > "." > text tokens: after '#' CodeIdent, callSuffix delimiters must win
|
||||
// over MarkupContent/StrongText/EmphText in the LALR-merged state.
|
||||
// "]" > LineCommentContent: inside #[...], ']' closes the ContentBlock even
|
||||
// if it appears after '//' (comment does not "protect" the bracket).
|
||||
@precedence { CodeKeyword CodeBool CodeIdent EscapeChar "(" "." "]" spaces MarkupContent StrongText EmphText LineCommentContent }
|
||||
}
|
||||
|
||||
@skip { spaces }
|
||||
|
||||
Reference in New Issue
Block a user