fix(typst): add newlines to @skip so multi-line code args parse cleanly
Build and Deploy Verso / deploy (push) Successful in 9m31s
Build and Deploy Verso / deploy (push) Successful in 9m31s
'\n' inside CodeArgs was an invalid token, triggering Lezer error recovery and resetting parser state before codeIdentTokenizer could fire. Heading detection is unaffected — headingTokenizer uses raw input.peek(-1) char reads which see the '\n' byte regardless of what @skip consumes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -27,7 +27,6 @@ item {
|
|||||||
Label |
|
Label |
|
||||||
Ref |
|
Ref |
|
||||||
Escape |
|
Escape |
|
||||||
Newline |
|
|
||||||
MarkupContent
|
MarkupContent
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -184,10 +183,12 @@ Escape { "\\" EscapeChar }
|
|||||||
|
|
||||||
// ── Regular tokens ────────────────────────────────────────────────────────
|
// ── Regular tokens ────────────────────────────────────────────────────────
|
||||||
@tokens {
|
@tokens {
|
||||||
// Horizontal whitespace only. Newlines are kept as explicit Newline items
|
// All whitespace including newlines. Heading detection still works because
|
||||||
// so that HeadingMark (which checks start-of-line via input.peek(-1)) can
|
// headingTokenizer uses input.peek(-1) on the raw character stream — it sees
|
||||||
// reliably detect newlines in the raw input stream.
|
// the '\n' byte regardless of what @skip consumes at the token level.
|
||||||
spaces { $[ \t]+ }
|
// Including '\n' here lets multi-line code expressions (e.g. #figure(\n ...\n))
|
||||||
|
// parse without error instead of triggering Lezer error recovery.
|
||||||
|
spaces { $[ \t\n\r]+ }
|
||||||
|
|
||||||
// Boolean / null literals — distinct from keywords for highlighting.
|
// Boolean / null literals — distinct from keywords for highlighting.
|
||||||
CodeBool { "true" | "false" | "none" | "auto" }
|
CodeBool { "true" | "false" | "none" | "auto" }
|
||||||
@@ -213,9 +214,6 @@ Escape { "\\" EscapeChar }
|
|||||||
// Escape: any single character after backslash.
|
// Escape: any single character after backslash.
|
||||||
EscapeChar { _ }
|
EscapeChar { _ }
|
||||||
|
|
||||||
// Newline item — kept out of @skip so heading detection works.
|
|
||||||
Newline { "\n" }
|
|
||||||
|
|
||||||
// Resolve ambiguities in merged states:
|
// Resolve ambiguities in merged states:
|
||||||
// EscapeChar > spaces: after '\', EscapeChar must win over the skip token.
|
// EscapeChar > spaces: after '\', EscapeChar must win over the skip token.
|
||||||
// "(" > "." > "]": callSuffix delimiters must win over MarkupContent after
|
// "(" > "." > "]": callSuffix delimiters must win over MarkupContent after
|
||||||
|
|||||||
Reference in New Issue
Block a user