LineCommentContent { ![\n]* } matches the empty string, which Lezer
rejects as a zero-length token (infinite-loop risk). Change to ![\n]+
and mark it optional in the LineComment rule so empty // comments parse.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Any item shared between headingTitleItem and document-level item causes
a shift/reduce conflict: the LALR automaton merges the two contexts and
makes the shared token ambiguous. The only structural fix is to make
HeadingTitle a terminal (external tokenizer) that reads greedily to EOL,
giving the LR state machine a context-isolated token that can never
collide with document-level item tokens.
Removes headingTitleItem sub-rule, HeadingText token, and updates
styleTags to match HeadingTitle directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removing Strong and Emphasis from headingTitleItem eliminates the
conflict: both appear in document-level item, causing the LR automaton
to merge heading-title states with document-item states and make "*"
ambiguous (Strong opener vs. end of heading).
HeadingText is widened to ![\n$#`<@\\]+ so "*" and "_" inside headings
are consumed as plain text rather than producing error nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
headingTitleItem* allowed an empty HeadingTitle, causing a shift/reduce
conflict: after HeadingMark, seeing "*" the LR parser couldn't decide
whether to shift it as a Strong inside the heading or reduce HeadingTitle
to empty and treat "*" as a document-level item.
Changing to headingTitleItem+ forces HeadingTitle to be non-empty, so
"*" after HeadingMark must be inside the heading. Empty headings are
handled by Lezer's error recovery.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Strong{strongItem{Emphasis}} and Emphasis{emphItem{Strong}} created a
mutual-recursion cycle that caused Lezer's LR automaton builder to
produce exponentially many states and crash.
Remove each construct from the other's item list. StrongText already
includes '_' and EmphText already includes '*', so nested delimiters
render as plain text inside the opposite construct rather than errors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the StreamLanguage tokenizer with a full LR grammar compiled by
@lezer/generator, giving Typst the same parse-tree infrastructure that
LaTeX and BibTeX already use.
Grammar features:
- Headings (=, ==, …) via SOL-detecting external tokenizer
- Code expressions (#keyword, #func(args), #ident.method, #{…}, #[…])
- Named argument highlighting (key: value in function calls)
- Inline and display math ($…$)
- Strong (*…*) and emphasis (_…_) with bold/italic formatting
- Raw blocks (```lang…```) and inline raw (`…`)
- Nested block comments (/* /* */ */) via depth-tracking external tokenizer
- Labels (<name>) and references (@name)
- Backslash escapes
Infrastructure changes:
- lezer-typst/typst.grammar — new Lezer grammar
- lezer-typst/tokens.mjs — external tokenizers for context-sensitive lexing
- scripts/lezer-latex/generate.mjs — Typst added to grammars array so the
existing lezer-latex:generate script (and Dockerfile step) compile it
- .gitignore — generated typst.mjs / typst.terms.mjs excluded from git
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the custom regex-based ViewPlugin with the official
@codemirror/lang-yaml package. yamlFrontmatter({ content: mdLS })
wraps the Markdown language with a mixed parser: the leading ---/---
block is handed to the full Lezer YAML parser (proper key/value/scalar/
anchor/alias highlighting), while the document body continues to use
the Markdown parser. The manual Frontmatter extension import is also
removed since yamlFrontmatter handles frontmatter recognition itself.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Typst: heading tokenizer now colors the entire heading line (not just the
'=' prefix), and bold/italic markers (*/_) map to strong/emphasis tags
rather than the generic operator tag. A typstHighlightStyle applies
bold/italic formatting even when the active theme lacks .tok-heading.
Quarto: enable @lezer/markdown's Frontmatter extension so the YAML header
is no longer mis-parsed as Setext headings. A new ViewPlugin decorates
frontmatter lines with type-appropriate CSS classes: keys (tok-typeName),
string/bool/number values, comments, and the --- delimiter markers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Apply the same CSS inversion filter to the HTML iframe as is already
applied to pdfjs PDF pages, so Quarto RevealJS presentations respect
the dark mode toggle. Also show the theme button for HTML outputs and
relax the darkModePdf condition to activate for iframes regardless of
the pdfViewer setting.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When output.html exists, findOutputFiles includes project media files
(images, videos) in outputFiles via the MEDIA_REGEX exception so they
get served from the cache. _removeExtraneousFiles then treated them
as extraneous and deleted them. On the next incremental compile,
unchanged binary files are not re-synced, so the files were gone when
Quarto ran and when _appendMissingResourceWarnings checked for them.
Fix: skip deletion for any file that is a project input resource.
Those files appear in outputFiles to be served, not cleaned up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When typst watch doesn't emit "compiled with errors" after a failed
compile, currentLines accumulates indefinitely. The next successful
compile then flushes the buffer including the stale error from the
prior cycle. Reset currentLines at the start of each compile cycle
("[HH:MM:SS] compiling ...") so each log only contains output from
one compile.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Per-project-type setting: Typst defaults to on, LaTeX defaults to off.
Toggle appears in the compile dropdown under "Smooth PDF transition".
The enableTransition flag is read via a ref so toggling does not
reload the current PDF.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The auto-compile effect was calling debouncedAutoCompile() on every changedAt
update (every keystroke), including while a compile was already running. With
a 1000ms maxWait the debounce fired every second even mid-compile, chaining
compiles back-to-back and making the user wait for all of them to drain.
Fix: add `compiling` to the effect's dependency array.
- While compiling: the effect cancels the debounce immediately, preventing
any new compile from being queued.
- When compile finishes (compiling → false): the effect re-runs; if changedAt
is still > 0 (changes were made during the compile), it re-arms the debounce
exactly once. One follow-up compile, then idle.
Also remove the debouncedAutoCompile() re-queue from compiler.ts's
wasCompiling guard — the effect now owns that responsibility.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
document.startViewTransition with an async callback places a ::view-transition
overlay on top of the entire page, intercepting pointer events for the duration
of the callback (up to the 1s safety timeout + 250ms animation). With rapid
auto-compiles this created interface freezes and overlapping transitions that
could leave the visual lock in a broken state, causing 'stuck on compiling'.
Replace with a canvas snapshot overlay + CSS opacity fade-out:
- pointer-events:none so the overlay never blocks input
- snapshot covers the canvas-clear from setDocument() (no white flash)
- on pagerendered: opacity transitions to 0 over 250ms, then overlay removed
- gives the same smooth visual crossfade, reliably, in all browsers
Chrome 126+ retains the element-level startViewTransition path which is
scoped to the PDF container and does not affect the rest of the page.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
document.startViewTransition generates both a named pseudo-element for the PDF
container (ol-pdf-viewer) and a root-level pseudo-element that covers the entire
page, causing the editor to fade along with the PDF.
Inject a temporary <style> that sets animation:none on the root pseudo-elements
before starting the transition, then remove it in transition.finished. Only the
named PDF container crossfades; the editor is unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The original code used container.startViewTransition(setDocument) with a
synchronous callback, giving a 250ms CSS crossfade that looked smooth when
the PDF happened to re-render before the animation ended — but was a race.
Now there are three tiers:
- Chrome 126+: element-level startViewTransition, async, waits for pagerendered
- Chrome 111+ (Brave 138, Edge 111+): document-level startViewTransition with
view-transition-name scoped to the PDF container, same async pattern
- Firefox / Safari / older Chromium: canvas snapshot overlay (no animation,
but seamless — introduced in build #108)
The document-level path restores the smooth fade the user saw on Edge build
#93, now guaranteed to crossfade old→new rather than old→blank.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rate limit: auto-compile requests already have a client-side debounce; skip
the 1-second server-side recently-compiled gate for them to avoid spurious
'too-recently-compiled' rejections that were blocking ~1/3 of Typst compiles.
PDF flicker: add _snapshotCanvases() fallback for browsers without element-level
View Transitions (Chrome <126, Firefox, Safari). Before setDocument() clears the
canvases it copies each rendered page to a positioned overlay; the overlay is
removed once the first page of the new document fires pagerendered, giving a
seamless old→new swap in all browsers. Chrome 126+ continues to use the
startViewTransition async callback path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous implementation used useState() to detect the project type, but the
file tree is loaded asynchronously after the WebSocket joinProject event, so
pathInFolder() always returns null on the initial render.
Use useEffect() instead — it re-runs when getRootDocInfo's reference changes
(i.e. when the file tree populates), correctly detecting .typ root docs.
Also adds updateAutoCompileDebounce() to DocumentCompiler so the tight
debounce can be applied at that point.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Typst projects default autocompile to enabled (300ms debounce / 1s max-wait
instead of 2.5s/5s), so the PDF refreshes nearly as the user types.
- Make startViewTransition wait for the first page to render before completing
the crossfade, eliminating the old-PDF→blank flash on Chrome 126+.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of cold-starting 'typst compile' on every request, TypstRunner
now maintains a long-lived 'typst watch' process per project. Subsequent
compiles reuse the warm process, which caches fonts, packages, and the
compiled AST via Typst's comemo framework — dramatically faster.
Architecture:
- WatchTable: maps compileName → live watcher process + state
- _startWatcher: spawns 'typst watch input.typ output.pdf', registers
stdout/close handlers, then immediately awaits the first compile result.
The resolver is pushed to pendingResolvers synchronously inside the
Promise constructor before any I/O event can fire — eliminating the
race between file-write detection and resolver registration.
- _onWatcherData: parses stdout line-by-line, resolves pending callers
on "compiled successfully/with warnings/with errors" (the three terminal
lines typst watch emits at the end of each compile cycle).
- Graceful restart: watcher is restarted after MAX_COMPILES_BEFORE_RESTART
(1000) cycles to stay clear of Typst's ~65k FileId limit, or immediately
if the "ran out of file ids" message is detected in stdout.
- killTypst: tears down both the watcher and any cold-start fallback job;
called by stopCompile (user-initiated) and clearProject/clearProjectWithListing
(before compile-dir deletion).
- Docker fallback: Settings.clsi.dockerRunner=true falls back to the
original cold-start 'typst compile' path unchanged.
- process.on('exit') kills all watcher process groups on CLSI shutdown.
CompileManager: call TypstRunner.promises.killTypst before deleting the
compile directory in both clearProject and clearProjectWithListing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Typst has no --synctex CLI option (open feature request #289 since 2023).
Revert the frontend guard back to LaTeX-only and remove --synctex from
the Typst compile command. Also remove the temporary logger.warn calls
added for diagnosing the LaTeX synctex issue (now resolved).
The official Typst binary installation in Dockerfile-base is kept as it
is cleaner than using Quarto's modified fork for .typ compilation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Quarto bundles a modified Typst fork that lacks --synctex, making
bidirectional sync impossible. Install the official Typst binary
(v0.13.1) from upstream and use it in TypstRunner instead.
This also means .typ projects now use the unmodified Typst compiler,
which is correct since TypstRunner handles plain .typ files (not .qmd).
QuartoRunner continues to use Quarto's bundled Typst internally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of going through 'quarto typst compile' (which intercepts
--synctex before it reaches Typst), call the Typst binary bundled in
the Quarto .deb directly at /opt/quarto/bin/tools/x86_64/typst.
This allows passing --synctex output.synctex.gz to generate the SyncTeX
file for bidirectional editor↔PDF sync.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Typst's CLI requires options before positional arguments (INPUT OUTPUT).
Placing --synctex after output.pdf caused it to be treated as an extra
positional arg and rejected with 'unexpected argument'.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TypstRunner: add --synctex output.synctex.gz to quarto typst compile,
generating a synctex file alongside the PDF (requires Typst 0.11+,
bundled in Quarto 1.5+).
- use-synctex: extend the root-doc guard from LaTeX-only to also cover
.typ files, enabling the Show in PDF / Show in code buttons for Typst.
The rest of the sync infrastructure (OutputCacheManager, synctex binary,
SynctexOutputParser, CLSI routes) is already format-agnostic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The synctex binary was not included in scheme-basic and was not
explicitly installed, causing `spawn synctex ENOENT` on every
sync request. Add it alongside latexmk and texcount.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Logs: request params, directory used, whether output.synctex.gz
is found, and the actual synctex binary output or error.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Verso added 'qmd' and 'typ' to validRootDocExtensions, which caused
isValidTeXFile() to return true for Typst/Quarto files — enabling
SyncTeX UI controls for projects that never produce output.synctex.gz.
Replace the open-doc extension check in canSyncToPdf with a
LaTeX-only regex on the project root document path (tex|ltx|Rtex|Rnw),
and add the same guard in _syncToCode so PDF-click sync never fires
an API request for non-LaTeX projects.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Replace token-link email with 6-digit code on SSO registration
Unverified SSO emails previously received a long-lived token link
(90-day TTL) via UserEmailsConfirmationHandler. This replaces that
flow with the same 6-digit code verification used for password
registration, redirecting through /registration/confirm-email.
- SSOManager.registerSSO now always confirms email (caller must
verify first); removes sendConfirmationEmail / _finishRegistration
- SSOController._signUp sends confirmation code and stores
pendingSSORegistration in session when IdP email_verified is false
- New SSOConfirmEmailHandler completes registration after code check
via completeSSOEmailConfirmation module hook
- OnboardingController confirm-email handlers accept
pendingSSORegistration alongside pendingUserRegistration
confirmEmailFromToken (POST /user/emails/confirm) removal is deferred
to a follow-up PR to avoid breaking in-flight 90-day tokens.
Closes#28607
* Fix unverified-email edge cases; Add ORCID e2e tests;
* Rename `confirmEmail` parameter to `emailVerifiedByIdP` in _signUp function
* Remove `sendConfirmationEmail`
* Mock getUserByAnyEmail in tests
* Extract _finishSSORegistration helper to deduplicate the register →
set session flags → allocate referral → finishSaasLogin → finishLogin
sequence shared by both the direct and deferred (code-confirmed) paths.
* Stop duplicating session data in pendingSSORegistration
analyticsId, splitTests, and referal_* are already in the session at
confirmation time — no need to copy them into pendingSSORegistration.
Re-fetch splitTests fresh on completion instead.
* Simplify the code
* Remove dead confirmEmail template
No callers remain after sendConfirmationEmail was deleted. The token-link
flow (confirmEmailFromToken) only validates tokens, never sends email.
* Remove dead reconfirmEmail template
* Address comments from Copilot
* Clear stale pending registration when starting a new flow
* Add unit tests for completeSSOEmailConfirmation
* Add `verificationMethod` param
* Fix camelcase issues
* Extract _createSSOUser and _registerAndFinish helpers to deduplicate registration logic
* Remove obscure "registration_error"
* Prevent FormTextIcon from shrinking
* Enable "email_already_registered_sso" error
* Misc. improvements to confirm-email-form.tsx
* Remove `UserEmailsConfirmationHandler` mock
Co-authored-by: Olzhas Askar <olzhas.askar@overleaf.com>
* Add info on sso_email.pug page
---------
Co-authored-by: Olzhas Askar <olzhas.askar@overleaf.com>
GitOrigin-RevId: d0196ebc6d81ff61bcd27726d0b899b743d08d64
* [monorepo] consolidate clsi-lb host/ip env-vars
Target env-var is CLSI_LB_HOST. Keep CLSI_LB_IP populated for a week.
* [clsi] initial version of /convert/pdf-to-jpeg
* [rails] use fake-secrets in CI and Codespaces
* [rails] adapt tests for using clsi to convert PDFs to image
* [rails] add rake task for comparing clsi conversion with transloadit
* [clsi] double check that output.jpg is a regular file
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [clsi] fix composing basename
* [monorepo] fix clsi-lb host env-var post merge
* [monorepo] sort dev-environment.env hosts
* [rails] use local pdf file rather than downloading it again
Download from the old renderer code path still. It's dead code.
* [terraform] clsi: enable pdf to jpg conversion
---------
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 5ecaa8559d299486340bb3961f06b29f7c4dfcca
claude