Instead of cold-starting 'typst compile' on every request, TypstRunner
now maintains a long-lived 'typst watch' process per project. Subsequent
compiles reuse the warm process, which caches fonts, packages, and the
compiled AST via Typst's comemo framework — dramatically faster.
Architecture:
- WatchTable: maps compileName → live watcher process + state
- _startWatcher: spawns 'typst watch input.typ output.pdf', registers
stdout/close handlers, then immediately awaits the first compile result.
The resolver is pushed to pendingResolvers synchronously inside the
Promise constructor before any I/O event can fire — eliminating the
race between file-write detection and resolver registration.
- _onWatcherData: parses stdout line-by-line, resolves pending callers
on "compiled successfully/with warnings/with errors" (the three terminal
lines typst watch emits at the end of each compile cycle).
- Graceful restart: watcher is restarted after MAX_COMPILES_BEFORE_RESTART
(1000) cycles to stay clear of Typst's ~65k FileId limit, or immediately
if the "ran out of file ids" message is detected in stdout.
- killTypst: tears down both the watcher and any cold-start fallback job;
called by stopCompile (user-initiated) and clearProject/clearProjectWithListing
(before compile-dir deletion).
- Docker fallback: Settings.clsi.dockerRunner=true falls back to the
original cold-start 'typst compile' path unchanged.
- process.on('exit') kills all watcher process groups on CLSI shutdown.
CompileManager: call TypstRunner.promises.killTypst before deleting the
compile directory in both clearProject and clearProjectWithListing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Typst has no --synctex CLI option (open feature request #289 since 2023).
Revert the frontend guard back to LaTeX-only and remove --synctex from
the Typst compile command. Also remove the temporary logger.warn calls
added for diagnosing the LaTeX synctex issue (now resolved).
The official Typst binary installation in Dockerfile-base is kept as it
is cleaner than using Quarto's modified fork for .typ compilation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Logs: request params, directory used, whether output.synctex.gz
is found, and the actual synctex binary output or error.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Base image: add opencv-python-headless (cv2) and tqdm to the bundled
scientific stack, and python3-venv (needed to build per-project venvs).
Per-project dependencies: a project's requirements.txt is now installed into a
venv cached by its sha256 (python3 -m venv --system-site-packages, so the
bundled stack stays visible and only extra packages are installed); QuartoRunner
points Quarto at it via QUARTO_PYTHON. A per-hash flock serialises concurrent
builds; pip output is merged into output.log; on failure the render falls back
to the base interpreter. Venvs live under PYTHON_VENVS_DIR
(default /var/lib/overleaf/data/python-venvs).
Gating: PythonVenvGate.userCanInstallPython restricts installs to the project
owner + invited collaborators (ignorePublicAccess excludes anonymous/link
users), threaded to CLSI as allowPythonInstall on the editor compile,
presentation export, and publish paths. Behind OVERLEAF_ENABLE_PROJECT_PYTHON_VENV
(enabled in the deployment). Design doc updated; Phase 2 (egress policy) and
Phase 3 (venv eviction) remain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In RevealJS mode the download button becomes a 2-choice menu:
- Standalone HTML: a one-off compile with embed-resources (chalkboard and other
runtime-only plugins are dropped, since they don't survive self-containment),
yielding a single portable .html.
- Slide PDF: render the deck, then print it with decktape (headless Chromium)
to a faithful one-slide-per-page PDF.
Implementation:
- Dockerfile-base: install decktape + headless Chromium (open-source; deps via
playwright install-deps for Ubuntu-Noble correctness). Base-only change.
- QuartoRunner honours options.exportMode ('html-standalone' | 'pdf-slides');
exportMode is threaded web ClsiManager -> CLSI RequestParser -> CompileManager
-> runner.
- New GET /project/:id/presentation-export/:format compiles in the matching
export mode and streams the result as a download (PresentationExportController,
reusing ClsiManager.getOutputFileStream).
- pdf-hybrid-download-button shows the dropdown when the output is output.html;
PDF/LaTeX projects keep the single download button.
- i18n: download_as_standalone_html / download_as_pdf_slides (en + fr +
extracted-translations.json).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A project whose root file is a .typ file now compiles straight to PDF with
Typst, as a third engine beside Quarto (.qmd) and latexmk (.tex). Dispatch
stays purely extension-based.
CLSI:
- New TypstRunner.js: runs `quarto typst compile <main>.typ output.pdf` (reuses
the Typst bundled in Quarto, so no extra binary / Docker change). stderr is
merged into output.log.
- CompileManager: _isTypstFile + a TypstRunner branch in _getRunner, and
TypstRunner added to the isRunning check and stopCompile kill list.
- RequestParser: 'typst' added to VALID_COMPILERS.
web:
- settings.defaults: 'typ' added to validRootDocExtensions and the text
extensions (so .typ opens in the editor); 'typst' added to safeCompilers.
- output-files: the Quarto/Typst log parser (which already understands Typst
`error:`/`warning:` + `┌─ file:line:col` diagnostics) now also handles .typ
compiles, so their errors/warnings populate the log tabs.
Polish:
- New-project menu: "Blank Typst project" + "Example Typst project" in both the
main and welcome dropdowns, backed by createBasicProject/createExampleProject
flavour 'typst', a new mainbasic.typ template and an example-project-typst
presentation (math, an image, a table, lists).
- Compiler dropdown gains a "Typst" option (cosmetic; dispatch is by extension).
README updated: three compilers side by side, with a Writing-a-Typst-document
section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verso now compiles both .tex (latexmk) and .qmd (Quarto) projects,
dispatching by the root file's extension rather than replacing one with
the other. LaTeX and Quarto projects can coexist on the same server.
CompileManager: re-import LatexRunner and add a _getRunner() dispatcher
that returns a uniform {run, isRunning, kill} interface. .qmd/.md/.Rmd
→ QuartoRunner; everything else (.tex/.ltx/.Rtex/.Rnw) → LatexRunner.
stopCompile now checks/kills both runners since it has no root path.
compiler-setting.tsx: restore the LaTeX engine choices (pdfLaTeX, LaTeX,
XeLaTeX, LuaLaTeX) alongside Quarto. The dropdown still controls which
TeX engine latexmk uses; actual engine dispatch is by file extension.
Dockerfile-base: reinstall TeX Live alongside Quarto (texlive-full minus
-doc/-lang- packages, plus xetex/luatex/biber/latexmk/texcount/chktex/
synctex). Restore TEXMFVAR for a writable LuaTeX cache. This brings back
a large image, which is the accepted cost of full LaTeX+Quarto support.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dockerfile-base: remove TeX Live (no longer needed), install Quarto
1.6.39 which bundles Typst for PDF output. This was the root cause
of all compile failures — the server-ce monolith never had Quarto.
QuartoRunner: run quarto via /bin/sh so stderr is merged into stdout
with 2>&1; write combined output to output.log (not output.stdout)
so the PDF-preview log panel picks it up and shows raw output.
Also write the log on error so failures are always visible.
CompileManager: guard DraftModeManager behind an isLatexFile check —
injecting LaTeX preamble commands into a .qmd file corrupts it and
causes a guaranteed compile failure when draft mode is requested.
ProjectCreationHandler + mainbasic.qmd: new projects now create
main.qmd with a minimal Quarto/Typst frontmatter instead of the
LaTeX main.tex; _createRootDoc names the file main.qmd accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add QuartoRunner.js: runs `quarto render --to typst --output output.pdf`,
using Typst (bundled with Quarto >= 1.4) so no separate LaTeX install needed
- Swap LatexRunner for QuartoRunner in CompileManager; remove latexmk-specific
stats, fdb metrics, and performance sampling that no longer apply
- Add 'quarto' to VALID_COMPILERS in RequestParser and set it as the default;
change default rootResourcePath from main.tex to main.qmd
- Add 'quarto' to safeCompilers and set it as the default in web settings
- Replace with-texlive Dockerfile stage with with-quarto (Quarto deb install);
add Quarto to the default final stage as well
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* [clsi] add request flag for isCompileFromHistory
* [clsi] derive cacheKey for history snapshot from compile dir
* [clsi] migrate convert project to document to compile from history
* [clsi] address review feedback
* [web] determine root doc at the time of converting the project
* [web] wait for flush before starting document conversion
* [saas-e2e] add tests for root doc override when converting project
GitOrigin-RevId: 71c578030949b89f3a74e7f7ab882dfa9c98c17a
* [clsi] handle draft mode and tikzexternalize as part of sync phase
* [clsi] emit empty string from SafeReader on ENOENT
* [clsi] persist history state after clearing dirty state without changes
GitOrigin-RevId: d9dcd2e6887017f7935b5e95bdbdc6e11a3b18f5
* [clsi] avoid server error when clearing cache while compiling
* [clsi] tweak API around releasing locks
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
---------
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: d3f171467d3bc26941758dd333f30049b37a05c8
* [saas-e2e] test gallery templates with binary file
* [rails] add make target for fixing rubocop errors
* [rails] migrate compiles of conversions/submissions to history mode
* [rails] forward version to clsi request
* [rails] trim down compile request
* [saas-e2e] source v1 secrets after make install
GitOrigin-RevId: 65269e1df1051c9f3b4f1813d2e9dcf32a01be50
* [clsi] initial implementation of compile from history
* [clsi] copy changes
* [saas-e2e] extend test case with nested folder
* [saas-e2e] add test case for tracked changes
* [web] fix accumulating changes from multiple chunks
* [web] optimize size check for compile request payload
* [clsi] deduplicate globalBlobs
* [clsi] add validation for request body details
* [clsi] add metrics for compile from history
* [clsi] download binary files concurrently
* [clsi] skip download of empty file blob
* [clsi] break down e2e compile time metric by compileFromHistory
GitOrigin-RevId: 0dadef93e89d8a172c35cb130a1042d9d1bec42a
* [clsi] remove all clsi-perf/health-check metrics
* [clsi] always emit E2E compile time metric
* [clsi] do not collect metrics for clsi-cache-template compiles
* [clsi] fix unit tests: request.metricsOpts always exists
* [clsi] use a gauge for the e2e compile time metric of clsi-perf
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* [clsi] remove metrics for binary file downloads from clsi-perf
---------
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 7995512e57c802086350e3d1a0ec5213ecdf0a05
* [clsi] add stats and timings to compile response from clsi-cache
* [clsi] set downloadedFromCache when previously downloaded for synctex
Assumption: every compile will emit an output.log. When the output.log
is missing, but the output.synctex.gz exists, it must have been
downloaded from the cache.
GitOrigin-RevId: 41ea34880931e3c43dda3bc9eb26c0d02054894d
* [clsi] tell frontend when synctex mapping was downloaded from clsi-cache
* [web] emit event when synctex mapping was downloaded from clsi-cache
GitOrigin-RevId: 1f6b7e0faaa7dd76449aad566802da971a4cf9ed
* [clsi] drop support for docker-in-docker
* [clsi] run SyncTeX in specific output dir rather than compile dir
* [clsi] store output.synctex.gz outside of tar-ball in clsi-cache
* [clsi] add documentation for rewriting of docker bind-mounts
* [server-pro] update env vars for sandboxed compiles in sample config
GitOrigin-RevId: 8debd7102ac612544961f237aa4ff1c530aa3da3
* [clsi-cache] initial revision of the clsi-cache service
* [clsi] send output files to clsi-cache and import from clsi-cache
* [web] pass editorId to clsi
* [web] clear clsi-cache when clearing clsi cache
* [web] add split-tests for controlling clsi-cache rollout
* [web] populate clsi-cache when cloning/creating project from template
* [clsi-cache] produce less noise when populating cache hits 404
* [clsi-cache] push docker image to AR
* [clsi-cache] push docker image to AR
* [clsi-cache] allow compileGroup in job payload
* [clsi-cache] set X-Zone header from latest endpoint
* [clsi-cache] use method POST for /enqueue endpoint
* [web] populate clsi-cache in zone b with template data
* [clsi-cache] limit number of editors per project/user folder to 10
* [web] clone: populate the clsi-cache unless the TeXLive release changed
* [clsi-cache] keep user folder when clearing cache as anonymous user
* [clsi] download old output.tar.gz when synctex finds empty compile dir
* [web] fix lint
* [clsi-cache] multi-zonal lookup of single build output
* [clsi-cache] add more validation and limits
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [clsi] do not include clsi-cache tar-ball in output.zip
* [clsi-cache] fix reference after remaining constant
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
* [web] consolidate validation of filename into ClsiCacheHandler
* [clsi-cache] extend metrics and event tracking
- break down most of the clsi metrics by label
- compile=initial - new compile dir without previous output files
- compile=recompile - recompile in existing compile dir
- compile=from-cache - compile using previous clsi-cache
- extend segmentation on compile-result-backend event
- isInitialCompile=true - found new compile dir at start of request
- restoredClsiCache=true - restored compile dir from clsi-cache
* [clsi] rename metrics labels for download of clsi-cache
This is in preparation for synctex changes.
* [clsi] use constant for limit of entries in output.tar.gz
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* [clsi-cache] fix cloning of project cache
---------
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Alf Eaton <alf.eaton@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 4901a65497af13be1549af7f38ceee3188fcf881
* [clsi] make error copying output files a fatal compile error
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
* [clsi] remove unused endpoints for downloading files from compileDir
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
* [clsi] avoid useless clone of output files
* [clsi] add test for output files when compile did not produce a PDF
---------
Co-authored-by: Rebeka <rebeka.dekany@overleaf.com>
GitOrigin-RevId: cb998b99b4d96cb48ddd70987958f614ad3b40fc
* Add `unicorn/prefer-node-protocol`
* Fix `unicorn/prefer-node-protocol` ESLint errors
* Run `npm run format:fix`
* Add sandboxed-module sourceTransformers in mocha setups
Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`
* Remove `node:` in the SandboxedModule requires
* Fix new linting errors with `node:`
GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
