275 Commits

Author SHA1 Message Date
claude 60d48ae532 docs: release alpha 3 — remove in-progress marker, fix export description
Build and Deploy Verso / deploy (push) Has been cancelled
Build and Deploy Verso (prod) / deploy (push) Successful in 1m19s
Mark Alpha 3 as released (drop the "in progress" qualifier) and correct
the bidirectional format export entry: only LaTeX ↔ Typst conversion is
available; DOCX, Markdown and HTML exports are not yet implemented.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 11:32:41 +00:00
claude a796577199 fix(security): restrict publish-presentation routes to project owners
Build and Deploy Verso / deploy (push) Successful in 10m54s
Read-only collaborators and token-link users could publish, unpublish,
and rotate presentation share tokens. Change all three write endpoints
from ensureUserCanReadProject to ensureUserCanAdminProject so only the
project owner can perform these actions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 10:17:24 +00:00
claude 1814cba458 fix(mobile): extend CSS breakpoints to cover Tor Browser spoofed viewport
Build and Deploy Verso / deploy (push) Successful in 10m11s
Tor Browser sets viewport width to ~980px to resist fingerprinting, so
@media (max-width: 767px) never fires on a real phone using it. Add the
secondary condition (pointer: coarse) and (max-width: 1024px) — same
dual-check already used in JS — to all mobile CSS overrides in
ide-lumiere.scss and project-list-lumiere.scss. This activates the font
scaling, toolbar height increase, auto-zoom prevention, and project page
layout fixes on Tor Browser.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 08:24:29 +00:00
claude 4c5db24963 fix(mobile): prevent editor auto-zoom on iOS/Android
Build and Deploy Verso / deploy (push) Successful in 10m28s
Browsers zoom any focused editable element (including contenteditable)
whose font-size is below 16px. Apply font-size: max(1rem, 1em) on
.cm-content inside the mobile media query so the 16px floor is enforced
while still respecting user-set sizes larger than that.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 07:01:20 +00:00
claude 2d44c920fd fix(mobile): scale up editor UI chrome fonts and toolbar on mobile
Build and Deploy Verso / deploy (push) Successful in 12m33s
Adds a @media (max-width: 767px) block scoped to .ide-redesign-main
that bumps the CSS font-size tokens one step each and increases the
toolbar height, making buttons, labels, and panel headers readable
on a phone without touching the CodeMirror editor font size (which
is controlled by user settings independently).

Also reverts the unintended rail auto-collapse from the previous
commit — collapsing the sidebar was not the requested change.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 22:23:23 +00:00
claude 7bc60a4e10 feat(mobile): collapse rail by default; tighten project page header layout
Build and Deploy Verso / deploy (push) Has been cancelled
Editor:
- Auto-collapse the file-tree rail on mobile so the editor+PDF panels
  occupy the full screen width instead of sharing it with a 15% sidebar.
  The user can still open the rail from the toolbar; the collapse only
  fires on first load or when switching to a mobile viewport.

Project page (Lumiere):
- Move the XS/M/L zoom buttons from the mobile filter-pill toolbar into
  a row with the New Project button, so neither is stranded alone.
- The search bar gets its own full-width row above the actions row.
- Desktop layout is unchanged (search + new-project stay side-by-side;
  zoom stays in the title row). `display:contents` makes the wrapper
  div transparent to the desktop flex container.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 22:18:28 +00:00
claude 51d0314c1c fix(mobile): detect touch devices with spoofed viewport width
Build and Deploy Verso / deploy (push) Successful in 10m17s
Tor Browser and Firefox with privacy.resistFingerprinting report a
desktop-sized viewport (~980px) even on a real Android phone. This made
window.matchMedia('(max-width: 767px)') return false, so isMobile was
always false on Tor, leaving the editor in side-by-side (horizontal)
layout instead of the expected vertical stack.

Fix: add a secondary check using `(pointer: coarse) and (max-width:
1024px)`. Touch hardware is not spoofed by fingerprinting resistance,
so this reliably catches phones and tablets regardless of the reported
viewport width. Applied to both getInitialLayout() and the live
isMobile state in main-layout.tsx.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 14:31:56 +00:00
claude 8fc71d677c Fix thumbnail quality and mobile vertical split layout
Build and Deploy Verso / deploy (push) Successful in 10m23s
Thumbnails: update the actual thumbnail endpoint (ConversionController.js
thumbnailFromBuild) to quality=90 and width=794. The previous fix targeted
ConversionManager.js which handles preview mode, not the thumbnail route
called by ThumbnailManager.mjs.

Mobile layout: move the isMobile guard before the stored-preference check
in getInitialLayout(). The autoSave race fix (build 274) stopped future
bad writes, but a stale 'flat' in localStorage was still being read on
every load, blocking the mobile check. Mobile now always starts in
verticalSplit regardless of any stored value.

CI: add node --check on all server-side .mjs files in the Dockerfile,
after source copy and before webpack compile, so syntax errors like the
escaped-backtick incident fail the build immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 13:48:44 +00:00
claude 8a821bc91d Fix escaped backtick syntax error in EmailBuilder.mjs
Build and Deploy Verso / deploy (push) Successful in 12m49s
sed substitution literal-escaped the backtick and \${ in the
groupSSOReauthenticate subject line, producing invalid JS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 13:13:05 +00:00
claude 33fa5986c8 Email rebranding, mobile filter alignment fix, minor UI cleanup
Build and Deploy Verso / deploy (push) Has been cancelled
Email:
- Replace Overleaf green palette with Verso teal (#2a9d8f) for CTA
  buttons, links, and logo text
- Rename force-overleaf-style CSS class to force-verso-style in all
  email body templates and layout
- Replace all user-visible "Overleaf" strings with settings.appName
  across email subjects, bodies, and sign-offs (EmailBuilder.mjs)
- Remove Overleaf CEO signature from onboarding email; substitute
  "The Verso Team"
- Fix utm_source=overleaf → utm_source=verso in onboarding links
- Point git token docs URL to local siteUrl instead of docs.overleaf.com
- Fix hard-coded Overleaf green (#0F7A06) in SSO disabled email link

Mobile filter chips:
- Switch from overflow-x:auto (still leaks through ancestor flex chain)
  to flex-wrap:wrap with flex:1 1 auto on each chip so all chips in a
  row grow to equal width — no overflow, clean alignment
- Toolbar becomes flex-column: chips on top, zoom below-right

Misc:
- Remove import_typst_file option from new-project menu
- Add interface_language to extracted-translations.json whitelist

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 12:56:34 +00:00
claude f629f6a50c Mobile polish: max thumbnail quality, tab-bar filter chips, fix vertical split default
Build and Deploy Verso / deploy (push) Successful in 10m12s
- CLSI thumbnails: bump to 794px/q90 (matches preview quality) for
  crisp display on 2x/3x phone screens
- Lumière filter chips → underline tab bar: single scrollable row with
  teal active indicator, no more wrapping alignment issues; zoom buttons
  separated by a vertical divider on the right
- Fix editor vertical split default on mobile: disable react-resizable-panels
  autoSaveId on mobile to prevent a stale collapsed PDF pane from firing
  onCollapse → changeLayout('flat') and overriding the verticalSplit
  default set by getInitialLayout()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 12:19:10 +00:00
claude c79ac23a15 Improve thumbnail quality and fix mobile editor default layout
Build and Deploy Verso / deploy (push) Successful in 10m18s
Thumbnails: increase CLSI thumbnail from 190px/q50 to 400px/q80.
At 190px/50% JPEG quality, images are noticeably blurry on 2x phone
screens (source needs 380px device pixels but source is only 190px).

Editor mobile layout: getInitialLayout() was returning sideBySide for
any stored 'split' preference (set from a desktop session), even on
mobile. sideBySide on mobile renders vertically via the isMobile check
in main-layout, but the stated default was still wrong. Now on mobile,
any stored value other than 'flat' maps to verticalSplit so the
top-bottom split is always the default; flat is preserved so a user
who explicitly chose editor-only keeps that preference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 11:44:34 +00:00
claude 5aab716245 lumiere mobile: wrap filter pills instead of horizontal scroll
Build and Deploy Verso / deploy (push) Successful in 12m8s
overflow-x: auto only clips content when every ancestor has a definite
width; that constraint wasn't met so the pills were still expanding the
page. Switch to flex-wrap: wrap so pills flow onto multiple rows and
never cause horizontal overflow. Also allow the toolbar itself to wrap
so the zoom control can drop to a new row if needed.

Remove redundant sidebar hide rule — SidebarDsNav already has d-none
d-md-flex so it was never contributing to the overflow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 11:07:16 +00:00
claude 90df7f5cc2 lumiere mobile: hide sidebar to fix 2x page width
Build and Deploy Verso / deploy (push) Successful in 11m15s
The project-list-wrapper flex row includes the sidebar (min-width:200px)
alongside the content area. On a 375px phone this totals ~575px, causing
the 2x horizontal overflow. Hide the sidebar on mobile — the filter pills
in the mobile toolbar already cover all its navigation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 10:50:37 +00:00
claude d12a39329b lumiere mobile: fix overflow, style ⋮ button, show tile actions on touch
Build and Deploy Verso / deploy (push) Successful in 16m58s
1. Page width overflow: stack lumiere-header as column on mobile so
   lumiere-header-actions (flex-shrink:0) no longer forces the container
   wider than the viewport. Search form and new-project button each take
   full width on their own line.

2. ⋮ button in compact row: override dropdown-table-button-toggle inside
   lumiere-compact-actions with Lumière-palette colours and tighter
   padding, replacing the table-context styling.

3. Tile view action strip: add @media (hover: none) rule so
   lumiere-card-actions is always opacity:1 on touch devices (phones have
   no cursor, so the hover-reveal pattern is unusable).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 10:00:50 +00:00
claude 6f419477df lumiere: fix mobile overflow + restore tile view with XS/M/L zoom
Build and Deploy Verso / deploy (push) Successful in 1m42s
Overflow fix: ActionsCell (up to 9 icon buttons) was assigned to the
`auto` column in the compact row, blowing out the row to ~300px.
Replace with ActionsDropdown (single ⋮ button) on mobile via
d-md-none / d-none d-md-flex, same pattern as the classic table.

Tile view: remove the isMobile force to compact (effectiveScale=0).
Add a mobile-only toolbar row (d-flex d-md-none) combining the filter
pills and a new XS/M/L zoom control:
  - XS (scale=0)  → compact rows
  - M  (scale=1)  → 2 tiles per row  (CSS: repeat(2, 1fr))
  - L  (scale≥1.35) → 1 tile per row (CSS: 1fr, class --mobile-1col)

The --lum-card-scale inline var is suppressed on mobile so CSS media
query controls thumbnail height (0.85 for 2-col, 1.3 for 1-col).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 08:54:30 +00:00
claude a004f21688 Fix mobile ergonomics on Lumière project list page
Build and Deploy Verso / deploy (push) Successful in 14m58s
Compact row (xs): wrap format/owner/date in a lumiere-compact-meta div
that uses display:contents on desktop (preserving the 6-column grid) and
switches to a 2-row grid layout on mobile — row 1: checkbox/name/actions,
row 2: format·owner·date. Actions are always visible on touch devices.

Filter access: add a horizontal scrollable filter-pill bar (d-md-none)
to the Lumière header so users can switch between All/Yours/Shared/Archived/
Trashed on phone without needing the desktop sidebar.

Also: hide zoom control on mobile (d-md-none), auto-force compact view on
mobile via useIsMobile hook, fix search form min-width overflow on xs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 08:22:27 +00:00
claude 065534819c Fix file tree refresh after convert and compiler sync on set-as-main
Build and Deploy Verso / deploy (push) Successful in 14m4s
- Convert: backend now returns parentFolderId+isNew; frontend calls
  dispatchCreateDoc directly so the new file appears without a page refresh
- Set as main: infer compiler from file extension and POST both rootDocId
  and compiler together; updateProject propagates the change to the
  editor settings dropdown and project list immediately

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 07:50:02 +00:00
claude b0b389dc4c feat: set-as-main for .typ/.qmd; fix compiler filter; fix ZIP import compiler
Build and Deploy Verso / deploy (push) Successful in 14m56s
Set as main document (context menu):
- canSetRootDocId used selectedEntityIds so .typ/.qmd files couldn't be
  set as main via right-click on an unselected file.
  Now computed locally from contextMenuEntityId (same pattern as convert)
  using isValidTeXFile which already covers .typ, .qmd, .tex etc.

Compiler filter (editor settings):
- docs?.find(id) could return undefined due to ID format mismatch,
  causing all engines to show as available for non-LaTeX projects.
  Added findInTree fallback so the root doc name is always resolved.

ZIP import compiler:
- Projects created from ZIP always got defaultLatexCompiler ('quarto')
  regardless of content.
- findRootDocFileFromDirectory now also searches for .typ and .qmd root
  files after finding no .tex file.
- ProjectUploadManager now infers the compiler from the root doc
  extension and sets it on the project after import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 21:56:39 +00:00
claude bc131f6440 fix: convert items show for unselected files; add success toast
Build and Deploy Verso / deploy (push) Successful in 15m23s
canRename required selectedEntityIds.size === 1, so right-clicking an
unselected file hid the convert items even though contextMenuEntityId
was correctly set. Replace canRename with !fileTreeReadOnly for the
convert-specific gate, which is the actual write-access check needed.

Also add showExportDocumentSuccess so the user sees the warning toast
on successful conversion instead of silent nothing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 21:18:24 +00:00
claude ff7de70a61 fix: per-file convert — DuplicateNameError + first-click no-op
Build and Deploy Verso / deploy (push) Successful in 21m44s
Two bugs:
1. Converting when output already exists threw DuplicateNameError (400).
   Now overwrites existing doc via setDocument instead of failing.

2. Right-clicking an unselected file left contextMenuEntityId null,
   so the first click on Convert silently did nothing. Added
   contextMenuEntityId to FileTreeMainContext, set it on right-click
   and on the … button click; FileTreeItemMenuItems now uses it for
   the convert hooks rather than relying on selectedEntityIds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 20:35:50 +00:00
claude dbb7899ca3 docs: clean up Alpha 3 section, add Known Issues
Build and Deploy Verso / deploy (push) Successful in 15m23s
Remove RevealJS thumbnails (redundant with Features section) and Upload
reliability (still unresolved). Add Known Issues section documenting
the large file upload timeout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 19:58:52 +00:00
claude 7eeabc93aa fix: send empty JSON body on convert to avoid body-parser 400
postJSON without options sends Content-Type: application/json with no
body; body-parser's BodyParserWrapper intercepts the resulting
SyntaxError and returns 400 before the route handler runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 19:58:48 +00:00
claude ac2315bc8e feat: per-file Convert in explorer menu + fix export success toast
Build and Deploy Verso / deploy (push) Successful in 9m50s
- Add "Convert to Typst (.typ)" in the file tree context menu for .tex
  docs, and "Convert to LaTeX (.tex)" for .typ docs. Clicking runs pandoc
  on the file content and creates the converted file in the same folder.
- New backend endpoint POST /project/:id/doc/:id/convert/:type that reads
  the doc from document-updater, runs pandoc directly, and creates the
  result via ProjectEntityUpdateHandler (file tree updates via socket).
- Rewrite the export success toast for typst and latex conversions: no
  more link to /contact, replaced with a plain warning that errors are
  expected (pandoc does not support all constructs).
- Add i18n keys: convert_to_typst, convert_to_latex,
  typst_export_feedback_message, latex_export_feedback_message (EN + FR)
  and all four to extracted-translations.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 15:18:31 +00:00
claude de22dcf87f docs: add mobile layout to Alpha 3 release notes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 14:58:28 +00:00
claude e82f5fbead docs: update Alpha 3 release notes and fix Overleaf description
Add top/bottom split view and bidirectional format export (LaTeX↔Typst,
LaTeX→DOCX/Markdown/HTML) to the Alpha 3 feature list. Fix the security
section which incorrectly referred to Overleaf as a LaTeX/Typst editor —
Overleaf only supports LaTeX.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 14:56:45 +00:00
claude d64365aa56 Add missing i18n keys to extracted-translations.json
Build and Deploy Verso / deploy (push) Successful in 14m28s
top_bottom_split_view, export_as_typst, and export_as_latex were absent
from the whitelist, so translations-loader.js stripped them from the
locale bundles at build time — causing raw keys to show in the UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 14:46:31 +00:00
claude 2d2a85f06f Fix typst export 500, add export-as-latex for typst projects
Build and Deploy Verso / deploy (push) Successful in 10m4s
- clsi-nginx: allow hyphens in project-id regex — conversion IDs are UUIDs
  which nginx was rejecting, causing 500 on file download after conversion
- CLSI ConversionController/Manager: add 'latex' export type (typst→latex via pandoc)
- Web: add 'latex' to SUPPORTED_CONVERSION_TYPES
- Frontend: add Export as LaTeX button (visible only for typst projects)
- Fix visibility logic: export-as-latex shows for typst, export-as-typst shows for latex
- Add export_as_latex translation key (en + fr)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 11:55:41 +00:00
claude b545d08939 Fix four reported bugs: typst export 500, export shown for all project types, Lumiere tile button layout, i18n
Build and Deploy Verso / deploy (push) Successful in 15m7s
- ConversionController.js: add typst to CONVERSION_CONFIGS (missing entry caused 400→500 chain)
- export-project-with-conversion-button: hide button for non-LaTeX projects (typst/quarto) via compiler check
- project-list-lumiere.tsx + scss: revert lumiere-card-actions back inside .lumiere-card (put them back like they were)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 11:12:50 +00:00
claude 9d11683920 feat: Typst → LaTeX import, and fix export button visibility
Build and Deploy Verso / deploy (push) Successful in 1m14s
Typst → LaTeX import:
- CLSI ConversionManager: add 'typst' to CONVERSION_CONFIGS
  (pandoc input.typ --from typst --to latex --standalone → zip archive)
- Web controller: allow 'typst' as a valid importDocument conversion type
- Frontend modal: add .typ file config to ImportDocumentModal
- New project button modal: add 'import_typst' variant + switch case
- New project button: show "Import Typst file" when enablePandocConversions
  is true (no split test gate — Verso has no SaaS split test infra)
- Locales: add choose_typst_file and import_typst_file keys (18 locales)

Export button fix:
- Remove featureFlag="export-typst" from ExportProjectWithConversionButton
  so the button shows whenever enablePandocConversions is true, without
  needing an unconfigured split test to return 'enabled'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 08:54:54 +00:00
claude 15ffaefb87 fix: install pandoc and enable pandoc conversions
Build and Deploy Verso / deploy (push) Failing after 1h2m54s
Pandoc was not installed in the base image, so the export buttons
(docx, markdown, html, typst) were hidden because ENABLE_PANDOC_CONVERSIONS
defaulted to false.

- Dockerfile-base: add pandoc via apt (Ubuntu Noble ships 3.1.3, which
  supports --to typst added in pandoc 3.0)
- env.sh: set ENABLE_PANDOC_CONVERSIONS=true so both the web and CLSI
  services expose and serve the export endpoints

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 08:46:09 +00:00
claude 32aec14c41 feat: export LaTeX project as Typst (.typ) via pandoc
Build and Deploy Verso / deploy (push) Successful in 9m58s
Adds a new "Export as Typst" option in the project title dropdown and
File menu, mirroring the existing docx/markdown/html export pipeline.

Changes:
- CLSI ConversionManager: add 'typst' to LATEX_EXPORT_CONFIGS
  (compressOutput: false, pandoc --from latex --to typst)
- Web controller: register 'typst' → 'typ' in SUPPORTED_CONVERSION_TYPES
- Frontend: extend conversionType union and add ExportProjectWithConversionButton
- File menu: add 'export-as-typst' to the download group command structure
- Locales: add export_as_typst key to all 18 locale files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 08:24:54 +00:00
claude f7d0369213 fix: translate hardcoded 'Editor settings' header in View menu
Build and Deploy Verso / deploy (push) Successful in 9m45s
The DropdownHeader in the View > Layout menu was using a raw English
string instead of t('editor_settings'). Added the key to all 18 locale
files with appropriate translations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 08:04:56 +00:00
claude 57e34aa104 fix: dropdown positioned at top-left on first click
Build and Deploy Verso / deploy (push) Successful in 15m17s
Popper lazy-initializes on first open, causing it to place the menu at
[0,0] before it has computed the toggle's position. renderOnMount forces
Popper to initialize while the component is first mounted, so the
position is ready before the user's first click.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 22:05:01 +00:00
claude 48d5e15f7f fix: Lumiere card dropdown clipped by card's backdrop-filter and transform
Build and Deploy Verso / deploy (push) Successful in 14m35s
.lumiere-card has both backdrop-filter and a hover transform, both of
which create a new containing block for position:fixed descendants,
trapping Popper dropdowns inside the card's overflow:hidden bounds.

Fix: move .lumiere-card-actions outside .lumiere-card (sibling inside
.lumiere-card-wrapper, which has position:relative but no filter or
transform). The actions strip is now absolutely positioned at the card
bottom with its own solid background and bottom border-radius.
No backdrop-filter on the strip to avoid re-introducing the same trap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 21:35:52 +00:00
claude b78845a926 fix: show presentation export dropdown on Lumiere tiles for Quarto projects
Build and Deploy Verso / deploy (push) Successful in 14m25s
ProjectCard in the Lumiere tile view always showed CompileAndDownloadProjectPDFButtonTooltip,
ignoring the project compiler. Quarto presentation projects need the
DownloadPresentationButtonTooltip (HTML/PDF dropdown) instead, matching
the logic already in actions-cell.tsx and actions-dropdown.tsx.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 21:08:43 +00:00
claude c10a13f605 fix: PC layout regression and silent PDF download failure
Build and Deploy Verso / deploy (push) Successful in 13m21s
layout-context: getInitialLayout() was returning verticalSplit for
any stored 'vertical' preference, including on desktop. Now checks
isMobile first so stored mobile preference doesn't bleed into PC.

compile-and-download-pdf: when compile succeeds but output.pdf is
absent from outputFiles, the code crashed silently at outputFile.build
leaving the user with no feedback. Now shows the error modal instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 20:42:22 +00:00
claude 762d3e75cf fix: footer translation, mobile search bar and table row layout
Build and Deploy Verso / deploy (push) Successful in 13m32s
- Footer: use translate('built_on') key instead of hardcoded 'Built on';
  update fr.json 'built_on' → 'Basé sur Overleaf'
- Mobile project list: move search bar + button outside the bordered
  TableContainer so its width is viewport-constrained, not affected by
  the table's fixed layout
- Mobile table rows: use width:100% (not auto) on cells so they fill the
  full tr width regardless of the higher-specificity column percentage
  rules; add explicit width:100% on tr to anchor flex-column sizing;
  keep width:auto on absolutely-positioned actions cell

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 15:31:09 +00:00
claude db5b2b1f82 fix: show presentation download dropdown for all Quarto projects
quartoFlavor is set only after a project is compiled with the current
build. Existing Quarto projects that haven't been recompiled have
quartoFlavor=undefined, so the old check (quartoFlavor === 'revealjs')
fell through to the regular PDF compile button with no dropdown.

Drop the quartoFlavor guard — compiler === 'quarto' is sufficient since
all Quarto projects in Verso are RevealJS presentations. Changes applied
to ActionsCell (desktop icon), DownloadPresentationButtonTooltip guard,
and ActionsDropdown (mobile three-dot menu).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:53:33 +00:00
claude 685a7ffca1 fix: language picker links, dropdown position, and editor layout defaults
Build and Deploy Verso / deploy (push) Successful in 14m34s
Language picker:
- Add fallback href in Pug so language links navigate even if JS fails
- Anchor dropdown to right edge (right:0) so it stays on-screen when
  the picker is near the right side of the footer on mobile

Editor layout:
- Read stored pdfLayout from localStorage on init so the last-used
  layout is remembered across sessions
- Default to verticalSplit (top/bottom) on mobile when no preference
  is stored, so the editor opens in a sensible layout on phones

Translations:
- Add top_bottom_split_view key to all 16 locales that were missing it

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:40:02 +00:00
claude 84d1efc271 fix: use <details>/<summary> for Pug language picker to get native toggle
Build and Deploy Verso / deploy (push) Successful in 14m10s
The manual JS click-handler approach (tried with stopPropagation,
containment check, and mousedown variants) never worked reliably on
login/password/settings pages. The browser's native <details>/<summary>
toggle behaviour requires no JavaScript and is immune to Bootstrap JS or
React event delegation interference.

The inline script now only builds the return_to hrefs and handles
outside-click-to-close (setting details.open=false). The CSS gains a
.language-picker-details rule that sets position:relative so the
absolutely-positioned dropdown-menu is positioned correctly, and
details[open] .dropdown-menu { display: block } to show the menu.

The #language-picker-toggle id remains on the <summary> so the existing
CSS (cursor, text-decoration, material-symbols alignment) continues to
apply. The React LanguagePicker is unaffected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 12:55:39 +00:00
claude b461343b23 fix: project list phone layout and language picker
Build and Deploy Verso / deploy (push) Successful in 14m3s
- Search bar overflow: min-width:0 on form prevents input min-content
  from overflowing flex container on narrow screens
- Remove duplicate New Project button from header row (tableTopArea
  already provides it for mobile)
- Simplify tableTopArea: single button+search layout regardless of
  isLibraryEnabled flag
- 2-line project rows on mobile: merge dash-cell-date-owner +
  dash-cell-tag into a single dash-cell-meta so date/owner/tags flow
  inline on line 2 below the project name
- Footer mobile: hide language-picker-text on mobile (icon only) to
  prevent 3rd line in 2-row footer
- Language picker: use mousedown instead of click for outside-close
  handler — click bubbling is unreliable on iOS for non-interactive
  elements; mousedown fires for all taps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 11:47:32 +00:00
claude be353d53bd Center footer items on mobile for harmonious two-line layout
Build and Deploy Verso / deploy (push) Successful in 9m44s
Both ul.site-footer-items rows are now justify-content: center on small
screens, so the copyright/language row and the licence/source row are
symmetrically aligned instead of left-justified.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 11:05:22 +00:00
claude 0a5bd4e47d Fix mobile layout key, language picker close handler, and presentation download
Build and Deploy Verso / deploy (push) Has been cancelled
- Mobile vertical layout: add key= based on direction so react-resizable-panels
  remounts cleanly when switching between horizontal and vertical; also use
  isVertical (not just pdfLayout) for the autoSaveId to avoid restoring a
  mismatched layout from localStorage
- Language picker: replace stopPropagation pattern with a containment check on
  the document click handler — more robust on React pages where Bootstrap JS or
  React's event delegation can interfere with stopPropagation
- Presentation download dropdown: use popperConfig strategy:'fixed' so the menu
  escapes overflow:hidden table cells; remove forced drop='up' and let Popper
  choose; defer URL.revokeObjectURL by 10 s to give the browser time to start
  the download before the blob URL is released

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 11:01:15 +00:00
claude 11227d59e3 Editor mobile ergonomics: vertical split layout on phones and as desktop option
Build and Deploy Verso / deploy (push) Successful in 14m3s
On mobile (< 768 px) the existing side-by-side layout automatically switches
to a vertical stack (editor on top, PDF/presentation on bottom) without
changing the stored layout preference.

A new "Top / bottom split" option is added to the layout menu so desktop
users can choose the same vertical split explicitly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 09:25:15 +00:00
claude 0f585ea5bb Fix: use picture_as_pdf icon for presentation download toggle
Using the generic 'download' icon duplicated the ZIP button icon,
giving two identical icons side by side. Switch to picture_as_pdf to
match the previous compile-and-download button's appearance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 08:51:29 +00:00
claude a6ea291ea8 Add PDF/HTML download dropdown for Quarto Slides in project list
Quarto Slides (RevealJS) projects compile to output.html, not output.pdf,
so the existing "Download PDF" button was meaningless for them. Replace it
with a two-option dropdown matching the editor's PdfHybridDownloadButton:

- Desktop (ActionsCell): icon button opens a dropup with
  "Download standalone HTML" and "Download PDF slides"
- Mobile (ActionsDropdown): two separate dropdown items with the same
  choices and per-format spinner while the export is in progress

Both use the same /project/:id/presentation-export/:format endpoint and
show a loading modal (with error reporting) during the server-side render,
exactly as the editor toolbar does.

Non-RevealJS projects continue to show the compile-and-download-PDF button
unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 08:48:03 +00:00
claude 703f4d6ee2 Replace native <select> language picker with styled dropdown
Build and Deploy Verso / deploy (push) Successful in 13m43s
The native <select> looked like a form control ("cheap"). Replace it with
the same HTML+CSS pattern as the React LanguagePicker: btn-inline-link
toggle with a translate icon, Bootstrap dropdown-menu for the list, active
item highlighted in green.

A tiny self-contained inline script handles the toggle since Bootstrap JS
is not loaded on React-layout pages. It builds the return_to URL from
window.location.pathname at click time (same as the old select approach).

Added top: auto / bottom: 100% to .language-picker .dropdown-menu so the
list always opens upward regardless of whether Popper.js is present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 08:29:10 +00:00
claude 86a902c197 Improve mobile ergonomics for footer and project list
Footer (both React and Pug):
- Below md: switch to flex-wrap so items wrap naturally instead of
  overflowing; reset line-height from the fixed 49px to normal; add
  row-gap so wrapped rows aren't crammed together
- Add footer-sep class to pipe separator <li>s so they can be hidden
  on small screens (wrapping mid-separator looks wrong)
- Change col-lg-3 text-end → text-lg-end so the right column (licence,
  source code) aligns left when it stacks full-width below lg

Project list:
- Show NewProjectButton on mobile in the header row (the sidebar that
  holds the button is already hidden below md via d-none d-md-flex,
  leaving users with no way to create projects on their phone)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 08:22:57 +00:00
claude 36f5e9fb06 Remove rounded top-left corner from project list content area
The content panel had border-top-left-radius: var(--border-radius-large)
at md+ breakpoint, left over from the old Overleaf theme. Everything else
is square now so this corner stood out. Removing it makes the edge flush.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 07:58:33 +00:00
claude 2b9ebe5522 Replace Overleaf O with Verso icon mark in editor state favicons
Build and Deploy Verso / deploy (push) Successful in 10m0s
favicon-compiled.svg, favicon-compiling.svg, favicon-error.svg all used
the Overleaf O logo as the base. Replace with the Verso icon mark (dark
background, four colored circles, white V polygon, clipped to a circle)
while keeping the existing status badge icons (✓ / ↻ / ✗) unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 07:54:33 +00:00
claude e4f1eead25 Branding polish: Verso favicons, OG image, and meta tag fixes
Build and Deploy Verso / deploy (push) Successful in 10m24s
Replace Overleaf favicons with Verso icon mark across all sizes (16x16,
32x32, 180x180 apple-touch, 192x192/512x512 android-chrome, ICO). Add
OG social preview image (1200x630) for Discord/Twitter link cards.

- New favicon.svg: Verso icon mark with four overlapping circles and V
- mask-favicon.svg: monochrome V polygon (was Overleaf chevron)
- og-image.png: 1200x630 social card with icon mark and "Verso" wordmark
- web.sitemanifest.json: rename "Overleaf" → "Verso", theme_color updated
- _metadata.pug: add og:url tag, fix og:type missing content= attribute,
  point CE default OG/twitter images to og-image.png instead of apple-touch-icon.png

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 20:19:01 +00:00
claude 323d74cc66 fix: replace Pug language picker dropdown with native select
Build and Deploy Verso / deploy (push) Successful in 14m31s
The old dropdown relied on data-bs-toggle and AngularJS directives,
neither of which are loaded on React-layout pages (layout-react.pug
intentionally excludes Bootstrap JS). The toggle button was inert on
pages like /user/settings.

Replace with a plain <select> that navigates via window.location.href
onchange — works without any framework on all page types.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 18:46:03 +00:00
claude 81c1fc92d1 fix: remove data-bs-toggle from LanguagePicker dropdown toggle
Build and Deploy Verso / deploy (push) Successful in 10m8s
Bootstrap vanilla JS and react-bootstrap were both handling the dropdown,
causing the toggle to be unresponsive. react-bootstrap manages its own
state and does not need this attribute.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 14:15:36 +00:00
claude 319ccc32ee feat: add language picker to logged-in footer and editor settings
Build and Deploy Verso / deploy (push) Successful in 18m46s
- Rewrites LanguagePicker to use availableLanguages from ol-footer meta
  instead of subdomainLang (which is always empty in single-domain setup)
- Passes availableLanguages through layout-react.pug → ol-footer meta so
  React footer picks it up
- Adds InterfaceLanguageSetting component to the editor settings modal
  ("Spelling and language" tab) for use when no footer is present
- Adds interface_language key to all five locale files (en/fr/de/es/it)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 12:22:55 +00:00
claude 1a0197812d feat: cookie+DB language preference, eliminating subdomain requirement
Build and Deploy Verso / deploy (push) Successful in 14m53s
Users can now select their UI language directly without relying on
subdomain routing (fr.verso.alocoq.fr etc.).

Resolution order: (1) verso-lang cookie, (2) subdomain host header,
(3) OVERLEAF_SITE_LANGUAGE default — fully backward compatible.

Changes:
- Translations.mjs: read verso-lang cookie in middleware; include all
  bundled locale files in availableLanguageCodes regardless of subdomain
  config so every loaded locale appears in the picker
- User.mjs: add languageCode field to persist preference per user
- UserController.mjs: setLanguage handler — sets cookie (1 year) and
  writes languageCode to DB when called by a logged-in user
- AuthenticationController.mjs: on login, sync DB languageCode to cookie
  so preference follows the user to any new browser/device after login
- ExpressLocals.mjs: expose availableLanguages to all Pug templates
- router.mjs: GET /set-language?lng=<code> (anonymous + logged-in),
  POST /user/language (logged-in, REST-style)
- language-picker.pug: replace subdomain href links with /set-language
  redirect links; iterate availableLanguages instead of subdomainLang
- thin-footer.pug: show picker whenever availableLanguages.length > 1,
  not only when multiple subdomains are configured

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:44:56 +00:00
claude 589c60a325 i18n(de/es/it): correct machine-translation errors in German, Spanish, Italian
Build and Deploy Verso / deploy (push) Successful in 14m22s
German (43 fixes): table→Tabelle (was Tisch/furniture), figure→Abbildung (was Figur),
no_borders→Keine Rahmen (was Grenzen/national borders), run→Ausführen (was Lauf),
right→Rechts (was Richtig/correct), unpausing was "Ununterbrochen" (uninterrupted),
unlink_provider said "verknüpfen" (link) instead of "trennen" (unlink),
unpause_subscription said "pausieren" (pause) instead of "fortsetzen",
zotero_reference_loading_error mentioned Mendeley, light_themes→Helle Themen,
review_panel→Überprüfungsbereich (was Gremium/committee), x_price_per_month untranslated,
Writefull/Papers/booktabs/Labs product names preserved

Spanish (58 fixes): no_borders→Sin bordes (was fronteras/national borders),
invite_resent→Invitación reenviada (was "Invita a resentirse"),
trash→Papelera, trashed→En papelera (was "Destrozado"/destroyed),
plan→Plan (was Planificar/verb), premium→Premium (was prima/bonus),
disabled→Desactivado (was Discapacitado/handicapped), breadcrumbs navigation fixed,
cookie_banner→Banner de cookies, search_match_case fixed, Writefull/Papers/Dropbox/Git preserved

Italian (85 fixes): no_borders→Nessun bordo (was confini/national borders),
right→Destra (was Giusto/correct), run→Esegui (was Corri/run physically),
standard→Standard (was Norma), premium→Premium (was Premio/prize),
characters→Caratteri (was Personaggi), editor→Editor (was Editore/publisher) throughout,
light_themes→Temi chiari (was leggeri/lightweight), unpausing→Riattivazione,
back_to_x buttons fixed from "Torniamo" (we return) to imperative "Torna",
~20 infinitives corrected to imperatives for button labels,
booktabs/Writefull/Papers product names preserved, triple-r typo fixed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:30:04 +00:00
claude 38144b1033 i18n(fr): correct 41 machine-translation errors in French locale
Fix a batch of severe mistranslations produced by automated translation:
- typographic terms: bold→Gras (was "Audacieux"), figure→Figure (was "Chiffre"),
  characters→Caractères (was "Personnages"), borders→bordures (was "frontières")
- UI action labels: apply→Appliquer (was "Postuler"/job application),
  run→Exécuter (was "Courir"/to run physically), chat→Chat (was "Discuter"/verb),
  code→Code (was "Coder"/verb), more_actions→Plus d'actions (was "propositions")
- subscription plan terms: plan→Forfait (was "Planifier"/verb),
  premium→Premium (was "Prime"), standard→Standard (was "Norme")
- role/position: role→Rôle, position→Poste (both were "Grade"/military rank)
- light mode: mode lumière→mode clair throughout
- upload direction: Télécharger→Téléverser (download vs upload)
- invite_resent: catastrophic mis-parse of "resent" as resentment
- search_match_case: "Étui de correspondance" (phone case)→"Respecter la casse"
- product names preserved: booktabs, Writefull, GPT (were translated literally)
- typo: "toruvé"→"trouvé", missing spaces: "viaGit"→"via Git", "SSOconfiguration"→"Configuration SSO"
- untranslated: x_price_per_month was left in English
- accent: "Suedois"→"Suédois", "Koréen"→"Coréen", "Turque"→"Turc"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 11:17:32 +00:00
claude 00ccb9748e docs: red CAUTION block for security warning, fix hallucinated Overleaf/Typst claim
Use GitHub's > [!CAUTION] admonition (renders with red background) for the
trusted-environment security warning, matching the style used by Collabst.

Remove invented claim that Overleaf is working on Typst support — that was
a hallucination. Replace with a plain "Verso is built on Overleaf's infra"
statement. Add RevealJS as a separate ecosystem project worth supporting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 09:31:35 +00:00
claude a16ad0b977 docs: no contributions/donations, add security model + Collabst + ecosystem support
Build and Deploy Verso / deploy (push) Successful in 15m37s
- Remove Contributing section (not accepting PRs/issues)
- Add Security model section: Verso is for trusted environments only;
  point untrusted-user use cases at Overleaf non-Community offerings
- Mention Collabst as a promising open-source Typst-only alternative
  in the Verso vs Typst.app comparison
- Add Supporting the ecosystem section redirecting to Overleaf (Typst +
  RevealJS work) and the Typst project instead of Verso donations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 09:22:34 +00:00
claude c2a21da47c docs: rewrite README — positioning vs Overleaf/Quarto/Typst.app, add releases section
Build and Deploy Verso / deploy (push) Has been cancelled
Replace the "how to write a .qmd / .typ / .tex" tutorial content with a
clear positioning narrative: what Verso is vs Overleaf (same engine + more
languages), vs Quarto CLI (browser-based, collaborative, multi-language),
and vs Typst.app (self-hosted, AGPL, OT-backed, three languages).

Add a Releases section covering Alpha 1 (core multi-compiler foundation),
Alpha 2 (Typst grammar overhaul, format sub-types, Python for collaborators),
and Alpha 3 in-progress (Lumière, i18n, visual editors, upload fix).

Keep Quick start, Architecture, Env vars, and License sections intact.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 09:11:02 +00:00
claude 94d3764c05 fix: stream HTTP 200 heartbeat before upload body to prevent proxy timeouts
Build and Deploy Verso / deploy (push) Has been cancelled
Uploads from slow connections consistently fail with 502 after ~60-120s
because an upstream proxy (Traefik or cloud load-balancer) has a
"first response byte" deadline that fires before the request body arrives.

Fix: add startStreamingResponse middleware (after auth, before multer)
that immediately writes HTTP 200 + Transfer-Encoding: chunked + '\n'.
With proxy_request_buffering off in Nginx, this reaches the proxy at T≈0,
so no timeout triggers. The upload body continues streaming; multer writes
to disk; the actual JSON result arrives as the final chunk. Periodic
heartbeat '\n' writes every 30s keep response-idle timeouts at bay too.

Client-side: override Uppy's getResponseData/validateStatus to trim
leading whitespace before JSON.parse so the extra '\n' bytes are ignored.
Server-side: sendUploadResponse() helper handles both streaming mode
(res.headersSent → res.end(json)) and normal mode (res.status(N).json()).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 08:59:02 +00:00
claude 8f372d13f8 fix: Lumière layout/navbar on settings+404, drop proxy_request_buffering
Build and Deploy Verso / deploy (push) Successful in 14m28s
- Move min-height:100vh from body to html:has(body[data-lumiere]) so the
  gradient fills the viewport on short pages without inflating document
  height or pushing the footer below the fold
- Remove min-height:60vh from .error-container (was causing scrollbar on
  404 when combined with thin footer)
- Replace Bootstrap 3 navbar selectors (.navbar-nav > li > a) with CSS
  custom property overrides (--navbar-link-color, --navbar-link-hover-*,
  --navbar-bg, etc.) consumed by navbar.scss — fixes header button colours
- Remove position:relative from .navbar-default override; base CSS already
  has position:absolute which provides the stacking context for ::before
- Drop proxy_request_buffering off from upload location: buffered mode +
  global client_body_timeout 15m (nginx.conf.template) is more compatible
  with multer's multipart stream handling on slow connections

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 20:38:00 +00:00
claude 211ca9c46d Fix Lumière theming + upload timeout via global middleware
Build and Deploy Verso / deploy (push) Successful in 14m26s
Theming: replace per-controller isLumiere lookups with a single
ExpressLocals middleware that sets res.locals.isLumiere for every
web request. Uses getOverallTheme() (now exported from
UserSettingsHelper) so the date-based default is handled correctly.
This covers 404, settings, setPassword, activate, and all future
server-rendered pages automatically.

Upload timeout: add client_body_timeout 15m to nginx.conf.template
at the http level (was defaulting to 60s globally). This is more
reliable than the location-specific override from build 229.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 19:33:53 +00:00
claude 2ec6ca827e Fix upload timeout + apply Lumière to settings/auth pages
Build and Deploy Verso / deploy (push) Successful in 15m0s
Nginx: add dedicated upload location with client_body_timeout 15m,
client_max_body_size 550m, and proxy_request_buffering off. Default
client_body_timeout of 60s was the actual culprit cutting slow uploads.
Node.js requestTimeout (build 228) remains as a backstop.

Lumière: pass isLumiere from UserPagesController (settings),
PasswordResetController (set-password), and UserActivateController
(first-time activation). auth.scss adds card styling for auth pages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 17:47:03 +00:00
claude 41d38a70ed Increase upload timeout to 15 min for slow connections
Build and Deploy Verso / deploy (push) Successful in 15m39s
Frontend fetch gets AbortSignal.timeout(15 min) so hung connections
fail cleanly. Server requestTimeout raised from Node default (5 min)
to match, preventing large-file uploads from being cut off server-side.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 16:58:10 +00:00
claude b8d5cb9816 fix: XS badge column, lumiere on welcome and 404 pages
Build and Deploy Verso / deploy (push) Successful in 14m39s
- XS compact row: format column 70px→96px so "QUARTO SLIDES" stays on one line;
  trim owner/date cols slightly to compensate
- Welcome page (0 projects): Lumière branch now renders before the 0-projects
  check; ProjectListLumiere renders WelcomePageContent when totalProjectsCount=0
  so new users get the full onboarding experience in the Lumière shell
- 404 page: notFound() now detects the user's overallTheme and passes isLumiere
  to the template; layout-base.pug sets data-lumiere on the body; error-pages.scss
  and project-list-lumiere.scss add [data-lumiere='true'] rules for the body
  background gradient, navbar white+stripe, and styled error box

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 16:33:23 +00:00
claude c5883e5954 feat: generate first-slide thumbnail for Quarto RevealJS presentations
Build and Deploy Verso / deploy (push) Successful in 14m10s
thumbnailFromBuild() now tries output.pdf → output-slides.pdf → decktape
on output.html (slide 1 only). The web service's ThumbnailManager already
calls this endpoint fire-and-forget on every successful compile, so RevealJS
project cards will show the first slide thumbnail automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 16:02:54 +00:00
claude 1ddd219449 fix: nowrap on format badges, widen search bar to 360px
Build and Deploy Verso / deploy (push) Successful in 14m25s
Prevents "Quarto Slides" from wrapping to two lines in XS view.
Widens search input from 300px to 360px so French placeholder text fits.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 15:44:57 +00:00
claude be6034afcf fix: remove block comments containing closing delimiter causing babel parse errors
Build and Deploy Verso / deploy (push) Successful in 15m19s
JSDoc blocks in typst-decorations.ts and quarto-decorations.ts contained
*/ sequences inside them, which Babel's parser treats as terminating the
block comment prematurely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 14:55:35 +00:00
claude 37ed70c7e9 build 223: Quarto visual editor — bold, italic, headings, inline code, strikethrough
Build and Deploy Verso / deploy (push) Has been cancelled
Add quarto-decorations.ts ViewPlugin for .qmd/.md files in visual mode:
- ATXHeading1-6: hide # prefix, apply font-size per level (2em → 1em)
- StrongEmphasis: hide ** markers, apply font-weight:700
- Emphasis: hide * or _ markers, apply font-style:italic
- Strikethrough: hide ~~ markers, apply text-decoration:line-through
- InlineCode: hide backtick markers, apply monospace + subtle bg
Markers reappear when cursor enters the node (selectionSet trigger).
Uses getChildren('EmphasisMark'/'StrikethroughMark'/'CodeMark') to locate
delimiters generically, handling both single- and double-backtick inline code.

CSS rules (.ol-cm-md-{strong,emph,strikethrough,inline-code,heading,h1-h6})
added to visual-theme.ts. Plugin wired into visual.ts after typstDecorations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 13:27:34 +00:00
claude e4dc5f3f5d build 222: Typst visual editor — bold, italic, headings, inline code
Build and Deploy Verso / deploy (push) Has been cancelled
Add typst-decorations.ts ViewPlugin that runs alongside the existing
LaTeX visual decorations. For Typst files in visual mode it:
- Hides *…* markers and applies font-weight:700 to the StrongBody
- Hides _…_ markers and applies font-style:italic to the EmphBody
- Hides = prefix marks and applies heading CSS (h1–h6 font sizes)
- Hides backtick delimiters and applies monospace to RawInlineContent
Markers reappear when the cursor enters the node (selectionSet trigger).

Register CSS rules for .ol-cm-typst-{strong,emph,heading,h1-h6,raw-inline}
in visual-theme.ts. Wire the plugin into visual.ts after markDecorations.

The visual editor toggle was already available for .typ files since 'typ'
is in validRootDocExtensions — no gating change needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 13:10:09 +00:00
claude 0058cc17b5 build 221: fix panel bg cascade, center toggle pill, Lumière XS compact rows
Build and Deploy Verso / deploy (push) Successful in 15m1s
- ide-lumiere.scss: declare --file-tree-bg/--outline-bg-color/--outline-container-color-bg:
  transparent at [data-lumiere] .ide-redesign-main scope to beat file-tree.scss +
  outline.scss which re-declare the same vars at .ide-redesign-main (closer ancestor)
- linked-file-highlight and disconnected-overlay get explicit fallback colors so they
  remain usable when --file-tree-bg is transparent
- custom-toggler: replace left:-5px with left:50%/transform:translateX(-50%) for
  reliable centering of the 16px pill over the 4px resize handle
- project-list-lumiere.scss: compact rows get teal-tinted bg + teal border (Lumière feel)
- project-format-badge-* overrides inside .project-list-lumiere so the XS compact view
  shows the soft Lumière palette (matching .lumiere-format-badge--* on cards)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 12:33:18 +00:00
claude 0754d986e9 build 220: XS compact view reuses classic table cells for full detail
Build and Deploy Verso / deploy (push) Successful in 20m14s
Replace the CSS-only compact card hack with a dedicated
ProjectCardCompact component that reuses FormatCell, OwnerCell,
LastUpdatedCell (with tooltip + updated-by), ActionsCell (full set),
and InlineTags — identical data to the classic table view. CSS Grid
aligns columns across rows: checkbox | name+tags | format | owner |
date | actions. Actions fade in on row hover.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 11:47:00 +00:00
claude 6e230e250e build 219: wider toggle pill, transparent panel bg, shorter panel names
Build and Deploy Verso / deploy (push) Has been cancelled
Resize handle toggle: 14px wide (centered over 4px handle) so the
arrow icon is clearly visible. File tree and outline backgrounds set
to transparent so the panel-group noise+gradient shows through.
FR: file_tree→"Fichiers", file_outline→"Plan" (+ hide variants).
IT: file_tree→"File", IT/ES hide keys shortened to match.
DE unchanged (Dateibaum/Gliederung already concise).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 11:35:12 +00:00
claude 4f299c6204 build 221: wider search bar, XS compact list view
Build and Deploy Verso / deploy (push) Successful in 14m55s
Search bar: set min-width: 300px so the placeholder text is fully
visible. XS zoom level (value 0): adds lumiere-card-grid--compact class
which switches cards to horizontal rows, hides the thumbnail, and shows
only name / badge / owner / date / actions in a compact strip. Stored
0.75 from old S already fell back to 1 (S); new 0 is cleanly additive.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 11:06:49 +00:00
claude 261ca98103 build 220: remap tile zoom levels S/M/L
Old S (0.75×) removed — too small. Old M→S (1×), old L→M (1.35×),
new L added at 1.75×. Stored 0.75 in localStorage falls back to 1×.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 10:58:55 +00:00
claude 60f1e2c511 build 219: Lumière styling for file tree, outline, and panel dividers
Outline: override all CSS variables to use teal palette (bg, border,
hover, highlight, guide line) — was completely unstyled.
File tree: add teal inset left-accent bar on selected item.
Resize handles: narrow from 7→4px, remove dot-grip SVGs, teal hover
highlight. Collapse toggle: teal pill instead of grey.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 10:54:34 +00:00
claude f6bded1596 build 218: Verso logos on set-password and email-check pages, i18n zoom aria-label
Replace Overleaf icons with Verso branding on setPassword.pug
(square icon) and primaryEmailCheck.pug (wordmark). Replace hardcoded
French aria-label "Taille des cartes" on zoom control with t('card_size')
and add card_size key to all 5 locale files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 10:48:20 +00:00
claude 2f4b60574c build 217: square logo on password reset, align tile colours to classic badges
Build and Deploy Verso / deploy (push) Successful in 15m11s
Use verso-square.svg (small icon) instead of the wordmark on the
password reset page. Align card thumbnail gradients and format badge
colours to match the classic project list badge colours (LaTeX green
#098842, Typst teal #239dad, Quarto blue #447099, Quarto Slides
pink-red #e4637c). Split quarto-slides badge from quarto badge.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 10:38:05 +00:00
claude 29880d1cf9 build 216: Verso logo on password reset page, fix SSO/ES translations
Build and Deploy Verso / deploy (push) Successful in 15m31s
Replace Overleaf icon with Verso wordmark on passwordReset.pug.
Fix ES password_reset_email_sent (meaning error: said the password
was sent, not a link). Translate <0>Log in with SSO</0> link text
in FR/DE/IT/ES (was left in English in all languages).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 10:15:37 +00:00
claude 9de127f879 i18n: fix login button and related strings in FR/DE/IT/ES
Build and Deploy Verso / deploy (push) Successful in 19m6s
- FR login button: 'Identifiant' (username field label) → 'Connexion'
- ES forgot_your_password: missing closing '?' added
- DE logging_in: 'Anmeldung' (noun) → 'Anmelden' (verb, fits spinner)
- IT logging_in: 'Entrata in corso' → 'Accesso in corso'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 09:51:16 +00:00
claude ae4e95312d i18n: translate hardcoded strings on project page and footer
Build and Deploy Verso / deploy (push) Successful in 17m39s
- Card owner: "You" now goes through t('you') so it renders as
  "Vous"/"Tu"/"Du" etc. instead of always "You"
- Relative dates (fromNow): moment.locale() is now set from the app
  language so "2 days ago" becomes "il y a 2 jours" etc.
- Footer: "Built on", "Source code", "AGPL licence" are now translated
  via t() with keys added to all locale files and extracted-translations
- New keys: built_on, source_code, agpl_licence (FR/DE/IT/ES translations
  added manually)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 09:33:22 +00:00
claude 093c25f3dd i18n: validate and fix 212 translation errors across FR/DE/IT/ES
Build and Deploy Verso / deploy (push) Successful in 13m46s
Automated validation pass found and corrected:
- Brand names translated literally (Overleaf→"au verso"/dorso/retro/umseitig,
  Verso→"verso", LaTeX→"látex", Quarto→"en cuarto", TeXGPT→"TestoGPT")
- React-Trans <N> tags eaten by Google Translate (5 strings)
- FR grammar: "va sera écrasé" → "sera écrasé"
- FR preposition: "en <b>__email__</b>" → "sur <b>__email__</b>" (×2)
- FR title: "Accepter l'erreur de modification" → "Erreur d'acceptation des modifications"
- FR redundancy: "la version Rolling TeX Live" → "le build Rolling TeX Live"
- ES: "mesa" (furniture) → "tabla" (document table) in 3 strings

Tooling committed: translate_missing.py, fix_translations.py,
validate_translations.py — reusable for future locale additions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 09:16:57 +00:00
claude 375c18873e i18n: complete FR/DE/IT/ES translations via Google Translate (free tier)
Build and Deploy Verso / deploy (push) Successful in 14m37s
All four locales are now at 100% key coverage (was FR 38%, DE 50%,
IT 12%, ES 21%). Translated ~7,700 missing keys using deep-translator
with placeholder preservation (__varName__, <N>…</N> React-Trans tags).
Manual corrections can be made on top of this baseline as needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 22:15:47 +00:00
claude 3ee564ef70 ui: fix header row layout, selected-tile contrast, toolbar gradient
Build and Deploy Verso / deploy (push) Successful in 14m19s
- Header: revert column layout; title+zoom stay on left, search+button
  on right — all on one row (flex-wrap handles narrow viewports)
- Selected cards: switch from teal tint (invisible on teal bg) to solid
  white + blue ring, clearly distinguishable from the page background
- Editor toolbar: replace flat teal wash with an exponential-like decay
  (20%→9%→3%→0% teal overlay) so only the very top has colour

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 20:47:31 +00:00
claude 4ca6ec2b58 ui: move zoom picker next to section title, fix tooltip hover-stealing
Build and Deploy Verso / deploy (push) Successful in 14m18s
S/M/L buttons now appear inline with the page title rather than in the
action bar. Bootstrap tooltips get pointer-events:none so they can no
longer steal :hover from the card beneath them, preventing the lift
animation from flickering when hovering over tag dots.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 20:09:48 +00:00
claude be529e53f6 Fix translation keys and remove sidebar logo border
Build and Deploy Verso / deploy (push) Successful in 14m14s
- extracted-translations.json: add n_projects_selected, n_projects_selected_plural,
  deselect_all — the translations-loader.js only bundles keys present in this file,
  which is why the FR translations were silently stripped from the webpack chunk
- Revert lang.default workaround in i18n.ts (not needed)
- SCSS: remove border-top above .ds-nav-verso-logo in sidebar

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 18:31:04 +00:00
claude 592b4d3dad Fix translations, center logo/footer, add tile zoom control
Build and Deploy Verso / deploy (push) Successful in 14m8s
- i18n: unwrap webpack module object on dynamic JSON import (lang.default ?? lang)
  so French bundle keys are correctly registered in the i18next store
- Login logo: use flex centering on wrapper instead of display:block + margin:auto
- Footer (project list + login): align-items:center on .row for vertical centering
- Tile zoom: S/M/L control in header with CSS custom property (--lum-card-scale)
  that scales grid column width and card thumbnail height; persisted in localStorage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 17:20:21 +00:00
claude d08e834f49 fix: footer default font size, tag dots inline in meta row, OLTooltip
Build and Deploy Verso / deploy (push) Successful in 14m57s
- Footer: remove font-size/letter-spacing overrides so the browser
  default applies; monospace right col keeps its uppercase + tracking
  but no longer shrinks the font
- Card tags: move coloured dots into the .lumiere-card-meta flex row
  instead of a separate div — zero added height, all cards uniform
- Card tags tooltip: replace native title attr with OLTooltip (React
  Bootstrap tooltip) for a consistent Verso look on hover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 13:55:36 +00:00
claude 93c00d51a7 fix: lighter footer, dot-only card tags, deselect_all fr translation
Build and Deploy Verso / deploy (push) Successful in 14m46s
- Footer: pale teal (#edf7f4) with noise grain instead of dark navy;
  serif author credit darker (#1a2e3b), right col monospace muted;
  teal→blue 2px stripe preserved; same treatment on login page
- Card tags: render as 8px coloured dots only (title attr for tooltip)
  — no text means no wrapping, consistent card heights across the grid
- i18n fr.json: add deselect_all → "Tout désélectionner"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 13:20:33 +00:00
claude 7df583a3b2 feat: creative footer, card tags, selection bar theme, i18n fixes
Build and Deploy Verso / deploy (push) Successful in 15m23s
- Footer: dark navy (#0d1b24) with noise texture, teal→blue gradient
  top stripe; serif author credit on the left, monospace/uppercase
  meta links on the right; same design on login page
- Thumbnail: larger inset (14px top, 12px sides) so more background
  gradient shows around the preview; subtle drop shadow added
- Card tags: projects now show their label chips on the card (colored
  dot + name, read-only, max 3, no close button inside the link)
- Selection bar: btn-secondary recoloured to Lumière palette; dropdown
  menus rounded with teal accent on hover
- i18n fr.json: add n_projects_selected, n_projects_selected_plural,
  toolbar_selected_projects (×4 variants)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 12:54:22 +00:00
claude 194ffe54db fix: target footer.site-footer (ThinFooter) in Lumière theme
Build and Deploy Verso / deploy (push) Successful in 14m49s
Verso forces showThinFooter=true for non-SaaS instances, rendering
footer.site-footer everywhere — never .fat-footer. All previous footer
theme rules silently matched nothing. Fix both project and login page
footer selectors. Also increase login logo max-width to 520px, and
remove bottom inset/radius from thumbnail (folder-behind-card effect).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 12:23:36 +00:00
claude f9622820c7 i18n: fix Verso-branded keys and wire welcome message titles
Build and Deploy Verso / deploy (push) Successful in 21m59s
en.json: add browse_templates and learn_latex_with_a_tutorial.

fr.json:
- fix agree_with_the_terms (said "Overleaf", now says "Verso")
- add browse_templates + learn_latex_with_a_tutorial (FR)
- add 7 Verso-branded keys missing entirely from FR:
  add_manager_user_not_found, compile_timeout_explanation,
  download_metadata, institution_has_overleaf_subscription,
  one_step_away_from_professional_features,
  to_confirm_email_address_you_must_be_logged_in_with_the_requesting_account,
  welcome_to_overleaf_opening_workspace

welcome-message.tsx: replace two hardcoded English title strings with
t('learn_latex_with_a_tutorial') and t('browse_templates').

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 11:00:02 +00:00
claude 933efb1ff9 fix: thumbnail sizing, parallax hover effect, login logo size
Build and Deploy Verso / deploy (push) Successful in 14m45s
Thumbnail: explicit top/left/right/bottom + width/height on the img so
object-fit has a well-defined box in all browsers (inset shorthand alone
was underspecified for some).

Hover effect: the card already rises translateY(-3px) on hover. Add a
matching translateY(+3px) counter-transform on the thumbnail image so
its net viewport motion is zero — the document preview appears to float
in place while the gradient tile and card body lift up around it.

Login logo: raise max-width from 300px to 400px now that the competing
inline style has been removed from the pug template.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 10:41:14 +00:00
claude 0bbc07e0b9 fix: improve thumbnail quality and show gradient border around preview
Build and Deploy Verso / deploy (push) Successful in 15m20s
Increase pdftocairo output from 190px/q50 to 380px/q82 — 2× resolution
for crisp rendering on retina displays, higher quality to eliminate
visible compression artefacts.

Inset the thumbnail image 6px from the tile edges (inset: 6px) with a
4px border-radius so the card's colour gradient is visible as a frame
around the document preview.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 10:10:24 +00:00
claude 7da02d9e3a fix: use pdftocairo directly for thumbnails, no Docker image needed
Build and Deploy Verso / deploy (push) Successful in 15m11s
The previous implementation delegated to ConversionManager which uses
the Docker-based CommandRunner and is gated behind enablePdfConversions
(ENABLE_PDF_CONVERSIONS env var). Neither is configured in the Verso
deployment, so every thumbnail request 404'd before doing any work.

poppler-utils (which provides pdftocairo) is already installed directly
in the CLSI base image via install_deps.sh. Rewrite thumbnailFromBuild
to call pdftocairo via execFile instead — no feature flag, no Docker
image, no ConversionManager indirection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 09:42:35 +00:00
claude b70c8ddd0e feat: show compiled PDF thumbnail in Lumière project cards
Build and Deploy Verso / deploy (push) Successful in 14m47s
After a successful compile, web service calls a new CLSI endpoint
(GET /project/:id/user/:uid/build/:bid/thumbnail) which runs pdftocairo
page-1 to a 190px-wide JPEG using the existing thumbnail preset. The
JPEG is stored in Redis (90-day TTL, overwritten on next compile) by
the new ThumbnailManager.

GET /project/:Project_id/thumbnail serves the cached JPEG to authenticated
users, returning 404 when no thumbnail exists. Project cards in the
Lumière grid show the image overlaying the coloured gradient tile; if
the image 404s (project never compiled or cache expired) the onerror
handler hides it and the gradient + initial letter shows through.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 09:20:48 +00:00
claude 88ddbd2513 fix: center Verso logo on all pages and theme both footers
Build and Deploy Verso / deploy (push) Successful in 14m59s
Trim SVG viewBox from 760 to 590 (content ends ~x=570; the blank
right whitespace was making the wordmark look left-biased). Remove the
scale(1.2) transform from the sidebar logo — the negative-margin
container already fills the sidebar width. Change login logo max-width
to be CSS-controlled only (removed inline 480px override).

Footer: switch to `background` shorthand !important so the dark-theme
`var(--footer-background)` shorthand can't compete; deepen the teal to
#c8e4de so the Lumière colour is clearly visible. Add a
`body:has(.login-page) .fat-footer` rule so the login-page footer gets
the same treatment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 08:59:22 +00:00
claude 464aee7612 Add per-card action buttons to Lumière project grid
Build and Deploy Verso / deploy (push) Successful in 14m46s
Restructures ProjectCard so the card is a <div> container instead of <a>
(buttons cannot be nested inside anchor elements). A .lumiere-card-link
anchor wraps the thumb+body area; a .lumiere-card-actions strip sits below
it and fades in on hover.

Buttons added (reusing the same tooltip components as the classic table):
  - Copy project (opens CloneProjectModal)
  - Download project zip
  - Compile & download PDF
  - Archive project (skipped when already archived)
  - Trash project (skipped when already trashed)

Action icons are coloured $lum-text-muted at rest and shift to $lum-teal
on hover, matching the Lumière palette.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 08:17:45 +00:00
claude e3929572c3 Fix Lumière visual regressions: login, footer, logo, notifications
Build and Deploy Verso / deploy (push) Successful in 14m31s
- Login: centre logo (display:block + margin:auto, max-width 300px) and
  increase h1 from 1.4rem to 1.75rem
- Sidebar logo: switch from width:120%/margin-left:-10% to transform:scale(1.2)
  so the image scales symmetrically from centre and isn't cut on the left
- Footer: use !important on background-color/color to beat the dark-theme
  selector's higher specificity (:root [data-theme] = 0,4,0 vs our 0,3,0)
- Notifications: replace near-transparent rgba backgrounds with solid
  opaque colours so the teal page gradient can't bleed through; make the
  CTA button neutral slate-grey (not teal) with border-radius:8px

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 08:10:52 +00:00
claude 7ed1d02271 Lumière: replace pill toggle with rounded-square + sliding indicator
Build and Deploy Verso / deploy (push) Successful in 14m52s
The Code/Visual editor toggle now uses a teal sliding indicator (::before
pseudo-element) that glides between tabs via translateX instead of the
plain background-color crossfade. Container and labels get border-radius:
10px/7px to match the rest of the Lumière toolbar's rounded-square style.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 16:44:44 +00:00
claude aec466c6f3 Enlarge sidebar logo 20% and fix compile button corner rounding
- Logo (all themes): scale the Verso wordmark to 120% width, centered and
  clipped to the sidebar column — the word mark visually fills the full
  sidebar width. Uses overflow:hidden + width:120% !important + margin-left:-10%
  to override the existing inline width style.
- Compile button (Lumière): replace the all-corners border-radius:7px on the
  split button group with corner-specific rules — .compile-button gets 7px on
  the left side only, .compile-dropdown-toggle gets 7px on the right side only,
  so the shared inner edge stays flat as expected for a joined split button.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 16:35:11 +00:00
claude 8d3f550cef Fix Lumière notification colors, style CTA button, and theme footer
- Warning notification: switch from teal to amber (#b45309) so it reads as
  a genuine warning and doesn't blend into the teal UI chrome
- Notification CTA button (.btn-secondary): style with teal tint so the
  'Send confirmation code' button matches the Lumière theme
- Footer: override dark footer on .project-list-lumiere with a light teal
  background (#edf7f5), dark text, teal section headings — selector has same
  specificity as the default-theme dark rule but appears later in the cascade

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 16:16:28 +00:00
claude e19cd6e8ba Add multi-select to Lumière project card grid
Each project card now has a checkbox (top-left corner, semi-transparent by
default, fully opaque on hover). When any card is selected a selection bar
slides in above the grid showing: select-all checkbox, count, the existing
bulk-action toolbar (archive, trash, tags, delete), and a deselect-all button.
:has(input:checked) keeps all checkboxes visible once a selection is active.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 16:03:15 +00:00
claude 484a2e3aac Apply Lumière theme to editor settings modal and login page
- Settings modal: teal accent stripe on header, teal gradient nav background,
  teal active-tab highlight, teal section titles, teal focus rings on form
  controls — scoped via :has(.ide-settings-modal-body) so other modals are
  unaffected
- Login page: grainy teal gradient background, white rounded-square card with
  teal/blue accent stripe, teal labels, focus rings, primary button — always
  applied since users haven't set a theme yet

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 15:53:10 +00:00
claude c70ec1c501 feat(lumiere): Lumière-styled notification banners
Build and Deploy Verso / deploy (push) Successful in 20m14s
Override Bootstrap orange/yellow warning and generic blue info colors
with the Lumière teal palette. Warning banners now use a soft teal
tint instead of orange; info banners use the Lumière blue. Both types
get 10px border-radius and a subtle shadow to match the card style.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 15:35:35 +00:00
claude 466f908f7c fix: admin button font size, extend button rounding to PDF toolbar
Build and Deploy Verso / deploy (push) Has been cancelled
- Remove font-size: 0.8rem from Admin navbar button (was shrinking text)
- Add border-radius: 7px to .toolbar-pdf .btn so the Recompile, Logs and
  Download buttons in the PDF panel get the Lumière rounded-square shape
- Add border-radius to compile-button-group .btn to cover the dropdown
  arrow toggle next to the Recompile button

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 15:33:44 +00:00
claude 9e618280b1 fix: restore theme toggle in navbar account menu, Lumière button styling
Build and Deploy Verso / deploy (push) Successful in 21m51s
- logged-in-items: pass showThemeToggle to AccountMenuItems so the theme
  switcher is accessible from the top-right navbar (was lost when the
  sidebar account icons were removed); AccountMenuItems already gates on
  hasOverallThemes so it's a no-op on non-themed pages
- project-list-lumiere: restyle Account + Admin navbar buttons — rounded
  square (8px) instead of pill, teal resting tint on Account, subtle
  teal border on hover; matches Lumière design language
- ide-lumiere: extend rounded-square styling to all toolbar action buttons
  (Share, Present, History, Layout, File/Edit/Help menu buttons) via
  .ide-redesign-toolbar-actions and .ide-redesign-toolbar-menu selectors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 14:59:48 +00:00
claude 62847fbc15 Fix remaining Overleaf branding in email confirmation notifications
Replaces "Overleaf subscription"/"Overleaf Commons"/"Overleaf premium features"
with Verso equivalents in the institution subscription and commons upgrade
notification strings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 14:37:33 +00:00
claude fc535590eb fix: restore account button, fix double border, logo, gradient, editor buttons
Build and Deploy Verso / deploy (push) Successful in 14m48s
- project-list-ds-nav.scss: remove display:none for .nav-item-account on
  desktop — it was hidden because the sidebar handled it, but now the sidebar
  no longer has the account icon so this made it invisible everywhere
- logged-in-items / nav-dropdown-menu: show User icon alongside 'Account'
  text in navbar dropdown so it's recognisable as an account button
- Lumière: remove border-top from .ds-nav-verso-logo (was doubling up with
  .ds-nav-sidebar-lower border)
- Logo hover: drop scale transform in both themes, use filter:brightness only
- Gradient: drop background-attachment:fixed (unreliable in scroll containers);
  switch to circle gradients at 0.60/0.45 opacity; base colour #e8f5f2
- Editor ide-lumiere: rounded square (7px) on .ol-cm-toolbar-button with teal
  hover/active states to match the Lumière design language

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 14:27:40 +00:00
claude 5fa73fa8e3 fix: rebrand Overleaf strings, move user menu, fix logo + button alignment
Build and Deploy Verso / deploy (push) Successful in 14m49s
- en.json: replace 'Overleaf' with 'Verso' in 6 user-visible strings
  (email_already_registered, add_manager_user_not_found, compile_timeout,
  download_metadata, to_confirm_email address, welcome_opening_workspace)
- groups-and-enterprise-banner: use dynamic appName instead of hard-coded
  'Overleaf'
- SidebarLowerSection: add showAccountIcons prop (default true); set false
  in project dashboard sidebar — account menu is already in the top-right
  navbar, so the bottom-left duplicate is removed for all themes
- ds-nav-verso-logo: replace opacity-fade hover with scale+brightness
  transform so logo is fully visible at rest
- Lumière: scope new-project-dropdown sidebar padding to avoid misaligning
  the button when it appears next to the search bar in the header

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 13:58:17 +00:00
claude 074ff2791d feat(lumiere): bold background + creative editor theme
Build and Deploy Verso / deploy (push) Successful in 14m43s
Dashboard:
- Grain opacity 0.12→0.28, teal orb 0.22→0.45, blue orb 0.16→0.32
- Added soft centre glow for depth; tinted base colour #f0f7f5

Editor:
- Toolbar: visible teal-wash gradient (dark→light) + 4px accent stripe
  + teal box-shadow
- Rail icon strip: teal-tinted gradient (#cceee8→#daf2ee)
- File tree / outline panel: full grainy gradient (matching dashboard)
  with teal/blue orbs + panel header with teal left-glow
- Compile button: teal gradient replacing the default blue

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 13:40:40 +00:00
claude a0d1829c1b fix(lumiere): fix toolbar dropdown clipping, boost grain/gradient visibility
Build and Deploy Verso / deploy (push) Successful in 14m29s
- Remove overflow:hidden from toolbar — it was clipping dropdown menus
- Increase SVG noise opacity 0.06→0.12 and gradient orb opacity for more
  visible texture on the dashboard background

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 12:21:34 +00:00
claude ac02fd707e feat(lumiere): grainy gradient background + editor chrome theme
Build and Deploy Verso / deploy (push) Successful in 15m0s
- Replace flat #f0f4f8 main content bg with layered grainy gradient:
  soft radial teal/blue orbs (background-attachment: fixed) plus
  SVG feTurbulence noise tile for organic depth
- Cards get backdrop-filter glass effect over the textured surface
- New ide-lumiere.scss: sets body[data-lumiere] via useThemedPage, then
  overrides toolbar (white + 3px teal→blue accent stripe), rail (teal
  active states), and file-tree (teal selected/hover) CSS variables

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 11:39:58 +00:00
claude 10a17c5443 feat: modernize Lumière sidebar and navbar styling
Build and Deploy Verso / deploy (push) Successful in 14m36s
- Navbar: teal-to-blue gradient accent stripe, soft shadow replacing
  hard border, pill-shaped hover on nav links
- New Project button: gradient teal fill, elevated shadow on hover
- Filter buttons: pill shape, solid teal active state, teal tint hover
- Tags section: uppercase teal section header, refined tag items
- Account/help icons: circular buttons with teal hover ring and glow
- Theme toggle: teal selected indicator
- Verso logo: subtle top separator with hover opacity transition

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 11:02:36 +00:00
claude 13577693f2 fix: restore sidebar CSS context in Lumière dashboard
Build and Deploy Verso / deploy (push) Successful in 15m50s
The Lumière container now carries project-ds-nav-page and
project-list-wrapper so the sidebar picks up all its existing styles.
The grey-rectangle button issue and broken sidebar layout were caused
by those expected parent classes being absent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 09:37:08 +00:00
claude 0b616436cf feat: toggle bold/italic and add underline, smallcaps, link for Typst editor
Build and Deploy Verso / deploy (push) Has been cancelled
Bold (Ctrl+B) and italic (Ctrl+I) now unwrap when the cursor is already
inside a Strong/Emphasis node. Added #underline[…] and #smallcaps[…]
wrap commands (toolbar only) and #link("")[…] with Ctrl+K shortcut that
places the cursor in the URL field.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 09:31:28 +00:00
claude 926b6f7cbb feat: add Verso Lumière theme with card-based project dashboard
Build and Deploy Verso / deploy (push) Successful in 14m45s
New theme with gradient document cards, serif title typography and a
light airy palette. Set as the default for new users. Existing users
keep their current theme; all users can switch via the theme toggle
(new sparkle icon). Classic Dark / Classic Light are renamed accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 09:13:46 +00:00
claude 31db7b2b4e feat: add bold/italic shortcuts and toolbar buttons for Typst editor
Build and Deploy Verso / deploy (push) Successful in 14m39s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 08:28:47 +00:00
claude 4899afd45f fix: enable shell-escape in test deployment for svg package support
Build and Deploy Verso / deploy (push) Successful in 1m6s
OVERLEAF_LATEX_SHELL_ESCAPE=true was added to the prod workflow but
missed in the test workflow, so the svg package still failed on
test.alocoq.fr despite inkscape being installed in the image.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:10:06 +00:00
claude 8b6b76c2fa fix: install inkscape after pip to avoid apt/pip numpy conflict
Build and Deploy Verso / deploy (push) Successful in 1m12s
inkscape's apt dependencies include python3-numpy, which pip can't
uninstall (no RECORD file). Moving inkscape to its own RUN layer after
the pip installs avoids the conflict: pip numpy lands in /usr/local/lib
first, then apt installs its numpy into /usr/lib alongside it, and
Python resolves /usr/local/lib first at import time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 12:49:08 +00:00
claude 4285d8d74c fix: add --ignore-installed so pip can coexist with inkscape's apt numpy
Build and Deploy Verso / deploy (push) Successful in 1m8s
inkscape pulls in python3-numpy 1.26.4 via apt; pip can't uninstall apt
packages (no RECORD file). --ignore-installed makes pip install its own
copy into /usr/local/lib without touching the apt version; /usr/local/lib
takes import precedence so runtime code gets the pip-managed numpy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 11:12:11 +00:00
claude d8ce7f9dc1 feat: support svg package — install Inkscape and enable shell-escape
Build and Deploy Verso / deploy (push) Has been cancelled
The LaTeX svg package converts .svg files to PDF at compile time by
shelling out to Inkscape (requires --shell-escape). Without Inkscape in
the image and the flag enabled, compilation fails with "Did you run the
export with Inkscape?".

- Dockerfile-base: add inkscape to the apt install block
- settings.js: expose OVERLEAF_LATEX_SHELL_ESCAPE env var → clsi.latexShellEscape
- LatexRunner.js: pass -shell-escape to latexmk when the setting is on
- deploy-verso-prod.yml: set OVERLEAF_LATEX_SHELL_ESCAPE=true (trusted-user instance)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 10:57:56 +00:00
claude 8b7da8296c fix: pass session token so anonymous users can install python packages
Build and Deploy Verso / deploy (push) Successful in 14m23s
Build and Deploy Verso (prod) / deploy (push) Successful in 3m55s
userCanInstallPython passed null as the token, so anonymous users
accessing via a share link got privilege level NONE from the WithoutUser
path and allowPythonInstall was always false for them.

Read the token from req.session.anonTokenAccess via
TokenAccessHandler.getRequestToken and forward it through
userCanInstallPython to getPrivilegeLevelForProject.  For TOKEN_BASED
projects this resolves the anonymous user's access level via
getPrivilegeLevelForProjectWithToken, enabling package installation.

Also update Quarto Slides badge color to #e4637c.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 09:17:30 +00:00
claude e6773c6baf fix: read stored project compiler for quartoFlavor update
Build and Deploy Verso / deploy (push) Successful in 20m28s
options.compiler is set from req.body.compiler which the frontend never
sends, so the condition was never true and quartoFlavor was never written.
Use ProjectGetter to read the stored compiler instead. Fire-and-forget so
it does not delay the compile response.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 08:48:25 +00:00
claude 2c8dad08f6 project-list: show plain 'Quarto' badge when quartoFlavor is unset
Build and Deploy Verso / deploy (push) Successful in 13m55s
Existing projects have no quartoFlavor value in the database (new field),
so defaulting to 'Quarto PDF' incorrectly labelled all of them. Show the
plain 'Quarto' label until the first compile sets the flavor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 08:30:34 +00:00
claude 7fecaf491a compile: allow python package install for all users who can access the project
Build and Deploy Verso / deploy (push) Successful in 9m23s
Previously userCanInstallPython used ignorePublicAccess: true, which
blocked token-link users (not-yet-joined) and logged-in readers of public
projects from installing packages. This caused Quarto presentations with
Python cells to fail for shared read-only users even when the required
packages were already listed in requirements.vrf.

The security model is: what gets installed is fully controlled by
requirements.vrf, which is only writable by members with write access.
There is therefore no security reason to block other readers from
triggering installation of already-approved packages.

Drop ignorePublicAccess so all users with any privilege level (direct,
token-based, or public-project) can trigger the venv install.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 08:20:46 +00:00
claude 9c7a10aa39 fix: correct import path and minor issues in quartoFlavor update
Build and Deploy Verso / deploy (push) Successful in 13m44s
- Fix wrong import path '../models/Project.mjs' → '../../models/Project.mjs'
  (from Features/Compile/, '..' is Features/, not src/; the server would
  crash on startup with ERR_MODULE_NOT_FOUND in Node.js ESM)
- Log MongoDB errors instead of silently swallowing them
- Remove null from Mongoose String enum (not a valid enum value for strings)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 08:06:00 +00:00
claude bbf532d282 project-list: distinguish Quarto PDF vs Quarto Slides in format badge
Build and Deploy Verso / deploy (push) Successful in 9m30s
Add a quartoFlavor field ('revealjs' | 'pdf') to the Project model.
After each successful Quarto compile, CompileController detects the output
type (output.html → revealjs, otherwise pdf) and persists it.
ProjectListController includes it in the projection and serialization so
it reaches the frontend without an extra round-trip.

Badge variants:
  - quartoFlavor unset (new/uncompiled) → "Quarto PDF" #447099
  - quartoFlavor 'pdf'                  → "Quarto PDF" #447099 (Quarto blue)
  - quartoFlavor 'revealjs'             → "Quarto Slides" #7e56c2 (purple)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 07:55:45 +00:00
claude eada1e9979 file-tree: show python packages button for all quarto projects
Build and Deploy Verso / deploy (push) Successful in 13m42s
The previous approach (pdfFile?.path === 'output.html') caused a
chicken-and-egg problem: the button only appeared after a successful
RevealJS compile, but you need to add packages before the first compile.

Use compiler === 'quarto' from ProjectSettingsContext instead — this is
set from project metadata and available immediately, before any compile.
Quarto supports Jupyter Python cells in all output formats (RevealJS HTML,
PDF via LaTeX, PDF via Typst), so showing the button for any Quarto project
is the correct behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 07:41:21 +00:00
claude 2f88ad124d ui: correct LaTeX badge to Overleaf button green #098842
Build and Deploy Verso / deploy (push) Successful in 13m47s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 07:05:08 +00:00
claude a398127522 ui: brand badge colors and hide python packages for non-revealjs
Build and Deploy Verso / deploy (push) Successful in 13m8s
- LaTeX badge: #13c965 (Overleaf brand green, from upstream overleaf/overleaf)
- Typst badge: #239dad (Typst brand blue/teal, from typst.app)
- Python packages toolbar button: only shown when the compiled output is
  output.html, i.e. a Quarto RevealJS presentation.  Uses the same
  pdfFile?.path === 'output.html' check as PresentationPreviewButton.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 06:51:30 +00:00
claude 083b195462 typst: support '+' binary operator in arg values
Build and Deploy Verso / deploy (push) Successful in 9m27s
stroke: 0.8pt + brand broke arg-list parsing because '+' was not a grammar
terminal. The parser exited CodeArgs via error recovery, so subsequent
named args (radius:, inset:, fill:) were never seen as CodeArgKey.

Add codeArgValue { codeValue | codeArgValue !add "+" codeValue } — a
left-recursive inline rule used only inside CodeArgs.  The !add cut point
gives the shift strict dominance over the reduce (prec add > 0 vs 0), so
a '+' after a value greedily extends the expression.  Because codeArgValue
only appears inside CodeArgs, the codeStatement* LALR-merging that caused
trouble for the earlier callSuffix* approach does not apply here.

Also add PLUS to codeIdentTokenizer's valid-predecessor list so identifiers
after '+' (the right-hand operand) are correctly tokenized as CodeIdent.
Add "+" to @tokens @precedence so it beats MarkupContent in merged states.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 06:07:32 +00:00
claude 0656ddfe52 fix(typst): highlight keywords/idents inside #{} code blocks
Build and Deploy Verso / deploy (push) Successful in 9m32s
Replace the opaque CodeBlockBody external tokenizer with grammar-parsed
codeStatement* so that keywords (show, let, set, …) and identifiers
inside #{ } code blocks receive proper Lezer nodes and are highlighted.

Key grammar changes:
- CodeBlock { "{" codeStatement* "}" } — structured, not opaque
- codeStatement uses two explicit alternatives for keyword lines:
    CodeKeyword !kw callOrValueAndBody  (grabs the subject eagerly)
    CodeKeyword keywordBody?            (bare keyword or body-only form)
  The !kw cut-point gives shift prec kw > 0 over the unannotated reduce,
  resolving the LALR merge ambiguity without @left/@right on kw.
- callOrValue { FuncExpr | CodeIdent | CodeString } — replaces CallExpr
  { CodeIdent !call callSuffix* }.  The * quantifier annotated both
  shift and reduce with !call, making them a same-prec tie that @right
  could not reliably resolve in merged states.  Using FuncExpr (required
  callSuffixes) + bare CodeIdent makes the tie strict (call > 0 for
  FuncExpr shift vs 0 for bare-ident reduce), then @right handles only
  the extension-of-callSuffixes case (shift = call<<2, FuncExpr reduce
  = call<<2 - 1 via @right encoding).
- KeywordExpr gets the same two-alternative structure as codeStatement
  so nested show/set/let inside a code block (e.g. show sel: set text)
  also parse without LALR state-merge conflicts.
- CallExpr removed; its role is split between FuncExpr (has args/chain)
  and bare CodeIdent (no args).  Styling updated: CodeExpr/CodeIdent
  replaces CallExpr/CodeIdent for bare #ident function-style highlights.
- codeKeywordTokenizer and codeIdentTokenizer already accept keywords /
  identifiers after { and ; (added in previous commit) — consistent with
  the new grammar.

Parse results:
  #{ show strong: link.with(url); body }
  → CodeKeyword "show", CodeIdent "strong", FuncExpr "link.with(url)",
    CodeIdent "body" — all properly highlighted, no ⚠ errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 21:47:21 +00:00
claude 056d9a7f47 fix(typst): add CodeArray so tuple args don't break nested call parsing
Build and Deploy Verso / deploy (push) Successful in 9m44s
align: (left, center, left) is a Typst array literal.  Without a grammar
rule for it, the parser treated the ')' closing the tuple as the ')' closing
the enclosing function call, so everything after align: — all ContentBlock
args and any subsequent named keys like 'caption:' — fell outside the parsed
call tree and was highlighted as MarkupContent.

Add CodeArray { "(" codeArgList? ")" } as a codeValue alternative so
parenthesised arrays and dictionaries parse correctly.  Also regenerate
typst.mjs / typst.terms.mjs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 20:13:38 +00:00
claude 8ed44cc352 fix(typst): style CodeArgKey via HighlightStyle, not theme CSS
Build and Deploy Verso / deploy (push) Successful in 13m38s
The tok-attributeName CSS class relies on each theme defining it, but
26 of 41 themes never had it. Defining the colour directly in
typstHighlightStyle (like we do for heading/strong/emphasis) applies
it universally regardless of which theme is active.

Amber #c47900 is legible on both light and dark backgrounds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:56:20 +00:00
claude 52ebff6286 debug(typst): force arg keys bright red to confirm token pipeline
Build and Deploy Verso / deploy (push) Successful in 10m0s
Adds { tag: t.attributeName, color: '#cc0000', fontWeight: 'bold' } to
typstHighlightStyle so named arg keys are unmistakably red if the
CodeArgKey token is reaching the highlighter.  Will be removed once
the pipeline is confirmed working and replaced with per-theme colors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:41:57 +00:00
claude 7c0ec9dd39 fix(typst): commit compiled grammar so CI always uses current parser
Build and Deploy Verso / deploy (push) Successful in 13m45s
The webpack plugin that compiles typst.grammar may silently skip
recompilation when file mtimes are ambiguous in Docker BuildKit layers.
Committing typst.mjs and typst.terms.mjs guarantees the build always
ships the correct parser without depending on build-time generation.

To regenerate after grammar changes:
  node -e "const {buildParserFile}=require('/tmp/lezertest/...'); ..."
  (or: yarn run lezer-latex:generate from services/web)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 19:02:16 +00:00
claude e9a34a5bd8 fix(typst): exclude ] from MarkupContent so ContentBlock always closes
Build and Deploy Verso / deploy (push) Successful in 13m35s
The core bug: MarkupContent { ![...]+ } did not exclude ']', so inside
#figure(table([A],[B]), caption:[...]) the tokenizer consumed ']' as
MarkupContent, ContentBlocks never closed, and all remaining args like
'caption:' were swallowed as MarkupContent instead of CodeArgKey.

Fix mirrors the LaTeX grammar pattern (its Normal token excludes \] and
\[): add ']' to MarkupContent's exclusion set and provide ClosingSquare
{ "]" } as an item alternative for bare ']' in body text.  The grammar's
existing @precedence { "]" ClosingSquare } ensures "]" wins and closes
the ContentBlock; outside a ContentBlock only ClosingSquare is valid.

Also change URL style tag from t.url (tok-url, unstyled in all themes)
to t.string (tok-string, styled in every theme).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 18:33:20 +00:00
claude f9fc0d9905 typst: fix URL comment split and heading label highlighting
Build and Deploy Verso / deploy (push) Successful in 13m56s
- Add URL token (https://... / http://...) so '://' is never split into
  ':' + LineComment '//', preventing URLs from being styled as comments
- Stop headingTitleTokenizer before '<label>' patterns so labels at the
  end of headings get Label node styling instead of being consumed as
  heading title text
- Style URL nodes with t.url tag

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 17:38:32 +00:00
claude d7ca7b194d feat(typst): parse show-rule bodies, let-value bindings, and content-block call args
Build and Deploy Verso / deploy (push) Successful in 14m13s
Three grammar gaps caused large blocks of code to be unhighlighted:

1. KeywordExpr now accepts an exclusive keywordBody: '#show sel: body' is
   parsed via ':', and '#let name = value' via '='.  callOrValue extends
   the subject to include CodeString so '#import "pkg"' highlights the path.

2. ContentBlock added to callSuffix so '#func("arg")[content]' and
   '#next-step("url")[...]' parse their trailing content block as code
   rather than falling back to markup.

3. Tokenizer: COLON added as a valid predecessor so identifiers (e.g. 'blue'
   in 'fill: blue') and keywords (e.g. 'set' in '#show link: set text(...)')
   are recognised after ':'.  EQUALS already added in the previous commit.
   The ident-chain backward scan now also skips whitespace before testing for
   '#' or ':', enabling 'text' in 'set text' to trace back to '#' through the
   keyword gap.  @precedence updated with CodeString, '[', ':' to resolve
   overlapping-token conflicts with MarkupContent in merged states.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 14:57:21 +00:00
claude 47cf84f20b fix(typst): highlight named arg keys after complex nested calls
Build and Deploy Verso / deploy (push) Successful in 13m48s
canShift(CodeIdent) returns false in LALR-merged states that arise after
reducing a complex first argument (e.g. figure(table(...), caption: ...)).
The previous guard `!couldBeIdent && !canShift(CodeArgKey)` then caused
an early exit before the character-level scan ran, silently dropping the
CodeArgKey token for any named arg key that follows such a reduction.

Fix: run the backward character scan first and derive `couldBeArgKey`
from the raw predecessor char ('(' or ',') rather than from canShift.
The early-exit now reads `!couldBeIdent && !couldBeArgKey`, so arg-key
positions always proceed to the full scan regardless of parser state.
Also stop calling canShift(CodeArgKey) entirely — it is unreliable here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 13:57:10 +00:00
claude 2d3e64da92 themes: add tok-attributeName color to 15 themes that were missing it
Build and Deploy Verso / deploy (push) Successful in 14m6s
The Typst grammar now emits CodeArgKey (mapped to tok-attributeName) for
named argument keys like 'columns:', 'align:', 'caption:'.  15 of 41
editor themes had no .tok-attributeName rule, so those keys appeared in
the default text color (black) despite the correct CSS class being set.

Chose colors that complement each theme's existing palette:
light themes → warm dark-orange family (#994409 / #7B3814 / #735C0F)
dark themes  → each theme's accent color (gold, warm red, lavender…)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 13:28:09 +00:00
claude 2db6e63162 typst: fix CodeArgKey detection using character-level context
Build and Deploy Verso / deploy (push) Successful in 9m31s
canShift(CodeArgKey) was consistently returning false because LALR
state merging folds the codeArgItem start state into others where
CodeArgKey is not in the valid set.  As a result, named arg keys like
'columns:', 'align:', 'caption:' were always falling through to
CodeIdent (black) instead of CodeArgKey (tok-attributeName).

Fix: detect named arg key position by inspecting the nearest
non-whitespace predecessor character instead of trusting canShift.
prev == '(' or ',' means we are inside a call argument list — the only
positions where a named arg key can appear.  prev == last char of a
keyword word (e.g. 'w' of 'show') correctly excludes '#show heading:'
from being treated as a named arg.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:38:05 +00:00
claude f2b7034b51 typst: distinguish function calls from value identifiers; fix math outline
Build and Deploy Verso / deploy (push) Successful in 9m49s
Named value identifiers like 'left', 'center', 'right' were being
highlighted as tok-function (blue) because codeValue used CallExpr
(callSuffix*), which styled any identifier in value position as a
function.  Fix: add FuncExpr { CodeIdent callSuffix+ } (requires at
least one argument list or method suffix) and use it in codeValue
instead of CallExpr.  Plain identifiers in value position now fall
through to CodeIdent → tok-variableName.  CallExpr (callSuffix*) is
kept for codeExprBody and KeywordExpr where zero-suffix idents are
valid.

Tokenizer safety: only acceptToken(CodeIdent) when canShift(CodeIdent)
is true, preventing emission in LALR-merged states where neither
CodeArgKey nor CodeIdent is expected.

Outline: track '$'-parity across lines so that lines inside a display
math block (e.g. '= b+c$') are not incorrectly reported as headings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 12:04:33 +00:00
claude 4c6032bce0 typst: fix named-arg key highlighting and multi-line math
Build and Deploy Verso / deploy (push) Successful in 13m41s
Named arg keys (columns:, align:, caption:) were appearing in black
because LALR state merging broke the CodeArgs/CodeIdent path for
multi-line expressions.  Fix: emit a dedicated CodeArgKey token from
codeIdentTokenizer (forward-peek for ':' to pre-disambiguate), declare
it in the grammar's codeArgItem rule, and map it to t.attributeName in
styleTags — bypassing LALR lookahead entirely.

Multi-line display math ($ ...\n... $) was consuming the rest of the
document as orange text when contextual:true caused a backward scan to
find a previous closing '$' and falsely set isDisplay=true.  Fix:
revert mathContentTokenizer to contextual:false with '\n' stop (each
MathContent token covers one line), and change InlineMath to
MathContent* so @skip consumes the newlines between lines.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:25:31 +00:00
claude 2fdb155547 fix(typst): support multi-line display math ($...$)
Build and Deploy Verso / deploy (push) Successful in 9m44s
mathContentTokenizer now detects inline vs display math by scanning back
to the opening '$': if @skip consumed whitespace between '$' and the
content the tokenizer removes the newline stop (display math), otherwise
it keeps it (inline math).  contextual: true prevents the tokenizer from
firing outside InlineMath entirely, avoiding the orange-body-text
regression seen when this was previously attempted without the guard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 10:52:29 +00:00
claude 75bc3bcc73 fix(typst): highlight idents after #keyword and wire tok-attributeName
Build and Deploy Verso / deploy (push) Successful in 9m34s
- codeIdentTokenizer: extend guard to scan back through the keyword word
  and accept when '#' immediately precedes it, so 'text' in '#set text(...)'
  and 'heading' in '#show heading:' are highlighted as function names
- classHighlighter: add tags.attributeName → tok-attributeName mapping;
  all 26 themes already define .tok-attributeName colours but the tag was
  never mapped to the class, leaving named arg keys (columns:, caption:)
  completely unstyled

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 09:18:31 +00:00
claude 1b773fdda0 fix(typst): add newlines to @skip so multi-line code args parse cleanly
Build and Deploy Verso / deploy (push) Successful in 9m31s
'\n' inside CodeArgs was an invalid token, triggering Lezer error recovery
and resetting parser state before codeIdentTokenizer could fire.  Heading
detection is unaffected — headingTokenizer uses raw input.peek(-1) char
reads which see the '\n' byte regardless of what @skip consumes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 08:47:47 +00:00
claude 019b4041a8 fix(typst): highlight function names across newlines in code args
Build and Deploy Verso / deploy (push) Successful in 9m35s
codeIdentTokenizer's backward scan stopped at \n, so identifiers at the
start of a new indented line inside multi-line arg lists (e.g. image(),
table() inside #figure(...)) never matched the '(' guard and stayed black.
Extend the whitespace skip to also cross newlines.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 07:28:43 +00:00
claude 4aca4aaac6 fix: make CodeIdent external and replace strongItem*/emphItem* with flat body tokens
Build and Deploy Verso / deploy (push) Successful in 13m36s
Two LALR state-merging bugs prevented Strong/Emphasis nodes from ever being
produced (confirmed: tok-strong/tok-emphasis count = 0 in browser diagnostic).

Bug 1 — _italic_ consumed as CodeIdent:
  CodeIdent was a @tokens rule with identHead = [A-Za-z_], so '_italic_' (the
  entire string including both underscores) matched as one CodeIdent token.
  LALR merging caused CodeIdent to be in item*'s valid set, and CodeIdent >
  "_" in @precedence, so the parser never opened Emphasis.

  Fix: move CodeIdent to an external tokenizer (codeIdentTokenizer) with a
  character-level guard — only fires when the preceding non-whitespace char
  is one of '#', '.', '(', ',' (genuine code-context positions).  In body
  text where peek-back finds a newline, space, or markup delimiter, the
  tokenizer returns without emitting, letting '"_"' open Emphasis correctly.

Bug 2 — StrongText never produced inside Strong:
  The strongItem* / emphItem* loops merged with item* states via Lezer's
  aggressive LALR merging.  In the merged state MarkupContent was in the
  valid set (from the item* side) and MarkupContent > StrongText in
  @precedence, so MarkupContent was always produced — not a valid strongItem,
  leading to error recovery with no StrongText in the tree.

  Fix: replace the recursive strongItem* / emphItem* loops with flat external
  tokens StrongBody / EmphBody (contextual: true).  These fire only inside
  Strong → "*" . StrongBody? "*" and Emphasis → "_" . EmphBody? "_", states
  specific enough that canShift is reliable.  They read everything up to the
  closing delimiter or newline in one token, bypassing the LALR merging
  entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 22:15:22 +00:00
claude f9d46aabeb fix: revert mathContentTokenizer regression (contextual + no-newline-stop)
Build and Deploy Verso / deploy (push) Successful in 9m43s
The previous change switched mathContentTokenizer to contextual:true with no
newline stop, intending to support multi-line Typst block math.  However,
LALR state merging causes canShift(MathContent) to spuriously return true in
body-text positions (e.g. after a RawInline backtick close), so the tokenizer
consumed everything until the next '$' — turning a full paragraph orange.

Revert to contextual:false with newline stop.  This correctly handles both
inline ($x^2$) and single-line block ($ integral ... $) math.  Multi-line
block math ($ formula\n continuation $) remains a separate issue for later.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:50:15 +00:00
claude 54ab282efc Fix Typst: support multi-line math blocks ($ ... multi-line ... $)
Build and Deploy Verso / deploy (push) Successful in 14m1s
mathContentTokenizer was stopping at newlines, causing a parse error for
Typst block math that spans multiple lines.  The parser then entered a bad
state that cascaded: the stale error-recovery left the item* parser in a
degraded mode, causing body text below the math block to be highlighted as
t.string (orange).

Fix: switch to contextual: true (only fires inside InlineMath where
MathContent is actually expected) and remove the newline restriction so
the tokenizer reads until the closing '$' regardless of line boundaries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 21:03:17 +00:00
claude f976c5ba92 Fix Typst: bold/italic rendering and keyword false-highlights in body text
Build and Deploy Verso / deploy (push) Successful in 14m15s
Add .tok-strong and .tok-emphasis CSS to the static editor theme so
bold/italic markup actually renders visually.

Move CodeKeyword from @tokens to an external tokenizer (codeKeywordTokenizer)
with a peek(-1)==='#' guard. LALR state-merging causes code-mode states to be
reachable in markup positions, making common English words like "in", "for",
"while", "return" trigger CodeKeyword highlighting in body text. The '#' guard
ensures keywords only fire immediately after the '#' sigil, never in prose.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 20:20:02 +00:00
claude f5a94c0ced fix(typst): guard RawBlockBody against LALR-merged body-text states
Build and Deploy Verso / deploy (push) Successful in 10m0s
canShift(RawBlockBody) returns true in states LALR-merged with the
post-RawBlockOpen state, causing the tokenizer to consume all remaining
body text as one giant RawBlockBody. Add a backward character scan:
require newline immediately before input.pos, then walk back past any
lang tag (A-Za-z0-9) and verify three backticks precede it. Body-text
positions never have backtick-backtick-backtick there, so the guard
rejects them.

This was the root cause of everything after the first heading being
black: RawBlockBody swallowed the entire document from the user-name
line onward, making headings, bold, italic and math invisible.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 19:44:05 +00:00
claude 11d852fe18 Revert "chore: add browser diagnostic script for Typst highlighting"
Build and Deploy Verso / deploy (push) Successful in 1m15s
This reverts commit 8c9088d054.
2026-06-08 19:29:46 +00:00
claude cc0d97903c Revert "chore: fix CodeMirror view accessor in diagnostic script"
This reverts commit 5850ffcad7.
2026-06-08 19:29:46 +00:00
claude 5850ffcad7 chore: fix CodeMirror view accessor in diagnostic script
Build and Deploy Verso / deploy (push) Successful in 1m4s
2026-06-08 19:27:06 +00:00
claude 8c9088d054 chore: add browser diagnostic script for Typst highlighting
Build and Deploy Verso / deploy (push) Successful in 1m12s
2026-06-08 19:21:30 +00:00
claude 974a9c4fb3 fix(typst): restore HeadingMark+HeadingTitle with character-level bleed guard
Build and Deploy Verso / deploy (push) Successful in 10m0s
The single-HeadingLine token approach caused everything after the first
heading to be unparsed. Reverting to the two-token structure but adding a
backward character scan in headingTitleTokenizer: after canShift(), walk
backward past whitespace and require '=' immediately before the current
position. Body-text positions in LALR-merged states will have a letter or
closing bracket there instead, so the tokenizer returns without accepting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 18:17:17 +00:00
claude 34025dc084 fix(typst): use HeadingLine single token to fix inline element highlighting
Build and Deploy Verso / deploy (push) Successful in 9m35s
The two-token approach (HeadingMark + HeadingTitle) caused LALR state
merging: the parser state waiting for HeadingTitle after HeadingMark was
merged into body-text item* states. In those merged states the
headingTitleTokenizer fired for every paragraph line, swallowing bold,
italic, math and inline function tokens — leaving body text black.

Fix: collapse the heading into a single HeadingLine external token that
covers the entire heading line (= prefix + title). A single-token Heading
rule leaves no post-token parser state waiting for a second token, so no
LALR merging can occur. The ViewPlugin and all HeadingMark/HeadingTitle
infrastructure are removed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 16:55:09 +00:00
claude 0099672015 fix(typst): restore HeadingTitle token to fix broken syntax highlighting
Build and Deploy Verso / deploy (push) Successful in 10m12s
Removing HeadingTitle from the grammar left HeadingTitle? undeclared,
causing the Lezer grammar compiler to fail and producing no parser
output — hence everything rendered as unstyled black text.

Dual approach to prevent heading style bleed:
- HeadingTitle exists in grammar with contextual: true + canShift guard
  (prevents it from matching in body-text LALR states)
- HeadingTitle is intentionally absent from styleTags so even spurious
  matches cannot apply heading colour to body text
- ViewPlugin styles heading titles by finding HeadingMark nodes and
  extending tok-heading decoration to end-of-line

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 15:38:38 +00:00
claude a5ca432396 Fix heading bleeding and smooth_pdf_transition translation
Build and Deploy Verso / deploy (push) Successful in 13m35s
Two fixes:

1. Heading style bleeding (Typst): the HeadingTitle external token approach
   was unreliable — even with contextual:true and canShift(), body text was
   being styled as headings. Remove HeadingTitle from the grammar entirely.
   Instead, a ViewPlugin (headingLinePlugin in languages/typst/index.ts)
   walks the syntax tree, finds HeadingMark nodes, and decorates the rest of
   the line with tok-heading class + bold. This is unconditionally correct
   because it is based on the syntax tree rather than the LR tokenizer state.

2. smooth_pdf_transition raw key shown in all locales: the key was in the
   JSON locale files but missing from extracted-translations.json, which is
   the allowlist the webpack translation loader uses to decide what to bundle.
   Add it there so all locales (including fr, es, de already added) resolve
   to their translated strings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 14:22:16 +00:00
claude f1abcaa4ce Hide LaTeX-only compile options for Typst/Quarto projects; add smooth_pdf_transition translations
Build and Deploy Verso / deploy (push) Successful in 9m35s
Draft compile mode and stop-on-first-error are LaTeX-only features not
supported by TypstRunner or QuartoRunner. Hide both sections from the
recompile dropdown for non-LaTeX projects. Also detect Quarto root files
(.qmd/.md/.Rmd) alongside Typst (.typ) to correctly set isLatexProject.

Add missing smooth_pdf_transition translations for French, Spanish, and
German (the English key already existed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:45:02 +00:00
claude 1c323351a2 fix: parse Quarto schema YAML errors and stop heading style bleeding
Build and Deploy Verso / deploy (push) Successful in 9m46s
Two unrelated fixes:

1. quarto-log-parser: handle the two-line Quarto schema-validation
   error format:
     ERROR: In file main.qmd
     (line 6, columns 24--27) Field "section-numbering" has value …
   Previously neither the file name nor the line number were extracted,
   so the error appeared without a red highlight. Now the first line
   stores the filename in pendingLocation and the second line creates
   the log entry with the correct file and line so the editor can jump
   to and highlight it.

2. headingTitleTokenizer: change contextual: false → contextual: true
   and guard with stack.canShift(HeadingTitle). With contextual: false
   Lezer calls the tokenizer speculatively at positions beyond the strict
   post-HeadingMark state; in some LALR-merged states the resulting token
   was accepted for body-text lines, making them render as bold-blue
   heading text. The contextual guard ensures the tokenizer only fires
   in the one state where HeadingTitle is legitimately valid.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:31:26 +00:00
claude 55e8208892 chore: add TODO.md with next-alpha Typst experience ideas
Build and Deploy Verso / deploy (push) Successful in 13m41s
Two ideas borrowed from the Collabst project (a Typst-native
collaborative editor): typst.ts WASM in-browser preview and Tinymist
LSP integration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 12:26:07 +00:00
claude b8543c8bb9 fix: capture typst diagnostics emitted after the status line
Build and Deploy Verso / deploy (push) Has been cancelled
typst watch outputs the "[HH:MM:SS] compiled with errors" status line
FIRST, then the full diagnostic output (file:line:col, source snippets,
hints) AFTERWARDS. The previous code resolved the pending compile
promise as soon as COMPILE_DONE_RE fired, discarding all post-status
diagnostic lines. Those lines then got cleared by the next cycle's
COMPILE_START_RE, so output.log only ever contained the bare status
line — explaining the "zero verbosity" symptom.

Fix: introduce a two-phase buffering model. When COMPILE_DONE_RE fires,
enter "post-done" phase (storing doneResult) and keep accumulating into
currentLines. _finalizeCompile() is called either when the next
COMPILE_START_RE arrives (zero added latency) or after FLUSH_DELAY_MS
(150 ms fallback for the last compile). It concatenates pre-done and
post-done lines before resolving, so output.log now contains the full
diagnostic output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 12:25:43 +00:00
claude 7e6c8c30cc fix: cache typst compile result to eliminate race-condition failures
Build and Deploy Verso / deploy (push) Successful in 14m6s
When typst watch detects a file change and compiles before the CLSI
resolver is registered (ResourceWriter writes files → typst compiles →
runTypst is called), _resolveAllPending was discarding the result
because pendingResolvers was empty. This caused two symptoms:

1. output.log only contained "compiled with errors" (no diagnostics)
   because the result carrying the full stdout was thrown away.

2. Every other manual compile failed with "compilation already gone"
   because the missed result caused a timeout, which killed the watcher
   and triggered a watcher restart cycle (success → miss → timeout →
   kill → restart → success → miss → ...).

Fix: when _resolveAllPending fires with no pending resolvers, store the
result in entry.pendingResult. _waitForNextCompile checks this field
first and resolves immediately if a cached result is present.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 11:48:46 +00:00
claude e0c717c131 fix: suppress interim compile errors while typing, show Typst error logs, dark-mode footer
Build and Deploy Verso / deploy (push) Successful in 1m43s
- local-compile-context: suppress failure/exited error state when
  changedAt > 0 (another compile is already queued), preventing the UI
  from flashing an error banner mid-typing that resolves moments later

- TypstRunner + CompileController: detect "compiled with errors" from
  typst watch and non-zero exit from typst compile, and signal
  status:'failure' to the frontend so the log panel opens automatically
  with the parsed error details (previously always returned 'success')

- footer.scss: add dark-mode overrides for footer.site-footer so the
  thin footer on project/marketing pages uses bg-dark-primary and
  content-primary-dark text in dark theme instead of hardcoded light bg

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 10:19:36 +00:00
claude 7a5218d472 typst: fix CodeIdent vs "_" token overlap after #keyword CallExpr?
Build and Deploy Verso / deploy (push) Failing after 8m41s
KeywordExpr { CodeKeyword CallExpr? } merges the post-keyword LR state
with document-level markup states, where "_" opens Emphasis.  CodeIdent
starts with identHead which includes "_", so the two tokens overlap.

Adding "_" after CodeIdent in @precedence resolves the conflict: CodeIdent
wins in the merged state (correct for '#set _name(...)'), and in pure markup
states CodeIdent is not in the valid set so "_" still opens Emphasis.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 09:16:35 +00:00
claude b16b096744 projection: visit overlay/mounted subtrees during tree iteration
Build and Deploy Verso / deploy (push) Successful in 22m39s
yamlFrontmatter() embeds the Markdown content as an overlay on the top-level
YAML-frontmatter tree.  The previous mode (IgnoreMounts | IgnoreOverlays)
skipped that overlay entirely, so ATXHeading nodes were never visited and the
Quarto (.qmd) file outline was always empty.

Dropping the mode flag lets the iterator descend into overlay and mounted
subtrees.  This is safe because every enterNode function already filters by
node name — visiting extra nodes from foreign-language mounts is a no-op.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 09:03:43 +00:00
claude e9cc63a261 typst: highlight function name after #set / #show keywords
Build and Deploy Verso / deploy (push) Has been cancelled
KeywordExpr now optionally includes a CallExpr, so '#set text(size: 12pt)'
parses 'text' as a CallExpr/CodeIdent and gets the function-name highlight
colour.  The optional CallExpr only shifts when the lookahead is CodeIdent,
so there is no shift/reduce conflict with other items.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 08:40:28 +00:00
claude 07c72cf7e5 typst: stop heading title at // or /* comment markers
Build and Deploy Verso / deploy (push) Successful in 9m24s
headingTitleTokenizer now stops reading when it encounters '//' or '/*',
so '= Heading // note' correctly produces a HeadingTitle token for 'Heading'
and a LineComment for the rest of the line.  Without this, the comment was
consumed into HeadingTitle, getting heading highlight and appearing verbatim
in the file outline.

Also strip trailing line comments from heading titles in the regex-based
document outline scanner, which reads raw text independently of the tree.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 08:27:02 +00:00
claude 4f98abbc5d classHighlighter: map function(variableName) to tok-function
Build and Deploy Verso / deploy (push) Has been cancelled
All cm6 themes define .tok-function but the classHighlighter had no entry
for tags.function(tags.variableName), so function-name tokens fell back to
tok-variableName (which themes leave unstyled).  This affected Typst function
calls (#func(...)) and would affect any future language that tags function
names with t.function(t.variableName).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 08:21:16 +00:00
claude 1dcd6e24f4 lezer-typst: convert LineCommentContent and MathContent to external tokenizers
Build and Deploy Verso / deploy (push) Successful in 10m10s
Both tokens are "read until delimiter" catchalls that match almost every
non-newline character, causing buildTokenGroups conflicts with every other
literal token in LALR-merged states.  Moving them to ExternalTokenizer (the
same pattern already used for HeadingTitle, RawBlockBody, etc.) makes them
context-isolated: the LR state machine only calls them when those tokens are
actually valid, so they never participate in the static token-group overlap
check.

Also exclude '<' from StrongText/EmphText so Label ('<' LabelName '>') is
recognised inside strong/emphasis spans rather than being consumed as plain
text.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 07:56:20 +00:00
claude e21f7cc0d5 fix(typst): resolve all overlapping-token errors via @precedence
Build and Deploy Verso / deploy (push) Has been cancelled
Lezer's buildTokenGroups rejects grammars with ambiguous token sets.
Eight overlaps existed:

  EscapeChar vs spaces       — EscapeChar { _ } matches \t; after '\'
                               it must win over the @skip spaces token.
  "(" / "." vs text tokens   — in the LALR-merged state after #CodeIdent,
                               callSuffix delimiters must beat
                               MarkupContent / StrongText / EmphText.
  "]" vs LineCommentContent  — inside #[...], the ContentBlock closer
                               must win even if it follows "//".

One extended @precedence declaration resolves all eight.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 07:41:03 +00:00
claude e4f5385e35 fix(typst): fix zero-length token error for LineCommentContent
Build and Deploy Verso / deploy (push) Has been cancelled
LineCommentContent { ![\n]* } matches the empty string, which Lezer
rejects as a zero-length token (infinite-loop risk). Change to ![\n]+
and mark it optional in the LineComment rule so empty // comments parse.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 06:54:56 +00:00
claude 2f3e3e7363 fix(typst): make HeadingTitle an external token to end LALR conflicts
Build and Deploy Verso / deploy (push) Has been cancelled
Any item shared between headingTitleItem and document-level item causes
a shift/reduce conflict: the LALR automaton merges the two contexts and
makes the shared token ambiguous. The only structural fix is to make
HeadingTitle a terminal (external tokenizer) that reads greedily to EOL,
giving the LR state machine a context-isolated token that can never
collide with document-level item tokens.

Removes headingTitleItem sub-rule, HeadingText token, and updates
styleTags to match HeadingTitle directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 21:14:18 +00:00
claude 94e8ff3503 fix(typst): eliminate LALR(1) conflict on heading title items
Build and Deploy Verso / deploy (push) Failing after 3h10m10s
Removing Strong and Emphasis from headingTitleItem eliminates the
conflict: both appear in document-level item, causing the LR automaton
to merge heading-title states with document-item states and make "*"
ambiguous (Strong opener vs. end of heading).

HeadingText is widened to ![\n$#`<@\\]+ so "*" and "_" inside headings
are consumed as plain text rather than producing error nodes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 20:58:05 +00:00
claude 26da1f6205 fix(editor): resolve HeadingTitle shift/reduce conflict in Typst grammar
Build and Deploy Verso / deploy (push) Has been cancelled
headingTitleItem* allowed an empty HeadingTitle, causing a shift/reduce
conflict: after HeadingMark, seeing "*" the LR parser couldn't decide
whether to shift it as a Strong inside the heading or reduce HeadingTitle
to empty and treat "*" as a document-level item.

Changing to headingTitleItem+ forces HeadingTitle to be non-empty, so
"*" after HeadingMark must be inside the heading. Empty headings are
handled by Lezer's error recovery.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 20:35:42 +00:00
claude 5287ea6f00 fix(editor): break Strong/Emphasis mutual recursion in Typst grammar
Build and Deploy Verso / deploy (push) Has been cancelled
Strong{strongItem{Emphasis}} and Emphasis{emphItem{Strong}} created a
mutual-recursion cycle that caused Lezer's LR automaton builder to
produce exponentially many states and crash.

Remove each construct from the other's item list. StrongText already
includes '_' and EmphText already includes '*', so nested delimiters
render as plain text inside the opposite construct rather than errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 20:14:33 +00:00
claude 045d458875 feat(editor): native Lezer grammar for Typst syntax highlighting
Build and Deploy Verso / deploy (push) Has been cancelled
Replace the StreamLanguage tokenizer with a full LR grammar compiled by
@lezer/generator, giving Typst the same parse-tree infrastructure that
LaTeX and BibTeX already use.

Grammar features:
- Headings (=, ==, …) via SOL-detecting external tokenizer
- Code expressions (#keyword, #func(args), #ident.method, #{…}, #[…])
- Named argument highlighting (key: value in function calls)
- Inline and display math ($…$)
- Strong (*…*) and emphasis (_…_) with bold/italic formatting
- Raw blocks (```lang…```) and inline raw (`…`)
- Nested block comments (/* /* */ */) via depth-tracking external tokenizer
- Labels (<name>) and references (@name)
- Backslash escapes

Infrastructure changes:
- lezer-typst/typst.grammar — new Lezer grammar
- lezer-typst/tokens.mjs — external tokenizers for context-sensitive lexing
- scripts/lezer-latex/generate.mjs — Typst added to grammars array so the
  existing lezer-latex:generate script (and Dockerfile step) compile it
- .gitignore — generated typst.mjs / typst.terms.mjs excluded from git

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 19:49:17 +00:00
claude 2c0f387cef feat(editor): use @codemirror/lang-yaml for Quarto YAML frontmatter
Build and Deploy Verso / deploy (push) Successful in 12m20s
Replace the custom regex-based ViewPlugin with the official
@codemirror/lang-yaml package. yamlFrontmatter({ content: mdLS })
wraps the Markdown language with a mixed parser: the leading ---/---
block is handed to the full Lezer YAML parser (proper key/value/scalar/
anchor/alias highlighting), while the document body continues to use
the Markdown parser. The manual Frontmatter extension import is also
removed since yamlFrontmatter handles frontmatter recognition itself.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 18:30:35 +00:00
claude f9788a1c69 feat(editor): improve syntax highlighting for Typst and Quarto documents
Build and Deploy Verso / deploy (push) Successful in 10m50s
Typst: heading tokenizer now colors the entire heading line (not just the
'=' prefix), and bold/italic markers (*/_) map to strong/emphasis tags
rather than the generic operator tag. A typstHighlightStyle applies
bold/italic formatting even when the active theme lacks .tok-heading.

Quarto: enable @lezer/markdown's Frontmatter extension so the YAML header
is no longer mis-parsed as Setext headings. A new ViewPlugin decorates
frontmatter lines with type-appropriate CSS classes: keys (tok-typeName),
string/bool/number values, comments, and the --- delimiter markers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 17:42:51 +00:00
claude 489bdb01ec feat(pdf): dark mode for Quarto RevealJS HTML output
Build and Deploy Verso / deploy (push) Successful in 10m36s
Apply the same CSS inversion filter to the HTML iframe as is already
applied to pdfjs PDF pages, so Quarto RevealJS presentations respect
the dark mode toggle. Also show the theme button for HTML outputs and
relax the darkModePdf condition to activate for iframes regardless of
the pdfViewer setting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 17:18:29 +00:00
claude 0b8897540d Fix Quarto RevealJS media missing on second compile
Build and Deploy Verso / deploy (push) Successful in 11m49s
When output.html exists, findOutputFiles includes project media files
(images, videos) in outputFiles via the MEDIA_REGEX exception so they
get served from the cache.  _removeExtraneousFiles then treated them
as extraneous and deleted them.  On the next incremental compile,
unchanged binary files are not re-synced, so the files were gone when
Quarto ran and when _appendMissingResourceWarnings checked for them.

Fix: skip deletion for any file that is a project input resource.
Those files appear in outputFiles to be served, not cleaned up.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 15:30:08 +00:00
claude 2ead377ebc Fix stale error lines bleeding into next Typst compile log
Build and Deploy Verso / deploy (push) Failing after 36m24s
When typst watch doesn't emit "compiled with errors" after a failed
compile, currentLines accumulates indefinitely. The next successful
compile then flushes the buffer including the stale error from the
prior cycle. Reset currentLines at the start of each compile cycle
("[HH:MM:SS] compiling ...") so each log only contains output from
one compile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 15:10:59 +00:00
claude 5a85e1b9d8 Add smooth PDF transition toggle to compile settings
Build and Deploy Verso / deploy (push) Successful in 11m53s
Per-project-type setting: Typst defaults to on, LaTeX defaults to off.
Toggle appears in the compile dropdown under "Smooth PDF transition".
The enableTransition flag is read via a ref so toggling does not
reload the current PDF.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:19:37 +00:00
claude 165219dcb1 fix(autocompile): prevent compile chaining — wait for previous compile before starting next
Build and Deploy Verso / deploy (push) Successful in 11m38s
The auto-compile effect was calling debouncedAutoCompile() on every changedAt
update (every keystroke), including while a compile was already running.  With
a 1000ms maxWait the debounce fired every second even mid-compile, chaining
compiles back-to-back and making the user wait for all of them to drain.

Fix: add `compiling` to the effect's dependency array.
- While compiling: the effect cancels the debounce immediately, preventing
  any new compile from being queued.
- When compile finishes (compiling → false): the effect re-runs; if changedAt
  is still > 0 (changes were made during the compile), it re-arms the debounce
  exactly once.  One follow-up compile, then idle.

Also remove the debouncedAutoCompile() re-queue from compiler.ts's
wasCompiling guard — the effect now owns that responsibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 13:38:43 +00:00
claude 71755e5cee fix(pdf): replace document.startViewTransition with non-blocking canvas fade
Build and Deploy Verso / deploy (push) Successful in 14m22s
document.startViewTransition with an async callback places a ::view-transition
overlay on top of the entire page, intercepting pointer events for the duration
of the callback (up to the 1s safety timeout + 250ms animation).  With rapid
auto-compiles this created interface freezes and overlapping transitions that
could leave the visual lock in a broken state, causing 'stuck on compiling'.

Replace with a canvas snapshot overlay + CSS opacity fade-out:
- pointer-events:none so the overlay never blocks input
- snapshot covers the canvas-clear from setDocument() (no white flash)
- on pagerendered: opacity transitions to 0 over 250ms, then overlay removed
- gives the same smooth visual crossfade, reliably, in all browsers

Chrome 126+ retains the element-level startViewTransition path which is
scoped to the PDF container and does not affect the rest of the page.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 13:09:18 +00:00
claude 453439e611 fix(pdf): suppress root view-transition animation to isolate fade to PDF pane only
Build and Deploy Verso / deploy (push) Successful in 11m47s
document.startViewTransition generates both a named pseudo-element for the PDF
container (ol-pdf-viewer) and a root-level pseudo-element that covers the entire
page, causing the editor to fade along with the PDF.

Inject a temporary <style> that sets animation:none on the root pseudo-elements
before starting the transition, then remove it in transition.finished.  Only the
named PDF container crossfades; the editor is unaffected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:39:01 +00:00
claude d895e14e48 feat(pdf): restore smooth crossfade for Chrome 111+ using document.startViewTransition
Build and Deploy Verso / deploy (push) Successful in 11m24s
The original code used container.startViewTransition(setDocument) with a
synchronous callback, giving a 250ms CSS crossfade that looked smooth when
the PDF happened to re-render before the animation ended — but was a race.

Now there are three tiers:
- Chrome 126+: element-level startViewTransition, async, waits for pagerendered
- Chrome 111+ (Brave 138, Edge 111+): document-level startViewTransition with
  view-transition-name scoped to the PDF container, same async pattern
- Firefox / Safari / older Chromium: canvas snapshot overlay (no animation,
  but seamless — introduced in build #108)

The document-level path restores the smooth fade the user saw on Edge build
#93, now guaranteed to crossfade old→new rather than old→blank.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:22:46 +00:00
claude 4410a83146 fix: eliminate too-recently-compiled error and PDF flicker on fast Typst compiles
Build and Deploy Verso / deploy (push) Successful in 11m25s
Rate limit: auto-compile requests already have a client-side debounce; skip
the 1-second server-side recently-compiled gate for them to avoid spurious
'too-recently-compiled' rejections that were blocking ~1/3 of Typst compiles.

PDF flicker: add _snapshotCanvases() fallback for browsers without element-level
View Transitions (Chrome <126, Firefox, Safari).  Before setDocument() clears the
canvases it copies each rendered page to a positioned overlay; the overlay is
removed once the first page of the new document fires pagerendered, giving a
seamless old→new swap in all browsers.  Chrome 126+ continues to use the
startViewTransition async callback path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:00:01 +00:00
claude db162e54af fix(typst): correct auto-compile default and debounce detection for Typst projects
Build and Deploy Verso / deploy (push) Successful in 11m37s
The previous implementation used useState() to detect the project type, but the
file tree is loaded asynchronously after the WebSocket joinProject event, so
pathInFolder() always returns null on the initial render.

Use useEffect() instead — it re-runs when getRootDocInfo's reference changes
(i.e. when the file tree populates), correctly detecting .typ root docs.
Also adds updateAutoCompileDebounce() to DocumentCompiler so the tight
debounce can be applied at that point.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 11:37:28 +00:00
claude 228ad00075 feat(typst): auto-compile on by default with fast debounce + smoother PDF transitions
Build and Deploy Verso / deploy (push) Successful in 14m23s
- Typst projects default autocompile to enabled (300ms debounce / 1s max-wait
  instead of 2.5s/5s), so the PDF refreshes nearly as the user types.
- Make startViewTransition wait for the first page to render before completing
  the crossfade, eliminating the old-PDF→blank flash on Chrome 126+.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 10:40:10 +00:00
claude 7eaeaedcd8 Implement persistent typst watch for incremental compilation
Build and Deploy Verso / deploy (push) Successful in 59m27s
Instead of cold-starting 'typst compile' on every request, TypstRunner
now maintains a long-lived 'typst watch' process per project. Subsequent
compiles reuse the warm process, which caches fonts, packages, and the
compiled AST via Typst's comemo framework — dramatically faster.

Architecture:
- WatchTable: maps compileName → live watcher process + state
- _startWatcher: spawns 'typst watch input.typ output.pdf', registers
  stdout/close handlers, then immediately awaits the first compile result.
  The resolver is pushed to pendingResolvers synchronously inside the
  Promise constructor before any I/O event can fire — eliminating the
  race between file-write detection and resolver registration.
- _onWatcherData: parses stdout line-by-line, resolves pending callers
  on "compiled successfully/with warnings/with errors" (the three terminal
  lines typst watch emits at the end of each compile cycle).
- Graceful restart: watcher is restarted after MAX_COMPILES_BEFORE_RESTART
  (1000) cycles to stay clear of Typst's ~65k FileId limit, or immediately
  if the "ran out of file ids" message is detected in stdout.
- killTypst: tears down both the watcher and any cold-start fallback job;
  called by stopCompile (user-initiated) and clearProject/clearProjectWithListing
  (before compile-dir deletion).
- Docker fallback: Settings.clsi.dockerRunner=true falls back to the
  original cold-start 'typst compile' path unchanged.
- process.on('exit') kills all watcher process groups on CLSI shutdown.

CompileManager: call TypstRunner.promises.killTypst before deleting the
compile directory in both clearProject and clearProjectWithListing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 09:09:33 +00:00
claude 54c510c818 Revert Typst SyncTeX attempt; clean up diagnostic logging
Build and Deploy Verso / deploy (push) Has been cancelled
Typst has no --synctex CLI option (open feature request #289 since 2023).
Revert the frontend guard back to LaTeX-only and remove --synctex from
the Typst compile command. Also remove the temporary logger.warn calls
added for diagnosing the LaTeX synctex issue (now resolved).

The official Typst binary installation in Dockerfile-base is kept as it
is cleaner than using Quarto's modified fork for .typ compilation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 08:43:18 +00:00
claude 5796c0157c Install official Typst binary and use it for .typ compilation
Build and Deploy Verso / deploy (push) Has been cancelled
Quarto bundles a modified Typst fork that lacks --synctex, making
bidirectional sync impossible. Install the official Typst binary
(v0.13.1) from upstream and use it in TypstRunner instead.

This also means .typ projects now use the unmodified Typst compiler,
which is correct since TypstRunner handles plain .typ files (not .qmd).
QuartoRunner continues to use Quarto's bundled Typst internally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 08:40:48 +00:00
claude 3f68c147a4 Call Typst binary directly for compile and SyncTeX support
Build and Deploy Verso / deploy (push) Successful in 13m8s
Instead of going through 'quarto typst compile' (which intercepts
--synctex before it reaches Typst), call the Typst binary bundled in
the Quarto .deb directly at /opt/quarto/bin/tools/x86_64/typst.

This allows passing --synctex output.synctex.gz to generate the SyncTeX
file for bidirectional editor↔PDF sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 08:24:20 +00:00
claude 0780963bc7 Fix --synctex argument order for Typst compile
Build and Deploy Verso / deploy (push) Successful in 11m33s
Typst's CLI requires options before positional arguments (INPUT OUTPUT).
Placing --synctex after output.pdf caused it to be treated as an extra
positional arg and rejected with 'unexpected argument'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 08:09:52 +00:00
claude 43a622cd71 Add SyncTeX support for Typst projects
Build and Deploy Verso / deploy (push) Successful in 11m58s
- TypstRunner: add --synctex output.synctex.gz to quarto typst compile,
  generating a synctex file alongside the PDF (requires Typst 0.11+,
  bundled in Quarto 1.5+).
- use-synctex: extend the root-doc guard from LaTeX-only to also cover
  .typ files, enabling the Show in PDF / Show in code buttons for Typst.

The rest of the sync infrastructure (OutputCacheManager, synctex binary,
SynctexOutputParser, CLSI routes) is already format-agnostic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 21:52:05 +00:00
claude 9079b545f7 Switch TeX Live from scheme-basic to scheme-full
Build and Deploy Verso / deploy (push) Successful in 50m16s
Replaces the minimal scheme-basic install (plus explicit latexmk,
texcount, synctex additions) with scheme-full, giving users access
to the complete LaTeX package ecosystem without manual tlmgr installs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 20:55:06 +00:00
claude eb45ececf0 Install synctex binary via tlmgr for SyncTeX support
Build and Deploy Verso / deploy (push) Has been cancelled
The synctex binary was not included in scheme-basic and was not
explicitly installed, causing `spawn synctex ENOENT` on every
sync request. Add it alongside latexmk and texcount.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 20:53:28 +00:00
claude e6add1e6f0 Add diagnostic logging to synctex to identify failure cause
Build and Deploy Verso / deploy (push) Successful in 9m42s
Logs: request params, directory used, whether output.synctex.gz
is found, and the actual synctex binary output or error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 16:21:35 +00:00
claude 170818e6fc fix(synctex): gate sync buttons to LaTeX-only projects
Build and Deploy Verso / deploy (push) Successful in 11m0s
Verso added 'qmd' and 'typ' to validRootDocExtensions, which caused
isValidTeXFile() to return true for Typst/Quarto files — enabling
SyncTeX UI controls for projects that never produce output.synctex.gz.

Replace the open-doc extension check in canSyncToPdf with a
LaTeX-only regex on the project root document path (tex|ltx|Rtex|Rnw),
and add the same guard in _syncToCode so PDF-click sync never fires
an API request for non-LaTeX projects.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:50:02 +00:00
claude 9ea904f78f Merge upstream Overleaf up to PR #34297 (68 commits)
Build and Deploy Verso / deploy (push) Successful in 11m30s
Conflicts resolved:
- fat-footer-website-redesign.pug: keep Verso footer (discard Overleaf marketing footer)
- MaterialSymbolsRoundedUnfilledPartialSlice.woff2: regenerated from merged
  unfilled-symbols.mjs (preserves Verso's deployed_code + adds upstream's spellcheck)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 13:39:32 +00:00
roo hutton 757735b075 Merge pull request #34297 from overleaf/rh-prev-plan-type-cancel
Update previous_plan_type on subscription expiry

GitOrigin-RevId: 19381e5516fdbfd2650a9a5b94b61791e0da909f
2026-06-05 08:07:27 +00:00
Mathias Jakobsen fa36cd508b Merge pull request #34310 from overleaf/mj-handle-lazy-errors-for-search-and-share
[web] Handle errors while loading full project search and share modal

GitOrigin-RevId: 29d863324a54fa872022002f612498335f88f377
2026-06-05 08:07:10 +00:00
Mathias Jakobsen b7735d402d Merge pull request #34195 from overleaf/mj-command-palette-menu-labels
[web] Add menu labels to commands, and add more commands to command palette

GitOrigin-RevId: 21e17142bb3112b5fdcda85a472122b011979f49
2026-06-05 08:07:06 +00:00
Mathias Jakobsen cabe0046c5 Merge pull request #34102 from overleaf/mj-document-import-errors
[web] Expose pandoc errors in import

GitOrigin-RevId: 55f89b91a52099a99a5d955bc05f3657b87b2cdc
2026-06-05 08:07:02 +00:00
Anna Claire Fields 97247b8ea5 [PnP migration] Remove mock-fs dependency (#33835)
GitOrigin-RevId: ff8df32d85b2ecd2837c9eee6d6d2b3b95285239
2026-06-05 08:06:54 +00:00
Anna Claire Fields 3fcd133198 [patch] update sandboxed-module for Yarn PnP compatibility and add mongodb-legacy type definitions (#33983)
GitOrigin-RevId: 8f1e9a4e4b4b5fbf3a770951a070b5a259abdcee
2026-06-05 08:06:50 +00:00
Anna Claire Fields 44dee7592a use require.resolve for mocha reporter paths (#34235)
GitOrigin-RevId: af607dfdeac8f91f63db294a964ade7622225932
2026-06-05 08:06:46 +00:00
Anna Claire Fields bfcf75855a [PnP migration] Convert .prettierrc to .prettierrc.cjs with require.resolve (#34237)
GitOrigin-RevId: ab57ca143bca8bfd2b44f03f9712a1aae70b2c1c
2026-06-05 08:06:42 +00:00
Antoine Clausse 3140e46e68 [web] Replace token-link email verification with 6-digit code on SSO registration (ORCID) (#33889)
* Replace token-link email with 6-digit code on SSO registration

Unverified SSO emails previously received a long-lived token link
(90-day TTL) via UserEmailsConfirmationHandler. This replaces that
flow with the same 6-digit code verification used for password
registration, redirecting through /registration/confirm-email.

- SSOManager.registerSSO now always confirms email (caller must
  verify first); removes sendConfirmationEmail / _finishRegistration
- SSOController._signUp sends confirmation code and stores
  pendingSSORegistration in session when IdP email_verified is false
- New SSOConfirmEmailHandler completes registration after code check
  via completeSSOEmailConfirmation module hook
- OnboardingController confirm-email handlers accept
  pendingSSORegistration alongside pendingUserRegistration

confirmEmailFromToken (POST /user/emails/confirm) removal is deferred
to a follow-up PR to avoid breaking in-flight 90-day tokens.

Closes #28607

* Fix unverified-email edge cases; Add ORCID e2e tests;

* Rename `confirmEmail` parameter to `emailVerifiedByIdP` in _signUp function

* Remove `sendConfirmationEmail`

* Mock getUserByAnyEmail in tests

* Extract _finishSSORegistration helper to deduplicate the register →
set session flags → allocate referral → finishSaasLogin → finishLogin
sequence shared by both the direct and deferred (code-confirmed) paths.

* Stop duplicating session data in pendingSSORegistration

analyticsId, splitTests, and referal_* are already in the session at
confirmation time — no need to copy them into pendingSSORegistration.
Re-fetch splitTests fresh on completion instead.

* Simplify the code

* Remove dead confirmEmail template

No callers remain after sendConfirmationEmail was deleted. The token-link
flow (confirmEmailFromToken) only validates tokens, never sends email.

* Remove dead reconfirmEmail template

* Address comments from Copilot

* Clear stale pending registration when starting a new flow

* Add unit tests for completeSSOEmailConfirmation

* Add `verificationMethod` param

* Fix camelcase issues

* Extract _createSSOUser and _registerAndFinish helpers to deduplicate registration logic

* Remove obscure "registration_error"

* Prevent FormTextIcon from shrinking

* Enable "email_already_registered_sso" error

* Misc. improvements to confirm-email-form.tsx

* Remove `UserEmailsConfirmationHandler` mock

Co-authored-by: Olzhas Askar <olzhas.askar@overleaf.com>

* Add info on sso_email.pug page

---------

Co-authored-by: Olzhas Askar <olzhas.askar@overleaf.com>
GitOrigin-RevId: d0196ebc6d81ff61bcd27726d0b899b743d08d64
2026-06-05 08:06:34 +00:00
Daniel Kontšek 2570b6559d Merge pull request #34112 from overleaf/dn0-migration-check-predeploy-hook
Add predeploy migration gate for Mongo-bundling services

GitOrigin-RevId: d9eb192ea32b5328fb24fd453ddb1370f373858e
2026-06-05 08:06:30 +00:00
Davinder Singh 6ce36a2606 adding web changes of Export HTML (#34117)
GitOrigin-RevId: 804c576faefebfc6683a0363b45372e66a43d8fc
2026-06-05 08:06:19 +00:00
Jakob Ackermann fc2abf5b24 [web] fix submit modal in Codespaces (#34137)
GitOrigin-RevId: dc057ed736e97265a901b1cf21995c1f391339a5
2026-06-05 08:06:15 +00:00
Malik Glossop b8fc478e1f Merge pull request #34185 from overleaf/worktree-mg-error-assist-paywall
Show paywall from gutter when user hits suggestion limit

GitOrigin-RevId: 36c09e3d93ac38e1e675aa8ffb419e928094d68e
2026-06-05 08:06:11 +00:00
Malik Glossop d25b032e16 Merge pull request #33450 from overleaf/worktree-mg-writefull-spelling-tab
Add writefull language suggestions section to Spelling and language tab

GitOrigin-RevId: 6195683ca175a4c3da25a7ab334a605c67db04b8
2026-06-05 08:06:07 +00:00
Mathias Jakobsen fc31a88767 Merge pull request #34145 from overleaf/ds-download-html-using-pandoc-clsi-1
[CLSI] Download as HTML feature

GitOrigin-RevId: 374101c1f957a00eda423a6be0363c08b5de7a95
2026-06-05 08:06:03 +00:00
Jakob Ackermann 0501586743 [latexqc] migrate to local s3, add codespaces support, add e2e tests (#34136)
GitOrigin-RevId: 167171103c14ed3c4ba2939d80231c343645e53a
2026-06-05 08:05:59 +00:00
Jakob Ackermann df61bfc788 [clsi] initial version of /convert/pdf-to-jpeg (#33752)
* [monorepo] consolidate clsi-lb host/ip env-vars

Target env-var is CLSI_LB_HOST. Keep CLSI_LB_IP populated for a week.

* [clsi] initial version of /convert/pdf-to-jpeg

* [rails] use fake-secrets in CI and Codespaces

* [rails] adapt tests for using clsi to convert PDFs to image

* [rails] add rake task for comparing clsi conversion with transloadit

* [clsi] double check that output.jpg is a regular file

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

* [clsi] fix composing basename

* [monorepo] fix clsi-lb host env-var post merge

* [monorepo] sort dev-environment.env hosts

* [rails] use local pdf file rather than downloading it again

Download from the old renderer code path still. It's dead code.

* [terraform] clsi: enable pdf to jpg conversion

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 5ecaa8559d299486340bb3961f06b29f7c4dfcca
2026-06-05 08:05:55 +00:00
Brian Gough 9ec0ff065d add missing mongo dependencies (#34298)
* add missing mongo dependency for analytics

* update build scripts for analytics

* add missing mongo dependency for third-party-datastore

* update build scripts for third-party-datastore

* add missing mongo dependency for third-party-references

* update build scripts for third-party-references

* update yarn.lock for buildscript changes

GitOrigin-RevId: 1c42e49af5075529a334d50648da990e4cedb1b4
2026-06-05 08:05:50 +00:00
Olzhas Askar 8e36f20950 Merge pull request #34267 from overleaf/oa-move-upgrade
[web] Moving the upgrade button

GitOrigin-RevId: 33dcdcfa4e816e29177abe2c045e919edd7a4e08
2026-06-04 08:07:21 +00:00
roo hutton 06e99fe62a Merge pull request #34130 from overleaf/rh-enterprise-cio
Expose enterprise indicators and previous_plan_type for first subscriptions to customer.io

GitOrigin-RevId: 693db7f796609f00ecd31216a6d6be32c1f569c8
2026-06-04 08:07:09 +00:00
Maria Florencia Besteiro Gonzalez d112271b1c Merge pull request #34184 from overleaf/cs-icon-button-labs-library
feat(library): add Labs feedback badge to Library heading

GitOrigin-RevId: 6dacc588cc58300a09b8195ca800d042d40f4c89
2026-06-04 08:06:52 +00:00
Antoine Clausse 0658bd9a31 [web] Change plans order in Change Plan modal (#34096)
* [web] Order plans in Change Plan modal consistently

Reorder the plans returned by `buildPlansListForSubscriptionDash` so the
Subscription page "Change plan" modal lists them top-to-bottom as:

  1. Student annual
  2. Student monthly
  3. Standard monthly
  4. Standard annual
  5. Pro monthly
  6. Pro annual

Previously `buildPlansList` produced three per-period buckets which the
dash function concatenated, giving an order that flipped per family.
Replace that with an explicit `CHANGE_PLAN_MODAL_PLAN_CODES` list so the
order matches the Design QA spec at a glance. The now-unused
`studentAccounts`, `individualMonthlyPlans`, `individualAnnualPlans`,
`groupMonthlyPlans`, and `groupAnnualPlans` buckets are dropped from
`buildPlansList` (no other callers).

Closes #34024

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [web] Update personal-plan acceptance test for new buildPlansList shape

The previous test asserted `buildPlansList().individualMonthlyPlans`,
which no longer exists after the change-plan modal reorder dropped the
per-period buckets. Move the assertion to
`buildPlansListForSubscriptionDash()`, which is where the personal-plan
exclusion is now enforced (via `CHANGE_PLAN_MODAL_PLAN_CODES`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [web] Drop now-dead client-side plan filter

`IndividualPlansTable` used to filter out `paid-personal`,
`paid-personal-annual` and `institutional_commons` defensively because
the old `buildPlansListForSubscriptionDash` returned every non-group
plan that wasn't `hideFromUsers`. The previous commit pins the modal to
an explicit six-plan list (`CHANGE_PLAN_MODAL_PLAN_CODES`), so none of
those plan codes ever reach the frontend and the filter is dead. Remove
it and the now-unused `useMemo` import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Revert "[web] Drop now-dead client-side plan filter"

This reverts commit 83e8448f2cfa2c68e44b749d5a2bc350a7443c6d.

We'll do that in a later cleanup

* Swap "Student monthly" and "Student annual" for consistency

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 046a235e14e7ad6622288f5a5a723f5a4f7f14da
2026-06-04 08:06:40 +00:00
Antoine Clausse b07d141397 [web] Fix /user/subscription/plans#ai-assist redirects (#34124)
* [web] Redirect missing AI add-on purchase to subscription dashboard

The two error paths in `previewAddonPurchase` redirected to
`/user/subscription/plans#ai-assist`, but the `#ai-assist` anchor was
removed when the AI Assist add-on was retired, so users land at the top
of the plans page with no context. Align both with the other error
branches in the same function and the `plans-2026-phase-1` enabled
branch, which already redirect to
`/user/subscription?redirect-reason=ai-assist-unavailable` — the
subscription dashboard shows the matching warning alert
(`redirect-alerts.tsx`).

Update the acceptance test to match the new redirect target.

Closes #34074

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [web] Update ai-assist-unavailable warning to reflect bundled AI features

The previous copy said "AI Assist isn't available to you due to your
current subscription type", which read as a hard block. Now that the AI
Assist add-on has been retired and AI features are included with every
paid plan, the warning should point users to the pricing page instead of
implying their plan can't access AI at all.

Keep the existing translation key for now — a follow-up can rename it
once #33624 (AI page CTA destination) is resolved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [web] Link the ai-assist-unavailable warning to the pricing page

* [web] Rename key `ai_assist_unavailable_due_to_subscription_type` -> `ai_assist_unavailable`

* [web] Update french and german translations

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: ae1319fa5b857d8f292de77c82ef0bda1c7ad144
2026-06-04 08:06:31 +00:00
Jakob Ackermann 6869ad5bdf [misc] remove HTTP method override capability (#34243)
GitOrigin-RevId: 2d88b9cdb63c7861e0604bb19d0e0c924701f3e4
2026-06-04 08:06:22 +00:00
Domagoj Kriskovic 9cf1085fbb [web] use updateProject for saving trackChangesState
GitOrigin-RevId: eecb2b78ff18547e8b3653fdff2d380d295c367f
2026-06-04 08:06:14 +00:00
Domagoj Kriskovic ea57ae9125 Rename sourceEditorVisualExtensions to sourceEditorMarkdownExtensions
GitOrigin-RevId: a242742c3844cccb355d4a98eb27b74123ad107e
2026-06-04 08:06:09 +00:00
Domagoj Kriskovic 5cf1b43ce7 Add Markdown visual editor support
GitOrigin-RevId: 4ec2ffb276c729a58f82ccb26ed571f4187a4178
2026-06-04 08:06:04 +00:00
Chris Dryden e38f4e18e4 Merge pull request #33868 from overleaf/dk-package-loading-tests
[web] Add tests for pyodide worker streams and output pane rendering

GitOrigin-RevId: 41ffc25230be23d68d50c61980cfaf1260a0247d
2026-06-04 08:06:00 +00:00
Liam O'Brien f1282ee5cd Helper script for changing expiry of git pat (#34234)
* Helper script for changing expiry of git pat

* Validation fail for invalid date

GitOrigin-RevId: 6786d4e808e0e4e87ef1293f4c22236257948128
2026-06-04 08:05:51 +00:00
Copilot a2f72adf67 Remove useSecondary from backup worker to fix false positive errors during MongoDB failover (#34186)
* Initial plan

* Remove useSecondary from configureBackup to fix false positive backup errors

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>
GitOrigin-RevId: f7c9bb88fb2f7526948fee196f0444bd33a96e56
2026-06-04 08:05:47 +00:00
Brian Gough 0da93aaab3 add script to finalise broken history-v1 chunks (#34005)
* add script to finalise broken history-v1 chunks

* use history-id instead of project-id

* update project-id to history-id in tests

* silence unwanted event emitter warnings

* fix up test for historyId

GitOrigin-RevId: 58d2a768f1eff296e921e2ed985f6faf3929f619
2026-06-04 08:05:42 +00:00
Liam O'Brien e53c6f2aea Notify users about expiring git PATs and expose PATs in admin panel (#33802)
* Allow admin access to user PATs

* Tests for new screen in admin panel

* Adding error for invalid token and way to parse error for OAuth 2

* Git bridge handles expired PAT

* Script for alerting on close to expiry and expired git tokens

* Refactoring and simplifying

* Updating email templates to match agreed docs

* tweak to email subject to include Overleaf

* Allowing dry run in scripts and general tidy up

* removing redundant tests and dry running script

* Fixing CI errors

* Adding new tab to admin test expectation

* Address PR feedback on oauth2-server changes

- Replace ad-hoc overleafErrorCode prop with a TokenExpiredError subclass
- Collapse listTokens/listTokensForAdmin into a single hook

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Adding cron definitions for alerting on expiring git pat

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
GitOrigin-RevId: 69b9fd901a201592a580c69abe7bd7d603e85d3a
2026-06-04 08:05:26 +00:00
jmescuderowritefull a553a8390d Clean 'writefull-keywords-generator' flag (#34200)
GitOrigin-RevId: 3014f02eba721b002eb35ec81750252993597748
2026-06-03 08:07:27 +00:00
Mathias Jakobsen 5ad548e7d7 Merge pull request #34199 from overleaf/mj-tabs-divider-tweaks
[web] Drop dividers next to active tab

GitOrigin-RevId: 9610e22b0aa7f036233108282687772c30f4c1b0
2026-06-03 08:07:06 +00:00
Maria Florencia Besteiro Gonzalez 021b2e305c Merge pull request #34108 from overleaf/mfb-show-warning-of-duplicate-citation-keys
Show duplicate citation keys as a warning beside the relevant entry

GitOrigin-RevId: e8506b2d77febec6d269a242f6d9b237171db66f
2026-06-03 08:06:44 +00:00
Mathias Jakobsen cc762bb7e6 Merge pull request #33994 from overleaf/mj-command-palette-synctex
[web] Add synctex to command palette

GitOrigin-RevId: 10e769dae6088d279d010fcfa3577b489c6ff89c
2026-06-03 08:06:40 +00:00
Brian Gough f8c7e092fa upgrade to eslint v10 (#34054)
* upgrade from eslint version 8 to eslint version 10

* remove unsupported eslint-env directive

* include jsx files in latexqc linting

* use basePath and extends to maintain paths in writefull eslint

* fix yarn.lock

with ./bin/yarn install

* preserve existing glob patterns in web eslint config

* restore original comments

* fix worker path

* corrected comment about eslint-plugin-mocha

* remove unused imports

* remove unused import of includeIgnoreFile

* switch to individual eslit.config.mjs files

* fix lint errors on eslint.config.mjs in web

* update build scripts for eslint.config.mjs

* update volumes for RUN_LINTING_CI_MONOREPO in web Makefile

updated manually as this makefile is not autogenerated
the RUN_LINTING_CI_MONOREPO command is only used for prettier, not eslint, but updating for consistency.

* migrate from mocha/no-skipped-tests to mocha/no-pending-tests

see https://github.com/lo1tuma/eslint-plugin-mocha/pull/365
"rule no-skipped-tests has been removed, its functionality has been merged into the existing no-pending-tests rule"

GitOrigin-RevId: 2c8f25c8049a0dba374a51df1214286bb5093a51
2026-06-03 08:06:29 +00:00
Mathias Jakobsen 98bd09c31d Merge pull request #34189 from overleaf/mj-fix-flaky-review-panel-tests
[web] Fix flaky <ReviewPanel /> Cypress tests

GitOrigin-RevId: b34dc9a0ca53da5a282513e8fb92297e4b2f702a
2026-06-03 08:06:17 +00:00
Alf Eaton 78bea8d574 Use Emulation.setFocusEmulationEnabled in Cypress (#33787)
GitOrigin-RevId: d3b9ba1b2362bdb23dbf8282514c972c52c83fec
2026-06-03 08:06:04 +00:00
Alf Eaton 979f065581 Upgrade to MathJax v4 (#15030)
GitOrigin-RevId: d1536bce67286da23e15aa18eb525dd83859978b
2026-06-03 08:05:55 +00:00
Andrew Rumble b6451d5bb0 Merge pull request #34179 from overleaf/ar-analytics-upgrade-development-postgres
[analytics] upgrade test postgres version to 14

GitOrigin-RevId: fac9d063c2572d4393d885554ef688876c300c29
2026-06-03 08:05:50 +00:00
Lucie Germain d52b5ae141 [Security upgrade] bump brace-expansion to 5.0.6 (#33915)
* Bump brace-expansion to 5.0.6 in linked-url-proxy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* drop unnecessary brace-expansion resolution; ^5.0.5 already permits 5.0.6

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
GitOrigin-RevId: 837dcd88e5e0a6181d3ac2fe4f512a6ec1904002
2026-06-03 08:05:41 +00:00
Jakob Ackermann ec84a88eb3 [server-ce] tests: build git-bridge latest locally in CI (#34177)
GitOrigin-RevId: 78bf654eee77a62185a53b1abdf2cd9ae0662802
2026-06-02 08:08:39 +00:00
Mathias Jakobsen 96e0830eef Merge pull request #34167 from overleaf/mj-conversion-error-update
[web] Point conversion errors to docs page

GitOrigin-RevId: 1a5208065252159b6a69bc6ae4cecae1dd0cd4d8
2026-06-02 08:08:31 +00:00
Copilot a9a9f6ee6b Migrate history-v1 recover_zip scripts from archiver to zip-stream (#32813)
* migrate recover_zip_from_backup from archiver to zip-stream

Replace the `archiver` package with `zip-stream` (the lower-level library
that `archiver` wraps) in the `recover_zip_from_backup.mjs` script and
`backupArchiver.mjs` library. The `archiver` package has known issues with
hanging when creating large zip files and is no longer actively maintained.

Changes:
- Add `zip-stream@^7.0.2` as a direct dependency
- Update `backupArchiver.mjs` to use promisified `ZipStream.entry()`
  instead of `Archiver.append()`
- Rewrite `recover_zip_from_backup.mjs` to use `ZipStream` with
  `stream/promises.pipeline` for cleaner async flow
- Keep `archiver` dependency for `project_archive.js` (separate code path)

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/0df27a8b-97f1-43cc-ac26-f5247a84313f

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* extract finalize timeout to named constant

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/0df27a8b-97f1-43cc-ac26-f5247a84313f

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* convert recover_zip.js to zip-stream, remove finalize timeout, add verbose logging

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/9380d08a-d813-4e9f-a2ac-4891122c163b

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* add acceptance tests for recover_zip_from_backup in raw and latest modes

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/9380d08a-d813-4e9f-a2ac-4891122c163b

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* fix comment formatting in recover_zip_from_backup.mjs

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/9380d08a-d813-4e9f-a2ac-4891122c163b

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* restore EventEmitter.defaultMaxListeners in recover_zip.js, add acceptance test

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/e7443126-22d5-4d0e-a176-a7a5dba49ffd

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* fix formatting

* refactor: simplify stream handling by using named imports for pipeline

* fix blob hash verification in backup acceptance tests

* fix recover_zip script and tests

* fix: exit with non-zero status on error in recover_zip.js

Agent-Logs-Url: https://github.com/overleaf/internal/sessions/ef3f109b-488f-47c9-84a5-b5269387166a

Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>

* migrate from npm to yarn

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: briangough <7457354+briangough@users.noreply.github.com>
Co-authored-by: Brian Gough <briangough@users.noreply.github.com>
GitOrigin-RevId: 6255f9610f3c846790e2ed8b1979ac08b7effece
2026-06-02 08:08:18 +00:00
Mathias Jakobsen 7053434da6 Merge pull request #34160 from overleaf/mj-tabs-design-tweaks
[web] Apply design tweaks to editor tabs

GitOrigin-RevId: c0b064c4f5977bb13a961f03d2c5f2949d338cfe
2026-06-02 08:08:13 +00:00
Mathias Jakobsen 24dba36060 Merge pull request #34152 from overleaf/mj-select-all
[web] Add select all to context menu

GitOrigin-RevId: ff5fb828db8e1cd57d1361a2e572918339e5e18b
2026-06-02 08:08:08 +00:00
Brian Gough fda0283490 Merge pull request #33377 from overleaf/lucie/js-yaml-security-fix
[Security Upgrade]: js-yaml in yarn.lock

GitOrigin-RevId: 4f388ca74de0e33a4f8894b1aa7e7963d1de552d
2026-06-02 08:07:45 +00:00
Antoine Clausse 2a5f1be811 [web] Fix "For students" link, fix toggles and navigation (#34051)
* [web] Fix footer For Students link to activate student toggle

The footer link only set itm_referrer plus a #student-annual hash. The
plans page reads the active plan/period from `plan` and `period` query
params (PlansHelper.getPlansPageViewOptions), so the student tab never
activated from the footer link.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* syncStudentModeFromPlanType after in handleDeprecatedHash

* Change URL update to use replaceState in the pricing page

* Revert "Change URL update to use replaceState in the pricing page"

This reverts commit eac71f193029e3f1c75e0c97261d8a5982c0d35c.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 69d689d0fe89fc68cefab9233739fc61da8f2ced
2026-06-02 08:07:40 +00:00
Antoine Clausse 105a0ff35c [web] Add nonprofit discount FAQ to plans page (#34126)
Insert a new "Do you offer discounts for nonprofits?" accordion item
under the educational group discount question in the "Overleaf
multi-license plans" FAQ tab. Routes the "contact sales" link through
the existing `faqContactLink` mixin so click tracking stays consistent
with the other FAQ contact links.

Closes #33494

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 582517f1d1f1f7934610253c252cf0f8af2b68a2
2026-06-02 08:07:35 +00:00
Antoine Clausse 2c7129be3a [web] Stop bolding AI features on the interstitial plans page (#34125)
* [web] Stop bolding AI features unconditionally on the interstitial

The four `strong: true` flags on the AI features in `sectionMain2026`
caused those rows to render bold on every interstitial visit, regardless
of paywall context. The original intent (per Design QA #34022) was for
boldness to highlight the features relevant to the specific paywall the
user came from (e.g. AI paywall -> AI features bolded) — that
conditional logic was never wired up, and currently no `purchaseReferrer`
or paywall reason is plumbed through to the feature config.

Remove the unconditional `strong: true` so the cards render consistently
with the pricing page. Reintroduce conditional bolding in a follow-up
once the paywall→features mapping is scoped by design.

Closes #34022

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Remove `card-include-strong` and related code

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: 2112214217f3b53d34518efbca546082ce559e26
2026-06-02 08:07:31 +00:00
Antoine Clausse bd4f73b836 [web] Render "Try for free instead" CTA as link, not button (#34098)
* [web] Render "Try for free instead" CTA as link, not button

Design QA wants the "Try for free instead" CTAs on the pricing and
interstitial pages styled as marketing links (`link-monospace link-lg`)
rather than the current `btn-ghost` button. Add a `link` button type to
the `plans-cta` mixin that drops the `btn` class and applies the link
classes, and set `buttonType: 'link'` on the six `try_for_free_instead`
CTAs (plans-individual, plans-student, interstitial-payment).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Make link smaller

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GitOrigin-RevId: f911698a9bfa19f8180e58edb3cebcea90468cbd
2026-06-02 08:07:26 +00:00
Antoine Clausse 0e4fe4090a [web] Migrate manual plurals to i18next _plural convention (#33989)
* Add tests on plurals

* Update `collabs_per_proj` and its pluralisations

* Update `n_user` and its pluralisations

* Update `showing_x_results` and its pluralisations

* Update `show_x_more_projects` and its pluralisations

* `bin/run web npm run extract-translations`

* Populate `_plural` keys in non-en locales

For 2-form languages (da, de, es, fi, fr, it, nl, no, pt, sv, tr), copy
the existing bare-key value into the new `_plural` sibling to prevent
i18next from falling back to English for count!=1.

Also remove orphan singular keys (`collabs_per_proj_single`,
`showing_1_result*`) left over from the previous commits.

Bare-key values remain in their original plural form pending translator
review — count=1 will still render the plural form in non-en until
translators flip those to singular. Multi-form (cs, pl, ru) and
single-form (ja, ko, zh-CN, zh-TW) locales are unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Flip non-en bare-key values to singular form

Per review, the i18next v3 plural convention uses the bare key for count=1
(singular) and `_plural` for count!=1. The non-en bare-key values were
left as the original plural form by the previous commit so the `_plural`
siblings could be copied from them; this commit flips the bare values to
the singular form per language.

Languages where singular and plural noun forms coincide (Finnish, Swedish,
Turkish) are unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Apply suggestions from code review

Co-authored-by: Olzhas Askar <olzhas.askar@gmail.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Olzhas Askar <olzhas.askar@gmail.com>
GitOrigin-RevId: 628513ca792c2dcce023247e52b7320e2741cc54
2026-06-02 08:07:17 +00:00
Antoine Clausse 58884231c1 [web] Redirect to plans page when previewing subscription change without an existing subscription (#33925)
GitOrigin-RevId: feb47fb519dd7872149d787a8543293cae66a908
2026-06-02 08:07:12 +00:00
Chris Dryden db1deb1617 Merge pull request #33895 from overleaf/cd-pyodide-unsupported-module-message
Improve error messaging for python modules unsupported by Pyodide

GitOrigin-RevId: 038c672ad9da46ea6d4640b8ed37426c92d22e72
2026-06-02 08:07:01 +00:00
Brian Gough b8067723b6 Merge pull request #33628 from overleaf/lg-otel-security-upgrade
Bump @opentelemetry/sdk-node and auto-instrumentations-node (GHSA-q7rr-3cgh-j5r3)

GitOrigin-RevId: 2d5bac25735e9ef8a462423505f142f49ef73d8b
2026-06-02 08:06:52 +00:00
Davinder Singh c09ada9ddb Revert "[WEB] Move Review Toggle into the toolbar (#34066)" (#34150)
This reverts commit 36847e0debdc4dce5f96492261d25e7cc46b2e96.

GitOrigin-RevId: 9bab306156006f683314fd59eea45854f62eae62
2026-06-02 08:06:47 +00:00
Davinder Singh 8b61e8cdca [WEB] removing the beta icon from import modal (#34025)
* removing the beta icon from import modal

* removing the unused classes

GitOrigin-RevId: 11dbe04f31ba831f96e30ab93f3f6c732166e08f
2026-06-02 08:06:38 +00:00
Davinder Singh e0f542a241 [WEB] Move Review Toggle into the toolbar (#34066)
* move Review Toggle into the toolbar

* cleaning up and adding a comment

* adding the cursor styling

* adding isolation on writefull toolbar to adjust z-index of writefull toolbar

* fixing the dark mode colours for review dropdown trigger

* Fix review mode switcher dark mode styles

GitOrigin-RevId: 36847e0debdc4dce5f96492261d25e7cc46b2e96
2026-06-02 08:06:34 +00:00
Davinder Singh ac83bc520c Merge pull request #34065 from overleaf/ds-move-toggle-to-right
[WEB] Move Code/Visual toggle to right-hand side and redesign

GitOrigin-RevId: efc1aa062fd44e20fdf719a6d4ecba9d8bb0e5e8
2026-06-02 08:06:28 +00:00
Eric Mc Sween 31fbc3daee Merge pull request #34128 from overleaf/em-library-analytics
Add analytics events to the account-level library page

GitOrigin-RevId: d0357f37a89ec29cca5b6b375a9553fbdf021b00
2026-06-01 08:05:02 +00:00
Gernot Schulz a0ca344065 Merge pull request #34127 from overleaf/gs-j-cd-hooks
Add deploy pipeline trigger hooks to Jenkinsfiles

GitOrigin-RevId: 80bb89615ae16b733009dca21a5fc41b5c30e993
2026-06-01 08:04:55 +00:00
Malik Glossop 54e122610e Merge pull request #34100 from overleaf/mg-fix-style
Stop inherited color overriding active list group item colour

GitOrigin-RevId: 7e36c2129661b4582658a5ccd9edfb15f12e701c
2026-06-01 08:04:52 +00:00
Domagoj Kriskovic 10ef1d0f34 [web] Fix empty lines being invisible in Python script output
GitOrigin-RevId: eb4b732cae74fa050384fd4cec6bd96a9caae152
2026-06-01 08:04:45 +00:00
Domagoj Kriskovic 987b3a1f71 Track script-runner-opened analytics event
GitOrigin-RevId: fb95aa2f5ad649061a6b8e9797789024a3345f3b
2026-06-01 08:04:41 +00:00
domagojk 270cbaf84e Move python labs icon next to run button
Closes #33892

GitOrigin-RevId: c48d920ee982ddd5e4295fc1279b0f70096820d1
2026-06-01 08:04:37 +00:00
Jakob Ackermann 3c763015ce [monorepo] consolidate clsi-lb host/ip env-vars (#33894)
* [monorepo] consolidate clsi-lb host/ip env-vars

Target env-var is CLSI_LB_HOST. Keep CLSI_LB_IP populated for a week.

* [monorepo] sort dev-environment.env hosts

GitOrigin-RevId: 95d12753c86ffb91264f8971e1c2c412c60de790
2026-06-01 08:04:31 +00:00
Olzhas Askar b5a73efaeb Merge pull request #34060 from overleaf/oa-timeout-cta
[web] Compile timeout CTA

GitOrigin-RevId: c1dd014150964ffec1b556943f572d3e5a8069ce
2026-06-01 08:04:24 +00:00
428 changed files with 23281 additions and 18306 deletions
+2
View File
@@ -329,6 +329,8 @@ jobs:
# (CE default): admin creates accounts / sends invites.
- name: OVERLEAF_ENABLE_PROJECT_PYTHON_VENV
value: "true"
- name: OVERLEAF_LATEX_SHELL_ESCAPE
value: "true"
# (SMTP email vars are loaded below via envFrom.)
# SMTP for password-reset / invite emails. All
# OVERLEAF_EMAIL_* vars come from the optional 'verso-smtp'
+2
View File
@@ -300,6 +300,8 @@ jobs:
# link-sharing users).
- name: OVERLEAF_ENABLE_PROJECT_PYTHON_VENV
value: "true"
- name: OVERLEAF_LATEX_SHELL_ESCAPE
value: "true"
---
apiVersion: v1
kind: Service
@@ -1,5 +1,5 @@
diff --git a/lib/sandboxed_module.js b/lib/sandboxed_module.js
index 1cd6743fe221cbe91ea92fea3707ed07a8a2ded3..46889217d96d5534a206549ae7bd97100e41c3e4 100644
index 1cd6743..4718b97 100644
--- a/lib/sandboxed_module.js
+++ b/lib/sandboxed_module.js
@@ -4,7 +4,7 @@ var Module = require('module');
@@ -11,3 +11,47 @@ index 1cd6743fe221cbe91ea92fea3707ed07a8a2ded3..46889217d96d5534a206549ae7bd9710
var parent = module.parent;
var globalOptions = {};
var registeredBuiltInSourceTransformers = ['coffee'];
@@ -157,12 +157,20 @@ SandboxedModule.prototype._createRecursiveRequireProxy = function() {
var cache = Object.create(null);
var required = this._getRequires();
for (var key in required) {
- var injectedFilename = requireLike(this.filename).resolve(key);
- cache[injectedFilename] = required[key];
+ // Under Yarn PnP, resolution from a transitive dependency's context may fail
+ // for packages not declared in that dependency's package.json. Silently skip
+ // cache pre-population on failure; the mock will still be injected via the
+ // inject map in requireInterceptor or resolved via RecursiveRequireProxy fallback.
+ try {
+ var injectedFilename = requireLike(this.filename).resolve(key);
+ cache[injectedFilename] = required[key];
+ } catch (e) {}
}
cache[this.filename] = this.exports;
var globals = this.globals;
+ // Store the top-level module's filename for PnP fallback resolution
+ var topLevelFilename = this.filename;
var options;
if(!this._options.sourceTransformersSingleOnly && this._options.sourceTransformers){
options = {
@@ -208,8 +216,18 @@ SandboxedModule.prototype._createRecursiveRequireProxy = function() {
if (request in cache) return cache[request];
return require(request);
}
- // cached modules
- var requestedFilename = requireLike(this.filename).resolve(request);
+ // Resolve the requested module filename.
+ // Under Yarn PnP, packages can only resolve their declared dependencies.
+ // When sandboxed-module loads a transitive dependency, the resolution context
+ // may not have access to all needed packages. Fall back to resolving from
+ // the top-level module's context (the module under test).
+ var requestedFilename;
+ try {
+ requestedFilename = requireLike(this.filename).resolve(request);
+ } catch (e) {
+ if (this.filename === topLevelFilename) throw e;
+ requestedFilename = requireLike(topLevelFilename).resolve(request);
+ }
if (requestedFilename in cache) return cache[requestedFilename];
var sandboxedModule = createInnerSandboxedModule(requestedFilename)
return sandboxedModule.exports;
+202 -139
View File
@@ -2,73 +2,182 @@
<img src="services/web/public/img/ol-brand/verso-logo.svg" alt="Verso" width="440">
</p>
**A collaborative, real-time editor for Quarto, LaTeX and Typst — documents and presentations.**
**A collaborative, real-time editor for LaTeX, Quarto and Typst — self-hosted.**
Verso is a fork of [Overleaf](https://github.com/overleaf/overleaf) that adds
first-class [Quarto](https://quarto.org) and [Typst](https://typst.app) support
alongside Overleaf's LaTeX toolchain. It keeps Overleaf's real-time
collaboration infrastructure and runs **three compilers side by side**, chosen
automatically from the root file's extension:
---
| Root file | Compiler | Typical output |
|-----------|----------|----------------|
| `.qmd` | Quarto | PDF (via Typst or LaTeX), or an HTML/RevealJS deck |
| `.tex` | `latexmk` / TeX Live | PDF |
| `.typ` | Typst | PDF |
## What is Verso?
All three coexist on one server; no per-project configuration is required to
pick the engine.
Verso is a fork of [Overleaf](https://github.com/overleaf/overleaf) that extends its
collaborative editing infrastructure to support [Quarto](https://quarto.org) and
[Typst](https://typst.app) projects alongside LaTeX. Think of it as Overleaf, but
not limited to LaTeX.
### Verso vs Overleaf
[Overleaf](https://www.overleaf.com) is the gold standard for collaborative LaTeX
editing. Verso keeps everything that makes Overleaf great — real-time co-editing,
operational-transformation history, auth, project management, file storage — and
adds:
- **Quarto and Typst compilers** running alongside TeX Live, dispatched
automatically from the root file's extension (`.qmd` → Quarto, `.typ` → Typst,
`.tex``latexmk`).
- **Language-aware editor** for Quarto and Typst (syntax highlighting, completions,
document outline) — not just LaTeX.
- **Publish & share compiled output** (`/p/:token` with tiered access links) — a
feature absent from Overleaf Community Edition.
- **Lumière theme** — a redesigned project dashboard and editor chrome with a
card-based grid, thumbnails, and a teal gradient identity.
- **Full i18n** — French, German, Italian, and Spanish UI translations on top of
Overleaf's English base.
- Completely **free and self-hosted**; no Overleaf subscription required.
### Verso vs Quarto
[Quarto](https://quarto.org) is a command-line tool: you install it locally, write
`.qmd` files in any text editor, and run `quarto render` in a terminal. It is
excellent for solo authors with full control over their environment.
Verso wraps Quarto in a collaborative web editor:
- **No local install** — Quarto, Typst, TeX Live and Python run on the server.
- **Real-time collaboration** — multiple people edit the same `.qmd` simultaneously
with live cursors and conflict-free merging.
- **Not just Quarto** — LaTeX and Typst projects live in the same workspace, under
the same auth and history system.
- **Publish in one click** — RevealJS decks and PDFs are served at a stable link
without leaving the browser.
Verso is not a replacement for Quarto's CLI — it is a platform that makes Quarto
accessible as a shared, always-on service.
### Verso vs Typst.app
[Typst.app](https://typst.app) is a cloud-hosted web editor for Typst. It is
polished and fast, but it is a proprietary SaaS product and only supports Typst.
Verso differs in that:
- It is **self-hosted** and open-source (AGPL v3) — you control your data.
- It supports **three languages** (Typst, LaTeX, Quarto) in one instance.
- Real-time collaboration is powered by **operational transformation** (the same
engine as Overleaf), not CRDTs, which means it handles concurrent edits
gracefully for long documents.
- It ships with a full **project history** and version-restore workflow.
If you only need Typst and want a lighter, Typst-focused alternative, have a look
at **[Collabst](https://github.com/herluf-ba/collabst)** — an open-source,
self-hosted collaborative Typst editor that is independent of the Overleaf
codebase and shows a lot of promise.
---
## Features
- **Real-time collaboration** — multiple people editing the same file at once,
powered by Overleaf's operational-transformation engine, with live cursors
and full project history.
- **Three compilers, auto-dispatched** — Quarto, LaTeX and Typst projects live
side by side; the runner is selected from the root file's extension.
- **Language-aware editor for all three**:
- *LaTeX* — syntax highlighting, command/environment/reference autocomplete,
linting (inherited from Overleaf).
- *Quarto (`.qmd`)* — Markdown highlighting plus Quarto-aware completions:
code chunks (```` ```{python} ````, `{r}`, `{julia}`, `{ojs}`…), callouts
and fenced divs (`::: {.callout-note}`, columns, tabsets) and
cross-references (`@fig-`, `@tbl-`, `@sec-`, `@eq-`).
- *Typst (`.typ`)* — syntax highlighting and completions for the common
functions and markup (`#import`, `#let`, `#set`, `#show`, `#figure`,
`#table`, `#cite`, …).
- **Document outline** — section headings are extracted into the sidebar
outline panel for LaTeX, Quarto (`#`, `##`, …) and Typst (`=`, `==`, …).
- **Format at a glance** — the project dashboard shows a per-project format
badge (Quarto / Typst / LaTeX), and the compiler dropdown greys out engines
that don't apply to the current root file.
- **Publish & share compiled output** — publish the compiled result as a
standalone page at `/p/:token`, with three independent access tiers (project
members / any logged-in user / public). Works for both HTML/RevealJS decks
(served live) and PDFs (embedded inline). HTML decks also get a one-click
**Present** button in the toolbar.
- **Quarto Python cells** — optional per-project virtual environment built from
the project's `requirements.txt`, so Python code chunks run during render
(gated to the project owner and invited collaborators).
- **Auto-compile** — the preview refreshes automatically shortly after you stop
typing.
- **Real-time collaboration** — multiple editors, live cursors, full project history and version restore.
- **Three compilers, auto-dispatched** by root file extension:
| Root file | Compiler | Typical output |
|-----------|----------|----------------|
| `.qmd` | Quarto | PDF (via Typst or LaTeX), HTML, or RevealJS |
| `.tex` | `latexmk` / TeX Live | PDF |
| `.typ` | Typst | PDF |
- **Language-aware editor for all three** — syntax highlighting, completions, and a document outline panel for LaTeX, Quarto and Typst.
- **Format badge** on the project dashboard; compiler dropdown greys out inapplicable engines.
- **Publish & share** — compile and snapshot to `/p/:token` with three independent access tiers (project members / any logged-in user / public). HTML/RevealJS decks are served live; PDFs are embedded inline. A **Present** toolbar button links directly to the published deck.
- **RevealJS thumbnails** — the first slide of a presentation is rendered as a preview card in the project list.
- **Quarto Python cells** — optional per-project virtual environment built from `requirements.txt`, so Python code chunks execute during render.
- **Visual formatting toolbar** — bold, italic, headings and inline code shortcuts for Quarto (`.qmd`) and Typst (`.typ`) files, in addition to Overleaf's existing LaTeX toolbar.
- **Lumière theme** — card-based project dashboard with PDF/slide thumbnails, a teal gradient identity, dark editor chrome, and an XS compact list view.
- **i18n** — French, German, Italian and Spanish UI translations.
- **Auto-compile** — preview refreshes automatically after you stop typing.
## Output formats
---
In the YAML frontmatter of a `.qmd` file:
## Releases
```yaml
format: typst # → PDF preview, rendered via Typst (no LaTeX required)
format: pdf # → PDF preview, rendered via LaTeX
format: revealjs # → interactive HTML slideshow preview
format: html # → a static HTML page
```
### Alpha 1
Typst ships inside Quarto, so `format: typst` needs no separate installation.
The initial public release. Established Verso as an Overleaf fork with first-class
multi-language support:
> **Note on display math**: keep `$$ … $$` blocks on a single line. Multi-line
> display-math blocks can trigger YAML parse errors in some Quarto versions.
- Quarto (`.qmd`) and Typst (`.typ`) compilers running alongside TeX Live,
dispatched automatically by root file extension — no per-project configuration.
- Language-aware editor for Quarto: Markdown highlighting, code-chunk completions
(`{python}`, `{r}`, `{julia}`, `{ojs}`…), callout and fenced-div completions,
cross-reference completions (`@fig-`, `@tbl-`, `@sec-`…).
- Language-aware editor for Typst: syntax highlighting and completions for
functions, imports, math and markup.
- Document outline panel for all three languages (LaTeX `\section`, Quarto `#`,
Typst `=`).
- Format badge on the project dashboard; compiler selector greys out inapplicable
engines for the current root file.
- Publish & share compiled output — HTML/RevealJS decks and PDFs hosted at
`/p/:token` with tiered access links (project / logged-in / public), each
independently resettable.
- Quarto Python code-cell execution via an optional per-project `requirements.txt`
virtual environment.
- Verso branding: name, logo and Kubernetes production deploy workflow.
### Alpha 2
Refinements to the Typst editor and the format badge system:
- **Quarto format sub-types** — the project badge now distinguishes *Quarto PDF*
from *Quarto Slides*, reading the frontmatter `format:` to pick the right label.
- **Python packages for collaborators** — Quarto Python package installation
extended to all users who have write access to the project, not only the owner.
- **Typst syntax highlighting overhaul** — complete grammar rewrite covering:
function calls and named argument keys, multi-line display math, `#{…}` code
blocks, content blocks, `show`-rule bodies, `let`-value bindings, and keyword
vs identifier disambiguation.
- **Typst visual formatting** — bold and italic toolbar buttons and keyboard
shortcuts (`Ctrl+B`, `Ctrl+I`), plus underline, small-caps and hyperlink
buttons, matching the Quarto and LaTeX toolbar experience.
### Alpha 3
- **Lumière theme** — redesigned project dashboard with a card grid, PDF/slide
thumbnails, parallax hover effects, a teal gradient identity and a dark editor
chrome. Includes an XS compact list view and a tile zoom slider.
- **Full i18n** — French, German, Italian and Spanish translations covering the
complete UI (login, dashboard, editor, settings, emails).
- **Visual editors for Quarto and Typst** — bold, italic, headings and inline code
shortcuts in the toolbar for `.qmd` and `.typ` files.
- **Top/bottom split view** — new editor layout that stacks the source editor
above the PDF preview vertically, in addition to the existing side-by-side mode.
- **Bidirectional format export** — LaTeX projects can be converted to Typst and
Typst projects to LaTeX via pandoc. Available from the File menu in the editor.
- **Mobile layout** — project dashboard, search bar, footer and editor all
adapted for phone screen sizes.
---
## Known issues
- **Large file upload timeouts** — uploads of large files on slow connections
can time out at the proxy layer. A streaming response fix is pending.
---
## Security model — trusted environments only
> [!CAUTION]
> Verso is designed for **closed groups of trusted users** (a lab, a class, a
> small team). All three compilers can execute arbitrary code on the server:
>
> - LaTeX with shell-escape enabled can run system commands.
> - Quarto Python cells execute Python code directly.
> - Typst's scripting layer is sandboxed by design, but runs server-side.
>
> There is **no per-project sandbox or resource isolation** beyond what the
> operating system provides. Exposing Verso to the public internet with open
> registration is not recommended. If you need to host a collaborative
> LaTeX editor for untrusted users or at scale, look at
> [Overleaf's non-Community offerings](https://www.overleaf.com/for/enterprises),
> which include proper sandboxing and enterprise access controls.
---
## Quick start
@@ -82,24 +191,23 @@ docker run -d \
registry.alocoq.fr/verso:latest
```
Open `http://localhost` in your browser, then visit `/launchpad` on first run to
create the admin account.
Open `http://localhost`, then visit `/launchpad` on first run to create the admin
account.
### Build from source
```bash
# Build the base image (system deps + Quarto + TeX Live)
cd server-ce
make build-base
# Build the application image
make build-community
make build-base # base OS image: system deps, Quarto, Typst, TeX Live
make build-community # application image: Node services + compiled frontend
```
| File | Purpose |
|------|---------|
| `server-ce/Dockerfile-base` | Base OS image — system deps, Quarto (with Typst) and a TeX Live (`latexmk`) toolchain |
| `server-ce/Dockerfile` | Application image — Node services and the compiled frontend |
| `server-ce/Dockerfile-base` | Base image — system deps, Quarto (with Typst) and TeX Live |
| `server-ce/Dockerfile` | App image — Node services and the compiled React frontend |
---
## Architecture
@@ -108,10 +216,10 @@ single container managed by `runit`, with `nginx` as the front router.
```
browser ──→ nginx:80
├── / ──────────────────→ web:4000 (main app, React UI)
├── /socket.io ──────────→ real-time:3026 (WebSocket, OT engine)
├── /p/:token ───────────→ web (published output)
└── /project/*/output/* → clsi-nginx:8080 (compiled output files)
├── / ──────────────────→ web:4000 (main app, React UI)
├── /socket.io ──────────→ real-time:3026 (WebSocket, OT engine)
├── /p/:token ───────────→ web (published output)
└── /project/*/output/* → clsi-nginx:8080 (compiled output files)
web → document-updater → Redis pub/sub → real-time → browser
web → CLSI (quarto render / latexmk / typst) → output files → nginx → browser
@@ -122,64 +230,12 @@ web → CLSI (quarto render / latexmk / typst) → output files → nginx → br
| `web` | HTTP API, React frontend, auth, project & sharing management |
| `real-time` | WebSocket layer, live cursors and edit sync |
| `document-updater` | Operational transformation, Redis pub/sub |
| `clsi` | Compiler — runs `quarto render` (`.qmd`), `latexmk` (`.tex`) or `typst` (`.typ`) and serves output |
| `clsi` | Compiler — runs `quarto render`, `latexmk` or `typst` and serves output |
| `docstore` | Document text storage (MongoDB) |
| `filestore` | Binary file storage (S3 or local) |
| `project-history` | Change history and version tracking |
## Writing documents
### Quarto (`main.qmd`)
```markdown
---
title: My Presentation
author: Your Name
date: today
format: revealjs
---
## Slide one
Write **Markdown** here.
## Mathematics
$$\int_0^\infty e^{-x^2}\,dx = \frac{\sqrt{\pi}}{2}$$
```
Switch `format: revealjs` to `format: typst` (or `pdf`) for a PDF preview.
### LaTeX (`main.tex`)
LaTeX works exactly as in Overleaf: a project whose root file is a `.tex` file
compiles with `latexmk`/TeX Live, no setting required. The **Example LaTeX
project** in the *New project* menu is a ready-made starting point.
> The bundled TeX Live is a minimal install. Documents that need extra packages
> may not build out of the box — see `server-ce/Dockerfile-base` for how to
> switch to a fuller TeX Live scheme.
### Typst (`main.typ`)
A project whose root file is a `.typ` file compiles directly to PDF with
[Typst](https://typst.app) — fast, modern markup with a real scripting
language. Verso drives the Typst bundled with Quarto, so no extra install is
needed. Use the **Blank Typst project** entry in the *New project* menu to get
started.
## Publishing compiled output
From **Share → Publish**, Verso compiles the project and snapshots the result to
a standalone page at `/p/:token`:
- **HTML / RevealJS** decks are served as a live page (the **Present** toolbar
button is a one-click shortcut to this).
- **PDF** output is embedded inline; the raw file stays reachable at
`/p/:token/output.pdf`.
Three stable links are issued, one per access tier — project members, any
logged-in user, or anyone — and each can be copied or independently reset.
## Environment variables
@@ -195,39 +251,46 @@ The most commonly needed:
| `OVERLEAF_SITE_URL` | — | Public URL (used in emails and published links) |
| `OVERLEAF_SITE_LANGUAGE` | `en` | Default UI language (e.g. `fr`) |
| `OVERLEAF_ENABLE_PROJECT_PYTHON_VENV` | `false` | Allow Quarto Python cells to use a project `requirements.txt` |
| `OVERLEAF_ADMIN_EMAIL` | — | Email for the first admin account |
| `OVERLEAF_ADMIN_EMAIL` | — | Email shown on the launchpad for the first admin account |
See the [Overleaf Server documentation](https://github.com/overleaf/overleaf/wiki)
for the full list.
---
## Relation to Overleaf
Verso is a fork of [Overleaf Community Edition](https://github.com/overleaf/overleaf).
The main additions on top of upstream are:
Everything that Overleaf CE provides — real-time collaboration, operational-transformation
history, auth, project management, binary file storage — is inherited unchanged. The
Verso-specific additions are listed in the Features section and tracked across releases above.
- Quarto and Typst compilers running alongside LaTeX, dispatched by the root
file's extension.
- Editor language support (highlighting, autocomplete, outline) for Quarto and
Typst.
- A per-project format badge on the dashboard and a root-file-aware compiler
selector.
- Publishing/sharing of compiled output (HTML decks and PDFs) via `/p/:token`
with tiered access links, and a toolbar **Present** shortcut.
- Optional per-project Python virtual environments for Quarto code execution.
- Verso branding (name, logo, palette, loading animation).
Verso is not affiliated with Overleaf Ltd.
All other infrastructure — real-time collaboration, history, auth, file
storage, project management — is unchanged from Overleaf.
---
## Contributing
## Supporting the ecosystem
Contributions are welcome — open an issue or pull request on the
[Verso repository](https://git.alocoq.fr/alois/verso). The upstream Overleaf
contribution guidelines are in [CONTRIBUTING.md](CONTRIBUTING.md).
Verso is not accepting contributions or donations at this time. If you find it
useful and want to support the broader ecosystem it builds on:
- **Support Overleaf** — Verso is built on Overleaf's infrastructure. The best
way to support their work is to use or subscribe to
[Overleaf](https://www.overleaf.com) and encourage your institution to do the
same.
- **Support Typst** — [Typst GmbH](https://typst.app) is the company behind the
Typst compiler. Using Typst.app or sponsoring the
[Typst project on GitHub](https://github.com/typst/typst) helps sustain the
language itself.
- **Support RevealJS** — Verso uses [Reveal.js](https://revealjs.com) for
HTML presentations. Consider sponsoring the
[RevealJS project on GitHub](https://github.com/hakimel/reveal.js).
---
## License
GNU Affero General Public License v3 — see [LICENSE](LICENSE).
Copyright © Overleaf, 20142026 (original code).
Copyright © Overleaf, 20142026 (original code).
Verso modifications © Aloïs Coquillard, 2026.
+41
View File
@@ -0,0 +1,41 @@
# Verso — Next Alpha Roadmap
Ideas and features deferred from the current alpha.
---
## Next alpha (post-current)
### Typst editing experience (inspired by Collabst)
- **typst.ts WASM preview** — Run the Typst compiler in the browser via
WebAssembly (typst.ts). This would give instant, sub-second preview
without a server round-trip, and would eliminate the entire class of
race conditions in the CLSI watcher (files written → typst compiles →
resolver missed). Could coexist with the CLSI watcher for PDF export
while using the WASM path for live preview.
- **Tinymist LSP integration** — Wire up
[Tinymist](https://github.com/Myriad-Dreamin/tinymist) (the Typst
language server) behind a WebSocket proxy. Would give Typst files
first-class autocomplete, hover docs, go-to-definition, and inline
error diagnostics — the main editing comfort gap vs. a native editor.
### Editor UX for non-LaTeX formats (.typ, .qmd, .md)
- **Visual/rich-text editing mode** — A toggle between raw source and a
rendered-in-place view for `.typ`, `.qmd`, and `.md` files (similar to
Overleaf's rich-text mode for LaTeX). Users who don't know Typst or
Markdown syntax should be able to edit content without seeing markup.
CodeMirror 6 already supports this pattern via a custom `NodeView` layer
or a separate Prosemirror bridge.
- **Toolbar / insertion shortcuts** — A formatting toolbar and keyboard
shortcuts for common operations, adapted per file type:
- **All formats**: bold, italic, underline, headings, bullet/numbered
lists, inline code, links.
- **Quarto / Markdown**: insert image, insert table, insert code block
with language tag.
- **Quarto RevealJS**: insert slide divider (`---`), insert speaker
notes (`::: notes`), insert columns layout, insert video embed
(using Quarto's `{{< video >}}` shortcode).
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6
View File
@@ -1,4 +1,10 @@
const pkg = require('./package.json')
module.exports = {
meta: {
name: pkg.name,
version: pkg.version,
},
rules: {
'no-unnecessary-trans': require('./no-unnecessary-trans'),
'prefer-kebab-url': require('./prefer-kebab-url'),
+2 -2
View File
@@ -8,10 +8,10 @@
"lodash": "^4.18.1"
},
"devDependencies": {
"@typescript-eslint/parser": "^8.50.0"
"@typescript-eslint/parser": "^8.59.4"
},
"peerDependencies": {
"eslint": "^8.51.0"
"eslint": "^10.4.0"
},
"scripts": {
"test": "node rules.test.js"
@@ -22,7 +22,7 @@ module.exports = {
},
},
create(context) {
const currentFilePath = context.getFilename()
const currentFilePath = context.filename
// ESLint can sometimes pass <text> or <input> for snippets not in a file
if (currentFilePath === '<text>' || currentFilePath === '<input>') {
return {}
@@ -81,9 +81,10 @@ module.exports = {
typeof firstArg.value !== 'string'
) {
if (firstArg.type === 'Identifier') {
const variable = context
.getScope()
.variables.find(v => v.name === firstArg.name)
const scope = context.sourceCode.getScope(node)
const variable = scope.variables.find(
v => v.name === firstArg.name
)
if (
variable &&
variable.defs.length > 0 &&
+18 -7
View File
@@ -1,4 +1,5 @@
const { RuleTester } = require('eslint')
const tsParser = require('@typescript-eslint/parser')
const noThrowInCallback = require('./no-throw-in-callback')
const preferKebabUrl = require('./prefer-kebab-url')
const noUnnecessaryTrans = require('./no-unnecessary-trans')
@@ -8,10 +9,10 @@ const viDoMockValidPath = require('./require-vi-doMock-valid-path')
const requireCioSnakeCaseProperties = require('./require-cio-snake-case-properties')
const ruleTester = new RuleTester({
parser: require.resolve('@typescript-eslint/parser'),
parserOptions: {
languageOptions: {
parser: tsParser,
ecmaVersion: 'latest',
ecmaFeatures: { jsx: true },
parserOptions: { ecmaFeatures: { jsx: true } },
},
})
@@ -33,19 +34,27 @@ ruleTester.run('prefer-kebab-url', preferKebabUrl, {
invalid: [
{
code: `app.get('/fooBar')`,
errors: [{ message: 'Route path should be in kebab-case.' }],
errors: [
{ message: 'Route path should be in kebab-case.', suggestions: 1 },
],
},
{
code: `app.get('/fooBar/:id')`,
errors: [{ message: 'Route path should be in kebab-case.' }],
errors: [
{ message: 'Route path should be in kebab-case.', suggestions: 1 },
],
},
{
code: `webRouter.get('/foo_bar/:id/FooBar/:name/fooBar')`,
errors: [{ message: 'Route path should be in kebab-case.' }],
errors: [
{ message: 'Route path should be in kebab-case.', suggestions: 1 },
],
},
{
code: `router.get(/^\\/downLoad\\/pro-ject\\/([^/]*)\\/OutPut\\/out-put\\.pdf$/)`,
errors: [{ message: 'Route path should be in kebab-case.' }],
errors: [
{ message: 'Route path should be in kebab-case.', suggestions: 1 },
],
},
],
})
@@ -153,6 +162,7 @@ ruleTester.run('domock-require-valid-path', viDoMockValidPath, {
{
message:
'The path "./require-vi-doMock-valid-path2" in vi.doMock() cannot be resolved relative to the current file.',
suggestions: [],
},
],
},
@@ -163,6 +173,7 @@ ruleTester.run('domock-require-valid-path', viDoMockValidPath, {
{
message:
'The first argument of vi.doMock() must be (or resolve to) a string literal representing a path.',
suggestions: [],
},
],
},
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -6
View File
@@ -10,12 +10,12 @@
"dependencies": {
"@google-cloud/opentelemetry-cloud-trace-exporter": "^3.0.0",
"@google-cloud/profiler": "^6.0.4",
"@opentelemetry/api": "1.9.0",
"@opentelemetry/auto-instrumentations-node": "^0.72.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.214.0",
"@opentelemetry/resources": "^2.6.0",
"@opentelemetry/sdk-node": "^0.214.0",
"@opentelemetry/semantic-conventions": "^1.39.0",
"@opentelemetry/api": "^1.9.1",
"@opentelemetry/auto-instrumentations-node": "^0.76.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.218.0",
"@opentelemetry/resources": "^2.7.1",
"@opentelemetry/sdk-node": "^0.218.0",
"@opentelemetry/semantic-conventions": "^1.41.1",
"compression": "^1.7.4",
"prom-client": "^14.1.1",
"yn": "^3.1.1"
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
-1
View File
@@ -38,7 +38,6 @@
"mocha": "^11.1.0",
"mocha-junit-reporter": "^2.2.1",
"mocha-multi-reporters": "^1.5.1",
"mock-fs": "^5.2.0",
"mongodb": "6.12.0",
"sandboxed-module": "^2.0.4",
"sinon": "^9.2.4",
@@ -3,9 +3,12 @@ const fs = require('node:fs')
const fsPromises = require('node:fs/promises')
const { glob } = require('glob')
const Path = require('node:path')
const { promisify } = require('node:util')
const { PassThrough } = require('node:stream')
const { pipeline } = require('node:stream/promises')
const openCb = promisify(fs.open)
const AbstractPersistor = require('./AbstractPersistor')
const { ReadError, WriteError, NotImplementedError } = require('./Errors')
const PersistorHelper = require('./PersistorHelper')
@@ -85,8 +88,9 @@ module.exports = class FSPersistor extends AbstractPersistor {
})
const fsPath = this._getFsPath(location, name, opts.useSubdirectories)
let fd
try {
opts.fd = await fsPromises.open(fsPath, 'r')
fd = await openCb(fsPath, 'r')
} catch (err) {
throw PersistorHelper.wrapError(
err,
@@ -96,7 +100,7 @@ module.exports = class FSPersistor extends AbstractPersistor {
)
}
const stream = fs.createReadStream(null, opts)
const stream = fs.createReadStream(null, { ...opts, fd })
// Return a PassThrough stream with a minimal interface. It will buffer until the caller starts reading. It will emit errors from the source stream (Stream.pipeline passes errors along).
const pass = new PassThrough()
pipeline(stream, observer, pass).catch(() => {})
@@ -1,6 +1,6 @@
const crypto = require('node:crypto')
const os = require('node:os')
const { expect } = require('chai')
const mockFs = require('mock-fs')
const fs = require('node:fs')
const fsPromises = require('node:fs/promises')
const Path = require('node:path')
@@ -10,22 +10,59 @@ const Errors = require('../../src/Errors')
const MODULE_PATH = '../../src/FSPersistor.js'
function createTree(base, tree) {
fs.mkdirSync(base, { recursive: true })
for (const [name, content] of Object.entries(tree)) {
const fullPath = Path.join(base, name)
if (Buffer.isBuffer(content) || typeof content === 'string') {
fs.writeFileSync(fullPath, content)
} else if (content && typeof content.symlink === 'string') {
fs.symlinkSync(content.symlink, fullPath)
} else {
createTree(fullPath, content)
}
}
}
describe('FSPersistorTests', function () {
const localFiles = {
'/uploads/info.txt': Buffer.from('This information is critical', {
const fileContents = {
'info.txt': Buffer.from('This information is critical', {
encoding: 'utf-8',
}),
'/uploads/other.txt': Buffer.from('Some other content', {
'other.txt': Buffer.from('Some other content', {
encoding: 'utf-8',
}),
}
const location = '/bucket'
let tmpDir
let location
let notADirPath
const files = {
wombat: 'animals/wombat.tex',
giraffe: 'animals/giraffe.tex',
potato: 'vegetables/potato.tex',
}
beforeEach(function () {
tmpDir = fs.mkdtempSync(Path.join(os.tmpdir(), 'fs-persistor-test-'))
createTree(tmpDir, {
uploads: {
'info.txt': fileContents['info.txt'],
'other.txt': fileContents['other.txt'],
},
'not-a-dir':
'This regular file is meant to prevent using this path as a directory',
directory: {
subdirectory: {},
},
})
notADirPath = Path.join(tmpDir, 'not-a-dir')
location = Path.join(tmpDir, 'bucket')
})
afterEach(function () {
fs.rmSync(tmpDir, { recursive: true })
})
const scenarios = [
{
description: 'default settings',
@@ -54,31 +91,26 @@ describe('FSPersistorTests', function () {
persistor = new FSPersistor(scenario.settings)
})
beforeEach(function () {
mockFs({
...localFiles,
'/not-a-dir':
'This regular file is meant to prevent using this path as a directory',
'/directory/subdirectory': {},
})
})
afterEach(function () {
mockFs.restore()
})
describe('sendFile', function () {
it('should copy the file', async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
it('should return an error if the file cannot be stored', async function () {
await expect(
persistor.sendFile('/not-a-dir', files.wombat, '/uploads/info.txt')
persistor.sendFile(
notADirPath,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
).to.be.rejectedWith(Errors.WriteError)
})
})
@@ -88,7 +120,9 @@ describe('FSPersistorTests', function () {
describe("when the file doesn't exist", function () {
beforeEach(function () {
stream = fs.createReadStream('/uploads/info.txt')
stream = fs.createReadStream(
Path.join(tmpDir, 'uploads', 'info.txt')
)
})
it('should write the stream to disk', async function () {
@@ -96,7 +130,7 @@ describe('FSPersistorTests', function () {
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
it('should delete the temporary file', async function () {
@@ -109,7 +143,7 @@ describe('FSPersistorTests', function () {
describe('on error', function () {
beforeEach(async function () {
await expect(
persistor.sendStream('/not-a-dir', files.wombat, stream)
persistor.sendStream(notADirPath, files.wombat, stream)
).to.be.rejectedWith(Errors.WriteError)
})
@@ -129,13 +163,12 @@ describe('FSPersistorTests', function () {
describe('when the md5 hash matches', function () {
it('should write the stream to disk', async function () {
await persistor.sendStream(location, files.wombat, stream, {
sourceMd5: md5(localFiles['/uploads/info.txt']),
sourceMd5: md5(fileContents['info.txt']),
})
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be
.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
})
@@ -169,9 +202,11 @@ describe('FSPersistorTests', function () {
await persistor.sendFile(
location,
files.wombat,
'/uploads/info.txt'
Path.join(tmpDir, 'uploads', 'info.txt')
)
stream = fs.createReadStream(
Path.join(tmpDir, 'uploads', 'other.txt')
)
stream = fs.createReadStream('/uploads/other.txt')
})
it('should write the stream to disk', async function () {
@@ -179,7 +214,7 @@ describe('FSPersistorTests', function () {
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/other.txt'])).to.be.true
expect(contents.equals(fileContents['other.txt'])).to.be.true
})
it('should delete the temporary file', async function () {
@@ -192,7 +227,7 @@ describe('FSPersistorTests', function () {
describe('on error', function () {
beforeEach(async function () {
await expect(
persistor.sendStream('/not-a-dir', files.wombat, stream)
persistor.sendStream(notADirPath, files.wombat, stream)
).to.be.rejectedWith(Errors.WriteError)
})
@@ -200,8 +235,7 @@ describe('FSPersistorTests', function () {
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be
.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
it('should delete the temporary file', async function () {
@@ -215,13 +249,12 @@ describe('FSPersistorTests', function () {
describe('when the md5 hash matches', function () {
it('should write the stream to disk', async function () {
await persistor.sendStream(location, files.wombat, stream, {
sourceMd5: md5(localFiles['/uploads/other.txt']),
sourceMd5: md5(fileContents['other.txt']),
})
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/other.txt'])).to.be
.true
expect(contents.equals(fileContents['other.txt'])).to.be.true
})
})
@@ -238,8 +271,7 @@ describe('FSPersistorTests', function () {
const contents = await fsPromises.readFile(
scenario.fsPath(files.wombat)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be
.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
it('should delete the temporary file', async function () {
@@ -254,13 +286,17 @@ describe('FSPersistorTests', function () {
describe('getObjectStream', function () {
beforeEach(async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
})
it('should return a string with the object contents', async function () {
const stream = await persistor.getObjectStream(location, files.wombat)
const contents = await streamToBuffer(stream)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
it('should support ranges', async function () {
@@ -274,8 +310,8 @@ describe('FSPersistorTests', function () {
)
const contents = await streamToBuffer(stream)
// end is inclusive in ranges, but exclusive in slice()
expect(contents.equals(localFiles['/uploads/info.txt'].slice(5, 17)))
.to.be.true
expect(contents.equals(fileContents['info.txt'].slice(5, 17))).to.be
.true
})
it('should give a NotFoundError if the file does not exist', async function () {
@@ -287,13 +323,17 @@ describe('FSPersistorTests', function () {
describe('getObjectSize', function () {
beforeEach(async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
})
it('should return the file size', async function () {
expect(
await persistor.getObjectSize(location, files.wombat)
).to.equal(localFiles['/uploads/info.txt'].length)
).to.equal(fileContents['info.txt'].length)
})
it('should throw a NotFoundError if the file does not exist', async function () {
@@ -305,7 +345,11 @@ describe('FSPersistorTests', function () {
describe('copyObject', function () {
beforeEach(async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
})
it('Should copy the file to the new location', async function () {
@@ -313,13 +357,17 @@ describe('FSPersistorTests', function () {
const contents = await fsPromises.readFile(
scenario.fsPath(files.potato)
)
expect(contents.equals(localFiles['/uploads/info.txt'])).to.be.true
expect(contents.equals(fileContents['info.txt'])).to.be.true
})
})
describe('deleteObject', function () {
beforeEach(async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
await fsPromises.access(scenario.fsPath(files.wombat))
})
@@ -337,7 +385,11 @@ describe('FSPersistorTests', function () {
describe('deleteDirectory', function () {
beforeEach(async function () {
for (const file of Object.values(files)) {
await persistor.sendFile(location, file, '/uploads/info.txt')
await persistor.sendFile(
location,
file,
Path.join(tmpDir, 'uploads', 'info.txt')
)
await fsPromises.access(scenario.fsPath(file))
}
})
@@ -365,7 +417,11 @@ describe('FSPersistorTests', function () {
describe('checkIfObjectExists', function () {
beforeEach(async function () {
await persistor.sendFile(location, files.wombat, '/uploads/info.txt')
await persistor.sendFile(
location,
files.wombat,
Path.join(tmpDir, 'uploads', 'info.txt')
)
})
it('should return true for existing files', async function () {
@@ -384,13 +440,17 @@ describe('FSPersistorTests', function () {
describe('directorySize', function () {
beforeEach(async function () {
for (const file of Object.values(files)) {
await persistor.sendFile(location, file, '/uploads/info.txt')
await persistor.sendFile(
location,
file,
Path.join(tmpDir, 'uploads', 'info.txt')
)
}
})
it('should sum directory files size', async function () {
expect(await persistor.directorySize(location, 'animals')).to.equal(
2 * localFiles['/uploads/info.txt'].length
2 * fileContents['info.txt'].length
)
})
@@ -404,7 +464,11 @@ describe('FSPersistorTests', function () {
describe('listDirectoryKeys', function () {
beforeEach(async function () {
for (const file of Object.values(files)) {
await persistor.sendFile(location, file, '/uploads/info.txt')
await persistor.sendFile(
location,
file,
Path.join(tmpDir, 'uploads', 'info.txt')
)
}
})
@@ -427,7 +491,11 @@ describe('FSPersistorTests', function () {
describe('listDirectoryStats', function () {
beforeEach(async function () {
for (const file of Object.values(files)) {
await persistor.sendFile(location, file, '/uploads/info.txt')
await persistor.sendFile(
location,
file,
Path.join(tmpDir, 'uploads', 'info.txt')
)
}
})
@@ -438,7 +506,7 @@ describe('FSPersistorTests', function () {
expect(keys).to.include(scenario.fsPath(files.wombat))
expect(keys).to.include(scenario.fsPath(files.giraffe))
for (const stat of stats) {
expect(stat.size).to.equal(localFiles['/uploads/info.txt'].length)
expect(stat.size).to.equal(fileContents['info.txt'].length)
}
})
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+17 -16
View File
@@ -3,39 +3,41 @@
"private": true,
"packageManager": "yarn@4.14.1",
"devDependencies": {
"@eslint/compat": "^2.1.0",
"@eslint/js": "^10.0.1",
"@overleaf/eslint-plugin": "workspace:*",
"@prettier/plugin-pug": "^3.4.0",
"@types/chai": "^4.3.0",
"@types/chai-as-promised": "^7.1.8",
"@types/mocha": "^10.0.6",
"@types/multer": "^2.1.0",
"@typescript-eslint/eslint-plugin": "8.50.0",
"@typescript-eslint/parser": "^8.50.0",
"@typescript-eslint/eslint-plugin": "^8.59.4",
"@typescript-eslint/parser": "^8.59.4",
"@vitest/eslint-plugin": "^1.5.0",
"eslint": "^8.15.0",
"eslint-config-prettier": "^8.5.0",
"eslint-config-standard": "^17.0.0",
"eslint-plugin-chai-expect": "^3.0.0",
"eslint-plugin-chai-friendly": "^0.7.2",
"eslint-plugin-cypress": "^2.15.1",
"eslint-plugin-import": "^2.26.0",
"eslint-plugin-mocha": "^10.1.0",
"eslint-plugin-n": "^15.7.0",
"eslint-plugin-prettier": "^4.0.0",
"eslint-plugin-promise": "^6.0.0",
"eslint": "^10.4.0",
"eslint-config-prettier": "^10.0.1",
"eslint-formatter-unix": "^8.40.0",
"eslint-plugin-chai-expect": "^4.0.0",
"eslint-plugin-chai-friendly": "^1.1.0",
"eslint-plugin-cypress": "^4.1.0",
"eslint-plugin-import": "^2.32.0",
"eslint-plugin-mocha": "^11.0.0",
"eslint-plugin-n": "^18.0.0",
"eslint-plugin-promise": "^7.2.1",
"eslint-plugin-unicorn": "^56.0.0",
"globals": "^17.6.0",
"prettier": "3.7.4",
"prettier-plugin-groovy": "0.2.1",
"typescript": "^5.9.3"
},
"engines": {
"node": ">=20.0.0"
"node": ">=20.19.0"
},
"resolutions": {
"@xmldom/xmldom": "0.8.13",
"argparse/underscore": "1.13.8",
"east/underscore": "1.13.8",
"referer-parser/js-yaml": "^4.1.0",
"referer-parser/js-yaml": "^4.1.1",
"sandboxed-module": "patch:sandboxed-module@npm%3A2.0.4#~/.yarn/patches/sandboxed-module-npm-2.0.4-f8b45aacc9.patch",
"request/tough-cookie": "5.1.2",
"request/form-data": "2.5.5",
@@ -99,7 +101,6 @@
"knip": "5.64.1",
"eslint-plugin-testing-library": "7.5.3",
"chart.js": "4.0.1",
"mock-fs": "5.2.0",
"@customerio/cdp-analytics-node": "0.3.9",
"@google-cloud/bigquery": "8.1.1",
"moment": "2.29.4",
-29
View File
@@ -1,29 +0,0 @@
{
"extends": [
"eslint:recommended",
"standard",
"prettier"
],
"plugins": [
"unicorn"
],
"parserOptions": {
"ecmaVersion": 2020
},
"env": {
"node": true
},
"rules": {
// Do not allow importing of implicit dependencies.
"import/no-extraneous-dependencies": "error",
"unicorn/prefer-node-protocol": "error"
},
"overrides": [
// Extra rules for Cypress tests
{ "files": ["**/*.spec.ts"], "extends": ["plugin:cypress/recommended"] }
],
"ignorePatterns": [
"hotfix/",
"develop/"
]
}
+7
View File
@@ -54,6 +54,13 @@ RUN --mount=type=cache,target=/root/.cache \
# Add the actual source files
# ---------------------------
COPY --parents libraries/ services/ tools/migrations/ /overleaf/
# Syntax-check all server-side ESM modules before the expensive webpack
# compile. node --check parses without executing, so it's fast and safe.
# Catches things like escaped backticks from sed substitutions that webpack
# never sees (it only bundles frontend code).
RUN find services/web/app/src services/web/modules -name '*.mjs' | xargs node --check
RUN --mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/root/.yarn/berry/cache,id=server-ce-yarn-cache \
--mount=type=tmpfs,target=/usr/local/share/.cache/yarn \
+17 -5
View File
@@ -25,6 +25,7 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
unattended-upgrades \
build-essential wget net-tools unzip time poppler-utils optipng strace nginx git python3 python-is-python3 zlib1g-dev libpcre3-dev gettext-base libwww-perl ca-certificates curl gnupg \
qpdf \
pandoc \
# upgrade base-image, batch all the upgrades together, rather than installing them on-by-one (which is slow!)
&& unattended-upgrade --verbose --no-minimal-upgrade-steps \
# install Node.js https://github.com/nodesource/distributions#nodejs
@@ -48,6 +49,12 @@ RUN curl -fsSL "https://github.com/quarto-dev/quarto-cli/releases/download/v${QU
&& mkdir -p /var/www/.cache/quarto /var/www/.local/share \
&& chown -R www-data:www-data /var/www/.cache /var/www/.local
# Install official Typst binary (Quarto bundles a modified fork without --synctex)
# ---------------------------------------------------------------------------------
ARG TYPST_VERSION=0.13.1
RUN curl -fsSL "https://github.com/typst/typst/releases/download/v${TYPST_VERSION}/typst-x86_64-unknown-linux-musl.tar.xz" \
| tar -xJC /usr/local/bin --strip-components=1 "typst-x86_64-unknown-linux-musl/typst"
# Pre-install popular Quarto extensions
# -----------------------------------------------------------------------
# Extensions land in /opt/quarto-extensions/_extensions/<author>/<name>/.
@@ -98,6 +105,15 @@ RUN apt-get update \
opencv-python-headless tqdm \
&& rm -rf /var/lib/apt/lists/* /root/.cache
# Install Inkscape (for the LaTeX svg package via shell-escape)
# Must come AFTER the pip installs above: inkscape pulls in python3-numpy via
# apt, which would block pip from upgrading numpy. With pip's numpy already in
# /usr/local/lib, apt installs its own copy into /usr/lib alongside it — no
# conflict — and Python resolves /usr/local/lib first at import time.
RUN apt-get update \
&& apt-get install -y inkscape \
&& rm -rf /var/lib/apt/lists/*
# Install decktape + headless Chromium (for exporting RevealJS decks to PDF)
# -----------------------------------------------------------------------
# decktape drives a headless Chromium (via Puppeteer) to print the rendered
@@ -142,15 +158,11 @@ RUN mkdir /install-tl-unx \
&& echo "tlpdbopt_autobackup 0" >> /install-tl-unx/texlive.profile \
&& echo "tlpdbopt_install_docfiles 0" >> /install-tl-unx/texlive.profile \
&& echo "tlpdbopt_install_srcfiles 0" >> /install-tl-unx/texlive.profile \
&& echo "selected_scheme scheme-basic" >> /install-tl-unx/texlive.profile \
&& echo "selected_scheme scheme-full" >> /install-tl-unx/texlive.profile \
&& echo "TEXDIR /usr/local/texlive" >> /install-tl-unx/texlive.profile \
&& /install-tl-unx/install-tl \
-profile /install-tl-unx/texlive.profile \
-repository ${TEXLIVE_MIRROR} \
&& /usr/local/texlive/bin/x86_64-linux/tlmgr install \
--repository ${TEXLIVE_MIRROR} \
latexmk \
texcount \
&& rm -rf /install-tl-unx
+1
View File
@@ -1,3 +1,4 @@
export ENABLE_PANDOC_CONVERSIONS=true
export CHAT_HOST=127.0.0.1
export CLSI_HOST=127.0.0.1
export DOCSTORE_HOST=127.0.0.1
+1
View File
@@ -50,6 +50,7 @@ const TMP_DIR = '/var/lib/overleaf/tmp'
const settings = {
clsi: {
optimiseInDocker: process.env.OPTIMISE_PDF === 'true',
latexShellEscape: process.env.OVERLEAF_LATEX_SHELL_ESCAPE === 'true',
},
brandPrefix: '',
+33
View File
@@ -0,0 +1,33 @@
import { defineConfig, globalIgnores } from 'eslint/config'
import cypress from 'eslint-plugin-cypress/flat'
import path from 'node:path'
import baseConfig from '../eslint.config.mjs'
const ROOT_DIR = path.resolve(import.meta.dirname, '..')
export default defineConfig([
globalIgnores(['**/hotfix/', '**/develop/']),
{
basePath: ROOT_DIR,
extends: baseConfig,
languageOptions: {
ecmaVersion: 2020,
},
},
{
// The cypress block in baseConfig has patterns rooted at the
// monorepo root (`server-ce/test/helpers/*.ts`). When ESLint loads
// this file (server-ce/eslint.config.mjs) as the closest config --
// which happens when running `yarn run lint` from
// /overleaf/server-ce/test/ -- patterns from baseConfig are
// resolved relative to /overleaf/server-ce/, so those cross-dir
// patterns don't match. Re-declare with paths relative to this
// config file.
files: [
'test/helpers/*.ts',
'test/cypress/support/*.{js,jsx,mjs,cjs,ts,tsx}',
'**/*.spec.ts',
],
...cypress.configs.recommended,
},
])
+3 -3
View File
@@ -48,9 +48,9 @@ server {
rewrite ^/project/([0-9a-f]+)/user/([0-9a-f]+)/build/([0-9a-f-]+)/output/(.+)$ /$4 break;
root /var/lib/overleaf/data/output/$1-$2/generated-files/$3/;
}
# handle output files for anonymous users
location ~ ^/project/([0-9a-f]+)/build/([0-9a-f-]+)/output/(.+)$ {
rewrite ^/project/([0-9a-f]+)/build/([0-9a-f-]+)/output/(.+)$ /$3 break;
# handle output files for anonymous users (project ID may be a UUID with hyphens for conversions)
location ~ ^/project/([0-9a-f-]+)/build/([0-9a-f-]+)/output/(.+)$ {
rewrite ^/project/([0-9a-f-]+)/build/([0-9a-f-]+)/output/(.+)$ /$3 break;
root /var/lib/overleaf/data/output/$1/generated-files/$2/;
}
+1
View File
@@ -47,6 +47,7 @@ http {
gzip_proxied any; # allow upstream server to compress.
client_max_body_size 500m;
client_body_timeout 15m;
# gzip_vary on;
# gzip_proxied any;
+20
View File
@@ -9,6 +9,26 @@ server {
internal;
}
# File upload endpoints: extended timeouts for large files on slow connections.
# proxy_request_buffering off: forward the request body to Node.js immediately
# rather than buffering first, so Node.js can send a keepalive response byte
# before the full body arrives (preventing upstream proxy "first-byte" timeouts).
# proxy_buffering off: forward that keepalive byte to Traefik/LB without delay.
location ~ ^/project/[^/]+/upload$ {
proxy_pass http://127.0.0.1:4000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_request_buffering off;
proxy_buffering off;
proxy_read_timeout 15m;
proxy_send_timeout 15m;
send_timeout 15m;
client_body_timeout 15m;
client_max_body_size 550m;
}
location / {
proxy_pass http://127.0.0.1:4000;
proxy_http_version 1.1;
+1
View File
@@ -11,6 +11,7 @@ export TEX_LIVE_DOCKER_IMAGE ?= us-east1-docker.pkg.dev/overleaf-ops/ol-docker/t
export ALL_TEX_LIVE_DOCKER_IMAGES ?= us-east1-docker.pkg.dev/overleaf-ops/ol-docker/texlive-full:2023.1,us-east1-docker.pkg.dev/overleaf-ops/ol-docker/texlive-full:2022.1
export IMAGE_TAG_CE ?= sharelatex/sharelatex:main
export IMAGE_TAG_PRO ?= us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pro:main
export IMAGE_TAG_GIT_BRIDGE ?= us-east1-docker.pkg.dev/overleaf-ops/ol-docker/git-bridge:main
export CYPRESS_SHARD ?=
export COMPOSE_PROJECT_NAME ?= test
export USER_UID=$(shell id -u)
+1
View File
@@ -245,6 +245,7 @@ describe('admin panel', function () {
'Deleted Projects',
'Audit Log',
'Sessions',
'Personal Access Tokens',
]
cy.findAllByRole('tab').should('have.length', tabs.length)
tabs.forEach(tabName => {
+7 -6
View File
@@ -78,12 +78,12 @@ services:
working_dir: $PWD
volumes:
- $PWD:$PWD
- $MONOREPO/libraries:$MONOREPO/libraries:ro
- $MONOREPO/node_modules:$MONOREPO/node_modules:ro
- $MONOREPO/.yarn:$MONOREPO/.yarn:ro
- $MONOREPO/.yarnrc.yml:$MONOREPO/.yarnrc.yml:ro
- $MONOREPO/package.json:$MONOREPO/package.json:ro
- $MONOREPO/yarn.lock:$MONOREPO/yarn.lock:ro
- $MONOREPO/libraries:$MONOREPO/libraries
- $MONOREPO/node_modules:$MONOREPO/node_modules
- $MONOREPO/.yarn:$MONOREPO/.yarn
- $MONOREPO/.yarnrc.yml:$MONOREPO/.yarnrc.yml
- $MONOREPO/package.json:$MONOREPO/package.json
- $MONOREPO/yarn.lock:$MONOREPO/yarn.lock
environment:
MONOREPO:
CYPRESS_SHARD:
@@ -130,6 +130,7 @@ services:
ALL_TEX_LIVE_DOCKER_IMAGES:
IMAGE_TAG_CE:
IMAGE_TAG_PRO:
IMAGE_TAG_GIT_BRIDGE:
healthcheck:
test: curl --fail http://localhost/status
interval: 3s
+2
View File
@@ -44,6 +44,7 @@ describe('editor', function () {
cy.log(`change project language to '${lng}'`)
cy.findByRole('button', { name: 'Settings' }).click()
cy.findByRole('dialog').within(() => {
cy.findByRole('tab', { name: 'Spelling and language' }).click()
cy.findByLabelText('Spellcheck language').select(lng)
})
cy.get('body').type('{esc}')
@@ -76,6 +77,7 @@ describe('editor', function () {
cy.log('remove word from dictionary')
cy.findByRole('button', { name: 'Settings' }).click()
cy.findByRole('dialog').within(() => {
cy.findByRole('tab', { name: 'Spelling and language' }).click()
cy.findByLabelText('Dictionary').click()
})
cy.findByTestId('dictionary-modal').within(() => {
+3 -2
View File
@@ -32,11 +32,12 @@ const PATHS = {
const IMAGES = {
CE: process.env.IMAGE_TAG_CE.replace(/:.+/, ''),
PRO: process.env.IMAGE_TAG_PRO.replace(/:.+/, ''),
GIT_BRIDGE: process.env.IMAGE_TAG_GIT_BRIDGE.replace(/:.+/, ''),
}
const LATEST = {
CE: process.env.IMAGE_TAG_CE.replace(/.+:/, '') || 'latest',
PRO: process.env.IMAGE_TAG_PRO.replace(/.+:/, '') || 'latest',
GIT_BRIDGE: 'latest', // TODO, build in CI?
GIT_BRIDGE: process.env.IMAGE_TAG_GIT_BRIDGE.replace(/.+:/, '') || 'latest',
}
function defaultDockerComposeOverride() {
@@ -242,7 +243,7 @@ function setVarsDockerCompose({
cfg.services.sharelatex.image = `${pro ? IMAGES.PRO : IMAGES.CE}:${version === 'latest' ? (pro ? LATEST.PRO : LATEST.CE) : version}`
cfg.services['git-bridge'].image =
`quay.io/sharelatex/git-bridge:${version === 'latest' ? LATEST.GIT_BRIDGE : version}`
`${IMAGES.GIT_BRIDGE}:${version === 'latest' ? LATEST.GIT_BRIDGE : version}`
cfg.services.sharelatex.environment = vars
+1 -1
View File
@@ -333,7 +333,7 @@ describe('SandboxedCompiles', function () {
})
// https://github.com/overleaf/internal/issues/20216
// eslint-disable-next-line mocha/no-skipped-tests
// eslint-disable-next-line mocha/no-pending-tests
describe.skip('unavailable in CE', function () {
if (isExcludedBySharding('CE_CUSTOM_1')) return
startWith({ pro: false, vars: enabledVars, resetData: true })
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -2
View File
@@ -54,6 +54,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -66,8 +70,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/chat/reports:/overleaf/services/chat/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
+1
View File
@@ -1,5 +1,6 @@
chat
--dependencies=mongo
--deploy-pipeline=chat
--env-add=
--env-pass-through=
--esmock-loader=False
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+2
View File
@@ -32,6 +32,7 @@ COPY libraries/overleaf-editor-core/package.json /overleaf/libraries/overleaf-ed
COPY libraries/promise-utils/package.json /overleaf/libraries/promise-utils/package.json
COPY libraries/settings/package.json /overleaf/libraries/settings/package.json
COPY libraries/stream-utils/package.json /overleaf/libraries/stream-utils/package.json
COPY libraries/validation-tools/package.json /overleaf/libraries/validation-tools/package.json
COPY services/clsi/package.json /overleaf/services/clsi/package.json
COPY .yarn/patches/ /overleaf/.yarn/patches/
@@ -45,6 +46,7 @@ COPY libraries/overleaf-editor-core/ /overleaf/libraries/overleaf-editor-core/
COPY libraries/promise-utils/ /overleaf/libraries/promise-utils/
COPY libraries/settings/ /overleaf/libraries/settings/
COPY libraries/stream-utils/ /overleaf/libraries/stream-utils/
COPY libraries/validation-tools/ /overleaf/libraries/validation-tools/
COPY services/clsi/ /overleaf/services/clsi/
FROM app AS with-quarto
+10 -4
View File
@@ -25,6 +25,7 @@ IMAGE_CACHE ?= $(IMAGE_REPO):cache-$(shell cat \
$(MONOREPO)/libraries/promise-utils/package.json \
$(MONOREPO)/libraries/settings/package.json \
$(MONOREPO)/libraries/stream-utils/package.json \
$(MONOREPO)/libraries/validation-tools/package.json \
$(MONOREPO)/services/clsi/package.json \
$(MONOREPO)/.yarn/patches/* \
| sha256sum | cut -d '-' -f1)
@@ -54,6 +55,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -67,8 +72,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/clsi/reports:/overleaf/services/clsi/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -165,8 +170,9 @@ test_acceptance_clean:
$(DOCKER_COMPOSE_TEST_ACCEPTANCE) down -v -t 0
test_acceptance_pre_run:
docker pull us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pandoc:3.9
docker pull us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pandoc-staging:3.9
-docker pull us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pandoc:3.9
-docker pull us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pandoc-staging:3.9
-cd ../../ && docker build -t us-east1-docker.pkg.dev/overleaf-ops/ol-docker/pdftocairo:24.02 dockerfiles/pdftocairo
ifneq (,$(wildcard test/acceptance/js/scripts/pre-run))
$(DOCKER_COMPOSE_TEST_ACCEPTANCE) run $(DC_RUN_FLAGS) test_acceptance test/acceptance/js/scripts/pre-run
endif
+9
View File
@@ -145,6 +145,15 @@ app.post(
bodyParser.json({ limit: Settings.compileSizeLimit }),
ConversionController.convertProjectToDocument
)
app.post(
'/convert/pdf-to-jpeg',
FileUploadMiddleware.multerMiddleware,
ConversionController.convertPDFToJPEG
)
app.get(
'/project/:project_id/user/:user_id/build/:build_id/thumbnail',
ConversionController.thumbnailFromBuild
)
if (process.env.NODE_ENV === 'development' && global.__coverage__) {
app.get('/coverage', (req, res) => {
@@ -80,6 +80,10 @@ function compile(req, res, next) {
{ err: error, projectId: request.project_id },
'timeout running compile'
)
} else if (error?.typstCompileFailure) {
// Typst compiled but with errors — treat as a compile failure so
// the frontend shows the error log rather than the old PDF.
status = 'failure'
} else if (error) {
status = 'error'
code = 500
+7
View File
@@ -386,11 +386,18 @@ async function stopCompile(projectId, userId) {
}
async function clearProject(projectId, userId) {
// Kill any live typst watcher before deleting its files.
const compileName = getCompileName(projectId, userId)
await TypstRunner.promises.killTypst(compileName)
const compileDir = getCompileDir(projectId, userId)
await fsPromises.rm(compileDir, { force: true, recursive: true })
}
async function clearProjectWithListing(projectId, userId, allEntries) {
// Kill any live typst watcher (e.g. timedout compile where killTypst
// was not already called) before removing files from under it.
const compileName = getCompileName(projectId, userId)
await TypstRunner.promises.killTypst(compileName)
const compileDir = getCompileDir(projectId, userId)
const exists = await _checkDirectory(compileDir)
+165 -1
View File
@@ -1,4 +1,7 @@
import crypto from 'node:crypto'
import { execFile } from 'node:child_process'
import os from 'node:os'
import { promisify } from 'node:util'
import logger from '@overleaf/logger'
import { expressify } from '@overleaf/promise-utils'
import fs from 'node:fs/promises'
@@ -14,10 +17,16 @@ import RequestParser from './RequestParser.js'
import { pipeline } from 'node:stream/promises'
import Settings from '@overleaf/settings'
import Path from 'node:path'
import { z } from '@overleaf/validation-tools'
const execFileAsync = promisify(execFile)
const CONVERSION_CONFIGS = {
docx: { extension: 'docx' },
markdown: { extension: 'zip' },
html: { extension: 'zip' },
typst: { extension: 'typ' },
latex: { extension: 'tex' },
}
async function convertDocumentToLaTeX(req, res) {
@@ -27,7 +36,7 @@ async function convertDocumentToLaTeX(req, res) {
await fs.unlink(path).catch(() => {})
return res.sendStatus(404)
}
if (!conversionType || !['docx', 'markdown'].includes(conversionType)) {
if (!conversionType || !['docx', 'markdown', 'typst'].includes(conversionType)) {
await fs.unlink(path).catch(() => {})
return res.sendStatus(400)
}
@@ -77,6 +86,51 @@ async function convertDocumentToLaTeX(req, res) {
}
}
const PDFToJPEGQuerySchema = z.object({
mode: z.enum(['preview', 'thumbnail']),
})
async function convertPDFToJPEG(req, res) {
const { path } = req.file
if (!Settings.enablePdfConversions) {
await fs.unlink(path).catch(() => {})
return res.sendStatus(404)
}
const parsed = PDFToJPEGQuerySchema.safeParse(req.query)
if (!parsed.success) {
await fs.unlink(path).catch(() => {})
return res.sendStatus(400)
}
const { mode } = parsed.data
logger.debug({ path, mode }, 'received pdf for conversion to jpeg')
const conversionId = crypto.randomUUID()
let jpegPath
try {
jpegPath = await ConversionManager.promises.convertPDFToJPEGWithLock(
conversionId,
path,
mode
)
} finally {
await fs.unlink(path).catch(() => {})
}
try {
const jpegStat = await fs.stat(jpegPath)
res.setHeader('Content-Length', jpegStat.size)
res.attachment('output.jpg')
res.setHeader('X-Content-Type-Options', 'nosniff')
const readStream = fsSync.createReadStream(jpegPath)
await pipeline(readStream, res)
} finally {
await fs
.rm(Path.dirname(jpegPath), { recursive: true, force: true })
.catch(() => {})
}
}
async function convertProjectToDocument(req, res) {
if (!Settings.enablePandocConversions) {
return res.sendStatus(404)
@@ -204,7 +258,117 @@ async function convertProjectToDocument(req, res) {
}
}
// Generates a JPEG thumbnail of page 1 of the compiled output using
// pdftocairo (poppler-utils). Tries output.pdf first (LaTeX / Quarto-PDF),
// then output-slides.pdf (Quarto RevealJS after a PDF-export compile), then
// falls back to rendering slide 1 of output.html via decktape (normal
// RevealJS preview compile). All temp dirs are cleaned up in finally.
async function thumbnailFromBuild(req, res) {
const { project_id: projectId, user_id: userId, build_id: buildId } = req.params
if (!buildId?.match(OutputCacheManager.BUILD_REGEX)) return res.sendStatus(400)
const compileName = userId ? `${projectId}-${userId}` : projectId
const buildDir = Path.join(
Settings.path.outputDir,
compileName,
OutputCacheManager.CACHE_SUBDIR,
buildId
)
let pdfPath = null
let deckTapeDir = null
for (const name of ['output.pdf', 'output-slides.pdf']) {
try {
const p = Path.join(buildDir, name)
await fs.access(p)
pdfPath = p
break
} catch {}
}
if (!pdfPath) {
const htmlPath = Path.join(buildDir, 'output.html')
try {
await fs.access(htmlPath)
deckTapeDir = await fs.mkdtemp(Path.join(os.tmpdir(), 'clsi-deck-'))
const chromeHome = Path.join(deckTapeDir, 'chrome')
await fs.mkdir(chromeHome, { recursive: true })
const slidePdf = Path.join(deckTapeDir, 'slide1.pdf')
await execFileAsync(
'decktape',
[
'--slides', '1',
'--chrome-arg=--no-sandbox',
'--chrome-arg=--disable-dev-shm-usage',
'--chrome-arg=--disable-gpu',
`--chrome-arg=--user-data-dir=${chromeHome}/data`,
htmlPath,
slidePdf,
],
{
timeout: 60000,
env: {
...process.env,
HOME: chromeHome,
XDG_CONFIG_HOME: chromeHome,
XDG_CACHE_HOME: chromeHome,
},
}
)
pdfPath = slidePdf
} catch (err) {
logger.warn({ err, projectId, buildId }, 'decktape slide1 thumbnail failed')
if (deckTapeDir) {
await fs.rm(deckTapeDir, { recursive: true, force: true }).catch(() => {})
deckTapeDir = null
}
return res.sendStatus(404)
}
}
const tmpDir = await fs.mkdtemp(Path.join(os.tmpdir(), 'clsi-thumb-'))
const outputBase = Path.join(tmpDir, 'thumb')
const jpegPath = outputBase + '.jpg'
try {
await execFileAsync(
'pdftocairo',
[
'-jpeg',
'-jpegopt', 'quality=90',
'-singlefile',
'-scale-to-x', '794',
'-scale-to-y', '-1',
'-f', '1',
'-l', '1',
pdfPath,
outputBase,
],
{ timeout: 30000 }
)
const jpegStat = await fs.stat(jpegPath)
res.setHeader('Content-Type', 'image/jpeg')
res.setHeader('Content-Length', jpegStat.size)
res.setHeader('Cache-Control', 'public, max-age=86400')
res.setHeader('X-Content-Type-Options', 'nosniff')
const readStream = fsSync.createReadStream(jpegPath)
await pipeline(readStream, res)
} catch (err) {
logger.warn({ err, projectId, buildId }, 'thumbnail generation failed')
if (!res.headersSent) res.sendStatus(500)
} finally {
await fs.rm(tmpDir, { recursive: true, force: true }).catch(() => {})
if (deckTapeDir) {
await fs.rm(deckTapeDir, { recursive: true, force: true }).catch(() => {})
}
}
}
export default {
convertDocumentToLaTeX: expressify(convertDocumentToLaTeX),
convertProjectToDocument: expressify(convertProjectToDocument),
convertPDFToJPEG: expressify(convertPDFToJPEG),
thumbnailFromBuild: expressify(thumbnailFromBuild),
}
+121
View File
@@ -16,8 +16,24 @@ const CONVERSION_CONFIGS = {
inputFilename: 'input.md',
pandocArgs: ['--from', 'markdown'],
},
typst: {
inputFilename: 'input.typ',
pandocArgs: ['--from', 'typst'],
},
}
const PDF_TO_JPEG_CONFIGS = {
preview: { width: 794, quality: 90 },
thumbnail: { width: 794, quality: 90 },
}
const PDF_TO_JPEG_INPUT_FILENAME = 'input.pdf'
const PDF_TO_JPEG_OUTPUT_FILENAME = 'output.jpg'
const PDF_TO_JPEG_OUTPUT_BASENAME = Path.basename(
PDF_TO_JPEG_OUTPUT_FILENAME,
'.jpg'
)
async function convertToLaTeXWithLock(conversionId, inputPath, conversionType) {
const conversionDir = Path.join(Settings.path.compilesDir, conversionId)
const lock = LockManager.acquire(conversionDir)
@@ -150,6 +166,44 @@ const LATEX_EXPORT_CONFIGS = {
'markdown',
],
},
html: {
fileExtension: 'html',
compressOutput: true,
getPandocArgs: ({ outputPath }) => [
'--output',
outputPath,
'--from',
'latex',
'--to',
'html',
'--standalone',
],
},
typst: {
fileExtension: 'typ',
compressOutput: false,
getPandocArgs: ({ outputPath }) => [
'--output',
outputPath,
'--from',
'latex',
'--to',
'typst',
],
},
latex: {
fileExtension: 'tex',
compressOutput: false,
getPandocArgs: ({ outputPath }) => [
'--output',
outputPath,
'--from',
'typst',
'--to',
'latex',
'--standalone',
],
},
}
async function convertLaTeXToDocumentInDirWithLock(
@@ -298,9 +352,76 @@ async function convertLaTeXToDocumentInDir(
return Path.join(compileDir, finalOutputName)
}
async function convertPDFToJPEGWithLock(conversionId, inputPath, mode) {
const conversionDir = Path.join(Settings.path.compilesDir, conversionId)
const lock = LockManager.acquire(conversionDir)
try {
return await convertPDFToJPEG(conversionId, conversionDir, inputPath, mode)
} finally {
lock.release()
}
}
async function convertPDFToJPEG(conversionId, conversionDir, inputPath, mode) {
const config = PDF_TO_JPEG_CONFIGS[mode]
await fs.mkdir(conversionDir, { recursive: true })
const newSourcePath = Path.join(conversionDir, PDF_TO_JPEG_INPUT_FILENAME)
await fs.copyFile(inputPath, newSourcePath)
const dstPath = Path.join(conversionDir, PDF_TO_JPEG_OUTPUT_FILENAME)
try {
const { stdout, stderr, exitCode } = await CommandRunner.promises.run(
conversionId,
[
'pdftocairo',
'-jpeg',
'-jpegopt',
`quality=${config.quality}`,
'-singlefile',
'-scale-to-x',
config.width.toString(),
'-scale-to-y',
'-1', // maintain aspect ratio
PDF_TO_JPEG_INPUT_FILENAME,
PDF_TO_JPEG_OUTPUT_BASENAME,
],
conversionDir,
Settings.pdftocairoImage,
Settings.conversionTimeoutSeconds * 1000,
{},
'conversions',
null
)
if (exitCode !== 0) {
throw new OError('Non-zero exit code from pdftocairo', {
exitCode,
stderr,
})
}
logger.debug(
{ stdout, stderr, exitCode },
'pdf-to-jpeg conversion completed'
)
const stat = await fs.lstat(dstPath)
if (!stat.isFile()) {
throw new OError('output.jpg is not a regular file', { stat })
}
// Clean up the source PDF to leave only the conversion result
await fs.unlink(newSourcePath).catch(() => {})
} catch (error) {
await fs.rm(conversionDir, { force: true, recursive: true }).catch(() => {})
throw new OError('pdf-to-jpeg conversion failed').withCause(error)
}
return dstPath
}
export default {
promises: {
convertToLaTeXWithLock,
convertLaTeXToDocumentInDirWithLock,
convertPDFToJPEGWithLock,
},
}
+4
View File
@@ -197,6 +197,10 @@ function _buildLatexCommand(mainFile, opts = {}) {
command.push(...opts.flags)
}
if (Settings.clsi?.latexShellEscape) {
command.push('-shell-escape')
}
// TeX Engine selection. A .tex project may carry a non-LaTeX compiler value
// (e.g. 'quarto', the fork-wide default for Project.compiler) because the
// runner is chosen by file extension, not by this setting. In that case fall
+8
View File
@@ -209,8 +209,16 @@ export default ResourceWriter = {
return callback(error)
}
// Project input resources are in outputFiles only to be served from
// the output cache (HTML media exception in OutputFileFinder). They
// must never be deleted here — incremental compiles don't re-sync
// unchanged binary files, so deleting them would leave them missing
// for Quarto and for _appendMissingResourceWarnings.
const incomingPaths = new Set(resources.map(r => r.path))
const jobs = []
for (const { path } of outputFiles || []) {
if (incomingPaths.has(path)) continue
const shouldDelete = ResourceWriter.isExtraneousFile(path)
if (shouldDelete) {
jobs.push(callback =>
+455 -66
View File
@@ -1,96 +1,485 @@
import Path from 'node:path'
import { spawn } from 'node:child_process'
import { promisify } from 'node:util'
import logger from '@overleaf/logger'
import Settings from '@overleaf/settings'
import CommandRunner from './CommandRunner.js'
import fs from 'node:fs'
// Maps currently-running Typst jobs: compileName → PID (or docker container id)
// ---------------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------------
// Max lines kept from the current compile cycle (prevents unbounded growth
// for documents that produce many warnings).
const MAX_LOG_LINES = 500
// How long to wait for the watcher process to emit its first output.
const WATCH_START_TIMEOUT_MS = 15_000
// Matches the start-of-compile marker typst watch emits before each cycle:
// "[HH:MM:SS] compiling ..."
// Used to reset the line buffer so stale output from a failed compile that
// didn't emit a "compiled with errors" footer cannot bleed into the next log.
const COMPILE_START_RE = /^\[\d{2}:\d{2}:\d{2}\] compiling/
// Matches the three terminal lines that typst watch emits at the end of each
// compile cycle regardless of outcome:
// "[HH:MM:SS] compiled successfully in 42ms"
// "[HH:MM:SS] compiled with warnings in 42ms"
// "[HH:MM:SS] compiled with errors"
const COMPILE_DONE_RE = /compiled (successfully|with (errors|warnings))/
// Signals FileId exhaustion in a long-lived typst process (typst issue #7434).
const FILE_ID_EXHAUSTION_RE = /ran out of file ids/i
// Proactively restart the watcher before FileId exhaustion.
// Typst uses ~65 IDs per compile; 1000 compiles ≈ 65 000 — safely under 65 535.
const MAX_COMPILES_BEFORE_RESTART = 1000
// typst watch emits the "[HH:MM:SS] compiled with errors" status line FIRST,
// then the full diagnostic output (file:line:col, code snippets) AFTERWARDS.
// We buffer post-done lines and resolve after this delay if no new compile
// cycle starts sooner. 150 ms is well above the ~1 chunk latency for typst's
// diagnostic flush and imperceptible on top of a typical compile time.
const FLUSH_DELAY_MS = 150
// ---------------------------------------------------------------------------
// State (module-level, never exported)
// ---------------------------------------------------------------------------
// Active cold-start compile jobs (Docker fallback): compileName → PID
const ProcessTable = {}
// Compiles a standalone Typst document (.typ) straight to output.pdf. We reuse
// the Typst that ships inside Quarto via `quarto typst compile`, so there is no
// extra binary to install. This is deliberately the simplest of the three
// runners: Typst only ever produces a PDF, so there is no format detection,
// no HTML asset directory and no extension merging (cf. QuartoRunner).
function runTypst(compileName, options, callback) {
const { directory, mainFile, image, environment, compileGroup } = options
const timeout = options.timeout || 60000
// Long-lived watcher processes: compileName → WatchEntry
// WatchEntry shape:
// proc ChildProcess
// directory compile dir (absolute path)
// mainFile root .typ filename
// environment env vars passed to the runner
// compilationCount total successful compile cycles on this watcher
// restartPending flag to restart at the next runTypst call
// accumulator incomplete trailing line from the last data chunk
// currentLines lines accumulated in the current phase (pre-done or
// post-done, see _onWatcherData for the two-phase logic)
// doneResult { preLines, compiledWithErrors } held between the
// COMPILE_DONE_RE line and the post-done diagnostic flush;
// null when not in post-done phase
// flushTimeout timeout handle that finalises doneResult when no next
// compile cycle starts within FLUSH_DELAY_MS
// pendingResolvers Array<{resolve, reject, timeoutHandle}>
// pendingResult compile result cached when typst finished before a
// resolver was registered (race-condition safety net)
const WatchTable = {}
logger.debug(
{ directory, timeout, mainFile, compileGroup },
'starting typst compile'
)
// PIDs we have intentionally killed, so the close handler can distinguish
// an expected exit from an unexpected crash.
const _killedWatchPids = new Set()
const command = _buildTypstCommand(mainFile)
// ---------------------------------------------------------------------------
// Public entry point
// ---------------------------------------------------------------------------
ProcessTable[compileName] = CommandRunner.run(
compileName,
command,
directory,
image,
timeout,
environment || {},
compileGroup,
null,
function (error, output) {
delete ProcessTable[compileName]
async function runTypstAsync(compileName, options) {
// Docker / sandboxed mode: fall back to a cold-start compile per request.
if (Settings.clsi?.dockerRunner) {
return _runColdStart(compileName, options)
}
// Propagate real process-level errors (killed, timed out) but NOT
// ordinary non-zero exit codes from Typst itself. A compile failure
// (exit code 1) is not a server error — the absence of output.pdf is
// enough for CompileController to return 'failure'.
if (error && (error.terminated || error.timedout)) {
return callback(error)
}
const timeout = options.timeout || 60_000
const entry = WatchTable[compileName]
// On exit-code-1 errors LocalCommandRunner attaches stdout to the error
// object; merge it so _writeLogOutput can persist it.
const combined = output || (error ? { stdout: error.stdout || '' } : null)
_writeLogOutput(compileName, directory, combined, () =>
callback(null, combined)
)
const needsStart =
!entry ||
entry.restartPending ||
entry.proc.exitCode !== null
if (needsStart) {
if (entry) _killWatchEntry(compileName)
// _startWatcher spawns the process, registers all handlers, then calls
// _waitForNextCompile synchronously — the resolver is in pendingResolvers
// before any I/O event can fire, eliminating the race condition.
const result = await _startWatcher(compileName, options)
await _writeLogOutputAsync(compileName, options.directory, result)
if (result.compiledWithErrors) {
throw Object.assign(new Error('typst-compile-failure'), {
typstCompileFailure: true,
})
}
return result
}
// Watcher is alive. _waitForNextCompile adds the resolver synchronously
// inside the Promise constructor, before this function yields — safe.
const result = await _waitForNextCompile(compileName, timeout)
await _writeLogOutputAsync(compileName, options.directory, result)
if (result.compiledWithErrors) {
throw Object.assign(new Error('typst-compile-failure'), {
typstCompileFailure: true,
})
}
return result
}
// ---------------------------------------------------------------------------
// Watcher lifecycle
// ---------------------------------------------------------------------------
async function _startWatcher(compileName, options) {
const { directory, mainFile, environment } = options
const timeout = options.timeout || 60_000
const absInput = Path.join(directory, mainFile)
const absOutput = Path.join(directory, 'output.pdf')
const env = { ...process.env, ...(environment || {}) }
logger.debug({ compileName, absInput }, 'starting typst watcher')
const proc = spawn(
'/bin/sh',
['-c', `typst watch "${absInput}" "${absOutput}" 2>&1`],
{
cwd: directory,
env,
stdio: ['ignore', 'pipe', 'ignore'],
detached: true,
}
)
const entry = {
proc,
directory,
mainFile,
environment,
compilationCount: 0,
restartPending: false,
accumulator: '',
currentLines: [],
doneResult: null,
flushTimeout: null,
pendingResolvers: [],
pendingResult: null,
}
WatchTable[compileName] = entry
// Register handlers synchronously — before any I/O events can fire.
proc.stdout.setEncoding('utf8')
proc.stdout.on('data', chunk => _onWatcherData(compileName, chunk))
proc.on('error', err => {
logger.error({ err, compileName }, 'typst watcher process error')
_rejectAllPending(
compileName,
Object.assign(err, { terminated: true })
)
if (WatchTable[compileName]?.proc === proc) {
delete WatchTable[compileName]
}
})
proc.on('close', (code, signal) => {
logger.warn({ code, signal, compileName }, 'typst watcher exited')
const wasKilled = _killedWatchPids.delete(proc.pid)
if (!wasKilled) {
_rejectAllPending(
compileName,
Object.assign(new Error('typst watcher exited unexpectedly'), {
terminated: true,
})
)
}
if (WatchTable[compileName]?.proc === proc) {
delete WatchTable[compileName]
}
})
// typst watch performs an initial compile immediately on startup.
// _waitForNextCompile adds the resolver synchronously here (inside the
// Promise constructor) before we yield, so it will catch that first event.
return _waitForNextCompile(compileName, timeout + WATCH_START_TIMEOUT_MS)
}
function _buildTypstCommand(mainFile) {
// Run through a POSIX shell so stderr (where Typst writes its diagnostics)
// is merged into stdout (2>&1). LocalCommandRunner replaces $COMPILE_DIR
// before the shell sees it; the output path is relative because the shell
// CWD is already the compile directory.
const inputPath = `$COMPILE_DIR/${mainFile}`
const cmd = `quarto typst compile ${inputPath} output.pdf 2>&1`
return ['/bin/sh', '-c', cmd]
function _killWatchEntry(compileName) {
const entry = WatchTable[compileName]
if (!entry) return
clearTimeout(entry.flushTimeout)
delete WatchTable[compileName]
try {
_killedWatchPids.add(entry.proc.pid)
process.kill(-entry.proc.pid) // kill entire process group
} catch (err) {
_killedWatchPids.delete(entry.proc.pid)
logger.warn({ err, compileName }, 'error killing typst watcher process group')
}
}
function _writeLogOutput(compileName, directory, output, callback) {
const content = (output && output.stdout) || ''
if (!content) return callback()
// Write to output.log so the PDF-preview log panel picks it up. Typst's
// `error:`/`warning:` + `┌─ file:line:col` diagnostics are understood by the
// Quarto/Typst log parser on the web side.
const logFile = Path.join(directory, 'output.log')
fs.unlink(logFile, () => {
fs.writeFile(logFile, content, { flag: 'wx' }, err => {
if (err) {
logger.error({ err, compileName, logFile }, 'error writing typst log')
// ---------------------------------------------------------------------------
// Stdout parsing
// ---------------------------------------------------------------------------
function _onWatcherData(compileName, chunk) {
const entry = WatchTable[compileName]
if (!entry) return
entry.accumulator += chunk
const lines = entry.accumulator.split('\n')
entry.accumulator = lines.pop() // keep the incomplete trailing fragment
for (const line of lines) {
if (COMPILE_START_RE.test(line)) {
// A new compile cycle is starting. If we were in the post-done phase
// (collecting diagnostic lines that typst emits AFTER the status line),
// finalise the previous result now — all diagnostics have arrived.
if (entry.doneResult) {
_finalizeCompile(compileName)
}
callback()
})
// Start fresh for the new cycle.
entry.currentLines = [line]
continue
}
entry.currentLines.push(line)
if (entry.currentLines.length > MAX_LOG_LINES) {
entry.currentLines.shift()
}
if (FILE_ID_EXHAUSTION_RE.test(line)) {
logger.warn({ compileName }, 'typst watcher: FileId exhaustion detected')
entry.restartPending = true
}
if (COMPILE_DONE_RE.test(line)) {
entry.compilationCount++
if (entry.compilationCount >= MAX_COMPILES_BEFORE_RESTART) {
logger.info(
{ compileName, compilationCount: entry.compilationCount },
'typst watcher: scheduling restart (FileId threshold)'
)
entry.restartPending = true
}
// typst watch outputs the "[HH:MM:SS] compiled with errors" status
// line FIRST, then the full diagnostics (file:line:col, code snippets)
// AFTERWARDS. Enter post-done phase: keep accumulating into currentLines
// and flush after FLUSH_DELAY_MS (or immediately when the next compile
// cycle's COMPILE_START_RE arrives, whichever comes first).
entry.doneResult = {
preLines: entry.currentLines,
compiledWithErrors: /compiled with errors/.test(line),
}
entry.currentLines = []
clearTimeout(entry.flushTimeout)
entry.flushTimeout = setTimeout(
() => _finalizeCompile(compileName),
FLUSH_DELAY_MS
)
}
}
}
// Combines the pre-done lines (up to/including the status line) with any
// post-done diagnostic lines and resolves all pending waiters.
function _finalizeCompile(compileName) {
const entry = WatchTable[compileName]
if (!entry || !entry.doneResult) return
clearTimeout(entry.flushTimeout)
entry.flushTimeout = null
const { preLines, compiledWithErrors } = entry.doneResult
entry.doneResult = null
// Merge: status line(s) first, then the post-done diagnostics.
const allLines = preLines.concat(entry.currentLines)
entry.currentLines = []
_resolveAllPending(compileName, {
stdout: allLines.join('\n'),
compiledWithErrors,
})
}
// ---------------------------------------------------------------------------
// Resolver helpers
// ---------------------------------------------------------------------------
function _waitForNextCompile(compileName, timeout) {
return new Promise((resolve, reject) => {
const entry = WatchTable[compileName]
if (!entry) {
return reject(new Error('no typst watcher for ' + compileName))
}
// If typst finished a compile cycle before this resolver was registered
// (race: ResourceWriter wrote files → typst compiled → runTypst called),
// consume the cached result immediately instead of waiting for a timeout.
if (entry.pendingResult) {
const result = entry.pendingResult
entry.pendingResult = null
return resolve(result)
}
// Push synchronously inside the Promise constructor — before the first
// await in the caller, so no data event can fire in the gap.
const timeoutHandle = setTimeout(() => {
entry.pendingResolvers = entry.pendingResolvers.filter(r => r !== resolver)
reject(
Object.assign(new Error('typst compile timed out'), { timedout: true })
)
}, timeout)
const resolver = { resolve, reject, timeoutHandle }
entry.pendingResolvers.push(resolver)
})
}
function _resolveAllPending(compileName, result) {
const entry = WatchTable[compileName]
if (!entry) return
const resolvers = entry.pendingResolvers.splice(0)
if (resolvers.length === 0) {
// typst compiled before a resolver was registered — cache the result so
// the next _waitForNextCompile call can consume it immediately.
entry.pendingResult = result
return
}
for (const { resolve, timeoutHandle } of resolvers) {
clearTimeout(timeoutHandle)
resolve(result)
}
}
function _rejectAllPending(compileName, err) {
const entry = WatchTable[compileName]
if (!entry) return
for (const { reject, timeoutHandle } of entry.pendingResolvers.splice(0)) {
clearTimeout(timeoutHandle)
reject(err)
}
}
// ---------------------------------------------------------------------------
// Log output
// ---------------------------------------------------------------------------
async function _writeLogOutputAsync(compileName, directory, output) {
const content = (output && output.stdout) || ''
if (!content) return
// Write to output.log so the PDF-preview log panel picks it up.
const logFile = Path.join(directory, 'output.log')
try {
await fs.promises.unlink(logFile)
} catch (err) {
if (err.code !== 'ENOENT') {
logger.error({ err, compileName, logFile }, 'error removing typst log')
}
}
try {
await fs.promises.writeFile(logFile, content, { flag: 'wx' })
} catch (err) {
if (err.code !== 'EEXIST') {
logger.error({ err, compileName, logFile }, 'error writing typst log')
}
}
}
// ---------------------------------------------------------------------------
// Cold-start fallback (Docker / sandboxed mode)
// ---------------------------------------------------------------------------
function _runColdStart(compileName, options) {
return new Promise((resolve, reject) => {
const { directory, mainFile, image, environment, compileGroup } = options
const timeout = options.timeout || 60_000
logger.debug({ directory, mainFile, compileGroup }, 'typst cold-start compile')
const inputPath = `$COMPILE_DIR/${mainFile}`
const command = ['/bin/sh', '-c', `typst compile ${inputPath} output.pdf 2>&1`]
ProcessTable[compileName] = CommandRunner.run(
compileName,
command,
directory,
image,
timeout,
environment || {},
compileGroup,
null,
function (error, output) {
delete ProcessTable[compileName]
if (error && (error.terminated || error.timedout)) {
return reject(error)
}
const combined =
output || (error ? { stdout: error.stdout || '' } : null)
_writeLogOutputAsync(compileName, directory, combined).then(
() => {
if (error && combined?.stdout) {
// Non-zero exit with output = typst compile error (not a
// system/infra error). Signal failure so the log panel opens.
reject(
Object.assign(new Error('typst-compile-failure'), {
typstCompileFailure: true,
})
)
} else if (error) {
reject(error)
} else {
resolve(combined)
}
},
reject
)
}
)
})
}
// ---------------------------------------------------------------------------
// Public interface
// ---------------------------------------------------------------------------
function isRunning(compileName) {
return ProcessTable[compileName] != null
return (
ProcessTable[compileName] != null ||
(WatchTable[compileName]?.pendingResolvers.length > 0)
)
}
function killTypst(compileName, callback) {
logger.debug({ compileName }, 'killing running typst compile')
if (!isRunning(compileName)) {
logger.warn({ compileName }, 'no such compile to kill')
return callback(null)
logger.debug({ compileName }, 'killing typst (watcher + any active compile)')
// Cold-start fallback path
if (ProcessTable[compileName] != null) {
CommandRunner.kill(ProcessTable[compileName], () => {})
delete ProcessTable[compileName]
}
CommandRunner.kill(ProcessTable[compileName], callback)
// Reject any in-flight waiters and tear down the watcher process
_rejectAllPending(
compileName,
Object.assign(new Error('terminated'), { terminated: true })
)
_killWatchEntry(compileName)
callback(null)
}
// Kill all watcher processes when the CLSI Node process exits.
process.on('exit', () => {
for (const compileName of Object.keys(WatchTable)) {
_killWatchEntry(compileName)
}
})
const runTypst = (compileName, options, callback) => {
runTypstAsync(compileName, options).then(
result => callback(null, result),
err => callback(err)
)
}
export default {
@@ -98,7 +487,7 @@ export default {
runTypst,
killTypst,
promises: {
runTypst: promisify(runTypst),
runTypst: runTypstAsync,
killTypst: promisify(killTypst),
},
}
+1 -1
View File
@@ -1,7 +1,7 @@
clsi
--data-dirs=cache,compiles,output
--dependencies=
--env-add=DOWNLOAD_HOST=http://clsi-nginx:8080,ALLOWED_COMPILE_GROUPS=clsi-perf simple-latex-file,ENABLE_PDF_CACHING=true,PDF_CACHING_ENABLE_WORKER_POOL=true,ALLOWED_IMAGES=quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9,TEXLIVE_IMAGE=quay.io/sharelatex/texlive-full:2025.1,TEX_LIVE_IMAGE_NAME_OVERRIDE=us-east1-docker.pkg.dev/overleaf-ops/ol-docker,TEXLIVE_IMAGE_USER=tex,SANDBOXED_COMPILES=true,SANDBOXED_COMPILES_HOST_DIR_COMPILES=$PWD/compiles,SANDBOXED_COMPILES_HOST_DIR_OUTPUT=$PWD/output,ENABLE_PANDOC_CONVERSIONS=true
--env-add=DOWNLOAD_HOST=http://clsi-nginx:8080,ALLOWED_COMPILE_GROUPS=clsi-perf simple-latex-file,ENABLE_PDF_CACHING=true,PDF_CACHING_ENABLE_WORKER_POOL=true,ALLOWED_IMAGES=quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9 quay.io/sharelatex/pdftocairo:24.02,TEXLIVE_IMAGE=quay.io/sharelatex/texlive-full:2025.1,TEX_LIVE_IMAGE_NAME_OVERRIDE=us-east1-docker.pkg.dev/overleaf-ops/ol-docker,TEXLIVE_IMAGE_USER=tex,SANDBOXED_COMPILES=true,SANDBOXED_COMPILES_HOST_DIR_COMPILES=$PWD/compiles,SANDBOXED_COMPILES_HOST_DIR_OUTPUT=$PWD/output,ENABLE_PANDOC_CONVERSIONS=true,ENABLE_PDF_CONVERSIONS=true
--env-pass-through=
--esmock-loader=False
--node-version=24.14.1
@@ -31,6 +31,9 @@ module.exports = {
parseInt(process.env.CLSI_CONVERSION_TIMEOUT_SECONDS, 10) || 60,
pandocImage: process.env.PANDOC_IMAGE || 'quay.io/sharelatex/pandoc:3.9',
enablePandocConversions: process.env.ENABLE_PANDOC_CONVERSIONS === 'true',
pdftocairoImage:
process.env.PDFTOCAIRO_IMAGE || 'quay.io/sharelatex/pdftocairo:24.02',
enablePdfConversions: process.env.ENABLE_PDF_CONVERSIONS === 'true',
maxUploadSize: 50 * 1024 * 1024,
internal: {
+2 -1
View File
@@ -30,7 +30,7 @@ services:
ALLOWED_COMPILE_GROUPS: clsi-perf simple-latex-file
ENABLE_PDF_CACHING: true
PDF_CACHING_ENABLE_WORKER_POOL: true
ALLOWED_IMAGES: quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9
ALLOWED_IMAGES: quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9 quay.io/sharelatex/pdftocairo:24.02
TEXLIVE_IMAGE: quay.io/sharelatex/texlive-full:2025.1
TEX_LIVE_IMAGE_NAME_OVERRIDE: us-east1-docker.pkg.dev/overleaf-ops/ol-docker
TEXLIVE_IMAGE_USER: tex
@@ -38,6 +38,7 @@ services:
SANDBOXED_COMPILES_HOST_DIR_COMPILES: $PWD/compiles
SANDBOXED_COMPILES_HOST_DIR_OUTPUT: $PWD/output
ENABLE_PANDOC_CONVERSIONS: true
ENABLE_PDF_CONVERSIONS: true
volumes:
- ./reports:/overleaf/services/clsi/reports
- ./compiles:/overleaf/services/clsi/compiles
+2 -1
View File
@@ -53,7 +53,7 @@ services:
ALLOWED_COMPILE_GROUPS: clsi-perf simple-latex-file
ENABLE_PDF_CACHING: true
PDF_CACHING_ENABLE_WORKER_POOL: true
ALLOWED_IMAGES: quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9
ALLOWED_IMAGES: quay.io/sharelatex/texlive-full:2017.1 quay.io/sharelatex/texlive-full:2025.1 quay.io/sharelatex/pandoc:3.9 quay.io/sharelatex/pdftocairo:24.02
TEXLIVE_IMAGE: quay.io/sharelatex/texlive-full:2025.1
TEX_LIVE_IMAGE_NAME_OVERRIDE: us-east1-docker.pkg.dev/overleaf-ops/ol-docker
TEXLIVE_IMAGE_USER: tex
@@ -61,6 +61,7 @@ services:
SANDBOXED_COMPILES_HOST_DIR_COMPILES: $PWD/compiles
SANDBOXED_COMPILES_HOST_DIR_OUTPUT: $PWD/output
ENABLE_PANDOC_CONVERSIONS: true
ENABLE_PDF_CONVERSIONS: true
depends_on:
clsi-nginx:
condition: service_started
+1 -1
View File
@@ -23,6 +23,7 @@
"@overleaf/promise-utils": "workspace:*",
"@overleaf/settings": "workspace:*",
"@overleaf/stream-utils": "workspace:*",
"@overleaf/validation-tools": "workspace:*",
"archiver": "5.3.2",
"async": "^3.2.5",
"body-parser": "1.20.4",
@@ -45,7 +46,6 @@
"mocha": "^11.1.0",
"mocha-junit-reporter": "^2.2.1",
"mocha-multi-reporters": "^1.5.1",
"mock-fs": "^5.1.2",
"node-fetch": "^2.7.0",
"nyc": "^17.1.0",
"sinon": "~9.0.1",
@@ -0,0 +1,83 @@
import Client from './helpers/Client.js'
import ClsiApp from './helpers/ClsiApp.js'
import Path from 'node:path'
import fs from 'node:fs/promises'
import { promisify } from 'node:util'
import { execFile as execFileCb } from 'node:child_process'
import { expect } from 'chai'
const execFile = promisify(execFileCb)
const FIXTURE_PDF = Path.join(import.meta.dirname, '../fixtures/minimal.pdf')
const MODE_EXPECTATIONS = {
preview: { width: 794 },
thumbnail: { width: 190 },
}
async function writeResponseToTempfile(response) {
const buffer = Buffer.from(await response.arrayBuffer())
const tmpPath = `/tmp/clsi-acceptance-pdf-to-jpeg-${crypto.randomUUID()}.jpg`
await fs.writeFile(tmpPath, buffer)
return { tmpPath, buffer }
}
describe('pdf-to-jpeg conversion', function () {
before(async function () {
await ClsiApp.ensureRunning()
})
for (const [mode, { width: expectedWidth }] of Object.entries(
MODE_EXPECTATIONS
)) {
describe(`with mode=${mode}`, function () {
let response
let tmpPath
let buffer
before(async function () {
response = await Client.convertPdfToJpeg(FIXTURE_PDF, mode)
expect(response.status).to.equal(200)
;({ tmpPath, buffer } = await writeResponseToTempfile(response))
})
after(async function () {
if (tmpPath) {
await fs.unlink(tmpPath).catch(() => {})
}
})
it('returns a JPEG (per `file`)', async function () {
const { stdout } = await execFile('file', ['--brief', tmpPath])
expect(stdout).to.match(/JPEG image data/)
})
it(`has the expected width of ${expectedWidth}px`, async function () {
const { stdout } = await execFile('identify', [
'-format',
'%w %h',
tmpPath,
])
const [width, height] = stdout.trim().split(' ').map(Number)
expect(width).to.equal(expectedWidth)
// A4 portrait is taller than wide; height must be positive and
// larger than the width (so the aspect ratio was preserved).
expect(height).to.be.greaterThan(width)
})
it('returns a non-empty body matching Content-Length', function () {
expect(buffer.length).to.be.greaterThan(0)
expect(buffer.length).to.equal(
Number(response.headers.get('content-length'))
)
})
})
}
describe('with an unsupported mode', function () {
it('returns 400', async function () {
const response = await Client.convertPdfToJpeg(FIXTURE_PDF, 'not-a-mode')
expect(response.status).to.equal(400)
})
})
})
@@ -53,6 +53,16 @@ async function convertDocument(path, type) {
}
}
async function convertPdfToJpeg(path, mode) {
const formData = new FormData()
formData.append('qqfile', await fsPromises.readFile(path), 'input.pdf')
return await fetch(`${host}/convert/pdf-to-jpeg?mode=${mode}`, {
method: 'POST',
headers: formData.getHeaders(),
body: formData.getBuffer(),
})
}
async function convertProjectToDocument(
projectId,
userId,
@@ -239,6 +249,7 @@ export default {
compile,
convertProjectToDocument,
convertDocument,
convertPdfToJpeg,
stopCompile,
clearCache,
getOutputFile,
@@ -18,6 +18,7 @@ describe('ConversionController', function () {
ctx.documentStat = { size: 5678 }
ctx.Settings = {
enablePandocConversions: true,
enablePdfConversions: true,
path: {
compilesDir: '/compiles',
outputDir: '/output',
@@ -591,6 +592,37 @@ describe('ConversionController', function () {
})
})
describe('with conversionType=html', function () {
beforeEach(async function (ctx) {
ctx.req.query = { type: 'html' }
ctx.fs.stat.resolves(ctx.documentStat)
await ctx.ConversionController.convertProjectToDocument(
ctx.req,
ctx.res,
sinon.stub()
)
})
it('should call convertLaTeXToDocumentInDirWithLock with type=html', function (ctx) {
sinon.assert.calledWith(
ctx.ConversionManager.promises.convertLaTeXToDocumentInDirWithLock,
sinon.match(
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/
),
sinon.match(
/^\/compiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/
),
'main.tex',
'html'
)
})
it('should set the attachment filename with .zip extension', function (ctx) {
sinon.assert.calledWith(ctx.res.attachment, 'output.zip')
})
})
describe('when conversion fails', function () {
beforeEach(async function (ctx) {
ctx.next = sinon.stub()
@@ -77,6 +77,24 @@ const LATEX_TO_DOCUMENT_CASES = [
'--extract-media=.',
],
},
{
type: 'html',
extension: 'html',
compressOutput: true,
pandocArgs: outputId => [
'pandoc',
Path.join('..', 'main.tex'),
'--output',
'main.html',
'--from',
'latex',
'--to',
'html',
'--standalone',
'--resource-path=..',
'--extract-media=.',
],
},
]
describe('ConversionManager', function () {
@@ -1,7 +1,8 @@
import { vi, expect, describe, beforeEach, afterEach, it } from 'vitest'
import Path from 'node:path'
import fs from 'node:fs'
import fsPromises from 'node:fs/promises'
import mockFs from 'mock-fs'
import os from 'node:os'
const MODULE_PATH = Path.join(
import.meta.dirname,
@@ -15,20 +16,19 @@ describe('DraftModeManager', () => {
}))
ctx.DraftModeManager = (await import(MODULE_PATH)).default
ctx.filename = '/mock/filename.tex'
ctx.tmpDir = fs.mkdtempSync(Path.join(os.tmpdir(), 'draft-mode-test-'))
ctx.filename = Path.join(ctx.tmpDir, 'filename.tex')
ctx.contents = `\
\\documentclass{article}
\\begin{document}
Hello world
\\end{document}\
`
mockFs({
[ctx.filename]: ctx.contents,
})
fs.writeFileSync(ctx.filename, ctx.contents)
})
afterEach(() => {
mockFs.restore()
afterEach(ctx => {
fs.rmSync(ctx.tmpDir, { recursive: true })
})
describe('injectDraftMode', () => {
@@ -1,6 +1,6 @@
import sinon from 'sinon'
import { expect, describe, beforeEach, afterEach, it } from 'vitest'
import mockFs from 'mock-fs'
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
const modulePath = path.join(
@@ -8,30 +8,40 @@ const modulePath = path.join(
'../../../app/js/OutputFileFinder'
)
function createTree(base, tree) {
fs.mkdirSync(base, { recursive: true })
for (const [name, content] of Object.entries(tree)) {
const fullPath = path.join(base, name)
if (Buffer.isBuffer(content) || typeof content === 'string') {
fs.writeFileSync(fullPath, content)
} else if (content && content.symlink) {
fs.symlinkSync(content.symlink, fullPath)
} else {
createTree(fullPath, content)
}
}
}
describe('OutputFileFinder', function () {
beforeEach(async function (ctx) {
ctx.OutputFileFinder = (await import(modulePath)).default
ctx.directory = '/test/dir'
ctx.callback = sinon.stub()
mockFs({
[ctx.directory]: {
resource: {
'path.tex': 'a source file',
},
'output.pdf': 'a generated pdf file',
extra: {
'file.tex': 'a generated tex file',
},
'sneaky-file': mockFs.symlink({
path: '../foo',
}),
ctx.directory = fs.mkdtempSync(
path.join(os.tmpdir(), 'output-finder-test-')
)
createTree(ctx.directory, {
resource: {
'path.tex': 'a source file',
},
'output.pdf': 'a generated pdf file',
extra: {
'file.tex': 'a generated tex file',
},
'sneaky-file': { symlink: '../foo' },
})
})
afterEach(function () {
mockFs.restore()
afterEach(function (ctx) {
fs.rmSync(ctx.directory, { recursive: true })
})
describe('findOutputFiles', function () {
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -2
View File
@@ -56,6 +56,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -68,8 +72,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/docstore/reports:/overleaf/services/docstore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
+1
View File
@@ -1,5 +1,6 @@
docstore
--dependencies=mongo,gcs
--deploy-pipeline=docstore
--env-add=
--env-pass-through=
--esmock-loader=False
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -2
View File
@@ -57,6 +57,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -69,8 +73,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/document-updater/reports:/overleaf/services/document-updater/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
@@ -1,5 +1,6 @@
document-updater
--dependencies=mongo,redis
--deploy-pipeline=document-updater
--env-add=
--env-pass-through=
--esmock-loader=False
+3
View File
@@ -0,0 +1,3 @@
declare module 'mongodb-legacy' {
export * from 'mongodb'
}
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -2
View File
@@ -53,6 +53,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -66,8 +70,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/filestore/reports:/overleaf/services/filestore/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
+1
View File
@@ -1,6 +1,7 @@
filestore
--data-dirs=uploads,template_files
--dependencies=s3,gcs
--deploy-pipeline=filestore-readonly
--env-add=ENABLE_CONVERSIONS=true,USE_PROM_METRICS=true,AWS_S3_USER_FILES_STORAGE_CLASS=REDUCED_REDUNDANCY,AWS_S3_USER_FILES_BUCKET_NAME=fake-user-files,AWS_S3_USER_FILES_DEK_BUCKET_NAME=fake-user-files-dek,AWS_S3_TEMPLATE_FILES_BUCKET_NAME=fake-template-files,GCS_USER_FILES_BUCKET_NAME=fake-gcs-user-files,GCS_TEMPLATE_FILES_BUCKET_NAME=fake-gcs-template-files
--env-pass-through=
--esmock-loader=False
+4
View File
@@ -15,6 +15,10 @@ IMAGE_REPO_BRANCH ?= $(IMAGE_REPO):$(BRANCH_NAME_TAG_SAFE)
IMAGE_REPO_MAIN ?= $(IMAGE_REPO):main
IMAGE_REPO_FINAL ?= $(IMAGE_REPO_BRANCH)-$(BUILD_NUMBER)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
runtime-conf:
/opt/envsubst < conf/envsubst_template.json > conf/runtime.json
@@ -5,6 +5,10 @@ import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpHeaders;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpResponse;
import com.google.gson.Gson;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonSyntaxException;
import jakarta.servlet.*;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@@ -107,15 +111,18 @@ public class Oauth2Filter implements Filter {
// fail later (for example, in the unlikely event that the token
// expired between the two requests). In that case, JGit will
// return a 401 without a custom error message.
int statusCode = checkAccessToken(this.oauth2Server, password, getClientIp(request));
if (statusCode == 429) {
AccessTokenCheck check = checkAccessToken(this.oauth2Server, password, getClientIp(request));
if (check.statusCode == 429) {
handleRateLimit(projectId, username, request, response);
return;
} else if (statusCode == 401) {
} else if (check.statusCode == 401 && "token_expired".equals(check.errorCode)) {
handleExpiredAccessToken(projectId, request, response);
return;
} else if (check.statusCode == 401) {
handleBadAccessToken(projectId, request, response);
return;
} else if (statusCode >= 400) {
handleUnknownOauthServerError(projectId, statusCode, request, response);
} else if (check.statusCode >= 400) {
handleUnknownOauthServerError(projectId, check.statusCode, request, response);
return;
}
cred.setAccessToken(password);
@@ -229,8 +236,52 @@ public class Oauth2Filter implements Filter {
"https://www.overleaf.com/learn/how-to/Git_integration"));
}
private int checkAccessToken(String oauth2Server, String accessToken, String clientIp)
private void handleExpiredAccessToken(
String projectId, HttpServletRequest request, HttpServletResponse response)
throws IOException {
Log.debug("[{}] Expired access token, ip={}", projectId, getClientIp(request));
sendResponse(
response,
401,
Arrays.asList(
"Your Overleaf Git authentication token has expired.",
"",
"Generate a new authentication token in your Overleaf Account Settings,",
"then run the git command again."));
}
static class AccessTokenCheck {
final int statusCode;
final String errorCode;
AccessTokenCheck(int statusCode, String errorCode) {
this.statusCode = statusCode;
this.errorCode = errorCode;
}
}
static String parseErrorCode(String body) {
if (body == null || body.isEmpty()) {
return null;
}
try {
JsonElement element = new Gson().fromJson(body, JsonElement.class);
if (element == null || !element.isJsonObject()) {
return null;
}
JsonObject obj = element.getAsJsonObject();
JsonElement codeElement = obj.get("error_code");
if (codeElement == null || codeElement.isJsonNull()) {
return null;
}
return codeElement.getAsString();
} catch (JsonSyntaxException | UnsupportedOperationException e) {
return null;
}
}
private AccessTokenCheck checkAccessToken(
String oauth2Server, String accessToken, String clientIp) throws IOException {
GenericUrl url = new GenericUrl(oauth2Server + "/oauth/token/info?client_ip=" + clientIp);
HttpRequest request = Instance.httpRequestFactory.buildGetRequest(url);
HttpHeaders headers = new HttpHeaders();
@@ -239,8 +290,12 @@ public class Oauth2Filter implements Filter {
request.setThrowExceptionOnExecuteError(false);
HttpResponse response = request.execute();
int statusCode = response.getStatusCode();
String errorCode = null;
if (statusCode >= 400 && statusCode < 500) {
errorCode = parseErrorCode(response.parseAsString());
}
response.disconnect();
return statusCode;
return new AccessTokenCheck(statusCode, errorCode);
}
private void handleUnknownOauthServerError(
@@ -0,0 +1,51 @@
package uk.ac.ic.wlgitbridge.server;
import org.junit.Assert;
import org.junit.Test;
public class Oauth2FilterTest {
@Test
public void parseErrorCode_returnsTokenExpired_whenBodyContainsIt() {
String body = "{\"error\":\"invalid_token\",\"error_code\":\"token_expired\"}";
Assert.assertEquals("token_expired", Oauth2Filter.parseErrorCode(body));
}
@Test
public void parseErrorCode_returnsTokenInvalid_whenBodyContainsIt() {
String body = "{\"error\":\"invalid_token\",\"error_code\":\"token_invalid\"}";
Assert.assertEquals("token_invalid", Oauth2Filter.parseErrorCode(body));
}
@Test
public void parseErrorCode_returnsNull_whenErrorCodeFieldIsMissing() {
String body = "{\"error\":\"invalid_token\"}";
Assert.assertNull(Oauth2Filter.parseErrorCode(body));
}
@Test
public void parseErrorCode_returnsNull_whenBodyIsNull() {
Assert.assertNull(Oauth2Filter.parseErrorCode(null));
}
@Test
public void parseErrorCode_returnsNull_whenBodyIsEmpty() {
Assert.assertNull(Oauth2Filter.parseErrorCode(""));
}
@Test
public void parseErrorCode_returnsNull_whenBodyIsNotJson() {
Assert.assertNull(Oauth2Filter.parseErrorCode("not json at all"));
}
@Test
public void parseErrorCode_returnsNull_whenBodyIsJsonArray() {
Assert.assertNull(Oauth2Filter.parseErrorCode("[1, 2, 3]"));
}
@Test
public void parseErrorCode_returnsNull_whenErrorCodeFieldIsJsonNull() {
String body = "{\"error\":\"invalid_token\",\"error_code\":null}";
Assert.assertNull(Oauth2Filter.parseErrorCode(body));
}
}
-2
View File
@@ -1,2 +0,0 @@
archive/
+1 -1
View File
@@ -1,7 +1,7 @@
let reporterOptions = {}
if (process.env.CI) {
reporterOptions = {
reporter: '/overleaf/node_modules/mocha-multi-reporters',
reporter: require.resolve('mocha-multi-reporters'),
'reporter-options': ['configFile=./test/mocha-multi-reporters.cjs'],
}
}
+6 -2
View File
@@ -59,6 +59,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -71,8 +75,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/history-v1/reports:/overleaf/services/history-v1/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
+1
View File
@@ -1,5 +1,6 @@
history-v1
--dependencies=postgres,gcs,mongo,redis,s3
--deploy-pipeline=history-v1
--env-add=
--env-pass-through=
--esmock-loader=False
+1 -1
View File
@@ -109,7 +109,7 @@ services:
- mongo:127.0.0.1
postgres:
image: postgres:10
image: postgres:14
environment:
POSTGRES_USER: overleaf
POSTGRES_PASSWORD: overleaf
+1 -1
View File
@@ -124,7 +124,7 @@ services:
- mongo:127.0.0.1
postgres:
image: postgres:10
image: postgres:14
environment:
POSTGRES_USER: overleaf
POSTGRES_PASSWORD: overleaf
+2 -1
View File
@@ -44,7 +44,8 @@
"temp": "^0.8.3",
"throng": "^4.0.0",
"tsscmp": "^1.0.6",
"utf-8-validate": "^5.0.4"
"utf-8-validate": "^5.0.4",
"zip-stream": "^7.0.2"
},
"devDependencies": {
"@overleaf/migrations": "workspace:*",
@@ -185,13 +185,30 @@ class BackupBlobStore {
*/
/**
* @typedef {(import('archiver').Archiver)} Archiver
* @typedef {(import('zip-stream').default)} ZipStream
*/
/**
* @typedef {(import('overleaf-editor-core').FileMap)} FileMap
*/
/**
* Promisified wrapper for ZipStream's entry method.
*
* @param {ZipStream} archive
* @param {Buffer|NodeJS.ReadableStream|string} source
* @param {{ name: string }} data
* @return {Promise<void>}
*/
function addEntry(archive, source, data) {
return new Promise((resolve, reject) => {
archive.entry(source, data, err => {
if (err) reject(err)
else resolve()
})
})
}
/**
*
* @param historyId
@@ -254,14 +271,15 @@ async function fetchBlob(historyId, hash, persistor) {
/**
* @typedef {object} AddChunkOptions
* @property {string} [prefix] Should include trailing slash (if length > 0)
* @property {string} [prefix]
* @property {boolean} [useBackupGlobalBlobs]
* @property {boolean} [verbose]
*/
/**
*
* @param {History} history
* @param {Archiver} archive
* @param {ZipStream} archive
* @param {CachedPerProjectEncryptedS3Persistor} projectCache
* @param {string} historyId
* @param {AddChunkOptions} [options]
@@ -272,7 +290,7 @@ async function addChunkToArchive(
archive,
projectCache,
historyId,
{ prefix = '', useBackupGlobalBlobs = false } = {}
{ prefix = '', useBackupGlobalBlobs = false, verbose = false } = {}
) {
const chunkBlobs = new Set()
history.findBlobHashes(chunkBlobs)
@@ -334,9 +352,16 @@ async function addChunkToArchive(
}
content = await blobStore.getStream(hash)
}
archive.append(content, {
if (content == null) {
logger.error({ filePath }, 'File content is empty')
continue
}
await addEntry(archive, content, {
name: `${prefix}${filePath}`,
})
if (verbose) {
logger.info({ filePath: `${prefix}${filePath}` }, 'added to archive')
}
}
})
}
@@ -358,17 +383,20 @@ async function findStartVersionOfLatestChunk(historyId) {
/**
* Restore a project from the latest snapshot
*
* There is an assumption that the database backup has been restored.
* There is an assumption that the database backup
* has been restored.
*
* @param {Archiver} archive
* @param {ZipStream} archive
* @param {string} historyId
* @param {boolean} [useBackupGlobalBlobs]
* @param {boolean} [verbose]
* @return {Promise<void>}
*/
export async function archiveLatestChunk(
archive,
historyId,
useBackupGlobalBlobs = false
useBackupGlobalBlobs = false,
verbose = false
) {
logger.info({ historyId, useBackupGlobalBlobs }, 'Archiving latest chunk')
@@ -386,20 +414,28 @@ export async function archiveLatestChunk(
await addChunkToArchive(backedUpChunk, archive, projectCache, historyId, {
useBackupGlobalBlobs,
verbose,
})
return archive
}
/**
* Fetches all raw blobs from the project and adds them to the archive.
* Fetches all raw blobs from the project and adds
* them to the archive.
*
* @param {string} historyId
* @param {Archiver} archive
* @param {ZipStream} archive
* @param {CachedPerProjectEncryptedS3Persistor} projectCache
* @param {boolean} [verbose]
* @return {Promise<void>}
*/
async function addRawBlobsToArchive(historyId, archive, projectCache) {
async function addRawBlobsToArchive(
historyId,
archive,
projectCache,
verbose = false
) {
const blobKeys = await projectCache.listDirectoryKeys(
projectBlobsBucket,
projectKey.format(historyId)
@@ -411,9 +447,13 @@ async function addRawBlobsToArchive(historyId, archive, projectCache) {
key,
{ autoGunzip: true }
)
archive.append(stream, {
name: path.join(historyId, 'blobs', key),
const entryName = path.join(historyId, 'blobs', key)
await addEntry(archive, stream, {
name: entryName,
})
if (verbose) {
logger.info({ entryName }, 'added to archive')
}
} catch (err) {
logger.warn({ err, path: key }, 'Failed to append blob to archive')
}
@@ -425,17 +465,20 @@ async function addRawBlobsToArchive(historyId, archive, projectCache) {
*
* This can work without the database being backed up.
*
* It will split the project into chunks per directory and download the blobs alongside the chunk.
* It will split the project into chunks per directory
* and download the blobs alongside the chunk.
*
* @param {Archiver} archive
* @param {ZipStream} archive
* @param {string} historyId
* @param {boolean} [useBackupGlobalBlobs]
* @param {boolean} [verbose]
* @return {Promise<void>}
*/
export async function archiveRawProject(
archive,
historyId,
useBackupGlobalBlobs = false
useBackupGlobalBlobs = false,
verbose = false
) {
const projectCache = await getProjectPersistor(historyId)
@@ -454,11 +497,15 @@ export async function archiveRawProject(
const { buffer } = await loadChunkByKey(projectCache, key)
archive.append(buffer, {
name: `${historyId}/chunks/${chunkId}/chunk.json`,
const entryName = `${historyId}/chunks/${chunkId}/chunk.json`
await addEntry(archive, buffer, {
name: entryName,
})
if (verbose) {
logger.info({ entryName }, 'added to archive')
}
}
await addRawBlobsToArchive(historyId, archive, projectCache)
await addRawBlobsToArchive(historyId, archive, projectCache, verbose)
}
export class BackupPersistorError extends OError {}
+4 -4
View File
@@ -6,19 +6,19 @@
*/
'use strict'
const Stream = require('node:stream')
const { pipeline } = require('node:stream/promises')
const zlib = require('node:zlib')
const { WritableBuffer } = require('@overleaf/stream-utils')
/**
* Create a promise for the result of reading a stream to a buffer.
*
* @param {Stream.Readable} readStream
* @param {import('node:stream').Readable} readStream
* @return {Promise<Buffer>}
*/
async function readStreamToBuffer(readStream) {
const bufferStream = new WritableBuffer()
await Stream.promises.pipeline(readStream, bufferStream)
await pipeline(readStream, bufferStream)
return bufferStream.contents()
}
@@ -33,7 +33,7 @@ exports.readStreamToBuffer = readStreamToBuffer
async function gunzipStreamToBuffer(readStream) {
const gunzip = zlib.createGunzip()
const bufferStream = new WritableBuffer()
await Stream.promises.pipeline(readStream, gunzip, bufferStream)
await pipeline(readStream, gunzip, bufferStream)
return bufferStream.contents()
}
@@ -19,7 +19,7 @@ const LAG_TIME_BUCKETS_HRS = [
] // hours
// Configure backup settings to match worker concurrency
configureBackup({ concurrency: UPLOAD_CONCURRENCY, useSecondary: true })
configureBackup({ concurrency: UPLOAD_CONCURRENCY })
let gracefulShutdownInitiated = false
@@ -0,0 +1,129 @@
// Finalise the current chunk for a project and start a new empty chunk
// whose starting snapshot is the end snapshot of the (now-closed) current
// chunk.
//
// This is intended as a recovery tool for projects whose current chunk has
// become corrupted in such a way that further changes can no longer be
// persisted, but where the end snapshot of the current chunk can still be
// computed.
import logger from '@overleaf/logger'
import commandLineArgs from 'command-line-args'
import { Change, Chunk, History, NoOperation } from 'overleaf-editor-core'
import * as redis from '../lib/redis.js'
import knex from '../lib/knex.js'
import knexReadOnly from '../lib/knex_read_only.js'
import { client as mongoClient } from '../lib/mongodb.js'
import chunkStore from '../lib/chunk_store/index.js'
import redisBackend from '../lib/chunk_store/redis.js'
import { loadGlobalBlobs } from '../lib/blob_store/index.js'
import { fileURLToPath } from 'node:url'
import { EventEmitter } from 'node:events'
EventEmitter.defaultMaxListeners = 20
logger.initialize('finalise-chunk')
const optionDefinitions = [
{ name: 'historyId', type: String },
{ name: 'dry-run', alias: 'd', type: Boolean },
]
const options = commandLineArgs(optionDefinitions)
const HISTORY_ID = options.historyId
const DRY_RUN = options['dry-run'] || false
if (!HISTORY_ID) {
console.error('Usage: finalise_chunk.mjs --historyId <id> [--dry-run]')
process.exit(2)
}
async function finaliseCurrentChunk(historyId) {
// Validates the history id and selects the backend (postgres or mongo).
chunkStore.getBackend(historyId)
await loadGlobalBlobs()
const currentChunk = await chunkStore.loadLatest(historyId, {
persistedOnly: true,
})
const startVersion = currentChunk.getStartVersion()
const endVersion = currentChunk.getEndVersion()
const numChanges = currentChunk.getChanges().length
logger.info(
{ historyId, startVersion, endVersion, numChanges },
'loaded current chunk'
)
if (endVersion === startVersion) {
throw new Error(
`current chunk for history ${historyId} is already empty (no changes); refusing to create another empty chunk`
)
}
let nonPersistedChanges
try {
nonPersistedChanges = await redisBackend.getNonPersistedChanges(
historyId,
endVersion
)
} catch (err) {
throw new Error(
`unable to read non-persisted changes from redis for history ${historyId}: ${err.message}`
)
}
if (nonPersistedChanges.length > 0) {
throw new Error(
`history ${historyId} has ${nonPersistedChanges.length} non-persisted change(s) in redis; persist or expire them before running this script`
)
}
const endSnapshot = currentChunk.getSnapshot().clone()
endSnapshot.applyAll(currentChunk.getChanges())
// The chunks table has a unique constraint on (doc_id, end_version), so the
// new chunk cannot share an end_version with the chunk we are closing. Add a
// single NoOperation change to bump end_version by 1 without mutating the
// snapshot.
const recoveryChange = new Change([new NoOperation()], new Date(), [])
const newChunk = new Chunk(
new History(endSnapshot, [recoveryChange]),
endVersion
)
if (DRY_RUN) {
logger.info(
{ historyId, endVersion },
'dry run: would close current chunk and create new empty chunk'
)
return
}
await chunkStore.create(historyId, newChunk)
logger.info(
{ historyId, endVersion },
'closed current chunk and created new empty chunk'
)
}
async function main() {
try {
await finaliseCurrentChunk(HISTORY_ID)
} catch (err) {
logger.fatal({ err, historyId: HISTORY_ID }, 'failed to finalise chunk')
process.exitCode = 1
} finally {
await redis.disconnect()
await mongoClient.close()
await knex.destroy()
await knexReadOnly.destroy()
}
}
const currentScriptPath = fileURLToPath(import.meta.url)
if (process.argv[1] === currentScriptPath) {
main()
}
export { finaliseCurrentChunk }
@@ -17,9 +17,10 @@ const fs = require('node:fs')
const os = require('node:os')
const path = require('node:path')
const util = require('node:util')
const { pipeline } = require('node:stream/promises')
// Something is registering 11 listeners, over the limit of 10, which generates
// a lot of warning noise.
// Something is registering 11 listeners, over the limit
// of 10, which generates a lot of warning noise.
require('node:events').EventEmitter.defaultMaxListeners = 11
const config = require('config')
@@ -27,11 +28,23 @@ const config = require('config')
// eslint-disable-next-line import/no-extraneous-dependencies
const { Storage } = require('@google-cloud/storage')
const isValidUtf8 = require('utf-8-validate')
// zip-stream@7 uses ESM default export
const ZipStream = require('zip-stream').default
function createStorage() {
const opts = {}
if (config.has('persistor.gcs.endpoint.apiEndpoint')) {
opts.apiEndpoint = config.get('persistor.gcs.endpoint.apiEndpoint')
}
if (config.has('persistor.gcs.endpoint.projectId')) {
opts.projectId = config.get('persistor.gcs.endpoint.projectId')
}
return new Storage(opts)
}
const core = require('overleaf-editor-core')
const projectKey = require('@overleaf/object-persistor/src/ProjectKey.js')
const streams = require('../lib/streams')
const ProjectArchive = require('../lib/project_archive')
const {
values: { verbose: VERBOSE },
@@ -53,7 +66,7 @@ if (HISTORY_IDS.length === 0) {
async function listDeletedChunks(historyId) {
const bucketName = config.get('chunkStore.bucket')
const storage = new Storage()
const storage = createStorage()
const [files] = await storage.bucket(bucketName).getFiles({
prefix: projectKey.format(historyId),
versions: true,
@@ -137,7 +150,7 @@ class RecoveryBlobStore {
if (VERBOSE) console.log('fetching blob', hash)
const bucketName = config.get('blobStore.projectBucket')
const storage = new Storage()
const storage = createStorage()
const [files] = await storage.bucket(bucketName).getFiles({
prefix: this.makeProjectBlobKey(hash),
versions: true,
@@ -158,7 +171,7 @@ class RecoveryBlobStore {
async fetchGlobalBlob(hash, destination) {
const bucketName = config.get('blobStore.globalBucket')
const storage = new Storage()
const storage = createStorage()
const file = storage.bucket(bucketName).file(this.makeGlobalBlobKey(hash))
await file.download({ destination })
}
@@ -203,9 +216,18 @@ class RecoveryBlobStore {
async function uploadZip(historyId, zipPathname) {
const bucketName = config.get('zipStore.bucket')
const deadline = 24 * 3600 * 1000 // lifecycle limit on the zips bucket
const storage = new Storage()
const storage = createStorage()
const destination = `${historyId}-recovered.zip`
await storage.bucket(bucketName).upload(zipPathname, { destination })
await storage.bucket(bucketName).upload(zipPathname, {
destination,
resumable: false,
})
if (config.has('persistor.gcs.endpoint.apiEndpoint')) {
// In emulator mode, signed URLs aren't available
const apiEndpoint = config.get('persistor.gcs.endpoint.apiEndpoint')
return `${apiEndpoint}/storage/v1/b/${bucketName}/o/${encodeURIComponent(destination)}?alt=media`
}
const signedUrls = await storage
.bucket(bucketName)
@@ -219,6 +241,23 @@ async function uploadZip(historyId, zipPathname) {
return signedUrls[0]
}
/**
* Promisified wrapper for ZipStream's entry method.
*
* @param {ZipStream} archive
* @param {Buffer|NodeJS.ReadableStream|string} source
* @param {{ name: string }} data
* @return {Promise<void>}
*/
function addEntry(archive, source, data) {
return new Promise((resolve, reject) => {
archive.entry(source, data, err => {
if (err) reject(err)
else resolve()
})
})
}
async function restoreProject(historyId) {
const tmp = await fs.promises.mkdtemp(
path.join(os.tmpdir(), historyId.toString())
@@ -237,9 +276,40 @@ async function restoreProject(historyId) {
if (VERBOSE) console.log('zipping', historyId)
const zipPathname = path.join(tmp, `${historyId}.zip`)
const zipTimeoutMs = 60 * 1000
const archive = new ProjectArchive(snapshot, zipTimeoutMs)
await archive.writeZip(blobStore, zipPathname)
const outputFile = fs.createWriteStream(zipPathname)
const archive = new ZipStream()
const pipelinePromise = pipeline(archive, outputFile)
for (const pathname of snapshot.getFilePathnames()) {
const file = snapshot.getFile(pathname)
if (!file) continue
await file.load('eager', blobStore)
let content = file.getContent({
filterTrackedDeletes: true,
})
if (content === null) {
const hash = file.getHash()
content = await blobStore.getStream(hash)
}
if (content == null) continue
if (typeof content === 'string') {
content = Buffer.from(content)
}
await addEntry(archive, content, { name: pathname })
if (VERBOSE) console.log(`${pathname} added`)
}
archive.finalize()
await pipelinePromise
if (VERBOSE) {
console.log(`Wrote ${archive.getBytesWritten()} bytes`)
}
if (VERBOSE) console.log('uploading', historyId)
@@ -252,4 +322,7 @@ async function main() {
console.log(signedUrl)
}
}
main().catch(console.error)
main().catch(err => {
console.error(err)
process.exit(1)
})
@@ -4,6 +4,7 @@ import commandLineArgs from 'command-line-args'
import assert from '../lib/assert.js'
import fs from 'node:fs'
import { setTimeout } from 'node:timers/promises'
import { pipeline } from 'node:stream/promises'
import {
archiveLatestChunk,
archiveRawProject,
@@ -11,17 +12,13 @@ import {
} from '../lib/backupArchiver.mjs'
import knex from '../lib/knex.js'
import { client } from '../lib/mongodb.js'
import archiver from 'archiver'
import Events from 'node:events'
import ZipStream from 'zip-stream'
import { Chunk } from 'overleaf-editor-core'
import _ from 'lodash'
// Silence warning.
Events.setMaxListeners(20)
const SUPPORTED_MODES = ['raw', 'latest']
// Pads the mode name to a fixed length for better alignment in output.
// Pads the mode name to a fixed length for alignment.
const padModeName = _.partialRight(
_.padEnd,
Math.max(...SUPPORTED_MODES.map(mode => mode.length))
@@ -65,25 +62,6 @@ function usage() {
})
}
/**
* @typedef {import('archiver').ZipArchive} ZipArchive
*/
/**
* @typedef {import('archiver').ProgressData} ProgressData
*/
/**
* @typedef {import('archiver').EntryData} EntryData
*/
/**
* @typedef {Object} ArchiverError
* @property {string} message
* @property {string} code
* @property {Object} data
*/
let historyId, help, mode, output, useBackupGlobalBlobs, verbose
try {
@@ -136,80 +114,37 @@ await loadGlobalBlobs()
outputFile = fs.createWriteStream(output)
const archive = archiver.create('zip', {})
const archive = new ZipStream()
archive.on('close', function () {
console.log(archive.pointer() + ' total bytes')
console.log(`Wrote ${output}`)
shutdown().catch(e => console.error('Error shutting down', e))
archive.on('error', function (e) {
console.error(`Error writing archive: ${e.message}`)
})
archive.on(
'error',
/**
*
* @param {ArchiverError} e
*/
function (e) {
console.error(`Error writing archive: ${e.message}`)
}
)
archive.on('end', function () {
console.log(`Wrote ${archive.pointer()} total bytes to ${output}`)
shutdown().catch(e => console.error('Error shutting down', e))
})
archive.on(
'progress',
/**
*
* @param {ProgressData} progress
*/
function (progress) {
if (verbose) {
console.log(
`${progress.entries.processed} processed out of ${progress.entries.total}`
)
}
}
)
archive.on(
'entry',
/**
*
* @param {EntryData} entry
*/
function (entry) {
if (verbose) {
console.log(`${entry.name} added`)
}
}
)
archive.on(
'warning',
/**
*
* @param {ArchiverError} warning
*/
function (warning) {
console.warn(`Warning encountered when writing archive: ${warning.message}`)
}
)
try {
// Pipe archive to the output file before adding entries.
// pipeline handles backpressure and will resolve when
// the archive stream ends.
const pipelinePromise = pipeline(archive, outputFile)
switch (mode) {
case 'latest':
await archiveLatestChunk(archive, historyId, useBackupGlobalBlobs)
await archiveLatestChunk(
archive,
historyId,
useBackupGlobalBlobs,
verbose
)
break
case 'raw':
default:
await archiveRawProject(archive, historyId, useBackupGlobalBlobs)
await archiveRawProject(archive, historyId, useBackupGlobalBlobs, verbose)
break
}
archive.pipe(outputFile)
archive.finalize()
await pipelinePromise
console.log(`Wrote ${archive.getBytesWritten()} total bytes to ${output}`)
} catch (error) {
if (error instanceof BackupPersistorError) {
console.error(error.message)
@@ -222,12 +157,7 @@ try {
} else {
console.error('Error encountered when writing archive')
}
} finally {
await Promise.race([
await archive.finalize(),
setTimeout(10000).then(() => {
console.error('Archive did not finalize in time')
return shutdown(1)
}),
])
await shutdown(1)
}
await shutdown(0)
@@ -507,7 +507,7 @@ describe('project controller', function () {
})
})
// eslint-disable-next-line mocha/no-skipped-tests
// eslint-disable-next-line mocha/no-pending-tests
describe.skip('getLatestContent', function () {
// TODO: remove this endpoint entirely, see
// https://github.com/overleaf/write_latex/pull/5120#discussion_r244291862
@@ -647,6 +647,68 @@ describe('backup script', function () {
expect(newBackupStatus.backupStatus.lastBackedUpVersion).to.equal(50) // backup fails on final chunk
expect(newBackupStatus.currentEndVersion).to.equal(54) // backup is incomplete due to missing blob
})
it('can recover zip file from backup in raw mode', async function () {
// First, run backup so data is available
await runBackupScript(['--projectId', projectId])
const zipPath = `/tmp/test-recover-raw-${historyId}.zip`
try {
await runRecoverZipFromBackupScript([
'--historyId',
historyId,
'--output',
zipPath,
'--mode=raw',
])
// Verify the zip file is valid
const { stdout } = await promisify(execFile)('unzip', ['-l', zipPath], {
encoding: 'utf-8',
})
// Raw mode includes chunk and blob keys
// Verify chunks are present
expect(stdout).to.include('chunk.json')
// Verify blob hashes are present (hashes are stored as {hash[0:2]}/{hash[2:]})
expect(stdout).to.include(testFiles.GRAPH_PNG_HASH.slice(2))
expect(stdout).to.include(testFiles.NON_BMP_TXT_HASH.slice(2))
} finally {
await fs.promises.unlink(zipPath).catch(() => {})
}
})
it('can recover zip file from backup in latest mode', async function () {
// First, run backup so data is available
await runBackupScript(['--projectId', projectId])
const zipPath = `/tmp/test-recover-latest-${historyId}.zip`
try {
await runRecoverZipFromBackupScript([
'--historyId',
historyId,
'--output',
zipPath,
'--mode=latest',
])
// Verify the zip file is valid
const { stdout } = await promisify(execFile)('unzip', ['-l', zipPath], {
encoding: 'utf-8',
})
// Latest mode includes the project files
expect(stdout).to.include('main.tex')
expect(stdout).to.include('chapter1.tex')
expect(stdout).to.include('chapter2.tex')
expect(stdout).to.include('bibliography.bib')
expect(stdout).to.include('graph.png')
expect(stdout).to.include('unicodeFile.tex')
} finally {
await fs.promises.unlink(zipPath).catch(() => {})
}
})
})
})
@@ -683,3 +745,37 @@ async function runBackupScript(args) {
}
return result
}
/**
* Run the recover_zip_from_backup script with given arguments
* @param {string[]} args
*/
async function runRecoverZipFromBackupScript(args) {
const TIMEOUT = 30 * 1000
let result
try {
result = await promisify(execFile)(
'node',
['storage/scripts/recover_zip_from_backup.mjs', ...args],
{
encoding: 'utf-8',
timeout: TIMEOUT,
env: {
...process.env,
LOG_LEVEL: 'debug',
},
}
)
result.status = 0
} catch (err) {
const { stdout, stderr, code } = err
if (typeof code !== 'number') {
console.log(err)
}
result = { stdout, stderr, status: code }
}
if (result.status !== 0) {
throw new Error(`recover_zip_from_backup failed: ${result.stderr}`)
}
return result
}
@@ -0,0 +1,205 @@
'use strict'
const { promisify } = require('node:util')
const { execFile } = require('node:child_process')
const { expect } = require('chai')
const {
Change,
AddFileOperation,
EditFileOperation,
TextOperation,
File,
} = require('overleaf-editor-core')
const cleanup = require('./support/cleanup')
const fixtures = require('./support/fixtures')
const { setupProjectState } = require('./support/redis')
const chunkStore = require('../../../../storage/lib/chunk_store')
const persistChanges = require('../../../../storage/lib/persist_changes')
const knex = require('../../../../storage/lib/knex')
const SCRIPT_PATH = 'storage/scripts/finalise_chunk.mjs'
async function runScript(args) {
try {
const result = await promisify(execFile)('node', [SCRIPT_PATH, ...args], {
encoding: 'utf-8',
timeout: 10000,
env: { ...process.env, LOG_LEVEL: 'debug' },
})
return { ...result, status: 0 }
} catch (err) {
if (typeof err.code !== 'number') {
throw err
}
return { stdout: err.stdout, stderr: err.stderr, status: err.code }
}
}
async function getChunkRows(projectId) {
return await knex('chunks')
.select('id', 'start_version', 'end_version', 'closed')
.where('doc_id', parseInt(projectId, 10))
.orderBy('end_version')
}
describe('finalise_chunk script', function () {
before(cleanup.everything)
let limitsToPersistImmediately
before(function () {
const farFuture = new Date()
farFuture.setTime(farFuture.getTime() + 7 * 24 * 3600 * 1000)
limitsToPersistImmediately = {
minChangeTimestamp: farFuture,
maxChangeTimestamp: farFuture,
maxChunkChanges: 100,
}
})
beforeEach(async function () {
await cleanup.everything()
await fixtures.create()
})
describe('with a populated current chunk', function () {
let projectId
let initialContent
let initialEndVersion
let initialChunkId
beforeEach(async function () {
projectId = await chunkStore.initializeProject()
initialContent = 'Hello world.'
const change1 = new Change(
[new AddFileOperation('main.tex', File.fromString(initialContent))],
new Date(Date.now() - 30000),
[]
)
const change2 = new Change(
[
new EditFileOperation(
'main.tex',
new TextOperation().retain(initialContent.length).insert(' More.')
),
],
new Date(Date.now() - 20000),
[]
)
await persistChanges(
projectId,
[change1, change2],
limitsToPersistImmediately,
0
)
const metadata = await chunkStore.getLatestChunkMetadata(projectId)
initialEndVersion = metadata.endVersion
initialChunkId = metadata.id
expect(initialEndVersion).to.equal(2)
})
it('closes the current chunk and creates a new empty chunk', async function () {
const result = await runScript(['--historyId', projectId])
expect(result.status, result.stderr).to.equal(0)
const rows = await getChunkRows(projectId)
expect(rows).to.have.length(2)
const oldRow = rows.find(r => r.id.toString() === initialChunkId)
expect(oldRow, 'old chunk still in DB').to.exist
expect(oldRow.closed).to.equal(true)
expect(oldRow.end_version).to.equal(initialEndVersion)
const newRow = rows.find(r => r.id.toString() !== initialChunkId)
expect(newRow, 'new chunk inserted').to.exist
expect(newRow.start_version).to.equal(initialEndVersion)
expect(newRow.end_version).to.equal(initialEndVersion + 1)
expect(newRow.closed).to.equal(false)
const newChunk = await chunkStore.loadLatest(projectId, {
persistedOnly: true,
})
expect(newChunk.getStartVersion()).to.equal(initialEndVersion)
expect(newChunk.getEndVersion()).to.equal(initialEndVersion + 1)
// The new chunk holds a single NoOperation change as a placeholder so
// its end_version is distinct from the closed chunk's end_version.
expect(newChunk.getChanges()).to.have.length(1)
const newSnapshot = newChunk.getSnapshot()
const file = newSnapshot.getFile('main.tex')
expect(file, 'main.tex carried over to new chunk snapshot').to.exist
})
it('does not modify state on a dry-run', async function () {
const result = await runScript(['--historyId', projectId, '--dry-run'])
expect(result.status, result.stderr).to.equal(0)
const rows = await getChunkRows(projectId)
expect(rows).to.have.length(1)
expect(rows[0].id.toString()).to.equal(initialChunkId)
expect(rows[0].closed).to.equal(false)
})
})
describe('safety checks', function () {
it('refuses to act on an already-empty current chunk', async function () {
const projectId = await chunkStore.initializeProject()
const result = await runScript(['--historyId', projectId])
expect(result.status).to.not.equal(0)
expect(result.stdout + result.stderr).to.match(/already empty/)
const rows = await getChunkRows(projectId)
expect(rows).to.have.length(1)
expect(rows[0].start_version).to.equal(0)
expect(rows[0].end_version).to.equal(0)
expect(rows[0].closed).to.equal(false)
})
it('refuses when there are non-persisted changes in redis', async function () {
const projectId = await chunkStore.initializeProject()
const initialContent = 'Initial.'
const initialChange = new Change(
[new AddFileOperation('main.tex', File.fromString(initialContent))],
new Date(Date.now() - 30000),
[]
)
await persistChanges(
projectId,
[initialChange],
limitsToPersistImmediately,
0
)
const metadata = await chunkStore.getLatestChunkMetadata(projectId)
const initialChunkId = metadata.id
const bufferedChange = new Change(
[
new EditFileOperation(
'main.tex',
new TextOperation()
.retain(initialContent.length)
.insert(' Buffered.')
),
],
new Date(Date.now() - 10000),
[]
)
await setupProjectState(projectId, {
persistTime: Date.now() - 1000,
headVersion: 2,
persistedVersion: 1,
changes: [bufferedChange],
expireTimeFuture: true,
})
const result = await runScript(['--historyId', projectId])
expect(result.status).to.not.equal(0)
expect(result.stdout + result.stderr).to.match(/non-persisted/)
const rows = await getChunkRows(projectId)
expect(rows).to.have.length(1)
expect(rows[0].id.toString()).to.equal(initialChunkId)
expect(rows[0].closed).to.equal(false)
})
})
})
@@ -0,0 +1,169 @@
import { expect } from 'chai'
import config from 'config'
import { execFile } from 'node:child_process'
import fs from 'node:fs'
import { promisify } from 'node:util'
import { Change, Operation, File, TextOperation } from 'overleaf-editor-core'
// We depend on this via object-persistor.
// eslint-disable-next-line import/no-extraneous-dependencies
import { Storage } from '@google-cloud/storage'
import {
loadGlobalBlobs,
BlobStore,
} from '../../../../storage/lib/blob_store/index.js'
import ChunkStore from '../../../../storage/lib/chunk_store/index.js'
import persistChanges from '../../../../storage/lib/persist_changes.js'
import testFiles from '../storage/support/test_files.js'
import cleanup from './support/cleanup.js'
import { getZipEntries } from './support/unzip.js'
describe('recover_zip script', function () {
let projectId
let limitsToPersistImmediately
before(async function () {
const farFuture = new Date()
farFuture.setTime(farFuture.getTime() + 7 * 24 * 3600 * 1000)
limitsToPersistImmediately = {
minChangeTimestamp: farFuture,
maxChangeTimestamp: farFuture,
maxChanges: 10,
maxChunkChanges: 10,
}
const gcsEndpoint = config.get('persistor.gcs.endpoint')
const storage = new Storage({
apiEndpoint: gcsEndpoint.apiEndpoint,
projectId: gcsEndpoint.projectId,
})
const bucketName = config.get('zipStore.bucket')
try {
const [exists] = await storage.bucket(bucketName).exists()
if (!exists) {
await storage.createBucket(bucketName)
}
} catch (err) {
if (err.code !== 409) throw err
}
})
beforeEach(cleanup.everything)
beforeEach(async function () {
await loadGlobalBlobs()
projectId = '123'
// Initialize the project in the chunk store
await ChunkStore.initializeProject(projectId)
const blobStore = new BlobStore(projectId)
// Upload binary file blob
await blobStore.putFile(testFiles.path('graph.png'))
// Create initial snapshot with text and binary files
const addMainTex = Operation.addFile(
'main.tex',
File.fromString('hello world')
)
const addGraphPng = Operation.addFile(
'graph.png',
File.fromHash(testFiles.GRAPH_PNG_HASH)
)
const change1 = new Change([addMainTex, addGraphPng], new Date(), [])
await persistChanges(projectId, [change1], limitsToPersistImmediately, 0)
// Add a text edit
const textOp = TextOperation.fromJSON({
textOperation: ['hello world'.length, ' more'],
})
const editOp = Operation.editFile('main.tex', textOp)
const change2 = new Change([editOp], new Date(), [])
await persistChanges(projectId, [change2], limitsToPersistImmediately, 1)
})
it('creates a valid zip from GCS data', async function () {
this.timeout(30 * 1000)
const zipPath = `/tmp/test-recover-zip-${projectId}.zip`
try {
const { stdout } = await runRecoverZipScript([projectId])
// The script logs the signed URL to stdout
const urlMatch = stdout.match(/(https?:\/\/[^\s]+)/)
expect(urlMatch).to.not.be.null
const signedUrl = urlMatch[1]
// Download the zip via fetch
const res = await fetch(signedUrl)
expect(res.ok).to.be.true
const buffer = await res.arrayBuffer()
await fs.promises.writeFile(zipPath, Buffer.from(buffer))
const zipEntries = await getZipEntries(zipPath)
const fileNames = zipEntries.map(e => e.fileName).sort()
expect(fileNames).to.deep.equal(['graph.png', 'main.tex'])
// Verify text content size (after edit)
const mainTexEntry = zipEntries.find(e => e.fileName === 'main.tex')
expect(mainTexEntry.uncompressedSize).to.equal('hello world more'.length)
// Verify binary content size
const graphEntry = zipEntries.find(e => e.fileName === 'graph.png')
expect(graphEntry.uncompressedSize).to.equal(
testFiles.GRAPH_PNG_BYTE_LENGTH
)
} finally {
await fs.promises.unlink(zipPath).catch(() => {})
}
})
it('supports the --verbose flag', async function () {
this.timeout(30 * 1000)
const { stdout } = await runRecoverZipScript(['--verbose', projectId])
// Verbose mode logs each file as it's added
expect(stdout).to.include('main.tex added')
expect(stdout).to.include('graph.png added')
})
})
/**
* Run the recover_zip.js script with given arguments
* @param {string[]} args
*/
async function runRecoverZipScript(args) {
const TIMEOUT = 30 * 1000
let result
try {
result = await promisify(execFile)(
'node',
['storage/scripts/recover_zip.js', ...args],
{
encoding: 'utf-8',
timeout: TIMEOUT,
env: {
...process.env,
LOG_LEVEL: 'debug',
},
}
)
result.status = 0
} catch (err) {
const { stdout, stderr, code } = err
if (typeof code !== 'number') {
console.log(err)
}
result = { stdout, stderr, status: code }
}
if (result.status !== 0 || result.stderr) {
throw new Error(
`recover_zip failed (exit ${result.status}):\n` +
`stdout: ${result.stdout}\n` +
`stderr: ${result.stderr}`
)
}
return result
}
+6 -2
View File
@@ -55,6 +55,10 @@ COMPOSE_PROJECT_NAME_TEST_UNIT ?= test_unit_$(BUILD_DIR_NAME)
DOCKER_COMPOSE_TEST_UNIT = \
COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME_TEST_UNIT) $(DOCKER_COMPOSE)
.PHONY: print-branch-tag-safe
print-branch-tag-safe:
@echo $(BRANCH_NAME_TAG_SAFE)
clean:
-docker rmi $(IMAGE_CI)
-docker rmi $(IMAGE_REPO_FINAL)
@@ -67,8 +71,8 @@ clean:
RUN_LINTING = ../../bin/run -w /overleaf/services/$(PROJECT_NAME) monorepo yarn run --silent
RUN_LINTING_MONOREPO = ../../bin/run monorepo yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/.eslintignore:/overleaf/.eslintignore --volume $(MONOREPO)/.eslintrc:/overleaf/.eslintrc --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc:/overleaf/.prettierrc --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache $(IMAGE_CI) yarn run --silent
RUN_LINTING_CI_MONOREPO = docker run --rm --volume $(MONOREPO)/.editorconfig:/overleaf/.editorconfig --volume $(MONOREPO)/eslint.config.mjs:/overleaf/eslint.config.mjs --volume $(MONOREPO)/.prettierignore:/overleaf/.prettierignore --volume $(MONOREPO)/.prettierrc.cjs:/overleaf/.prettierrc.cjs --volume $(MONOREPO)/tsconfig.backend.json:/overleaf/tsconfig.backend.json --volume $(MONOREPO)/services/notifications/reports:/overleaf/services/notifications/reports --volume $(MONOREPO)/node_modules/.cache:/overleaf/node_modules/.cache -w /overleaf $(IMAGE_CI) yarn run --silent
SHELLCHECK_OPTS = \
--shell=bash \
-2
View File
@@ -9,7 +9,6 @@ import express, {
type ErrorRequestHandler,
type NextFunction,
} from 'express'
import methodOverride from 'method-override'
import { mongoClient } from './app/js/mongodb.js'
import NotificationsController from './app/js/NotificationsController.ts'
import HealthCheckController from './app/js/HealthCheckController.ts'
@@ -25,7 +24,6 @@ logger.initialize('notifications')
metrics.memory.monitor(logger)
metrics.open_sockets.monitor()
app.use(methodOverride())
app.use(express.json())
app.use(metrics.http.monitor(logger))
+1
View File
@@ -1,5 +1,6 @@
notifications
--dependencies=mongo
--deploy-pipeline=notifications
--env-add=
--env-pass-through=
--esmock-loader=False
-2
View File
@@ -30,14 +30,12 @@
"body-parser": "1.20.4",
"bunyan": "^1.8.15",
"express": "4.22.1",
"method-override": "^3.0.0",
"mongodb-legacy": "6.1.3",
"zod": "^4.1.7",
"zod-validation-error": "^4.0.1"
},
"devDependencies": {
"@overleaf/migrations": "workspace:*",
"@types/method-override": "^3.0.0",
"chai": "^4.3.6",
"chai-as-promised": "^7.1.1",
"mocha": "^11.1.0",
-1
View File
@@ -1 +0,0 @@
app/lib/*.js

Some files were not shown because too many files have changed in this diff Show More