Commit Graph
118 Commits
Author SHA1 Message Date
Eric Mc Sween 20c7f14b3c Merge pull request #24461 from overleaf/em-non-editable-file-in-history
Graceful fallback for getRangesSnapshot()

GitOrigin-RevId: 2f0e70dbcdd38de87d0b1c7590f7d8a5a7112937
2025-03-26 09:05:21 +00:00
Eric Mc Sween 374acf8119 Merge pull request #24462 from overleaf/em-resync-error
Fix error reporting on resyncs in project-history

GitOrigin-RevId: 9e8a454f746c2b4db7c28fafcd52e50935af5957
2025-03-25 09:05:12 +00:00
Brian Gough 31b57e2991 Merge pull request #24367 from overleaf/bg-add-new-project-history-flush-script
add new project history flush script

GitOrigin-RevId: 4d6f3be1ada7191334b934e34e1c9eac59a816d0
2025-03-24 10:46:34 +00:00
Eric Mc Sween 35dc7faab6 Merge pull request #24224 from overleaf/em-resync-on-flush-failure
Immediately attempt a resync when a history flush fails

GitOrigin-RevId: 098a0a7edf55c0ed47c48e0a3c080e3562cdceaa
2025-03-18 09:05:36 +00:00
Eric Mc Sween 064baf31ad Merge pull request #23478 from overleaf/em-doc-hash-diff
Fix doc hash on delete + insert combination

GitOrigin-RevId: ecea5f8ab01f24a9064be88611256b502500d91f
2025-02-10 09:05:38 +00:00
Jakob Ackermann 6b8a50eb92 [project-history] trim logging (#23408)
* [project-history] trim logging when projectVersion is out of order

* [editor-core] record reason rather than full pathname when not clean

* [editor-core] remove pathname from error messages

* [project-history] avoid logging updates when historyId is inconsistent

* [project-history] avoid logging failed history-v1 requests repeatedly

* [project-history] log once when doc version is out of order in chunk

GitOrigin-RevId: 5ac431ac87b57c00fac9d5ab486ca305b1775372
2025-02-06 09:04:35 +00:00
Jakob Ackermann 81aab1e159 [misc] fix logger.error(err) and logger.warn(err) calls (#23369)
* [misc] fix logger.error(err) calls

The signature is "logger.error({ err }, 'MESSAGE')".

* [project-history] remove duplicate logger.err calls in health check

The call-site is already logging any errors. Also, the logger.err call
signature was not quite right.

* [web] log userId when removeDropbox/removeGithub hook fails

* [misc] fix logger.warn(err) calls

The signature is "logger.warn({ err }, 'MESSAGE')".

* [misc] fix logger.error(OError.tag(err)) calls

* [web] make eslint happy

GitOrigin-RevId: 7f528113a3f7e9f6293b7d2d45adc079380325bb
2025-02-06 09:04:31 +00:00
Eric Mc Sween 39842c9857 Merge pull request #23250 from overleaf/em-project-history-doc-hash
Propagate the doc hash from project-history to history

GitOrigin-RevId: 341df52df41f7a5f8dbebbce53a47f9d5e1f8175
2025-02-04 09:05:03 +00:00
3a4c5a0d0f [history-v1] add readOnly lookup for raw chunks (#23318)
* [history-v1] add readOnly lookup for raw chunks

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [history-v1] reduce min poolsize for readOnly pool to 0

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a711c4ee4f3ea3775bd090e620d1ef52689fa1f4
2025-02-04 09:04:52 +00:00
Jakob Ackermann c6c623da78 [project-history] script for fixing-up files/metadata with bulk resync (#23184)
* [history-v1] add cheap endpoint for checking time of last history write

The /raw endpoint skips the GCS lookup for the chunk.

* [project-history] script for fixing-up files/metadata with bulk resync

* [project-history] upgrade structure only resync when full sync is needed

* [project-history] start resync and process resync updates under lock

* [project-history] stop retrying during graceful shutdown

GitOrigin-RevId: 73184d5786e1d40f5b7e21f387fc37cf43f0ac2d
2025-02-03 09:05:43 +00:00
Jakob Ackermann b0c9176634 [project-history] add resync for project structure only (#23271)
* [project-history] add resync for project structure only

* [project-history] block resyncProjectStructureOnly that update docs

* [project-history] tweak test description

GitOrigin-RevId: ce2749566c36e04dab21c26dd60dd75d93c0d4c0
2025-02-03 09:05:39 +00:00
Jakob Ackermann ba60f885a4 [project-history] fetch most recent chunk once when processing updates (#23261)
GitOrigin-RevId: 08ddd1f953b0dbae0541a60b45fec0f88e0a1d06
2025-01-31 09:05:14 +00:00
Jakob Ackermann 72b4083318 [project-history] forward all the optional file attributes during resync (#23260)
Notably, 'createdBlob' was missing. And others like metadata should not
be added if not set.

GitOrigin-RevId: c4a6746c4824d637f7c54b4c68a7025b60c15ff2
2025-01-31 09:05:09 +00:00
Jakob Ackermann ae171dbced [project-history] fix name for error in logging statement (#23110)
Only the err/error fields will use the Error serializer and emit all the
OError details into the log.

GitOrigin-RevId: 4b0e16c00be5ea7c0aea747f49b46897ebd100cc
2025-01-27 09:05:22 +00:00
Eric Mc Sween 3e4e9b298f Merge pull request #23089 from overleaf/em-fix-ranges-snapshot
Fix edge cases when restoring comments

GitOrigin-RevId: 6ce2426bfb7233a3c0915bcc8c53bf0588702847
2025-01-27 09:05:06 +00:00
Jakob Ackermann 762266cd87 [project-history] block filestore reads from old queue entries (#23096)
GitOrigin-RevId: 9952cb66e542b17a6a3b5e3b2609d53dc8c371fd
2025-01-27 09:04:54 +00:00
Jakob Ackermann 8d99ad3964 [web] do not send filestore URLs when feature is disabled (#23095)
* [web] do not return createdBlob=true from error path

Defense in-depth, currently the only call-site bails out on error.

* [web] do not send filestore URLs when feature is disabled

GitOrigin-RevId: 7e90cf4c4babffeec337702502234bac73c1f116
2025-01-27 09:04:50 +00:00
Jakob Ackermann 3b602ed93a [project-history] log previous error when retrying createBlob operation (#23103)
GitOrigin-RevId: ff59d15e93c406e17775e5e4acac1f13da9a7788
2025-01-27 09:04:46 +00:00
Jakob Ackermann 36c9772e0e [project-history] log context of failed history-v1 requests (#23078)
GitOrigin-RevId: a597efc46e00fc07cc6e5b71938c82f513c0b288
2025-01-24 09:05:44 +00:00
Jakob Ackermann 2640d4a566 [project-history] fixes for checking blob exists (#23051)
* [project-history] skip HEAD on blob without file-hash

* [project-history] only 404s indicate a missing blob

* [project-history] error checking for existing blob is fatal

* [project-history] always skip creating blob if already exists

Ignore createdBlob detail after performing check on blob. It might not
 exist on old operations that have been queued.

* [project-history] fix unit tests

* [project-history] migrate test to new fetch call

GitOrigin-RevId: 457963042e9d836647d3c122516329d4dc9b544f
2025-01-24 09:05:09 +00:00
Eric Mc Sween 30e5c95a27 Merge pull request #22871 from overleaf/em-history-flush-metrics
History flush metrics

GitOrigin-RevId: eb8b357427942e9816ad92ccd46c0dd8a65ab939
2025-01-17 09:05:00 +00:00
Andrew Rumble e64e69f539 Merge pull request #22652 from overleaf/ar-handle-filestore-404-when-copying-in-saas
[web] Do not do filestore copy in SAAS

GitOrigin-RevId: 83456ca57f706246a86b997a038968aecfbae4c5
2024-12-24 09:04:48 +00:00
Andrew Rumble e6a2caa088 Merge pull request #22421 from overleaf/ar-pass-created-blob-for-filetree-uploads
[web] Pass createdBlob through for file creation everywhere

GitOrigin-RevId: 880bebd0a55e351a6a61555a46e80faef22db7cb
2024-12-12 09:05:02 +00:00
Andrew Rumble ad8026038c Merge pull request #22397 from overleaf/revert-22320-revert-22213-ar-avoid-duplicate-blob-writes
Revert "Revert "Avoid duplicate blob writes""

GitOrigin-RevId: f6a0083c0edf9ede14dfb52dbf1809a843aa5ff4
2024-12-10 09:05:07 +00:00
Andrew Rumble ffa28b1a58 Merge pull request #22320 from overleaf/revert-22213-ar-avoid-duplicate-blob-writes
Revert "Avoid duplicate blob writes"

GitOrigin-RevId: 9f86bcea654cd3fa5f66fbdf42080e7f6e2861a7
2024-12-05 09:04:48 +00:00
Andrew Rumble 40b781eb0a Merge pull request #22213 from overleaf/ar-avoid-duplicate-blob-writes
Avoid duplicate blob writes

GitOrigin-RevId: 5e7a472b235bae5ef84389dbbe791189e951908a
2024-12-04 09:05:08 +00:00
Jakob Ackermann ce0d5fd383 Merge pull request #22177 from overleaf/jpa-file-view-hash-1
[web] migrate file-view to download from history-v1 (via web) 1/2

GitOrigin-RevId: b787e90c57af5e2704b06ba63502aa6fc09ea1df
2024-11-28 09:06:33 +00:00
Brian GoughandAntoine Clausse 6672372828 remove sentry from backend services (#20752)
* remove sentry from backend services - no longer required

* Remove Sentry integration from logging manager

* Remove Sentry from clsi default settings

* Remove `initializeErrorReporting` in libraries/logger

* Remove `@sentry/node` from `libraries/logger`

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 8149a885f5258804b93ae39cde7b7333e992532a
2024-11-27 09:04:50 +00:00
Brian Gough fc84bdf68b Merge pull request #21751 from overleaf/bg-record-background-flush-operations
record background flush operations in project-history

GitOrigin-RevId: 80169b3f93a844d7162322ca21d605fffc64d687
2024-11-11 09:05:35 +00:00
Antoine Clausse 7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Miguel Serrano bafda84420 Merge pull request #21072 from overleaf/msm-cleanup-post-labels
[project-history] Cleanup POST /labels

GitOrigin-RevId: 129f83ab72e4a9cd6f0f5fd7935fafa813ead919
2024-10-16 09:35:48 +00:00
Jakob Ackermann fab69a16c4 Merge pull request #20979 from overleaf/jpa-handle-error
[project-history] gracefully handle errors when loading Changes

GitOrigin-RevId: 214fadb3c779551ea90f38b0fdf77c58ed5df178
2024-10-15 08:05:15 +00:00
Jakob Ackermann 663f6605d7 Merge pull request #20978 from overleaf/jpa-fix-unit-tests
[project-history] fix unit tests for processing updates

GitOrigin-RevId: ce87296b9b97dcfd7598fbf25b3066b7c145fb6f
2024-10-15 08:05:10 +00:00
Domagoj Kriskovic 0998cc04df [project-history] fix diff view on file revert (#20769)
GitOrigin-RevId: 6c95dab9328ecab636272ad507505f5342ec8709
2024-10-14 11:06:36 +00:00
Miguel Serrano cd6631c105 Fix history label creation for anonymous users (#20200)
* Remove decaffeination artifacts in LabelsTests

* Remove decaffeination artifacts in LabelsManagerTests

* Fix label creation for anonymous users

* Update label creation route in MockProjectHistoryApi tests

* Support both endpoints for backwards compatibility

GitOrigin-RevId: 50ce1ba49388e50f147fb620e0425fea83301c9d
2024-10-14 10:57:33 +00:00
Jakob Ackermann 3ba3ffe56d Merge pull request #20277 from overleaf/jpa-ro-mirror-chunk
[misc] use Chunk for read-only mirror in frontend

GitOrigin-RevId: b97e520c7a63015179792542310d18f346752e92
2024-09-24 08:05:42 +00:00
Eric Mc Sween 250acbf33c Merge pull request #20520 from overleaf/em-ts-import
Use new import JSDoc syntax for Typescript annotations

GitOrigin-RevId: 782456d637fc8f2de6163b5d70fabf06c1d74964
2024-09-23 08:04:47 +00:00
Mathias Jakobsen 8245a95b4e Merge pull request #20266 from overleaf/mj-revert-linked-file
[project-history+web] Restore metadata when reverting file

GitOrigin-RevId: dbfa8202a2fe0bf077d8eedb51a2a13b9f1e8a83
2024-09-09 08:04:53 +00:00
Jakob Ackermann 989c48978a Merge pull request #20067 from overleaf/jpa-fix-old-blob-download
[web] project-history expects history-v1 id in blob requests

GitOrigin-RevId: d9170a12fc6070811c188b346dbac32accabbfd7
2024-08-22 14:01:12 +00:00
Jakob Ackermann 577497b655 Merge pull request #19842 from overleaf/jpa-ro-mirror-on-client
[misc] add readonly mirror of full project content on the client

GitOrigin-RevId: 940bd93bfd587f83ca383d10fc44579b38fc3e88
2024-08-22 08:05:13 +00:00
Mathias Jakobsen 654d96ace6 Merge pull request #19743 from overleaf/mj-restore-project
[web+project-history] Add project version reverting

GitOrigin-RevId: 0f77cec730393187d531c0c6561faaa652bebf29
2024-08-21 08:05:08 +00:00
Eric Mc Sween e10478f256 Merge pull request #19867 from overleaf/em-hide-history-resync-details
Hide history resync changes

GitOrigin-RevId: a36876ff637d0a4c34150b21f4b4e623dff0ab03
2024-08-14 08:04:54 +00:00
Jakob Ackermann 86f26043d2 Merge pull request #19847 from overleaf/jpa-rename-history-v1
[misc] rename docker compose service history_v1 to history-v1 in dev-env

GitOrigin-RevId: 0991ee4297bae1ed9e9068af1889966ceeeba66d
2024-08-09 08:04:29 +00:00
Jakob Ackermann 074de0cc02 Merge pull request #19790 from overleaf/jpa-silence-missing
[project-history] silence warning due to missing web hash

GitOrigin-RevId: 04ff878c1b356333ff9d515af6a5cba8b31a1ff2
2024-08-06 08:04:54 +00:00
Jakob Ackermann d21874b076 Merge pull request #19741 from overleaf/jpa-check-filestore-write
[misc] verify file hash when downloading binary file in project-history

GitOrigin-RevId: 0ef56a0753cdfd55fdea921b3555dea48036766b
2024-08-06 08:04:49 +00:00
Jakob Ackermann e26b6de51b Merge pull request #19740 from overleaf/jpa-linked-file-metadata
[misc] persist linkedFileData in full project history as file metadata

GitOrigin-RevId: f3e8ba947ea34b6796e210a076a248c57188d148
2024-08-06 08:04:44 +00:00
Domagoj Kriskovic 6afb067737 Hide history-resync updates from "All history" (#19435)
* [web] hide history-resync updates from "All history"

* Revert "[web] hide history-resync updates from "All history""

This reverts commit e9d5e7638eabe2baccf36f8c80bb4cd619e383ea.

* filter history resync update in summarizeUpdates

* used isHistoryResyncUpdate for merging

* remove merging for history-resync

* Revert "remove merging for history-resync"

This reverts commit 6ce48bc3f906867a64c0acc12e2dc61c05436c41.

GitOrigin-RevId: 0335eb9c05815fb66188d453a90dd09531504a55
2024-07-19 08:04:21 +00:00
Eric Mc Sween 58ffefc8bb Merge pull request #19298 from overleaf/em-fix-restore-empty-file
Fix file restore when the file is empty

GitOrigin-RevId: be08305c9a41265acfb12046aeb2d003cda92b54
2024-07-15 09:00:51 +00:00
Eric Mc Sween e4cc2a0816 Fix history diffs when deleting over many tracked deletes (#19193)
* Fix history diffs when deleting over many tracked deletes

As we are looping through tracked deletes, the offset between the result
positions and the source positions must be kept constant. Otherwise, the
tracked deletes are translated as we delete text and move the source
cursor.

GitOrigin-RevId: b2417a75219aaa16bf5c61e0ebcb0586cae6aef2
2024-07-01 08:04:25 +00:00
Eric Mc Sween 67dbf743e1 Merge pull request #19192 from overleaf/em-project-history-avoid-crash
Prevent project-history crashes when generating a diff

GitOrigin-RevId: 513fb846f0884a68d4550783497ae436e1cba00f
2024-06-28 08:05:05 +00:00