* [monorepo] record ERROR/FATAL log messages in junit report
* [web] put SaaS specific code behind feature flag
* [web] use split test cache for getting user assignments
The unit tests needed updating as they did not replicate any of the
mongo filtering. The acceptance tests cover this logic.
* [web] make better use of existing indexes
* [web] avoid col-scan in tests of notifications module
* [web] remove cleanup of empty feedbacks collection
* [web] add assertion for reason of rejected request in launchpad test
* [web] add missing indexes
* [web] enable mongo notablescan
* [web] make emailNotifications tests compatible with notablescan
GitOrigin-RevId: b888f2feeb3a0e915f068ae1c4ea23ec17821221
* [migrations] make old migrations compatible with --notablescan in mongo
Re-running the migration for db.users.emails.reversedHostname on all
Server Pro/CE users will not be harmfull.
* [monorepo] enable mongo notablescan for all services but web
GitOrigin-RevId: e2a710c9de08de9ac2aa9df1f3082d5f1c33ea05
* fix: correct typedef for Document in helpers.mjs
* add move-migrations codemod
* update migration paths to use shared migrations directory
* move migrations to shared location
* fix: update Dockerfile and docker-compose.ci.yml to include migrations directory
* feat: add migrations tool to workspaces in package.json
* [monorepo] Fix order of docker ignore rules
* [web] remove unused docker ignore file
* [monorepo] replace old references to migrations folder
* [server-ce] copy migrations from new place
* [migrations] Inline web scripts
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] move three web scripts over
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] add missing collection
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] remove lodash dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] avoid mongodb-legacy dependency
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [monorepo] run migrations from tools/migrations
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [migrations] simplify migration for adding gitBridge feature to users
* [monorepo] run migrations from tests in all the services
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] fixup running web migrations everywhere
* [monorepo] trigger Jenkins builds on changes to mongo migrations
* [migrations] add Jenkins pipeline for linting/formatting
* [monorepo] build scripts: update devDependencies before deps scanning
* [monorepo] build scripts: formerly depend on tools/migrations
* [monorepo] run eslint on .mjs files
* [migrations] enable more eslint rules and fix all the errors
* [rake] fix migrations:list task
---------
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
* Use snapshot timestamp, remove request for paths at version
* Add timestamp to core Snapshot object
* Avoid mutating function argument
* Explain assumption about editable files
* snapshot.toRaw() in getContentAtVersion
* fix project-history acceptance test
* fix history_v1 test
* fix web tests
* Include the snapshot timestamp in stored chunks
---------
Co-authored-by: Eric Mc Sween <5454374+emcsween@users.noreply.github.com>
GitOrigin-RevId: 376a53e1f927cb56544e6782b47d80345655038c
* Upgrade `aws-sdk` to v3
Also vendored nodemailer-ses-transport
* Moved default region to CE settings
* Ensure timeout is added to `requestHandler` and `region` is populated
* fix unsigned header in signed URL request
The x-amz-acl header is not needed when using the signed request,
as ACL are already defined when creating the signed URL in
PutObjectCommand constructor.
* Add default AWS region for latexqc service
* remove unnecessary region in compose files
* Use AWS_REGION=us-west-2 for history-v1
* prevent retries uploading streams with PutObjectCommand
* Remove AWS SDK JS message suppression
GitOrigin-RevId: 6fda6f02160023ffed76143397bbd965f86a9509
* [history-v1] add endpoint for getting blob stats of projects
* [web] display project history blob size on admin pages
* [web] break down history storage size by text and binary blob
GitOrigin-RevId: bfa4d56cf2c503d03005c13a0f7ac38158156cd2
* [history-v1] add migrations record after full binary file migration
* [server-pro] add hotfix 5.5.5
* [server-ce] test: build hotfix 5.5.5 and use it in tests
GitOrigin-RevId: fb84e5710c59f466a3305de5f32f78e0ac9ce15d
* [history-v1] use String.padStart instead of lodash.padStart
* [web] download binary files in clsi from filestore via new endpoints
* [server-ce] tests: Cypress.env() is parsing boolean values
* [server-ce] tests: run history migration as root
GitOrigin-RevId: bdf6c0e542531ccc4b3f13d2ed68ca0d31e580e9
* [history-v1] put logs of back_fill_file_hash script in /var/log/overleaf
* [history-v1] do not leave tmp dir behind when using --report
* [history-v1] extend --report with number of files to check/missing
* [history-v1] move all the options into command line flags
Change default values:
- concurrency to 10
- concurrent batches to 1
- stream high water mark to 1MiB
* [web] find_malformed_filetrees: log progress on stderr
* [history-v1] fix percentage of projects that need back-filling
GitOrigin-RevId: ebd7889840f97b24eda43774ffe7524998fb634b
* [history-v1] move MockFilestore into shared place
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* [history-v1] make back_fill_file_hash_fix_up compatible with Server Pro
---------
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: 70ea57e1503031d9f14dcd60c4c110e746450587
add support for fetching files via http from filestore in back_fill_file_hash script and tests
GitOrigin-RevId: 8dea6383ed6fe9ee6786a5695e2deee93b1cdd84
* [history-v1] use POST requests for expiring redis buffer from cron
(cherry picked from commit 15780ac54e36b96e1aed9fd9eb6dfe9d4fbf842f)
* [history-v1] remove double claim of expire job
GitOrigin-RevId: 8b2eab07006a5819a47eed3f646b2a4d75f86e5b
