alois/Verso
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
17,975 Commits 8 Branches 11 Tags
msm-github-codespaces
Commit Graph
194 Commits
Author SHA1 Message Date
Jakob Ackermann 2b9d0868c5 [misc] test/acceptance: retrieve ingress metrics just before using it 
The upload request can bump the ingress metric.
The content hash validation might require a full download
in case the ETag field of the upload response is not a md5 sum.
 2020-02-28 12:27:06 +01:00
Jakob Ackermann 516102e6fb [misc] test/acceptance: do not hard code fake credentials 2020-02-28 12:26:46 +01:00
Jakob Ackermann 847f124d7b [misc] test/acceptance: skip the shutdown in case we did not start yet 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-02-28 12:26:24 +01:00
Jakob Ackermann 6589aa6ae0 [misc] test/acceptance: harden the startup check for s3 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-02-28 12:26:24 +01:00
Jakob Ackermann 94b8a7f891 [misc] test/unit: KeybuilderTests: use a unique suite label 2020-02-28 12:26:23 +01:00
Jakob Ackermann 54e3b57772 [misc] test/unit: add missing globals that are lazy loaded 2020-02-28 12:26:23 +01:00
Jakob Ackermann 7920570dd8 [misc] test/unit: add missing require stubs for metrics and settings 2020-02-28 12:26:22 +01:00
Simon Detheridge 8acc9ef0ae Make S3 partSize a setting 2020-02-17 14:06:15 +00:00
Simon Detheridge 9e7eec7a19 Add acceptance test for large files 2020-02-17 14:05:56 +00:00
Simon Detheridge 2f2a819b74 Re-fetch file to calculate md5 if etag is not in correct format 2020-02-17 14:05:43 +00:00
Simon Detheridge 5adfb3e2c0 Use large upload chunks 2020-02-17 14:05:31 +00:00
Simon Detheridge 4e507197fc Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor" 
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
 2020-02-17 14:04:42 +00:00
Simon DetheridgeandGitHub df2ddbe0e1 Revert "Add Migration Persistor, to send missing file requests to a fallback persistor" 2020-02-14 14:26:33 +00:00
Simon Detheridge 93cd55fb79 Refactor persistors to use a helper for common things 2020-02-12 13:27:12 +00:00
Simon Detheridge 304fdfd35c Explicitly resume stream after adding listener 2020-02-12 13:21:53 +00:00
Simon Detheridge 6dcf351377 Remove unnecessary 'async' 2020-02-12 13:21:53 +00:00
Simon Detheridge f4a16cd972 Update tests to properly check for copied files 2020-02-12 13:21:53 +00:00
Simon Detheridge 9e0b378948 Remove minipass as dependency and refactor to make things clearer 2020-02-12 13:21:52 +00:00
Simon Detheridge b4b7fd226e Add mechanisms to transfer files with md5-based integrity checks 
Fix error in settings and tidy up tests

Remove unused variable declaration

Remove .only from tests and update eslint rules to catch it in future

Use  to catch errors more safely getting md5 hash

Avoid unnecessary call to S3 to get md5 response
 2020-02-12 13:21:52 +00:00
Simon Detheridge 2625e03a31 Add MigrationPersistor for sending 404 requests to a fallback persistor 2020-02-12 13:19:48 +00:00
Simon Detheridge f877f51775 Rename *PersistorManager to *Persistor 2020-02-12 13:19:48 +00:00
Simon Detheridge 28fb998719 Refactor request logger into a class 2020-01-14 12:02:39 +00:00
Simon Detheridge 9615a06e0f Improve logging middleware to add info methods to request object 2020-01-10 17:22:06 +00:00
Simon Detheridge f40fbe77aa Simplify logging to log only once per http request 2020-01-10 15:41:36 +00:00
Simon Detheridge e331a3130c Handle AccessDenied and stream-premature-close errors 
These errors tend to occur as part of normal operation and should not generate `logger.err` messages
 2020-01-09 14:15:01 +00:00
Simon Detheridge d8470d4a59 Use correct object path for s3BucketCreds 2020-01-08 11:32:05 +00:00
Simon Detheridge 80d41cf51b Move bucket-specific file endpoint into FileController 2020-01-07 15:05:51 +00:00
Simon Detheridge 6cc5d94f13 Re-add bucket-specific credentials support for S3 2020-01-07 10:55:00 +00:00
Simon Detheridge 2bb2caf7b3 Clean up settings tests 2020-01-07 09:46:53 +00:00
Simon Detheridge b642b3cf3a Rename incorrectly-named 'getDirectorySize' method 2020-01-07 09:30:30 +00:00
Simon Detheridge a6d9d34fe3 Configure PersistorManager to use the new S3 manager for 'aws-sdk' 2020-01-07 09:30:30 +00:00
Simon Detheridge 2ca74fdf15 Cleanup and refactor S3PersistorManager to use aws-sdk only 2020-01-07 09:30:30 +00:00
Simon Detheridge 473aea4e60 Remove AWSSDKPersistorManager 2020-01-07 09:30:30 +00:00
Simon Detheridge 87b51d0c35 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:30:28 +00:00
Simon Detheridge 74b480fc55 Post-decaf cleanup of app.js 2020-01-07 09:29:33 +00:00
Simon Detheridge 85d3c0a852 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:27:02 +00:00
Simon Detheridge fce275e1d4 Post-decaf cleanup of app.js 2020-01-07 09:27:02 +00:00
Simon Detheridge dbfacce988 Post-decaf cleanup of PersistorManager 2020-01-07 09:26:43 +00:00
Simon Detheridge 10f7087912 Post-decaf cleanup of KeyBuilderTests 2020-01-07 09:26:43 +00:00
Simon Detheridge ec80052d9a Decaf cleanup of FileController 
Remove deprecated res.send
 2020-01-07 09:26:23 +00:00
Simon Detheridge 37d44d5d86 Return full path in stubbed 'glob' method 2020-01-07 09:26:01 +00:00
Simon Detheridge f1b6b35c69 Throw errors more consistently in FSPersistorManager 2020-01-07 09:26:00 +00:00
Simon Detheridge 25f1c2bfc4 Delete temporary file when error in writing to stream 2020-01-07 09:26:00 +00:00
Simon Detheridge 039bec02f7 Use directory names in acceptance tests, and add test for getting size 2020-01-07 09:26:00 +00:00
Simon Detheridge 21ba083877 Use 'glob' to determine 'directory' size. 
Check for filtered directory paths in all tests.
 2020-01-07 09:26:00 +00:00
Simon Detheridge ce90292394 Decaf cleanup for FSPersistorManager 2020-01-07 09:26:00 +00:00
Simon Detheridge 4315824d3c Minor cleanup of FilestoreApp acceptance-test helper 2020-01-07 09:25:04 +00:00
Simon Detheridge a8158d6c8c Cleanup and promisify acceptance tests. Run tests for all backends. 2020-01-07 09:25:04 +00:00
Simon Detheridge 006f84abeb Decaf and promisify ImageOptimiser 2020-01-07 09:24:29 +00:00
Simon Detheridge b6a1ea6a30 Cleanup and promisify FileConverter 2020-01-07 09:22:53 +00:00