Commit Graph
61 Commits
Author SHA1 Message Date
Shane Kilkelly fa7ee739bb Update to new metrics api 2017-03-17 14:58:35 +00:00
Shane Kilkelly c30e672549 Merge branch 'master' into sk-upgrade-metrics 2017-03-16 15:17:57 +00:00
Shane Kilkelly c303a1a386 wip: upgrade metrics 2017-03-16 15:17:38 +00:00
James Allen f8865e616d Update ensureIndices to reflect reality 2016-09-30 14:34:38 +01:00
Brian Gough fd49601716 preserve existing history when user upgrades 2016-04-07 15:16:38 +01:00
Brian Gough 6ab75795a2 archive head packs after sufficient time 2016-04-06 13:30:09 +01:00
Brian Gough 795f717bab added index definitions 2016-03-01 11:38:23 +00:00
Brian Gough 3d9dfeccc3 remove pack worker
remove the op-specific code

remove tests for ops, now only packing

remove unused packing code

work in progress

store index for completed packs only

support archiving and unarchiving of individual packs

remove support for archiving whole document history

split out ArchiveManager, IndexManager

remove old DocArchive code

remove docHistoryStats collection

comment about archiving

added method to look at index when last pack has been archived

added start of iterator for project results

use a proper iterator

added heap module

getting it working

increase pack size since bulk operations no longer needed

remove unused MongoAWSexternal

cleanup

added doc iterator

remove old query code

added missing files

cleanup

clean upclean up

started adding pack worker for archiving

work in progress

work in progress

getting pack worker working

updating worker

getting packworker working

added lock

use correct key name for track changes aws access

use correct key name for track changes aws access

always send back users array

fix up comparison of retrieved objects

handle op ids inside packs

log when s3 download completes

comments

cleanup, remove finalisation ideacleanup, remove finalisation idea

remove logging
2016-03-01 10:10:02 +00:00
Brian Gough 0532a4daaa use compound index to replace separate index for packs 2016-01-19 15:56:09 +00:00
Brian Gough 5e830cbbdb put all new ops into packs 2016-01-19 15:56:09 +00:00
Brian Gough dc564fd5d0 archiving document history now sends all changes to s3 2016-01-15 15:54:46 +00:00
Brian Gough 5153ed8217 make peekLastUpdate alway return lastVersion when available 2016-01-15 15:54:44 +00:00
Brian Gough c7b4062412 remove unsupported options argument in count() method of mongojs 1.x 2015-12-22 14:20:34 +00:00
Brian Gough 8ebc069ddb modify last compressed op in place 2015-11-26 16:17:18 +00:00
Brian Gough ad144371d0 gracefully handle updates marked as broken
set update.broken == true to allow the user to view history without a
crash
2015-10-16 11:24:50 +01:00
James Allen 2a03591030 Stub out noisy/slow logger-sharelatex and mongojs modules in tests 2015-09-25 13:46:20 +01:00
Brian Gough e683b0275a bug fix for clear archive in progress flag 2015-09-24 09:09:49 +01:00
Brian Gough 847a553344 prevent double archiving by checking if any inS3 field is already present 2015-09-23 13:29:32 +01:00
Brian Gough e49f260507 allow rollback/locking by setting inS3:false when starting the archive process 2015-09-23 13:28:07 +01:00
Brian Gough 551e8334cf compressedUpdates are now never inserted with inS3
it is now always added later, and a new update is forced for any
addition to an archived update
2015-09-23 13:25:10 +01:00
Brian Gough d6b827426c support forcing new compressed update in popLastCompressedUpdate
callback with a null update, passing the version as an additional
argument
2015-09-23 13:22:38 +01:00
Henrique Dias 1abcea1a66 add some unit test 2015-08-31 18:13:18 -03:00
Henrique Dias efff026a79 handle easier propagation 2015-08-25 16:52:28 -03:00
Henrique Dias fcbe4aa925 fix inS3 propagation 2015-08-24 12:19:19 -03:00
Henrique Dias 98ce03b2f2 replace docs collection to DocstoreHandler 2015-08-24 10:38:31 -03:00
Henrique Dias 04ec45529f restore updates from S3 when exists
fix: avoid rearchiving
2015-08-18 17:11:19 -03:00
Henrique Dias fd4afb3574 Archive changes, care about: versioin, expiresAt and Lock 2015-08-14 15:07:16 -03:00
Henrique Dias 6bc9c9010a handle auto unarchive track changes 2015-08-09 19:52:32 -03:00
Henrique Dias 3bc5380468 handle inS3 flag 2015-08-09 17:50:15 -03:00
Henrique Dias bca48ac117 add unarchive doc track from s3 2015-08-06 17:09:36 -03:00
Henrique Dias 028fe2fa03 archive docChanges list to s3 2015-08-06 11:11:43 -03:00
Brian Gough 66bca8d05c include the current date in the updates to docHistoryStats 2015-05-26 11:00:55 +01:00
Brian Gough 5c4afd5303 add docHistoryStats collection to keep track of updates to docs 2015-05-22 15:40:01 +01:00
Brian Gough 78f0bdbae3 fix name of temporary parameter to match other methods 2015-05-22 15:40:01 +01:00
Brian Gough 4c0eea9916 return an error if trying to pop the last update from a pack 2015-02-17 14:53:50 +00:00
Brian Gough 76cbf32e6a cleanup
remove unnecessary comments, whitespace and unused modules
2015-02-17 13:41:31 +00:00
Brian Gough 448a7b7f36 rename MongoPackManager to PackManager 2015-02-17 11:14:13 +00:00
Brian Gough 4f36ccd519 moved pack code into MongoPackManager 2015-02-13 16:18:15 +00:00
Brian Gough 682e8e8ab6 add a sparse mongo index for finding packs 2015-02-10 16:54:58 +00:00
Brian Gough 1d7f0919a4 fix mongo logic for project search with packs 2015-02-10 16:54:45 +00:00
Brian Gough 7fdce8fc48 split pack handling code into separate parts for docs and projects
need to debug projects
2015-02-09 16:53:44 +00:00
Brian Gough d8dc56b031 don't try to getLastCompressedUpdates from packs 2015-02-09 16:53:05 +00:00
Brian Gough f50f091fc6 added comments 2015-02-06 16:59:09 +00:00
Brian Gough e47476369a clean up docHistory pack decoding 2015-02-06 15:04:46 +00:00
Brian Gough 715b0df4a1 prototype for expanding packs 2015-02-05 16:36:41 +00:00
James Allen eb7bcc6922 Remove old history deletion method 2014-05-29 15:37:16 +01:00
James Allen cc962e1c44 Add indexes in background 2014-05-16 17:00:30 +01:00
James Allen 6a371c267f Fix indexes 2014-05-16 16:41:40 +01:00
James Allen 29ad81c134 Use expiresAt rather than tempCreatedAt 2014-05-16 16:41:14 +01:00
James Allen 8b0b79bc32 Use TTL index to auto-delete updates after a week when versioning is not enabled 2014-05-16 15:59:12 +01:00