Commit Graph

92 Commits

Author SHA1 Message Date
James Allen 4fadd75ef3 Track changes based on flag on op, not global setting 2016-12-02 11:04:21 +00:00
James Allen f43355b74d Fix UpdateManager tests 2016-12-01 18:19:47 +00:00
James Allen ce93a76e7f Fix ShareJsUpdateManager tests 2016-12-01 18:11:03 +00:00
James Allen e7ff05e792 Fix UpdateManager tests 2016-12-01 18:06:33 +00:00
James Allen 889f5fdf9f Fix ShareJsDB tests 2016-12-01 17:14:40 +00:00
James Allen b6c93c718d Update TrackChangesManagerTests -> HistoryManagerTests 2016-12-01 16:50:55 +00:00
James Allen d878dd5758 Fix RedisManagerTests 2016-12-01 16:49:53 +00:00
James Allen 9ee913be39 Update PersistenceManagerTests 2016-12-01 16:40:15 +00:00
James Allen fb39e37fe0 Update DocumentManager tests 2016-12-01 16:27:40 +00:00
James Allen efab68e6b2 Add in guards on bad data from web API 2016-11-29 17:13:16 +00:00
James Allen 546d9e9cee Remove left over test 2016-11-29 17:13:07 +00:00
James Allen 2852043a6d Don't store doc version in Mongo directly, instead use docstore 2016-11-29 17:06:23 +00:00
James Allen 5ce15c4d60 Move check of zero length op array up a level 2016-09-12 10:37:41 +01:00
James Allen 993aab7a78 Don't try to redis rpush with no arguments 2016-09-12 10:37:32 +01:00
James Allen 2d82d56f93 Process updates one at a time and write into redis after each one 2016-09-09 11:01:14 +01:00
James Allen c85fd7ef41 Merge branch 'master' into ja-consistency-refactor 2016-09-07 11:08:04 +01:00
Brian Gough 9bc7594226 clean up redis query 2016-09-05 10:31:54 +01:00
James Allen 8779f3f686 Only write DocOps atomically with version and lines, after all docs are applied 2016-08-23 16:00:46 +01:00
James Allen 87f3e5e809 Remove old migration code 2016-08-23 11:03:37 +01:00
James Allen 1c62a1c5a1 Don't propagate error on secondary timeout 2016-07-07 09:46:15 +01:00
James Allen 59883023ca Ignore different order of results from smembers 2016-07-06 14:24:27 +01:00
James Allen 89f90c1b04 Timeout secondary requests if they take longer than 200ms 2016-07-06 12:25:36 +01:00
James Allen 508a95c19b Use the main redis instance for locks 2016-07-06 11:50:02 +01:00
James Allen dfd45bd23c Add timers to time how long each redis request takes 2016-07-04 11:14:23 +01:00
James Allen fde334acc4 Create script that will ensure data is migrate between redis backends 2016-06-29 12:57:56 +01:00
James Allen dbe03e2757 Track metrics of when backends match or disagree 2016-06-28 16:49:15 +01:00
James Allen e04c946ecd Fix unit tests 2016-06-23 18:04:26 +01:00
James Allen da89ff7172 Add in external health check rather than internal 2016-06-23 18:00:03 +01:00
James Allen 414ab5d6a9 Refactor TCP redis monitoring to be more explicit 2016-06-23 15:59:25 +01:00
James Allen 8ef03c3d2f Add in application layer monitoring of the health of each cluster node 2016-06-23 15:38:51 +01:00
James Allen bc00aab7b1 Explicitly separate calls to web and docupdater redis instances 2016-06-17 12:17:22 +01:00
James Allen 9ff3026807 Merge branch 'master' into ja_redis_cluster 2016-06-09 10:00:58 +01:00
James Allen 48a92b28e5 Don't run redis commands in parallel for easier consistency reasoning 2016-06-09 09:54:13 +01:00
James Allen c823e06912 Don't run redis commands in parallel for easier consistency reasoning 2016-06-08 16:42:09 +01:00
James Allen b4936f62af Check that return values from different redis backends match 2016-06-08 12:18:37 +01:00
James Allen 1db6f8f159 Pull out rclient into RedisBackend that supports sending requests to multiple rclients 2016-06-07 17:58:18 +01:00
James Allen e38e883081 Clean up and consolidate RedisManager tests for easy refactoring later 2016-06-01 12:22:47 +01:00
James Allen 8004e022fe Fix async version and fix affected unit tests 2016-06-01 11:49:24 +01:00
James Allen 05b09a447e Ensure that all multi call keys will hash to the same node in cluster 2016-06-01 11:40:12 +01:00
James Allen f707783aba Remove old methods of triggering doc updates
Remove the old pub/sub listener which is no longer used.

Also remove the DocsWithPendingUpdates set, which used to track
docs waiting to be updated. This was necessary incase messages were
missed on the pub/sub channel, so we knew which docs still had pending
updates. However, now we use the BLPOP queue, these updates just sit in
the queue until a consumer comes back to continue consuming them.
2016-06-01 11:28:23 +01:00
James Allen b2e58266ae Remove unused total doc counting redis commands 2016-06-01 11:03:39 +01:00
James Allen 32ce44f9c6 Remove unused gzipping logic 2016-06-01 10:53:50 +01:00
James Allen 1d7cbd8a78 Fix unit tests 2016-05-31 13:32:15 +01:00
Shane Kilkelly 048fd19418 Add one to size of line.
To account for newline characters in the original document
2016-05-12 09:26:50 +01:00
Shane Kilkelly 82d5a7fafd check total size of lines, rather than content-length 2016-05-11 15:55:21 +01:00
Shane Kilkelly 210a61112b reject payloads larger than 2MB for setDoc 2016-04-29 15:08:21 +01:00
James Allen 945c728db2 Use signed locks so only the locking party can remove their lock 2016-04-13 11:59:56 +01:00
James Allen 6c79ab4321 Don't let HTTP calls take longer than 5 seconds since we're inside a 30 second lock 2016-04-12 17:10:39 +01:00
James Allen 3c7c318ea0 Clean up and speed up unit tests by making sure requires are mocked 2016-01-20 17:36:06 +00:00
James Allen af5d01e440 Flush track changes when unloading data from redis 2016-01-20 15:05:31 +00:00