CI: stop Yarn git-dep prepare from corrupting the shared fallback cache
Build and Deploy Verso / deploy (push) Successful in 12m50s
Build and Deploy Verso / deploy (push) Successful in 12m50s
The web build's 'yarn install' re-prepares the git-sourced @replit/codemirror-*
deps whenever the Berry cache misses (BuildKit GCs it between builds). Each
prepare uses Yarn Classic, which pulls every esbuild platform binary into the
single shared /usr/local/share/.cache/yarn folder; running several prepares in
parallel races and corrupts it ('tar content corrupt', EEXIST, missing
.yarn-tarball.tgz). Bumping the cache id only cleared it until the next
cache-miss build (#69).
Serialise Yarn's fetch with YARN_NETWORK_CONCURRENCY=1 on the install and
compile steps so the prepares no longer write that cache concurrently, and bump
the fallback cache id (v2 -> v3) once more to discard the currently-corrupt
cache. Slightly slower fetch, but no more random cache corruption.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -25,20 +25,25 @@ ENV COREPACK_ENABLE_NETWORK=0
|
||||
|
||||
# Install yarn dependencies
|
||||
# -------------------------
|
||||
# YARN_NETWORK_CONCURRENCY=1 serialises Yarn's fetch step. The git-sourced
|
||||
# @replit/codemirror-* deps are prepared with Yarn Classic, which pulls *all*
|
||||
# esbuild platform binaries into the one shared /usr/local/share/.cache/yarn
|
||||
# folder; preparing several of them in parallel races and corrupts that cache
|
||||
# (EEXIST / missing .yarn-tarball.tgz). Serialising the fetch removes the race.
|
||||
RUN --mount=type=cache,target=/root/.cache \
|
||||
--mount=type=cache,target=/root/.yarn/berry/cache,id=server-ce-yarn-cache \
|
||||
--mount=type=cache,target=/usr/local/share/.cache/yarn,id=server-ce-yarn-fallback-cache-v2 \
|
||||
--mount=type=tmpfs,target=/tmp node genScript install | bash
|
||||
--mount=type=cache,target=/usr/local/share/.cache/yarn,id=server-ce-yarn-fallback-cache-v3 \
|
||||
--mount=type=tmpfs,target=/tmp node genScript install | YARN_NETWORK_CONCURRENCY=1 bash
|
||||
|
||||
# Add the actual source files
|
||||
# ---------------------------
|
||||
COPY --parents libraries/ services/ tools/migrations/ /overleaf/
|
||||
RUN --mount=type=cache,target=/root/.cache \
|
||||
--mount=type=cache,target=/root/.yarn/berry/cache,id=server-ce-yarn-cache \
|
||||
--mount=type=cache,target=/usr/local/share/.cache/yarn,id=server-ce-yarn-fallback-cache-v2 \
|
||||
--mount=type=cache,target=/usr/local/share/.cache/yarn,id=server-ce-yarn-fallback-cache-v3 \
|
||||
--mount=type=cache,target=/overleaf/services/web/node_modules/.cache,id=server-ce-webpack-cache \
|
||||
--mount=type=tmpfs,target=/tmp \
|
||||
node genScript compile | bash || \
|
||||
node genScript compile | YARN_NETWORK_CONCURRENCY=1 bash || \
|
||||
(echo "==== PACK LOGS ====" && find /tmp -name pack.log -exec cat {} \; && exit 1)
|
||||
# Copy runit service startup scripts to its location
|
||||
# --------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user