CI: push images to in-cluster registry to bypass Traefik
Build and Deploy Verso / deploy (push) Failing after 1s

The TeX Live layer (~3.5 GB) failed to push to registry.alocoq.fr:
Traefik severed the upload mid-stream ("client disconnected during blob
PUT ... unexpected EOF"), buildkit retried at the wrong offset, and the
registry returned "blob upload invalid".

Push to the in-cluster registry Service (registry.git.svc.cluster.local:5000)
instead, so the upload never traverses Traefik. Changes:
- buildctl outputs use registry.insecure=true (registry is plain HTTP)
- add a verso-buildkitd-config ConfigMap with buildkitd.toml marking the
  registry http/insecure, so the second build can pull the base image back
- the verso Deployment and rolling update reference the in-cluster image

NOTE: the cluster nodes' containerd must also treat
registry.git.svc.cluster.local:5000 as an insecure registry, otherwise
the kubelet image pull for the test deployment will fail. That is node-
level config outside this repo.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
claude
2026-05-31 20:46:30 +00:00
parent 3bb293f7a7
commit 8f2f6d1684
+33 -5
View File
@@ -19,6 +19,21 @@ jobs:
run: |
kubectl -n ci delete job verso-buildkit --ignore-not-found=true --wait=true
# buildkitd config: mark the in-cluster registry as http (insecure)
# so the second build can resolve/pull the base image we just pushed.
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: verso-buildkitd-config
namespace: ci
data:
buildkitd.toml: |
[registry."registry.git.svc.cluster.local:5000"]
http = true
insecure = true
EOF
cat <<'EOF' | kubectl apply -f -
apiVersion: batch/v1
kind: Job
@@ -52,27 +67,40 @@ jobs:
- |
set -eux
# Push to the in-cluster registry (plain HTTP) to bypass
# the Traefik ingress, whose read timeout was killing the
# multi-GB TeX Live layer upload mid-stream. The base
# image is pulled back in for the second build, so the
# registry must be marked insecure for both push and pull
# (buildkitd.toml handles the pull/resolve side).
REG=registry.git.svc.cluster.local:5000
buildctl-daemonless.sh build \
--frontend=dockerfile.v0 \
--local context=/workspace/repo \
--local dockerfile=/workspace/repo/server-ce \
--opt filename=Dockerfile-base \
--output type=image,name=registry.alocoq.fr/verso-base:latest,push=true
--output type=image,name=$REG/verso-base:latest,push=true,registry.insecure=true
buildctl-daemonless.sh build \
--frontend=dockerfile.v0 \
--local context=/workspace/repo \
--local dockerfile=/workspace/repo/server-ce \
--opt filename=Dockerfile \
--opt build-arg:OVERLEAF_BASE_TAG=registry.alocoq.fr/verso-base:latest \
--output type=image,name=registry.alocoq.fr/verso:latest,push=true
--opt build-arg:OVERLEAF_BASE_TAG=$REG/verso-base:latest \
--output type=image,name=$REG/verso:latest,push=true,registry.insecure=true
volumeMounts:
- name: workspace
mountPath: /workspace
- name: buildkitd-config
mountPath: /etc/buildkit
volumes:
- name: workspace
emptyDir: {}
- name: buildkitd-config
configMap:
name: verso-buildkitd-config
EOF
- name: Wait for build
@@ -212,7 +240,7 @@ jobs:
spec:
containers:
- name: verso
image: registry.alocoq.fr/verso:latest
image: registry.git.svc.cluster.local:5000/verso:latest
ports:
- containerPort: 80
env:
@@ -244,7 +272,7 @@ jobs:
- name: Deploy Verso image
run: |
kubectl -n test set image deployment/verso \
verso=registry.alocoq.fr/verso:latest
verso=registry.git.svc.cluster.local:5000/verso:latest
kubectl -n test rollout restart deployment/verso
kubectl -n test rollout status deployment/verso --timeout=300s