Extend run-openfront skill with headless in-game WebGL testing (#4245)

## What

Extends the `run-openfront` Claude Code skill so agents can test the
*whole game* headlessly, not just the home page and modals. New
`game.mjs` driver plays an actual singleplayer game end-to-end:

- start a solo game with chosen options (bots, map, difficulty, …)
- spawn, attack/expand, open the radial build menu
- read ground-truth sim state (`ticks`, `inSpawnPhase`, `myPlayer`
troops/gold/tiles, `outgoingAttacks`) instead of guessing from pixels
- take real WebGL screenshots (SwiftShader renders the map fine
headless)

`node .claude/skills/run-openfront/game.mjs` runs a ~2 min smoke flow
that asserts territory growth after an expansion attack and that the
radial menu opens.

## How

No game-code changes were needed:

- `hud/GameRenderer.ts` already assigns the `GameView` and
`TransformHandler` onto the `<build-menu>` element, so page JS reaches
live sim state and world↔screen conversion through it.
- `launch({ rafIntervalMs })` stubs `requestAnimationFrame` to one frame
per interval. SwiftShader needs seconds of CPU per frame, and an
unthrottled frame loop starves the main thread — the singleplayer turn
loop drops from 10 ticks/s to ~0.3. Throttled, the sim runs near full
speed while frames still render for screenshots.
- `clickWorld()` absorbs the canvas-click pitfalls discovered while
testing: aims at tile centers (corner clicks floor onto the neighboring
tile), refuses to click through HUD elements covering
`#game-input-overlay`, and freezes the post-spawn camera animation so
computed coordinates don't go stale.

## Testing

Smoke flow run repeatedly on a headless 4-core box: game starts (123
players), spawn lands on the clicked tile, expansion attack grows
territory 52 → ~275 tiles, radial menu opens, screenshots show the
rendered map.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

.claude/skills/run-openfront/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: run-openfront
-description: Build, run, and drive OpenFront locally. Use when asked to run the game, start the dev server, take a screenshot of the UI, verify a client change in the real app, or interact with the running game (lobby, modals, map picker).
+description: Build, run, and drive OpenFront locally — including full in-game WebGL testing. Use when asked to run the game, start the dev server, take a screenshot of the UI, verify a client change in the real app, or interact with the running game (lobby, modals, map picker, starting a singleplayer game, spawning, attacking, build menu, reading live sim state).
 ---
 
 OpenFront is a browser game (Lit + Pixi.js client, Node game server).
@@ -66,6 +66,99 @@ const s = await page.evaluate(
 await browser.close();
 ```
 
+## Drive a full game (WebGL, in-game interaction)
+
+`game.mjs` drives an actual singleplayer game end-to-end: start, spawn,
+attack/expand, open the radial menu, and read **ground-truth sim state**.
+WebGL works headless via SwiftShader (no extra flags needed), and the
+screenshots show the real rendered map.
+
+Smoke flow (≈2 min: starts a 50-bot game, spawns, expands, opens the
+radial menu, asserts territory growth):
+
+```bash
+node .claude/skills/run-openfront/game.mjs
+# screenshots: /tmp/openfront-run/game-{spawn-phase,spawned,expanded,radial-menu}.png
+```
+
+For ad-hoc in-game flows, import the helpers (script must live inside the
+repo):
+
+```js
+import {
+  launch,
+  gotoHome,
+  openSoloModal,
+} from "./.claude/skills/run-openfront/driver.mjs";
+import {
+  startSoloGame, // set modal options ({bots, map, difficulty, instantBuild, …}), click Start, wait for sim
+  gameState, // {ticks, inSpawnPhase, numPlayers, myPlayer: {troops, gold, tilesOwned, isAlive}, …}
+  findSpawnTile,
+  spawn, // pick land + click it; waits until myPlayer owns tiles
+  waitForSpawnPhaseEnd,
+  waitForTick,
+  findExpansionTile,
+  attack,
+  clickWorld,
+  panTo,
+  setAttackRatio,
+  openRadialMenu, // right-click on own territory; returns true if the menu opened
+} from "./.claude/skills/run-openfront/game.mjs";
+
+const { browser, page } = await launch({ rafIntervalMs: 3000 }); // throttle is REQUIRED in-game, see below
+await gotoHome(page);
+await openSoloModal(page);
+await startSoloGame(page, { bots: 50 });
+const tile = await spawn(page);
+await waitForSpawnPhaseEnd(page);
+const target = await findExpansionTile(page, tile);
+await attack(page, target.x, target.y);
+await browser.close();
+```
+
+### How it works / in-game gotchas
+
+- **Ground-truth state without any repo changes**: `hud/GameRenderer.ts`
+  assigns the `GameView` and `TransformHandler` onto the `<build-menu>`
+  Lit element (light DOM). From page JS:
+  `document.querySelector("build-menu").game` / `.transformHandler`.
+  GameView has `ticks()`, `inSpawnPhase()`, `myPlayer()`, `players()`,
+  `ref(x,y)`, `isLand()`, `hasOwner()`; PlayerView has `troops()`,
+  `numTilesOwned()`, `gold()`, `isAlive()`, `outgoingAttacks()`.
+- **`launch({ rafIntervalMs: 3000 })` is mandatory for in-game work.**
+  SwiftShader needs seconds of CPU per frame; an unthrottled rAF loop
+  starves the main thread (0.8 fps, 100 ms timers firing every ~4 s) and
+  the singleplayer turn loop crawls at ~0.3 ticks/s instead of 10/s. The
+  throttle stubs `requestAnimationFrame` to one frame per interval —
+  sim runs near full speed, frames still render for screenshots.
+- **Solo modal options are settable as element properties** before
+  clicking Start: `document.querySelector("single-player-modal").bots = 50`
+  (`@state` fields are TS-private only). `startSoloGame` does this.
+- **Click tile centers, not corners.** World coords address a tile's
+  top-left corner and `screenToWorldCoordinates` floors — a corner click
+  can land on the neighboring tile (and clicking your own tile is a
+  silent no-op). `clickWorld` aims at `+0.5,+0.5`.
+- **HUD elements swallow canvas clicks.** The leaderboard / control panel
+  / modals sit above the `#game-input-overlay`. `clickWorld` verifies
+  `document.elementFromPoint` hits the overlay and recenters the camera
+  (`panTo`) if not — never click raw screen coords yourself.
+- **The camera animates on its own** (post-spawn go-to-player), so screen
+  coords computed before the click go stale. `clickWorld` calls
+  `transformHandler.clearTarget()` first to freeze it.
+- **Spawning**: during the spawn phase a left click on unowned land sends
+  the spawn intent; in singleplayer the spawn phase ends as soon as the
+  human spawns. `nameLocation()` can still be `{0,0}` for the first ticks
+  after spawning — pass the spawn tile as fallback origin (helpers do).
+- **Attacking**: a left click outside the spawn phase attacks/expands if
+  `canAttack` (unowned land must be connected to your border through
+  unowned land). Troops drop and `outgoingAttacks()` becomes non-empty on
+  success. The radial menu (right click) is a DOM/SVG overlay —
+  `.radial-menu-container` exists from startup; check
+  `style.display !== "none"` for "open".
+- Verify rendering visually by reading the screenshots — a blank WebGL
+  canvas means SwiftShader broke (check `webgl2` context creation and
+  `LD_LIBRARY_PATH`/fontconfig from setup.sh).
+
 ## Run (human path)
 
 `npm run dev`, open http://localhost:9000 in a browser. Useless headless.

.claude/skills/run-openfront/driver.mjs

@@ -16,7 +16,14 @@ export const BASE_URL = process.env.OPENFRONT_URL ?? "http://localhost:9000";
 // Launch chromium with the locally-extracted system libraries and fontconfig
 // (see setup.sh). Without them the headless shell dies on libnspr4.so, and
 // later Skia FATALs on the missing fontconfig.
-export async function launch() {
+// opts:
+//   viewport      - {width, height}, default 1400x1000
+//   rafIntervalMs - throttle requestAnimationFrame to one frame per interval.
+//                   Essential for in-game testing: SwiftShader needs seconds
+//                   of CPU per frame, and an unthrottled rAF loop starves the
+//                   main thread (timers, the singleplayer turn loop, input).
+//                   ~1000 is a good value; frames still render for screenshots.
+export async function launch({ viewport, rafIntervalMs } = {}) {
   const env = { ...process.env };
   const libs = path.join(CACHE, "extracted", "usr", "lib", "x86_64-linux-gnu");
   if (fs.existsSync(libs)) {
@@ -30,8 +37,22 @@ export async function launch() {
     env,
   });
   const context = await browser.newContext({
-    viewport: { width: 1400, height: 1000 },
+    viewport: viewport ?? { width: 1400, height: 1000 },
   });
+  if (rafIntervalMs) {
+    await context.addInitScript((interval) => {
+      let last = 0;
+      window.requestAnimationFrame = (cb) => {
+        const now = performance.now();
+        const wait = Math.max(0, interval - (now - last));
+        return setTimeout(() => {
+          last = performance.now();
+          cb(last);
+        }, wait);
+      };
+      window.cancelAnimationFrame = (id) => clearTimeout(id);
+    }, rafIntervalMs);
+  }
   const page = await context.newPage();
   page.on("pageerror", (e) =>
     console.log("PAGEERROR:", e.message.split("\n")[0]),

.claude/skills/run-openfront/game.mjs

@@ -0,0 +1,366 @@
+// In-game driver for OpenFront: start a singleplayer game headless, spawn,
+// attack, and read ground-truth simulation state from the page.
+//
+// Run the smoke flow from the repo root (dev server must be up):
+//   node .claude/skills/run-openfront/game.mjs
+// or import the helpers in an ad-hoc script inside the repo.
+//
+// Ground-truth access — no repo changes needed: src/client/hud/GameRenderer.ts
+// assigns the GameView and TransformHandler onto the <build-menu> Lit element
+// (light DOM), so page JS can reach them via
+//   document.querySelector("build-menu").game / .transformHandler
+import fs from "fs";
+import { gotoHome, launch, openSoloModal } from "./driver.mjs";
+
+// ---------- game lifecycle ----------
+
+// From an open single-player modal: tweak options and click Start, then wait
+// until the game is fully loaded (renderer mounted, sim ticking).
+// opts: { bots, instantBuild, infiniteGold, infiniteTroops, map, difficulty }
+export async function startSoloGame(page, opts = {}) {
+  if (Object.keys(opts).length > 0) {
+    await page.evaluate((o) => {
+      const modal = document.querySelector("single-player-modal");
+      if (o.bots !== undefined) modal.bots = o.bots;
+      if (o.instantBuild !== undefined) modal.instantBuild = o.instantBuild;
+      if (o.infiniteGold !== undefined) modal.infiniteGold = o.infiniteGold;
+      if (o.infiniteTroops !== undefined)
+        modal.infiniteTroops = o.infiniteTroops;
+      if (o.map !== undefined) modal.selectedMap = o.map;
+      if (o.difficulty !== undefined) modal.selectedDifficulty = o.difficulty;
+    }, opts);
+    await page.waitForTimeout(300); // let Lit re-render
+  }
+  await page
+    .locator('o-button[translationKey="single_modal.start"] button:visible')
+    .first()
+    .click();
+  await waitForGameReady(page);
+}
+
+// Game is "ready" when the HUD has its GameView and the sim has ticked.
+export async function waitForGameReady(page, timeout = 180_000) {
+  await page.waitForFunction(
+    () => {
+      const bm = document.querySelector("build-menu");
+      return bm?.game !== undefined && bm.game.ticks() > 0;
+    },
+    undefined,
+    { timeout, polling: 500 },
+  );
+}
+
+// Snapshot of ground-truth sim state (everything serializable happens
+// in-page; live objects can't cross the evaluate boundary).
+export async function gameState(page) {
+  return await page.evaluate(() => {
+    const g = document.querySelector("build-menu")?.game;
+    if (!g) return null;
+    const me = g.myPlayer();
+    const players = g.players();
+    return {
+      ticks: g.ticks(),
+      inSpawnPhase: g.inSpawnPhase(),
+      mapSize: { width: g.width(), height: g.height() },
+      numPlayers: players.length,
+      numAlive: players.filter((p) => p.isAlive()).length,
+      myPlayer:
+        me === null
+          ? null
+          : {
+              name: me.name(),
+              isAlive: me.isAlive(),
+              troops: me.troops(),
+              gold: String(me.gold()),
+              tilesOwned: me.numTilesOwned(),
+            },
+    };
+  });
+}
+
+// ---------- coordinates & clicking ----------
+
+// World (tile) coords -> screen px, or null if off-viewport.
+// worldToScreenCoordinates only reads cell.x/.y, so a plain object works.
+export async function worldToScreen(page, x, y) {
+  return await page.evaluate(
+    ([wx, wy]) => {
+      const t = document.querySelector("build-menu")?.transformHandler;
+      const s = t.worldToScreenCoordinates({ x: wx, y: wy });
+      if (
+        s.x < 0 ||
+        s.y < 0 ||
+        s.x > window.innerWidth ||
+        s.y > window.innerHeight
+      )
+        return null;
+      return s;
+    },
+    [x, y],
+  );
+}
+
+// Click a world tile on the canvas (left button = spawn during spawn phase,
+// attack/expand after it). If the tile is off-screen (e.g. the camera is
+// still animating to the player after spawn), recenter on my player and wait.
+export async function clickWorld(page, x, y, button = "left") {
+  // Stop any in-flight camera animation (e.g. the post-spawn go-to-player),
+  // otherwise the transform changes between computing coords and clicking.
+  await page.evaluate(() => {
+    document.querySelector("build-menu").transformHandler.clearTarget();
+  });
+  // Aim at the tile center: world coords address the tile's top-left corner
+  // and screenToWorld floors, so a corner click can land on the neighbor.
+  const [cx, cy] = [x + 0.5, y + 0.5];
+  const clickable = async () => {
+    const s = await worldToScreen(page, cx, cy);
+    if (s === null) return null;
+    // HUD elements (leaderboard, control panel, modals) sit above the input
+    // overlay and swallow pointer events — only click if the overlay is hit.
+    const hit = await page.evaluate(
+      ([px, py]) => document.elementFromPoint(px, py)?.id ?? "",
+      [s.x, s.y],
+    );
+    return hit === "game-input-overlay" ? s : null;
+  };
+  let s = await clickable();
+  if (s === null) {
+    await panTo(page, x, y); // viewport center is clear of HUD chrome
+    s = await clickable();
+  }
+  if (s === null)
+    throw new Error(`world tile (${x},${y}) is off-screen or covered by HUD`);
+  await page.mouse.click(s.x, s.y, { button });
+  return s;
+}
+
+// Snap the camera so world point (x,y) is at the viewport center.
+// screenCenter() returns world coords and offsetX/Y are in world units,
+// so the pan is a plain delta — no animation to wait on.
+export async function panTo(page, x, y) {
+  await page.evaluate(
+    ([wx, wy]) => {
+      const t = document.querySelector("build-menu").transformHandler;
+      const c = t.screenCenter();
+      t.offsetX += wx - c.screenX;
+      t.offsetY += wy - c.screenY;
+    },
+    [x, y],
+  );
+  await page.waitForTimeout(300); // let the frame loop redraw
+}
+
+// ---------- spawn ----------
+
+// Find a spawnable tile (land, unowned, on-screen, away from HUD edges).
+export async function findSpawnTile(page, margin = 200) {
+  return await page.evaluate((m) => {
+    const bm = document.querySelector("build-menu");
+    const g = bm.game;
+    const t = bm.transformHandler;
+    const w = g.width();
+    const h = g.height();
+    // Deterministic grid sweep, center-out, so we don't need randomness.
+    const cells = [];
+    for (let i = 0; i < 4000; i++) {
+      const gx = Math.floor((i * 79) % w);
+      const gy = Math.floor((i * 131) % h);
+      cells.push([gx, gy]);
+    }
+    for (const [x, y] of cells) {
+      if (!g.isValidCoord(x, y)) continue;
+      const ref = g.ref(x, y);
+      if (!g.isLand(ref) || g.hasOwner(ref)) continue;
+      const s = t.worldToScreenCoordinates({ x, y });
+      if (
+        s.x < m ||
+        s.y < m ||
+        s.x > window.innerWidth - m ||
+        s.y > window.innerHeight - m
+      )
+        continue;
+      return { x, y, screen: s };
+    }
+    return null;
+  }, margin);
+}
+
+// Click a spawn point and wait until the player exists and owns tiles.
+// Returns the spawn tile. In singleplayer the spawn phase ends as soon as
+// the human spawns (SpawnExecution).
+export async function spawn(page, tile = null) {
+  tile = tile ?? (await findSpawnTile(page));
+  if (tile === null) throw new Error("no spawnable tile found on screen");
+  await clickWorld(page, tile.x, tile.y);
+  await page.waitForFunction(
+    () => {
+      const g = document.querySelector("build-menu")?.game;
+      const me = g?.myPlayer();
+      return me !== null && me !== undefined && me.numTilesOwned() > 0;
+    },
+    undefined,
+    { timeout: 30_000, polling: 250 },
+  );
+  return tile;
+}
+
+export async function waitForSpawnPhaseEnd(page, timeout = 60_000) {
+  await page.waitForFunction(
+    () => {
+      const g = document.querySelector("build-menu")?.game;
+      return g !== undefined && !g.inSpawnPhase();
+    },
+    undefined,
+    { timeout, polling: 500 },
+  );
+}
+
+// ---------- actions ----------
+
+// Set the troop fraction used per attack (0.01–1).
+export async function setAttackRatio(page, ratio) {
+  await page.evaluate((r) => {
+    document.querySelector("control-panel").uiState.attackRatio = r;
+  }, ratio);
+}
+
+// Attack/expand toward a world tile: a plain left click outside spawn phase
+// triggers ClientGameRunner.inputEvent -> SendAttackIntentEvent if attackable.
+export async function attack(page, x, y) {
+  await clickWorld(page, x, y);
+}
+
+// Find an unowned land tile near (and outside) my territory border — the
+// natural "expand" click target right after spawning. `near` is a {x,y}
+// fallback (e.g. the spawn tile): nameLocation() can still be (0,0) in the
+// first ticks after spawning, before name render data is computed.
+export async function findExpansionTile(page, near = null) {
+  return await page.evaluate((fallback) => {
+    const bm = document.querySelector("build-menu");
+    const g = bm.game;
+    const me = g.myPlayer();
+    if (!me) return null;
+    const loc = me.nameLocation();
+    const origin = loc && loc.size > 0 ? loc : fallback;
+    if (!origin) return null;
+    const cx = Math.round(origin.x);
+    const cy = Math.round(origin.y);
+    for (let r = 2; r < 100; r += 2) {
+      for (const [dx, dy] of [
+        [r, 0],
+        [-r, 0],
+        [0, r],
+        [0, -r],
+        [r, r],
+        [-r, -r],
+      ]) {
+        const x = cx + dx;
+        const y = cy + dy;
+        if (!g.isValidCoord(x, y)) continue;
+        const ref = g.ref(x, y);
+        if (g.isLand(ref) && !g.hasOwner(ref)) return { x, y };
+      }
+    }
+    return null;
+  }, near);
+}
+
+// Open the radial (build) menu with a right click on my territory.
+// Returns true if the radial menu became visible (it's a DOM/SVG overlay).
+// `at` falls back to my nameLocation when omitted.
+export async function openRadialMenu(page, at = null) {
+  at ??= await page.evaluate(() => {
+    const g = document.querySelector("build-menu").game;
+    const me = g.myPlayer();
+    if (!me) return null;
+    const loc = me.nameLocation();
+    if (!loc || loc.size <= 0) return null;
+    return { x: Math.round(loc.x), y: Math.round(loc.y) };
+  });
+  if (at === null) throw new Error("no territory location — spawn first");
+  await clickWorld(page, at.x, at.y, "right");
+  await page.waitForTimeout(800);
+  // The container div always exists (created hidden at startup) — visibility
+  // is the actual open/closed signal.
+  return await page.evaluate(() => {
+    const el = document.querySelector(".radial-menu-container");
+    return el !== null && el.style.display !== "none";
+  });
+}
+
+// ---------- waiting / verification ----------
+
+// Wait until the sim reaches a given tick (game ticks are 100ms).
+export async function waitForTick(page, tick, timeout = 120_000) {
+  await page.waitForFunction(
+    (target) => {
+      const g = document.querySelector("build-menu")?.game;
+      return g !== undefined && g.ticks() >= target;
+    },
+    tick,
+    { timeout, polling: 500 },
+  );
+}
+
+// ---------- smoke flow ----------
+
+const isMain =
+  process.argv[1] && import.meta.url === `file://${process.argv[1]}`;
+if (isMain) {
+  const out = "/tmp/openfront-run";
+  fs.mkdirSync(out, { recursive: true });
+  const shot = (page, name) =>
+    page.screenshot({ path: `${out}/${name}.png` }).then(() => {
+      const kb = Math.round(fs.statSync(`${out}/${name}.png`).size / 1024);
+      console.log(`screenshot ${out}/${name}.png (${kb} KB)`);
+    });
+
+  // rAF throttle is what makes in-game testing viable: SwiftShader frames
+  // cost seconds of CPU and an unthrottled frame loop starves the sim.
+  const { browser, page } = await launch({ rafIntervalMs: 3000 });
+  page.on("console", (m) => {
+    if (/clicked cell/.test(m.text())) console.log("   PAGE:", m.text());
+  });
+  try {
+    console.log("1. home + solo modal");
+    await gotoHome(page);
+    await openSoloModal(page);
+
+    console.log("2. starting solo game (50 bots)…");
+    await startSoloGame(page, { bots: 50 });
+    console.log("   game ready:", JSON.stringify(await gameState(page)));
+    await shot(page, "game-spawn-phase");
+
+    console.log("3. spawning…");
+    const tile = await spawn(page, await findSpawnTile(page));
+    console.log(`   spawned at (${tile.x},${tile.y})`);
+    await waitForSpawnPhaseEnd(page);
+    console.log("   spawn phase over:", JSON.stringify(await gameState(page)));
+    await shot(page, "game-spawned");
+
+    console.log("4. expanding (attack unowned land)…");
+    const before = await gameState(page);
+    const target = await findExpansionTile(page, tile);
+    if (target === null) throw new Error("no expansion tile found");
+    await attack(page, target.x, target.y);
+    await waitForTick(page, before.ticks + 50); // let 5s of sim run
+    const after = await gameState(page);
+    console.log("   after:", JSON.stringify(after));
+    if (after.myPlayer.tilesOwned <= before.myPlayer.tilesOwned) {
+      throw new Error("territory did not grow after attack");
+    }
+    console.log(
+      `   territory grew ${before.myPlayer.tilesOwned} -> ${after.myPlayer.tilesOwned} ✓`,
+    );
+    await shot(page, "game-expanded");
+
+    console.log("5. radial menu (right click)…");
+    const radialOpen = await openRadialMenu(page, tile);
+    console.log(`   radial menu visible: ${radialOpen}`);
+    await shot(page, "game-radial-menu");
+
+    console.log("SMOKE OK");
+  } finally {
+    await browser.close();
+  }
+}