Pathfinding - optimize naval invasions (#2932)

# Pathfinding pt. 4

https://pf-pt-4.openfront.dev/

## Description:

Hello again! Pathfinding. It's fast, but inaccurate. This PR makes it
more accurate and actually faster. Sadly it is _faster_ because of a
blunder in previous PR (using BucketQueue where MinHeap would be
better), not because of a new tech. More importantly, it is more
accurate. And that's what people apparently want.

## What changed?

Most of the functional changes relate to `SpatialQuery` module. This is
the thingy that answers "we know the target, which tile of my territory
is the best to launch an invasion". To make it compute a path from South
America to the deep inland China river, it has to work on a coerced map,
one with a very small resolution, so small in fact, that every 4096 map
tiles gets compressed to just one pixel. I hope you see where this is
going.

Previously we selected a random coastal tile within this big pixel
(honestly it wasn't random at all, but could very well be for the
illustrative purposes). Now, we try to be a bit more deliberate. Since
we already know the rough location of the probably best tile, we can
exclude all other tiles from the computation. Imagine a player's
territory spans both Americas on global map - that's a lot of shores.
But since we already know the best tile is somewhere close to Miami, the
problem space was greatly reduced, no need to consider all other shores.
But pathing to the target in China from Miami is still crazy expensive.

This is where second trick comes to play - instead of pathing all the
way to China, we select a _waypoint_ in the rough direction of China,
about 100 to 200 tiles away. This way we fairly cheaply select best tile
to launch an invasion towards this abstract point. And chances are, this
point is far enough, the newly computed path is very close to being
optimal. When you throw a dart from far away, the difference between
scoring 10 and missing is very small. This is why aiming in the general
direction of the board - as opposed to the ceiling - is usually good
enough.

## Okay, but what about the crazy paths when I send invasion to the
opposed bank of a river?!

Well, pathing from America to China is cool, but most players wouldn't
notice the difference on such long paths, what about the short ones? We
now try more accurate pathing first and defer to hierarchy only if it
fails. This produces much better paths for short invasions. While the
fix described above ensures the accuracy is improved also on
medium-to-long routes.

## Playground

Yes.


https://github.com/user-attachments/assets/9cf9586f-c99a-416d-b856-8cf0a21c35ed

## CodeRabbit

Grab a 🥕. Remember `tests/pathfinding/playground` is mostly generated
code and go easy on it. It's enough for it to work and do it's job of
visualizing the paths. No need for throughout review of these files.

## Please complete the following:

- [x] I have added screenshots for all UI updates
- [x] I process any text displayed to the user through translateText()
and I've added it to the en.json file
- [x] I have added relevant tests to the test directory
- [x] I confirm I have thoroughly tested these changes and take full
responsibility for any bugs introduced

## Please put your Discord username so you can be contacted if a bug or
regression is found:

moleole

src/core/pathfinding/algorithms/AStar.Water.ts

@@ -1,6 +1,6 @@
 import { GameMap, TileRef } from "../../game/GameMap";
 import { PathFinder } from "../types";
-import { BucketQueue, PriorityQueue } from "./PriorityQueue";
+import { MinHeap, PriorityQueue } from "./PriorityQueue";
 
 const LAND_BIT = 7; // Bit 7 in terrain indicates land
 const MAGNITUDE_MASK = 0x1f;
@@ -45,11 +45,7 @@ export class AStarWater implements PathFinder<number> {
     this.gScore = new Uint32Array(this.numNodes);
     this.cameFrom = new Int32Array(this.numNodes);
 
-    // Account for scaled costs + tie-breaker headroom
-    const maxDim = map.width() + map.height();
-    const maxF =
-      (this.heuristicWeight + 1) * BASE_COST * maxDim + COST_SCALE * maxDim;
-    this.queue = new BucketQueue(maxF);
+    this.queue = new MinHeap(this.numNodes);
   }
 
   findPath(start: number | number[], goal: number): number[] | null {

src/core/pathfinding/algorithms/AStar.WaterBounded.ts

@@ -1,6 +1,6 @@
 import { GameMap, TileRef } from "../../game/GameMap";
 import { PathFinder } from "../types";
-import { BucketQueue } from "./PriorityQueue";
+import { MinHeap } from "./PriorityQueue";
 
 const LAND_BIT = 7;
 const MAGNITUDE_MASK = 0x1f;
@@ -33,7 +33,7 @@ export class AStarWaterBounded implements PathFinder<number> {
   private readonly gScoreStamp: Uint32Array;
   private readonly gScore: Uint32Array;
   private readonly cameFrom: Int32Array;
-  private readonly queue: BucketQueue;
+  private readonly queue: MinHeap;
   private readonly terrain: Uint8Array;
   private readonly mapWidth: number;
   private readonly heuristicWeight: number;
@@ -54,11 +54,7 @@ export class AStarWaterBounded implements PathFinder<number> {
     this.gScore = new Uint32Array(maxSearchArea);
     this.cameFrom = new Int32Array(maxSearchArea);
 
-    const maxDim = Math.ceil(Math.sqrt(maxSearchArea));
-    // Account for scaled costs + tie-breaker headroom
-    const maxF =
-      (this.heuristicWeight + 1) * BASE_COST * maxDim * 2 + COST_SCALE * maxDim;
-    this.queue = new BucketQueue(maxF);
+    this.queue = new MinHeap(maxSearchArea * 4);
   }
 
   findPath(start: number | number[], goal: number): number[] | null {
@@ -209,6 +205,8 @@ export class AStarWaterBounded implements PathFinder<number> {
           closedStamp[neighborLocal] !== stamp &&
           (neighbor === goal || (neighborTerrain & landMask) === 0)
         ) {
+          const ny = currentY - 1;
+          const distToGoal = Math.abs(currentX - goalX) + Math.abs(ny - goalY);
           const magnitude = neighborTerrain & MAGNITUDE_MASK;
           const cost = BASE_COST + getMagnitudePenalty(magnitude);
           const tentativeG = currentG + cost;
@@ -219,11 +217,7 @@ export class AStarWaterBounded implements PathFinder<number> {
             cameFrom[neighborLocal] = currentLocal;
             gScore[neighborLocal] = tentativeG;
             gScoreStamp[neighborLocal] = stamp;
-            const ny = currentY - 1;
-            const h =
-              weight *
-              BASE_COST *
-              (Math.abs(currentX - goalX) + Math.abs(ny - goalY));
+            const h = weight * BASE_COST * distToGoal;
             const f = tentativeG + h + crossTieBreaker(currentX, ny);
             queue.push(neighborLocal, f);
           }
@@ -238,6 +232,8 @@ export class AStarWaterBounded implements PathFinder<number> {
           closedStamp[neighborLocal] !== stamp &&
           (neighbor === goal || (neighborTerrain & landMask) === 0)
         ) {
+          const ny = currentY + 1;
+          const distToGoal = Math.abs(currentX - goalX) + Math.abs(ny - goalY);
           const magnitude = neighborTerrain & MAGNITUDE_MASK;
           const cost = BASE_COST + getMagnitudePenalty(magnitude);
           const tentativeG = currentG + cost;
@@ -248,11 +244,7 @@ export class AStarWaterBounded implements PathFinder<number> {
             cameFrom[neighborLocal] = currentLocal;
             gScore[neighborLocal] = tentativeG;
             gScoreStamp[neighborLocal] = stamp;
-            const ny = currentY + 1;
-            const h =
-              weight *
-              BASE_COST *
-              (Math.abs(currentX - goalX) + Math.abs(ny - goalY));
+            const h = weight * BASE_COST * distToGoal;
             const f = tentativeG + h + crossTieBreaker(currentX, ny);
             queue.push(neighborLocal, f);
           }
@@ -267,6 +259,8 @@ export class AStarWaterBounded implements PathFinder<number> {
           closedStamp[neighborLocal] !== stamp &&
           (neighbor === goal || (neighborTerrain & landMask) === 0)
         ) {
+          const nx = currentX - 1;
+          const distToGoal = Math.abs(nx - goalX) + Math.abs(currentY - goalY);
           const magnitude = neighborTerrain & MAGNITUDE_MASK;
           const cost = BASE_COST + getMagnitudePenalty(magnitude);
           const tentativeG = currentG + cost;
@@ -277,11 +271,7 @@ export class AStarWaterBounded implements PathFinder<number> {
             cameFrom[neighborLocal] = currentLocal;
             gScore[neighborLocal] = tentativeG;
             gScoreStamp[neighborLocal] = stamp;
-            const nx = currentX - 1;
-            const h =
-              weight *
-              BASE_COST *
-              (Math.abs(nx - goalX) + Math.abs(currentY - goalY));
+            const h = weight * BASE_COST * distToGoal;
             const f = tentativeG + h + crossTieBreaker(nx, currentY);
             queue.push(neighborLocal, f);
           }
@@ -296,6 +286,8 @@ export class AStarWaterBounded implements PathFinder<number> {
           closedStamp[neighborLocal] !== stamp &&
           (neighbor === goal || (neighborTerrain & landMask) === 0)
         ) {
+          const nx = currentX + 1;
+          const distToGoal = Math.abs(nx - goalX) + Math.abs(currentY - goalY);
           const magnitude = neighborTerrain & MAGNITUDE_MASK;
           const cost = BASE_COST + getMagnitudePenalty(magnitude);
           const tentativeG = currentG + cost;
@@ -306,11 +298,7 @@ export class AStarWaterBounded implements PathFinder<number> {
             cameFrom[neighborLocal] = currentLocal;
             gScore[neighborLocal] = tentativeG;
             gScoreStamp[neighborLocal] = stamp;
-            const nx = currentX + 1;
-            const h =
-              weight *
-              BASE_COST *
-              (Math.abs(nx - goalX) + Math.abs(currentY - goalY));
+            const h = weight * BASE_COST * distToGoal;
             const f = tentativeG + h + crossTieBreaker(nx, currentY);
             queue.push(neighborLocal, f);
           }

src/core/pathfinding/algorithms/AStar.WaterHierarchical.ts

@@ -12,6 +12,7 @@ export class AStarWaterHierarchical implements PathFinder<number> {
   private abstractAStar: AbstractGraphAStar;
   private localAStar: AStarWaterBounded;
   private localAStarMultiCluster: AStarWaterBounded;
+  private localAStarShortPath: AStarWaterBounded;
   private sourceResolver: SourceResolver;
 
   constructor(
@@ -41,6 +42,11 @@ export class AStarWaterHierarchical implements PathFinder<number> {
       maxMultiClusterNodes,
     );
 
+    // BoundedAStar for short path multi-source (120 + 2*10 padding = 140)
+    const shortPathSize = 140;
+    const maxShortPathNodes = shortPathSize * shortPathSize;
+    this.localAStarShortPath = new AStarWaterBounded(map, maxShortPathNodes);
+
     // SourceResolver for multi-source search
     this.sourceResolver = new SourceResolver(this.map, this.graph);
   }
@@ -62,6 +68,10 @@ export class AStarWaterHierarchical implements PathFinder<number> {
     sources: TileRef[],
     target: TileRef,
   ): TileRef[] | null {
+    // Early exit: try bounded A* for sources close to target
+    const shortPath = this.tryShortPathMultiSource(sources, target);
+    if (shortPath) return shortPath;
+
     // 1. Resolve target to abstract node
     const targetNode = this.sourceResolver.resolveTarget(target);
     if (!targetNode) return null;
@@ -82,6 +92,44 @@ export class AStarWaterHierarchical implements PathFinder<number> {
     return this.findPathSingle(winningSource, target);
   }
 
+  private tryShortPathMultiSource(
+    sources: TileRef[],
+    target: TileRef,
+  ): TileRef[] | null {
+    const SHORT_PATH_THRESHOLD = 120;
+    const PADDING = 10;
+
+    const candidates = sources.filter(
+      (s) => this.map.manhattanDist(s, target) <= SHORT_PATH_THRESHOLD,
+    );
+    if (candidates.length === 0) return null;
+
+    const toX = this.map.x(target);
+    const toY = this.map.y(target);
+    let minX = toX,
+      maxX = toX,
+      minY = toY,
+      maxY = toY;
+
+    for (const s of candidates) {
+      const sx = this.map.x(s);
+      const sy = this.map.y(s);
+      minX = Math.min(minX, sx);
+      maxX = Math.max(maxX, sx);
+      minY = Math.min(minY, sy);
+      maxY = Math.max(maxY, sy);
+    }
+
+    const bounds = {
+      minX: Math.max(0, minX - PADDING),
+      maxX: Math.min(this.map.width() - 1, maxX + PADDING),
+      minY: Math.max(0, minY - PADDING),
+      maxY: Math.min(this.map.height() - 1, maxY + PADDING),
+    };
+
+    return this.localAStarShortPath.searchBounded(candidates, target, bounds);
+  }
+
   findPathSingle(from: TileRef, to: TileRef): TileRef[] | null {
     const dist = this.map.manhattanDist(from, to);
 

src/core/pathfinding/algorithms/PriorityQueue.ts

@@ -18,7 +18,20 @@ export class MinHeap implements PriorityQueue {
 
   push(node: number, priority: number): void {
     if (this.size >= this.capacity) {
-      throw new Error(`MinHeap capacity exceeded: ${this.capacity}`);
+      console.error(
+        `MinHeap capacity exceeded (${this.capacity}). ` +
+          "Resizing, but this indicates a bug. Please investigate.",
+      );
+
+      this.capacity *= 2;
+
+      const newHeap = new Int32Array(this.capacity);
+      const newPri = new Float32Array(this.capacity);
+      newHeap.set(this.heap);
+      newPri.set(this.priorities);
+
+      this.heap = newHeap;
+      this.priorities = newPri;
     }
 
     let i = this.size++;
@@ -94,6 +107,8 @@ export class MinHeap implements PriorityQueue {
 export class BucketQueue implements PriorityQueue {
   private buckets: Int32Array[];
   private bucketSizes: Int32Array;
+  private bucketStamp: Uint32Array;
+  private stamp = 0;
   private minBucket: number;
   private maxBucket: number;
   private size: number;
@@ -102,6 +117,7 @@ export class BucketQueue implements PriorityQueue {
     this.maxBucket = maxPriority + 1;
     this.buckets = new Array(this.maxBucket);
     this.bucketSizes = new Int32Array(this.maxBucket);
+    this.bucketStamp = new Uint32Array(this.maxBucket);
     this.minBucket = this.maxBucket;
     this.size = 0;
   }
@@ -113,7 +129,9 @@ export class BucketQueue implements PriorityQueue {
       this.buckets[bucket] = new Int32Array(64);
     }
 
-    const size = this.bucketSizes[bucket];
+    const size =
+      this.bucketStamp[bucket] === this.stamp ? this.bucketSizes[bucket] : 0;
+
     if (size >= this.buckets[bucket].length) {
       const newBucket = new Int32Array(this.buckets[bucket].length * 2);
       newBucket.set(this.buckets[bucket]);
@@ -121,7 +139,8 @@ export class BucketQueue implements PriorityQueue {
     }
 
     this.buckets[bucket][size] = node;
-    this.bucketSizes[bucket]++;
+    this.bucketSizes[bucket] = size + 1;
+    this.bucketStamp[bucket] = this.stamp;
     this.size++;
 
     if (bucket < this.minBucket) {
@@ -131,11 +150,13 @@ export class BucketQueue implements PriorityQueue {
 
   pop(): number {
     while (this.minBucket < this.maxBucket) {
-      const size = this.bucketSizes[this.minBucket];
-      if (size > 0) {
-        this.bucketSizes[this.minBucket]--;
-        this.size--;
-        return this.buckets[this.minBucket][size - 1];
+      if (this.bucketStamp[this.minBucket] === this.stamp) {
+        const size = this.bucketSizes[this.minBucket];
+        if (size > 0) {
+          this.bucketSizes[this.minBucket]--;
+          this.size--;
+          return this.buckets[this.minBucket][size - 1];
+        }
       }
       this.minBucket++;
     }
@@ -147,7 +168,11 @@ export class BucketQueue implements PriorityQueue {
   }
 
   clear(): void {
-    this.bucketSizes.fill(0);
+    this.stamp++;
+    if (this.stamp > 0xffffffff) {
+      this.bucketStamp.fill(0);
+      this.stamp = 1;
+    }
     this.minBucket = this.maxBucket;
     this.size = 0;
   }

src/core/pathfinding/spatial/SpatialQuery.ts

@@ -1,12 +1,27 @@
 import { Game, Player, TerraNullius } from "../../game/Game";
 import { TileRef } from "../../game/GameMap";
+import { DebugSpan } from "../../utilities/DebugSpan";
 import { PathFinding } from "../PathFinder";
+import { AStarWaterBounded } from "../algorithms/AStar.WaterBounded";
 
 type Owner = Player | TerraNullius;
 
+const REFINE_MAX_SEARCH_AREA = 100 * 100;
+
 export class SpatialQuery {
+  private boundedAStar: AStarWaterBounded | null = null;
+
   constructor(private game: Game) {}
 
+  private getBoundedAStar(): AStarWaterBounded {
+    this.boundedAStar ??= new AStarWaterBounded(
+      this.game.map(),
+      REFINE_MAX_SEARCH_AREA,
+    );
+
+    return this.boundedAStar;
+  }
+
   /**
    * Find nearest tile matching predicate using BFS traversal.
    * Uses Manhattan distance filter, ignores terrain barriers.
@@ -64,27 +79,125 @@ export class SpatialQuery {
    * Returns null for terra nullius (no borderTiles).
    */
   closestShoreByWater(owner: Owner, target: TileRef): TileRef | null {
-    if (!owner.isPlayer()) return null;
+    return DebugSpan.wrap("SpatialQuery.closestShoreByWater", () => {
+      if (!owner.isPlayer()) return null;
 
-    const gm = this.game;
-    const player = owner as Player;
+      const gm = this.game;
+      const player = owner as Player;
 
-    // Target must be water or shore (land adjacent to water)
-    if (!gm.isWater(target) && !gm.isShore(target)) return null;
+      // Target must be water or shore (land adjacent to water)
+      if (!gm.isWater(target) && !gm.isShore(target)) return null;
 
-    const targetComponent = gm.getWaterComponent(target);
-    if (targetComponent === null) return null;
+      const targetComponent = gm.getWaterComponent(target);
+      if (targetComponent === null) return null;
 
-    const isValidTile = (t: TileRef) => {
-      if (!gm.isShore(t) || !gm.isLand(t)) return false;
-      const tComponent = gm.getWaterComponent(t);
-      return tComponent === targetComponent;
+      const isValidTile = (t: TileRef) => {
+        if (!gm.isShore(t) || !gm.isLand(t)) return false;
+        const tComponent = gm.getWaterComponent(t);
+        return tComponent === targetComponent;
+      };
+
+      const shores = Array.from(player.borderTiles()).filter(isValidTile);
+      if (shores.length === 0) return null;
+
+      const path = PathFinding.Water(gm).findPath(shores, target);
+      if (!path || path.length === 0) return null;
+
+      return DebugSpan.wrap("SpatialQuery.refineStartTile", () =>
+        this.refineStartTile(path, shores, gm),
+      );
+    });
+  }
+
+  private refineStartTile(
+    path: TileRef[],
+    shores: TileRef[],
+    gm: Game,
+  ): TileRef {
+    const CANDIDATE_RADIUS = 20;
+    const MIN_WAYPOINT_DIST = 50;
+    const MAX_WAYPOINT_DIST = 200;
+    const PADDING = 10;
+
+    if (path.length <= MIN_WAYPOINT_DIST) {
+      return path[0];
+    }
+
+    const bestTile = path[0];
+    const map = gm.map();
+
+    const candidates = shores.filter(
+      (s) => map.manhattanDist(s, bestTile) <= CANDIDATE_RADIUS,
+    );
+
+    if (candidates.length <= 1) return bestTile;
+
+    // Precompute candidate bounds
+    let candMinX = map.x(candidates[0]);
+    let candMaxX = candMinX;
+    let candMinY = map.y(candidates[0]);
+    let candMaxY = candMinY;
+
+    for (let i = 1; i < candidates.length; i++) {
+      const sx = map.x(candidates[i]);
+      const sy = map.y(candidates[i]);
+      candMinX = Math.min(candMinX, sx);
+      candMaxX = Math.max(candMaxX, sx);
+      candMinY = Math.min(candMinY, sy);
+      candMaxY = Math.max(candMaxY, sy);
+    }
+
+    // Binary search for furthest waypoint that keeps bounds within limit
+    let lo = MIN_WAYPOINT_DIST;
+    let hi = Math.min(MAX_WAYPOINT_DIST, path.length - 1);
+    let bestWaypointIdx = lo;
+
+    for (let i = 0; i < 5 && lo <= hi; i++) {
+      const mid = (lo + hi) >> 1;
+      const wp = path[mid];
+      const wpX = map.x(wp);
+      const wpY = map.y(wp);
+
+      const minX = Math.min(candMinX, wpX) - PADDING;
+      const maxX = Math.max(candMaxX, wpX) + PADDING;
+      const minY = Math.min(candMinY, wpY) - PADDING;
+      const maxY = Math.max(candMaxY, wpY) + PADDING;
+
+      const area = (maxX - minX + 1) * (maxY - minY + 1);
+      if (area <= REFINE_MAX_SEARCH_AREA) {
+        bestWaypointIdx = mid;
+        lo = mid + 1;
+      } else {
+        hi = mid - 1;
+      }
+    }
+
+    const waypoint = path[bestWaypointIdx];
+    const wpX = map.x(waypoint);
+    const wpY = map.y(waypoint);
+
+    const bounds = {
+      minX: Math.max(0, Math.min(candMinX, wpX) - PADDING),
+      maxX: Math.min(map.width() - 1, Math.max(candMaxX, wpX) + PADDING),
+      minY: Math.max(0, Math.min(candMinY, wpY) - PADDING),
+      maxY: Math.min(map.height() - 1, Math.max(candMaxY, wpY) + PADDING),
     };
 
-    const shores = Array.from(player.borderTiles()).filter(isValidTile);
-    if (shores.length === 0) return null;
+    const boundsArea =
+      (bounds.maxX - bounds.minX + 1) * (bounds.maxY - bounds.minY + 1);
+    if (boundsArea > REFINE_MAX_SEARCH_AREA) return bestTile;
 
-    const path = PathFinding.Water(gm).findPath(shores, target);
-    return path?.[0] ?? null;
+    const refinedPath = this.getBoundedAStar().searchBounded(
+      candidates,
+      waypoint,
+      bounds,
+    );
+
+    DebugSpan.set("$candidates", () => candidates);
+    DebugSpan.set("$refinedPath", () => refinedPath);
+    DebugSpan.set("$originalBestTile", () => bestTile);
+    DebugSpan.set("$newBestTile", () => refinedPath?.[0] ?? bestTile);
+
+    return refinedPath?.[0] ?? bestTile;
   }
 }

src/core/pathfinding/transformers/SmoothingWaterTransformer.ts

@@ -54,6 +54,9 @@ export class SmoothingWaterTransformer implements PathFinder<TileRef> {
       this.refineEndpoints(smoothed),
     );
 
+    // Pass 3: LOS smoothing again (refinement may create new shortcut opportunities)
+    smoothed = DebugSpan.wrap("smoother:los2", () => this.losSmooth(smoothed));
+
     return smoothed;
   }