diff --git a/docs/replay_design.md b/docs/replay_design.md new file mode 100644 index 0000000..c9432c6 --- /dev/null +++ b/docs/replay_design.md @@ -0,0 +1,408 @@ +# Replay — Design + +This document captures the design for the replay record/playback feature. It records the +decisions made during design discussion; it is a complement to `architecture.md`. No +implementation exists yet — this is the agreed design to implement against. + +## Goal + +Record every play session and allow it to be played back later. Playback is **view-only** +(no interaction) with **manual game-speed selection** (including pause). Playback is launched +via a command-line argument to the executable. + +## Approach: deterministic command-replay (re-simulation) + +We record **player intent** (commands) plus the inputs needed to reproduce the run, and on +playback we **re-run the real simulation**, injecting the recorded commands at their recorded +ticks. We do **not** record per-tick state snapshots. + +This is viable because the simulation is already built for it (see `architecture.md`: +"determinism, replayability ... fall out for free"): + +- Fixed 30 Hz tick-based simulation, decoupled from render rate via `TickDriver`. +- Game speed (0/0.5/1/2/4×) and pause are tick-rate multipliers — they change *how many* + ticks run per frame, never the *outcome* of a tick. So speed, pause, camera scroll, and + selection are pure presentation and are **not recorded**. +- A single deterministic RNG stream: `Simulation::m_rng` (one `std::mt19937`) is passed by + reference into `WaveSystem` and `BuildingSystem`, the only two consumers. ECS combat/AI/ + movement/scrap systems use no RNG. The `utility::getRandom*` global is not used by the sim. +- Config is immutable after load; a replay is pinned to the config it was recorded with. + +A replay run is therefore a pure function of `(seed, config, ordered commands)`. + +### What we do NOT do (now) + +- No per-tick / keyframe state snapshots. +- No backward seek / scrubbing (would require snapshots). +- No save/load. (See "Future direction".) +- No interactive playback (no taking over a replay mid-run). + +## Replay commands + +A *replay command* is the resolved, serializable **intent** behind a player action — the +data, not the UI gesture. Example: placing a miner records +`PlaceBuilding{type=Miner, anchor=(3,5), rotation=East}`, not the mouse pixel that produced it. + +- Commands are at **intent level, resolved to tile coordinates / domain ids** — independent + of window size, camera scroll, and DPI. +- Command payloads reference **stable, deterministic domain ids** (`BuildingId`, tile + coordinates, choice indices) — never raw `entt::entity` handles. These ids are sim-allocated + deterministically, so a recorded command resolves to the same entity on replay. +- Camera scroll, selection, game speed, and pause are **not** commands. + +### Command vocabulary + +One command per sim-mutating operation (the complete mutation surface): + +- `PlaceBuilding` +- `Demolish` +- `RotateInPlace` +- `SetRecipe` +- `SetShipLayout` +- `SetSplitterFilters` (building-site and belt variants) +- `ClearBeltTiles` +- `ApplySchematicChoice` +- `Reset` / restart — see "Restart is a boundary". + +### Command representation + +Commands use a **base class + derived classes** (mirroring the existing `Event` hierarchy +idiom, so it is native to this codebase). They are routed through a dedicated command path, +**not** through `EventManager` (see next section). + +## Command system: reuse the *pattern*, not the EventManager singleton + +We reuse the **pattern** of the existing event system (a polymorphic base + small derived +types), but the sim-mutating command path is a **dedicated, ordered queue**, not the +`EventManager` pub/sub bus. Reasons: + +1. **Determinism / ordering.** Sim mutations must apply in a strict, tick-pinned, recorded + order. `architecture.md` deliberately keeps the sim free of `EventManager` for exactly this + reason (determinism, tick-order fidelity, headless testability — why `BeamFiredEvent` uses a + plain vector). Routing commands into the sim via the singleton would break that. +2. **Single consumer.** A command has exactly one recipient (the `Simulation`); pub/sub + N-handler fan-out is the wrong shape. +3. **Recording chokepoint.** One place must see every command, stamp its tick, append it to the + file, and apply it. A direct queue gives that; a multi-handler bus does not. +4. **Headless tests.** Tests link only `lib` and build a `Simulation` directly; the command + type and apply path live in `lib` and must work with no UI and no singleton. + +### Structure + +- **In `lib`:** a `Command` base class + derived command types, plus a `CommandManager` + (ordered queue) and a single `Simulation::apply(command)` chokepoint. +- **UI fan-in still uses `EventManager`:** widgets emit a UI event as today; a single + dispatcher/recorder catches it, builds the `lib` command, and hands it to the + `CommandManager`. This keeps widgets decoupled (consistent with current architecture). +- **Replay** skips the UI half and feeds commands straight into the same `CommandManager` / + `Simulation::apply` chokepoint. + +### The completeness invariant (enforced structurally) + +**Every** sim mutation must flow through the single `CommandManager → Simulation::apply` +chokepoint. Any path that mutates the sim directly would not be recorded and would silently +desync the replay. + +This is enforced **structurally, not by discipline**: the `Simulation` mutators +(`tryPlaceBuilding`, `demolish`, `setRecipe`, …) are made private/non-public so the only way to +reach them is `apply(command)`. A stray direct call then fails to compile rather than compiling +and silently desyncing. + +Recording happens **at the apply chokepoint**, not at the UI gesture — so only commands that +actually reached the sim are recorded, and they replay through the identical apply path. +UI-side validation (placement validity, affordability) remains a pre-filter that simply does +not produce a command unless the action reaches the sim. + +## Command timing: drain once per frame, before the tick batch + +- During live play, input pushes commands onto the `CommandManager` queue (not applied + synchronously). +- The queue is drained at **one defined point: once per frame, before stepping the tick + batch.** The whole queue is drained in FIFO order (not one-per-tick), so bursts (e.g. laying + many belts quickly) apply immediately instead of dribbling across ticks, and it matches the + lockstep model wanted later. +- Each drained command is **tagged with the current completed-tick count**, recorded at drain + time (so record-order == apply-order canonically), and applied. + +### Build-while-paused is preserved + +The drain runs every frame including at 0× (the tick batch is simply empty when paused). So a +player can place buildings while paused and **see the construction sites immediately**. This is +still fully deterministic: replay applies each command at its recorded tick regardless of the +frame cadence that produced it. + +On replay, there is no input; the player applies each pre-filled command at its recorded tick +through the same drain path, preserving order. + +The Qt single-threaded event loop guarantees input events and the `onFrame` tick-batch never +interleave, so the completed-tick count at drain time is unambiguous. (If the sim is ever moved +to a worker thread, this needs a lock at the sim boundary.) + +## Determinism: checksums and verification + +We do not verify EnTT iteration order statically. EnTT view iteration is a pure function of the +sequence of spawn/destroy/add/remove operations, so on a fixed binary it contributes zero +run-to-run nondeterminism. Instead we verify **end-to-end determinism** with a state checksum, +and any divergence (EnTT order, float, container ordering, etc.) surfaces loudly. + +### What is checksummed (now) + +- **RNG state only**, for now. The `mt19937` state is fingerprinted into a 64-bit value. +- The hash can be extended later (entity positions/HP, belt items, building buffers, scalars) + without changing the format. + +### Cadence + +- **In the replay file:** every **30 ticks**, **and** after **every command** is applied. The + per-command checksum pins any divergence to the action that triggered it; the periodic one + localizes drift to a ~1 s window. On playback the recomputed checksum is compared; a mismatch + reports "desync at tick N". +- **In tests:** the Catch2 **double-run determinism test** hashes **full sim state every tick** + (not just RNG). It runs a scripted command sequence twice from the same seed and asserts + per-tick checksums match. This keeps the file lean while still catching non-RNG determinism + bugs during development. + +### Known limitation of the RNG-only file checksum (accepted) + +An RNG-only checksum only catches divergences that change **how much randomness is consumed** +(wave composition, recipe rolls, scrap). Float or iteration drift that does **not** alter RNG +draw counts passes the checksum undetected. This is acceptable for same-binary Windows replay +(no float drift expected on an identical binary; the checksum's real job there is catching +determinism *bugs*). When cross-platform replay becomes a goal, the **file** hash must be +expanded to include entity state. + +## Cross-platform: Windows-first, portable by construction + +The replay file is platform-neutral data; `std::mt19937` is bit-identical across platforms, so +RNG is not a cross-platform problem. The only real cross-platform issue is **floating-point +reproducibility** — the sim does heavy `QVector2D` float math, and a 1-ULP difference (compiler +/ CPU / SIMD / FMA contraction) can flip an in-range comparison and cascade into different ship +behavior (the classic lockstep-RTS problem). + +Decision: **Windows-only first**, but make the later swap cheap and bounded by, from day one: + +- a **per-period state checksum** in the file (above), and +- a **build/version + config-hash identity tag** in the header. + +Then cross-platform later is a contained float-hardening pass (`/fp:strict`, no FMA contraction, +possibly fixed-point positions) guided by the checksums — **not** a redesign of the +command-replay architecture. + +Note: even a new Windows *build* of the game can desync old replays for the same float reasons, +so the version tag + "warn on mismatch" is needed regardless of cross-platform ambitions. + +## Seed and config + +- **Seed:** a **random** seed is generated at the start of each run, **outside** the sim (e.g. + `std::random_device` in `main`/reset), so the `Simulation` stays a pure function of + `(seed, config, commands)`. The seed is written to the replay header. +- **Config:** the header stores a **config hash** (not a full config snapshot). On playback the + current config is hashed and compared; a mismatch warns/refuses. The hash is taken over the + actually-loaded config (so editing config files and restarting yields a new, consistent + replay). + +## File format: line-oriented append-friendly text + +Non-binary, chosen for readability and crash-safety. Size is a non-issue: the command log is +sparse (only ticks with a player action), so even a multi-hour game is tens of KB in any text +format. + +- A small keyed/header section: seed, config hash, build/version, start timestamp. +- One line per command, e.g. `1234 place miner 3 5 E`. +- Periodic checksum lines interleaved, e.g. `# checksum 9000 a1b2c3...`. + +Why line-oriented text: + +- **Append-friendly** — the recorder stream-appends as the game runs, so a crash does not lose + the replay (a crash is exactly when you would want it). A format that must be rewritten/closed + as a whole is rejected for this reason. +- **No new dependency** — the project has no JSON lib; toml++ is parse-oriented and clunky for a + long event stream (fine for the header, awkward as an array-of-tables of thousands of + entries). +- Greppable, diffable, tiny. +- Aligns with the project's existing text-serialization idiom (`BlueprintSerializer`, + `ShipLayoutBlueprintSerializer`). + +## Recording lifecycle + +- **Record every run.** A new replay file is created at `Simulation` construction and at each + `reset()`. +- **Restart is a boundary.** Restart (escape menu → restart, which reloads config and resets) + closes the current file and opens a new one with a fresh seed and header. One replay file = + one contiguous run from tick 0 to game-over/quit. +- **Retention: keep everything.** Files live in the existing `data/` directory, named by + timestamp + seed. (No automatic pruning for now.) + +## Playback + +Launched via a command-line argument, e.g. `DotaFactory.exe --replay `. + +`main` for the `--replay` path: + +1. Read the header → validate config hash and build/version (warn on mismatch). +2. Construct the `Simulation` from the recorded seed + config. +3. Construct the `CommandManager` in **Replay mode**, **pre-filled** with the whole command list + from the file. (Pre-fill memory is trivial; streaming-read is a later optimization if files + ever get huge — not needed now.) +4. Run the driver in replay mode: each frame, drain commands due at the reached tick (same drain + path as live), step ticks, compare checksums. + +### Replay mode rules + +Reframe: the schematic-choice modal is **an input source** (the device that produces an +`ApplySchematicChoice` command in live play), exactly like the mouse. Replay's single rule is +"**disable live input sources**", which the modal falls under. + +- **`CommandManager` in replay mode:** `addCommand` is a no-op; the queue is pre-filled from the + file. Live input therefore produces nothing. +- **Only two reactions need explicit gating** — the sim-state *polls* in `onFrame` that emit + `SchematicChoicesAvailableEvent` and `GameOverEvent`. In replay these polls do not run, so no + modal opens, no auto-pause occurs, and there is no deadlock against the recorded command. +- **Everything else falls away for free** because it is click-driven, not sim-state-driven: the + recipe dialog (`RecipeSelectionRequestedEvent`), ship-layout dialog + (`LayoutDialogRequestedEvent`), and escape menu are all triggered by player input, which is + disabled — so they never open and need no special handling. +- **Schematic choice still resolves with no UI:** the sim regenerates identical choices + deterministically (same seed + prior commands), and the pre-filled `ApplySchematicChoice` + applies itself at its recorded tick through the normal drain path. The tick-tag invariant + places it correctly relative to when the choices became pending, in both record and replay. +- **Game-over is replaced, not just suppressed:** instead of the live restart/quit dialog, + playback detects the end condition (command stream exhausted / recorded game-over reached) and + stops, showing a passive "replay ended" state. +- **Kept in replay:** the renderer/view and **manual game-speed selection** (including pause / + 0× and fast-forward via high speed). Playback only ever moves forward. + +## Future direction (informs the design, not built now) + +Save/load and (deterministic lockstep) multiplayer are wanted later. The command bus is the +shared foundation; two cheap shaping decisions now keep that path open: + +1. **Each command carries a source/player id** (always "player 0" in single-player). Lockstep + multiplayer is just commands from multiple sources merged into one ordered stream. +2. **Commands are applied at a defined tick boundary** (already required for replay). Multiplayer + schedules them a few ticks in the future to hide latency; single-player uses the next drain. + +Implications to note: + +- Multiplayer makes cross-platform float determinism mandatory and promotes the checksum to + load-bearing desync-detection (rather than a test aid) — reinforcing doing the checksum now. +- **Save/load** is the one feature that needs a *different* mechanism: either "replay to current + tick" on load (reuses 100% of replay machinery; load time grows with game length, though + fast-forward usually replays hours in seconds), or a full **state-snapshot serializer** + (EnTT registry + belts + buildings + scalars). The snapshot serializer is also what + backward-seek/scrubbing would need. Building the command bus now does not block adding it + later; it is explicitly out of scope here. + +## Summary of decisions + +- Approach: **A — deterministic command-replay** (re-simulation), no snapshots. +- Scope: **view-only** playback + **manual speed selection**; launched via CLI argument. +- Commands: **base class + derived types**, routed through a dedicated `CommandManager` queue + and a single `Simulation::apply` chokepoint; sim mutators made non-public to **enforce** the + chokepoint. UI fan-in still uses `EventManager`. +- Timing: queue **drained once per frame before the tick batch**, whole queue FIFO, each command + tick-tagged; **build-while-paused preserved**. +- Determinism: **RNG-state checksum** in the file every **30 ticks + after each command**; + **full-state per-tick hashing** in the Catch2 double-run test. Known RNG-only blind spot + accepted for now. +- Platform: **Windows-first**; file format + version/config-hash make a later cross-platform + pass contained. +- Seed: **random**, generated outside the sim, written to the header. +- Config: **config hash** in the header, validated on playback. +- File: **line-oriented append-friendly text**, kept in `data/`, **one file per run**, + **retain everything**. +- Restart: **a boundary** — new file, new seed. +- Replay mode: `CommandManager` `addCommand` is a no-op + pre-filled; gate the two sim-state + polls (schematic choices, game-over); passive "replay ended" instead of the game-over dialog; + keep view + speed. + +## Implementation plan + +Ordered to de-risk: prove determinism first, then build the command path, then recording, then +playback. Each phase is independently testable and leaves the game in a working state. Phases +0 → 1 → 2 → 3 are strictly sequential; Phase 4 tests can start as soon as their subject exists. + +### Phase 0 — Determinism foundation & verification (no replay yet) + +The whole feature rests on a deterministic sim, so prove that before building on it. + +- Add a `mt19937` state **fingerprint** (fold its serialized state into a 64-bit value). +- Add a **full-state checksum** path (positions, HP, velocities, belt items, building buffers, + scalars), used by tests; each subsystem contributes via its own `appendChecksum(Hasher&)` so + no state knowledge is duplicated. +- Add a Catch2 **double-run determinism test**: run a scripted sequence twice from the same + seed, assert per-tick **full-state** checksums match. +- **Files:** new `lib/sim` checksum helper; small additions to `Simulation`, `BeltSystem`, + `BuildingSystem`, ECS state; new test. +- **Exit criteria:** the double-run test passes. If it fails, fix the nondeterminism here before + proceeding. + +### Phase 1 — Command model + enforced chokepoint (no recording yet) + +Reshape mutations to flow through one path; behaviour unchanged. + +- Define `Command` base + derived types (`PlaceBuilding`, `Demolish`, `RotateInPlace`, + `SetRecipe`, `SetShipLayout`, `SetSplitterFilters`, `ClearBeltTiles`, `ApplySchematicChoice`, + `Reset`) in `lib`. Each carries a source/player id (always 0 now) for the future-multiplayer + shape. +- Add `CommandManager` (FIFO queue, drain) in `lib`, holding a `Simulation&`. +- Add `Simulation::apply(const Command&)` dispatching to the existing mutators; then **make + those mutators non-public** so `apply` is the only entry — compile-enforces the completeness + invariant. +- Wire the drain: in `GameWorldView::onFrame`, call `CommandManager::drain()` **once per frame, + before** the tick batch; tag each command with the current completed-tick count. +- Refactor every UI mutation site to **emit a single `CommandRequestedEvent`** (carrying a + `shared_ptr`) via the existing `EventManager`; one dispatcher subscribes and enqueues + onto `CommandManager`. +- **Files:** new `lib` command + `CommandManager`; `Simulation.h/.cpp`; call sites in + `GameWorldView.cpp`, `MainWindow.cpp`, `SelectedBuildingPanel.cpp`; new dispatcher in + `ui`/`app`. +- **Exit criteria:** game plays identically (including build-while-paused), determinism test + still passes, no widget can call a sim mutator directly (won't compile). + +### Phase 2 — Recording + +- Implement the **line-oriented append writer**: header (seed, config hash, build/version, + timestamp) + one line per command + checksum lines. +- Generate a **random seed outside the sim** (in `main`/reset), write to header. +- Compute the **config hash** over the loaded config. +- Hook the **recorder at the apply chokepoint** in `CommandManager::drain()`: append each command + (tick-tagged), append an **RNG checksum after every command** and **every 30 ticks**. +- **Lifecycle:** open a new file on `Simulation` construction and on each `reset()` (restart = + boundary); store in `data/`, named by timestamp+seed; retain everything. +- **Files:** new replay writer in `lib`; `main.cpp` (seed), config hash helper; + `CommandManager`/`Simulation` for the tick checksum hook. +- **Exit criteria:** every run produces a well-formed, growing replay file; a crash mid-run still + leaves a valid partial file. + +### Phase 3 — Playback + +- Implement the **reader/parser** for the format (header + commands + checksums). +- Add the `--replay ` CLI path in `main`: validate config hash + version (warn on + mismatch), construct `Simulation` from seed+config, construct `CommandManager` in **replay + mode** (pre-filled, `addCommand` is a no-op). +- Replay driver: each frame, drain commands due at the reached tick (same drain path), step + ticks, **keep manual speed/pause**, playback only moves forward. +- **Gate the two `onFrame` polls** (schematic-choices, game-over) off in replay mode; everything + else (recipe/layout dialogs, escape menu) is input-driven and falls away automatically. +- **Desync detection:** recompute the RNG checksum at each checkpoint, compare to the file, + report "desync at tick N" on mismatch. +- **Passive end:** when the command stream is exhausted / recorded game-over is reached, stop + with a "replay ended" overlay instead of the restart dialog. +- **Files:** new reader in `lib`; `main.cpp`; `CommandManager` (replay mode); `GameWorldView.cpp` + (poll gating, end overlay). +- **Exit criteria:** a recorded file plays back identically; checksums match throughout. + +### Phase 4 — Closing tests & polish + +- **Round-trip test:** serialize → parse → assert command equality. +- **Replay-equivalence test (headless):** record a scripted run, play it back through the same + `lib` path, assert per-tick checksums match end-to-end — the real proof, no UI needed. +- Mismatch-warning UX, end-of-replay overlay polish. + +### Notes + +- Phase 1 is the largest (the mutation-site refactor); Phase 0 is the riskiest (it may surface + latent nondeterminism that must be fixed first). +- Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float + hardening, expanding the file checksum beyond RNG.