dota_factory/docs/replay_design.md

# Replay — Design

This document captures the design for the replay record/playback feature. It records the
decisions made during design discussion; it is a complement to `architecture.md`. No
implementation exists yet — this is the agreed design to implement against.

## Goal

Record every play session and allow it to be played back later. Playback is **view-only**
(no interaction) with **manual game-speed selection** (including pause). Playback is launched
via a command-line argument to the executable.

## Approach: deterministic command-replay (re-simulation)

We record **player intent** (commands) plus the inputs needed to reproduce the run, and on
playback we **re-run the real simulation**, injecting the recorded commands at their recorded
ticks. We do **not** record per-tick state snapshots.

This is viable because the simulation is already built for it (see `architecture.md`:
"determinism, replayability ... fall out for free"):

- Fixed 30 Hz tick-based simulation, decoupled from render rate via `TickDriver`.
- Game speed (0/0.5/1/2/4×) and pause are tick-rate multipliers — they change *how many*
  ticks run per frame, never the *outcome* of a tick. So speed, pause, camera scroll, and
  selection are pure presentation and are **not recorded**.
- A single deterministic RNG stream: `Simulation::m_rng` (one `std::mt19937`) is passed by
  reference into `WaveSystem` and `BuildingSystem`, the only two consumers. ECS combat/AI/
  movement/scrap systems use no RNG. The `utility::getRandom*` global is not used by the sim.
- Config is immutable after load; a replay is pinned to the config it was recorded with.

A replay run is therefore a pure function of `(seed, config, ordered commands)`.

### What we do NOT do (now)

- No per-tick / keyframe state snapshots.
- No backward seek / scrubbing (would require snapshots).
- No save/load. (See "Future direction".)
- No interactive playback (no taking over a replay mid-run).

## Replay commands

A *replay command* is the resolved, serializable **intent** behind a player action — the
data, not the UI gesture. Example: placing a miner records
`PlaceBuilding{type=Miner, anchor=(3,5), rotation=East}`, not the mouse pixel that produced it.

- Commands are at **intent level, resolved to tile coordinates / domain ids** — independent
  of window size, camera scroll, and DPI.
- Command payloads reference **stable, deterministic domain ids** (`BuildingId`, tile
  coordinates, choice indices) — never raw `entt::entity` handles. These ids are sim-allocated
  deterministically, so a recorded command resolves to the same entity on replay.
- Camera scroll, selection, game speed, and pause are **not** commands.

### Command vocabulary

One command per sim-mutating operation (the complete mutation surface):

- `PlaceBuilding`
- `Demolish`
- `RotateInPlace`
- `SetRecipe`
- `SetShipLayout`
- `SetSplitterFilters` (building-site and belt variants)
- `ClearBeltTiles`
- `ApplySchematicChoice`
- `Reset` / restart — see "Restart is a boundary".

### Command representation

Commands use a **base class + derived classes** (mirroring the existing `Event` hierarchy
idiom, so it is native to this codebase). They are routed through a dedicated command path,
**not** through `EventManager` (see next section).

> **Implementation refinement (Phase 1).** `PlaceBuilding` is **atomic**: it carries the
> optional recipe / ship-layout / splitter-filters to configure the new building in the same
> command. This is forced by the deferred-drain timing — commands apply at a later tick
> boundary, so the caller never sees the new `BuildingId` and therefore cannot issue a
> follow-up `SetRecipe`/`SetShipLayout` against it. The standalone `SetRecipe`,
> `SetShipLayout`, and the two `SetSplitterFilters` commands remain for the dialog-driven
> edits on *existing* buildings (which reference a known id). `Reset` carries the (move-only)
> `GameConfig` via `shared_ptr` and is moved into the sim on apply; a null config means "keep
> current config".

## Command system: reuse the *pattern*, not the EventManager singleton

We reuse the **pattern** of the existing event system (a polymorphic base + small derived
types), but the sim-mutating command path is a **dedicated, ordered queue**, not the
`EventManager` pub/sub bus. Reasons:

1. **Determinism / ordering.** Sim mutations must apply in a strict, tick-pinned, recorded
   order. `architecture.md` deliberately keeps the sim free of `EventManager` for exactly this
   reason (determinism, tick-order fidelity, headless testability — why `BeamFiredEvent` uses a
   plain vector). Routing commands into the sim via the singleton would break that.
2. **Single consumer.** A command has exactly one recipient (the `Simulation`); pub/sub
   N-handler fan-out is the wrong shape.
3. **Recording chokepoint.** One place must see every command, stamp its tick, append it to the
   file, and apply it. A direct queue gives that; a multi-handler bus does not.
4. **Headless tests.** Tests link only `lib` and build a `Simulation` directly; the command
   type and apply path live in `lib` and must work with no UI and no singleton.

### Structure

- **In `lib`:** a `Command` base class + derived command types, plus a `CommandManager`
  (ordered queue) and a single `Simulation::apply(command)` chokepoint.
- **UI fan-in still uses `EventManager`:** widgets emit a UI event as today; a single
  dispatcher/recorder catches it, builds the `lib` command, and hands it to the
  `CommandManager`. This keeps widgets decoupled (consistent with current architecture).
- **Replay** skips the UI half and feeds commands straight into the same `CommandManager` /
  `Simulation::apply` chokepoint.

### The completeness invariant (enforced structurally)

**Every** sim mutation must flow through the single `CommandManager → Simulation::apply`
chokepoint. Any path that mutates the sim directly would not be recorded and would silently
desync the replay.

This was originally intended to be enforced **structurally** (make the `Simulation` mutators
non-public so the only way to reach them is `apply(command)`). 

> **Implementation decision (Phase 1).** The structural-enforcement plan was **dropped in
> favour of convention**, because the test suite legitimately drives the same mutators
> directly (`sim.tryPlaceBuilding(...)` and its returned id, `buildings().setRecipe(...)`,
> `applySchematicChoice`, `reset`, `placeImmediate`, …) and relies on their return values —
> making them non-public would break ~30 test call sites, and `apply()` cannot hand a new
> `BuildingId` back to a caller. So the mutators stay **public**; the rule "every UI mutation
> goes through a command" is upheld by convention and a documented chokepoint comment on
> `Simulation::apply`. A `[command]` Catch2 suite asserts `apply(...)` produces byte-identical
> state to the direct mutator path, guarding the equivalence the replay relies on. Tests are
> not gameplay (they never record), so direct mutator use there does not affect replay
> correctness.

Recording happens **at the apply chokepoint**, not at the UI gesture — so only commands that
actually reached the sim are recorded, and they replay through the identical apply path.
UI-side validation (placement validity, affordability) remains a pre-filter that simply does
not produce a command unless the action reaches the sim.

## Command timing: drain once per frame, before the tick batch

- During live play, input pushes commands onto the `CommandManager` queue (not applied
  synchronously).
- The queue is drained at **one defined point: once per frame, before stepping the tick
  batch.** The whole queue is drained in FIFO order (not one-per-tick), so bursts (e.g. laying
  many belts quickly) apply immediately instead of dribbling across ticks, and it matches the
  lockstep model wanted later.
- Each drained command is **tagged with the current completed-tick count**, recorded at drain
  time (so record-order == apply-order canonically), and applied.

### Build-while-paused is preserved

The drain runs every frame including at 0× (the tick batch is simply empty when paused). So a
player can place buildings while paused and **see the construction sites immediately**. This is
still fully deterministic: replay applies each command at its recorded tick regardless of the
frame cadence that produced it.

On replay, there is no input; the player applies each pre-filled command at its recorded tick
through the same drain path, preserving order.

The Qt single-threaded event loop guarantees input events and the `onFrame` tick-batch never
interleave, so the completed-tick count at drain time is unambiguous. (If the sim is ever moved
to a worker thread, this needs a lock at the sim boundary.)

## Determinism: checksums and verification

We do not verify EnTT iteration order statically. EnTT view iteration is a pure function of the
sequence of spawn/destroy/add/remove operations, so on a fixed binary it contributes zero
run-to-run nondeterminism. Instead we verify **end-to-end determinism** with a state checksum,
and any divergence (EnTT order, float, container ordering, etc.) surfaces loudly.

### What is checksummed (now)

- **RNG state only**, for now. The `mt19937` state is fingerprinted into a 64-bit value.
- The hash can be extended later (entity positions/HP, belt items, building buffers, scalars)
  without changing the format.

### Cadence

- **In the replay file:** every **30 ticks**, **and** after **every command** is applied. The
  per-command checksum pins any divergence to the action that triggered it; the periodic one
  localizes drift to a ~1 s window. On playback the recomputed checksum is compared; a mismatch
  reports "desync at tick N".
- **In tests:** the Catch2 **double-run determinism test** hashes **full sim state every tick**
  (not just RNG). It runs a scripted command sequence twice from the same seed and asserts
  per-tick checksums match. This keeps the file lean while still catching non-RNG determinism
  bugs during development.

### Known limitation of the RNG-only file checksum (accepted)

An RNG-only checksum only catches divergences that change **how much randomness is consumed**
(wave composition, recipe rolls, scrap). Float or iteration drift that does **not** alter RNG
draw counts passes the checksum undetected. This is acceptable for same-binary Windows replay
(no float drift expected on an identical binary; the checksum's real job there is catching
determinism *bugs*). When cross-platform replay becomes a goal, the **file** hash must be
expanded to include entity state.

## Cross-platform: Windows-first, portable by construction

The replay file is platform-neutral data; `std::mt19937` is bit-identical across platforms, so
RNG is not a cross-platform problem. The only real cross-platform issue is **floating-point
reproducibility** — the sim does heavy `QVector2D` float math, and a 1-ULP difference (compiler
/ CPU / SIMD / FMA contraction) can flip an in-range comparison and cascade into different ship
behavior (the classic lockstep-RTS problem).

Decision: **Windows-only first**, but make the later swap cheap and bounded by, from day one:

- a **per-period state checksum** in the file (above), and
- a **build/version + config-hash identity tag** in the header.

Then cross-platform later is a contained float-hardening pass (`/fp:strict`, no FMA contraction,
possibly fixed-point positions) guided by the checksums — **not** a redesign of the
command-replay architecture.

Note: even a new Windows *build* of the game can desync old replays for the same float reasons,
so the version tag + "warn on mismatch" is needed regardless of cross-platform ambitions.

## Seed and config

- **Seed:** a **random** seed is generated at the start of each run, **outside** the sim (e.g.
  `std::random_device` in `main`/reset), so the `Simulation` stays a pure function of
  `(seed, config, commands)`. The seed is written to the replay header.
- **Config:** the header stores a **config hash** (not a full config snapshot). On playback the
  current config is hashed and compared; a mismatch warns/refuses. The hash is taken over the
  actually-loaded config (so editing config files and restarting yields a new, consistent
  replay).

## File format: line-oriented append-friendly text

Non-binary, chosen for readability and crash-safety. Size is a non-issue: the command log is
sparse (only ticks with a player action), so even a multi-hour game is tens of KB in any text
format.

- A small keyed/header section: seed, config hash, build/version, start timestamp.
- One line per command, e.g. `1234 place miner 3 5 E`.
- Periodic checksum lines interleaved, e.g. `# checksum 9000 a1b2c3...`.

Why line-oriented text:

- **Append-friendly** — the recorder stream-appends as the game runs, so a crash does not lose
  the replay (a crash is exactly when you would want it). A format that must be rewritten/closed
  as a whole is rejected for this reason.
- **No new dependency** — the project has no JSON lib; toml++ is parse-oriented and clunky for a
  long event stream (fine for the header, awkward as an array-of-tables of thousands of
  entries).
- Greppable, diffable, tiny.
- Aligns with the project's existing text-serialization idiom (`BlueprintSerializer`,
  `ShipLayoutBlueprintSerializer`).

## Recording lifecycle

- **Record every run.** A new replay file is created at `Simulation` construction and at each
  `reset()`.
- **Restart is a boundary.** Restart (escape menu → restart, which reloads config and resets)
  closes the current file and opens a new one with a fresh seed and header. One replay file =
  one contiguous run from tick 0 to game-over/quit.
- **Retention: keep everything.** Files live in the existing `data/` directory, named by
  timestamp + seed. (No automatic pruning for now.)

## Playback

Launched via a command-line argument, e.g. `DotaFactory.exe --replay <file>`.

`main` for the `--replay` path:

1. Read the header → validate config hash and build/version (warn on mismatch).
2. Construct the `Simulation` from the recorded seed + config.
3. Construct the `CommandManager` in **Replay mode**, **pre-filled** with the whole command list
   from the file. (Pre-fill memory is trivial; streaming-read is a later optimization if files
   ever get huge — not needed now.)
4. Run the driver in replay mode: each frame, drain commands due at the reached tick (same drain
   path as live), step ticks, compare checksums.

### Replay mode rules

Reframe: the schematic-choice modal is **an input source** (the device that produces an
`ApplySchematicChoice` command in live play), exactly like the mouse. Replay's single rule is
"**disable live input sources**", which the modal falls under.

- **`CommandManager` in replay mode:** `addCommand` is a no-op; the queue is pre-filled from the
  file. Live input therefore produces nothing.
- **Only two reactions need explicit gating** — the sim-state *polls* in `onFrame` that emit
  `SchematicChoicesAvailableEvent` and `GameOverEvent`. In replay these polls do not run, so no
  modal opens, no auto-pause occurs, and there is no deadlock against the recorded command.
- **Everything else falls away for free** because it is click-driven, not sim-state-driven: the
  recipe dialog (`RecipeSelectionRequestedEvent`), ship-layout dialog
  (`LayoutDialogRequestedEvent`), and escape menu are all triggered by player input, which is
  disabled — so they never open and need no special handling.
- **Schematic choice still resolves with no UI:** the sim regenerates identical choices
  deterministically (same seed + prior commands), and the pre-filled `ApplySchematicChoice`
  applies itself at its recorded tick through the normal drain path. The tick-tag invariant
  places it correctly relative to when the choices became pending, in both record and replay.
- **Game-over is replaced, not just suppressed:** instead of the live restart/quit dialog,
  playback detects the end condition (command stream exhausted / recorded game-over reached) and
  stops, showing a passive "replay ended" state.
- **Kept in replay:** the renderer/view and **manual game-speed selection** (including pause /
  0× and fast-forward via high speed). Playback only ever moves forward.

## Future direction (informs the design, not built now)

Save/load and (deterministic lockstep) multiplayer are wanted later. The command bus is the
shared foundation; two cheap shaping decisions now keep that path open:

1. **Each command carries a source/player id** (always "player 0" in single-player). Lockstep
   multiplayer is just commands from multiple sources merged into one ordered stream.
2. **Commands are applied at a defined tick boundary** (already required for replay). Multiplayer
   schedules them a few ticks in the future to hide latency; single-player uses the next drain.

Implications to note:

- Multiplayer makes cross-platform float determinism mandatory and promotes the checksum to
  load-bearing desync-detection (rather than a test aid) — reinforcing doing the checksum now.
- **Save/load** is the one feature that needs a *different* mechanism: either "replay to current
  tick" on load (reuses 100% of replay machinery; load time grows with game length, though
  fast-forward usually replays hours in seconds), or a full **state-snapshot serializer**
  (EnTT registry + belts + buildings + scalars). The snapshot serializer is also what
  backward-seek/scrubbing would need. Building the command bus now does not block adding it
  later; it is explicitly out of scope here.

## Summary of decisions

- Approach: **A — deterministic command-replay** (re-simulation), no snapshots.
- Scope: **view-only** playback + **manual speed selection**; launched via CLI argument.
- Commands: **base class + derived types**, routed through a dedicated `CommandManager` queue
  and a single `Simulation::apply` chokepoint; sim mutators made non-public to **enforce** the
  chokepoint. UI fan-in still uses `EventManager`.
- Timing: queue **drained once per frame before the tick batch**, whole queue FIFO, each command
  tick-tagged; **build-while-paused preserved**.
- Determinism: **RNG-state checksum** in the file every **30 ticks + after each command**;
  **full-state per-tick hashing** in the Catch2 double-run test. Known RNG-only blind spot
  accepted for now.
- Platform: **Windows-first**; file format + version/config-hash make a later cross-platform
  pass contained.
- Seed: **random**, generated outside the sim, written to the header.
- Config: **config hash** in the header, validated on playback.
- File: **line-oriented append-friendly text**, kept in `data/`, **one file per run**,
  **retain everything**.
- Restart: **a boundary** — new file, new seed.
- Replay mode: `CommandManager` `addCommand` is a no-op + pre-filled; gate the two sim-state
  polls (schematic choices, game-over); passive "replay ended" instead of the game-over dialog;
  keep view + speed.

## Implementation plan

Ordered to de-risk: prove determinism first, then build the command path, then recording, then
playback. Each phase is independently testable and leaves the game in a working state. Phases
0 → 1 → 2 → 3 are strictly sequential; Phase 4 tests can start as soon as their subject exists.

### Phase 0 — Determinism foundation & verification (no replay yet)

The whole feature rests on a deterministic sim, so prove that before building on it.

- Add a `mt19937` state **fingerprint** (fold its serialized state into a 64-bit value).
- Add a **full-state checksum** path (positions, HP, velocities, belt items, building buffers,
  scalars), used by tests; each subsystem contributes via its own `appendChecksum(Hasher&)` so
  no state knowledge is duplicated.
- Add a Catch2 **double-run determinism test**: run a scripted sequence twice from the same
  seed, assert per-tick **full-state** checksums match.
- **Files:** new `lib/sim` checksum helper; small additions to `Simulation`, `BeltSystem`,
  `BuildingSystem`, ECS state; new test.
- **Exit criteria:** the double-run test passes. If it fails, fix the nondeterminism here before
  proceeding.

### Phase 1 — Command model + chokepoint (no recording yet) — DONE

Reshape mutations to flow through one path; behaviour unchanged.

- Defined `Command` base + derived types (`PlaceBuilding`, `Demolish`, `RotateInPlace`,
  `SetRecipe`, `SetShipLayout`, `SetSiteSplitterFilters`, `SetSplitterFilters`,
  `ClearBeltTiles`, `ApplySchematicChoice`, `Reset`) in `lib`, each with a `playerId` (always 0
  now). `PlaceBuilding` is atomic (carries optional config — see the refinement note above).
- Added `CommandManager` (FIFO queue, `enqueue`/`drain`) in `lib`, holding a `Simulation&`.
- Added `Simulation::apply(const Command&)` dispatching by `CommandKind` to the existing
  mutators — the single documented chokepoint. (Mutators stay public; enforced by convention,
  see the decision note above.)
- Wired the drain: `GameWorldView::onFrame` calls `CommandManager::drain()` once per frame,
  before the tick batch (runs even at 0× → build-while-paused preserved). A drained `Reset`
  triggers the view reset.
- Refactored every UI mutation site: `GameWorldView` owns the `CommandManager` and enqueues
  directly; `MainWindow` and `SelectedBuildingPanel` emit `CommandRequestedEvent` (carrying a
  `shared_ptr<const Command>`) which `GameWorldView` subscribes to and enqueues.
- **Files:** new `lib/sim/Command.h`, `CommandManager.{h,cpp}`; `CommandRequestedEvent.h`;
  `Simulation.{h,cpp}` (`apply`); `GameWorldView.{h,cpp}`, `MainWindow.cpp`,
  `SelectedBuildingPanel.cpp`; new `CommandTest.cpp`.
- **Exit criteria:** game plays identically (including build-while-paused); determinism test
  still passes; `[command]` equivalence tests pass; no UI call site mutates the sim directly
  (verified by grep — convention, not compile-enforced).

### Phase 2 — Recording — DONE

- `ReplayRecorder` (lib) writes the **line-oriented append file**: header (`version`, `build`,
  `seed`, `config_hash`, `timestamp`) then `---`, then one tick-tagged line per command
  interleaved with `# checksum <tick> <hex>` lines. Each line is flushed, so a crash mid-run
  leaves a valid partial file. `CommandSerializer` produces the per-command text (length-prefixed
  variable parts; `ShipLayoutConfig`/filters serialized inline). The build tag is
  `__DATE__ " " __TIME__`; the config hash is a 64-bit FNV over the `*.toml` files in the config
  dir (re-hashed on playback to detect mismatch).
- **Random seed** generated in `main` (and on each restart in `MainWindow`) via
  `std::random_device`; `Simulation` retains it (`getSeed()`) for the header.
- **Recorder hooked at the chokepoint:** `CommandManager` owns an optional `ReplayRecorder`;
  `drain()` records each applied command (tick-tagged) + a post-apply RNG checksum, and
  `recordTickCheckpoint()` (called per tick from the `onFrame` loop) writes a checksum every 30
  ticks. A drained `Reset` rolls the recorder to a new file (restart = boundary).
- **Lifecycle:** `GameWorldView` attaches the recorder at construction (opens the first file with
  the initial seed + a tick-0 checksum); files live in `<data>/replays`, named
  `<timestamp>_<seed>.replay`; everything is retained.
- **Files:** new `lib/sim/ReplayRecorder.{h,cpp}`, `CommandSerializer.{h,cpp}`; `Simulation`
  (`getSeed`); `CommandManager` (recorder + tick checkpoint); `main.cpp` (seed);
  `MainWindow.cpp` / `GameWorldView.{h,cpp}` (wiring); new `ReplayRecorderTest.cpp`.
- **Exit criteria met:** recorder + serializer + drain-integration tests pass; the format is
  well-formed and flushed per line. (Live GUI recording is wired but not auto-tested here.)

### Phase 3 — Playback — DONE

- `ReplayReader` (lib) parses the file into `{ header, entries }`, where each entry is a command
  (with its tick) or a checksum (with its tick), kept in **file order**. `CommandSerializer`
  gained the inverse `parseCommand` (round-tripping every verb).
- `--replay <file>` CLI path in `main`: reads the file, **warns** on version / config-hash
  mismatch (proceeds anyway), constructs the `Simulation` from the header seed, and threads the
  parsed replay through `MainWindow` to `GameWorldView`.
- `ReplayPlayer` (lib) is the playback driver. Rather than reproduce frame batching, it applies
  each command at its **exact recorded tick** and verifies checksums **in file order**:
  `start()` processes the tick-0 entries, then after every `sim.tick()` `advanceTo(tick)`
  consumes that tick's entries (periodic checksum first, then command + its checksum — the order
  the file already has). This makes playback independent of replay-time speed/pause.
- `GameWorldView` runs the player in `onFrame` when in replay mode (manual speed/pause kept,
  forward-only); `CommandManager` is put in **replay mode** so live input is a no-op. The two
  sim-state polls (schematic-choices, game-over) are **gated off**; dialog/escape paths are
  input-driven and fall away. A **"REPLAY"** tag plus a passive **"Replay ended"** /
  **"Desync at tick N"** overlay replaces the restart dialog.
- **Files:** new `lib/sim/ReplayReader.{h,cpp}`, `ReplayPlayer.{h,cpp}`; `CommandSerializer`
  (`parseCommand`); `ReplayRecorder` (shared `computeReplayConfigHash`); `CommandManager`
  (replay mode); `main.cpp`; `MainWindow.{h,cpp}`; `GameWorldView.{h,cpp}`; new
  `ReplayPlaybackTest.cpp`.
- **Exit criteria met:** the headless `ReplayPlaybackTest` records a scripted run, reads it back,
  replays it, and asserts **no desync** and a **byte-identical final state checksum** — including
  the periodic-checksum-then-command ordering at a shared tick. (Live GUI playback is wired but
  not auto-tested here.)

### Phase 4 — Closing tests & polish

- **Round-trip test:** serialize → parse → assert command equality.
- **Replay-equivalence test (headless):** record a scripted run, play it back through the same
  `lib` path, assert per-tick checksums match end-to-end — the real proof, no UI needed.
- Mismatch-warning UX, end-of-replay overlay polish.

### Notes

- Phase 1 is the largest (the mutation-site refactor); Phase 0 is the riskiest (it may surface
  latent nondeterminism that must be fixed first).
- Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float
  hardening, expanding the file checksum beyond RNG.