# Replay — Design This document captures the design for the replay record/playback feature. It records the decisions made during design discussion; it is a complement to `architecture.md`. No implementation exists yet — this is the agreed design to implement against. ## Goal Record every play session and allow it to be played back later. Playback is **view-only** (no interaction) with **manual game-speed selection** (including pause). Playback is launched via a command-line argument to the executable. ## Approach: deterministic command-replay (re-simulation) We record **player intent** (commands) plus the inputs needed to reproduce the run, and on playback we **re-run the real simulation**, injecting the recorded commands at their recorded ticks. We do **not** record per-tick state snapshots. This is viable because the simulation is already built for it (see `architecture.md`: "determinism, replayability ... fall out for free"): - Fixed 30 Hz tick-based simulation, decoupled from render rate via `TickDriver`. - Game speed (0/0.5/1/2/4×) and pause are tick-rate multipliers — they change *how many* ticks run per frame, never the *outcome* of a tick. So speed, pause, camera scroll, and selection are pure presentation and are **not recorded**. - A single deterministic RNG stream: `Simulation::m_rng` (one `std::mt19937`) is passed by reference into `WaveSystem` and `BuildingSystem`, the only two consumers. ECS combat/AI/ movement/scrap systems use no RNG. The `utility::getRandom*` global is not used by the sim. - Config is immutable after load; a replay is pinned to the config it was recorded with. A replay run is therefore a pure function of `(seed, config, ordered commands)`. ### What we do NOT do (now) - No per-tick / keyframe state snapshots. - No backward seek / scrubbing (would require snapshots). - No save/load. (See "Future direction".) - No interactive playback (no taking over a replay mid-run). ## Replay commands A *replay command* is the resolved, serializable **intent** behind a player action — the data, not the UI gesture. Example: placing a miner records `PlaceBuilding{type=Miner, anchor=(3,5), rotation=East}`, not the mouse pixel that produced it. - Commands are at **intent level, resolved to tile coordinates / domain ids** — independent of window size, camera scroll, and DPI. - Command payloads reference **stable, deterministic domain ids** (`BuildingId`, tile coordinates, choice indices) — never raw `entt::entity` handles. These ids are sim-allocated deterministically, so a recorded command resolves to the same entity on replay. - Camera scroll, selection, game speed, and pause are **not** commands. ### Command vocabulary One command per sim-mutating operation (the complete mutation surface): - `PlaceBuilding` - `Deconstruct` - `RotateInPlace` - `SetRecipe` - `SetShipLayout` - `SetSplitterFilters` (building-site and belt variants) - `ClearBeltTiles` - `ApplySchematicChoice` - `Reset` / restart — see "Restart is a boundary". ### Command representation Commands use a **base class + derived classes** (mirroring the existing `Event` hierarchy idiom, so it is native to this codebase). They are routed through a dedicated command path, **not** through `EventManager` (see next section). > **Implementation refinement (Phase 1).** `PlaceBuilding` is **atomic**: it carries the > optional recipe / ship-layout / splitter-filters to configure the new building in the same > command. This is forced by the deferred-drain timing — commands apply at a later tick > boundary, so the caller never sees the new `BuildingId` and therefore cannot issue a > follow-up `SetRecipe`/`SetShipLayout` against it. The standalone `SetRecipe`, > `SetShipLayout`, and the two `SetSplitterFilters` commands remain for the dialog-driven > edits on *existing* buildings (which reference a known id). `Reset` carries the (move-only) > `GameConfig` via `shared_ptr` and is moved into the sim on apply; a null config means "keep > current config". ## Command system: reuse the *pattern*, not the EventManager singleton We reuse the **pattern** of the existing event system (a polymorphic base + small derived types), but the sim-mutating command path is a **dedicated, ordered queue**, not the `EventManager` pub/sub bus. Reasons: 1. **Determinism / ordering.** Sim mutations must apply in a strict, tick-pinned, recorded order. `architecture.md` deliberately keeps the sim free of `EventManager` for exactly this reason (determinism, tick-order fidelity, headless testability — why `BeamFiredEvent` uses a plain vector). Routing commands into the sim via the singleton would break that. 2. **Single consumer.** A command has exactly one recipient (the `Simulation`); pub/sub N-handler fan-out is the wrong shape. 3. **Recording chokepoint.** One place must see every command, stamp its tick, append it to the file, and apply it. A direct queue gives that; a multi-handler bus does not. 4. **Headless tests.** Tests link only `lib` and build a `Simulation` directly; the command type and apply path live in `lib` and must work with no UI and no singleton. ### Structure - **In `lib`:** a `Command` base class + derived command types, plus a `CommandManager` (ordered queue) and a single `Simulation::apply(command)` chokepoint. - **UI fan-in still uses `EventManager`:** widgets emit a UI event as today; a single dispatcher/recorder catches it, builds the `lib` command, and hands it to the `CommandManager`. This keeps widgets decoupled (consistent with current architecture). - **Replay** skips the UI half and feeds commands straight into the same `CommandManager` / `Simulation::apply` chokepoint. ### The completeness invariant (enforced structurally) **Every** sim mutation must flow through the single `CommandManager → Simulation::apply` chokepoint. Any path that mutates the sim directly would not be recorded and would silently desync the replay. This is enforced **structurally**: the `Simulation` player-action mutators are **private**, so the only way production code can reach them is `apply(command)`. > **Implementation decision (Phase 1, revised post-Phase 4).** Structural enforcement was > initially deferred in favour of convention, because the test suite legitimately drives the > same mutators directly and relies on their return values (notably the `BuildingId` from > placement, which `apply()` cannot hand back to a caller). It was later restored once a key > observation made the change cheap: **the UI's only handle to a mutable subsystem is through > `Simulation`** — no production code in `ui`/`app`/`balancing` holds a `BuildingSystem`/ > `BeltSystem` directly, and every production `buildings()`/`belts()` call is a const query. > So: > > - `Simulation::tryPlaceBuilding`, `deconstruct`, and `applySchematicChoice` are **private**. > - The mutable subsystem accessors are private and renamed `buildingsMutable()` / > `beltsMutable()`; only `const BuildingSystem& buildings() const` / `belts() const` are > public (queries). UI query sites bind to the const overload unchanged. > - `Simulation::apply` still mutates through the private members directly, so the chokepoint > itself is unaffected. > - Tests reach the private mutators through `SimulationTestAccess` (src/test, a `friend struct` > of `Simulation`), so they keep calling the real mutators **and keep getting return values** > — no id-by-position recovery needed. This header is not on the lib/ui/app include path, so > only test translation units can use it. > > The `BuildingSystem` subsystem mutators (`place`, `setRecipe`, `placeImmediate`, > `forEachBuilding`, …) stay **public**: `BuildingTest` unit-tests a bare `BuildingSystem` with > no `Simulation`/command layer, and that surface is unreachable from production anyway (you > cannot obtain a mutable subsystem without the private accessor). A `[command]` Catch2 suite > still asserts `apply(...)` produces byte-identical state to the direct mutator path, guarding > the equivalence the replay relies on. Tests are not gameplay (they never record), so direct > mutator use there does not affect replay correctness. Recording happens **at the apply chokepoint**, not at the UI gesture — so only commands that actually reached the sim are recorded, and they replay through the identical apply path. UI-side validation (placement validity, affordability) remains a pre-filter that simply does not produce a command unless the action reaches the sim. ## Command timing: drain once per frame, before the tick batch - During live play, input pushes commands onto the `CommandManager` queue (not applied synchronously). - The queue is drained at **one defined point: once per frame, before stepping the tick batch.** The whole queue is drained in FIFO order (not one-per-tick), so bursts (e.g. laying many belts quickly) apply immediately instead of dribbling across ticks, and it matches the lockstep model wanted later. - Each drained command is **tagged with the current completed-tick count**, recorded at drain time (so record-order == apply-order canonically), and applied. ### Build-while-paused is preserved The drain runs every frame including at 0× (the tick batch is simply empty when paused). So a player can place buildings while paused and **see the construction sites immediately**. This is still fully deterministic: replay applies each command at its recorded tick regardless of the frame cadence that produced it. On replay, there is no input; the player applies each pre-filled command at its recorded tick through the same drain path, preserving order. The Qt single-threaded event loop guarantees input events and the `onFrame` tick-batch never interleave, so the completed-tick count at drain time is unambiguous. (If the sim is ever moved to a worker thread, this needs a lock at the sim boundary.) ## Determinism: checksums and verification We do not verify EnTT iteration order statically. EnTT view iteration is a pure function of the sequence of spawn/destroy/add/remove operations, so on a fixed binary it contributes zero run-to-run nondeterminism. Instead we verify **end-to-end determinism** with a state checksum, and any divergence (EnTT order, float, container ordering, etc.) surfaces loudly. ### What is checksummed (now) - **RNG state only**, for now. The `mt19937` state is fingerprinted into a 64-bit value. - The hash can be extended later (entity positions/HP, belt items, building buffers, scalars) without changing the format. ### Cadence - **In the replay file:** every **30 ticks**, **and** after **every command** is applied. The per-command checksum pins any divergence to the action that triggered it; the periodic one localizes drift to a ~1 s window. On playback the recomputed checksum is compared; a mismatch reports "desync at tick N". - **In tests:** the Catch2 **double-run determinism test** hashes **full sim state every tick** (not just RNG). It runs a scripted command sequence twice from the same seed and asserts per-tick checksums match. This keeps the file lean while still catching non-RNG determinism bugs during development. ### Known limitation of the RNG-only file checksum (accepted) An RNG-only checksum only catches divergences that change **how much randomness is consumed** (wave composition, recipe rolls, scrap). Float or iteration drift that does **not** alter RNG draw counts passes the checksum undetected. This is acceptable for same-binary Windows replay (no float drift expected on an identical binary; the checksum's real job there is catching determinism *bugs*). When cross-platform replay becomes a goal, the **file** hash must be expanded to include entity state. ## Cross-platform: Windows-first, portable by construction The replay file is platform-neutral data; `std::mt19937` is bit-identical across platforms, so RNG is not a cross-platform problem. The only real cross-platform issue is **floating-point reproducibility** — the sim does heavy `QVector2D` float math, and a 1-ULP difference (compiler / CPU / SIMD / FMA contraction) can flip an in-range comparison and cascade into different ship behavior (the classic lockstep-RTS problem). Decision: **Windows-only first**, but make the later swap cheap and bounded by, from day one: - a **per-period state checksum** in the file (above), and - a **build/version + config-hash identity tag** in the header. Then cross-platform later is a contained float-hardening pass (`/fp:strict`, no FMA contraction, possibly fixed-point positions) guided by the checksums — **not** a redesign of the command-replay architecture. Note: even a new Windows *build* of the game can desync old replays for the same float reasons, so the version tag + "warn on mismatch" is needed regardless of cross-platform ambitions. ## Seed and config - **Seed:** a **random** seed is generated at the start of each run, **outside** the sim (e.g. `std::random_device` in `main`/reset), so the `Simulation` stays a pure function of `(seed, config, commands)`. The seed is written to the replay header. - **Config:** the header stores a **config hash** (not a full config snapshot). On playback the current config is hashed and compared; a mismatch warns/refuses. The hash is taken over the actually-loaded config (so editing config files and restarting yields a new, consistent replay). ## File format: line-oriented append-friendly text Non-binary, chosen for readability and crash-safety. Size is a non-issue: the command log is sparse (only ticks with a player action), so even a multi-hour game is tens of KB in any text format. - A small keyed/header section: seed, config hash, build/version, start timestamp. - One line per command, e.g. `1234 place miner 3 5 E`. - Periodic checksum lines interleaved, e.g. `# checksum 9000 a1b2c3...`. Why line-oriented text: - **Append-friendly** — the recorder stream-appends as the game runs, so a crash does not lose the replay (a crash is exactly when you would want it). A format that must be rewritten/closed as a whole is rejected for this reason. - **No new dependency** — the project has no JSON lib; toml++ is parse-oriented and clunky for a long event stream (fine for the header, awkward as an array-of-tables of thousands of entries). - Greppable, diffable, tiny. - Aligns with the project's existing text-serialization idiom (`BlueprintSerializer`, `ShipLayoutBlueprintSerializer`). ## Recording lifecycle - **Record every run.** A new replay file is created at `Simulation` construction and at each `reset()`. - **Restart is a boundary.** Restart (escape menu → restart, which reloads config and resets) closes the current file and opens a new one with a fresh seed and header. One replay file = one contiguous run from tick 0 to game-over/quit. - **Retention: keep everything.** Files live in the existing `data/` directory, named by timestamp + seed. (No automatic pruning for now.) ## Playback Launched via a command-line argument, e.g. `DotaFactory.exe --replay `. `main` for the `--replay` path: 1. Read the header → validate config hash and build/version (warn on mismatch). 2. Construct the `Simulation` from the recorded seed + config. 3. Construct the `CommandManager` in **Replay mode**, **pre-filled** with the whole command list from the file. (Pre-fill memory is trivial; streaming-read is a later optimization if files ever get huge — not needed now.) 4. Run the driver in replay mode: each frame, drain commands due at the reached tick (same drain path as live), step ticks, compare checksums. ### Replay mode rules Reframe: the schematic-choice modal is **an input source** (the device that produces an `ApplySchematicChoice` command in live play), exactly like the mouse. Replay's single rule is "**disable live input sources**", which the modal falls under. - **`CommandManager` in replay mode:** `addCommand` is a no-op; the queue is pre-filled from the file. Live input therefore produces nothing. - **Only two reactions need explicit gating** — the sim-state *polls* in `onFrame` that emit `SchematicChoicesAvailableEvent` and `GameOverEvent`. In replay these polls do not run, so no modal opens, no auto-pause occurs, and there is no deadlock against the recorded command. - **Everything else falls away for free** because it is click-driven, not sim-state-driven: the recipe dialog (`RecipeSelectionRequestedEvent`), ship-layout dialog (`LayoutDialogRequestedEvent`), and escape menu are all triggered by player input, which is disabled — so they never open and need no special handling. - **Schematic choice still resolves with no UI:** the sim regenerates identical choices deterministically (same seed + prior commands), and the pre-filled `ApplySchematicChoice` applies itself at its recorded tick through the normal drain path. The tick-tag invariant places it correctly relative to when the choices became pending, in both record and replay. - **Game-over is replaced, not just suppressed:** instead of the live restart/quit dialog, playback detects the end condition (command stream exhausted / recorded game-over reached) and stops, showing a passive "replay ended" state. - **Kept in replay:** the renderer/view and **manual game-speed selection** (including pause / 0× and fast-forward via high speed). Playback only ever moves forward. ## Future direction (informs the design, not built now) Save/load and (deterministic lockstep) multiplayer are wanted later. The command bus is the shared foundation; two cheap shaping decisions now keep that path open: 1. **Each command carries a source/player id** (always "player 0" in single-player). Lockstep multiplayer is just commands from multiple sources merged into one ordered stream. 2. **Commands are applied at a defined tick boundary** (already required for replay). Multiplayer schedules them a few ticks in the future to hide latency; single-player uses the next drain. Implications to note: - Multiplayer makes cross-platform float determinism mandatory and promotes the checksum to load-bearing desync-detection (rather than a test aid) — reinforcing doing the checksum now. - **Save/load** is the one feature that needs a *different* mechanism: either "replay to current tick" on load (reuses 100% of replay machinery; load time grows with game length, though fast-forward usually replays hours in seconds), or a full **state-snapshot serializer** (EnTT registry + belts + buildings + scalars). The snapshot serializer is also what backward-seek/scrubbing would need. Building the command bus now does not block adding it later; it is explicitly out of scope here. ## Summary of decisions - Approach: **A — deterministic command-replay** (re-simulation), no snapshots. - Scope: **view-only** playback + **manual speed selection**; launched via CLI argument. - Commands: **base class + derived types**, routed through a dedicated `CommandManager` queue and a single `Simulation::apply` chokepoint; sim mutators made non-public to **enforce** the chokepoint. UI fan-in still uses `EventManager`. - Timing: queue **drained once per frame before the tick batch**, whole queue FIFO, each command tick-tagged; **build-while-paused preserved**. - Determinism: **RNG-state checksum** in the file every **30 ticks + after each command**; **full-state per-tick hashing** in the Catch2 double-run test. Known RNG-only blind spot accepted for now. - Platform: **Windows-first**; file format + version/config-hash make a later cross-platform pass contained. - Seed: **random**, generated outside the sim, written to the header. - Config: **config hash** in the header, validated on playback. - File: **line-oriented append-friendly text**, kept in `data/`, **one file per run**, **retain everything**. - Restart: **a boundary** — new file, new seed. - Replay mode: `CommandManager` `addCommand` is a no-op + pre-filled; gate the two sim-state polls (schematic choices, game-over); passive "replay ended" instead of the game-over dialog; keep view + speed. ## Implementation plan Ordered to de-risk: prove determinism first, then build the command path, then recording, then playback. Each phase is independently testable and leaves the game in a working state. Phases 0 → 1 → 2 → 3 are strictly sequential; Phase 4 tests can start as soon as their subject exists. ### Phase 0 — Determinism foundation & verification (no replay yet) The whole feature rests on a deterministic sim, so prove that before building on it. - Add a `mt19937` state **fingerprint** (fold its serialized state into a 64-bit value). - Add a **full-state checksum** path (positions, HP, velocities, belt items, building buffers, scalars), used by tests; each subsystem contributes via its own `appendChecksum(Hasher&)` so no state knowledge is duplicated. - Add a Catch2 **double-run determinism test**: run a scripted sequence twice from the same seed, assert per-tick **full-state** checksums match. - **Files:** new `lib/sim` checksum helper; small additions to `Simulation`, `BeltSystem`, `BuildingSystem`, ECS state; new test. - **Exit criteria:** the double-run test passes. If it fails, fix the nondeterminism here before proceeding. ### Phase 1 — Command model + chokepoint (no recording yet) — DONE Reshape mutations to flow through one path; behaviour unchanged. - Defined `Command` base + derived types (`PlaceBuilding`, `Deconstruct`, `RotateInPlace`, `SetRecipe`, `SetShipLayout`, `SetSiteSplitterFilters`, `SetSplitterFilters`, `ClearBeltTiles`, `ApplySchematicChoice`, `Reset`) in `lib`, each with a `playerId` (always 0 now). `PlaceBuilding` is atomic (carries optional config — see the refinement note above). - Added `CommandManager` (FIFO queue, `enqueue`/`drain`) in `lib`, holding a `Simulation&`. - Added `Simulation::apply(const Command&)` dispatching by `CommandKind` to the underlying mutators — the single chokepoint. The `Simulation` player-action mutators are **private** (compile-time enforced; tests reach them via the `SimulationTestAccess` friend) — see the decision note above. - Wired the drain: `GameWorldView::onFrame` calls `CommandManager::drain()` once per frame, before the tick batch (runs even at 0× → build-while-paused preserved). A drained `Reset` triggers the view reset. - Refactored every UI mutation site: `GameWorldView` owns the `CommandManager` and enqueues directly; `MainWindow` and `SelectionPanel` emit `CommandRequestedEvent` (carrying a `shared_ptr`) which `GameWorldView` subscribes to and enqueues. - **Files:** new `lib/sim/Command.h`, `CommandManager.{h,cpp}`; `CommandRequestedEvent.h`; `Simulation.{h,cpp}` (`apply`); `GameWorldView.{h,cpp}`, `MainWindow.cpp`, `SelectionPanel.cpp`; new `CommandTest.cpp`. - **Exit criteria:** game plays identically (including build-while-paused); determinism test still passes; `[command]` equivalence tests pass; no production call site can mutate the sim directly (compile-enforced: the `Simulation` mutators are private, tests excepted via `SimulationTestAccess`). ### Phase 2 — Recording — DONE - `ReplayRecorder` (lib) writes the **line-oriented append file**: header (`version`, `build`, `seed`, `config_hash`, `timestamp`) then `---`, then one tick-tagged line per command interleaved with `# checksum ` lines. Each line is flushed, so a crash mid-run leaves a valid partial file. `CommandSerializer` produces the per-command text (length-prefixed variable parts; `ShipLayoutConfig`/filters serialized inline). The build tag is `__DATE__ " " __TIME__`; the config hash is a 64-bit FNV over the `*.toml` files in the config dir (re-hashed on playback to detect mismatch). - **Random seed** generated in `main` (and on each restart in `MainWindow`) via `std::random_device`; `Simulation` retains it (`getSeed()`) for the header. - **Recorder hooked at the chokepoint:** `CommandManager` owns an optional `ReplayRecorder`; `drain()` records each applied command (tick-tagged) + a post-apply RNG checksum, and `recordTickCheckpoint()` (called per tick from the `onFrame` loop) writes a checksum every 30 ticks. A drained `Reset` rolls the recorder to a new file (restart = boundary). - **Lifecycle:** `GameWorldView` attaches the recorder at construction (opens the first file with the initial seed + a tick-0 checksum); files live in `/replays`, named `_.replay`; everything is retained. - **Files:** new `lib/sim/ReplayRecorder.{h,cpp}`, `CommandSerializer.{h,cpp}`; `Simulation` (`getSeed`); `CommandManager` (recorder + tick checkpoint); `main.cpp` (seed); `MainWindow.cpp` / `GameWorldView.{h,cpp}` (wiring); new `ReplayRecorderTest.cpp`. - **Exit criteria met:** recorder + serializer + drain-integration tests pass; the format is well-formed and flushed per line. (Live GUI recording is wired but not auto-tested here.) ### Phase 3 — Playback — DONE - `ReplayReader` (lib) parses the file into `{ header, entries }`, where each entry is a command (with its tick) or a checksum (with its tick), kept in **file order**. `CommandSerializer` gained the inverse `parseCommand` (round-tripping every verb). - `--replay ` CLI path in `main`: reads the file, **warns** on version / config-hash mismatch (proceeds anyway), constructs the `Simulation` from the header seed, and threads the parsed replay through `MainWindow` to `GameWorldView`. - `ReplayPlayer` (lib) is the playback driver. Rather than reproduce frame batching, it applies each command at its **exact recorded tick** and verifies checksums **in file order**: `start()` processes the tick-0 entries, then after every `sim.tick()` `advanceTo(tick)` consumes that tick's entries (periodic checksum first, then command + its checksum — the order the file already has). This makes playback independent of replay-time speed/pause. - `GameWorldView` runs the player in `onFrame` when in replay mode (manual speed/pause kept, forward-only); `CommandManager` is put in **replay mode** so live input is a no-op. The two sim-state polls (schematic-choices, game-over) are **gated off**; dialog/escape paths are input-driven and fall away. A **"REPLAY"** tag plus a passive **"Replay ended"** / **"Desync at tick N"** overlay replaces the restart dialog. - **Files:** new `lib/sim/ReplayReader.{h,cpp}`, `ReplayPlayer.{h,cpp}`; `CommandSerializer` (`parseCommand`); `ReplayRecorder` (shared `computeReplayConfigHash`); `CommandManager` (replay mode); `main.cpp`; `MainWindow.{h,cpp}`; `GameWorldView.{h,cpp}`; new `ReplayPlaybackTest.cpp`. - **Exit criteria met:** the headless `ReplayPlaybackTest` records a scripted run, reads it back, replays it, and asserts **no desync** and a **byte-identical final state checksum** — including the periodic-checksum-then-command ordering at a shared tick. (Live GUI playback is wired but not auto-tested here.) ### Phase 4 — Closing tests & polish — DONE - **Round-trip:** every command verb serializes → parses → re-serializes identically; malformed input is rejected (`parseCommand` returns nullptr). - **Replay-equivalence (headless):** a short scripted run and a **long ~2400-tick run through waves/combat** each record → read → replay with **no desync** and a **byte-identical final state checksum**. - **Desync detection:** corrupting one recorded checksum makes the player report the exact desync tick. - **Reset boundary:** a `Reset` drained through `CommandManager` rolls the recorder to a new file (named by the new seed). - **Polish:** the end-of-replay / desync overlay dims the world behind the message for readability; config/version mismatch is warned to the log on launch (its visible consequence, a desync, is already surfaced by the overlay). ### Status Record + playback is functionally complete and covered by headless tests. Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float hardening, expanding the file checksum beyond RNG. Known minor rough edge: in replay mode the recipe/layout dialogs and escape→restart can still open but do nothing (their commands hit the no-op enqueue); fully disabling that input UI is polish, not correctness. ### Notes - Phase 1 is the largest (the mutation-site refactor); Phase 0 is the riskiest (it may surface latent nondeterminism that must be fixed first). - Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float hardening, expanding the file checksum beyond RNG.