dota_factory/docs/replay_design.md

Replay — Design

This document captures the design for the replay record/playback feature. It records the decisions made during design discussion; it is a complement to architecture.md. No implementation exists yet — this is the agreed design to implement against.

Goal

Record every play session and allow it to be played back later. Playback is view-only (no interaction) with manual game-speed selection (including pause). Playback is launched via a command-line argument to the executable.

Approach: deterministic command-replay (re-simulation)

We record player intent (commands) plus the inputs needed to reproduce the run, and on playback we re-run the real simulation, injecting the recorded commands at their recorded ticks. We do not record per-tick state snapshots.

This is viable because the simulation is already built for it (see architecture.md: "determinism, replayability ... fall out for free"):

• Fixed 30 Hz tick-based simulation, decoupled from render rate via TickDriver.
• Game speed (0/0.5/1/2/4×) and pause are tick-rate multipliers — they change how many ticks run per frame, never the outcome of a tick. So speed, pause, camera scroll, and selection are pure presentation and are not recorded.
• A single deterministic RNG stream: Simulation::m_rng (one std::mt19937) is passed by reference into WaveSystem and BuildingSystem, the only two consumers. ECS combat/AI/ movement/scrap systems use no RNG. The utility::getRandom* global is not used by the sim.
• Config is immutable after load; a replay is pinned to the config it was recorded with.

A replay run is therefore a pure function of (seed, config, ordered commands).

What we do NOT do (now)

• No per-tick / keyframe state snapshots.
• No backward seek / scrubbing (would require snapshots).
• No save/load. (See "Future direction".)
• No interactive playback (no taking over a replay mid-run).

Replay commands

A replay command is the resolved, serializable intent behind a player action — the data, not the UI gesture. Example: placing a miner records PlaceBuilding{type=Miner, anchor=(3,5), rotation=East}, not the mouse pixel that produced it.

• Commands are at intent level, resolved to tile coordinates / domain ids — independent of window size, camera scroll, and DPI.
• Command payloads reference stable, deterministic domain ids (BuildingId, tile coordinates, choice indices) — never raw entt::entity handles. These ids are sim-allocated deterministically, so a recorded command resolves to the same entity on replay.
• Camera scroll, selection, game speed, and pause are not commands.

Command vocabulary

One command per sim-mutating operation (the complete mutation surface):

• PlaceBuilding
• Demolish
• RotateInPlace
• SetRecipe
• SetShipLayout
• SetSplitterFilters (building-site and belt variants)
• ClearBeltTiles
• ApplySchematicChoice
• Reset / restart — see "Restart is a boundary".

Command representation

Commands use a base class + derived classes (mirroring the existing Event hierarchy idiom, so it is native to this codebase). They are routed through a dedicated command path, not through EventManager (see next section).

Implementation refinement (Phase 1). PlaceBuilding is atomic: it carries the optional recipe / ship-layout / splitter-filters to configure the new building in the same command. This is forced by the deferred-drain timing — commands apply at a later tick boundary, so the caller never sees the new BuildingId and therefore cannot issue a follow-up SetRecipe/SetShipLayout against it. The standalone SetRecipe, SetShipLayout, and the two SetSplitterFilters commands remain for the dialog-driven edits on existing buildings (which reference a known id). Reset carries the (move-only) GameConfig via shared_ptr and is moved into the sim on apply; a null config means "keep current config".

Command system: reuse the pattern, not the EventManager singleton

We reuse the pattern of the existing event system (a polymorphic base + small derived types), but the sim-mutating command path is a dedicated, ordered queue, not the EventManager pub/sub bus. Reasons:

1. Determinism / ordering. Sim mutations must apply in a strict, tick-pinned, recorded order. architecture.md deliberately keeps the sim free of EventManager for exactly this reason (determinism, tick-order fidelity, headless testability — why BeamFiredEvent uses a plain vector). Routing commands into the sim via the singleton would break that.
2. Single consumer. A command has exactly one recipient (the Simulation); pub/sub N-handler fan-out is the wrong shape.
3. Recording chokepoint. One place must see every command, stamp its tick, append it to the file, and apply it. A direct queue gives that; a multi-handler bus does not.
4. Headless tests. Tests link only lib and build a Simulation directly; the command type and apply path live in lib and must work with no UI and no singleton.

Structure

• In lib: a Command base class + derived command types, plus a CommandManager (ordered queue) and a single Simulation::apply(command) chokepoint.
• UI fan-in still uses EventManager: widgets emit a UI event as today; a single dispatcher/recorder catches it, builds the lib command, and hands it to the CommandManager. This keeps widgets decoupled (consistent with current architecture).
• Replay skips the UI half and feeds commands straight into the same CommandManager / Simulation::apply chokepoint.

The completeness invariant (enforced structurally)

Every sim mutation must flow through the single CommandManager → Simulation::apply chokepoint. Any path that mutates the sim directly would not be recorded and would silently desync the replay.

This is enforced structurally: the Simulation player-action mutators are private, so the only way production code can reach them is apply(command).

Implementation decision (Phase 1, revised post-Phase 4). Structural enforcement was initially deferred in favour of convention, because the test suite legitimately drives the same mutators directly and relies on their return values (notably the BuildingId from placement, which apply() cannot hand back to a caller). It was later restored once a key observation made the change cheap: the UI's only handle to a mutable subsystem is through Simulation — no production code in ui/app/balancing holds a BuildingSystem/ BeltSystem directly, and every production buildings()/belts() call is a const query. So:

• Simulation::tryPlaceBuilding, demolish, and applySchematicChoice are private.
• The mutable subsystem accessors are private and renamed buildingsMutable() / beltsMutable(); only const BuildingSystem& buildings() const / belts() const are public (queries). UI query sites bind to the const overload unchanged.
• Simulation::apply still mutates through the private members directly, so the chokepoint itself is unaffected.
• Tests reach the private mutators through SimulationTestAccess (src/test, a friend struct of Simulation), so they keep calling the real mutators and keep getting return values — no id-by-position recovery needed. This header is not on the lib/ui/app include path, so only test translation units can use it.

The BuildingSystem subsystem mutators (place, setRecipe, placeImmediate, forEachBuilding, …) stay public: BuildingTest unit-tests a bare BuildingSystem with no Simulation/command layer, and that surface is unreachable from production anyway (you cannot obtain a mutable subsystem without the private accessor). A [command] Catch2 suite still asserts apply(...) produces byte-identical state to the direct mutator path, guarding the equivalence the replay relies on. Tests are not gameplay (they never record), so direct mutator use there does not affect replay correctness.

Recording happens at the apply chokepoint, not at the UI gesture — so only commands that actually reached the sim are recorded, and they replay through the identical apply path. UI-side validation (placement validity, affordability) remains a pre-filter that simply does not produce a command unless the action reaches the sim.

Command timing: drain once per frame, before the tick batch

• During live play, input pushes commands onto the CommandManager queue (not applied synchronously).
• The queue is drained at one defined point: once per frame, before stepping the tick batch. The whole queue is drained in FIFO order (not one-per-tick), so bursts (e.g. laying many belts quickly) apply immediately instead of dribbling across ticks, and it matches the lockstep model wanted later.
• Each drained command is tagged with the current completed-tick count, recorded at drain time (so record-order == apply-order canonically), and applied.

Build-while-paused is preserved

The drain runs every frame including at 0× (the tick batch is simply empty when paused). So a player can place buildings while paused and see the construction sites immediately. This is still fully deterministic: replay applies each command at its recorded tick regardless of the frame cadence that produced it.

On replay, there is no input; the player applies each pre-filled command at its recorded tick through the same drain path, preserving order.

The Qt single-threaded event loop guarantees input events and the onFrame tick-batch never interleave, so the completed-tick count at drain time is unambiguous. (If the sim is ever moved to a worker thread, this needs a lock at the sim boundary.)

Determinism: checksums and verification

We do not verify EnTT iteration order statically. EnTT view iteration is a pure function of the sequence of spawn/destroy/add/remove operations, so on a fixed binary it contributes zero run-to-run nondeterminism. Instead we verify end-to-end determinism with a state checksum, and any divergence (EnTT order, float, container ordering, etc.) surfaces loudly.

What is checksummed (now)

• RNG state only, for now. The mt19937 state is fingerprinted into a 64-bit value.
• The hash can be extended later (entity positions/HP, belt items, building buffers, scalars) without changing the format.

Cadence

• In the replay file: every 30 ticks, and after every command is applied. The per-command checksum pins any divergence to the action that triggered it; the periodic one localizes drift to a ~1 s window. On playback the recomputed checksum is compared; a mismatch reports "desync at tick N".
• In tests: the Catch2 double-run determinism test hashes full sim state every tick (not just RNG). It runs a scripted command sequence twice from the same seed and asserts per-tick checksums match. This keeps the file lean while still catching non-RNG determinism bugs during development.

Known limitation of the RNG-only file checksum (accepted)

An RNG-only checksum only catches divergences that change how much randomness is consumed (wave composition, recipe rolls, scrap). Float or iteration drift that does not alter RNG draw counts passes the checksum undetected. This is acceptable for same-binary Windows replay (no float drift expected on an identical binary; the checksum's real job there is catching determinism bugs). When cross-platform replay becomes a goal, the file hash must be expanded to include entity state.

Cross-platform: Windows-first, portable by construction

The replay file is platform-neutral data; std::mt19937 is bit-identical across platforms, so RNG is not a cross-platform problem. The only real cross-platform issue is floating-point reproducibility — the sim does heavy QVector2D float math, and a 1-ULP difference (compiler / CPU / SIMD / FMA contraction) can flip an in-range comparison and cascade into different ship behavior (the classic lockstep-RTS problem).

Decision: Windows-only first, but make the later swap cheap and bounded by, from day one:

• a per-period state checksum in the file (above), and
• a build/version + config-hash identity tag in the header.

Then cross-platform later is a contained float-hardening pass (/fp:strict, no FMA contraction, possibly fixed-point positions) guided by the checksums — not a redesign of the command-replay architecture.

Note: even a new Windows build of the game can desync old replays for the same float reasons, so the version tag + "warn on mismatch" is needed regardless of cross-platform ambitions.

Seed and config

• Seed: a random seed is generated at the start of each run, outside the sim (e.g. std::random_device in main/reset), so the Simulation stays a pure function of (seed, config, commands). The seed is written to the replay header.
• Config: the header stores a config hash (not a full config snapshot). On playback the current config is hashed and compared; a mismatch warns/refuses. The hash is taken over the actually-loaded config (so editing config files and restarting yields a new, consistent replay).

File format: line-oriented append-friendly text

Non-binary, chosen for readability and crash-safety. Size is a non-issue: the command log is sparse (only ticks with a player action), so even a multi-hour game is tens of KB in any text format.

• A small keyed/header section: seed, config hash, build/version, start timestamp.
• One line per command, e.g. 1234 place miner 3 5 E.
• Periodic checksum lines interleaved, e.g. # checksum 9000 a1b2c3....

Why line-oriented text:

• Append-friendly — the recorder stream-appends as the game runs, so a crash does not lose the replay (a crash is exactly when you would want it). A format that must be rewritten/closed as a whole is rejected for this reason.
• No new dependency — the project has no JSON lib; toml++ is parse-oriented and clunky for a long event stream (fine for the header, awkward as an array-of-tables of thousands of entries).
• Greppable, diffable, tiny.
• Aligns with the project's existing text-serialization idiom (BlueprintSerializer, ShipLayoutBlueprintSerializer).

Recording lifecycle

• Record every run. A new replay file is created at Simulation construction and at each reset().
• Restart is a boundary. Restart (escape menu → restart, which reloads config and resets) closes the current file and opens a new one with a fresh seed and header. One replay file = one contiguous run from tick 0 to game-over/quit.
• Retention: keep everything. Files live in the existing data/ directory, named by timestamp + seed. (No automatic pruning for now.)

Playback

Launched via a command-line argument, e.g. DotaFactory.exe --replay <file>.

main for the --replay path:

1. Read the header → validate config hash and build/version (warn on mismatch).
2. Construct the Simulation from the recorded seed + config.
3. Construct the CommandManager in Replay mode, pre-filled with the whole command list from the file. (Pre-fill memory is trivial; streaming-read is a later optimization if files ever get huge — not needed now.)
4. Run the driver in replay mode: each frame, drain commands due at the reached tick (same drain path as live), step ticks, compare checksums.

Replay mode rules

Reframe: the schematic-choice modal is an input source (the device that produces an ApplySchematicChoice command in live play), exactly like the mouse. Replay's single rule is "disable live input sources", which the modal falls under.

• CommandManager in replay mode: addCommand is a no-op; the queue is pre-filled from the file. Live input therefore produces nothing.
• Only two reactions need explicit gating — the sim-state polls in onFrame that emit SchematicChoicesAvailableEvent and GameOverEvent. In replay these polls do not run, so no modal opens, no auto-pause occurs, and there is no deadlock against the recorded command.
• Everything else falls away for free because it is click-driven, not sim-state-driven: the recipe dialog (RecipeSelectionRequestedEvent), ship-layout dialog (LayoutDialogRequestedEvent), and escape menu are all triggered by player input, which is disabled — so they never open and need no special handling.
• Schematic choice still resolves with no UI: the sim regenerates identical choices deterministically (same seed + prior commands), and the pre-filled ApplySchematicChoice applies itself at its recorded tick through the normal drain path. The tick-tag invariant places it correctly relative to when the choices became pending, in both record and replay.
• Game-over is replaced, not just suppressed: instead of the live restart/quit dialog, playback detects the end condition (command stream exhausted / recorded game-over reached) and stops, showing a passive "replay ended" state.
• Kept in replay: the renderer/view and manual game-speed selection (including pause / 0× and fast-forward via high speed). Playback only ever moves forward.

Future direction (informs the design, not built now)

Save/load and (deterministic lockstep) multiplayer are wanted later. The command bus is the shared foundation; two cheap shaping decisions now keep that path open:

1. Each command carries a source/player id (always "player 0" in single-player). Lockstep multiplayer is just commands from multiple sources merged into one ordered stream.
2. Commands are applied at a defined tick boundary (already required for replay). Multiplayer schedules them a few ticks in the future to hide latency; single-player uses the next drain.

Implications to note:

• Multiplayer makes cross-platform float determinism mandatory and promotes the checksum to load-bearing desync-detection (rather than a test aid) — reinforcing doing the checksum now.
• Save/load is the one feature that needs a different mechanism: either "replay to current tick" on load (reuses 100% of replay machinery; load time grows with game length, though fast-forward usually replays hours in seconds), or a full state-snapshot serializer (EnTT registry + belts + buildings + scalars). The snapshot serializer is also what backward-seek/scrubbing would need. Building the command bus now does not block adding it later; it is explicitly out of scope here.

Summary of decisions

• Approach: A — deterministic command-replay (re-simulation), no snapshots.
• Scope: view-only playback + manual speed selection; launched via CLI argument.
• Commands: base class + derived types, routed through a dedicated CommandManager queue and a single Simulation::apply chokepoint; sim mutators made non-public to enforce the chokepoint. UI fan-in still uses EventManager.
• Timing: queue drained once per frame before the tick batch, whole queue FIFO, each command tick-tagged; build-while-paused preserved.
• Determinism: RNG-state checksum in the file every 30 ticks + after each command; full-state per-tick hashing in the Catch2 double-run test. Known RNG-only blind spot accepted for now.
• Platform: Windows-first; file format + version/config-hash make a later cross-platform pass contained.
• Seed: random, generated outside the sim, written to the header.
• Config: config hash in the header, validated on playback.
• File: line-oriented append-friendly text, kept in data/, one file per run, retain everything.
• Restart: a boundary — new file, new seed.
• Replay mode: CommandManager addCommand is a no-op + pre-filled; gate the two sim-state polls (schematic choices, game-over); passive "replay ended" instead of the game-over dialog; keep view + speed.

Implementation plan

Ordered to de-risk: prove determinism first, then build the command path, then recording, then playback. Each phase is independently testable and leaves the game in a working state. Phases 0 → 1 → 2 → 3 are strictly sequential; Phase 4 tests can start as soon as their subject exists.

Phase 0 — Determinism foundation & verification (no replay yet)

The whole feature rests on a deterministic sim, so prove that before building on it.

• Add a mt19937 state fingerprint (fold its serialized state into a 64-bit value).
• Add a full-state checksum path (positions, HP, velocities, belt items, building buffers, scalars), used by tests; each subsystem contributes via its own appendChecksum(Hasher&) so no state knowledge is duplicated.
• Add a Catch2 double-run determinism test: run a scripted sequence twice from the same seed, assert per-tick full-state checksums match.
• Files: new lib/sim checksum helper; small additions to Simulation, BeltSystem, BuildingSystem, ECS state; new test.
• Exit criteria: the double-run test passes. If it fails, fix the nondeterminism here before proceeding.

Phase 1 — Command model + chokepoint (no recording yet) — DONE

Reshape mutations to flow through one path; behaviour unchanged.

• Defined Command base + derived types (PlaceBuilding, Demolish, RotateInPlace, SetRecipe, SetShipLayout, SetSiteSplitterFilters, SetSplitterFilters, ClearBeltTiles, ApplySchematicChoice, Reset) in lib, each with a playerId (always 0 now). PlaceBuilding is atomic (carries optional config — see the refinement note above).
• Added CommandManager (FIFO queue, enqueue/drain) in lib, holding a Simulation&.
• Added Simulation::apply(const Command&) dispatching by CommandKind to the underlying mutators — the single chokepoint. The Simulation player-action mutators are private (compile-time enforced; tests reach them via the SimulationTestAccess friend) — see the decision note above.
• Wired the drain: GameWorldView::onFrame calls CommandManager::drain() once per frame, before the tick batch (runs even at 0× → build-while-paused preserved). A drained Reset triggers the view reset.
• Refactored every UI mutation site: GameWorldView owns the CommandManager and enqueues directly; MainWindow and SelectedBuildingPanel emit CommandRequestedEvent (carrying a shared_ptr<const Command>) which GameWorldView subscribes to and enqueues.
• Files: new lib/sim/Command.h, CommandManager.{h,cpp}; CommandRequestedEvent.h; Simulation.{h,cpp} (apply); GameWorldView.{h,cpp}, MainWindow.cpp, SelectedBuildingPanel.cpp; new CommandTest.cpp.
• Exit criteria: game plays identically (including build-while-paused); determinism test still passes; [command] equivalence tests pass; no production call site can mutate the sim directly (compile-enforced: the Simulation mutators are private, tests excepted via SimulationTestAccess).

Phase 2 — Recording — DONE

• ReplayRecorder (lib) writes the line-oriented append file: header (version, build, seed, config_hash, timestamp) then ---, then one tick-tagged line per command interleaved with # checksum <tick> <hex> lines. Each line is flushed, so a crash mid-run leaves a valid partial file. CommandSerializer produces the per-command text (length-prefixed variable parts; ShipLayoutConfig/filters serialized inline). The build tag is __DATE__ " " __TIME__; the config hash is a 64-bit FNV over the *.toml files in the config dir (re-hashed on playback to detect mismatch).
• Random seed generated in main (and on each restart in MainWindow) via std::random_device; Simulation retains it (getSeed()) for the header.
• Recorder hooked at the chokepoint: CommandManager owns an optional ReplayRecorder; drain() records each applied command (tick-tagged) + a post-apply RNG checksum, and recordTickCheckpoint() (called per tick from the onFrame loop) writes a checksum every 30 ticks. A drained Reset rolls the recorder to a new file (restart = boundary).
• Lifecycle: GameWorldView attaches the recorder at construction (opens the first file with the initial seed + a tick-0 checksum); files live in <data>/replays, named <timestamp>_<seed>.replay; everything is retained.
• Files: new lib/sim/ReplayRecorder.{h,cpp}, CommandSerializer.{h,cpp}; Simulation (getSeed); CommandManager (recorder + tick checkpoint); main.cpp (seed); MainWindow.cpp / GameWorldView.{h,cpp} (wiring); new ReplayRecorderTest.cpp.
• Exit criteria met: recorder + serializer + drain-integration tests pass; the format is well-formed and flushed per line. (Live GUI recording is wired but not auto-tested here.)

Phase 3 — Playback — DONE

• ReplayReader (lib) parses the file into { header, entries }, where each entry is a command (with its tick) or a checksum (with its tick), kept in file order. CommandSerializer gained the inverse parseCommand (round-tripping every verb).
• --replay <file> CLI path in main: reads the file, warns on version / config-hash mismatch (proceeds anyway), constructs the Simulation from the header seed, and threads the parsed replay through MainWindow to GameWorldView.
• ReplayPlayer (lib) is the playback driver. Rather than reproduce frame batching, it applies each command at its exact recorded tick and verifies checksums in file order: start() processes the tick-0 entries, then after every sim.tick() advanceTo(tick) consumes that tick's entries (periodic checksum first, then command + its checksum — the order the file already has). This makes playback independent of replay-time speed/pause.
• GameWorldView runs the player in onFrame when in replay mode (manual speed/pause kept, forward-only); CommandManager is put in replay mode so live input is a no-op. The two sim-state polls (schematic-choices, game-over) are gated off; dialog/escape paths are input-driven and fall away. A "REPLAY" tag plus a passive "Replay ended" / "Desync at tick N" overlay replaces the restart dialog.
• Files: new lib/sim/ReplayReader.{h,cpp}, ReplayPlayer.{h,cpp}; CommandSerializer (parseCommand); ReplayRecorder (shared computeReplayConfigHash); CommandManager (replay mode); main.cpp; MainWindow.{h,cpp}; GameWorldView.{h,cpp}; new ReplayPlaybackTest.cpp.
• Exit criteria met: the headless ReplayPlaybackTest records a scripted run, reads it back, replays it, and asserts no desync and a byte-identical final state checksum — including the periodic-checksum-then-command ordering at a shared tick. (Live GUI playback is wired but not auto-tested here.)

Phase 4 — Closing tests & polish — DONE

• Round-trip: every command verb serializes → parses → re-serializes identically; malformed input is rejected (parseCommand returns nullptr).
• Replay-equivalence (headless): a short scripted run and a long ~2400-tick run through waves/combat each record → read → replay with no desync and a byte-identical final state checksum.
• Desync detection: corrupting one recorded checksum makes the player report the exact desync tick.
• Reset boundary: a Reset drained through CommandManager rolls the recorder to a new file (named by the new seed).
• Polish: the end-of-replay / desync overlay dims the world behind the message for readability; config/version mismatch is warned to the log on launch (its visible consequence, a desync, is already surfaced by the overlay).

Status

Record + playback is functionally complete and covered by headless tests. Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float hardening, expanding the file checksum beyond RNG. Known minor rough edge: in replay mode the recipe/layout dialogs and escape→restart can still open but do nothing (their commands hit the no-op enqueue); fully disabling that input UI is polish, not correctness.

Notes

• Phase 1 is the largest (the mutation-site refactor); Phase 0 is the riskiest (it may surface latent nondeterminism that must be fixed first).
• Still deferred (per this design): snapshots, save/load, backward-seek, cross-platform float hardening, expanding the file checksum beyond RNG.