From 68cf36a3d043446e7381a8284d736ad6f49af582 Mon Sep 17 00:00:00 2001 From: pyr0ball Date: Fri, 3 Jul 2026 10:02:44 -0700 Subject: [PATCH] docs: add self-contained session handoff for continuing on Linux docs/HANDOFF.md bootstraps a fresh session (no prior conversation memory): project goal + variants, current state (game boots, formats cracked), the Windows->Linux tooling map, repo/git constraints (copyright, no game data), the binary's section FO<->VA formulas and verified control-flow address map, format status, the Ghidra decompilation targets (FPS-coupling/render modes, cheats, weapons, AI, level loaders), the format-cracking methodology, and suggested next steps. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01TN4Ytn3gdWRonNmHpisWQv --- docs/HANDOFF.md | 237 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 237 insertions(+) create mode 100644 docs/HANDOFF.md diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md new file mode 100644 index 0000000..cb85556 --- /dev/null +++ b/docs/HANDOFF.md @@ -0,0 +1,237 @@ +# Havoc Remaster - Session Handoff + +Read this first. It is a self-contained bootstrap for continuing the Havoc reverse-engineering +and remaster work in a fresh session (including on Linux). It assumes no prior conversation +memory. Companion docs: `docs/PATCHES.md`, `docs/FORMATS.md`, `docs/METHODOLOGY.md`, +`docs/WEAPONS.md`. + +Last updated at the end of the session that got the game booting and cracked GRAFIX. + +--- + +## 1. What this project is + +Reviving **Havoc** (Reality Bytes, Inc., 1995): a first-person, **software-rendered** 3D +vehicular-combat shooter for Win95/98. The binaries are **32-bit PE (i386), DirectDraw + +DirectSound, no Direct3D** (the 3D is a software rasterizer). It is NOT DOS/16-bit, so DOSBox is +the wrong tool. + +**End goal:** port to **Godot** as a remaster with QoL (widescreen, higher-res, rebindable +input, fixed-timestep physics). Three planned variants: +1. Faithful 1:1 (original bugs preserved as an option, including FPS-coupled cockpit physics). +2. Remaster (bug fixes, widescreen, rebindable input, higher-res). +3. Roguelite (procedural worlds, extra tanks) as a separate mode. + +**Two active work streams for the next sessions:** +- **A. Ghidra decompilation** of `HAVOC.EXE` to recover game logic (weapons, AI, physics, cheats, + the FPS-coupling, level/entity loaders). +- **B. Asset cracking** of the remaining `WORLDS/` and `.FF` formats. + +The user wants a **detailed blog post** of the whole RE process, so when you crack something, +write down the *discovery steps and dead ends*, not just the final spec. Keep notes in `docs/`. + +--- + +## 2. Current state (what works / what is done) + +- **The game boots and is fully playable on Windows 11** via dgVoodoo2 (DirectDraw wrapper, + windowed). Loads levels, enemies shoot, etc. This was the big unlock this session. +- **NOCD + copy-protection fix:** `HAVOC_NOCD.EXE` is the patched original (28 patches). The + final blocker was a copy-protection check in the data-file loader `0x42a480` that only accepted + files loaded from the CD drive; a 14-byte patch makes it accept the local files. Full write-up + and patch table in `docs/PATCHES.md`. +- **Formats cracked:** `.FF` archives, MUSIC.FF (17 songs + FL Studio pack), INTRFACE.FF bitmaps, + and now **GRAFIX0N.DAT** (64x64 texture tilesets, `tools/grafix_extract.py`). See + `docs/FORMATS.md`. +- **MAP layout partially decoded:** `MAP0101.MAP` is a 2-byte-per-cell grid, ~192x144 (384-byte + rows, found via row autocorrelation). Cell semantics (which byte is tile vs height) still TBD. +- **Known open gameplay issue:** movement/physics is **FPS-coupled** (one sim step per rendered + frame, no fixed timestep). Workaround in place: dgVoodoo `FPSLimit` set to 30 (tunable). The + user's memory says the in-game "minimal" HUD mode was NOT FPS-coupled while "cockpit" mode was, + which suggests a limiter/delta path already exists in the binary for one mode. Finding that in + Ghidra is a priority (see section 7). + +--- + +## 3. Environment (Windows -> Linux) + +The repo lives on **navi** at `/Library/Development/devl/Havoc` (mounted on the Windows box as +`Z:\Development\devl\Havoc`). On Linux you work directly in that path. + +Tooling used this session and how it maps to Linux: + +| Purpose | Windows (this session) | Linux equivalent | +|---|---|---| +| Python | `C:/Python313/python.exe` | `python3` | +| Disassembly | `radare2` + `r2pipe` (pip) | same: `apt/pacman install radare2`, `pip install r2pipe` | +| Image output | Pillow (`pip install pillow`) | same | +| Decompiler | (not yet used) | **Ghidra** (Java, cross-platform) - the main Linux task | +| Run the game | dgVoodoo2 on native Windows | Wine + dgVoodoo (test only; not needed for static work) | + +**Windows-only tools that do NOT port (and are no longer needed):** the crash diagnosis used WER +minidumps from `%LOCALAPPDATA%\CrashDumps` parsed in pure Python, and a `ctypes` Win32 debugger +(`tools/dbg_path.py`). Those were for the boot crash, which is solved. Decompilation and asset +cracking are pure static analysis and run fine on Linux. + +All the extraction tools (`ff_extract.py`, `grafix_extract.py`) are cross-platform Python +(they use `os.path`, avoid Windows-isms). + +**Note:** the game itself only runs on Windows (or Wine). Linux sessions are for Ghidra + +asset extraction, which need only the files, not a running game. + +--- + +## 4. Repo, git, and constraints + +- **Remote:** Forgejo, private repo `git.opensourcesolarpunk.com/pyr0ball/havoc-remaster` + (personal `pyr0ball` account, not the CircuitForge org). Origin is already configured. +- **Pushing:** the user pushes on request. Token lives in a local `.env.mirror` file (outside the + repo) as `FORGEJO_TOKEN` - **never print it, never store it in `.git/config`**. Push via + `git -c http.extraheader="Authorization: token $TOKEN" push` for a one-time push. +- **Commits:** solo repo, history is on `main`. Commit style: conventional prefixes + (`feat:`, `docs:`, `chore:`). Every commit message ends with the two trailer lines the harness + provides (Co-Authored-By + Claude-Session). + +**Hard constraints (do not violate):** +- **Never commit original game data or binaries** (copyright, Reality Bytes 1995): + `*.EXE *.DLL *.FF *.DAT *.MAP *.INI *.cue *.bin`, `BIGFILE.DAT`, `WORLDS/`, `tmp_cd/`, the + Ghidra install/project, and derived audio/art are all in `.gitignore`. When staging, use + explicit paths (not `git add -A`) and sanity-check `git diff --cached --name-only`. +- **No emdashes in any output** (user style preference). Use plain hyphens. +- Document discovery process for the blog (see section 1). + +--- + +## 5. The binary: layout and key addresses + +`HAVOC_NOCD.EXE` = patched `HAVOC.EXE`, 485,376 bytes, 32-bit PE i386, image base `0x400000`, +loads at preferred base (so runtime VAs == static VAs). + +**Section file-offset (FO) to virtual-address (VA):** + +| Section | Raw FO | RVA | FO -> VA | +|---|---|---|---| +| `.text` | 0x00400 | 0x01000 | `VA = FO + 0x400C00` | +| `.rdata` | 0x67000 | 0x79000 | `VA = FO + 0x412000` | +| `.data` | 0x69400 | 0x7C000 | `VA = FO + 0x412C00` | +| `.idata` | 0x6EC00 | 0x82000 | `VA = FO + 0x413400` | + +**Control-flow map (verified this session):** + +| VA | What it is | +|---|---| +| `0x464657` | CRT entry point | +| `0x407C90` | WinMain. Allocs game object (vtable `0x4791A0`), calls `0x434040` (init); on success calls `vtable[7]` = `0x43AC40` (main loop) | +| `0x434040` | Main init: allocates ~30 subsystem objects, loads title/GAMIMAGE, palette fade. Calls `0x418600`. Returns AX=0 on success | +| `0x418600` | DirectDraw + archives + sound init. **GRAFIX/world files load during this** | +| `0x43AC40` | **MAIN GAME LOOP** (vtable[7]). State machine; state byte at `[this+0x18]`; loops while `0x10 <= state < 0xf0`. Per-frame calls `vtable[8]=0x436170`, `vtable[17]=0x43adc0` (msg pump), `vtable[9]=0x4361d0`. Exits when state = 0xf0. **No frame limiter here = FPS-coupled** | +| `0x4791A0` | Game object vtable (0x14-entry) | +| `0x42a480` | WORLDS\ data-file loader (the copy-protection function, now patched at FO 0x29a2b) | +| `0x439380` | Stream open method: `CreateFileA([this+4], ...)`, vtable `0x479af8` | +| `0x444db0` | GRAFIX0N.DAT loader (byte-swaps BE records; template for reading other WORLDS loaders) | +| `0x40C8E0` | Busy-wait frame limiter (~60/sec via timeGetTime), used in palette-fade / state transitions (`0x40CAE0`), NOT in the gameplay loop | +| `0x40b5b0` | Win32 WndProc dispatch (msg 2 WM_DESTROY -> `0x40b680` sets state=0xf0 + PostQuitMessage; focus msgs 7/0x1c handled) | +| `0x409FC0` | DirectSound init | +| `0x42D970` | show_error_dialog (patched to `RET` so failures do not popup) | +| `0x4824E4` | timeGetTime IAT; `[0x47e56c]` holds a 60Hz tick baseline (set once, not used as a per-frame delta) | + +Reproduce any disassembly with r2: +```sh +python3 -c "import r2pipe; r=r2pipe.open('HAVOC_NOCD.EXE',['-e','bin.relocs.apply=true']); print(r.cmd('pd 60 @ 0x43ac40'))" +``` + +--- + +## 6. File formats: status + +Cracked (see `docs/FORMATS.md` for specs and `tools/` for extractors): +- `.FF` "FlashFile" archive (`tools/ff_extract.py`). +- MUSIC.FF: 97 WAV segments + 17 PLST-sequenced songs + FL Studio pack. +- INTRFACE.FF: bitmap system (BitB/BitR headers, XPAL palettes, STR# string tables). +- **GRAFIX0N.DAT** (`tools/grafix_extract.py`): BE header (size@0x08, count@0x0c), 16-byte entry + records (BE offset + id), per-graphic blocks (BE w/h/frames/id + 8-bit indexed pixels). 64x64. + +TODO (the asset-cracking stream): +- **MAP.MAP**: 2-byte-per-cell grid, ~192x144 confirmed. Next: disassemble its loader to get + exact dims + which byte is tile vs height. Find the loader by searching for the code that opens + "MAP" filenames (same pattern as GRAFIX via `0x42a480`). +- **STUF.DAT**: entity placement. Contains ASCII `"Item"` records - very parseable. Start + here for a quick win. +- **LAND.DAT**: terrain geometry (hardest). BE fields; music-track id is a BE u16 at offset + 0x12 (per prior notes). Has 4-byte records around 0x20 (`17 70 00 c0` ...). +- **OBJECTS.FF**: 3D models (vehicles, enemies, pickups) - not yet touched. +- **World palette:** GRAFIX tiles currently render with a UI XPAL palette. The true per-world 3D + palette is TBD - likely loaded during `0x418600` init or embedded per-world. Finding it makes + all extracted art color-correct. + +--- + +## 7. Ghidra decompilation stream (primary Linux task) + +Load `HAVOC.EXE` (or `HAVOC_NOCD.EXE`) into Ghidra (x86 32-bit, image base 0x400000). Import the +key addresses from section 5 as labels to bootstrap. High-value targets, roughly in order: + +1. **The FPS-coupling / render modes.** The user reports cockpit mode is FPS-coupled but the + minimal HUD mode is not. Find the mode toggle and the two render/update paths. Determine + whether minimal mode uses a frame limiter (Sleep/timeGetTime) or a fixed-timestep/delta update. + That path is the template for the remaster's fixed-timestep, and possibly a cleaner in-binary + fix than the dgVoodoo FPS cap. Start from the main loop `0x43AC40` and its per-frame methods + `0x436170` / `0x4361d0`, and the busy-wait limiter `0x40C8E0`. +2. **Cheat code handler.** Codes `mmm` (max heatseeker), `vvv` (free vehicle), `sss` (shield), + `hhh` (health), `aaa` (ammo). Find the keyboard-buffer matcher; it will reveal the full cheat + list and the player/inventory structs. +3. **Weapon system.** 6 primary classes (Laser, Disc, Shockwave, Fireball, Plasma, VARG) infinite + ammo; 14 secondary lines (see `docs/WEAPONS.md`, from string tables STR#7D02/7D03). Recover the + weapon data tables, upgrade tiers, and firing logic. +4. **Player / vehicle physics** (movement constants, the FPS-coupled step) and **enemy AI**. +5. **Level + entity loaders** (MAP/LAND/STUF) - decompiling these also directly finishes the + asset-cracking stream (get exact record layouts instead of guessing). +6. **World table**: 6 worlds stored in reverse (Overlord, Wasteland, Malterra, Tyrak, Fallout, + Badlands), 3 levels + boss + bonus each. See `docs/FORMATS.md` and game-mechanics notes. + +--- + +## 8. Methodology playbook (how the cracking was done) + +- **Crack a format via its loader, not guesswork.** Find where the game opens the file + (filenames are in `.data`; the loader `0x42a480` builds `WORLDS\` paths), then read the + loader's parse code. GRAFIX was solved by reading `0x444db0` (the 16-byte record stride and + BE->LE byte-swap gave the record layout exactly). +- **These WORLDS files are big-endian** (Mac-origin data; the loaders byte-swap). Always try BE + first for headers/records. +- **Find grid dimensions** by row autocorrelation (minimize mean abs diff between byte j and + j+stride over candidate strides); then render grayscale at that width and eyeball for coherent + structure. That is how MAP's 384-byte row (192x2) was found. +- **Verify visually.** Render extracted pixel data to PNG and build contact sheets; correct + decodes look like real art. Pillow `Image.frombytes('P', (w,h), data); putpalette(...)`. +- **r2pipe pattern** for scanning: open with `bin.relocs.apply=true`, iterate `.text` FOs looking + for opcode patterns (e.g. `ff 15` CALL [IAT], `a1`/`8b 05` mem reads) and convert FO->VA. + +--- + +## 9. Tool inventory (`tools/`) + +- `ff_extract.py` - `.FF` archive extractor (archives, songs, FL pack). `--help` for flags. +- `grafix_extract.py` - GRAFIX0N.DAT -> PNG (`-p <768-byte palette>`). +- `dbg_path.py` - Windows-only ctypes debugger (used for the boot crash; keep for reference). +- Many `r2*.py` / `find_*.py` / `analyze*.py` - one-off RE probes from the crash hunt and format + work. Useful as r2pipe examples; not all are load-bearing. +- `run_probe.bat` / `run_and_log.bat` - Windows launch helpers (repro/testing on Windows only). + +Palettes: extract with `python3 tools/ff_extract.py INTRFACE.FF -o tools/out_intrface`, palettes +land in `tools/out_intrface/raw/XPAL*.DAT` (768-byte RGB). + +--- + +## 10. Immediate next steps (suggested) + +1. (Asset, quick win) Crack `STUF.DAT` entity records (ASCII `"Item"` markers). +2. (Asset) Disassemble the MAP loader to lock MAP dims + cell meaning; render a proper level map. +3. (Ghidra) Stand up the project, label the section-5 addresses, and chase the render-mode / + FPS-coupling path (item 7.1) - it informs both the exe fix and the Godot physics. +4. (Remaster) Once STUF + MAP + a world palette are done, there is enough to scaffold the Godot + project (empty today) and load a real level. + +The FPS workaround (dgVoodoo `FPSLimit = 30`) is a stopgap; the user is tuning the value. A proper +fix is either a frame limiter patched into `0x43AC40` (the game imports `Sleep`) or, better, +adopting the minimal-mode timing path once Ghidra reveals it.