From 534c41af73fc21d0ac82fcfdee62b1cb81d3474b Mon Sep 17 00:00:00 2001 From: pyr0ball Date: Mon, 29 Jun 2026 00:29:50 -0700 Subject: [PATCH] docs: add full RE methodology for blog post Documents the process (not just the results) for each format cracked: - Binary identification: why 32-bit PE not 16-bit, what actually broke - .FF archive discovery: speculative parsing from file count heuristic - MUSIC.FF playlists: big-endian payload in LE container, segment ID math - INTRFACE.FF bitmaps: BitB/BitR/XPAL system, index-2 black patch discovery Includes tools, dead ends, corrections (2D->3D), and what's next. Co-Authored-By: Claude Sonnet 4.6 --- docs/METHODOLOGY.md | 189 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 189 insertions(+) create mode 100644 docs/METHODOLOGY.md diff --git a/docs/METHODOLOGY.md b/docs/METHODOLOGY.md new file mode 100644 index 0000000..84ab1f1 --- /dev/null +++ b/docs/METHODOLOGY.md @@ -0,0 +1,189 @@ +# Havoc RE Methodology: How We Cracked the Formats + +This document records the *process* of reverse-engineering Havoc (Reality Bytes, Inc., 1995), +including the wrong turns, so the blog post can tell the full story. + +--- + +## 1. Starting point: what do we have? + +The game files had been sitting on a network share untouched. The top level: + +``` +HAVOC.EXE 485 KB +LAUNCHER.EXE 17 KB +SETUP.EXE 110 KB +BIGFILE.DAT 64 MB +MUSIC.FF ~6 MB +INTRFACE.FF ~12 MB +OBJECTS.FF ~2 MB +SOUND.FF ~1 MB +WORLDS/ terrain data +DIRECTX/ bundled DirectX 5 +AUTORUN.INF OPEN=LAUNCHER.EXE +.windows-serial contains "0" +``` + +The first question: **why did it stop running?** The common assumption for old Windows games +is "16-bit" or "DOS". That assumption was wrong. + +--- + +## 2. Identifying the binaries (correcting the 16-bit assumption) + +Approach: read the PE header directly in Python rather than trusting assumptions. + +Every Windows executable starts with an MZ stub. At offset `0x3C` is a uint32 pointing to +the PE signature. We read that offset for all three EXEs: + +``` +HAVOC.EXE PE offset = 0x80 +LAUNCHER.EXE PE offset = 0x80 +SETUP.EXE PE offset = 0x80 +``` + +At `0x80` in each: `50 45 00 00` = `PE\x00\x00`. Confirmed 32-bit PE. The Machine field +(`0x014C`) = i386. The Magic field (`0x010B`) = PE32 (not PE32+). Subsystem = 2 (GUI). + +**Key finding:** these are 32-bit Win32 applications. They will run under WOW64 on 64-bit +Windows today. The game itself is not the problem. + +The import table confirmed the real story: `DDRAW.dll`, `DSOUND.dll`, `WINMM.dll`, +`WSOCK32.dll`, `GDI32.dll`, `USER32.dll`, `KERNEL32.dll`. Notably: **no Direct3D at all**. + +**What actually broke:** the bundled `DIRECTX/` folder ships `DSETUP16.DLL` and `DDRAW16.DLL` +-- genuine 16-bit components used by the installer path. Those cannot run on 64-bit Windows. +But the game doesn't need the installer to run once files are in place. + +The second problem: legacy DirectDraw palettized fullscreen (Mode X / 8-bit indexed) is +unreliable on modern GPUs without a compatibility wrapper like dgVoodoo2. + +**Correction during process:** we initially described Havoc as "2D". Looking at the +screenshots on myabandonware corrected this -- it is a first-person software-rendered 3D +vehicular combat game. The software rasterizer blits into a DirectDraw surface, explaining +why there is no Direct3D import despite full 3D gameplay. + +--- + +## 3. Identifying the archive format (.FF files) + +The `.FF` files are the main asset containers. No magic signature at the start, so we +read the first few bytes as a uint32: `116` for `MUSIC.FF`. That looked like a file count. + +We then parsed speculatively: if byte 4 is an offset and the following bytes are a NUL- +terminated filename, does it make sense? + +```python +count = struct.unpack_from(" +``` + +Finding `FFCREATE.EXE` (a DOS MZ binary) embedded inside the archive was a nice confirmation: +this was the tool Reality Bytes used to *build* the `.FF` files, shipped alongside the data it +created. The name "FF" almost certainly stands for "FlashFile." + +Gotcha: the last TOC entry may have an empty name and offset pointing past the end. Skip it. + +--- + +## 4. Cracking MUSIC.FF: the playlist format + +`MUSIC.FF` contains 97 `WAVE*.DAT` entries and 17 `PLST*.DAT` entries, plus `FFCREATE.EXE`. + +The WAVE entries were straightforward: `RIFF` signature, standard PCM WAV, mono 22050 Hz +8-bit. Directly playable with any audio tool. + +The PLST (playlist) entries were more interesting. Initial read (little-endian): + +``` +first two uint16s of PLST07D1: 0x0004, 0x0014 = (4, 20) +``` + +A 4-segment song with 20 playback steps. Then 20 uint16s for the sequence. But when we +tried to unpack them little-endian, the step count came out as `10240` -- clearly wrong for +a 44-byte file. **The PLST payload is big-endian** despite the `.FF` container being +little-endian. Swapping to `">HH"` gave the correct values. + +Segment resolution: a playlist `PLST07D1` references segments by 1-based index into a group. +First attempt mapped by the last hex digit of the filename (`07D1` -> segment 1, `07D2` -> 2). +This failed for multi-digit segment counts. Correct rule: **step N maps to wave ID +`base_id + N - 1`**, where `base_id = int("07D1", 16) = 2001`. Step 2 -> wave ID 2002 = +`WAVE07D2`. Verified by assembling all 17 songs and confirming they sound correct. + +--- + +## 5. Cracking INTRFACE.FF: the bitmap system + +`INTRFACE.FF` (547 entries) contained several entry types by name prefix: + +| Prefix | Count | Hypothesis | +|---|---|---| +| `BitR` | 239 | "Bit Raster" -- raw pixel data | +| `BitB` | 239 | "Bit header/Border" -- paired with BitR by same hex ID | +| `XPAL`/`xpal` | ~47 | "eXtended PALette" -- 768-byte palette = 256 * RGB | +| `STR#` | 16 | String tables | +| `RTbl` | 3 | "Remap Table" (function TBD) | +| `FFCREATE.EXE` | 1 | Archive builder | + +The `BitB` entries were all exactly 4 bytes. Reading as two big-endian uint16s gave +dimensions: `(48, 48)` for `BitB4E21`. Checking `BitR4E21`: 2304 bytes = 48 * 48. Every +single BitB/BitR pair (239 pairs) matched exactly. Format confirmed. + +The `BitR` data is raw 8-bit palette-indexed pixels, row-major, no header. The `XPAL` files +are 768 bytes = 256 RGB triplets, 8-bit per channel (NOT 6-bit VGA -- we verified by +checking that ~67% of values exceed 63). + +**The index-2 black patch (key discovery):** + +First renders of most screens showed a salmon/pink background (#e78484) instead of the +expected dark industrial look. One screen, `7A12`, rendered correctly with a dark background. + +Comparing palettes `xpal7A12` vs `xpal7A1C` (same screen layout, different background color): +only 3 of 256 palette entries differed -- indices 0, 1, and 2. In `7A12`: index 2 = `#000000` +(black). In `7A1C`: index 2 = `#e78484` (salmon). **The game engine forces palette index 2 to +black at runtime.** The XPAL files store development/authoring-time values; the runtime +patches index 2 before display. One-line fix: `pal[6], pal[7], pal[8] = 0, 0, 0` after +loading any XPAL. + +Assets extracted from INTRFACE.FF include: 32 full-screen menu/UI backgrounds, cockpit +windshield frame layers (three color variants), vehicle selection screens with pre-rendered +3D vehicle thumbnails, cockpit side panel strips, menu button strips in multiple states, +and 57 vehicle HUD icons in the game's monochrome green HUD style. + +The "minorly off" colors in the renders vs reference screenshots are consistent with +CRT gamma -- the palette values were tuned for a 1995 CRT display (~2.2 gamma) and display +directly on a modern LCD without gamma compensation. This is a Godot shader concern, not +an extraction accuracy issue. + +--- + +## 6. Tools used + +- **Python 3** (stdlib only for format parsing: `struct`, `os`, `collections`) +- **Pillow** (`PIL`) for bitmap rendering and PNG export +- **ffmpeg** (via scoop on Windows) for WAV to OGG/MP3 conversion +- **No disassembler yet** -- all format work done from data patterns alone +- **Ghidra** (planned) for game logic / binary RE + +--- + +## 7. What's next + +- `SOUND.FF`: same format as MUSIC.FF, just raw SFX WAVs (no playlists) +- `WORLDS/`: terrain data -- `MAP*.MAP`, `LAND*.DAT`, `GRAFIX*.DAT`, `STUF*.DAT` +- `OBJECTS.FF`: 3D models -- the hard one; `ASND*` entries TBD +- `BIGFILE.DAT`: 64 MB bulk data pool (starts with copyright string then RIFF data) +- Ghidra decompilation of `HAVOC.EXE` for game logic, AI, cheat codes, weapon system