headless: split --timeout into --timeout-wall and --timeout-emulated

--timeout was wall-clock seconds, which is what CI wants but not what you want
when the question is whether the game has had long enough to get somewhere: a
heavy scene runs many times slower than real time and a near-idle one much
faster, so the same budget means very different amounts of game time. Booting a
firmware VSH is a good example - 10 emulated seconds is about 25 real ones on
6.61 and about 7 on 2.00, and judging those two by the same wall-clock number
makes a working shell look stuck.

Both limits can be set at once and whichever is reached first ends the run,
which also says which one it was. --timeout still works as the old name for
--timeout-wall. The IsDebuggerPresent() exemption stays on the wall-clock check
only; the emulated one doesn't need it, since sitting at a native breakpoint
burns no emulated time.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Henrik RydgårdandClaude Opus 5 committed 2026-09-17 16:01:47 -06:00
1 parent ce42033686
commit e8fa4e3f56
7 files changed
+78 -25

No files matched your search

+14 -6
View File
@@ -47,16 +47,24 @@ platforms. Where arch is x64 or ARM64.
A working invocation, and the traps around it:
```bash
./Windows/x64/Debug/PPSSPPHeadless.exe -i --debugger=34567 --timeout=100000 --graphics=software --log \
./Windows/x64/Debug/PPSSPPHeadless.exe -i --debugger=34567 --timeout-wall=100000 --graphics=software --log \
--root pspautotests/tests/../ pspautotests/tests/cpu/cpu_alu/cpu_alu.prx > hl.log 2>&1 &
# wait for "Listening on port" in hl.log, then:
./Tools/wsdbg/target/release/wsdbg.exe 34567 --sync --sync-timeout 15 < script.txt
```
- **`--timeout` is wall-clock seconds for the whole session**, not per test - the default is infinity, but as soon as
you pass one it applies to your whole interactive debugging session too. Pass something huge (`--timeout=100000`);
otherwise the process prints `TIMEOUT` and exits out from under you mid-session. (There's an escape hatch: the
deadline check is skipped while `IsDebuggerPresent()`, i.e. under a native debugger.)
- **The timeouts apply to the whole session**, not per test. `--timeout-wall=N` is real seconds and
`--timeout-emulated=N` is emulated ones; both default to infinity, both can be set at once, and whichever
is reached first ends the run (`TIMEOUT` or `TIMEOUT (emulated)`). `--timeout` is the old name for
`--timeout-wall`. As soon as you pass one it applies to your whole interactive debugging session too, so
pass something huge (`--timeout-wall=100000`) or the process exits out from under you mid-session. The
wall-clock check is skipped while `IsDebuggerPresent()`; the emulated one doesn't need that escape hatch,
since sitting at a native breakpoint burns no emulated time.
- **Which one you want depends on the question.** Wall-clock is what stops a hang from hanging the machine.
Emulated is what you want for "has the game had long enough to get somewhere" - a heavy scene runs many
times slower than real time and a near-idle one much faster, so the same wall-clock budget means very
different amounts of game time. Booting a firmware VSH to its XMB is a good example: 10 emulated seconds
is about 25 real ones on 6.61 and about 7 on 2.00.
- **Prefer `--debugger=0` and scrape `Listening on port N` from that run's own log** over hardcoding a port. Also
`taskkill //F //IM PPSSPPHeadless.exe` between runs for hygiene (Git Bash here has no `pkill`) - leftover
instances are easy to accumulate when a script leaves the CPU stopped at a breakpoint.
@@ -112,7 +120,7 @@ A working invocation, and the traps around it:
- **Exception and crash messages do not reach the log in headless.** It registers its own debug-output listener
(`SendDebugOutput` in `headless/Headless.cpp`) that `fwrite`s to stdout, which is block-buffered when you
redirect it to a file - so the output sits in the CRT buffer while the process runs, and `taskkill //F` throws
it away rather than flushing. To actually read a crash trace, give that run a short `--timeout` and `wait` for
it away rather than flushing. To actually read a crash trace, give that run a short `--timeout-wall` and `wait` for
the process to exit on its own.
- **`0xFFFFFFFF` is not an invalid instruction** - it decodes to `vflush`, a real Allegrex VFPU op, so writing it
over code to test illegal-instruction handling just runs it. Check what an encoding actually is with
+4 -3
View File
@@ -45,7 +45,7 @@ Make sure Python is available. On Windows the Microsoft Store alias may interfer
### Direct headless invocation
```
Windows/x64/Debug/PPSSPPHeadless.exe --root pspautotests/tests/../ --compare --timeout=5 --graphics=software pspautotests/tests/audio/atrac/addstreamdata.prx
Windows/x64/Debug/PPSSPPHeadless.exe --root pspautotests/tests/../ --compare --timeout-wall=5 --graphics=software pspautotests/tests/audio/atrac/addstreamdata.prx
```
Instead of a single PRX, you can pass a directory (e.g. `pspautotests/tests/threads/mbx/...`) to run all tests under it, recursively.
@@ -53,7 +53,8 @@ Instead of a single PRX, you can pass a directory (e.g. `pspautotests/tests/thre
**Key flags:**
- `--root` — points to the directory above `tests/` so the headless can find the expected directory layout.
- `--compare` — enables output comparison against `.expected` files.
- `--timeout=N` — seconds per test before killing it (default 5).
- `--timeout-wall=N` — real seconds per test before killing it (default 5). `--timeout-emulated=N` is the
same idea in emulated time, and both can be set at once. `--timeout` is the old name for `--timeout-wall`.
- `--graphics=software` — uses software GPU backend (required for headless; no real GPU available).
### What you'll see
@@ -138,7 +139,7 @@ returns. Don't go hunting for a wrong value; there isn't one.
- The diff output compares the full text output line-by-line. To see PPSSPP's raw output
without the diff overlay, omit `--compare`:
```
Windows/x64/Debug/PPSSPPHeadless.exe --root pspautotests/tests/../ --timeout=5 --graphics=software path/to/test.prx
Windows/x64/Debug/PPSSPPHeadless.exe --root pspautotests/tests/../ --timeout-wall=5 --graphics=software path/to/test.prx
```
There's another trick too, --print-equal-lines, which prints matching lines with a '=' prefix, so you can see the full output with context.
- Tests can show contradictory expected outputs at first glance. For example, the mbx/send