The game screen already reports which updater a disc carries, but there was no
way to act on it - the only route to installing one was picking an updater PBP
out of the browser. The unpacker already handles being pointed at a disc, so
this just wires the menu entry to it.
Shown only when the disc actually has one, and not while a game is running:
installing wipes the NAND that game has mounted.
InstallUpdateScreen takes the archive size now, because for a disc the size of
the file it was handed is the game's and says nothing about the firmware.
Co-Authored-By: Claude Opus 5 <[email protected]>
Backwards from 5.01, one release at a time, against all 39 versions that ship
on a disc plus the download-only 6.61. Nothing here is an offset - it's almost
entirely Sony renumbering the kernel *_driver NIDs, which sends an import we
mean to HLE into the real firmware module instead.
Four more NIDs each for sceRtc_driver/sceRtcSetAlarmTick and
sceHprm_driver/sceHprmReadLatch, covering 1.50 up. The rtc one is what parked
every thread on a SceSysconSync semaphore; the hprm one runs once a frame, so
unresolved it was most of the boot log. Also sceImposeGetParam/sceImposeChanges
(1.50 - 2.xx) and sceKernelLoadModuleVSH (1.x, which is how the shell loads its
own plugins - unresolved it got module id 0 and StartModule failed).
sceRtcIsAlarmed had to be implemented too; it returns 0, as in JPCSP. As a null
entry it returned LIBRARY_NOT_YET_LINKED, and the 3.0x-3.5x VSH read that as
"ask the hardware instead" and went back to blocking on syscon.
Two structural findings:
- Up to 4.05, scePaf's heap allocator is a separate heaparea1.prx that paf
imports as scePafHeaparea. Load it when it's there. Its pool pointer needs
the same pre-fill paf's does, at gp - 0x7FCC rather than gp - 0x7E88.
- 1.50's vshmain.prx declares no module attributes at all - PSP_MODULE_VSH_MODE
only appears from 1.52 - so the whole VSH bootstrap was being skipped. Accept
the module name too.
Co-Authored-By: Claude Opus 5 <[email protected]>
The blocker below 6.60 wasn't offsets, it was that Sony renumbered the kernel
*_driver NIDs between versions. A function we HLE under its 6.6x NID is a
stranger on an older build, so the import lands in the real firmware module
instead - and that's where it goes wrong:
- sceRtc_driver sceRtcSetAlarmTick. Without the HLE the VSH's alarm call ran
the real rtc.prx, which called on into syscon.prx and blocked forever on a
SceSysconSync semaphore. That was the whole "stalls with every thread parked"
symptom; the tell was a fourth SceSysconSync waiter a healthy boot lacks.
- sceHprm_driver sceHprmReadLatch, called once a frame - so before this an
older firmware's 12-second boot logged ~20000 lines of one unresolved import.
Three extra NIDs each, found by disassembling the module from both firmwares
and matching on the address of the user-mode export whose NID never changed
(sceRtc/0x7D1FBED3, sceHprm/0x40D2F9F0).
5.55 additionally needed two PRX decryption keys we didn't have (0x4C941AF0
and 0x4C941BF0) - without them none of flash0:/kd decrypted and the shell came
up with no drivers behind it at all.
Checked one release at a time against every version that ships on a disc, plus
6.61. 4.05 and below still die on a null write inside vsh_module.
Co-Authored-By: Claude Opus 5 <[email protected]>
With no kernel modules there's no firmware to speak of - it's the fonts we
pulled off a game's disc - and rows reading "Kernel modules: 0 / XMB: No"
say nothing. Keep the font count and the size.
Co-Authored-By: Claude Opus 5 <[email protected]>
6.60 ships byte-identical paf.prx and vshmain.prx to 6.61 - all 6338 + 669
functions disassemble the same - and boots to an interactive XMB, so let
FirmwareVersionSupportsVSH accept it. That matters because no UMD carries 6.61
(it was download-only), so 6.60 is the best a disc-installed firmware can be.
The two module patches were hardcoded offsets from the module base applied to
any module of the right name, which is quietly wrong on any other build:
- The scePaf heap arena slot moves with every build (0x18CCD8 on 6.00 through
0x18D728 on 6.60/6.61) but sits at gp - 0x7E88 in all of them, so find it
that way. On its own this turns an immediate SIGSEGV inside scePaf into a
clean stall on 6.00 through 6.39 - they still don't reach an XMB, they get
stuck in sceVshBridge_Driver instead.
- The vsh_module alarm-category offset has no such anchor, so check the word
there is the one the patch was derived from. On 6.20 and 6.00 it's ASCII
string data - the unconditional write was corrupting a string table.
Also resolve the per-model kernel drivers (memlmd, loadexec, wlanfirm) to the
model being emulated. They were asked for as _01g, which a firmware unpacked
for a single model doesn't have - and our own updater unpack defaults to 02g.
Co-Authored-By: Claude Opus 5 <[email protected]>
KIRK CMD1 writes header + data_offset + align16(data_size) bytes into outbuf,
and all three come out of the header the decrypter just decrypted, not from the
caller. The SHA1 check doesn't bound them - it only covers the header, so it
passes just as happily for a block that's been cut short.
The PSAR walker has to guess how long an updater's second block is (nothing
records it, so it tries the sizes real updaters use), and a wrong guess sent
KIRK off the end of the buffer: unpacking a firmware crashed roughly half the
time, on every version and disc I tried, depending on the heap layout.
Bound the write against the size the caller gave us, in all six decrypt types.
Co-Authored-By: Claude Opus 5 <[email protected]>
Shows what's actually in PSP/NAND - nothing, a fonts-only partial install, or
a full firmware with its version, build date and region - read from
flash0:/vsh/etc/version.txt, which is present both in a PSAR-unpacked install
and a NAND dumped off hardware.
Also offers to install an updater, erase the NAND, and launch the XMB, the
last one gated on FirmwareVersionSupportsVSH() since the module patches that
get vshmain.prx running are tied to 6.61's offsets.
Installing a firmware now erases flash0/flash1/ipl first - two firmwares can't
be merged, a file the new one doesn't have would linger and still get loaded.
Co-Authored-By: Claude Opus 5 <[email protected]>
An ME image is two KL4E streams back to back: the code that runs at
0x88300000, then a blob the image decompresses to ME local RAM at
0x00101000, which is where its data segment lives. Decompressing only the
first left every global unaccounted for.
--re-decrypt now reports how much of the plaintext the first stream used
and writes the remainder to <out>.tail, and accepts an already-plain
KL4E/KL3E file so that tail can be fed straight back in.
--re-decrypt runs pspDecryptPRX() over a file and unpacks the KL4E/KL3E
stream behind it. This opens up flash0:/kd/resource/*.img, the images the
Media Engine actually runs: they are ordinary tagged containers (tag
862648D1, which PrxDecrypter already has a key for) with the ~PSP
signature blanked, so the normal module loader never touches them.
--re-raw-base analyzes --re-module as a flat code image at a given
address rather than as a PRX. The decrypted ME images are raw MIPS with
no ELF around them; the address they were linked for is recoverable from
their own jal targets (0x08300000 for meimg.img).
Also makes PrxDecrypter.h self-contained - PSP_Header is built from _le
types, so it needs Common/Swap.h rather than relying on the includer.
The channel took only 256, 1024 and 2048 samples, so a game that hands it MP3
frames got SCE_KERNEL_ERROR_INVALID_SIZE and no music. Dead or Alive Paradise
does exactly that from its music player: sceVaudioChReserve(1152, 44100, 2),
1152 being the MPEG-1 Layer III frame size.
The format check moves below the sample count check to match: the module
returns 0x80000104 for a bad count before it ever looks at the format, so a
call with both wrong got the wrong error out of us. The two error codes we
already returned are the ones it uses.
Co-Authored-By: Claude Opus 5 <[email protected]>
PPSSPP decided whether a caller was privileged with hleIsKernelMode(), which reports whether the
syscall being executed is itself a kernel-only export. That's a different question from the one
the hardware answers: on a PSP the privilege belongs to the calling module, and a kernel module
reaches sceKernelCreateTlspl through the ordinary ThreadManForUser NID like anything else. So a
kernel module asking for partition 1, 3 or 4 got ILLEGAL_PERM where a real PSP hands it over,
which the new threads/tls/kernel/partition test shows directly.
BlockAllocatorFromID now also accepts a caller whose thread belongs to a kernel module, via a new
__KernelCurThreadIsKernelMode(). It checks the thread's own attribute first and then the owning
module, because a kernel module's main thread isn't necessarily flagged kernel - the attribute
comes from PSP_MAIN_THREAD_ATTR, which needn't set it. That mirrors how sceKernelCreateThread
already works out allowKernel.
This only ever widens access, and only for threads belonging to kernel modules, so games are
unaffected - they run in user modules and see exactly what they saw before.
threads/tls/kernel/partition now records the sweep from a kernel module, which settles the range
question the user-mode recording couldn't: privilege changes the permission check, not the range.
Partitions 1, 3 and 4 are ILLEGAL_PERM from user mode and fine from kernel mode, while 7 and up
are ILLEGAL_ARGUMENT either way. So the check goes back to a plain 1-6 for both, and the
kernel-mode carve-out from the last commit - which would have let 8 and 9 through - is gone.
The hardware doc gains a section on kernel-mode tests: what COMMON_KERNEL does, why the stock
crt0 makes a kernel PRX unloadable, which libraries can't be imported, and how much room there
actually is in the kernel partition.
sceKernelUtilsSha1Block* had the same single global context that MD5 did, so it gets the same
treatment: state, counters and block buffer now live at ctxAddr in the layout hash/sha1ctx
records off hardware. Unlike MD5, SHA-1 does not stream whole blocks through buf, which happens
to be what our sha1_update already does - so no fill-in step is needed there.
The Tlspl partition range from the last commit was too broad a cut. Hardware says only 1-6 exist,
but that recording is from user mode, and BlockAllocatorFromID deliberately maps 8 and 10 to the
user partition for a kernel-mode caller - rejecting them outright would have taken that away.
The tightened range now applies to user mode only and kernel mode keeps what it had.
threads/tls/partition also shows the answer doesn't depend on the compiled SDK version, checked
across 1.00 through 6.06, and that partition 5 is accepted - which no test had covered.
Three fixes, all of them things the new hardware tests turned up.
sceMd5Block* and sceKernelUtilsMd5Block* shared one static md5_context and ignored the context
pointer the caller passed in, with a TODO saying it would do "unless games do several MD5
concurrently". hash/md5ctx shows a real PSP keeps everything in the caller's 96 bytes and happily
runs two digests at once, so do that instead: the state, the counters and the block buffer now
live at ctxAddr in the game's own memory, in the layout the test pins down. Two interleaved
digests come out right, and a context that gets copied mid-digest carries on correctly. As a
side effect the state is now covered by savestates, which a file-static never was.
MersenneTwister masked both halves with 0x80000000 where the low half needs 0x7FFFFFFF, so
sceMt19937UInt and sceKernelUtilsMt19937UInt were returning a sequence that isn't MT19937 at
all - every number differed from hardware from the first draw. hash/mt19937ctx computes the
reference sequence itself and confirms the PSP is plain MT19937; with the mask fixed we match it
for both seeds tested. Init also twists the array immediately, as hardware does, so a context
that has been seeded but not drawn from now holds what a real one would.
sceKernelCreateTlspl accepted partitions up to 9 before falling through to the permission check.
Hardware draws the line at 6 - threads/tls/create records 7, 8, 9 and 10 all returning
ILLEGAL_ARGUMENT - so 8 and 9 were coming back ILLEGAL_PERM. Note this is genuinely different
from sceKernelCreateVpl right above it, which does let 8 and 9 through to ILLEGAL_PERM; the two
had been sharing a check that was only ever right for Vpl.
Risk worth naming: the MT19937 change alters the numbers any game gets from these calls. That's
the point - they were wrong - but a savestate taken mid-sequence will resume with a generator
that behaves differently from the one that made it.
Every test directory builds under pspdev GCC 15 again, so gentest.py no longer
aborts on a neighbour's compile error before it reaches the PSP. No .prx was
regenerated, so the suite behaves exactly as before - 319/319 still pass.
Also updates the hardware doc: the "several tests don't build" workaround is
gone, and it now records what rebuilding a .prx actually costs (64-bit time_t
changes what rtc/convert tests), that host0: differs per host OS, and that
PRXs stay resident so you need a reset between runs.
Not every disc image is a whole number of 2048-byte sectors - tools that build
pre-patched ISOs write images that stop partway through their last one, with a
file legitimately ending there. Two things then conspired to lose that tail.
FileBlockDevice::GetNumBlocks() rounds down, so the partial sector isn't
counted, and the file size clamp in ISOFileSystem measured what the image holds
in whole blocks. A file running to the last byte of such an image got clamped
short - by up to a sector - before anything read it.
FileBlockDevice::ReadBlock() then returned false for a short read of that
sector, and ISOFileSystem::ReadFile substitutes an all-zero sector when a read
fails, so even the bytes that were there came back as zeroes.
Measure the clamp in bytes via GetUncompressedSize() instead of blocks, and
treat a short read at the end of the image as a success with the rest of the
sector zeroed. GetUncompressedSize() defaults to the block-based value and is
only overridden by FileBlockDevice, so nothing else changes behaviour.
Also report why a module was rejected. "Failed to load module" named the file
and nothing else, and the truncation check logged only the byte count, which
points at the executable when the real cause is that the loader was handed
fewer bytes than the file has. ElfReader now keeps the reason for a failed
LoadInto, __KernelLoadELFFromPtr puts it in the error string that reaches the
user, and both messages say which header table overran and by how much.
Co-Authored-By: Claude Opus 5 <[email protected]>
We had a doc for running the existing tests against headless, but nothing on
the other half - bringing up PSPLink and usbhostfs_pc, what gentest.py does,
and how to get an .expected out of real hardware. Write that down, including
the parts that cost time to rediscover: usbhostfs_pc's working directory is
host0:/ so it has to start in the pspautotests root, gentest.py makes the
whole test directory and several old tests no longer build under pspdev's
GCC 15 (use -k), rebuilding a .prx with a newer toolchain balloons it, and
host0: is not FAT so anything testing FAT semantics needs ms0:.
Also adds the io/shortname test the doc uses as its worked example. It stays
in tests_next: hardware preserves the case of d_name where we uppercase it,
and appends ~1 to the short name of anything that isn't already valid
uppercase 8.3 where we only do that on a collision.
threads/tls/create moves to tests_next as well. It's collateral from the
submodule bump - upstream 1dcefeb regenerated its .expected on a PSP with
less free memory, so allocations at 1MB and above now expect failure, and
partitions 8 and 9 now expect 800200D2 where we return 800200D1.
kirk_engine.h and amctrl.h guard their declarations, but AES.h and SHA1.h
never did, and kirk_engine.h includes them from outside its own guard. So the
AES_* and SHA1* functions got C++ linkage in any C++ file that reached them
through there, and only linked for callers that happened to wrap the whole
header in an extern "C" of their own. Nothing had called AES_* from C++
before, so it stayed hidden until something did.
Guarding the two headers instead lets every caller include them plainly, and
the wrappers scattered around the tree come out. Both are pure declarations
over kirk_common.h's typedefs with no system headers behind them, so there's
nothing in there that shouldn't be wrapped.
kirk_engine.h also uses size_t without including anything that defines it,
which only held together because its includers happened to have it already.
Co-Authored-By: Claude Opus 5 <[email protected]>
A .sprx from one of these packages is an NPDRM "\0PSPEDAT" container: a
0x90-byte header, then an ordinary ~PSP PRX. The loader only ever saw the
EDAT magic and gave up with SCE_KERNEL_ERROR_UNSUPPORTED_PRX_TYPE.
Step over the header, then derive the key the PRX inside is really
encrypted against: sceNpDrmGetFixedKey() over the content ID, XOR in the
licensee key the game handed us through sceNpDrmSetLicenseeKey(), then AES
under a module key that had to be added. Both halves of that were already
lying around unused - sceNpDrmGetFixedKey() had no callers at all, and the
licensee key was being kept and never read.
The rest of it is a fixed XOR that the PRX header's decrypt_mode selects
rather than its tag, so it's applied on the mode the way JPCSP does it and
the tag table is left alone - tag 0x407810F0 carries no seed of its own
there either, so ours was never wrong about it. pspDecryptType5() already
had a slot for both XORs; no new decryption logic was needed.
Decryption is only half of it: these modules are KL4E-compressed rather
than gzipped, so they also need Core/Util/KL4E.cpp, which is already there
for the firmware modules that use the same compression. With both halves
Shiren 4 Plus loads its one big .sprx and runs. God Eater 2 needed one
further fix that isn't in this commit - the type-B relocation bug in
ElfReader::LoadRelocations2, issue #8075 - and then plays.
docs/pkg_notes.md has the container layout and the key derivation.
Co-Authored-By: Claude Opus 5 <[email protected]>
An installed update silently replaces what the game boots, so the info
pane now says when there is one - version, size and where it lives - and
the context menu offers to remove it again.
Removing takes the whole PSP/GAME/<DISC_ID> folder when the update is all
that's in it. When a digital game shares the folder, only PBOOT.PBP goes,
since deleting the folder would take the game with it and nothing records
what the install wrote. The confirmation names the exact path either way.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_018izZ1mGTWhz2RqeudqsDQR
Opening a .pkg now offers to install it, the way a .zip does - see the new
InstallPkgScreen, which shows what the update patches, what it'll take up
on disk (exact, since package contents aren't compressed) and where it's
going. The package's PS3-style USRDIR/CONTENT wrapping is stripped so the
files land where the PSP expects them, in PSP/GAME/<DISC_ID>.
Booting a disc then looks for PSP/GAME/<DISC_ID>/PBOOT.PBP and boots that
instead of the disc's own EBOOT, leaving the disc mounted - so the update
overrides the files it ships and the disc supplies the rest. The update's
DISC_ID has to match; a DISC_VERSION mismatch only warns, since updates do
get used with slightly different dumps in practice.
Verified against the whole corpus: every one installs, and the
digital NP* update/base-image pairs that could be assembled all boot the
patch rather than the disc's executable. That includes Super Robot Taisen
Operation Extend from a real NPUMDIMG EBOOT.PBP, which settles that
ISO.BIN.EDAT does not re-key the PBOOT - a digital title's patched EBOOT is
encrypted exactly like a UMD one. On the UMD side, the patched
LittleBigPlanet reads PATCH.ARC out of the install alongside the disc's own
archive.
The DISC_VERSION warning turns out to be load-bearing: many of the pairs
mismatch, because the dumps in circulation are later disc revisions than the
updates were built against. docs/pkg_notes.md has the numbers, and the one
thing that doesn't work - PGD-wrapped .sprx modules, which
sceKernelLoadModuleNpDrm can't load.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
PSP game updates were distributed as NPDRM .pkg files holding a patched
EBOOT (PBOOT.PBP) plus the data files the patch replaces. PkgUnpack reads
one: header, item table, both PARAM.SFOs, and the AES-128-CTR that covers
everything past the header - including the per-item key split, where an
item's pspType byte picks between the PSP and PS3 keys.
Nothing new is needed to decrypt these. All packages checked use PRX
tag 0x2E5E10F0 for their PBOOT, which PrxDecrypter already has a key for.
Also adds "PPSSPPHeadless --install-pkg=DIR", a sibling of --unpack-updater,
which installs without any UI. All packages install through it byte
-identically to a reference implementation.
Format notes are in docs/pkg_notes.md.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
A standalone Python reader for the .pkg containers Sony shipped PSP game
updates in: it prints a package's header, metadata, both PARAM.SFOs and its
item table, and extracts the payload. Nothing in PPSSPP calls it - it exists to
work the format out and to have a second implementation to check the C++ one
against, the way Tools/ already holds a few other one-off analysis scripts.
docs/pkg_notes.md is what it was written from: the header and item table
layout, the two AES-CTR keys a single package mixes, and the detail that trips
up a first attempt - which key applies is per item, not per package, so a
reader that picks one produces garbage filenames for most of a package while a
few entries decode perfectly.
Co-Authored-By: Claude Opus 5 <[email protected]>
LoadRelocations2 declared last_type, initialised it to -1, read it once - and
never assigned it. So the (flag & 0x38) == 0x08 case, which means "reuse the
lo16 the previous relocation carried", always saw last_type != 4 and reset
lo16 to 0 instead.
That matters because R_MIPS_HI16 computes ((op << 16) + lo16) + relocate_to and
then adds 0x10000 if bit 15 of the result is set, to pre-compensate the sign
extension the paired addiu will do. With lo16 wrongly 0 the carry decision is
made on the load address alone, so for any base whose low half has bit 15 set
the high half comes out one too high and the pointer lands 0x10000 past what it
should be.
A compiler emits exactly this pattern around a branch-likely: one lui in the
delay slot, another on the fall-through path, both for the same symbol, sharing
a single addiu after the paths converge. Only the second lui is adjacent to a
HI16, so the first one silently got the wrong high half.
last_type is assigned where JPCSP assigns its R_TYPE_OLD: at the end of the
branch that actually relocates something, so the commands that only move the
base around don't count as "the previous relocation" and a HI16/HI16/LO16 group
still pairs up across them. R_MIPS_NONE stops continuing the loop for the same
reason - it has to clear last_type, or a HI16 after it would reuse a lo16 that
isn't its own.
Co-Authored-By: Claude Opus 5 <[email protected]>
The tracer records every basic block the CPU executes and can write the
instruction stream out to a file, which is the tool you want when something
corrupts state and the question is "what actually ran just before". It was
only reachable from Developer Tools in the UI, so a scripted session had no
way to turn it on, and reconstructing the same thing from log-only breakpoints
means guessing what to watch before you know what happened.
Five events: cpu.tracer.start/stop/flush/clear/status. start takes the two
buffer sizes and clears the JIT cache by default, because blocks compiled
before tracing was on don't carry the LogIRBlock instruction the tracer feeds
on - without that a hot loop compiled earlier simply never appears. The trace
ring is cyclic, so a finished recording holds the last maxTraceSize blocks:
start it, run into a crash, and the tail of the file is the instructions that
led there.
Only the IR cores drive the tracer, so start refuses on the others and says
which core is loaded rather than recording nothing; status reports the same
thing as `supported` so a client can tell that apart from "nothing executed".
Everything that mutates tracer state goes through Core_RunOnCPUThread, per
docs/DebuggerThreading.md.
Co-Authored-By: Claude Opus 5 <[email protected]>
DebuggerLogListener keeps a ring of the last BUFFER_SIZE messages. read_ and
count_ count messages ever read/written, while messages_ is indexed modulo
BUFFER_SIZE - but the non-overflow path used read_ directly as the index. Once
a session had logged BUFFER_SIZE messages that index was past the end of the
array, so the first copy loop (bounded by BUFFER_SIZE) ran zero times and the
second one handed back messages_[0..readCount-1] instead: real log messages,
just the wrong ones, and every later poll stayed the same distance out of step.
It looks like the tail of the log going missing rather than being wrong, which
is a bad way to find out. A log-only breakpoint in a hot loop reaches 1024
messages in seconds, and then "the last thing logged before the breakpoint hit"
- exactly what such a breakpoint is for - names an event thousands of messages
old. Confirmed against a case with an independently known answer: a log-only
breakpoint recording a register in a loop that runs ~20k times now ends with
the value that register actually held at the final hit, where before it ended
several hundred iterations short of it.
The overflow path was already correct - it starts from nextMessage_, which is
an index - so only the one line changes.
Co-Authored-By: Claude Opus 5 <[email protected]>
The log manager had a single external-callback slot, but the WebSocket
debugger registers one per *connection* - LogBroadcaster is a local in the
per-connection handler. So with two clients attached (the bundled JS debugger
in a browser and Tools/wsdbg, say) the second one to connect silently took the
log stream away from the first, and then whichever disconnected first cleared
the slot and stopped delivery to the other as well. A one-shot wsdbg command is
enough to do it: connect, take the stream, exit, and the long-lived listener
that was watching the log goes quiet with nothing to say why.
Make it a list with add/remove by handle. The dispatch loop holds the lock
across the callbacks so a listener can't be freed while one is running - which
is what lets LogBroadcaster delete its listener straight after removing it.
Enabling and disabling LogOutput::ExternalCallback belongs to the list now, and
disabling only happens when the last callback goes away.
libretro registers one of these too, and never removes it; it just moves to the
new call. It can't actually collide with the debugger - the libretro build
doesn't compile Core/Debugger/WebSocket at all - but there's no reason for it
to keep using an API that only has room for one caller.
Co-Authored-By: Claude Opus 5 <[email protected]>
The log manager had a single external-callback slot, but the WebSocket
debugger registers one per *connection* - LogBroadcaster is a local in the
per-connection handler. So with two clients attached (the bundled JS debugger
in a browser and Tools/wsdbg, say) the second one to connect silently took the
log stream away from the first, and then whichever disconnected first cleared
the slot and stopped delivery to the other as well. A one-shot wsdbg command is
enough to do it: connect, take the stream, exit, and the long-lived listener
that was watching the log goes quiet with nothing to say why.
Make it a list with add/remove by handle. The dispatch loop holds the lock
across the callbacks so a listener can't be freed while one is running - which
is what lets LogBroadcaster delete its listener straight after removing it.
Enabling and disabling LogOutput::ExternalCallback belongs to the list now, and
disabling only happens when the last callback goes away.
libretro registers one of these too, and never removes it; it just moves to the
new call. It can't actually collide with the debugger - the libretro build
doesn't compile Core/Debugger/WebSocket at all - but there's no reason for it
to keep using an API that only has room for one caller.
Co-Authored-By: Claude Opus 5 <[email protected]>
nt.runForClocks was zeroed when a thread was created and copied out by
sceKernelReferThreadStatus, but nothing ever added to it, so every thread
reported having run for zero time forever.
Crazy Taxi: Fare Wars uses it as a liveness check. Its music state machine
samples the mp3 thread's run time once every 60 frames and compares it with
the previous two samples; when it doesn't move it concludes playback is
wedged, sets the stop bit, and the thread tears itself down and exits. The
game restarts it, and about a second later decides it's wedged again - custom
soundtracks restarted roughly once a second forever, whatever the file.
Bill the time since the previous switch to the outgoing thread, which is
exactly the thread that was running for it. The field is already part of the
serialized thread struct, so savestates don't change format; the timestamp
itself is re-based on load rather than saved, and only on load - saving runs
a measure pass and a write pass, and re-basing in those would discard the
time the running thread had accumulated since the last switch, letting a save
change what the game can observe.
Risk: this runs on every context switch, the hottest path in the scheduler.
It adds one CoreTiming read and a 64-bit add. Games that poll thread run
times will now see them move, which is correct but is new behavior.
Co-Authored-By: Claude Opus 5 <[email protected]>
sceIoDread hands back a dirent whose d_private holds the 8.3 short name
ahead of the long name, and we never wrote the short name at all - the game
got whatever was on the stack there. Crazy Taxi: Fare Wars reads it rather
than d_name, so it rejected every file in ms0:/MUSIC, ended up with an empty
playlist and never even reserved an mp3 handle: custom soundtracks were
silently dead, with the game spinning on InitResource/SetLoopNum forever.
Generate the names from the directory listing, and resolve them back in
DirectoryFileSystem so a game can open a file by the short name it was given.
Both sides come from the same function, so they agree.
We can't lean on the host for any of this. Linux, macOS and Android have no
8.3 names at all, and while Windows does keep aliases it generates them by a
different rule - it counts to ~4 and then switches to a hash - so resolution
runs before the literal path is tried rather than as a fallback, or on
Windows we'd quietly open a different file than the one we handed the game.
The exact names a real PSP produces are still unverified - no pspautotest
covers d_private - so this implements the ordinary FAT rule and the new
FatShortNames unit test pins that down until hardware can settle it.
Co-Authored-By: Claude Opus 5 <[email protected]>
SetFontPixel refused to write any pixel whose x fell outside
bytesPerLine, so a glyph drawn into a buffer with a bytesPerLine
narrower than its rows came out mostly blank. The hardware doesn't
bound it that way - it works out an address and writes, so the rows
overlap and the glyph smears across them. The declared bufWidth and
bufHeight, plus the address check, are what keep it in bounds.
Cache invalidation now covers the wider of bytesPerLine * bufHeight and
where the last row actually ends, since those are no longer the same
thing when the rows overlap.
Fixes font/charglyphimage and font/charglyphimageclip, moved from
tests_next to tests_good. The other font tests are unaffected, so
whatever ails them is something else.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
sceMp3ReserveMp3Handle derived the new handle from g_mp3Map.size(), which
collides as soon as handles are released out of order: with 0 and 1 open,
releasing 0 leaves size at 1, so the next reserve returns 1 again. That
replaced the live context in the map without deleting it, leaking it and
handing the game a handle aliasing a stream it was still playing.
Co-Authored-By: Claude Opus 5 <[email protected]>
A game can notify more data than the file actually had - audio/mp3/stream
asks for 3360 bytes and notifies all of them even when the read came up
short - so the tail of the buffer holds stale bytes from the previous half.
We happily decoded those, six frames past the end of the stream, because the
end flag only suppressed the zero fill and never stopped the decoder.
Check it before decoding too. The post-decode check stays where it was: the
hardware rewinds in the same call that decodes the last frame, so the sum
reads back as zero right after it, which is what audio/mp3/getsumdecoded
records. Moving the whole thing up front breaks that test.
Fixes audio/mp3/stream, added to tests_good - it walks 27 refills end to end,
so it also covers the half-buffer handout.
Co-Authored-By: Claude Opus 5 <[email protected]>
The area after the 0x5c0 workarea is double buffered - a half only becomes
writable again once the decoder has consumed past its end, so decoding a
single frame usually frees nothing at all. We instead reported every byte a
decode had just consumed, which made sceMp3CheckStreamDataNeeded() answer
"yes" after every single frame.
Beats sleeps 50ms whenever that call says the file thread is behind, so it
slept once per decoded frame and delivered audio at 46% of realtime - the
badly stuttering custom soundtracks. It now decodes 3-4 frames per 3360 byte
refill, with the write pointer alternating between the two halves exactly as
audio/mp3/stream records from hardware, and keeps up.
AuGetInfoToAddStreamData/AuNotifyAddStreamData now derive the write position
from how much has been added rather than from how much is still buffered,
since the write pointer walks the halves in turn and doesn't follow the
decoder.
Fixes audio/mp3/notifyadd, moved to tests_good, and the "after decode" case
in audio/mp3/checkneeded.
sceMp3: note that the half-buffer split is only verified at 8192 bytes
The command splits $1/$2/$3 off the invocation positionally and then states them
as fact, so calling it with a sentence rather than `/add-string <Section> "<Key>"`
yields three arbitrary words - and nothing downstream notices. Ask for them to be
checked against en_US.ini first, and to re-derive the real section and key from
the request if they don't hold up.
Also spell out, in both the command and docs/translations.md, that the en_US line
in the scratch file overwrites en_US.ini like any other language, so for an
existing key it has to match the current English text exactly - otherwise it
quietly rewords the string every other language was translated from.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
36 languages, following each file's existing handling of the sibling RAIntegration
string - most keep the name as-is, pl_PL uses "Integracja RA", tr_TR "RA Entegrasyonu",
ja_JP "RAインテグレーション" - and its formality. The rest are left to fall back to
English rather than guessed at.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
RAIntegration keeps its cache and local achievement data next to the
executable. If PPSSPP is installed somewhere that needs elevation to write -
Program Files being the obvious case - that write fails and takes the emulator
down as soon as a set or code notes are loaded, with no log to show for it since
the log can't be written either.
Check whether the exe directory is writable before handing the DLL to rcheevos,
and if it isn't, say so and point at the portable .zip instead. Achievements
themselves still work, so carry on to the normal login.
Fixes#21260
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
SDL doesn't reliably pick Wayland on its own - on a machine with a
working Wayland compositor but no XDG_SESSION_TYPE it still chose x11,
putting us on XWayland. Ask for Wayland when WAYLAND_DISPLAY is set,
respect SDL_VIDEO_DRIVER if the user set it, and fall back to letting
SDL choose if Wayland then fails to initialize.
Also log the video driver we ended up with, which should help triage
the Wayland/X11 reports.
Fixes#21080
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
sceMp3GetInfoToAddStreamData always handed back the start of the work
area, so the pointer never moved as data was added - the hardware walks
it forward past what's already buffered. AuNotifyAddStreamData now
takes the new bytes from where the game was actually told to write, and
checks that range fits the buffer rather than just comparing the size.
Also compare readPos against endPos as signed. readPos is an int and a
game can notify a negative size, which made it promote to a huge u64
and look like the end of the stream, so we reported nothing left to
write where the hardware still wanted 6721 bytes.
Fixes audio/mp3/infotoadd, moved to tests_good. audio/mp3/notifyadd
gets both of its value differences fixed but still fails: after a
decode the hardware reports no space at all, while we free what the
decode consumed, so we do one round more than it does.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Same problem as the hardcore checks: SaveState.cpp asked NetworkAllowSaveState()
and just returned, so a load or save refused because you're connected did
nothing at all, with no explanation. Switched all eight to
NetworkWarnUserIfOnlineAndCantSavestate(), which is the same predicate plus the
standard message; its OSD id already collapses duplicates for the paths that
check twice on the way in.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
Freeze-frame restores a savestate every frame, straight through
SaveState::LoadFromRam(), so it never touched the operation queue and neither
hardcore check saw it. Blocked at the toggle in the dev menu, and again in the
render loop, since hardcore mode can come up after the fact once the game has
been identified.
Enqueue also just dropped operations silently, so a load that arrived through a
path with no check of its own (--state, auto-load) did nothing with no
explanation. Both it and Process now go through WarnUserIfHardcoreModeActive,
which is the same predicate plus the standard message. Callers that already ask
it themselves return before reaching Enqueue, so nothing shows the message twice.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
The check lived only in Enqueue, but operations don't run there - they're
queued and applied later by Process(). During boot, HardcoreModeActive() reads
false even when hardcore is on, since it requires rc_client_is_processing_required(),
which only becomes true once RetroAchievements has finished identifying the game
asynchronously. Anything queued in that window passed the check, and was then
applied by Process() after identification completed and hardcore came up.
Auto-load wasn't even a race: EmuScreen::bootComplete() calls Achievements::SetGame(),
which starts the identify, and then checks HardcoreModeActive() a few lines below -
always false at that point. So "Auto load savestate" quietly worked in hardcore mode.
--state and a load-state hotkey pressed during boot got through the same way.
Re-checking per operation in Process() covers every entry point at once, and by
then identification has finished, so the answer is authoritative.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01GgACRqkQNpfJQ4fjwyoEup
naettClose cleared res->request and then asked the backend to close the
response - but the request is what a backend needs to do that. On Windows it
owns the WinHTTP handles, so the close had nothing to work with, which is part
of why it did nothing at all. Backend first, then clear.
With that in place, the Windows close unhooks the status callback and shuts the
request handle, so a completion raised later can't write through the response
after it's freed. It stops short of a full cancel: a callback already running on
another thread isn't waited for, which needs the HANDLE_CLOSING handshake.
On Apple, invalidateAndCancel returns before the session has finished with its
delegate, so clear the delegate's pointer back to the response first and have
didReceiveData check it, the way didCompleteWithError already did.
Android is still the only backend that genuinely cancels and waits, so naett.h
now says a response should be complete before it's closed rather than leaving
that to be discovered.
getEnv called AttachCurrentThread and nothing detached. processRequest gets away
with it by detaching its own thread at the end, but naettPlatformInitRequest and
naettPlatformFreeRequest run on whatever thread the caller is using - and a
thread that exits while still attached takes the process down on Android, with
"Native thread exiting without having called DetachCurrentThread". They now
attach only when the thread isn't already, and detach on the way out.
pthread_create's result was ignored. Without a worker nothing ever sets
res->complete, so the caller polls naettComplete forever.
getOutputStream throws for anything from a refused connection onwards, and the
calls after it ran with that exception still pending, which most of JNI doesn't
allow. GetMethodID also returns NULL for a method it can't find, and calling
with a NULL jmethodID aborts the VM - so the call helpers check. Same for a
header whose value list is empty, which handed GetStringUTFChars a null.
Checked with the NDK's clang; the other backends were syntax-checked the same
way, against stub headers.
The worker reads a queued CURL* out of a pipe eight bytes at a time, tracking
how much it has so far - but it always read into the start of the buffer rather
than at that offset. A short read would have resumed mid-pointer and eventually
handed curl_multi_add_handle a spliced pointer. It never bit because a write
that size to a pipe is atomic, so reads are all-or-nothing, but the code is
written as though it isn't.
The write that queues the request was unchecked. If it ever failed, the request
was never run and never completed, and the caller sits in naettComplete
forever. It reports the failure now, and retries on EINTR.
curl wants an easy handle out of its multi before cleanup; that needs
curl_multi_remove_handle, which meant adding it - and curl_multi_cleanup, for
the init failure paths that leaked the multi handle and the pipe - to the dlopen
table.
workerRunning is written by the worker and read by the request path, so it's an
atomic rather than a plain int. And the read/write callbacks passed the body
callbacks' int return straight back to curl, which takes a size_t: a negative
came through as an enormous count rather than the error it was.
WinHttpQueryHeaders only writes the size it needs when it fails with
ERROR_INSUFFICIENT_BUFFER. Any other failure left bufSize at zero, so we
allocated nothing and unpackHeaders walked wcslen over it looking for the
double-null that terminates the list. Check the size, check the second query,
and allocate zeroed with room for a terminator.
winToUTF8, winFromUTF8 and wcsndup can all return NULL - on a failed conversion
or a failed allocation - and not one caller checked. packHeaders is the one that
mattered: its result goes straight into headers[0].
res->bytesLeft is a size_t counting down from what WinHTTP announced. If a read
ever returned more than that, it wrapped to an enormous count and the loop kept
reading.
createDelegate built its NSURLSessionDataDelegate with objc_allocateClassPair
and then sent it +alloc without ever calling objc_registerClassPair. The runtime
requires registration before a class pair can be used; everything up to then is
still being assembled. Register it, after the methods and the ivar go on.
The response header arrays were VLAs sized from the count the server sent, so a
response with enough headers walked the stack off the end, and one with no
headers at all declared zero-length VLAs, which is undefined by itself. Heap
now, and skipped when there's nothing to read.
The NSURLSession was also kept in the response without retaining it, while the
autorelease pool it came from is drained on the way out of the function. It only
survived because a session keeps itself alive while it has tasks running. Retain
it, release it when the response closes.
Finally, addMethod/addIvar signalled failure with assert alone, which is
compiled out in release - a delegate missing its methods would just never
receive data and the request would hang with nothing logged.
naettFree frees the method and url it strdup'd but never the user agent, which
stringSetter allocates exactly the same way. We set a user agent on every
request, so that leaked on every one of them, on all platforms.
On Linux, headerCallback strndup's each header line and only hands it to the
header list when it finds a colon - the status line and the blank line that ends
the block don't have one, so it leaked those every response, and again per hop
when following redirects.
defaultBodyWriter grew its capacity by doubling an int until the new data fit.
Both the length and the resulting capacity come from the response, so that's
signed overflow on a large one, and a negative capacity then reaches realloc as
a huge size_t. It also assigned the realloc result straight over the old
pointer, so a failed allocation lost the buffer and the memcpy below went
through NULL. Grow in int64_t, cap at INT_MAX, and report failure by returning
short - which is what every caller already checks for.
DecodeDXTBlocks limited its x loop to min(bufw, w), so when w > bufw everything
from bufw to w was simply never written - the destination is sized for w, so
those columns kept whatever the buffer held.
The software sampler doesn't do that. It addresses a block as
(v >> 2) * (texbufw >> 2) + (u >> 2), which for u past bufw runs on into the
next row's blocks, the same way the linear formats run into the next row. So the
two renderers disagreed on the same texture.
Follow the sampler: decode w texels per row and let the block index carry on,
with the range check widened to cover the blocks that reach past the last row's
stride. Shares the SourceExtent helper from the previous commit, counting 4x4
blocks rather than texels.
Rendering-visible where w > bufw for a DXT texture, which is the case that used
to leave stale contents behind.
The non-DXT decode paths read w texels per row from a source whose stride, range
check and unswizzle buffer were all sized from bufw alone. w and bufw are
independent GE registers, so w > bufw is reachable, and the last row then runs
off the end of both the validated guest range and the temp buffer.
The previous commit clamped w down to bufw, which stops the overrun but is the
wrong shape twice over: the game asked for w texels and the destination is sized
for w, so the tail of every row is left holding whatever was in the buffer, and
it silently narrows a texture the hardware would have decoded in full.
Size the checks from both instead. The extent a decode touches is bufw per row
plus however far the last row reaches past its own stride - the same adjustment
TextureReplacer::ComputeHash and the GE recorder already make - so the range
check, the height it falls back to when the range is short, and the unswizzle
buffer are all computed that way now.
Pull the five copies of "resize tmpTexBuf32_, unswizzle into it" into a helper
while we're here, since they all need the same sizing. It zeroes the buffer when
w reaches past bufw, as UnswizzleFromMem only fills the stride and the tail would
otherwise be stale heap.
ComputeTextureHash has the same bufw-only assumption, but it's left alone
deliberately - changing what goes into a texture hash isn't worth the risk here.
DXT is left alone too - it clamps to minw, but there the limit is the block index
within a bufw/4 block row, so its range check already covers what it reads.
PrepareBuildTexture's mip scan stopped at the first level with a dimension of 1
*before* running the mip size check for that level, so a level like 1x256 under a
256x256 level 0 was accepted as a valid mip. The backend then sized the level from
the halved level-0 dimensions (128x128) while LoadTextureLevel re-read the real
height (256) from the GE state, writing past the allocation. Do the check first.
That check was also gated on GPU_USE_SAMPLER_LOD_CONTROL, so backends without it
did no mip dimension validation at all - there's no reason for the guard, the
result only feeds badMipSizes, so drop it.
Separately, the non-DXT decode paths read w texels per row from a source whose
stride, range check and unswizzle buffer are all computed from bufw. w and bufw
are independent GE registers, so w > bufw is reachable and read past the end of
both the validated guest range and the temp buffer. Clamp w to bufw in
DecodeTextureLevel, which is what the DXT paths have always done via minw.
Note: ungating the mip check means backends without SAMPLER_LOD_CONTROL can now
set badMipSizes where they previously didn't, which collapses such textures to a
single level. That's the intended behavior, but it is a rendering-visible change
on those backends.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
DirectoryFileSystem passed the host's permission bits and directory
size straight through to the game. The PSP has neither - its FAT driver
makes a mode up from the entry type and whether it's writable, and
reports no size for a directory. So a game saw 0644/0755 and a 4096
byte directory on Linux, but 0664/0777 and 0 on Windows, where the file
layer already synthesizes those bits.
Now both platforms report what the PSP does: 0777 for directories, 0664
for writable files, 0444 for read-only ones, and no size on a
directory. The parent ".." entry keeps its 4096, which is what the
hardware reports for that one.
Also fixes the synthetic PSP directory entry using 0x777 where 0777 was
meant.
Fixes io/directory/directory, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The constructor leaves treeroot null when it can't find a CD001 volume
descriptor, but GetFromPath walked into it anyway - TreeEntry *entry = treeroot;
then entry->valid - so any path lookup on a failed mount dereferenced null.
Reachable from the firmware installer, which mounts whatever file it's handed
and asks for PSP_GAME/SYSDIR/UPDATE without consulting Error() first. Point it
at a PlayStation disc image, whose descriptor sits behind a Mode 2 subheader and
so fails the signature check, and PPSSPP goes down. Identify_File checks for
CD001 before reporting PSP_ISO, so the game browser was never exposed.
Return null instead, which is what the rest of the function already does for a
path that isn't there, and what every caller expects.
The disc-updater install ran when ltn0.pgf was missing, so a font set unpacked
from an old UMD looked complete forever, and a later game wanting a font its own
firmware added silently got a bundled substitute instead.
Requiring the whole registry doesn't work either: firmwares older than a font
can never satisfy it, so we'd unpack the same updater on every launch and
announce it each time. What settles it is that a game can't ask for a font that
didn't exist when it was made. Record the earliest firmware known to ship each
font in the registry, and only require the ones the running game's firmware
would have had.
The version comes from PARAM.SFO's PSP_SYSTEM_VER, with the bundled updater's
version as a fallback. That keeps the whole thing stateless - nothing recorded
that could go stale when flash0 or the ini gets moved around.
Survey of a large library, unpacking flash0:/font from each disc's updater
across firmware 1.50 to 6.60: jpn0 and ltn0..ltn15 are in every one of them, and
kr0.pgf is the only registry font that arrived later - absent in 1.50, present
from 1.52.
Uses one directory listing rather than a stat per font, since on Android's
scoped storage the individual checks are slow.
Centralizing command line parsing replaced the hand-rolled --gamesettings and
--touchscreentest argv checks in NativeInit with a single --start-screen=<name>
option, and made an unrecognized "--" argument a hard parse error. The two
System_RestartApp callers still passed the old flags.
So changing the graphics backend killed PPSSPP for good: the new process starts,
fails to parse --gamesettings, and returns 1 from WinMain before a window ever
exists. The error goes to stderr, which nobody sees in a GUI build, so it just
looks like the app quit instead of restarting.
Restarts that pass no arguments (the memstick screen, and the edit-then-restore
path) were unaffected, since an empty argument string makes ExitAndRestart reuse
the original command line.
Also move the TouchTestScreen push inside the touchscreentest branch - it looks
like a brace that didn't move during the refactor, and it would otherwise push a
touch test screen for every --start-screen value.
Decrypting an entry's contents is by far the most expensive part of
walking an archive, and it happened for every entry before the filter
was even consulted. Now it waits until entryData()/entryCompression()
asks, so pulling just the fonts out of an updater no longer costs a
full firmware decrypt. Records are decrypted independently of each
other, so deferring one is safe.
Unpacking fonts from a 3.11 updater goes 0.365s -> 0.133s; a full
unpack is unchanged and produces identical output. The compression
counts now describe the entries we actually decoded rather than
everything in the archive.
Also adds --unpack-updater-filter to headless, which is how the above
was measured.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
sceFont never looked at flash0:, so a firmware the user installed was
ignored and we always fell back to our bundled substitutes - despite
the "ignoring NAND" warning suggesting otherwise. It reads flash0:/font
now, after the game's own fonts and the classic ms0 override.
And if there's nothing in NAND, we unpack just flash0:/font out of the
firmware updater on the running disc, which most UMDs carry. That turns
"install a firmware first" into something that happens by itself for
anyone playing a retail game.
EmulatedModelGeneration moves from InstallUpdateScreen into PSARUnpack
so both callers pick the same firmware file list.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Henrik RydgårdandClaude Opus 5