It asserted that it was the first call, so net::Init - which is documented as
safe to call repeatedly, and is - needed a bool of its own purely to guard this
one line. That's bookkeeping the library may as well do itself, and the WSA half
of the same function already says as much: it doesn't track anything because
WSAStartup counts its own references.
So a repeat call does nothing, and net::Init loses g_naettInitialized. Asking
HTTPSAvailable() twice costs nothing - on Linux it's a cached dlopen result and
everywhere else it's a constant.
Found reviewing the commits before this one. On Windows a short write marked the
request complete and then queued another read regardless, which is worse than it
sounds: the caller is free to close a response the moment it sees complete, so
WinHTTP carried on writing into memory that was being freed. It also meant
cancelling didn't actually stop anything there. Stop at that point, and ignore
anything raised after the request is complete.
The Apple callbacks do the same check, so the cancellation we ask for after a
failed write doesn't overwrite the error that caused it.
naettMake only asserted that its request was non-NULL, and the request
constructors do return NULL when a platform can't set one up - an unparseable
URL is enough on Windows. Asserts are compiled out in release, so that was a
null dereference. It returns NULL now, and HTTPSRequest checks for it and fails
the request instead of handing it on.
Also: a request cancelled between construction and Start() didn't carry that
into the sink, and -1000 - our own cancellation code, not naett's - was logging
"Unhandled naett error" on a perfectly normal cancel.
A writer that takes less than it was given is failing the request. Windows and
Linux already treat it that way, and it's how the default writer reports that it
couldn't grow its buffer - but the Apple and Android backends threw the result
away, so a short write silently truncated the body and still reported success.
That also means it's how a caller stops a transfer in progress, which is what
the next commit uses, so document what the return value means while we're here.
Apple cancels the data task as well, or the chunks just keep coming.
naettClose cleared res->request and then asked the backend to close the
response - but the request is what a backend needs to do that. On Windows it
owns the WinHTTP handles, so the close had nothing to work with, which is part
of why it did nothing at all. Backend first, then clear.
With that in place, the Windows close unhooks the status callback and shuts the
request handle, so a completion raised later can't write through the response
after it's freed. It stops short of a full cancel: a callback already running on
another thread isn't waited for, which needs the HANDLE_CLOSING handshake.
On Apple, invalidateAndCancel returns before the session has finished with its
delegate, so clear the delegate's pointer back to the response first and have
didReceiveData check it, the way didCompleteWithError already did.
Android is still the only backend that genuinely cancels and waits, so naett.h
now says a response should be complete before it's closed rather than leaving
that to be discovered.
getEnv called AttachCurrentThread and nothing detached. processRequest gets away
with it by detaching its own thread at the end, but naettPlatformInitRequest and
naettPlatformFreeRequest run on whatever thread the caller is using - and a
thread that exits while still attached takes the process down on Android, with
"Native thread exiting without having called DetachCurrentThread". They now
attach only when the thread isn't already, and detach on the way out.
pthread_create's result was ignored. Without a worker nothing ever sets
res->complete, so the caller polls naettComplete forever.
getOutputStream throws for anything from a refused connection onwards, and the
calls after it ran with that exception still pending, which most of JNI doesn't
allow. GetMethodID also returns NULL for a method it can't find, and calling
with a NULL jmethodID aborts the VM - so the call helpers check. Same for a
header whose value list is empty, which handed GetStringUTFChars a null.
Checked with the NDK's clang; the other backends were syntax-checked the same
way, against stub headers.
The worker reads a queued CURL* out of a pipe eight bytes at a time, tracking
how much it has so far - but it always read into the start of the buffer rather
than at that offset. A short read would have resumed mid-pointer and eventually
handed curl_multi_add_handle a spliced pointer. It never bit because a write
that size to a pipe is atomic, so reads are all-or-nothing, but the code is
written as though it isn't.
The write that queues the request was unchecked. If it ever failed, the request
was never run and never completed, and the caller sits in naettComplete
forever. It reports the failure now, and retries on EINTR.
curl wants an easy handle out of its multi before cleanup; that needs
curl_multi_remove_handle, which meant adding it - and curl_multi_cleanup, for
the init failure paths that leaked the multi handle and the pipe - to the dlopen
table.
workerRunning is written by the worker and read by the request path, so it's an
atomic rather than a plain int. And the read/write callbacks passed the body
callbacks' int return straight back to curl, which takes a size_t: a negative
came through as an enormous count rather than the error it was.
WinHttpQueryHeaders only writes the size it needs when it fails with
ERROR_INSUFFICIENT_BUFFER. Any other failure left bufSize at zero, so we
allocated nothing and unpackHeaders walked wcslen over it looking for the
double-null that terminates the list. Check the size, check the second query,
and allocate zeroed with room for a terminator.
winToUTF8, winFromUTF8 and wcsndup can all return NULL - on a failed conversion
or a failed allocation - and not one caller checked. packHeaders is the one that
mattered: its result goes straight into headers[0].
res->bytesLeft is a size_t counting down from what WinHTTP announced. If a read
ever returned more than that, it wrapped to an enormous count and the loop kept
reading.
createDelegate built its NSURLSessionDataDelegate with objc_allocateClassPair
and then sent it +alloc without ever calling objc_registerClassPair. The runtime
requires registration before a class pair can be used; everything up to then is
still being assembled. Register it, after the methods and the ivar go on.
The response header arrays were VLAs sized from the count the server sent, so a
response with enough headers walked the stack off the end, and one with no
headers at all declared zero-length VLAs, which is undefined by itself. Heap
now, and skipped when there's nothing to read.
The NSURLSession was also kept in the response without retaining it, while the
autorelease pool it came from is drained on the way out of the function. It only
survived because a session keeps itself alive while it has tasks running. Retain
it, release it when the response closes.
Finally, addMethod/addIvar signalled failure with assert alone, which is
compiled out in release - a delegate missing its methods would just never
receive data and the request would hang with nothing logged.
naettFree frees the method and url it strdup'd but never the user agent, which
stringSetter allocates exactly the same way. We set a user agent on every
request, so that leaked on every one of them, on all platforms.
On Linux, headerCallback strndup's each header line and only hands it to the
header list when it finds a colon - the status line and the blank line that ends
the block don't have one, so it leaked those every response, and again per hop
when following redirects.
defaultBodyWriter grew its capacity by doubling an int until the new data fit.
Both the length and the resulting capacity come from the response, so that's
signed overflow on a large one, and a negative capacity then reaches realloc as
a huge size_t. It also assigned the realloc result straight over the old
pointer, so a failed allocation lost the buffer and the memcpy below went
through NULL. Grow in int64_t, cap at INT_MAX, and report failure by returning
short - which is what every caller already checks for.
naett has had a complete libcurl backend all along; we just never built it,
so Linux ran with HTTPS_NOT_AVAILABLE. That means no homebrew store over
HTTPS, and RetroAchievements talking to plain http://retroachievements.org.
libcurl is loaded with dlopen rather than linked, the same way we handle the
Vulkan loader, so it stays a soft dependency: we need the curl headers at
build time, but a build made here still starts on a machine without libcurl
installed - it just reports HTTPS as unavailable, exactly like today. Distro
packagers get the behavior they'd expect either way, and certificate
validation comes free from the system CA store.
New net::HTTPSAvailable() answers "did that work", and SDLMain folds it into
SYSPROP_SUPPORTS_HTTPS, which everything downstream already degrades on.
Four fixes to the backend itself, all noted in ext/naett/README-ppsspp.md:
- panic() called exit(1) on a pipe or curl_multi_perform failure. Taking the
emulator down because a download failed isn't acceptable - the backend now
disables itself and requests complete with naettGenericError.
- CURLINFO_RESPONSE_CODE writes a long into res->code, which is an int. Eight
bytes into four, getting away with it only because the next field absorbs
the zeroes.
- curl_easy_setopt is varargs and wants a long for these options; int literals
and int variables are UB on LP64.
- naettPlatformCloseResponse called through a null function pointer when
libcurl was missing. Found by testing that path, which segfaulted.
CI needs libcurl4-openssl-dev (curl-dev on Alpine) or it would quietly keep
building without HTTPS.
naett has been a submodule pinned at v0.3.3; upstream has had no commits since
April 2024, and we want to carry local changes (next up: a libcurl-backed HTTPS
path for Linux). It's ~1500 lines of MIT C, smaller than several things we
already vendor, so bring it in-tree and drop the submodule.
Also drop the generated single-file amalgam (naett.c) that every build system
was compiling, and build src/*.c directly instead - otherwise the file you edit
isn't the file that gets compiled, which is a trap for anyone patching this.
example/ and testrig/ (a whole Android Studio project) are gone with it.
Two changes were needed to make the sources build on their own, both noted in
ext/naett/README-ppsspp.md along with the upstream commit:
- naett_internal.h now includes naett.h, which the amalgam pulled in first.
- naett_linux.c now includes stdio.h/stdlib.h. It calls exit/calloc/realloc/
free/fprintf without ever including either, and only got away with it because
naett_core.c sat above it in the concatenation.
No functional change - Linux still has HTTPS_NOT_AVAILABLE set, so it doesn't
build naett at all yet.
Rename naett to naett-lib
Henrik Rydgård