Alexander J. Semenuk
5d5e35fb9b
## Problem Windows builds break with a current local toolchain (Scoop LLVM 22.1.8, CMake 4.4.0, VS 2026), in two independent ways: 1. The build stops at curl's deliberate guard: `#error "no non-blocking method was found/used/set"` in `third-party/curl/lib/nonblock.c`. 2. From the second configure onward, `cmake --build` re-runs CMake in an endless loop (observed 42 consecutive reconfigure cycles in a single build). Likely the same mechanism behind the "endlessly building" VS 2026 note in `docs/setup/dev/vs.md`. ## Root cause 1. `third-party/curl/CMake/CurlTests.c` passes `int *` to `ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes `-Wincompatible-pointer-types` to a hard error in C, so the `HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and `curl_config.h` never defines it. Upstream CI does not see this because the windows-2022 runner image ships an older LLVM. GCC 14 promotes the same warning to a hard error, which is very likely the `CurlTests.c.obj` failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0 (curl/curl#13578). 2. The root CMakeLists copies the build tree's `compile_commands.json` into `<src>/build/` for clangd using `configure_file()`, which registers its input as a configure dependency. CMake rewrites `compile_commands.json` late in every generation, after `CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same Ninja regen rule), so once the dependency is registered the rule is deterministically dirty and every `ninja` invocation re-runs CMake. A pristine first configure is safe (the file does not exist yet, so the `if(EXISTS ...)` guard skips the copy), which is why the loop looks machine- or IDE-specific. ## Fix 1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag (previously `curl-8_3_0`), which carries the upstream probe fix plus two years of upstream development; `vendor.yaml` updated to match. Adjustments the version jump forced: - curl 8.15 removed the native macOS Secure Transport backend (`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like Linux. The two macOS workflows install Homebrew `openssl@3` and export `OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same two lines. - `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10; pinned OFF to keep the previous no-compression behavior and avoid silently linking whatever the CI images happen to have. - curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked into discord-rpc's identically named option and broke configure at a nonexistent `examples/send-presence` directory; pinned OFF ahead of the third-party subdirectories. The diff is dominated by the mechanical tag-tree swap under `third-party/curl` (linguist-vendored, collapsed in review). The hand-written changes are `CMakeLists.txt`, the two macOS workflows, `docs/setup/system/macos.md`, and `vendor.yaml`. 2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy with no configure dependency registered. (`file(COPY_FILE ... ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21, above the declared `cmake_minimum_required(VERSION 3.10)`.) ## Test plan - [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache seeding) completes and logs `Enabled SSL backends: Schannel`; the FIONBIO probe passes without the previous `#error` - [x] Full Windows Release build from scratch in the branch worktree (all 1422 targets) - [x] goalc-test suite: 1509 passed, 0 failed - [x] Second consecutive configure with `compile_commands.json` present: the regen rule in `build.ninja` has no `compile_commands.json` input; `<src>/build/compile_commands.json` is still refreshed for clangd - [x] Repeated `ninja` invocations after a full build no longer re-run CMake - [x] macOS Intel and ARM CI green (first exercise of the OpenSSL backend switch) --- I work off a self-hosted forge, so this GitHub account is quiet; the configure logs and ninja dirty-node traces from the investigation are available if anyone wants the raw data. (AI-assisted)
3.2 KiB
Vendored
Generated
Multi Identifiers (mid)
All transfers (easy handles) added to a multi handle are assigned
a unique identifier until they are removed again. The multi handle
keeps a table multi->xfers that allow O(1) access to the easy
handle by its mid.
References to other easy handles should keep their mids instead
of a pointer (not all code has been converted as of now). This solves
problems in easy and multi handle life cycle management as well as
iterating over handles where operations may add/remove other handles.
Values and Lifetime
An mid is an unsigned int. There are two reserved values:
0: is themidof an internal "admin" handle. Multi and share handles each have their own admin handle for maintenance operations, like shutting down connections.UINT_MAX: the "invalid"mid. Easy handles are initialized with this value. They get it assigned again when removed from a multi handle.
This makes potential range of mids from 1 to UINT_MAX - 1 inside the
same multi handle at the same time. The multi->xfers table reuses mid
values from previous transfers that have been removed.
multi->xfers is created with an initial capacity. At the time of this
writing that is 16 for "multi_easy" handles (used in curl_easy_perform()
and 512 for multi handles created with curl_multi_init().
The first added easy handle gets mid == 1 assigned. The second one receives 2,
even when the first one has been removed already. Every added handle gets an
mid one larger than the previously assigned one. Until the capacity of
the table is reached and it starts looking for a free id at 1 again (0
is always in the table).
When adding a new handle, the multi checks the amount of free entries
in the multi->xfers table. If that drops below a threshold (currently 25%),
the table is resized. This serves two purposes: one, a previous mid is not
reused immediately and second, table resizes are not needed that often.
The table is implemented in uint-table.[ch]. More details in UINT_SETS.
Tracking mids
There are several places where transfers need to be tracked:
- the multi tracks
process,pendingandmsgsenttransfers. A transfer is in at most one of these at a time. - connections track the transfers that are attached to them.
- multi event handling tracks transfers interested in a specific socket.
- DoH handles track the handle they perform lookups for (and vice versa).
There are two bitset implemented for storing mids: uint_bset and uint_spbset.
The first is a bitset optimal for storing a large number of unsigned int values.
The second one is a "sparse" variant good for storing a small set of numbers.
More details about these in UINT_SETS.
A multi uses uint_bsets for process, pending and msgsent. Connections
and sockets use the sparse variant as both often track only a single transfer
and at most 100 on an HTTP/2 or HTTP/3 connection/socket.
These sets allow safe iteration while being modified. This allows a multi to iterate over its "process" set while existing transfers are removed or new ones added.