Files
Alexander J. Semenuk 5d5e35fb9b fix: Windows toolchain compatibility (curl 8.21 re-vendor, endless reconfigure loop) (#4355)
## Problem

Windows builds break with a current local toolchain (Scoop LLVM 22.1.8,
CMake 4.4.0, VS 2026), in two independent ways:

1. The build stops at curl's deliberate guard: `#error "no non-blocking
method was found/used/set"` in `third-party/curl/lib/nonblock.c`.
2. From the second configure onward, `cmake --build` re-runs CMake in an
endless loop (observed 42 consecutive reconfigure cycles in a single
build). Likely the same mechanism behind the "endlessly building" VS
2026 note in `docs/setup/dev/vs.md`.

## Root cause

1. `third-party/curl/CMake/CurlTests.c` passes `int *` to
`ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes
`-Wincompatible-pointer-types` to a hard error in C, so the
`HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and
`curl_config.h` never defines it. Upstream CI does not see this because
the windows-2022 runner image ships an older LLVM. GCC 14 promotes the
same warning to a hard error, which is very likely the `CurlTests.c.obj`
failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl
hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0
(curl/curl#13578).
2. The root CMakeLists copies the build tree's `compile_commands.json`
into `<src>/build/` for clangd using `configure_file()`, which registers
its input as a configure dependency. CMake rewrites
`compile_commands.json` late in every generation, after
`CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same
Ninja regen rule), so once the dependency is registered the rule is
deterministically dirty and every `ninja` invocation re-runs CMake. A
pristine first configure is safe (the file does not exist yet, so the
`if(EXISTS ...)` guard skips the copy), which is why the loop looks
machine- or IDE-specific.

## Fix

1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag
(previously `curl-8_3_0`), which carries the upstream probe fix plus two
years of upstream development; `vendor.yaml` updated to match.
Adjustments the version jump forced:
- curl 8.15 removed the native macOS Secure Transport backend
(`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like
Linux. The two macOS workflows install Homebrew `openssl@3` and export
`OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same
two lines.
- `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10;
pinned OFF to keep the previous no-compression behavior and avoid
silently linking whatever the CI images happen to have.
- curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked
into discord-rpc's identically named option and broke configure at a
nonexistent `examples/send-presence` directory; pinned OFF ahead of the
third-party subdirectories.

The diff is dominated by the mechanical tag-tree swap under
`third-party/curl` (linguist-vendored, collapsed in review). The
hand-written changes are `CMakeLists.txt`, the two macOS workflows,
`docs/setup/system/macos.md`, and `vendor.yaml`.
2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy
with no configure dependency registered. (`file(COPY_FILE ...
ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21,
above the declared `cmake_minimum_required(VERSION 3.10)`.)

## Test plan

- [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache
seeding) completes and logs `Enabled SSL backends: Schannel`; the
FIONBIO probe passes without the previous `#error`
- [x] Full Windows Release build from scratch in the branch worktree
(all 1422 targets)
- [x] goalc-test suite: 1509 passed, 0 failed
- [x] Second consecutive configure with `compile_commands.json` present:
the regen rule in `build.ninja` has no `compile_commands.json` input;
`<src>/build/compile_commands.json` is still refreshed for clangd
- [x] Repeated `ninja` invocations after a full build no longer re-run
CMake
- [x] macOS Intel and ARM CI green (first exercise of the OpenSSL
backend switch)

---

I work off a self-hosted forge, so this GitHub account is quiet; the
configure logs and ninja dirty-node traces from the investigation are
available if anyone wants the raw data.

(AI-assisted)
2026-07-27 19:19:18 -04:00

293 lines
6.7 KiB
C
Vendored
Generated

/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at https://curl.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
* SPDX-License-Identifier: curl
*
***************************************************************************/
#include "curl_setup.h"
#include "curlx/dynbuf.h"
#include "curl_printf.h"
#define MIN_FIRST_ALLOC 32
#ifdef DEBUGBUILD
#define DYNINIT 0xbee51da /* random pattern */
#endif
/*
* Init a dynbuf struct.
*/
void curlx_dyn_init(struct dynbuf *s, size_t toobig)
{
DEBUGASSERT(s);
DEBUGASSERT(toobig);
DEBUGASSERT(toobig <= MAX_DYNBUF_SIZE); /* catch crazy mistakes */
s->bufr = NULL;
s->leng = 0;
s->allc = 0;
s->toobig = toobig;
#ifdef DEBUGBUILD
s->init = DYNINIT;
#endif
}
/*
* free the buffer and re-init the necessary fields. It does not touch the
* 'init' field and thus this buffer can be reused to add data to again.
*/
void curlx_dyn_free(struct dynbuf *s)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
curlx_safefree(s->bufr);
s->leng = s->allc = 0;
}
/*
* Store/append an chunk of memory to the dynbuf.
*/
static CURLcode dyn_nappend(struct dynbuf *s,
const unsigned char *mem, size_t len)
{
size_t idx = s->leng;
size_t a = s->allc;
size_t fit = len + idx + 1; /* new string + old string + zero byte */
/* try to detect if there is rubbish in the struct */
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(s->toobig);
DEBUGASSERT(idx < s->toobig);
DEBUGASSERT(!s->leng || s->bufr);
DEBUGASSERT(a <= s->toobig);
DEBUGASSERT(!len || mem);
if(fit > s->toobig) {
curlx_dyn_free(s);
return CURLE_TOO_LARGE;
}
else if(!a) {
DEBUGASSERT(!idx);
/* first invoke */
if(MIN_FIRST_ALLOC > s->toobig)
a = s->toobig;
else if(fit < MIN_FIRST_ALLOC)
a = MIN_FIRST_ALLOC;
else
a = fit;
}
else {
while(a < fit)
a *= 2;
if(a > s->toobig)
/* no point in allocating a larger buffer than this is allowed to use */
a = s->toobig;
}
if(a != s->allc) {
void *p = curlx_realloc(s->bufr, a);
if(!p) {
curlx_dyn_free(s);
return CURLE_OUT_OF_MEMORY;
}
s->bufr = p;
s->allc = a;
}
if(len)
memcpy(&s->bufr[idx], mem, len);
s->leng = idx + len;
s->bufr[s->leng] = 0;
return CURLE_OK;
}
/*
* Clears the string, keeps the allocation. This can also be called on a
* buffer that already was freed.
*/
void curlx_dyn_reset(struct dynbuf *s)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
if(s->leng)
s->bufr[0] = 0;
s->leng = 0;
}
/*
* Specify the size of the tail to keep (number of bytes from the end of the
* buffer). The rest is dropped.
*/
CURLcode curlx_dyn_tail(struct dynbuf *s, size_t trail)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
if(trail > s->leng)
return CURLE_BAD_FUNCTION_ARGUMENT;
else if(trail == s->leng)
return CURLE_OK;
else if(!trail) {
curlx_dyn_reset(s);
}
else {
memmove(&s->bufr[0], &s->bufr[s->leng - trail], trail);
s->leng = trail;
s->bufr[s->leng] = 0;
}
return CURLE_OK;
}
/*
* Appends a buffer with length.
*/
CURLcode curlx_dyn_addn(struct dynbuf *s, const void *mem, size_t len)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
return dyn_nappend(s, mem, len);
}
/*
* Append a null-terminated string at the end.
*/
CURLcode curlx_dyn_add(struct dynbuf *s, const char *str)
{
size_t n;
DEBUGASSERT(str);
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
n = strlen(str);
return dyn_nappend(s, (const unsigned char *)str, n);
}
/*
* Append a string vprintf()-style
*/
CURLcode curlx_dyn_vaddf(struct dynbuf *s, const char *fmt, va_list ap)
{
#ifdef BUILDING_LIBCURL
int rc;
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
DEBUGASSERT(fmt);
rc = curlx_dyn_vprintf(s, fmt, ap);
if(!rc)
return CURLE_OK;
else if(rc == MERR_TOO_LARGE)
return CURLE_TOO_LARGE;
return CURLE_OUT_OF_MEMORY;
#else
char *str;
str = curl_mvaprintf(fmt, ap); /* this allocs a new string to append */
if(str) {
CURLcode result = dyn_nappend(s, (const unsigned char *)str, strlen(str));
curl_free(str);
return result;
}
/* If we failed, we cleanup the whole buffer and return error */
curlx_dyn_free(s);
return CURLE_OUT_OF_MEMORY;
#endif
}
/*
* Append a string printf()-style
*/
CURLcode curlx_dyn_addf(struct dynbuf *s, const char *fmt, ...)
{
CURLcode result;
va_list ap;
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
DEBUGASSERT(strcmp(fmt, "%s")); /* use curlx_dyn_add instead */
va_start(ap, fmt);
result = curlx_dyn_vaddf(s, fmt, ap);
va_end(ap);
return result;
}
/*
* Returns a pointer to the buffer.
*/
char *curlx_dyn_ptr(const struct dynbuf *s)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
return s->bufr;
}
char *curlx_dyn_take(struct dynbuf *s, size_t *plen)
{
char *ptr = s->bufr;
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
*plen = s->leng;
s->bufr = NULL;
s->leng = 0;
s->allc = 0;
return ptr;
}
/*
* Returns an unsigned pointer to the buffer.
*/
unsigned char *curlx_dyn_uptr(const struct dynbuf *s)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
return (unsigned char *)s->bufr;
}
/*
* Returns the length of the buffer.
*/
size_t curlx_dyn_len(const struct dynbuf *s)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
return s->leng;
}
/*
* Set a new (smaller) length.
*/
CURLcode curlx_dyn_setlen(struct dynbuf *s, size_t set)
{
DEBUGASSERT(s);
DEBUGASSERT(s->init == DYNINIT);
DEBUGASSERT(!s->leng || s->bufr);
if(set > s->leng)
return CURLE_BAD_FUNCTION_ARGUMENT;
s->leng = set;
s->bufr[s->leng] = 0;
return CURLE_OK;
}