Files
Alexander J. Semenuk 5d5e35fb9b fix: Windows toolchain compatibility (curl 8.21 re-vendor, endless reconfigure loop) (#4355)
## Problem

Windows builds break with a current local toolchain (Scoop LLVM 22.1.8,
CMake 4.4.0, VS 2026), in two independent ways:

1. The build stops at curl's deliberate guard: `#error "no non-blocking
method was found/used/set"` in `third-party/curl/lib/nonblock.c`.
2. From the second configure onward, `cmake --build` re-runs CMake in an
endless loop (observed 42 consecutive reconfigure cycles in a single
build). Likely the same mechanism behind the "endlessly building" VS
2026 note in `docs/setup/dev/vs.md`.

## Root cause

1. `third-party/curl/CMake/CurlTests.c` passes `int *` to
`ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes
`-Wincompatible-pointer-types` to a hard error in C, so the
`HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and
`curl_config.h` never defines it. Upstream CI does not see this because
the windows-2022 runner image ships an older LLVM. GCC 14 promotes the
same warning to a hard error, which is very likely the `CurlTests.c.obj`
failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl
hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0
(curl/curl#13578).
2. The root CMakeLists copies the build tree's `compile_commands.json`
into `<src>/build/` for clangd using `configure_file()`, which registers
its input as a configure dependency. CMake rewrites
`compile_commands.json` late in every generation, after
`CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same
Ninja regen rule), so once the dependency is registered the rule is
deterministically dirty and every `ninja` invocation re-runs CMake. A
pristine first configure is safe (the file does not exist yet, so the
`if(EXISTS ...)` guard skips the copy), which is why the loop looks
machine- or IDE-specific.

## Fix

1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag
(previously `curl-8_3_0`), which carries the upstream probe fix plus two
years of upstream development; `vendor.yaml` updated to match.
Adjustments the version jump forced:
- curl 8.15 removed the native macOS Secure Transport backend
(`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like
Linux. The two macOS workflows install Homebrew `openssl@3` and export
`OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same
two lines.
- `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10;
pinned OFF to keep the previous no-compression behavior and avoid
silently linking whatever the CI images happen to have.
- curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked
into discord-rpc's identically named option and broke configure at a
nonexistent `examples/send-presence` directory; pinned OFF ahead of the
third-party subdirectories.

The diff is dominated by the mechanical tag-tree swap under
`third-party/curl` (linguist-vendored, collapsed in review). The
hand-written changes are `CMakeLists.txt`, the two macOS workflows,
`docs/setup/system/macos.md`, and `vendor.yaml`.
2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy
with no configure dependency registered. (`file(COPY_FILE ...
ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21,
above the declared `cmake_minimum_required(VERSION 3.10)`.)

## Test plan

- [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache
seeding) completes and logs `Enabled SSL backends: Schannel`; the
FIONBIO probe passes without the previous `#error`
- [x] Full Windows Release build from scratch in the branch worktree
(all 1422 targets)
- [x] goalc-test suite: 1509 passed, 0 failed
- [x] Second consecutive configure with `compile_commands.json` present:
the regen rule in `build.ninja` has no `compile_commands.json` input;
`<src>/build/compile_commands.json` is still refreshed for clangd
- [x] Repeated `ninja` invocations after a full build no longer re-run
CMake
- [x] macOS Intel and ARM CI green (first exercise of the OpenSSL
backend switch)

---

I work off a self-hosted forge, so this GitHub account is quiet; the
configure logs and ninja dirty-node traces from the investigation are
available if anyone wants the raw data.

(AI-assisted)
2026-07-27 19:19:18 -04:00

473 lines
12 KiB
C
Vendored
Generated

/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at https://curl.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
* SPDX-License-Identifier: curl
*
***************************************************************************/
#include "curl_setup.h"
#include "urldata.h"
#include "multiif.h"
#include "curl_threads.h"
#include "curl_share.h"
#include "vtls/vtls.h"
#include "vtls/vtls_scache.h"
#include "hsts.h"
#include "url.h"
static void share_destroy(struct Curl_share *share)
{
if(share->specifier & (1 << CURL_LOCK_DATA_CONNECT)) {
Curl_cpool_destroy(&share->cpool);
}
Curl_dnscache_destroy(&share->dnscache);
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_COOKIES)
Curl_cookie_cleanup(share->cookies);
#endif
#ifndef CURL_DISABLE_HSTS
Curl_hsts_cleanup(&share->hsts);
#endif
#ifdef USE_SSL
if(share->ssl_scache) {
Curl_ssl_scache_destroy(share->ssl_scache);
share->ssl_scache = NULL;
}
#endif
Curl_psl_destroy(&share->psl);
Curl_close(&share->admin);
#ifdef USE_MUTEX
Curl_mutex_destroy(&share->lock);
#endif
share->magic = 0;
curlx_free(share);
}
CURLSH *curl_share_init(void)
{
struct Curl_share *share = curlx_calloc(1, sizeof(struct Curl_share));
if(share) {
share->magic = CURL_GOOD_SHARE;
share->specifier |= (1 << CURL_LOCK_DATA_SHARE);
#ifdef USE_MUTEX
Curl_mutex_init(&share->lock);
#endif
share->ref_count = 1;
Curl_dnscache_init(&share->dnscache, 23);
share->admin = curl_easy_init();
if(!share->admin) {
share_destroy(share);
return NULL;
}
/* admin handles have mid 0 */
share->admin->mid = 0;
share->admin->state.internal = TRUE;
#ifdef DEBUGBUILD
if(getenv("CURL_DEBUG"))
share->admin->set.verbose = TRUE;
#endif
}
return share;
}
static uint32_t share_ref_inc(struct Curl_share *share)
{
uint32_t n;
#ifdef USE_MUTEX
Curl_mutex_acquire(&share->lock);
n = ++(share->ref_count);
share->has_been_shared = TRUE;
Curl_mutex_release(&share->lock);
#else
n = ++(share->ref_count);
share->has_been_shared = TRUE;
#endif
return n;
}
static uint32_t share_ref_dec(struct Curl_share *share)
{
uint32_t n;
#ifdef USE_MUTEX
Curl_mutex_acquire(&share->lock);
DEBUGASSERT(share->ref_count);
n = --(share->ref_count);
Curl_mutex_release(&share->lock);
#else
n = --(share->ref_count);
#endif
return n;
}
static bool share_has_been_shared(struct Curl_share *share)
{
bool was_shared;
#ifdef USE_MUTEX
Curl_mutex_acquire(&share->lock);
was_shared = share->has_been_shared;
Curl_mutex_release(&share->lock);
#else
was_shared = share->has_been_shared;
#endif
return was_shared;
}
static bool share_lock_acquire(struct Curl_share *share,
struct Curl_easy *data)
{
if(share->lockfunc && share->unlockfunc &&
(data || share_has_been_shared(share))) {
share->lockfunc(data, CURL_LOCK_DATA_SHARE, CURL_LOCK_ACCESS_SINGLE,
share->clientdata);
return TRUE;
}
return FALSE;
}
static void share_lock_release(struct Curl_share *share,
struct Curl_easy *data,
bool locked)
{
if(locked) {
DEBUGASSERT(share->unlockfunc);
if(share->unlockfunc)
share->unlockfunc(data, CURL_LOCK_DATA_SHARE, share->clientdata);
}
}
static bool share_in_use(struct Curl_share *share)
{
bool in_use;
#ifdef USE_MUTEX
Curl_mutex_acquire(&share->lock);
in_use = (share->ref_count > 1);
Curl_mutex_release(&share->lock);
#else
bool locked = share_lock_acquire(share, NULL);
in_use = (share->ref_count > 1);
share_lock_release(share, NULL, locked);
#endif
return in_use;
}
static void share_unlink(struct Curl_share **pshare,
struct Curl_easy *data,
bool locked)
{
struct Curl_share *share = *pshare;
uint32_t n;
*pshare = NULL;
n = share_ref_dec(share);
if(locked)
share_lock_release(share, data, locked);
if(!n) /* last reference gone */
share_destroy(share);
}
#undef curl_share_setopt
CURLSHcode curl_share_setopt(CURLSH *sh, CURLSHoption option, ...)
{
va_list param;
int type;
curl_lock_function lockfunc;
curl_unlock_function unlockfunc;
void *ptr;
CURLSHcode res = CURLSHE_OK;
struct Curl_share *share = sh;
if(!GOOD_SHARE_HANDLE(share))
return CURLSHE_INVALID;
if(share_in_use(share)) {
/* do not allow setting options while one or more handles are already
using this share */
return CURLSHE_IN_USE;
}
va_start(param, option);
switch(option) {
case CURLSHOPT_SHARE:
/* this is a type this share will share */
type = va_arg(param, int);
switch(type) {
case CURL_LOCK_DATA_DNS:
break;
case CURL_LOCK_DATA_COOKIE:
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_COOKIES)
if(!share->cookies) {
share->cookies = Curl_cookie_init();
if(!share->cookies)
res = CURLSHE_NOMEM;
}
#else /* CURL_DISABLE_HTTP || CURL_DISABLE_COOKIES */
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_HSTS:
#ifndef CURL_DISABLE_HSTS
if(!share->hsts) {
share->hsts = Curl_hsts_init();
if(!share->hsts)
res = CURLSHE_NOMEM;
}
#else /* CURL_DISABLE_HSTS */
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_SSL_SESSION:
#ifdef USE_SSL
if(!share->ssl_scache) {
/* There is no way (yet) for the application to configure the
* session cache size, shared between many transfers. As for curl
* itself, a high session count will impact startup time. Also, the
* scache is not optimized for several hundreds of peers.
* Keep it at a reasonable level. */
if(Curl_ssl_scache_create(25, 2, &share->ssl_scache))
res = CURLSHE_NOMEM;
}
#else
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_CONNECT:
/* It is safe to set this option several times on a share. */
if(!share->cpool.initialized) {
Curl_cpool_init(&share->cpool, share->admin, share, 103);
}
break;
case CURL_LOCK_DATA_PSL:
#ifndef USE_LIBPSL
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
default:
res = CURLSHE_BAD_OPTION;
}
if(!res)
share->specifier |= (unsigned int)(1 << type);
break;
case CURLSHOPT_UNSHARE:
/* this is a type this share will no longer share */
type = va_arg(param, int);
switch(type) {
case CURL_LOCK_DATA_DNS:
break;
case CURL_LOCK_DATA_COOKIE:
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_COOKIES)
if(share->cookies) {
Curl_cookie_cleanup(share->cookies);
share->cookies = NULL;
}
#else /* CURL_DISABLE_HTTP || CURL_DISABLE_COOKIES */
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_HSTS:
#ifndef CURL_DISABLE_HSTS
if(share->hsts) {
Curl_hsts_cleanup(&share->hsts);
}
#else /* CURL_DISABLE_HSTS */
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_SSL_SESSION:
#ifdef USE_SSL
if(share->ssl_scache) {
Curl_ssl_scache_destroy(share->ssl_scache);
share->ssl_scache = NULL;
}
#else
res = CURLSHE_NOT_BUILT_IN;
#endif
break;
case CURL_LOCK_DATA_CONNECT:
break;
default:
res = CURLSHE_BAD_OPTION;
break;
}
if(!res)
share->specifier &= ~(unsigned int)(1 << type);
break;
case CURLSHOPT_LOCKFUNC:
lockfunc = va_arg(param, curl_lock_function);
share->lockfunc = lockfunc;
break;
case CURLSHOPT_UNLOCKFUNC:
unlockfunc = va_arg(param, curl_unlock_function);
share->unlockfunc = unlockfunc;
break;
case CURLSHOPT_USERDATA:
ptr = va_arg(param, void *);
share->clientdata = ptr;
break;
default:
res = CURLSHE_BAD_OPTION;
break;
}
va_end(param);
return res;
}
CURLSHcode curl_share_cleanup(CURLSH *sh)
{
struct Curl_share *share = sh;
bool locked;
if(!GOOD_SHARE_HANDLE(share))
return CURLSHE_INVALID;
if(share_in_use(share))
return CURLSHE_IN_USE;
locked = share_lock_acquire(share, NULL);
share_unlink(&share, NULL, locked);
return CURLSHE_OK;
}
CURLSHcode Curl_share_lock(struct Curl_easy *data, curl_lock_data type,
curl_lock_access accesstype)
{
struct Curl_share *share = data->share;
if(!share)
return CURLSHE_INVALID;
if(share->specifier & (unsigned int)(1 << type)) {
if(share->lockfunc) /* only call this if set! */
share->lockfunc(data, type, accesstype, share->clientdata);
}
/* else if we do not share this, pretend successful lock */
return CURLSHE_OK;
}
CURLSHcode Curl_share_unlock(struct Curl_easy *data, curl_lock_data type)
{
struct Curl_share *share = data->share;
if(!share)
return CURLSHE_INVALID;
if(share->specifier & (unsigned int)(1 << type)) {
if(share->unlockfunc) /* only call this if set! */
share->unlockfunc(data, type, share->clientdata);
}
return CURLSHE_OK;
}
CURLcode Curl_share_easy_unlink(struct Curl_easy *data)
{
struct Curl_share *share = data->share;
if(share) {
bool locked = share_lock_acquire(share, data);
/* If data has a connection from this share, detach it. */
if(data->conn && (share->specifier & (1 << CURL_LOCK_DATA_CONNECT)))
Curl_detach_connection(data);
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_COOKIES)
if(share->cookies == data->cookies)
data->cookies = NULL;
#endif
#ifndef CURL_DISABLE_HSTS
if(share->hsts == data->hsts)
data->hsts = NULL;
#endif
#ifdef USE_LIBPSL
if(&share->psl == data->psl)
data->psl = data->multi ? &data->multi->psl : NULL;
#endif
share_unlink(&data->share, data, locked);
}
return CURLE_OK;
}
CURLcode Curl_share_easy_link(struct Curl_easy *data,
struct Curl_share *share)
{
if(data->share) {
DEBUGASSERT(0);
return CURLE_FAILED_INIT;
}
if(share) {
bool locked = share_lock_acquire(share, data);
share_ref_inc(share);
data->share = share;
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_COOKIES)
if(share->cookies) {
/* use shared cookie list, first free own one if any */
Curl_cookie_cleanup(data->cookies);
/* enable cookies since we now use a share that uses cookies! */
data->cookies = share->cookies;
}
#endif /* CURL_DISABLE_HTTP */
#ifndef CURL_DISABLE_HSTS
if(share->hsts) {
/* first free the private one if any */
Curl_hsts_cleanup(&data->hsts);
data->hsts = share->hsts;
}
#endif
#ifdef USE_LIBPSL
if(share->specifier & (1 << CURL_LOCK_DATA_PSL))
data->psl = &share->psl;
#endif
/* check for host cache not needed,
* it will be done by curl_easy_perform */
share_lock_release(share, data, locked);
}
return CURLE_OK;
}