Files
Alexander J. Semenuk 5d5e35fb9b fix: Windows toolchain compatibility (curl 8.21 re-vendor, endless reconfigure loop) (#4355)
## Problem

Windows builds break with a current local toolchain (Scoop LLVM 22.1.8,
CMake 4.4.0, VS 2026), in two independent ways:

1. The build stops at curl's deliberate guard: `#error "no non-blocking
method was found/used/set"` in `third-party/curl/lib/nonblock.c`.
2. From the second configure onward, `cmake --build` re-runs CMake in an
endless loop (observed 42 consecutive reconfigure cycles in a single
build). Likely the same mechanism behind the "endlessly building" VS
2026 note in `docs/setup/dev/vs.md`.

## Root cause

1. `third-party/curl/CMake/CurlTests.c` passes `int *` to
`ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes
`-Wincompatible-pointer-types` to a hard error in C, so the
`HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and
`curl_config.h` never defines it. Upstream CI does not see this because
the windows-2022 runner image ships an older LLVM. GCC 14 promotes the
same warning to a hard error, which is very likely the `CurlTests.c.obj`
failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl
hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0
(curl/curl#13578).
2. The root CMakeLists copies the build tree's `compile_commands.json`
into `<src>/build/` for clangd using `configure_file()`, which registers
its input as a configure dependency. CMake rewrites
`compile_commands.json` late in every generation, after
`CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same
Ninja regen rule), so once the dependency is registered the rule is
deterministically dirty and every `ninja` invocation re-runs CMake. A
pristine first configure is safe (the file does not exist yet, so the
`if(EXISTS ...)` guard skips the copy), which is why the loop looks
machine- or IDE-specific.

## Fix

1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag
(previously `curl-8_3_0`), which carries the upstream probe fix plus two
years of upstream development; `vendor.yaml` updated to match.
Adjustments the version jump forced:
- curl 8.15 removed the native macOS Secure Transport backend
(`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like
Linux. The two macOS workflows install Homebrew `openssl@3` and export
`OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same
two lines.
- `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10;
pinned OFF to keep the previous no-compression behavior and avoid
silently linking whatever the CI images happen to have.
- curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked
into discord-rpc's identically named option and broke configure at a
nonexistent `examples/send-presence` directory; pinned OFF ahead of the
third-party subdirectories.

The diff is dominated by the mechanical tag-tree swap under
`third-party/curl` (linguist-vendored, collapsed in review). The
hand-written changes are `CMakeLists.txt`, the two macOS workflows,
`docs/setup/system/macos.md`, and `vendor.yaml`.
2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy
with no configure dependency registered. (`file(COPY_FILE ...
ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21,
above the declared `cmake_minimum_required(VERSION 3.10)`.)

## Test plan

- [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache
seeding) completes and logs `Enabled SSL backends: Schannel`; the
FIONBIO probe passes without the previous `#error`
- [x] Full Windows Release build from scratch in the branch worktree
(all 1422 targets)
- [x] goalc-test suite: 1509 passed, 0 failed
- [x] Second consecutive configure with `compile_commands.json` present:
the regen rule in `build.ninja` has no `compile_commands.json` input;
`<src>/build/compile_commands.json` is still refreshed for clangd
- [x] Repeated `ninja` invocations after a full build no longer re-run
CMake
- [x] macOS Intel and ARM CI green (first exercise of the OpenSSL
backend switch)

---

I work off a self-hosted forge, so this GitHub account is quiet; the
configure logs and ninja dirty-node traces from the investigation are
available if anyone wants the raw data.

(AI-assisted)
2026-07-27 19:19:18 -04:00

537 lines
22 KiB
C
Vendored
Generated

#ifndef HEADER_LIBTEST_FIRST_H
#define HEADER_LIBTEST_FIRST_H
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at https://curl.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
* SPDX-License-Identifier: curl
*
***************************************************************************/
#define CURL_NO_OLDIES
#define CURL_DISABLE_DEPRECATION
/* Now include the curl_setup.h file from libcurl's private libdir (the source
version, but that might include "curl_config.h" from the build directory so
we need both of them in the include path), so that we get good in-depth
knowledge about the system we are building this on */
#include "curl_setup.h"
#include "testutil.h"
typedef CURLcode (*entry_func_t)(const char *);
struct entry_s {
const char *name;
entry_func_t ptr;
};
extern const struct entry_s s_entries[];
extern int unitfail; /* for unittests */
#include "curlx/base64.h" /* for curlx_base64* */
#include "curlx/dynbuf.h" /* for curlx_dyn_*() */
#include "curlx/fopen.h" /* for curlx_f*() */
#include "curlx/strcopy.h" /* for curlx_strcopy() */
#include "curlx/strerr.h" /* for curlx_strerror() */
#include "curlx/strparse.h" /* for curlx_str_* parsing functions */
#include "curlx/timediff.h" /* for timediff_t type and related functions */
#include "curlx/timeval.h" /* for curlx_now type and related functions */
#include "curlx/wait.h" /* for curlx_wait_ms() */
#ifdef HAVE_SYS_SELECT_H
/* since so many tests use select(), we can as well include it here */
#include <sys/select.h>
#endif
extern const char *libtest_arg2; /* set by first.c to the argv[2] or NULL */
extern const char *libtest_arg3; /* set by first.c to the argv[3] or NULL */
extern const char *libtest_arg4; /* set by first.c to the argv[4] or NULL */
extern const char *libtest_arg5; /* set by first.c to the argv[5] or NULL */
/* argc and argv as passed in to the main() function */
extern int test_argc;
extern const char **test_argv;
extern int testnum;
extern struct curltime tv_test_start; /* for test timing */
extern int coptind;
extern const char *coptarg;
int cgetopt(int argc, const char * const argv[], const char *optstring);
extern int select_wrapper(int nfds, fd_set *rd, fd_set *wr, fd_set *exc,
struct timeval *tv);
extern char *hexdump(const unsigned char *buf, size_t len);
#ifndef CURL_DISABLE_WEBSOCKETS
CURLcode ws_send_ping(CURL *curl, const char *send_payload);
CURLcode ws_recv_pong(CURL *curl, const char *expected_payload);
void ws_close(CURL *curl); /* close the connection */
#endif
/*
* TEST_ERR_* values must be within the CURLcode range to not cause compiler
* errors.
*
* For portability reasons TEST_ERR_* values should be less than 127.
*/
#define TEST_ERR_MAJOR_BAD CURLE_OBSOLETE20
#define TEST_ERR_RUNS_FOREVER CURLE_OBSOLETE24
#define TEST_ERR_EASY_INIT CURLE_OBSOLETE29
#define TEST_ERR_MULTI CURLE_OBSOLETE32
#define TEST_ERR_NUM_HANDLES CURLE_OBSOLETE34
#define TEST_ERR_SELECT CURLE_OBSOLETE40
#define TEST_ERR_SUCCESS CURLE_OBSOLETE41
#define TEST_ERR_FAILURE CURLE_OBSOLETE44
#define TEST_ERR_USAGE CURLE_OBSOLETE46
#define TEST_ERR_FOPEN CURLE_OBSOLETE50
#define TEST_ERR_FSTAT CURLE_OBSOLETE51
#define TEST_ERR_BAD_TIMEOUT CURLE_OBSOLETE57
/*
* Macros for test source code readability/maintainability.
*
* All of the following macros require that an int data type 'res' variable
* exists in scope where macro is used, and that it has been initialized to
* zero before the macro is used.
*
* exe_* and chk_* macros are helper macros not intended to be used from
* outside of this header file. Arguments 'Y' and 'Z' of these represent
* source code file and line number, while Arguments 'A', 'B', etc, are
* the arguments used to actually call a libcurl function.
*
* All easy_* and multi_* macros call a libcurl function and evaluate if
* the function has succeeded or failed. When the function succeeds 'res'
* variable is not set nor cleared and program continues normal flow. On
* the other hand if function fails 'res' variable is set and a jump to
* label 'test_cleanup' is performed.
*
* Every easy_* and multi_* macros have a res_easy_* and res_multi_* macro
* counterpart that operates in the same way with the exception that no
* jump takes place in case of failure. res_easy_* and res_multi_* macros
* should be immediately followed by checking if 'res' variable has been
* set.
*
* 'res' variable when set holds a CURLcode, CURLMcode, or any of the
* TEST_ERR_* values defined above. It is advisable to return this value
* as test result.
*/
#ifndef UNITTESTS
/* ---------------------------------------------------------------- */
#define exe_easy_init(A, Y, Z) \
do { \
(A) = curl_easy_init(); \
if(!(A)) { \
curl_mfprintf(stderr, "%s:%d curl_easy_init() failed\n", Y, Z); \
result = TEST_ERR_EASY_INIT; \
} \
} while(0)
#define res_easy_init(A) \
exe_easy_init(A, __FILE__, __LINE__)
#define chk_easy_init(A, Y, Z) \
do { \
exe_easy_init(A, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define easy_init(A) \
chk_easy_init(A, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_init(A, Y, Z) \
do { \
(A) = curl_multi_init(); \
if(!(A)) { \
curl_mfprintf(stderr, "%s:%d curl_multi_init() failed\n", Y, Z); \
result = TEST_ERR_MULTI; \
} \
} while(0)
#define res_multi_init(A) \
exe_multi_init(A, __FILE__, __LINE__)
#define chk_multi_init(A, Y, Z) \
do { \
exe_multi_init(A, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_init(A) \
chk_multi_init(A, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_easy_setopt(A, B, C, Y, Z) \
do { \
result = curl_easy_setopt(A, B, C); \
if(result) \
curl_mfprintf(stderr, \
"%s:%d curl_easy_setopt() failed, " \
"with code %d (%s)\n", \
Y, Z, (int)result, curl_easy_strerror(result)); \
} while(0)
#define res_easy_setopt(A, B, C) \
exe_easy_setopt(A, B, C, __FILE__, __LINE__)
#define chk_easy_setopt(A, B, C, Y, Z) \
do { \
exe_easy_setopt(A, B, C, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define easy_setopt(A, B, C) \
chk_easy_setopt(A, B, C, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_setopt(A, B, C, Y, Z) \
do { \
CURLMcode ec = curl_multi_setopt(A, B, C); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_setopt() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
} while(0)
#define chk_multi_setopt(A, B, C, Y, Z) \
do { \
exe_multi_setopt(A, B, C, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_setopt(A, B, C) \
chk_multi_setopt(A, B, C, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_add_handle(A, B, Y, Z) \
do { \
CURLMcode ec = curl_multi_add_handle(A, B); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_add_handle() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
} while(0)
#define res_multi_add_handle(A, B) \
exe_multi_add_handle(A, B, __FILE__, __LINE__)
#define chk_multi_add_handle(A, B, Y, Z) \
do { \
exe_multi_add_handle(A, B, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_add_handle(A, B) \
chk_multi_add_handle(A, B, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_remove_handle(A, B, Y, Z) \
do { \
CURLMcode ec = curl_multi_remove_handle(A, B); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_remove_handle() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
} while(0)
#define chk_multi_remove_handle(A, B, Y, Z) \
do { \
exe_multi_remove_handle(A, B, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_remove_handle(A, B) \
chk_multi_remove_handle(A, B, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_perform(A, B, Y, Z) \
do { \
CURLMcode ec = curl_multi_perform(A, B); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_perform() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
else if(*(B) < 0) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_perform() succeeded, " \
"but returned invalid running_handles value (%d)\n", \
Y, Z, (int)*(B)); \
result = TEST_ERR_NUM_HANDLES; \
} \
} while(0)
#define res_multi_perform(A, B) \
exe_multi_perform(A, B, __FILE__, __LINE__)
#define chk_multi_perform(A, B, Y, Z) \
do { \
exe_multi_perform(A, B, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_perform(A, B) \
chk_multi_perform(A, B, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_fdset(A, B, C, D, E, Y, Z) \
do { \
CURLMcode ec = curl_multi_fdset(A, B, C, D, E); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_fdset() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
else if(*(E) < -1) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_fdset() succeeded, " \
"but returned invalid max_fd value (%d)\n", \
Y, Z, (int)*(E)); \
result = TEST_ERR_NUM_HANDLES; \
} \
} while(0)
#define res_multi_fdset(A, B, C, D, E) \
exe_multi_fdset(A, B, C, D, E, __FILE__, __LINE__)
#define chk_multi_fdset(A, B, C, D, E, Y, Z) \
do { \
exe_multi_fdset(A, B, C, D, E, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_fdset(A, B, C, D, E) \
chk_multi_fdset(A, B, C, D, E, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_timeout(A, B, Y, Z) \
do { \
CURLMcode ec = curl_multi_timeout(A, B); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_timeout() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_BAD_TIMEOUT; \
} \
else if(*(B) < -1L) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_timeout() succeeded, " \
"but returned invalid timeout value (%ld)\n", \
Y, Z, (long)*(B)); \
result = TEST_ERR_BAD_TIMEOUT; \
} \
} while(0)
#define res_multi_timeout(A, B) \
exe_multi_timeout(A, B, __FILE__, __LINE__)
#define chk_multi_timeout(A, B, Y, Z) \
do { \
exe_multi_timeout(A, B, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_timeout(A, B) \
chk_multi_timeout(A, B, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_poll(A, B, C, D, E, Y, Z) \
do { \
CURLMcode ec = curl_multi_poll(A, B, C, D, E); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_poll() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
else if(*(E) < 0) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_poll() succeeded, " \
"but returned invalid numfds value (%d)\n", \
Y, Z, (int)*(E)); \
result = TEST_ERR_NUM_HANDLES; \
} \
} while(0)
#define chk_multi_poll(A, B, C, D, E, Y, Z) \
do { \
exe_multi_poll(A, B, C, D, E, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define multi_poll(A, B, C, D, E) \
chk_multi_poll(A, B, C, D, E, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_multi_wakeup(A, Y, Z) \
do { \
CURLMcode ec = curl_multi_wakeup(A); \
if(ec != CURLM_OK) { \
curl_mfprintf(stderr, \
"%s:%d curl_multi_wakeup() failed, " \
"with code %d (%s)\n", \
Y, Z, ec, curl_multi_strerror(ec)); \
result = TEST_ERR_MULTI; \
} \
} while(0)
#define res_multi_wakeup(A) \
exe_multi_wakeup(A, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define exe_select_test(A, B, C, D, E, Y, Z) \
do { \
if(select_wrapper(A, B, C, D, E) == -1) { \
int sockerr = SOCKERRNO; \
char sockerrbuf[STRERROR_LEN]; \
curl_mfprintf(stderr, \
"%s:%d select() failed, with errno %d (%s)\n", Y, Z, \
sockerr, curlx_strerror(sockerr, sockerrbuf, \
sizeof(sockerrbuf))); \
result = TEST_ERR_SELECT; \
} \
} while(0)
#define res_select_test(A, B, C, D, E) \
exe_select_test(A, B, C, D, E, __FILE__, __LINE__)
#define chk_select_test(A, B, C, D, E, Y, Z) \
do { \
exe_select_test(A, B, C, D, E, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define select_test(A, B, C, D, E) \
chk_select_test(A, B, C, D, E, __FILE__, __LINE__)
/* ---------------------------------------------------------------- */
#define start_test_timing() \
do { \
tv_test_start = curlx_now(); \
} while(0)
#define TEST_HANG_TIMEOUT (60 * 1000) /* global default */
#define exe_test_timedout(T, Y, Z) \
do { \
timediff_t timediff = curlx_timediff_ms(curlx_now(), tv_test_start); \
if(timediff > (T)) { \
curl_mfprintf(stderr, \
"%s:%d ABORTING TEST, since it seems " \
"that it would have run forever (%ld ms > %ld ms)\n", \
Y, Z, (long)timediff, (long)TEST_HANG_TIMEOUT); \
result = TEST_ERR_RUNS_FOREVER; \
} \
} while(0)
#define res_test_timedout() \
exe_test_timedout(TEST_HANG_TIMEOUT, __FILE__, __LINE__)
#define chk_test_timedout(T, Y, Z) \
do { \
exe_test_timedout(T, Y, Z); \
if(result) \
goto test_cleanup; \
} while(0)
#define abort_on_test_timeout() \
chk_test_timedout(TEST_HANG_TIMEOUT, __FILE__, __LINE__)
#define abort_on_test_timeout_custom(T) \
chk_test_timedout(T, __FILE__, __LINE__)
#define NUM_HANDLES 4 /* global default */
#define res_global_init(A) \
exe_global_init(A, __FILE__, __LINE__)
#endif /* !UNITTESTS */
#if !defined(UNITTESTS) || defined(BUILDING_LIBCURL)
/* ---------------------------------------------------------------- */
#define exe_global_init(A, Y, Z) \
do { \
result = curl_global_init(A); \
if(result) \
curl_mfprintf(stderr, \
"%s:%d curl_global_init() failed, " \
"with code %d (%s)\n", \
Y, Z, (int)result, curl_easy_strerror(result)); \
} while(0)
#define chk_global_init(A, Y, Z) \
do { \
exe_global_init(A, Y, Z); \
if(result) \
return result; \
} while(0)
/* global_init() is different than other macros. In case of
failure it 'return's instead of going to 'test_cleanup'. */
#define global_init(A) \
chk_global_init(A, __FILE__, __LINE__)
#endif /* !UNITTESTS || BUILDING_LIBCURL */
#endif /* HEADER_LIBTEST_FIRST_H */