mirror/jak-project
1
0
Fork 0
mirror of https://github.com/open-goal/jak-project synced 2026-08-20 22:35:07 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix: Windows toolchain compatibility (curl 8.21 re-vendor, endless reconfigure loop) (#4355)

Browse Source 
## Problem

Windows builds break with a current local toolchain (Scoop LLVM 22.1.8,
CMake 4.4.0, VS 2026), in two independent ways:

1. The build stops at curl's deliberate guard: `#error "no non-blocking
method was found/used/set"` in `third-party/curl/lib/nonblock.c`.
2. From the second configure onward, `cmake --build` re-runs CMake in an
endless loop (observed 42 consecutive reconfigure cycles in a single
build). Likely the same mechanism behind the "endlessly building" VS
2026 note in `docs/setup/dev/vs.md`.

## Root cause

1. `third-party/curl/CMake/CurlTests.c` passes `int *` to
`ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes
`-Wincompatible-pointer-types` to a hard error in C, so the
`HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and
`curl_config.h` never defines it. Upstream CI does not see this because
the windows-2022 runner image ships an older LLVM. GCC 14 promotes the
same warning to a hard error, which is very likely the `CurlTests.c.obj`
failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl
hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0
(curl/curl#13578).
2. The root CMakeLists copies the build tree's `compile_commands.json`
into `<src>/build/` for clangd using `configure_file()`, which registers
its input as a configure dependency. CMake rewrites
`compile_commands.json` late in every generation, after
`CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same
Ninja regen rule), so once the dependency is registered the rule is
deterministically dirty and every `ninja` invocation re-runs CMake. A
pristine first configure is safe (the file does not exist yet, so the
`if(EXISTS ...)` guard skips the copy), which is why the loop looks
machine- or IDE-specific.

## Fix

1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag
(previously `curl-8_3_0`), which carries the upstream probe fix plus two
years of upstream development; `vendor.yaml` updated to match.
Adjustments the version jump forced:
- curl 8.15 removed the native macOS Secure Transport backend
(`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like
Linux. The two macOS workflows install Homebrew `openssl@3` and export
`OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same
two lines.
- `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10;
pinned OFF to keep the previous no-compression behavior and avoid
silently linking whatever the CI images happen to have.
- curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked
into discord-rpc's identically named option and broke configure at a
nonexistent `examples/send-presence` directory; pinned OFF ahead of the
third-party subdirectories.

The diff is dominated by the mechanical tag-tree swap under
`third-party/curl` (linguist-vendored, collapsed in review). The
hand-written changes are `CMakeLists.txt`, the two macOS workflows,
`docs/setup/system/macos.md`, and `vendor.yaml`.
2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy
with no configure dependency registered. (`file(COPY_FILE ...
ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21,
above the declared `cmake_minimum_required(VERSION 3.10)`.)

## Test plan

- [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache
seeding) completes and logs `Enabled SSL backends: Schannel`; the
FIONBIO probe passes without the previous `#error`
- [x] Full Windows Release build from scratch in the branch worktree
(all 1422 targets)
- [x] goalc-test suite: 1509 passed, 0 failed
- [x] Second consecutive configure with `compile_commands.json` present:
the regen rule in `build.ninja` has no `compile_commands.json` input;
`<src>/build/compile_commands.json` is still refreshed for clangd
- [x] Repeated `ninja` invocations after a full build no longer re-run
CMake
- [x] macOS Intel and ARM CI green (first exercise of the OpenSSL
backend switch)

---

I work off a self-hosted forge, so this GitHub account is quiet; the
configure logs and ninja dirty-node traces from the investigation are
available if anyone wants the raw data.

(AI-assisted)
This commit is contained in:
Alexander J. Semenuk
2026-07-27 19:19:18 -04:00
committed by GitHub
parent 888b300487
commit 5d5e35fb9b
5480 changed files with 345923 additions and 288294 deletions
Download Patch File Download Diff File Expand all files Collapse all files
third-party/curl/docs/examples/htmltitle.cpp
Generated Vendored
+76 -77
View File
@@ -22,107 +22,105 @@
*
***************************************************************************/
/* <DESC>
* Get a web page, extract the title with libxml.
* Get a webpage, extract the title with libxml.
* </DESC>
Written by Lars Nilsson
Written by Lars Nilsson
GNU C++ compile command line suggestion (edit paths accordingly):
GNU C++ compile command line suggestion (edit paths accordingly):
g++ -Wall -I/opt/curl/include -I/opt/libxml/include/libxml2 htmltitle.cpp \
-o htmltitle -L/opt/curl/lib -L/opt/libxml/lib -lcurl -lxml2
g++ -Wall -I/opt/curl/include -I/opt/libxml/include/libxml2 htmltitle.cpp \
-o htmltitle -L/opt/curl/lib -L/opt/libxml/lib -lcurl -lxml2
*/
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <string>
#include <curl/curl.h>
#include <libxml/HTMLparser.h>
//
// Case-insensitive string comparison
// Case-insensitive string comparison
//
#ifdef _MSC_VER
#define COMPARE(a, b) (!_stricmp((a), (b)))
#ifdef _WIN32
#define COMPARE(a, b) (!_stricmp(a, b))
#else
#define COMPARE(a, b) (!strcasecmp((a), (b)))
#define COMPARE(a, b) (!strcasecmp(a, b))
#endif
//
// libxml callback context structure
// libxml callback context structure
//
struct Context
{
Context(): addTitle(false) { }
struct Context {
Context() : addTitle(false) {}
bool addTitle;
std::string title;
};
//
// libcurl variables for error strings and returned data
// libcurl variables for error strings and returned data
static char errorBuffer[CURL_ERROR_SIZE];
static std::string buffer;
//
// libcurl write callback function
// libcurl write callback function
//
static int writer(char *data, size_t size, size_t nmemb,
std::string *writerData)
static size_t writer(char *data, size_t size, size_t nmemb,
std::string *writerData)
{
if(writerData == NULL)
if(!writerData)
return 0;
writerData->append(data, size*nmemb);
writerData->append(data, size * nmemb);
return size * nmemb;
}
//
// libcurl connection initialization
// libcurl connection initialization
//
static bool init(CURL *&conn, char *url)
static bool init(CURL *&curl, const char *url)
{
CURLcode code;
CURLcode result;
conn = curl_easy_init();
curl = curl_easy_init();
if(conn == NULL) {
fprintf(stderr, "Failed to create CURL connection\n");
exit(EXIT_FAILURE);
}
code = curl_easy_setopt(conn, CURLOPT_ERRORBUFFER, errorBuffer);
if(code != CURLE_OK) {
fprintf(stderr, "Failed to set error buffer [%d]\n", code);
if(!curl) {
fprintf(stderr, "Failed to create CURL handle\n");
return false;
}
code = curl_easy_setopt(conn, CURLOPT_URL, url);
if(code != CURLE_OK) {
result = curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errorBuffer);
if(result != CURLE_OK) {
fprintf(stderr, "Failed to set error buffer [%d]\n", result);
return false;
}
result = curl_easy_setopt(curl, CURLOPT_URL, url);
if(result != CURLE_OK) {
fprintf(stderr, "Failed to set URL [%s]\n", errorBuffer);
return false;
}
code = curl_easy_setopt(conn, CURLOPT_FOLLOWLOCATION, 1L);
if(code != CURLE_OK) {
result = curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
if(result != CURLE_OK) {
fprintf(stderr, "Failed to set redirect option [%s]\n", errorBuffer);
return false;
}
code = curl_easy_setopt(conn, CURLOPT_WRITEFUNCTION, writer);
if(code != CURLE_OK) {
result = curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writer);
if(result != CURLE_OK) {
fprintf(stderr, "Failed to set writer [%s]\n", errorBuffer);
return false;
}
code = curl_easy_setopt(conn, CURLOPT_WRITEDATA, &buffer);
if(code != CURLE_OK) {
result = curl_easy_setopt(curl, CURLOPT_WRITEDATA, &buffer);
if(result != CURLE_OK) {
fprintf(stderr, "Failed to set write data [%s]\n", errorBuffer);
return false;
}
@@ -131,51 +129,48 @@ static bool init(CURL *&conn, char *url)
}
//
// libxml start element callback function
// libxml start element callback function
//
static void StartElement(void *voidContext,
const xmlChar *name,
const xmlChar **attributes)
{
Context *context = static_cast<Context *>(voidContext);
if(COMPARE(reinterpret_cast<char *>(name), "TITLE")) {
if(COMPARE(reinterpret_cast<const char *>(name), "TITLE")) {
context->title = "";
context->addTitle = true;
}
(void) attributes;
(void)attributes;
}
//
// libxml end element callback function
// libxml end element callback function
//
static void EndElement(void *voidContext,
const xmlChar *name)
{
Context *context = static_cast<Context *>(voidContext);
if(COMPARE(reinterpret_cast<char *>(name), "TITLE"))
if(COMPARE(reinterpret_cast<const char *>(name), "TITLE"))
context->addTitle = false;
}
//
// Text handling helper function
// Text handling helper function
//
static void handleCharacters(Context *context,
const xmlChar *chars,
int length)
{
if(context->addTitle)
context->title.append(reinterpret_cast<char *>(chars), length);
context->title.append(reinterpret_cast<const char *>(chars),
(unsigned long)length);
}
//
// libxml PCDATA callback function
// libxml PCDATA callback function
//
static void Characters(void *voidContext,
const xmlChar *chars,
int length)
@@ -186,9 +181,8 @@ static void Characters(void *voidContext,
}
//
// libxml CDATA callback function
// libxml CDATA callback function
//
static void cdata(void *voidContext,
const xmlChar *chars,
int length)
@@ -199,11 +193,9 @@ static void cdata(void *voidContext,
}
//
// libxml SAX callback structure
// libxml SAX callback structure
//
static htmlSAXHandler saxHandler =
{
static htmlSAXHandler saxHandler = {
NULL,
NULL,
NULL,
@@ -230,13 +222,17 @@ static htmlSAXHandler saxHandler =
NULL,
NULL,
cdata,
NULL,
0,
0,
0,
0,
NULL
};
//
// Parse given (assumed to be) HTML text and return the title
// Parse given (assumed to be) HTML text and return the title
//
static void parseHtml(const std::string &html,
std::string &title)
{
@@ -246,7 +242,7 @@ static void parseHtml(const std::string &html,
ctxt = htmlCreatePushParserCtxt(&saxHandler, &context, "", 0, "",
XML_CHAR_ENCODING_NONE);
htmlParseChunk(ctxt, html.c_str(), html.size(), 0);
htmlParseChunk(ctxt, html.c_str(), (int)html.size(), 0);
htmlParseChunk(ctxt, "", 0, 1);
htmlFreeParserCtxt(ctxt);
@@ -254,36 +250,39 @@ static void parseHtml(const std::string &html,
title = context.title;
}
int main(int argc, char *argv[])
int main(int argc, const char *argv[])
{
CURL *conn = NULL;
CURLcode code;
CURL *curl = NULL;
CURLcode result;
std::string title;
// Ensure one argument is given
if(argc != 2) {
fprintf(stderr, "Usage: %s <url>\n", argv[0]);
exit(EXIT_FAILURE);
return EXIT_FAILURE;
}
curl_global_init(CURL_GLOBAL_DEFAULT);
result = curl_global_init(CURL_GLOBAL_ALL);
if(result != CURLE_OK)
return (int)result;
// Initialize CURL connection
// Initialize CURL handle
if(!init(conn, argv[1])) {
fprintf(stderr, "Connection initialization failed\n");
exit(EXIT_FAILURE);
if(!init(curl, argv[1])) {
fprintf(stderr, "Handle initialization failed\n");
curl_global_cleanup();
return EXIT_FAILURE;
}
// Retrieve content for the URL
code = curl_easy_perform(conn);
curl_easy_cleanup(conn);
result = curl_easy_perform(curl);
curl_easy_cleanup(curl);
if(code != CURLE_OK) {
if(result != CURLE_OK) {
fprintf(stderr, "Failed to get '%s' [%s]\n", argv[1], errorBuffer);
exit(EXIT_FAILURE);
return EXIT_FAILURE;
}
// Parse the (assumed) HTML code
@@ -292,5 +291,5 @@ int main(int argc, char *argv[])
// Display the extracted title
printf("Title: %s\n", title.c_str());
return EXIT_SUCCESS;
return (int)result;
}