mirror of
https://github.com/open-goal/jak-project
synced 2026-09-10 04:22:13 -04:00
5d5e35fb9b
## Problem Windows builds break with a current local toolchain (Scoop LLVM 22.1.8, CMake 4.4.0, VS 2026), in two independent ways: 1. The build stops at curl's deliberate guard: `#error "no non-blocking method was found/used/set"` in `third-party/curl/lib/nonblock.c`. 2. From the second configure onward, `cmake --build` re-runs CMake in an endless loop (observed 42 consecutive reconfigure cycles in a single build). Likely the same mechanism behind the "endlessly building" VS 2026 note in `docs/setup/dev/vs.md`. ## Root cause 1. `third-party/curl/CMake/CurlTests.c` passes `int *` to `ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes `-Wincompatible-pointer-types` to a hard error in C, so the `HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and `curl_config.h` never defines it. Upstream CI does not see this because the windows-2022 runner image ships an older LLVM. GCC 14 promotes the same warning to a hard error, which is very likely the `CurlTests.c.obj` failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0 (curl/curl#13578). 2. The root CMakeLists copies the build tree's `compile_commands.json` into `<src>/build/` for clangd using `configure_file()`, which registers its input as a configure dependency. CMake rewrites `compile_commands.json` late in every generation, after `CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same Ninja regen rule), so once the dependency is registered the rule is deterministically dirty and every `ninja` invocation re-runs CMake. A pristine first configure is safe (the file does not exist yet, so the `if(EXISTS ...)` guard skips the copy), which is why the loop looks machine- or IDE-specific. ## Fix 1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag (previously `curl-8_3_0`), which carries the upstream probe fix plus two years of upstream development; `vendor.yaml` updated to match. Adjustments the version jump forced: - curl 8.15 removed the native macOS Secure Transport backend (`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like Linux. The two macOS workflows install Homebrew `openssl@3` and export `OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same two lines. - `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10; pinned OFF to keep the previous no-compression behavior and avoid silently linking whatever the CI images happen to have. - curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked into discord-rpc's identically named option and broke configure at a nonexistent `examples/send-presence` directory; pinned OFF ahead of the third-party subdirectories. The diff is dominated by the mechanical tag-tree swap under `third-party/curl` (linguist-vendored, collapsed in review). The hand-written changes are `CMakeLists.txt`, the two macOS workflows, `docs/setup/system/macos.md`, and `vendor.yaml`. 2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy with no configure dependency registered. (`file(COPY_FILE ... ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21, above the declared `cmake_minimum_required(VERSION 3.10)`.) ## Test plan - [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache seeding) completes and logs `Enabled SSL backends: Schannel`; the FIONBIO probe passes without the previous `#error` - [x] Full Windows Release build from scratch in the branch worktree (all 1422 targets) - [x] goalc-test suite: 1509 passed, 0 failed - [x] Second consecutive configure with `compile_commands.json` present: the regen rule in `build.ninja` has no `compile_commands.json` input; `<src>/build/compile_commands.json` is still refreshed for clangd - [x] Repeated `ninja` invocations after a full build no longer re-run CMake - [x] macOS Intel and ARM CI green (first exercise of the OpenSSL backend switch) --- I work off a self-hosted forge, so this GitHub account is quiet; the configure logs and ninja dirty-node traces from the investigation are available if anyone wants the raw data. (AI-assisted)
367 lines
11 KiB
C
Vendored
Generated
367 lines
11 KiB
C
Vendored
Generated
/***************************************************************************
|
|
* _ _ ____ _
|
|
* Project ___| | | | _ \| |
|
|
* / __| | | | |_) | |
|
|
* | (__| |_| | _ <| |___
|
|
* \___|\___/|_| \_\_____|
|
|
*
|
|
* Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
|
|
*
|
|
* This software is licensed as described in the file COPYING, which
|
|
* you should have received as part of this distribution. The terms
|
|
* are also available at https://curl.se/docs/copyright.html.
|
|
*
|
|
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
|
|
* copies of the Software, and permit persons to whom the Software is
|
|
* furnished to do so, under the terms of the COPYING file.
|
|
*
|
|
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
|
|
* KIND, either express or implied.
|
|
*
|
|
* SPDX-License-Identifier: curl
|
|
*
|
|
***************************************************************************/
|
|
#include "tool_setup.h"
|
|
|
|
#include "tool_cfgable.h"
|
|
#include "tool_msgs.h"
|
|
#include "tool_cb_wrt.h"
|
|
#include "tool_operate.h"
|
|
|
|
#ifdef _WIN32
|
|
#define OPENMODE (_S_IREAD | _S_IWRITE)
|
|
#else
|
|
#define OPENMODE (S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH)
|
|
#endif
|
|
|
|
/* create/open a local file for writing, return TRUE on success */
|
|
bool tool_create_output_file(struct OutStruct *outs,
|
|
struct OperationConfig *config)
|
|
{
|
|
FILE *file = NULL;
|
|
const char *fname = outs->filename;
|
|
DEBUGASSERT(outs);
|
|
DEBUGASSERT(config);
|
|
DEBUGASSERT(fname && *fname);
|
|
|
|
if(config->file_clobber_mode == CLOBBER_ALWAYS ||
|
|
(config->file_clobber_mode == CLOBBER_DEFAULT &&
|
|
!outs->is_cd_filename)) {
|
|
/* open file for writing */
|
|
file = curlx_fopen(fname, "wb");
|
|
}
|
|
else {
|
|
int fd;
|
|
do {
|
|
fd = curlx_open(fname, O_CREAT | O_WRONLY | O_EXCL | CURL_O_BINARY,
|
|
OPENMODE);
|
|
/* Keep retrying in the hope that it is not interrupted sometime */
|
|
/* !checksrc! disable ERRNOVAR 1 */
|
|
} while(fd == -1 && errno == EINTR);
|
|
if(config->file_clobber_mode == CLOBBER_NEVER && fd == -1) {
|
|
int next_num = 1;
|
|
struct dynbuf fbuffer;
|
|
char *newfile;
|
|
curlx_dyn_init(&fbuffer, 1025);
|
|
/* !checksrc! disable ERRNOVAR 1 */
|
|
while(fd == -1 && /* have not successfully opened a file */
|
|
(errno == EEXIST || errno == EISDIR) &&
|
|
/* because we keep having files that already exist */
|
|
next_num < 100 /* and we have not reached the retry limit */) {
|
|
curlx_dyn_reset(&fbuffer);
|
|
if(curlx_dyn_addf(&fbuffer, "%s.%d", fname, next_num))
|
|
return FALSE;
|
|
next_num++;
|
|
do {
|
|
fd = curlx_open(curlx_dyn_ptr(&fbuffer),
|
|
O_CREAT | O_WRONLY | O_EXCL | CURL_O_BINARY,
|
|
OPENMODE);
|
|
/* Keep retrying in the hope that it is not interrupted sometime */
|
|
} while(fd == -1 && errno == EINTR);
|
|
}
|
|
newfile = curlx_dyn_ptr(&fbuffer); /* remember the new one */
|
|
if(newfile) {
|
|
outs->filename = newfile;
|
|
outs->alloc_filename = TRUE;
|
|
}
|
|
}
|
|
/* An else statement to not overwrite existing files and not retry with
|
|
new numbered names (which would cover
|
|
config->file_clobber_mode == CLOBBER_DEFAULT && outs->is_cd_filename)
|
|
is not needed because we would have failed earlier, in the while loop
|
|
and `fd` would now be -1 */
|
|
if(fd != -1) {
|
|
file = curlx_fdopen(fd, "wb");
|
|
if(!file)
|
|
curlx_close(fd);
|
|
}
|
|
}
|
|
|
|
if(!file) {
|
|
char errbuf[STRERROR_LEN];
|
|
warnf("Failed to open the file %s: %s", fname,
|
|
curlx_strerror(errno, errbuf, sizeof(errbuf)));
|
|
return FALSE;
|
|
}
|
|
outs->regular_file = TRUE;
|
|
outs->fopened = TRUE;
|
|
outs->stream = file;
|
|
outs->bytes = 0;
|
|
outs->init = 0;
|
|
return TRUE;
|
|
}
|
|
|
|
#ifdef _WIN32
|
|
static size_t win_console(intptr_t fhnd, struct OutStruct *outs,
|
|
char *buffer, size_t bytes,
|
|
size_t *retp)
|
|
{
|
|
DWORD chars_written;
|
|
unsigned char *rbuf = (unsigned char *)buffer;
|
|
DWORD rlen = (DWORD)bytes;
|
|
|
|
#define IS_TRAILING_BYTE(x) (0x80 <= (x) && (x) < 0xC0)
|
|
|
|
/* attempt to complete an incomplete UTF-8 sequence from previous call. the
|
|
sequence does not have to be well-formed. */
|
|
if(outs->utf8seq[0] && rlen) {
|
|
bool complete = FALSE;
|
|
/* two byte sequence (lead byte 110yyyyy) */
|
|
if(0xC0 <= outs->utf8seq[0] && outs->utf8seq[0] < 0xE0) {
|
|
outs->utf8seq[1] = *rbuf++;
|
|
--rlen;
|
|
complete = TRUE;
|
|
}
|
|
/* three byte sequence (lead byte 1110zzzz) */
|
|
else if(0xE0 <= outs->utf8seq[0] && outs->utf8seq[0] < 0xF0) {
|
|
if(!outs->utf8seq[1]) {
|
|
outs->utf8seq[1] = *rbuf++;
|
|
--rlen;
|
|
}
|
|
if(rlen && !outs->utf8seq[2]) {
|
|
outs->utf8seq[2] = *rbuf++;
|
|
--rlen;
|
|
complete = TRUE;
|
|
}
|
|
}
|
|
/* four byte sequence (lead byte 11110uuu) */
|
|
else if(0xF0 <= outs->utf8seq[0] && outs->utf8seq[0] < 0xF8) {
|
|
if(!outs->utf8seq[1]) {
|
|
outs->utf8seq[1] = *rbuf++;
|
|
--rlen;
|
|
}
|
|
if(rlen && !outs->utf8seq[2]) {
|
|
outs->utf8seq[2] = *rbuf++;
|
|
--rlen;
|
|
}
|
|
if(rlen && !outs->utf8seq[3]) {
|
|
outs->utf8seq[3] = *rbuf++;
|
|
--rlen;
|
|
complete = TRUE;
|
|
}
|
|
}
|
|
|
|
if(complete) {
|
|
WCHAR prefix[3] = { 0 }; /* UTF-16 (1-2 WCHARs) + NUL */
|
|
|
|
if(MultiByteToWideChar(CP_UTF8, 0, (LPCSTR)outs->utf8seq, -1,
|
|
prefix, CURL_ARRAYSIZE(prefix))) {
|
|
DEBUGASSERT(prefix[2] == L'\0');
|
|
if(!WriteConsoleW((HANDLE)fhnd, prefix, prefix[1] ? 2 : 1,
|
|
&chars_written, NULL)) {
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
}
|
|
/* else: UTF-8 input was not well formed and OS is pre-Vista which drops
|
|
invalid characters instead of writing U+FFFD to output. */
|
|
memset(outs->utf8seq, 0, sizeof(outs->utf8seq));
|
|
}
|
|
}
|
|
|
|
/* suppress an incomplete utf-8 sequence at end of rbuf */
|
|
if(!outs->utf8seq[0] && rlen && (rbuf[rlen - 1] & 0x80)) {
|
|
/* check for lead byte from a two, three or four byte sequence */
|
|
if(0xC0 <= rbuf[rlen - 1] && rbuf[rlen - 1] < 0xF8) {
|
|
outs->utf8seq[0] = rbuf[rlen - 1];
|
|
rlen -= 1;
|
|
}
|
|
else if(rlen >= 2 && IS_TRAILING_BYTE(rbuf[rlen - 1])) {
|
|
/* check for lead byte from a three or four byte sequence */
|
|
if(0xE0 <= rbuf[rlen - 2] && rbuf[rlen - 2] < 0xF8) {
|
|
outs->utf8seq[0] = rbuf[rlen - 2];
|
|
outs->utf8seq[1] = rbuf[rlen - 1];
|
|
rlen -= 2;
|
|
}
|
|
else if(rlen >= 3 && IS_TRAILING_BYTE(rbuf[rlen - 2])) {
|
|
/* check for lead byte from a four byte sequence */
|
|
if(0xF0 <= rbuf[rlen - 3] && rbuf[rlen - 3] < 0xF8) {
|
|
outs->utf8seq[0] = rbuf[rlen - 3];
|
|
outs->utf8seq[1] = rbuf[rlen - 2];
|
|
outs->utf8seq[2] = rbuf[rlen - 1];
|
|
rlen -= 3;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
if(rlen) {
|
|
/* calculate buffer size for wide characters */
|
|
DWORD len = (DWORD)MultiByteToWideChar(CP_UTF8, 0, (LPCSTR)rbuf,
|
|
(int)rlen, NULL, 0);
|
|
if(!len)
|
|
return CURL_WRITEFUNC_ERROR;
|
|
|
|
/* grow the buffer if needed */
|
|
if(len > global->term.len) {
|
|
wchar_t *buf = curlx_realloc(global->term.buf, len * sizeof(wchar_t));
|
|
if(!buf)
|
|
return CURL_WRITEFUNC_ERROR;
|
|
global->term.len = len;
|
|
global->term.buf = buf;
|
|
}
|
|
|
|
len = (DWORD)MultiByteToWideChar(CP_UTF8, 0, (LPCSTR)rbuf, (int)rlen,
|
|
global->term.buf,
|
|
(int)len);
|
|
if(!len)
|
|
return CURL_WRITEFUNC_ERROR;
|
|
|
|
if(!WriteConsoleW((HANDLE) fhnd, global->term.buf,
|
|
len, &chars_written, NULL))
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
|
|
*retp = bytes;
|
|
return 0;
|
|
}
|
|
#endif /* _WIN32 */
|
|
|
|
/*
|
|
** callback for CURLOPT_WRITEFUNCTION
|
|
*/
|
|
|
|
size_t tool_write_cb(char *buffer, size_t sz, size_t nmemb, void *userdata)
|
|
{
|
|
size_t rc;
|
|
struct per_transfer *per = userdata;
|
|
struct OutStruct *outs = &per->outs;
|
|
struct OperationConfig *config = per->config;
|
|
size_t bytes = sz * nmemb;
|
|
bool is_tty = (bool)global->isatty;
|
|
#ifdef _WIN32
|
|
CONSOLE_SCREEN_BUFFER_INFO console_info;
|
|
intptr_t fhnd;
|
|
#endif
|
|
|
|
if(outs->out_null)
|
|
return bytes;
|
|
|
|
#ifdef DEBUGBUILD
|
|
{
|
|
char *tty = curl_getenv("CURL_ISATTY");
|
|
if(tty) {
|
|
is_tty = TRUE;
|
|
curl_free(tty);
|
|
}
|
|
}
|
|
|
|
if(config->show_headers) {
|
|
if(bytes > (size_t)CURL_MAX_HTTP_HEADER) {
|
|
warnf("Header data size exceeds write limit");
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
}
|
|
else {
|
|
if(bytes > (size_t)CURL_MAX_WRITE_SIZE) {
|
|
warnf("Data size exceeds write limit");
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
}
|
|
|
|
{
|
|
/* Some internal congruency checks on received OutStruct */
|
|
bool check_fails = FALSE;
|
|
if(outs->filename) {
|
|
/* regular file */
|
|
if(!*outs->filename)
|
|
check_fails = TRUE;
|
|
if(!outs->regular_file)
|
|
check_fails = TRUE;
|
|
if(outs->fopened && !outs->stream)
|
|
check_fails = TRUE;
|
|
if(!outs->fopened && outs->stream)
|
|
check_fails = TRUE;
|
|
if(!outs->fopened && outs->bytes)
|
|
check_fails = TRUE;
|
|
}
|
|
else {
|
|
/* standard stream */
|
|
if(!outs->stream || outs->regular_file || outs->fopened)
|
|
check_fails = TRUE;
|
|
if(outs->alloc_filename || outs->is_cd_filename || outs->init)
|
|
check_fails = TRUE;
|
|
}
|
|
if(check_fails) {
|
|
warnf("Invalid output struct data for write callback");
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
}
|
|
#endif /* DEBUGBUILD */
|
|
|
|
if(!outs->stream && !tool_create_output_file(outs, per->config))
|
|
return CURL_WRITEFUNC_ERROR;
|
|
|
|
if(is_tty && (outs->bytes < 2000) && !config->terminal_binary_ok) {
|
|
/* binary output to terminal? */
|
|
if(memchr(buffer, 0, bytes)) {
|
|
warnf("Binary output can mess up your terminal. "
|
|
"Use \"--output -\" to tell curl to output it to your terminal "
|
|
"anyway, or consider \"--output <FILE>\" to save to a file.");
|
|
config->synthetic_error = TRUE;
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
}
|
|
|
|
#ifdef _WIN32
|
|
fhnd = _get_osfhandle(fileno(outs->stream));
|
|
/* if Windows console then UTF-8 must be converted to UTF-16 */
|
|
if(isatty(fileno(outs->stream)) &&
|
|
GetConsoleScreenBufferInfo((HANDLE)fhnd, &console_info)) {
|
|
size_t retval = win_console(fhnd, outs, buffer, bytes, &rc);
|
|
if(retval)
|
|
return retval;
|
|
}
|
|
else
|
|
#endif
|
|
{
|
|
if(per->hdrcbdata.headlist) {
|
|
if(tool_write_headers(&per->hdrcbdata, outs->stream))
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
rc = fwrite(buffer, sz, nmemb, outs->stream);
|
|
}
|
|
|
|
if(bytes == rc)
|
|
/* we added this amount of data to the output */
|
|
outs->bytes += bytes;
|
|
|
|
if(config->readbusy) {
|
|
config->readbusy = FALSE;
|
|
curl_easy_pause(per->curl, CURLPAUSE_CONT);
|
|
}
|
|
|
|
if(config->nobuffer) {
|
|
/* output buffering disabled */
|
|
int res;
|
|
do {
|
|
res = fflush(outs->stream);
|
|
/* Keep retrying in the hope that it is not interrupted sometime */
|
|
/* !checksrc! disable ERRNOVAR 1 */
|
|
} while(res && errno == EINTR);
|
|
if(res)
|
|
return CURL_WRITEFUNC_ERROR;
|
|
}
|
|
|
|
return rc;
|
|
}
|