mirror of
https://github.com/open-goal/jak-project
synced 2026-09-03 02:14:07 -04:00
Alexander J. Semenuk
5d5e35fb9b
## Problem Windows builds break with a current local toolchain (Scoop LLVM 22.1.8, CMake 4.4.0, VS 2026), in two independent ways: 1. The build stops at curl's deliberate guard: `#error "no non-blocking method was found/used/set"` in `third-party/curl/lib/nonblock.c`. 2. From the second configure onward, `cmake --build` re-runs CMake in an endless loop (observed 42 consecutive reconfigure cycles in a single build). Likely the same mechanism behind the "endlessly building" VS 2026 note in `docs/setup/dev/vs.md`. ## Root cause 1. `third-party/curl/CMake/CurlTests.c` passes `int *` to `ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes `-Wincompatible-pointer-types` to a hard error in C, so the `HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and `curl_config.h` never defines it. Upstream CI does not see this because the windows-2022 runner image ships an older LLVM. GCC 14 promotes the same warning to a hard error, which is very likely the `CurlTests.c.obj` failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0 (curl/curl#13578). 2. The root CMakeLists copies the build tree's `compile_commands.json` into `<src>/build/` for clangd using `configure_file()`, which registers its input as a configure dependency. CMake rewrites `compile_commands.json` late in every generation, after `CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same Ninja regen rule), so once the dependency is registered the rule is deterministically dirty and every `ninja` invocation re-runs CMake. A pristine first configure is safe (the file does not exist yet, so the `if(EXISTS ...)` guard skips the copy), which is why the loop looks machine- or IDE-specific. ## Fix 1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag (previously `curl-8_3_0`), which carries the upstream probe fix plus two years of upstream development; `vendor.yaml` updated to match. Adjustments the version jump forced: - curl 8.15 removed the native macOS Secure Transport backend (`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like Linux. The two macOS workflows install Homebrew `openssl@3` and export `OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same two lines. - `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10; pinned OFF to keep the previous no-compression behavior and avoid silently linking whatever the CI images happen to have. - curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked into discord-rpc's identically named option and broke configure at a nonexistent `examples/send-presence` directory; pinned OFF ahead of the third-party subdirectories. The diff is dominated by the mechanical tag-tree swap under `third-party/curl` (linguist-vendored, collapsed in review). The hand-written changes are `CMakeLists.txt`, the two macOS workflows, `docs/setup/system/macos.md`, and `vendor.yaml`. 2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy with no configure dependency registered. (`file(COPY_FILE ... ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21, above the declared `cmake_minimum_required(VERSION 3.10)`.) ## Test plan - [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache seeding) completes and logs `Enabled SSL backends: Schannel`; the FIONBIO probe passes without the previous `#error` - [x] Full Windows Release build from scratch in the branch worktree (all 1422 targets) - [x] goalc-test suite: 1509 passed, 0 failed - [x] Second consecutive configure with `compile_commands.json` present: the regen rule in `build.ninja` has no `compile_commands.json` input; `<src>/build/compile_commands.json` is still refreshed for clangd - [x] Repeated `ninja` invocations after a full build no longer re-run CMake - [x] macOS Intel and ARM CI green (first exercise of the OpenSSL backend switch) --- I work off a self-hosted forge, so this GitHub account is quiet; the configure logs and ninja dirty-node traces from the investigation are available if anyone wants the raw data. (AI-assisted)
447 lines
13 KiB
Perl
Vendored
Generated
447 lines
13 KiB
Perl
Vendored
Generated
#!/usr/bin/env perl
|
|
#***************************************************************************
|
|
# _ _ ____ _
|
|
# Project ___| | | | _ \| |
|
|
# / __| | | | |_) | |
|
|
# | (__| |_| | _ <| |___
|
|
# \___|\___/|_| \_\_____|
|
|
#
|
|
# Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
|
|
#
|
|
# This software is licensed as described in the file COPYING, which
|
|
# you should have received as part of this distribution. The terms
|
|
# are also available at https://curl.se/docs/copyright.html.
|
|
#
|
|
# You may opt to use, copy, modify, merge, publish, distribute and/or sell
|
|
# copies of the Software, and permit persons to whom the Software is
|
|
# furnished to do so, under the terms of the COPYING file.
|
|
#
|
|
# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
|
|
# KIND, either express or implied.
|
|
#
|
|
# SPDX-License-Identifier: curl
|
|
#
|
|
###########################################################################
|
|
#
|
|
# Example input:
|
|
#
|
|
# MEM mprintf.c:1094 malloc(32) = e5718
|
|
# MEM mprintf.c:1103 realloc(e5718, 64) = e6118
|
|
# MEM sendf.c:232 free(f6520)
|
|
|
|
package memanalyzer;
|
|
|
|
use strict;
|
|
use warnings;
|
|
|
|
BEGIN {
|
|
use base qw(Exporter);
|
|
|
|
our @EXPORT = qw(
|
|
memanalyze
|
|
);
|
|
}
|
|
|
|
my $memsum;
|
|
my $maxmem;
|
|
|
|
sub newtotal {
|
|
my ($newtot) = @_;
|
|
# count a max here
|
|
|
|
if($newtot > $maxmem) {
|
|
$maxmem = $newtot;
|
|
}
|
|
}
|
|
|
|
sub memanalyze {
|
|
my ($file, $verbose, $trace, $showlimit) = @_;
|
|
my @res;
|
|
|
|
my $mallocs = 0;
|
|
my $callocs = 0;
|
|
my $reallocs = 0;
|
|
my $strdups = 0;
|
|
my $wcsdups = 0;
|
|
my $sockets = 0;
|
|
|
|
$memsum = 0; # the total number of memory allocated over the lifetime
|
|
$maxmem = 0; # the high water mark
|
|
|
|
open(my $fileh, "<", $file) or return ();
|
|
|
|
if($showlimit) {
|
|
while(<$fileh>) {
|
|
if(/^LIMIT.*memlimit$/) {
|
|
push @res, $_;
|
|
last;
|
|
}
|
|
}
|
|
close($fileh);
|
|
return @res;
|
|
}
|
|
|
|
my %sizeataddr;
|
|
my %getmem;
|
|
|
|
my $totalmem = 0;
|
|
my $frees = 0;
|
|
|
|
my $dup;
|
|
my $size;
|
|
my $addr;
|
|
|
|
my %filedes;
|
|
my %getfile;
|
|
|
|
my %fopen;
|
|
my %fopenfile;
|
|
my $openfile = 0;
|
|
my $fopens = 0;
|
|
|
|
my %addrinfo;
|
|
my %addrinfofile;
|
|
my $addrinfos = 0;
|
|
|
|
my $source;
|
|
my $linenum;
|
|
my $function;
|
|
|
|
my $lnum = 0;
|
|
|
|
while(<$fileh>) {
|
|
chomp $_;
|
|
my $line = $_;
|
|
$lnum++;
|
|
if($line =~ /^BT/) {
|
|
# back-trace, ignore
|
|
}
|
|
elsif($line =~ /^LIMIT ([^ ]*):(\d*) (.*)/) {
|
|
# new memory limit test prefix
|
|
my $i = $3;
|
|
my ($source, $linenum) = ($1, $2);
|
|
if($trace && ($i =~ /([^ ]*) reached memlimit/)) {
|
|
push @res, "LIMIT: $1 returned error at $source:$linenum\n";
|
|
}
|
|
}
|
|
elsif($line =~ /^MEM ([^ ]*):(\d*) (.*)/) {
|
|
# generic match for the filename+linenumber
|
|
$source = $1;
|
|
$linenum = $2;
|
|
$function = $3;
|
|
|
|
if($function =~ /free\((\(nil\)|0x([0-9a-f]*))/) {
|
|
$addr = $2;
|
|
if($1 eq "(nil)") {
|
|
; # do nothing when free(NULL)
|
|
}
|
|
elsif(!exists $sizeataddr{$addr}) {
|
|
push @res, "FREE ERROR: No memory allocated: $line\n";
|
|
}
|
|
elsif(-1 == $sizeataddr{$addr}) {
|
|
push @res, "FREE ERROR: Memory freed twice: $line\n";
|
|
push @res, "FREE ERROR: Previously freed at: $getmem{$addr}\n";
|
|
}
|
|
else {
|
|
$totalmem -= $sizeataddr{$addr};
|
|
if($trace) {
|
|
push @res, "FREE: malloc at $getmem{$addr} is freed again at $source:$linenum\n";
|
|
push @res, "FREE: $sizeataddr{$addr} bytes freed, left allocated: $totalmem bytes\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$frees++;
|
|
|
|
$sizeataddr{$addr} = -1; # set -1 to mark as freed
|
|
$getmem{$addr} = "$source:$linenum";
|
|
}
|
|
}
|
|
elsif($function =~ /malloc\((\d*)\) = 0x([0-9a-f]*)/) {
|
|
$size = $1;
|
|
$addr = $2;
|
|
|
|
if($sizeataddr{$addr} && $sizeataddr{$addr}>0) {
|
|
# this means weeeeeirdo
|
|
push @res, "Mixed debug compile ($source:$linenum at line $lnum), rebuild curl now\n";
|
|
push @res, "We think $sizeataddr{$addr} bytes are already allocated at that memory address: $addr!\n";
|
|
}
|
|
|
|
$sizeataddr{$addr} = $size;
|
|
$totalmem += $size;
|
|
$memsum += $size;
|
|
|
|
if($trace) {
|
|
push @res, "MALLOC: malloc($size) at $source:$linenum makes totally $totalmem bytes\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$mallocs++;
|
|
|
|
$getmem{$addr} = "$source:$linenum";
|
|
}
|
|
elsif($function =~ /calloc\((\d*),(\d*)\) = 0x([0-9a-f]*)/) {
|
|
$size = $1 * $2;
|
|
$addr = $3;
|
|
|
|
my $arg1 = $1;
|
|
my $arg2 = $2;
|
|
|
|
if($sizeataddr{$addr} && $sizeataddr{$addr}>0) {
|
|
# this means weeeeeirdo
|
|
push @res, "Mixed debug compile ($source:$linenum at line $lnum), rebuild curl now\n";
|
|
push @res, "We think $sizeataddr{$addr} bytes are already allocated at that memory address: $addr!\n";
|
|
}
|
|
|
|
$sizeataddr{$addr} = $size;
|
|
$totalmem += $size;
|
|
$memsum += $size;
|
|
|
|
if($trace) {
|
|
push @res, "CALLOC: calloc($arg1,$arg2) at $source:$linenum makes totally $totalmem bytes\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$callocs++;
|
|
|
|
$getmem{$addr} = "$source:$linenum";
|
|
}
|
|
elsif($function =~ /realloc\((\(nil\)|0x([0-9a-f]*)), (\d*)\) = 0x([0-9a-f]*)/) {
|
|
my ($oldaddr, $newsize, $newaddr) = ($2, $3, $4);
|
|
my $oldsize = '-';
|
|
|
|
if($oldaddr) {
|
|
$oldsize = $sizeataddr{$oldaddr} ? $sizeataddr{$oldaddr} : 0;
|
|
|
|
$totalmem -= $oldsize;
|
|
if($trace) {
|
|
}
|
|
$sizeataddr{$oldaddr} = 0;
|
|
|
|
$getmem{$oldaddr} = "";
|
|
}
|
|
|
|
$totalmem += $newsize;
|
|
$memsum += $newsize;
|
|
$sizeataddr{$newaddr} = $newsize;
|
|
|
|
if($trace) {
|
|
push @res, "REALLOC: $oldsize less bytes and $newsize more bytes ($source:$linenum)\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$reallocs++;
|
|
|
|
$getmem{$newaddr} = "$source:$linenum";
|
|
}
|
|
elsif($function =~ /strdup\(0x([0-9a-f]*)\) \((\d*)\) = 0x([0-9a-f]*)/) {
|
|
# strdup(a5b50) (8) = df7c0
|
|
|
|
$dup = $1;
|
|
$size = $2;
|
|
$addr = $3;
|
|
$getmem{$addr} = "$source:$linenum";
|
|
$sizeataddr{$addr} = $size;
|
|
|
|
$totalmem += $size;
|
|
$memsum += $size;
|
|
|
|
if($trace) {
|
|
push @res, "STRDUP: $size bytes at $getmem{$addr}, makes totally: $totalmem bytes\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$strdups++;
|
|
}
|
|
elsif($function =~ /wcsdup\(0x([0-9a-f]*)\) \((\d*)\) = 0x([0-9a-f]*)/) {
|
|
# wcsdup(a5b50) (8) = df7c0
|
|
|
|
$dup = $1;
|
|
$size = $2;
|
|
$addr = $3;
|
|
$getmem{$addr} = "$source:$linenum";
|
|
$sizeataddr{$addr} = $size;
|
|
|
|
$totalmem += $size;
|
|
$memsum += $size;
|
|
|
|
if($trace) {
|
|
push @res, "WCSDUP: $size bytes at $getmem{$addr}, makes totally: $totalmem bytes\n";
|
|
}
|
|
|
|
newtotal($totalmem);
|
|
$wcsdups++;
|
|
}
|
|
else {
|
|
push @res, "Not recognized input line: $function\n";
|
|
}
|
|
}
|
|
# FD url.c:1282 socket() = 5
|
|
elsif($_ =~ /^FD ([^ ]*):(\d*) (.*)/) {
|
|
# generic match for the filename+linenumber
|
|
$source = $1;
|
|
$linenum = $2;
|
|
$function = $3;
|
|
|
|
if($function =~ /socket\(\) = (\d*)/) {
|
|
$filedes{$1} = 1;
|
|
$getfile{$1} = "$source:$linenum";
|
|
$openfile++;
|
|
$sockets++; # number of socket() calls
|
|
}
|
|
elsif($function =~ /socketpair\(\) = (\d*) (\d*)/) {
|
|
$filedes{$1} = 1;
|
|
$getfile{$1} = "$source:$linenum";
|
|
$openfile++;
|
|
$filedes{$2} = 1;
|
|
$getfile{$2} = "$source:$linenum";
|
|
$openfile++;
|
|
}
|
|
elsif($function =~ /accept\(\) = (\d*)/) {
|
|
$filedes{$1} = 1;
|
|
$getfile{$1} = "$source:$linenum";
|
|
$openfile++;
|
|
}
|
|
elsif($function =~ /sclose\((\d*)\)/) {
|
|
if($filedes{$1} != 1) {
|
|
push @res, "Close without open: $line\n";
|
|
}
|
|
else {
|
|
$filedes{$1} = 0; # closed now
|
|
$openfile--;
|
|
}
|
|
}
|
|
}
|
|
# FILE url.c:1282 fopen("blabla") = 0x5ddd
|
|
elsif($_ =~ /^FILE ([^ ]*):(\d*) (.*)/) {
|
|
# generic match for the filename+linenumber
|
|
$source = $1;
|
|
$linenum = $2;
|
|
$function = $3;
|
|
|
|
if($function =~ /f[d]*open\(\"(.*)\",\"([^\"]*)\"\) = (\(nil\)|0x([0-9a-f]*))/) {
|
|
if($3 eq "(nil)") {
|
|
;
|
|
}
|
|
else {
|
|
$fopen{$4} = 1;
|
|
$fopenfile{$4} = "$source:$linenum";
|
|
$fopens++;
|
|
}
|
|
}
|
|
# fclose(0x1026c8)
|
|
elsif($function =~ /fclose\(0x([0-9a-f]*)\)/) {
|
|
if(!$fopen{$1}) {
|
|
push @res, "fclose() without fopen(): $line\n";
|
|
}
|
|
else {
|
|
$fopen{$1} = 0;
|
|
$fopens--;
|
|
}
|
|
}
|
|
}
|
|
# GETNAME url.c:1901 getnameinfo()
|
|
elsif($_ =~ /^GETNAME ([^ ]*):(\d*) (.*)/) {
|
|
# not much to do
|
|
}
|
|
|
|
# ADDR url.c:1282 getaddrinfo() = 0x5ddd
|
|
elsif($_ =~ /^ADDR ([^ ]*):(\d*) (.*)/) {
|
|
# generic match for the filename+linenumber
|
|
$source = $1;
|
|
$linenum = $2;
|
|
$function = $3;
|
|
|
|
if($function =~ /getaddrinfo\(\) = (\(nil\)|0x([0-9a-f]*))/) {
|
|
my $add = $1;
|
|
if($add eq "(nil)") {
|
|
;
|
|
}
|
|
else {
|
|
$addrinfo{$add} = 1;
|
|
$addrinfofile{$add} = "$source:$linenum";
|
|
$addrinfos++;
|
|
}
|
|
if($trace) {
|
|
push @res, "GETADDRINFO ($source:$linenum)\n";
|
|
}
|
|
}
|
|
# fclose(0x1026c8)
|
|
elsif($function =~ /freeaddrinfo\((0x[0-9a-f]*)\)/) {
|
|
my $addr = $1;
|
|
if(!$addrinfo{$addr}) {
|
|
push @res, "freeaddrinfo() without getaddrinfo(): $line\n";
|
|
}
|
|
else {
|
|
$addrinfo{$addr} = 0;
|
|
$addrinfos--;
|
|
}
|
|
if($trace) {
|
|
push @res, "FREEADDRINFO ($source:$linenum)\n";
|
|
}
|
|
}
|
|
}
|
|
else {
|
|
push @res, "Not recognized prefix line: $line\n";
|
|
}
|
|
}
|
|
close($fileh);
|
|
|
|
if($totalmem) {
|
|
push @res, "Leak detected: memory still allocated: $totalmem bytes\n";
|
|
|
|
for(keys %sizeataddr) {
|
|
$addr = $_;
|
|
$size = $sizeataddr{$addr};
|
|
if($size > 0) {
|
|
push @res, "At $addr, there is $size bytes.\n";
|
|
push @res, " allocated by $getmem{$addr}\n";
|
|
}
|
|
}
|
|
}
|
|
|
|
if($openfile) {
|
|
for(keys %filedes) {
|
|
if($filedes{$_} == 1) {
|
|
push @res, "Open file descriptor created at $getfile{$_}.\n";
|
|
}
|
|
}
|
|
}
|
|
|
|
if($fopens) {
|
|
push @res, "Open FILE handles left at:\n";
|
|
for(keys %fopen) {
|
|
if($fopen{$_} == 1) {
|
|
push @res, "fopen() called at $fopenfile{$_}.\n";
|
|
}
|
|
}
|
|
}
|
|
|
|
if($addrinfos) {
|
|
push @res, "IPv6-style name resolve data left at:\n";
|
|
for(keys %addrinfofile) {
|
|
if($addrinfo{$_} == 1) {
|
|
push @res, "getaddrinfo() called at $addrinfofile{$_}.\n";
|
|
}
|
|
}
|
|
}
|
|
|
|
if($verbose) {
|
|
push @res,
|
|
"Mallocs: $mallocs\n",
|
|
"Reallocs: $reallocs\n",
|
|
"Callocs: $callocs\n",
|
|
"Strdups: $strdups\n",
|
|
"Wcsdups: $wcsdups\n",
|
|
"Frees: $frees\n",
|
|
"Sockets: $sockets\n",
|
|
"Allocations: ".($mallocs + $callocs + $reallocs + $strdups + $wcsdups)."\n",
|
|
"Operations: ".($mallocs + $callocs + $reallocs + $strdups + $wcsdups + $sockets)."\n",
|
|
"Maximum allocated: $maxmem\n",
|
|
"Total allocated: $memsum\n";
|
|
}
|
|
|
|
return @res;
|
|
}
|
|
|
|
1;
|