Alexander J. Semenuk
5d5e35fb9b
## Problem Windows builds break with a current local toolchain (Scoop LLVM 22.1.8, CMake 4.4.0, VS 2026), in two independent ways: 1. The build stops at curl's deliberate guard: `#error "no non-blocking method was found/used/set"` in `third-party/curl/lib/nonblock.c`. 2. From the second configure onward, `cmake --build` re-runs CMake in an endless loop (observed 42 consecutive reconfigure cycles in a single build). Likely the same mechanism behind the "endlessly building" VS 2026 note in `docs/setup/dev/vs.md`. ## Root cause 1. `third-party/curl/CMake/CurlTests.c` passes `int *` to `ioctlsocket()`, whose third parameter is `u_long *`. Clang 22 promotes `-Wincompatible-pointer-types` to a hard error in C, so the `HAVE_IOCTLSOCKET_FIONBIO` try_compile silently fails and `curl_config.h` never defines it. Upstream CI does not see this because the windows-2022 runner image ships an older LLVM. GCC 14 promotes the same warning to a hard error, which is very likely the `CurlTests.c.obj` failure reported from MSYS2 in open-goal/jak-project#3551. Upstream curl hit the identical problem with GCC 14 and fixed the probe in curl 8.8.0 (curl/curl#13578). 2. The root CMakeLists copies the build tree's `compile_commands.json` into `<src>/build/` for clangd using `configure_file()`, which registers its input as a configure dependency. CMake rewrites `compile_commands.json` late in every generation, after `CTestTestfile.cmake` and `cmake_install.cmake` (outputs of the same Ninja regen rule), so once the dependency is registered the rule is deterministically dirty and every `ninja` invocation re-runs CMake. A pristine first configure is safe (the file does not exist yet, so the `if(EXISTS ...)` guard skips the copy), which is why the loop looks machine- or IDE-specific. ## Fix 1. Per review, re-vendor `third-party/curl` at the `curl-8_21_0` tag (previously `curl-8_3_0`), which carries the upstream probe fix plus two years of upstream development; `vendor.yaml` updated to match. Adjustments the version jump forced: - curl 8.15 removed the native macOS Secure Transport backend (`CURL_USE_SECTRANSP`), so macOS now builds curl against OpenSSL like Linux. The two macOS workflows install Homebrew `openssl@3` and export `OPENSSL_ROOT_DIR` (keg-only), and the macOS setup docs gained the same two lines. - `CURL_BROTLI` / `CURL_ZSTD` switched to AUTO-detection in curl 8.10; pinned OFF to keep the previous no-compression behavior and avoid silently linking whatever the CI images happen to have. - curl's new top-level `BUILD_EXAMPLES` cache option (default ON) leaked into discord-rpc's identically named option and broke configure at a nonexistent `examples/send-presence` directory; pinned OFF ahead of the third-party subdirectories. The diff is dominated by the mechanical tag-tree swap under `third-party/curl` (linguist-vendored, collapsed in review). The hand-written changes are `CMakeLists.txt`, the two macOS workflows, `docs/setup/system/macos.md`, and `vendor.yaml`. 2. Swap `configure_file()` for `file(COPY ...)`: the same clangd copy with no configure dependency registered. (`file(COPY_FILE ... ONLY_IF_DIFFERENT)` would be cleaner still but requires CMake 3.21, above the declared `cmake_minimum_required(VERSION 3.10)`.) ## Test plan - [x] Fresh `cmake --preset Release-windows-clang` (LLVM 22, no cache seeding) completes and logs `Enabled SSL backends: Schannel`; the FIONBIO probe passes without the previous `#error` - [x] Full Windows Release build from scratch in the branch worktree (all 1422 targets) - [x] goalc-test suite: 1509 passed, 0 failed - [x] Second consecutive configure with `compile_commands.json` present: the regen rule in `build.ninja` has no `compile_commands.json` input; `<src>/build/compile_commands.json` is still refreshed for clangd - [x] Repeated `ninja` invocations after a full build no longer re-run CMake - [x] macOS Intel and ARM CI green (first exercise of the OpenSSL backend switch) --- I work off a self-hosted forge, so this GitHub account is quiet; the configure logs and ninja dirty-node traces from the investigation are available if anyone wants the raw data. (AI-assisted)
5.9 KiB
Vendored
Generated
TLS Certificate Verification
Native vs file based
If curl was built with Schannel support, then curl uses the Windows native CA store for verification. On Apple operating systems, it is possible to use Apple's "SecTrust" services for certain TLS backends, details below. All other TLS libraries use a file based CA store by default.
Verification
Every trusted server certificate is digitally signed by a Certificate Authority, a CA.
In your local CA store you have a collection of certificates from trusted certificate authorities that TLS clients like curl use to verify servers.
curl does certificate verification by default. This is done by verifying the signature and making sure the certificate was crafted for the server name provided in the URL.
If you communicate with HTTPS, FTPS or other TLS-using servers using certificates signed by a CA whose certificate is present in the store, you can be sure that the remote server really is the one it claims to be.
If the remote server uses a self-signed certificate, if you do not install a CA cert store, if the server uses a certificate signed by a CA that is not included in the store you use or if the remote host is an impostor impersonating your favorite site, the certificate check fails and reports an error.
If you think it wrongly failed the verification, consider one of the following sections.
Skip verification
Tell curl to not verify the peer with -k/--insecure.
We strongly recommend this is avoided and that even if you end up doing this for experimentation or development, never skip verification in production.
Use a custom CA store
Get a CA certificate that can verify the remote server and use the proper option to point out this CA cert for verification when connecting - for this specific transfer only.
With the curl command line tool: --cacert [file]
If you use the curl command line tool without a native CA store, then you can
specify your own CA cert file by setting the environment variable
CURL_CA_BUNDLE to the path of your choice. SSL_CERT_FILE and SSL_CERT_DIR
are also supported.
If you are using the curl command line tool on Windows, curl searches for a CA
cert file named curl-ca-bundle.crt in these directories and in this order:
- application's directory
- current working directory
- Windows System directory (e.g. C:\Windows\System32)
- Windows Directory (e.g. C:\Windows)
- all directories along %PATH%
curl 8.11.0 added a build-time option to disable this search behavior, and another option to restrict search to the application's directory.
curl 8.19.0 added a build-time option to enable Native CA by default on
Windows. This build-time option by default also disables searching for
a curl-ca-bundle.crt on disk.
Use the native store
In several environments, in particular on Microsoft and Apple operating systems, you can ask curl to use the system's native CA store when verifying the certificate. Depending on how curl was built, this may already be the default.
With the curl command line tool: --ca-native.
Modify the CA store
Add the CA cert for your server to the existing default CA certificate store.
Usually you can figure out the path to the local CA store by looking at the
verbose output that curl -v shows when you connect to an HTTPS site.
Change curl's default CA store
The default CA certificate store curl uses is set at build time. When you build curl you can point out your preferred path.
Extract CA cert from a server
curl -w %{certs} https://example.com > cacert.pem
The certificate has BEGIN CERTIFICATE and END CERTIFICATE markers.
Get the Mozilla CA store
Download a version of the Firefox CA store converted to PEM format on the CA Extract page. It always features the latest Firefox bundle.
Native CA store
Windows + Schannel
If curl was built with Schannel, then curl uses the certificates that are built into the OS. These are the same certificates that appear in the Internet Options control panel (under Windows). Any custom security rules for certificates are honored.
Schannel runs CRL checks on certificates unless peer verification is disabled.
Apple + OpenSSL/GnuTLS
When curl is built with Apple SecTrust enabled and uses an OpenSSL compatible TLS backend or GnuTLS, the default verification is handled by that Apple service. As in:
curl https://example.com
You may still provide your own certificates on the command line, such as:
curl --cacert mycerts.pem https://example.com
In this situation, Apple SecTrust is not used and verification is done
only with the trust anchors found in mycerts.pem. If you want both
Apple SecTrust and your own file to be considered, use:
curl --ca-native --cacert mycerts.pem https://example.com
Other Combinations
How well the use of native CA stores work in all other combinations depends on the TLS backend and the OS. Many TLS backends offer functionality to access the native CA on a range of operating systems. Some provide this only on specific configurations.
Specific support in curl exists for Windows and OpenSSL compatible TLS backends. It tries to load the certificates from the Windows "CA" and "ROOT" stores for transfers requesting the native CA. Due to Window's delayed population of those stores, this might not always find all certificates.
HTTPS proxy
curl can do HTTPS to the proxy separately from the connection to the server.
This TLS connection is handled and verified separately from the server
connection so instead of --insecure and --cacert to control the
certificate verification, you use --proxy-insecure and --proxy-cacert.
With these options, you make sure that the TLS connection and the trust of the
proxy can be kept totally separate from the TLS connection to the server.