gh-91349: Replace zlib with zlib-ng in Windows build by zooba · Pull Request #131438 · python/cpython

zooba · 2025-03-19T01:11:08Z

Issue: Use zlib-ng (fast!) rather than mainline stale zlib in binary releases #91349

zooba · 2025-03-19T01:12:37Z

        {
          "referenceCategory": "SECURITY",
-          "referenceLocator": "cpe:2.3:a:zlib:zlib:1.3.1:*:*:*:*:*:*:*",
+          "referenceLocator": "cpe:2.3:a:zlib-ng:zlib-ng:2.2.4:*:*:*:*:*:*:*",


I'll admit I just guessed this, in order to unblock my testing (couldn't build at all with an invalid SBOM). If it's not right, let me know

This is likely the CPE that would get used, although we can't be for certain until a CVE exists with it... the only one I could find with some searching is cpe:...:zlib-ng:minizip-ng which is for a different component but at least the organization is a match.

zooba · 2025-03-19T01:13:14Z

        return -1;
    }
+#ifdef ZLIBNG_VERSION
+    if (PyModule_Add(mod, "ZLIBNG_VERSION",


Figured we'd want some way to detect this other than looking at the text in ZLIB_VERSION

makes sense. annoying to have optional attributes that one would use hasattr or getattr on instead of blindly accessing, but realistically nobody should care as this is more internal informational so this is fine.

Yeah, the main place it'll be used is in pythoninfo, which already handles absent attributes.

We have more offensive optional attributes in our extension modules 😆

zooba · 2025-03-19T01:24:45Z

I've reviewed the three header files added to PC and they seem generic. It's a shame they aren't part of the sources, but figured it's easier/safer to just copy them into our source tree so that the cpython-source-deps repo is a straight import.

bedevere-bot · 2025-03-19T01:35:18Z

🤖 New build scheduled with the buildbot fleet by @zooba for commit 08eecb1 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131438%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

zooba · 2025-03-19T13:45:34Z

The two failing buildbots appear unrelated, and everything else passed. All I changed in the last commit was docs, so I'm not going to rerun the buildbots unless someone thinks it's necessary or worthwhile.

sethmlarson

SBOM changes LGTM!

sethmlarson · 2025-03-19T13:38:26Z

        {
          "referenceCategory": "SECURITY",
-          "referenceLocator": "cpe:2.3:a:zlib:zlib:1.3.1:*:*:*:*:*:*:*",
+          "referenceLocator": "cpe:2.3:a:zlib-ng:zlib-ng:2.2.4:*:*:*:*:*:*:*",


This is likely the CPE that would get used, although we can't be for certain until a CVE exists with it... the only one I could find with some searching is cpe:...:zlib-ng:minizip-ng which is for a different component but at least the organization is a match.

mdboom · 2025-03-19T14:22:55Z

I'm kicking off a pyperformance benchmarking run with this -- we have a non-zero usage of zlib in that suite.

mdboom · 2025-03-19T18:02:38Z

The results for this on pyperformance are basically within the noise / no change. Not all that surprising, given that there are no benchmarks specifically aimed at zlib. https://github.com/faster-cpython/benchmarking-public/tree/main/results/bm-20250319-3.14.0a6+-548daa7

gpshead · 2025-03-19T18:21:32Z

        return -1;
    }
+#ifdef ZLIBNG_VERSION
+    if (PyModule_Add(mod, "ZLIBNG_VERSION",


makes sense. annoying to have optional attributes that one would use hasattr or getattr on instead of blindly accessing, but realistically nobody should care as this is more internal informational so this is fine.

chris-eibl · 2025-03-20T16:57:13Z

This has broken the tailcall builds on Windows: https://github.com/python/cpython/actions/runs/13967899073/job/39102464260

D:\a\cpython\cpython\externals\\zlib-ng-2.2.4\\functable.c(79,26): error : use of undeclared identifier 'slide_hash_sse2'; did you mean 'slide_hash_c'? [D:\a\cpython\cpython\PCbuild\zlib-ng.vcxproj]
D:\a\cpython\cpython\externals\\zlib-ng-2.2.4\\functable.c(123,26): error : use of undeclared identifier 'slide_hash_avx2'; did you mean 'slide_hash_c'? [D:\a\cpython\cpython\PCbuild\zlib-ng.vcxproj]

zooba · 2025-03-20T17:01:30Z

Probably needs some fixed preprocessor checks (hopefully only in what's in the PC directory) to better handle the intrinsics being present/absent.

Though I notice that build warns NOTE: Visual Studio not detected. LLVM does not provide a C/C++ standard library and may be unable to locate MSVC headers., which might also be relevant.

chris-eibl · 2025-03-20T17:24:07Z

I think, this is because here
https://github.com/zlib-ng/zlib-ng/blob/860e4cff7917d93f54f5d7f0bc1d0e8b1a3cb988/fallback_builtins.h#L4-L25
in line 4 __clang__ gets excluded and hence #define HAVE_BUILTIN_CTZ is not set (but in the MSVC case it is).

Then, in
https://github.com/zlib-ng/zlib-ng/blob/860e4cff7917d93f54f5d7f0bc1d0e8b1a3cb988/arch/x86/x86_functions.h#L13-L20
we dont get the prototype for slide_hash_sse2 in line 17, hence the error.

Sorry, I cannot get the permalinks to work here the usual way I am doing it :(

chris-eibl · 2025-03-20T17:30:51Z

Though I notice that build warns NOTE: Visual Studio not detected. LLVM does not provide a C/C++ standard library and may be unable to locate MSVC headers., which might also be relevant.

FWIW, when building with the bundled clang-cl of VS I do not have these warnings, but get the same error.

zooba · 2025-03-20T17:37:20Z

Most likely zlib-ng would generate different versions of the header files I checked into the PC directory for Clang. Figuring out what differences belong in there and detecting it dynamically rather than using CMake is the way forward.

This should get a new issue (because clang-cl isn't a supported configuration, so we don't have to revert this change until it's working). Tag me on it and we can continue there.

chris-eibl · 2025-03-20T17:44:41Z

+      <PrecompiledHeader>NotUsing</PrecompiledHeader>
+      <AdditionalIncludeDirectories>$(zlibNgDir);$(PySourceDir)PC;$(GeneratedZlibNgDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>%(PreprocessorDefinitions);ZLIB_COMPAT;WITH_GZFILEOP;NO_FSEEKO;HAVE_BUILTIN_ASSUME_ALIGNED;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;</PreprocessorDefinitions>
+      <PreprocessorDefinitions Condition="$(Platform) == 'Win32' or $(Platform) == 'x64'">%(PreprocessorDefinitions);X86_FEATURES;X86_HAVE_XSAVE_INTRIN;X86_SSE2;X86_SSSE3;X86_SSE42;X86_PCLMULQDQ_CRC;X86_AVX2;X86_AVX512;X86_AVX512VNNI;X86_VPCLMULQDQ_CRC</PreprocessorDefinitions>


Does this mean that for all executables X86_AVX512 code will be emitted and used?
See e.g. adler32_avx512.c, where the whole file is guarded with X86_AVX512 and there is code like __m512i in it.

I believe it's detected and called dynamically based on availability, but it's all compiled at compile time.

Yeah, I hope so, too.

chris-eibl · 2025-03-20T20:34:53Z

+      <PreprocessorDefinitions>%(PreprocessorDefinitions);ZLIB_COMPAT;WITH_GZFILEOP;NO_FSEEKO;HAVE_BUILTIN_ASSUME_ALIGNED;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;</PreprocessorDefinitions>
+      <PreprocessorDefinitions Condition="$(Platform) == 'Win32' or $(Platform) == 'x64'">%(PreprocessorDefinitions);X86_FEATURES;X86_HAVE_XSAVE_INTRIN;X86_SSE2;X86_SSSE3;X86_SSE42;X86_PCLMULQDQ_CRC;X86_AVX2;X86_AVX512;X86_AVX512VNNI;X86_VPCLMULQDQ_CRC</PreprocessorDefinitions>
+      <PreprocessorDefinitions Condition="$(Configuration) == 'Debug'">%(PreprocessorDefinitions);ZLIB_DEBUG</PreprocessorDefinitions>
+      <EnableEnhancedInstructionSet Condition="$(Platform) == 'Win32' or $(Platform) == 'x64'">AdvancedVectorExtensions2</EnableEnhancedInstructionSet>


Unconditionally letting the compiler generate code up to AVX2 for all *.c files here might result in the binary not running on all CPUs?

The big changes here are: - Switching to zlib-ng on Windows (python/cpython#131438) - Using hmac for hashing functions (python/cpython#130157) --------- Co-authored-by: Geoffrey Thomas <geofft@ldpreload.com>

maikschulze · 2026-11-09T16:14:16Z

Hi @zooba, a heads-up on zlib-ng 2.2.4, in case it's useful to anyone: PGO builds made with Visual Studio 17.11 (MSVC 14.41) decompress deflate data wrongly on CPUs with AVX-512 (test_zlib: "incorrect data check", test_gzip: "CRC check failed"). It's the known zlib-ng bug zlib-ng/zlib-ng#1883 (MSVC before 17.13 miscompiles the AVX-512 GET_CHUNK_MAG()), worked around in zlib-ng 2.2.5 by zlib-ng/zlib-ng#1884. On a Ryzen 9 7950X, adding just that pragma to 2.2.4 fixes the build, and removing it brings the wrong output back.

It doesn't affect CI or the python.org builds, which use newer compilers, but it does affect building from source with an older Visual Studio, which is how I stumbled upon it. Is there any chance of updating the Windows build to zlib-ng 2.2.5? main, 3.15 and 3.14 still bundle 2.2.4.

zooba · 2026-11-09T16:52:44Z

@maikschulze Yes, that sounds like an update we'd make. Can you file a new issue?

maikschulze · 2026-11-09T17:25:12Z

Hi, thanks for the quick response! I will file an issue and along with a pull request tomorrow.

zooba · 2026-11-09T17:34:16Z

I think we keep the zlib-ng sources in a separate repo, so just file the issue without a PR as we'll need a core developer to update that. We don't accept PRs, as it's too hard to validate that the sources are unmodified (but if you've got a link to the right download page, that will be appreciated)

zooba added 4 commits March 19, 2025 00:56

pythongh-91349: Replace zlib with zlib-ng in Windows build

de56164

Updated readme.txt

1347c50

Fix includes for PGO build

7fd8f26

Use correct include dirs

f919749

zooba requested a review from gpshead March 19, 2025 01:11

zooba requested review from a team and sethmlarson as code owners March 19, 2025 01:11

bedevere-app Bot added the awaiting core review label Mar 19, 2025

bedevere-app Bot mentioned this pull request Mar 19, 2025

Use zlib-ng (fast!) rather than mainline stale zlib in binary releases #91349

Closed

Add NEWS

fb6500d

zooba commented Mar 19, 2025

View reviewed changes

zooba added 2 commits March 19, 2025 01:25

Fix debug reference and some warnings

89d44a1

Exclude Intel optimizations better on ARM64

08eecb1

zooba added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 19, 2025

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 19, 2025

rhpvorderman reviewed Mar 19, 2025

View reviewed changes

Comment thread PCbuild/pythoncore.vcxproj Resolved

Update docs and whatsnew

548daa7

sethmlarson approved these changes Mar 19, 2025

View reviewed changes

Remove unresolved reference from docs

f108044

gpshead approved these changes Mar 19, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Mar 19, 2025

Include ZLIBNG_VERSION in pythoninfo

085fa10

zooba enabled auto-merge (squash) March 19, 2025 18:38

zooba merged commit 63a638c into python:main Mar 19, 2025

bedevere-app Bot removed the awaiting merge label Mar 19, 2025

zooba deleted the gh-91349 branch March 19, 2025 19:25

chris-eibl reviewed Mar 20, 2025

View reviewed changes

chris-eibl mentioned this pull request Mar 20, 2025

clang-cl on Windows fails to compile after zlib-ng #131521

Closed

chris-eibl reviewed Mar 20, 2025

View reviewed changes

hugovk mentioned this pull request Mar 21, 2025

Consider distributing zlib-ng astral-sh/python-build-standalone#464

Open

chris-eibl mentioned this pull request Mar 22, 2025

GH-131521: fix clangcl build on Windows for zlib-ng #131526

Merged

zanieb mentioned this pull request Apr 9, 2025

Bump to 3.14.0a7 astral-sh/python-build-standalone#580

Merged

andersonhc mentioned this pull request Oct 15, 2025

Add Python 3.14 support py-pdf/fpdf2#1622

Merged

1 of 4 tasks

maikschulze mentioned this pull request Nov 10, 2026

Windows: update zlib-ng to 2.2.5 (PGO builds with Visual Studio 17.11 decompress wrongly on AVX-512 CPUs) #161735

Open

Conversation

zooba commented Mar 19, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zooba commented Mar 19, 2025

Uh oh!

bedevere-bot commented Mar 19, 2025

Uh oh!

Uh oh!

zooba commented Mar 19, 2025

Uh oh!

sethmlarson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdboom commented Mar 19, 2025

Uh oh!

mdboom commented Mar 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chris-eibl commented Mar 20, 2025

Uh oh!

zooba commented Mar 20, 2025

Uh oh!

chris-eibl commented Mar 20, 2025

Uh oh!

chris-eibl commented Mar 20, 2025

Uh oh!

zooba commented Mar 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maikschulze commented Nov 9, 2026

Uh oh!

zooba commented Nov 9, 2026

Uh oh!

maikschulze commented Nov 9, 2026

Uh oh!

zooba commented Nov 9, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

zooba commented Mar 19, 2025 •

edited by bedevere-app Bot

Loading