Add Lzma library files to the repository by rzikm · Pull Request #124003 · dotnet/runtime

rzikm · 2026-02-04T13:22:41Z

Preparatory PR for #1542

libSystem.IO.Compression.Native.so size (Release)

	Size	Diff
Before	1 747 040 B	-
After	1 810 704 B	+3.64%

Copilot

Pull request overview

This PR adds the XZ Utils library (version 5.8.2) to the dotnet/runtime repository as preparatory work for issue #1542. The addition includes the complete LZMA compression library source code, build configurations, documentation, and integration scripts.

Changes:

Adds XZ Utils 5.8.2 library files from the tukaani-project/xz repository
Includes CMake and Autotools build system configurations
Adds documentation, examples, and internationalization support files
Configures the library for static linking with multithreading disabled

Reviewed changes

Copilot reviewed 75 out of 497 changed files in this pull request and generated no comments.

File	Description
src/native/external/xz/*	Complete XZ Utils 5.8.2 library source code including build scripts, documentation, examples, and localization files
src/native/external/xz.cmake	CMake configuration to integrate XZ Utils into dotnet/runtime build system with specific feature flags
src/native/external/xz.version	Version tracking file indicating XZ Utils v5.8.2 from tukaani-project
src/native/external/cgmanifest.json	Component governance manifest entry for XZ Utils dependency

dotnet-policy-service · 2026-02-04T13:25:43Z

Tagging subscribers to this area: @karelz, @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

src/native/libs/System.IO.Compression.Native/CMakeLists.txt

Copilot

Pull request overview

Copilot reviewed 79 out of 284 changed files in this pull request and generated 7 comments.

src/native/external/xz/src/common/tuklib_open_stdxxx.c

src/native/external/xz/src/common/tuklib_open_stdxxx.h

src/native/external/xz/cmake/tuklib_physmem.cmake

src/native/external/xz/cmake/tuklib_cpucores.cmake

src/native/external/xz.version

src/native/external/xz/src/common/tuklib_gettext.h

src/native/external/xz.cmake

rzikm · 2026-02-09T12:21:55Z

I am getting compile errors on linux:

  ERROR: /source/dotnet/runtime/src/native/libs/System.IO.Compression.Native/entrypoints.c file did not match entries exported from /source/dotnet/runtime/artifacts/obj/native/net11.0-linux-Debug-x64/System.IO.Compression.Native/libSystem.IO.Compression.Native.so
  DIFFERENCES FOUND: 
  lzma_stream_decoder

Seems like there is something wrong with the exports specification:

System.IO.Compression.Native$ nm libSystem.IO.Compression.Native.so | grep " lzma_stream_decoder"
00000000003c51c0 t lzma_stream_decoder
00000000003c4410 t lzma_stream_decoder_init

All the other exported entries have T in the nm output, e.g.

000000000019dd90 T BrotliDecoderDecompress

But the exports file seems to be generated correctly:

        ZSTD_CCtx_setPledgedSrcSize;
        ZDICT_trainFromBuffer;
        lzma_stream_decoder;
    local: *;
};

I am not sure what is going on here, maybe there is a problem because the library functions are declared as extern?

runtime/src/native/external/xz/src/liblzma/api/lzma/container.h

Lines 838 to 840 in d8731e0

    
           extern LZMA_API(lzma_ret) lzma_alone_decoder( 
        
           		lzma_stream *strm, uint64_t memlimit) 
        
           		lzma_nothrow lzma_attr_warn_unused_result;

I will need to investigate a bit more

Copilot

Pull request overview

Copilot reviewed 79 out of 284 changed files in this pull request and generated 2 comments.

src/native/external/xz/src/common/tuklib_open_stdxxx.h

src/native/external/xz.version

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

rzikm · 2026-02-10T17:38:54Z

I will need to investigate a bit more

Should be working now

Copilot

Pull request overview

Copilot reviewed 79 out of 284 changed files in this pull request and generated 3 comments.

src/native/external/xz.cmake

src/native/external/xz/src/common/w32_application.manifest.comments.txt

src/native/external/xz/src/common/tuklib_gettext.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 79 out of 284 changed files in this pull request and generated no new comments.

…rsions

Copilot

Pull request overview

Copilot reviewed 82 out of 288 changed files in this pull request and generated 2 comments.

src/native/external/xz/cmake/tuklib_common.cmake

src/native/external/xz/cmake/tuklib_cpucores.cmake

rzikm · 2026-02-18T08:21:53Z

Build is passing, but no new functionality was introduced yet, so the build failures are not related

Copilot

Pull request overview

Copilot reviewed 81 out of 288 changed files in this pull request and generated 5 comments.

src/native/external/xz/src/common/tuklib_open_stdxxx.c

src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

src/native/external/xz/cmake/tuklib_cpucores.cmake

src/native/external/xz/src/common/tuklib_mbstr_wrap.h

…e.Unix.targets Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 81 out of 288 changed files in this pull request and generated 1 comment.

src/native/external/xz/cmake/tuklib_common.cmake

ericstj · 2026-02-19T15:56:34Z

src/native/external/xz/COPYING

@@ -0,0 +1,70 @@
+
+XZ Utils Licensing
+==================


This is a non-traditional license. We should make sure to review this with our OSS team and understand what's required to use this. I didn't see an update to our repo's third-party-notices.

I see we are subsetting what we copy - does that mean we've omitted all the things under GPL? If so, can those licenses be omitted? IANAL but this makes me a little nervous and is the sort of thing we want to make doubly sure we have correct so that we understand that our usage / copying is permitted by the license and we are meeting all the requirements set out by that license.

I removed all the code that we don't use, the parts that are left (liblzma, common) are under 0BSD license. I also added third party notice

Copilot

Pull request overview

Copilot reviewed 79 out of 230 changed files in this pull request and generated no new comments.

ericstj · 2026-02-19T19:14:38Z

What sort of size increase do we see to our system.io.compression.native with this, across different platforms? Have you confirmed we are linking correctly and getting the minimal set of things we need?

rzikm · 2026-02-20T09:11:58Z

What sort of size increase do we see to our system.io.compression.native with this, across different platforms? Have you confirmed we are linking correctly and getting the minimal set of things we need?

the PR as-is increases the native .so by about 60 kB in release on my desktop, there is a reference to lzma_stream_decoder_init in the entryponts.c to validate if it links properly. I don't think we can whittle down any more of the compiled source without manual modifications to the build scripts.

jkotas · 2026-02-20T16:36:56Z

src/native/external/xz/src/scripts/xzgrep.in

@@ -0,0 +1,302 @@
+#!@POSIX_SHELL@
+# SPDX-License-Identifier: GPL-2.0-or-later


Do we really need these GPL licensed sources?

jkotas · 2026-02-20T16:51:43Z

the PR as-is increases the native .so by about 60 kB in release on my desktop, there is a reference to lzma_stream_decoder_init in the entryponts.c to validate if it links properly

Does lzma_stream_decoder_init bring the full compression / decompression implementation into the binary?

FWIW, I do not see a problem with the extra code that this brings. We add several megabytes of code to netcoreapp every release. The problem I see is that the overall compression design is not trimming friendly. The moment you enable compression for your HttpClient, you get all compression algorithms in our portfolio in your (AOT compiled) binary - discussed at #123531 (comment) .

rzikm added 2 commits January 23, 2026 13:31

Include xz library in the repo

d8731e0

Link lzma to System.IO.Compression.Native

2b5700f

Copilot AI review requested due to automatic review settings February 4, 2026 13:22

rzikm added the area-System.IO.Compression label Feb 4, 2026

Copilot AI reviewed Feb 4, 2026

View reviewed changes

rzikm marked this pull request as draft February 4, 2026 13:23

dotnet-policy-service bot assigned rzikm Feb 4, 2026

build-analysis bot mentioned this pull request Feb 4, 2026

[android][arm64] System.Net.Sockets.Tests.SendTo_SyncForceNonBlocking.Datagram_UDP_ShouldImplicitlyBindLocalEndpoint fails with NetworkUnreachable #120526

Open

iremyux reviewed Feb 6, 2026

View reviewed changes

src/native/libs/System.IO.Compression.Native/CMakeLists.txt Outdated Show resolved Hide resolved

rzikm added 4 commits February 9, 2026 10:42

Remove unnecessary xz repo files

6b2729e

Don't copy unnecessary files during install

9e22f20

Attempt to fix build on linux

b577d95

Fix newlines at the end of file

829ad77

Copilot AI review requested due to automatic review settings February 9, 2026 12:14

Copilot AI reviewed Feb 9, 2026

View reviewed changes

rzikm added 2 commits February 10, 2026 18:06

Fix build on Linux

daf697d

Remove libgnu workaround

4a58132

Copilot AI review requested due to automatic review settings February 10, 2026 17:24

Copilot AI reviewed Feb 10, 2026

View reviewed changes

src/native/external/xz/src/common/tuklib_open_stdxxx.h Show resolved Hide resolved

src/native/external/xz.version Outdated Show resolved Hide resolved

Update src/native/external/xz.version

29847da

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 10, 2026 17:38

Copilot AI reviewed Feb 10, 2026

View reviewed changes

src/native/external/xz.cmake Outdated Show resolved Hide resolved

src/native/external/xz/src/common/w32_application.manifest.comments.txt Show resolved Hide resolved

src/native/external/xz/src/common/tuklib_gettext.h Show resolved Hide resolved

Update src/native/external/xz.cmake

5abc214

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 10, 2026 17:49

Copilot AI reviewed Feb 10, 2026

View reviewed changes

This was referenced Feb 10, 2026

"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886

Open

Test failure: baseservices/exceptions/stackoverflow/stackoverflowtester/stackoverflowtester.cmd #110173

Open

rzikm added 2 commits February 12, 2026 14:33

Fix C4244 warnings in liblzma: add explicit casts for data loss conve…

54d96e2

…rsions

Add patch file

340d547

Copilot AI review requested due to automatic review settings February 12, 2026 14:58

Copilot AI reviewed Feb 12, 2026

View reviewed changes

src/native/external/xz/cmake/tuklib_common.cmake Show resolved Hide resolved

src/native/external/xz/cmake/tuklib_cpucores.cmake Show resolved Hide resolved

build-analysis bot mentioned this pull request Feb 12, 2026

/root/helix/work/correlation/scripts/<hash>/execute.sh: Permission denied dotnet/dnceng#3412

Open

3 tasks

More x86 fixes

ebbf71a

rzikm marked this pull request as ready for review February 18, 2026 08:20

rzikm requested a review from MichalStrehovsky as a code owner February 18, 2026 08:20

Copilot AI review requested due to automatic review settings February 18, 2026 08:20

Copilot AI reviewed Feb 18, 2026

View reviewed changes

Update src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Nativ…

4107e59

…e.Unix.targets Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 18, 2026 08:23

Copilot AI reviewed Feb 18, 2026

View reviewed changes

src/native/external/xz/cmake/tuklib_common.cmake Show resolved Hide resolved

Minor fixes

2ab542a

This was referenced Feb 18, 2026

Mono interpreter Segmentation fault crashes #124525

Closed

[android][clr] No peer certificates when executing System.Net.Http.Functional.Tests on Android emulator #124526

Open

ericstj reviewed Feb 19, 2026

View reviewed changes

remove more unnecessary code, add copyright notice

85ba08b

Copilot AI review requested due to automatic review settings February 19, 2026 18:16

Copilot AI reviewed Feb 19, 2026

View reviewed changes

build-analysis bot mentioned this pull request Feb 19, 2026

[mono] The type initializer for System.Drawing.DrawingCom threw an exception: Operation is not supported on this platform #110150

Open

Restore scripts folder

8aa31aa

jkotas reviewed Feb 20, 2026

View reviewed changes

		@@ -0,0 +1,302 @@
		#!@POSIX_SHELL@
		# SPDX-License-Identifier: GPL-2.0-or-later

Conversation

rzikm commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

dotnet-policy-service bot commented Feb 4, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rzikm commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

rzikm commented Feb 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

rzikm commented Feb 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

ericstj Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

rzikm Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

ericstj commented Feb 19, 2026

Uh oh!

rzikm commented Feb 4, 2026 •

edited

Loading

rzikm commented Feb 9, 2026 •

edited

Loading

rzikm Feb 19, 2026 •

edited

Loading

jkotas Feb 20, 2026 •

edited

Loading