Implement ZStandard Stream, Encoder, Decoder

[rzikm]

Implements #59591

There are some patches applied to vendored zstd sources which are listed in src/native/zstd-version.txt, neither of them should be necessary once we update to next release (maintainers didn't commit to a release date, but hopefully they will release something early next year).

Benchmarks results (taken from dotnet/performance repo) look great so far

Benchmark results

Method	level	file	Algorithm	Mean	Error	StdDev	Size	Compressed
Compress	Optimal	alice29.txt	Brotli	2,306.40 μs	45.807 μs	71.316 μs	148481	54517
Compress	Optimal	alice29.txt	Zstandard	842.743 μs	8.3719 μs	7.8310 μs	148481	54919
Compress	Optimal	alice29.txt	Deflate	2,421.16 μs	41.274 μs	38.608 μs	148481	53890
Decompress	Optimal	alice29.txt	Brotli	320.82 μs	4.802 μs	4.257 μs
Decompress	Optimal	alice29.txt	Zstandard	139.251 μs	2.0885 μs	1.7440 μs
Decompress	Optimal	alice29.txt	Deflate	275.43 μs	4.849 μs	4.535 μs
Compress	Optimal	sum	Brotli	577.62 μs	7.647 μs	5.971 μs	38240	12527
Compress	Optimal	sum	Zstandard	155.050 μs	3.0367 μs	3.3753 μs	38240	13389
Compress	Optimal	sum	Deflate	404.70 μs	7.732 μs	9.779 μs	38240	13388
Decompress	Optimal	sum	Brotli	101.26 μs	2.781 μs	7.566 μs
Decompress	Optimal	sum	Zstandard	31.333 μs	0.6172 μs	0.5774 μs
Decompress	Optimal	sum	Deflate	54.86 μs	0.676 μs	0.632 μs
Compress	Optimal	TestDocument.pdf	Brotli	763.43 μs	13.851 μs	23.520 μs	121993	115863
Compress	Optimal	TestDocument.pdf	Zstandard	182.115 μs	3.3456 μs	2.9658 μs	121993	119285
Compress	Optimal	TestDocument.pdf	Deflate	2,133.16 μs	42.543 μs	104.358 μs	121993	115369
Decompress	Optimal	TestDocument.pdf	Brotli	267.64 μs	5.281 μs	10.047 μs
Decompress	Optimal	TestDocument.pdf	Zstandard	17.508 μs	0.3336 μs	0.3708 μs
Decompress	Optimal	TestDocument.pdf	Deflate	224.92 μs	2.642 μs	2.342 μs
Compress	Fastest	alice29.txt	Brotli	732.30 μs	14.288 μs	19.074 μs	148481	59265
Compress	Fastest	alice29.txt	Zstandard	509.3 us	5.10 us	4.53 us	148481	59891
Compress	Fastest	alice29.txt	Deflate	612.28 μs	11.300 μs	16.564 μs	148481	83053
Decompress	Fastest	alice29.txt	Brotli	380.92 μs	7.570 μs	9.844 μs
Decompress	Fastest	alice29.txt	Zstandard	111.14 us	1.321 us	1.103 us
Decompress	Fastest	alice29.txt	Deflate	361.78 μs	6.700 μs	14.847 μs
Compress	Fastest	sum	Brotli	125.26 μs	2.489 μs	3.801 μs	38240	14305
Compress	Fastest	sum	Zstandard	108.0 us	1.05 us	0.88 us	38240	13848
Compress	Fastest	sum	Deflate	121.81 μs	2.409 μs	4.699 μs	38240	17222
Decompress	Fastest	sum	Brotli	99.19 μs	1.974 μs	3.243 μs
Decompress	Fastest	sum	Zstandard	29.01 us	0.559 us	0.599 us
Decompress	Fastest	sum	Deflate	58.68 μs	0.999 μs	0.934 μs
Compress	Fastest	TestDocument.pdf	Brotli	300.40 μs	5.753 μs	6.394 μs	121993	117104
Compress	Fastest	TestDocument.pdf	Zstandard	137.3 us	2.00 us	1.67 us	121993	119405
Compress	Fastest	TestDocument.pdf	Deflate	947.61 μs	17.861 μs	18.342 μs	121993	121404
Decompress	Fastest	TestDocument.pdf	Brotli	233.14 μs	3.625 μs	3.391 μs
Decompress	Fastest	TestDocument.pdf	Zstandard	17.22 us	0.260 us	0.244 us
Decompress	Fastest	TestDocument.pdf	Deflate	238.81 μs	2.153 μs	1.908 μs

[Copilot]

Pull Request Overview

This PR implements ZStandard stream, encoder, and decoder functionality for .NET by integrating the zstd native library. The changes primarily consist of adding the external zstd library source files and build infrastructure to support compression and decompression capabilities.

Key Changes:

• Integration of zstd native library source code (version metadata suggests 0.9.0)
• Build system configuration files for Meson and CMake
• Single-file library generation scripts and examples
• Test infrastructure for the zstd implementation

Reviewed Changes

Copilot reviewed 142 out of 237 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/native/external/zstd/lib/common/fse_decompress.c	FSE (Finite State Entropy) decompression implementation
src/native/external/zstd/lib/common/fse.h	FSE codec public API and data structures
src/native/external/zstd/lib/common/error_private.h	Internal error handling macros and definitions
src/native/external/zstd/lib/common/error_private.c	Error code to string mapping implementation
src/native/external/zstd/lib/common/entropy_common.c	Common entropy encoding/decoding functions
src/native/external/zstd/lib/common/debug.h	Debug logging and assertion macros
src/native/external/zstd/lib/common/debug.c	Debug level global variable definition
src/native/external/zstd/lib/common/cpu.h	CPU feature detection (BMI2, AVX, etc.)
src/native/external/zstd/lib/common/compiler.h	Compiler-specific macros and attributes
src/native/external/zstd/lib/common/bitstream.h	Bitstream encoding/decoding utilities
src/native/external/zstd/lib/common/bits.h	Bit manipulation helper functions
src/native/external/zstd/lib/common/allocations.h	Custom memory allocation wrappers
src/native/external/zstd/lib/README.md	Documentation for library structure and build options
src/native/external/zstd/lib/Makefile	Build configuration for the zstd library
src/native/external/zstd/lib/BUCK	Buck build system configuration
src/native/external/zstd/lib/.gitignore	Git ignore rules for build artifacts
src/native/external/zstd/build/single_file_libs/*	Scripts and examples for single-file library generation
src/native/external/zstd/build/meson/*	Meson build system configuration files
src/native/external/zstd/build/cmake/tests/.gitignore	Git ignore rules for CMake test artifacts

[rzikm]

I think this PR is ready for the first round of reviews

[rzikm]

Well, retargeting required a bit of a dance, but hopefully everything went well, this PR now contains only the work that needs to be reviewed (minus a couple of zstd patches taken from their upstream to get around build errors)