Skip to content

Replace stdlib gzip/flate with klauspost/compress in compress filter#3933

Closed
ivan-digital wants to merge 1 commit intozalando:masterfrom
ivan-digital:optimize/klauspost-compress
Closed

Replace stdlib gzip/flate with klauspost/compress in compress filter#3933
ivan-digital wants to merge 1 commit intozalando:masterfrom
ivan-digital:optimize/klauspost-compress

Conversation

@ivan-digital
Copy link
Copy Markdown
Contributor

Summary

Replace compress/gzip and compress/flate with github.com/klauspost/compress in the compress filter. Drop-in API-compatible replacement with significantly better CPU performance.

Scoped to the compress filter only — decompress, routesrv, and OIDC cookie compression can follow up separately.

Motivation

Profiling shows 30% CPU spent in compress/flate on high-traffic clusters (#3932).

Benchmark results (Apple M2 Max, 6 samples)

Default level (BestSpeed — production path)

                     │  stdlib   │ klauspost │  change  │
CompressGzip4  10KB    16.12µs    9.38µs      -42%
CompressGzip6  1MB     985.9µs   401.1µs      -59%
CompressGzip8  100MB   88.76ms   24.56ms      -72%

JSON payload (compressible, realistic)

                     │  stdlib   │ klauspost │  change  │
GzipJSON6      1MB    113.9µs    72.4µs      -36%
GzipJSON8      100MB  10.68ms    6.77ms      -37%

Deflate-only

                     │  stdlib   │ klauspost │  change  │
Deflate4       10KB   16.50µs    9.06µs      -45%
Deflate6       1MB    1041.2µs   366.0µs     -65%

Higher compression level (level 6)

                     │  stdlib   │ klauspost │  change  │
GzipLevel6_6   1MB   2293.3µs   464.0µs     -80%

Memory allocation nearly identical. Brotli unchanged (separate library).

Closes #3932

Test plan

  • All existing compress filter tests pass
  • Benchmarks across random data, JSON, deflate, and compression levels

Closes zalando#3932

Signed-off-by: ivan-digital <root@ivan.digital>
@szuecs szuecs mentioned this pull request Mar 26, 2026
szuecs added a commit that referenced this pull request Mar 26, 2026
Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
szuecs added a commit that referenced this pull request Mar 26, 2026
see commit message

ref: #3933
fix: #3932

---------

Signed-off-by: Sandor Szücs <sandor.szuecs@zalando.de>
@szuecs
Copy link
Copy Markdown
Member

szuecs commented Mar 26, 2026

thanks for the PR, I took the test to increase coverage on my PR.

@szuecs szuecs closed this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

optimize compression

2 participants