Commit 701ca28
authored
flate: reduce stateless allocations (#1106)
After updating GO to v1.24+, a sharp increase in CPU utilization was detected. Heap profile helped to reveal increased memory allocations by Write and Close methods of stateless gzip.Writer mode. This PR optimizes problem area by using sync.Pool and later allocation of tokens object.
Benchmarks:
BEFORE
```
BenchmarkEncodeDigitsSL1e4-12 10141 115946 ns/op 86.25 MB/s 542379 B/op 3 allocs/op
BenchmarkEncodeDigitsSL1e5-12 1602 730674 ns/op 136.86 MB/s 541377 B/op 2 allocs/op
BenchmarkEncodeDigitsSL1e6-12 175 6851506 ns/op 145.95 MB/s 541542 B/op 2 allocs/op
BenchmarkEncodeTwainSL1e4-12 9708 131564 ns/op 76.01 MB/s 542146 B/op 3 allocs/op
BenchmarkEncodeTwainSL1e5-12 1663 684854 ns/op 146.02 MB/s 541463 B/op 2 allocs/op
BenchmarkEncodeTwainSL1e6-12 177 6435648 ns/op 155.38 MB/s 541654 B/op 2 allocs/op
```
AFTER
```
BenchmarkEncodeDigitsSL1e4-12 34747 33800 ns/op 295.86 MB/s 8 B/op 0 allocs/op
BenchmarkEncodeDigitsSL1e5-12 1771 640723 ns/op 156.07 MB/s 160 B/op 0 allocs/op
BenchmarkEncodeDigitsSL1e6-12 181 6759226 ns/op 147.95 MB/s 1573 B/op 0 allocs/op
BenchmarkEncodeTwainSL1e4-12 35294 35304 ns/op 283.26 MB/s 8 B/op 0 allocs/op
BenchmarkEncodeTwainSL1e5-12 1939 585755 ns/op 170.72 MB/s 146 B/op 0 allocs/op
BenchmarkEncodeTwainSL1e6-12 181 6505389 ns/op 153.72 MB/s 1573 B/op 0 allocs/op
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Optimized compression internals to reuse buffers via pooling, improving throughput and reducing memory use during repeated operations.
- Enhances performance and consistency for both dictionary and non-dictionary compression paths across large blocks.
- No changes to public APIs or user-facing behavior; workflows remain the same.
- Users may see faster compression and lower memory footprint under sustained/high-volume workloads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 parent e0b47ff commit 701ca28
1 file changed
+16
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
64 | 71 | | |
65 | 72 | | |
66 | 73 | | |
67 | 74 | | |
68 | 75 | | |
69 | 76 | | |
70 | | - | |
71 | 77 | | |
72 | 78 | | |
73 | 79 | | |
| |||
91 | 97 | | |
92 | 98 | | |
93 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
94 | 106 | | |
95 | 107 | | |
96 | 108 | | |
| |||
113 | 125 | | |
114 | 126 | | |
115 | 127 | | |
116 | | - | |
| 128 | + | |
117 | 129 | | |
118 | | - | |
| 130 | + | |
119 | 131 | | |
120 | 132 | | |
121 | 133 | | |
| |||
129 | 141 | | |
130 | 142 | | |
131 | 143 | | |
132 | | - | |
| 144 | + | |
133 | 145 | | |
134 | 146 | | |
135 | 147 | | |
| |||
0 commit comments