Rewrite gzip component to support trailing bytes without external tool and compress with PIGZ. Addresses #476#485
Conversation
…l (using zlib library instead of gzip)
…stead of temp file
Co-authored-by: Jacob Strieb <99368685+rbs-jacob@users.noreply.github.com>
rbs-jacob
left a comment
There was a problem hiding this comment.
This needs a changelog entry, but after that, should probably be good to go.
Last change was still in unreleased section
|
Based on benchmarks of zlib module vs pigz, no benefit to using pigz for decompression, but significant benefit to using it for compression. https://gist.github.com/alchzh/24ca62a6f87548ea2b1911c174e0ed54 |
whyitfor
left a comment
There was a problem hiding this comment.
Looks good to me.
Especially like the profiling and data-driven approach.
rbs-jacob
left a comment
There was a problem hiding this comment.
I did not review the tests in great detail, but I like the way that this implementation ended up. Seems like a big improvement!
rbs-jacob
left a comment
There was a problem hiding this comment.
I relented and included one minor change that I would like to see before merging this.
rbs-jacob
left a comment
There was a problem hiding this comment.
Thanks for this fix. Looks ready to merge once the tests pass!
Rewrite gzip component to support trailing bytes without external tool.
Link to Related Issue(s)
#476
python/cpython#68489
Please describe the changes in your request.
gziplibrary with calls to the underlyingzliblibrary directly, in a way that doesn't fail with trailing garbage at the end of files.