Skip to content

LZ4 decompression forward compatibility#4824

Merged
drcrallen merged 2 commits intoapache:masterfrom
metamx:lz4compression-forward-compatibility
Sep 19, 2017
Merged

LZ4 decompression forward compatibility#4824
drcrallen merged 2 commits intoapache:masterfrom
metamx:lz4compression-forward-compatibility

Conversation

@leventov
Copy link
Copy Markdown
Member

This PR adds some forward-compatibility with #4762, not always expecting compressed buffers of the same size. There is no noticeable performance difference:

After
LongCompressionBenchmark.readContinuous  longCompress/   enumerate     longs         lz4  avgt   25  19.520 ± 0.359  ms/op
LongCompressionBenchmark.readContinuous  longCompress/     zipfLow     longs         lz4  avgt   25  35.228 ± 0.709  ms/op
LongCompressionBenchmark.readContinuous  longCompress/    zipfHigh     longs         lz4  avgt   25  22.461 ± 0.316  ms/op
LongCompressionBenchmark.readContinuous  longCompress/  sequential     longs         lz4  avgt   25  60.459 ± 0.938  ms/op
LongCompressionBenchmark.readContinuous  longCompress/     uniform     longs         lz4  avgt   25  33.912 ± 0.465  ms/op
LongCompressionBenchmark.readSkipping    longCompress/   enumerate     longs         lz4  avgt   25   6.340 ± 0.089  ms/op
LongCompressionBenchmark.readSkipping    longCompress/     zipfLow     longs         lz4  avgt   25  22.306 ± 0.423  ms/op
LongCompressionBenchmark.readSkipping    longCompress/    zipfHigh     longs         lz4  avgt   25   9.470 ± 0.126  ms/op
LongCompressionBenchmark.readSkipping    longCompress/  sequential     longs         lz4  avgt   25  47.260 ± 0.726  ms/op
LongCompressionBenchmark.readSkipping    longCompress/     uniform     longs         lz4  avgt   25  20.669 ± 0.366  ms/op

Before
LongCompressionBenchmark.readContinuous  longCompress/   enumerate     longs         lz4  avgt   25  19.553 ± 0.264  ms/op
LongCompressionBenchmark.readContinuous  longCompress/     zipfLow     longs         lz4  avgt   25  34.843 ± 0.463  ms/op
LongCompressionBenchmark.readContinuous  longCompress/    zipfHigh     longs         lz4  avgt   25  22.523 ± 0.293  ms/op
LongCompressionBenchmark.readContinuous  longCompress/  sequential     longs         lz4  avgt   25  59.861 ± 0.673  ms/op
LongCompressionBenchmark.readContinuous  longCompress/     uniform     longs         lz4  avgt   25  34.256 ± 0.548  ms/op
LongCompressionBenchmark.readSkipping    longCompress/   enumerate     longs         lz4  avgt   25   6.414 ± 0.092  ms/op
LongCompressionBenchmark.readSkipping    longCompress/     zipfLow     longs         lz4  avgt   25  21.927 ± 0.518  ms/op
LongCompressionBenchmark.readSkipping    longCompress/    zipfHigh     longs         lz4  avgt   25   9.588 ± 0.142  ms/op
LongCompressionBenchmark.readSkipping    longCompress/  sequential     longs         lz4  avgt   25  46.621 ± 0.667  ms/op
LongCompressionBenchmark.readSkipping    longCompress/     uniform     longs         lz4  avgt   25  20.716 ± 0.305  ms/op

@himanshug
Copy link
Copy Markdown
Contributor

👍

@drcrallen drcrallen merged commit 24646ac into apache:master Sep 19, 2017
@drcrallen drcrallen deleted the lz4compression-forward-compatibility branch September 19, 2017 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants