Skip to content

[opt] Fix oss-fuzz bug in optimal parser#2980

Merged
terrelln merged 1 commit intofacebook:devfrom
terrelln:opt-oss-fuzz-fix
Jan 7, 2022
Merged

[opt] Fix oss-fuzz bug in optimal parser#2980
terrelln merged 1 commit intofacebook:devfrom
terrelln:opt-oss-fuzz-fix

Conversation

@terrelln
Copy link
Contributor

@terrelln terrelln commented Jan 7, 2022

oss-fuzz uncovered a scenario where we're evaluating the cost of litLength = 131072,
which can't be represented in the zstd format, so we accessed 1 beyond LL_bits.

Fix the issue by making it cost 1 bit more than litLength = 131071.

There are still follow ups:

  1. This happened because literals_cost[0] = 0, so the optimal parser chose 36 literals
    over a match. Should we bound literals_cost[literal] > 0, unless the block truly only
    has one literal value?
  2. When no matches are found, the cost model isn't updated. In this case no matches were
    found for an entire block. So the literals cost model wasn't updated at all. That made
    the optimal parser think literals_cost[0] = 0, where it is actually quite high, since
    the block was entirely random noise.

Credit to OSS-Fuzz.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants