Skip to content

Most fuzz tests should not use LLVM #9044

@abadams

Description

@abadams

Any fuzz test that uses JIT compilation implicitly also fuzz tests all of LLVM. We do not have the capacity to handle a steady stream of fuzzer-found bugs in LLVM. On the other hand, JIT compiling exprs lets us know if we're using LLVM incorrectly, so they're nice to have. It would just be good to isolate that into its own test. I propose we restructure things like so:

  • (Add a tiny fuzz test framework #9050) A small interface to switch between libFuzzer or a simple RNG based one.
  • (Shared fuzz expression generator. #9048) A shared file that includes a random expr generator
  • (ExprInterpreter. #9047) and an evaluator/interpreter of (vector) exprs
  • A test that generates random exprs and checks JIT-compiling and running them produces the same results as the interpreter
  • A test that checks that interpret(simplify(random_expr)) == interpret(random_expr)
  • A test that checks that interpret(extract_lanes(random_expr)) == extract_lanes(interpret(random_expr))
  • ... other tests that do something to an Expr and want to know if it's correct (e.g. lossless_cast, bounds)

The latter tests would then not break due to llvm bugs, but we would still find out about the llvm bugs.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions