[Benchmarking] Decide which benchmarks are part of regression checks

[asfimport]

Reporter: Francois Saint-Jacques / @fsaintjacques

Related issues:

• [Benchmarking] Performance Benchmarking Dashboard (blocks)

_{Note: This issue was originally created as ARROW-5175. Please see the migration documentation for further details.}