[AutoTVM] Introducing multi_filter into ConfigSpace autotvm

[Icemist]

multi_filter can restrict combination of parameters in difference to the knob filter, that restricts only single parameter.

Here is a small example to show the workflow and the difference:

Pre-requisites

candidates = [[16, 64], [32, 32], [64, 16]]
filter = lambda v: v.size[0] != 16
multi_filter = lambda e: (e["tile_x"].size[0] + e["tile_y"].size[0]) <= 64

Case 1 - without filtering

cfg.define_split("tile_x", x, num_outputs=2, policy="candidate", candidate=candidates)
cfg.define_split("tile_y", y, num_outputs=2, policy="candidate", candidate=candidates)

[('tile_x', [16, 64]), ('tile_y', [16, 64])],None,0
[('tile_x', [32, 32]), ('tile_y', [16, 64])],None,1
[('tile_x', [64, 16]), ('tile_y', [16, 64])],None,2
[('tile_x', [16, 64]), ('tile_y', [32, 32])],None,3
[('tile_x', [32, 32]), ('tile_y', [32, 32])],None,4
[('tile_x', [64, 16]), ('tile_y', [32, 32])],None,5
[('tile_x', [16, 64]), ('tile_y', [64, 16])],None,6
[('tile_x', [32, 32]), ('tile_y', [64, 16])],None,7
[('tile_x', [64, 16]), ('tile_y', [64, 16])],None,8

Case 2 - with filter

cfg.define_split("tile_x", x, num_outputs=2, policy="candidate", candidate=candidates, filter=filter)
cfg.define_split("tile_y", y, num_outputs=2, policy="candidate", candidate=candidates, filter=filter)

[('tile_x', [32, 32]), ('tile_y', [32, 32])],None,0
[('tile_x', [64, 16]), ('tile_y', [32, 32])],None,1
[('tile_x', [32, 32]), ('tile_y', [64, 16])],None,2
[('tile_x', [64, 16]), ('tile_y', [64, 16])],None,3

Case 3 - with filter and multi_filter

cfg.define_split("tile_x", x, num_outputs=2, policy="candidate", candidate=candidates, filter=filter)
cfg.define_split("tile_y", y, num_outputs=2, policy="candidate", candidate=candidates, filter=filter)
cfg.multi_filter(filter=multi_filter)

[('tile_x', [32, 32]), ('tile_y', [32, 32])],None,0

Co-authored-by: Andrey Malyshev elvin.nnov@gmail.com
Co-authored-by: Egor Churaev egor.churaev@gmail.com

[Icemist]

CC @merrymercy @eqy

[TejashShah]

cc @csullivan

[TejashShah]

cc @vinx13 @junrushao @masahi

[echuraev]

Quickly looked into this PR. I don't really like that now we need to call tsk.config_space.filtered_length instead of len(tsk.config_space). I think that len(smth) is more intuitive. But I didn't dive deeply into the details. I'll do it later. I think that @elvin-n can provide more useful and detailed review.

[elvin-n]

@masahi @merrymercy could you please take a look?

[tmoreau89]

CC @tqchen @merrymercy this PR makes some in depth changes to autoTVM, PTAL

[elvin-n]

can we merge this PR into main?