[AutoScheduler] Enable schedule sharing in dispatch context

[comaniac]

This is the follow up PR for #7317 to enable the schedule sharing in the auto_scheduler diaptch context.

cc @merrymercy @jcf94

[masahi]

@comaniac Does this mean tuning support for dynamic workload (dynamic batch size etc) is coming soon? I'm very excited for this, that would tremendously help my MaskRCNN!!

[comaniac]

@comaniac Does this mean tuning support for dynamic workload (dynamic batch size etc) is coming soon? I'm very excited for this, that would tremendously help my MaskRCNN!!

Ah this is not the perfect solution for dynamic shape. This is more like a solution to make tuned logs more useful. For example, you can apply the tuning log with batch 1 to all batch sizes. You can even tune several batch sizes in prime numbers to achieve better performance to their multiples. Meanwhile, we do work on the dynamic shape support in auto_scheduler, but it may not be ready to be upstreamed before this summer or fall.

[jcf94]

@comaniac Does this mean tuning support for dynamic workload (dynamic batch size etc) is coming soon? I'm very excited for this, that would tremendously help my MaskRCNN!!

Ah this is not the perfect solution for dynamic shape. This is more like a solution to make tuned logs more useful. For example, you can apply the tuning log with batch 1 to all batch sizes. You can even tune several batch sizes in prime numbers to achieve better performance to their multiples. Meanwhile, we do work on the dynamic shape support in auto_scheduler, but it may not be ready to be upstreamed before this summer or fall.

😃 Looking forward to the dynamic shape support, too! It will be great useful.

[masahi]

Yes, I believe dynamic tuning and codegen is one of the biggest challenges of TVM this year. I'm glad at least there are folks looking at the problem.

MaskRCNN should serve as a good benchmark, it has both dynamic dense (very large) and dynamic conv2d + conv2d transpose. All of them are current bottleneck, without tuning them I cannot beat pytorch.

[comaniac]

Thanks @merrymercy