[AutoTVM] AutoTVM incorrect measurement

[FrozenGene]

Our AutoTVM use debug_skip_region to skip our precompute stage (like precompute kernel). However, debug_skip_region can not reflect practical running, which just only create empty tensor for it. Empty tensor will make AutoTVM record the performance incorrect. This condition will become worse like Winograd. In Winograd, our weight shape will be changed from (KH, KW, CO, CI) to (alpha, alpha, CO, CI). Alpha equals to tile_size + KH - 1. If tile_size is 4 (KH is 3), then alpha is 6. So to avoid this issue, we should avoid using debug_skip_region in all platforms of AutoTVM template and use the technique like this PR: #5200.

Open this issue to track this.

• Intel CPU Fix intel conv2d auto tune #5200
• ARM CPU
• Bifrost
• CUDA
• Intel Graphics
• Mali [AutoTVM][TOPI] AutoTVM incorrect measurement #5511

[tqchen]

@FrozenGene what is the status of this issue?

[FrozenGene]

@FrozenGene what is the status of this issue?

This issue wants to track all template of using debug_skip_region(like arm / cuda and so on). Currently, we only fix intel conv2d. I think I should list the target we should fix.

[kevinthesun]

@FrozenGene In #5200 we discussed another source of autotvm inaccurate measurement due to empty input tensor. Do we have a timeline to fix that?

[FrozenGene]

@FrozenGene In #5200 we discussed another source of autotvm inaccurate measurement due to empty input tensor. Do we have a timeline to fix that?

I hope we could bring this with Auto Scheduler this month if every thing goes well.

[FrozenGene]

@FrozenGene In #5200 we discussed another source of autotvm inaccurate measurement due to empty input tensor. Do we have a timeline to fix that?

@kevinthesun FYI. We are rebasing our code based on the laster master. After completing it, this fix will be brought in.

[tqchen]

@FrozenGene please update the status of this issue

[tqchen]

ping @FrozenGene

[FrozenGene]

Currently, as listed, we have completed Intel CPU / ARM CPU / Bifrost / Mali (thanks to community's help) implementation except cuda (some progress we have done like conv2d_nhwc_winograd / conv3d_winograd when we review related prs) / intel graphics.

[tqchen]

OK, we will leave the issue open then, please remember to update it as we add more improvements.

[tqchen]

@FrozenGene please check if we can close this issue either due to non-actionable status(stale) or completion

[FrozenGene]

i think we could close it as Ansor solves the problem for all platforms. For AutoTVM's cuda platform, there is no activity on it. If anyone is interested on it, he / she could do it for cuda on AutoTVM too.