[Bug][VTA][OpenCL] If allowed to allocate in stack, VTA multiple target test will fail

[zhanghaohit]

In src/tir/transforms/lower_tvm_builtin.cc:

    if (device_type_.defined()) {
      if (const auto* dev_type = device_type_.as<IntImmNode>()) {
        if (dev_type->value == kDLCPU) {
          int32_t constant_size = op->constant_allocation_size();
          if (constant_size > 0 && constant_size * nbytes < runtime::kMaxStackAlloca) {
            return stmt;
          }
        }
      }
    }

If the above optimization code is here, it will raise LLVM function signature errors when there are multiple targets. This piece of code was removed in #6126 to fix this issue. But it was reverted back in #8274, as removal of this code will cause some VM test failed.

To quick fix, the multiple target unittests are disabled in the current main branch. But we need further investigation on how to resolve this nicely.

cc @echuraev @elvin-n

[manupak]

Thanks for flagging this! and following up from #8274

IIUC,

This is problematic for micro because now the constant sized allocates are forced to be placed on stack (bypassing TVMPlatformAllocate abstraction) because that is how the codegen_c lowers the allocate.

tvm/src/target/source/codegen_c.cc

Lines 860 to 877 in 1b99adc

	void CodeGenC::VisitStmt_(const AllocateNode* op) {
	ICHECK(!is_zero(op->condition));
	std::string vid = AllocVarID(op->buffer_var.get());
	this->PrintIndent();
	int32_t constant_size = op->constant_allocation_size();
	ICHECK_GT(constant_size, 0) << "Can only handle constant size stack allocation for now";
	auto scope = GetPtrStorageScope(op->buffer_var);
	alloc_storage_scope_[op->buffer_var.get()] = scope;
	PrintStorageScope(scope, stream);
	PrintType(op->dtype, stream);
	stream << ' ' << vid << '[' << constant_size << "];\n";
	RegisterHandleType(op->buffer_var.get(), op->dtype);
	this->PrintStmt(op->body);
	}

I dont think is desired.

Can someone explain why the guarded TVMBackendAllocWorkspace calls to allocate memory is problematic ? -- which are the ones I see generated in TIR prior to the last revert : #8274 ?

cc: @jroesch @mbrookhart

[manupak]

Also for some reason, if we really need the tir.allocate to be translated down to as a stack placement, we should probably make the tir.allocate with storage_scope = "local" to PrimFunc that is placed on CPU.

keep the global ones served via TVMBAWs, because TVMBAW could serve as the 'global' allocator for memory.
In this way, in micro we could still use TVMBAWs to serve memory from the application/platform layer for 'global' allocates.

[mbrookhart]

FYI - this PR renables the test that this change breaks #9019

[mbrookhart]

@mbs-octoml opened this issue to track an approach to better distinguish multiple targets #9022