[RFC] Support Codegen Using Texture Memory

[antinucleon]

Current TVM is not supporting mixture texture and buffer. This pattern is essential to obtain state-of-art performance on iOS device, and potentially better OpenGL backend for legacy devices.

A potential solution is providing extra device hints, eg:

a = tvm.placeholder((32,), name="a", storage_dev="2d_texture")
b = tvm.placeholder((32,), name="b", storage_dev="buffer")

In lowering process, some extra work is required I think:

1. We need a pass to infer output storage dev for each node
2. We need special data access lowing rule for texture, especially different layout of texture (1D/2D texture)
3. We need modify codegen
4. We need to modify runtime, to support NDArray laying on texture memory.

Once we support 2D texture, I think we are able to achieve state-of-art performance on iOS, and OpenGL.

[tqchen]

I would recommend us to start with backend support for texture and list them as a separate device type. Once we have enough evidence to support the texture code, then we can start this migration

[antinucleon]

@dmlc/tvm-comitter

Just FYI there is other effort on this topic: https://medium.com/tensorflow/tensorflow-lite-now-faster-with-mobile-gpus-developer-preview-e15797e6dee7

I want to check whether there is other people in community plan to work on this problem so we may avoid redundant work.

[masahi]

This should be supported now.

[tmoreau89]

Technically it hasn't been merged in but it's worth pointing to the RFCs and PRs for posterity @masahi:

See this RFC to understand the broader contributions of the texture enablement work.

There are several PRs that need to land in order for the texture support to be fully enabled in Open Source TVM. Some of these PRs will need to be revisited extensively before they can be merged, for instance in light of the work on TECompiler refactor, and efforts to provide physical and logical layout separation.

• MERGED [Texture support][Part 0] Device API and runtime support [Texture support][Part 0] Device API and runtime support #7711

• MERGED [Texture support][Part 1] TIR lowering and OpenCL support [Texture support][Part 1] TIR lowering and OpenCL support #7686

• [Texture support][Part 2] Add opencl adreno target, topi schedules, and relay op strategies [Texture support][Part 2] Add opencl adreno target, topi schedules, and relay op strategies  #7687

• [Texture support][Part 3] Support storage scope tag in graph runtime codegen, planning, runtime and compile engine [Texture support][Part 3] Support storage scope tag in graph runtime codegen, planning, runtime and compile engine  #7688

• [Texture support][Part 4] Add CollectStorageInfo and CollectBufferBinds relay passes for Adreno GPU [Texture support][Part 4] Add CollectStorageInfo and CollectBufferBinds relay passes for Adreno GPU  #7689

• [Texture support][Part 5] Graph runtime and memory planning support for 2d allocations [Texture support][Part 5] Graph runtime and memory planning support for 2d allocations  #7690

[masahi]

Thanks @tmoreau89, I've opened a fresh tracking #9891

[tmoreau89]

Good idea, thank you @masahi !