[RUNTIME][ALIGNMENT] Configurable kAllocAlignment if needed #13307
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment. Generated by tvm-bot |
srkreddy1238
force-pushed
the
adreno_kalignment
branch
from
0aea9f6 to
6328202
Compare
Not all plarforms 64bit aligned allocations. Platforms with 32bit alignment fail to support set_input_zero_copy even though the ndarray is allocated by the tvm runtime itself. This change enabled configurable option for such targets.
srkreddy1238
force-pushed
the
adreno_kalignment
branch
from
6328202 to
9077d0a
Compare
junrushao
reviewed
Nov 11, 2022
junrushao
reviewed
Nov 11, 2022
srkreddy1238
force-pushed
the
adreno_kalignment
branch
from
b0ddce4 to
907a521
Compare
Contributor
Author
|
cc @junrushao |
Member
|
Thanks @srkreddy1238 for the PR! |
xinetzone
pushed a commit
to daobook/tvm
that referenced
this pull request
Nov 25, 2022
…3307) Not all plarforms 64bit aligned allocations. Platforms with 32bit alignment fail to support set_input_zero_copy even though the ndarray is allocated by the tvm runtime itself. This change enabled configurable option for such targets. Co-authored-by: Siva Rama Krishna Reddy B <sivb@blr-ubuntu-ripper.qualcomm.com>
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
May 4, 2023
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught.
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
May 8, 2023
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught.
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
Jun 5, 2023
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught.
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
Jun 12, 2023
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught.
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
Jul 5, 2023
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught. This check is also restricted to targets whose `void*` opaque pointer can be interpreted as a pointer to the data array. (e.g. No such check applies on Vulkan, as the `void*` is a pointer to a struct that contains additional bookkeeping.)
Lunderberg
added a commit
to Lunderberg/tvm
that referenced
this pull request
May 23, 2024
Prior to this commit, any use of `tvm.nd.from_dlpack` to create a strided `NDArray`, or a `NDArray` whose alignment was less than `tvm::runtime::kAllocAlignment` would raise an error. As a result, views into larger arrays, which are unlikely to be aligned and compact, could only be shared when copied into an aligned and compact buffer. This commit moves the compact/aligned check from the `NDArray` class into the generated TIR code as part of DLTensor unpacking. These checks were initially introduced in apache#11391, to avoid segfaults caused by use of non-aligned buffers in code intended for aligned buffers. The new checks will provide the same safeguard as the alignment is checked prior to use, but allows the alignment requirement to be relaxed on a per-buffer basis. This approach also removes a potential bug resulting from compile-time configuration of `tvm::runtime::kAllocAlignment`, first introduced in apache#13307. Since TVM supports cross-compiling, the installation of TVM used to compile a kernel may assume a larger value of `kAllocAlignment` than is provided by the runtime installation of TVM. By validating the alignment within the generated kernel, rather than as part of the runtime, this potential inconsistency would be caught. This check is also restricted to targets whose `void*` opaque pointer can be interpreted as a pointer to the data array. (e.g. No such check applies on Vulkan, as the `void*` is a pointer to a struct that contains additional bookkeeping.)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not all plarforms 64bit aligned allocations. Platforms with 32bit alignment fail to support set_input_zero_copy even though the ndarray is allocated by the tvm runtime itself.
This change enabled configurable option for such targets.