Skip to content

aoti_torch_create_tensor_from_blob_v2#14687

Merged
larryliu0820 merged 1 commit intogh/larryliu0820/77/basefrom
gh/larryliu0820/77/head
Sep 30, 2025
Merged

aoti_torch_create_tensor_from_blob_v2#14687
larryliu0820 merged 1 commit intogh/larryliu0820/77/basefrom
gh/larryliu0820/77/head

Conversation

@larryliu0820
Copy link
Contributor

@larryliu0820 larryliu0820 commented Sep 30, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

Summary:

This function introduce aoti_torch_create_tensor_from_blob_v2, a function that create tensor from data blob and custom stride and size.

Worth to notice that unlike aoti_torch_empty_strided, the tensor created by aoti_torch_create_tensor_from_blob_v2 will not have the control of the memory blob. Therefore when we delete it, the memory will not be freed.

Reviewed By:

Differential Revision:

@larryliu0820
aoti_torch_create_tensor_from_blob_v2
975fb6b 
Summary:

This function introduce aoti_torch_create_tensor_from_blob_v2, a function that create tensor from data blob and custom stride and size.

Worth to notice that unlike aoti_torch_empty_strided, the tensor created by aoti_torch_create_tensor_from_blob_v2 will not have the control of the memory blob. Therefore when we delete it, the memory will not be freed.

Reviewed By:

Differential Revision:

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 30, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14687

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Cancelled Jobs, 1 Unrelated Failure

As of commit 975fb6b with merge base db8d04f (image):

NEW FAILURE - The following job has failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 30, 2025
This was referenced Sep 30, 2025
Merged
Merged
Merged
Merged
@github-actions
Copy link

github-actions bot commented Sep 30, 2025

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
tensor destroy (#14686)
ac2cc6c 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #14690
* #14689
* #14688
* #14687
* __->__ #14686

Summary:

This diff introduce `aoti_torch_delete_tensor_object` for deleting
tensors created during cuda backend inference.

Reviewed By:
@larryliu0820 larryliu0820 merged commit 56b6ac6 into gh/larryliu0820/77/base Sep 30, 2025
127 of 133 checks passed
@larryliu0820 larryliu0820 deleted the gh/larryliu0820/77/head branch September 30, 2025 17:40
larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
Revert "aoti_torch_create_tensor_from_blob_v2 (#14687)"
62771a1 
This reverts commit 56b6ac6.
@larryliu0820 larryliu0820 restored the gh/larryliu0820/77/head branch September 30, 2025 17:45
@larryliu0820 larryliu0820 mentioned this pull request Sep 30, 2025
Merged
larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
aoti_torch_create_tensor_from_blob_v2 (#14700)
41379fb 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #14690
* #14689
* #14688
* __->__ #14700
* #14686

Summary:
This is a manual cherry pick of #14687 

This function introduce aoti_torch_create_tensor_from_blob_v2, a
function that create tensor from data blob and custom stride and size.

Worth to notice that unlike aoti_torch_empty_strided, the tensor created
by aoti_torch_create_tensor_from_blob_v2 will not have the control of
the memory blob. Therefore when we delete it, the memory will not be
freed.

Reviewed By:

Differential Revision:
larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
aoti_torch__reinterpret_tensor (#14688)
696bf19 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #14690
* #14689
* __->__ #14688
* #14687
* #14686

Summary:

Introduced aoti_torch__reinterpret_tensor, which creates a new tensor
view that reinterprets the same underlying memory with custom shape and
strides.

Reviewed By:

Differential Revision:
larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
aoti_torch_copy_ (#14689)
65100f6 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #14690
* __->__ #14689
* #14688
* #14687
* #14686

Summary:

This diff introduce `aoti_torch_copy_`, the function for copying tensor
inside cuda backend.

Right now it only support copy between tensors with same dtype.

Reviewed By:

Differential Revision:
larryliu0820 added a commit that referenced this pull request Sep 30, 2025
@larryliu0820
use ET_CHECK macro for sanity checks in memory shim layer (#14690)
258bce3 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* __->__ #14690
* #14689
* #14688
* #14687
* #14686

Summary:

this is a comprehensive update to use ET_CHECK macro to replace original
if..else check for better follow et's law

Reviewed By:

Differential Revision:
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
@larryliu0820 @jirioc
aoti_torch_create_tensor_from_blob_v2 (pytorch#14700)
841498f 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* pytorch#14690
* pytorch#14689
* pytorch#14688
* __->__ pytorch#14700
* pytorch#14686

Summary:
This is a manual cherry pick of pytorch#14687 

This function introduce aoti_torch_create_tensor_from_blob_v2, a
function that create tensor from data blob and custom stride and size.

Worth to notice that unlike aoti_torch_empty_strided, the tensor created
by aoti_torch_create_tensor_from_blob_v2 will not have the control of
the memory blob. Therefore when we delete it, the memory will not be
freed.

Reviewed By:

Differential Revision:
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
@larryliu0820 @jirioc
aoti_torch__reinterpret_tensor (pytorch#14688)
fedb733 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* pytorch#14690
* pytorch#14689
* __->__ pytorch#14688
* pytorch#14687
* pytorch#14686

Summary:

Introduced aoti_torch__reinterpret_tensor, which creates a new tensor
view that reinterprets the same underlying memory with custom shape and
strides.

Reviewed By:

Differential Revision:
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
@larryliu0820 @jirioc
aoti_torch_copy_ (pytorch#14689)
b443ce2 
Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* pytorch#14690
* __->__ pytorch#14689
* pytorch#14688
* pytorch#14687
* pytorch#14686

Summary:

This diff introduce `aoti_torch_copy_`, the function for copying tensor
inside cuda backend.

Right now it only support copy between tensors with same dtype.

Reviewed By:

Differential Revision:
jirioc pushed a commit to nxp-upstream/executorch that referenced this pull request Dec 19, 2025
@larryliu0820 @jirioc
use ET_CHECK macro for sanity checks in memory shim layer (pytorch#14690
7b854df 
)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* __->__ pytorch#14690
* pytorch#14689
* pytorch#14688
* pytorch#14687
* pytorch#14686

Summary:

this is a comprehensive update to use ET_CHECK macro to replace original
if..else check for better follow et's law

Reviewed By:

Differential Revision:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manuelcandales manuelcandales manuelcandales approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@larryliu0820 @manuelcandales