Add ONNX export support for TE modules#41
Conversation
|
Hi @asfiyab-nvidia, what is that libcustom so file? |
|
@ptrendx it contains the onnxruntime (ORT) implementations for FP8 functionality. This is used to test the ONNX export and validate the ORT outputs against TE outputs. (code under |
|
Does that have to be closed source? If so, can we at least move it to tests directory instead of the top level one? If it does not have to be closed source then maybe we can have the source inside tests and compile it on the fly? |
asfiyab-nvidia
force-pushed
the
dev-onnx-export-support
branch
from
693ee53 to
8ed54a8
Compare
|
Moving the .so to the tests directory seems to be a better approach at the moment. We can potentially include the source code in a follow up PR. |
asfiyab-nvidia
force-pushed
the
dev-onnx-export-support
branch
from
b39f87e to
0cf5e16
Compare
asfiyab-nvidia
force-pushed
the
dev-onnx-export-support
branch
from
65f4196 to
b9b5477
Compare
|
/te-ci |
|
Please fix the tests (see the results for commit 4812408) - the biggest problem is that you try to run tests requiring FP8 on non-Hopper, which triggers the assertion failure. I am working on enabling Hopper GPU in CI, so we should be able to get the FP8 tests running soon too. |
|
@ptrendx is there some code in TE we can leverage to query the SM version, or do you recommend us installing some lib (e.g. pynvml)? |
|
/te-ci |
1 similar comment
|
/te-ci |
|
@ptrendx can you please authorize a pipeline run for the latest commit? It contains fixes for the failures from the last run. Thanks |
|
/te-ci |
* Add TorchScript Operators * Add symbolic methods to ONNX exporter * Add tests for the ONNX export Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
* Increase layernorm FP16 threshold * Normalize onnx file names: _ separates configs; - separates words in a single config * Add get_attn_mask_str and fix mask string * Add missing ONNX files * Moved generated ONNX files to tests/gen_onnx_models/ Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
asfiyab-nvidia
force-pushed
the
dev-onnx-export-support
branch
from
119a0ec to
ab4410f
Compare
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> Signed-off-by: asfiyab-nvidia <117682710+asfiyab-nvidia@users.noreply.github.com>
|
/te-ci |
1 similar comment
|
/te-ci |
[New API] Added support for Reshape operation. [New API] Added support for DgradDreluBNBwdWeight operation [Minor Enhancement] Added cudnn frontend enums to simplify Resample operation creation. [Minor Enhancement] Added alpha and beta values as key for the plan caches. [Bug Fix] Fixed an error which was causing reference code to fail with segmentation fault. [Bug Fix] Fixed an issue where stride/padding and dilation values were incorrectly cached for 2d convolutions. [Bug Fix] Fixed issues where error statuses were not handled correctly during tensor creation. [Samples] Added a new sample to show case how fMHA graph can be programmed through FE API. This sample contains both fprop and backprop graphs. [Samples] Added a new sample to show case DgradDreluBNBwdWeight operation. [Samples] Added a modular block which models fprop of residual block resnet. Co-authored-by: Anerudhan Gopal <agopal@nvidia.com>
4 participants
Signed-off-by: Asfiya Baig asfiyab@nvidia.com
Signed-off-by: Neta Zmora nzmora@nvidia.com