[Hexagon] Add support for instrumentation based profiling for Hexagon #12971

jverma-quic · 2022-10-03T17:40:04Z

This is done by instrumenting the code with profiling builtin calls using a TIR pass. During codegen, these builtin calls are replaced with the calls to a hexagon specific handler which records the runtime information into a buffer. This buffer is written into a JSON file ('lwp.json') which is processed to construct function and loop-level profiling information as a csv file.

At a high-level, this PR makes the following changes:

Add a TIR pass (src/tir/transforms/profile_instrumentation.cc) to instrument the functions and loops with profilging builtins.
Hexagon codegen changes to replace profilng builtin calls with the call to Hexagon specific handler. This handler records the runtime data into a buffer. For all other targets, these builtin calls are ignored.
Add API to RPC Launcher to get the profiling data as a JSON file
Add a python script (python/tvm/contrib/hexagon/profiling/process_lwp_data.py) to process the profiling data and construct a CSV file
Add TVM script based unit tests to test and demonstrate various profiling config flags: tests/python/unittest/test_tir_transform_profiling_instr.py
Adds two tests in tests/python/contrib/test_hexagon/test_launcher.py to demonstrate necessary changes to enable profiling and to collect and process runtime data.

For additional details, please refer to src/runtime/hexagon/profiler/README.md and apps/hexagon_launcher/README.md.

This's done by instrumenting the code with profiling builtin calls using a TIR pass. During codegen, these builtin calls are replaced with the calls to a hexagon specific handler which records the runtime information into a buffer. This buffer is written into a JSON file ('lwp.json') which is processed to construct function and loop-level profiling information as a csv file. At a high-level, this PR makes the following changes: 1) Add a TIR pass (src/tir/transforms/profile_instrumentation.cc) to instrument the functions and loops with profilging builtins. 2) Hexagon codegen changes to replace profilng builtin calls with the call to Hexagon specific handler. This handler record the runtime data into a buffer. For all other targets, these builtin calls are ignored. 3) Add API to RPC Launcher to get the profiling data as a JSON file 4) A python script to process the profiling data and construct a CSV file 5) Add TVM script based unit tests to test and demonstrate various profiling config flags: tests/python/unittest/test_tir_transform_profiling_instr.py 6) Adds two tests in tests/python/contrib/test_hexagon/test_launcher.py to demonstrate necessary changes to enable profiling and to collect and process runtime data. For additional details, please refer to src/runtime/hexagon/profiler/README.md

tmoreau89 · 2022-10-03T18:05:49Z

CC @tkonolige

tmoreau89 · 2022-10-03T18:08:43Z

apps/hexagon_launcher/README.md

+Here, `instrument_lwp` is used to enable the tir pass which instruments the code with the builtin calls.
+
+During codegen, profiling builtin calls can be replaced with a target specific handler to record runtime
+information into a buffer. This buffer is written into a JSON file which is proccessed to construct


proccessed -> processed

tmoreau89 · 2022-10-03T18:13:39Z

src/runtime/hexagon/profiler/README.md

+# Hexagon lightweight instrumentation based profiling (LWP)
+
+For Hexagon, LWP can be used to get function and loop level processor cycle count.
+This's done by instrumenting the code with profiling builtin calls using a TIR pass.


This's -> This is

tmoreau89 · 2022-10-05T14:40:32Z

@tkonolige - do you have some cycles to take a look at profile_instrumentation.cc specifically? thanks

tmoreau89 · 2022-10-05T14:44:03Z

@jverma-quic are you able to resolve the merge conflicts in tests/python/contrib/test_hexagon/test_launcher.py

tkonolige

@jverma-quic this is pretty awesome! I've wanted something like this for a while.

Thinking longer-term, I'd like to see this supported for more targets than hexagon. Maybe we could discuss (not in this PR) how we could integrate this work into the existing profiling tools. I'd especially love to see support for different runtime metrics (hardware counters, etc). Do you think it would be possible to integrate the (MetricCollector)[https://github.com/apache/tvm/blob/main/include/tvm/runtime/profiling.h#L278] interface into this? (Once again, I'm just thinking about the future. Its not necessary for this PR)

src/tir/transforms/profile_instrumentation.cc

src/runtime/hexagon/profiler/lwp_handler.S

tkonolige · 2022-10-05T16:04:33Z

src/runtime/hexagon/profiler/prof_utils.cc

+unsigned int lwp_counter[LWP_COUNTER_SIZE] = {0};
+unsigned int lwp_buffer[LWP_BUFFER_SIZE];
+unsigned int* __lwp_counter = lwp_counter;
+unsigned int* __lwp_buffer_ptr = lwp_buffer;


Maybe use a fixed size type (uint32_t) for this?

tkonolige · 2022-10-05T16:08:04Z

src/runtime/hexagon/profiler/README.md

+This is done by instrumenting the code with profiling builtin calls using a TIR pass.
+During codegen, these builtin calls are replaced with the calls to a hexagon specific
+handler which records the runtime information into a buffer.
+This buffer is written into a JSON file ('lwp.json') which is processed to construct


What is the reason to save the data to disk vs just sending it directly over the network? Seems like it would simplify the interface and the code to not have to write out the data and then re-parse it.

There is no easy way of getting the data from Hexagon. It needs to be saved into a file. Since this file contains the processor cycles for each handler invocation, it needs to be processed for ease of understanding.

tkonolige · 2022-10-05T16:11:27Z

src/runtime/hexagon/profiler/README.md

+    profiler = HexagonProfiler()
+```
+
+4) Run the model and get profile data (`lwp.json`) from the device (or the simulator):


Can you include a fully runnable example here? I think it would be helpful for people trying to use the profiler.

Yes, there're two examples, (test_lwp and test_lwp_multiple_conv2d) in tests/python/contrib/test_hexagon/test_launcher.py. You should be able to run them on the hexagon simulator using pytest.

tkonolige · 2022-10-05T16:12:29Z

src/runtime/hexagon/profiler/README.md

@@ -0,0 +1,152 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->


I think this file belongs in the docs directory.

src/driver/driver_api.cc

python/tvm/contrib/hexagon/build.py

cmake/modules/Hexagon.cmake

jverma-quic · 2022-10-05T20:43:35Z

@jverma-quic this is pretty awesome! I've wanted something like this for a while.

Thinking longer-term, I'd like to see this supported for more targets than hexagon. Maybe we could discuss (not in this PR) how we could integrate this work into the existing profiling tools. I'd especially love to see support for different runtime metrics (hardware counters, etc). Do you think it would be possible to integrate the (MetricCollector)[https://github.com/apache/tvm/blob/main/include/tvm/runtime/profiling.h#L278] interface into this? (Once again, I'm just thinking about the future. Its not necessary for this PR)

@tkonolige - Thanks for the review comments and for suggestions for the future enhancements! Let me look into MetricCollector to see if it can be integrated with what I have here. The only part that makes it Hexagon specific is the LLVM intrinsic (which doesn't have to be hexagon specific) and its lowering in LLVM backend to a hexagon specific handler function that collects runtime data.

Add type hint Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com>

Simplify the interface to the lightweight profiling.

jverma-quic · 2022-10-13T21:19:47Z

Hi @tkonolige, Sorry it took me a while to address all your review comments. I think I have been able to resolve most of them. Could you please take a look?
One thing that remains unresolved is your comment regarding README.md belonging to the docs directory. Do I need to move it under ./docs?

tkonolige

Thanks for all the changes. Looking at apps/hexagon_launcher/README.md, it seems to a be a big file. I think leaving it where it is for this PR is fine. It would be good to move it to docs/how_to/deploy in a separate PR.

tkonolige · 2022-10-13T21:23:46Z

apps/hexagon_launcher/cmake/hexagon/CMakeLists.txt

  "${LAUNCHER_SRC}/launcher_core.cc"
  "${LAUNCHER_SRC}/launcher_hexagon.cc"
 )
+set(PROFILER_DIR "${TVM_SOURCE_DIR}/src/runtime/hexagon/profiler")


Suggested change

set(PROFILER_DIR "${TVM_SOURCE_DIR}/src/runtime/hexagon/profiler")

set(HEXAGON_PROFILER_DIR "${TVM_SOURCE_DIR}/src/runtime/hexagon/profiler")

jverma-quic · 2022-10-14T19:50:17Z

@tkonolige - I have made a change in codegen_hexagon.cc to ignore profile builtins for llvm versions < 15.0. This builtin is lowered into a Hexagon specific llvm intrinsic, which was added prior to LLVM 15.0 release, causing upstream CI builds to fail.

jverma-quic · 2022-10-14T21:28:53Z

@tmoreau89: My PR contains a .S file which is the hand written assembly code for the profiling handler for hexagon. The CI is failing as it doesn't expect .S file to be checked in. Can I get an exception for this?

joshherr-quic

Just one comment regarding the name used for the binary. It is hardcoded in the python profiler class.

joshherr-quic · 2022-10-20T09:09:16Z

python/tvm/contrib/hexagon/hexagon_profiler.py

+    def __init__(self, module: ExecutorFactoryModule, hexagon_server_process, enable_debug):
+        """Configure HexagonProfiler"""
+        # Save test .so to process profiling data
+        dso_binary = "test_binary.so"


I think this needs to be abstracted to allow for other binary names

That's a good suggestion, @joshherr-quic. Thanks!

tvm-bot · 2022-10-25T18:37:24Z

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Built docs for commit 774f61b can be found here.

_{Generated by tvm-bot}

…apache#12971) * [Hexagon] Add support for instrumentation based profiling for Hexagon This's done by instrumenting the code with profiling builtin calls using a TIR pass. During codegen, these builtin calls are replaced with the calls to a hexagon specific handler which records the runtime information into a buffer. This buffer is written into a JSON file ('lwp.json') which is processed to construct function and loop-level profiling information as a csv file. At a high-level, this PR makes the following changes: 1) Add a TIR pass (src/tir/transforms/profile_instrumentation.cc) to instrument the functions and loops with profilging builtins. 2) Hexagon codegen changes to replace profilng builtin calls with the call to Hexagon specific handler. This handler record the runtime data into a buffer. For all other targets, these builtin calls are ignored. 3) Add API to RPC Launcher to get the profiling data as a JSON file 4) A python script to process the profiling data and construct a CSV file 5) Add TVM script based unit tests to test and demonstrate various profiling config flags: tests/python/unittest/test_tir_transform_profiling_instr.py 6) Adds two tests in tests/python/contrib/test_hexagon/test_launcher.py to demonstrate necessary changes to enable profiling and to collect and process runtime data. For additional details, please refer to src/runtime/hexagon/profiler/README.md * Fix typos * Update python/tvm/contrib/hexagon/build.py Add type hint Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com> * Address review comments Simplify the interface to the lightweight profiling. * Ignore profile builtins if llvm version < 15.0 * Add src/runtime/hexagon/profiler/lwp_handler.S to allowed list * Address reformatting issues * Fix pylint errors * Address remaining linter failures * clang-format issue * Fix builtin names * Resolve test failure for the simulator run * Allow for the tests to provide .so name Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com>

tmoreau89 reviewed Oct 3, 2022

View reviewed changes

Fix typos

5d39093

Merge branch 'main' into hexagon-instr-profiling

4d0a141

tkonolige reviewed Oct 5, 2022

View reviewed changes

jverma-quic and others added 3 commits October 5, 2022 15:59

Update python/tvm/contrib/hexagon/build.py

ca5e6f3

Add type hint Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com>

Merge branch 'main' into hexagon-instr-profiling

0213cd6

Address review comments

8b8e3b9

Simplify the interface to the lightweight profiling.

jverma-quic requested review from tkonolige and tmoreau89 and removed request for tkonolige and tmoreau89 October 13, 2022 21:15

tkonolige approved these changes Oct 13, 2022

View reviewed changes

Ignore profile builtins if llvm version < 15.0

2b6b15d

Merge branch 'apache:main' into hexagon-instr-profiling

470d047

jverma-quic added 2 commits October 18, 2022 11:50

Add src/runtime/hexagon/profiler/lwp_handler.S to allowed list

b9bd7c3

Address reformatting issues

9ab2ccd

areusch added needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it and removed needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it labels Oct 19, 2022

jverma-quic and others added 7 commits October 19, 2022 11:02

Fix pylint errors

fd52d05

Merge branch 'main' into hexagon-instr-profiling

1754d2d

Address remaining linter failures

39204ae

clang-format issue

d1aae57

Fix builtin names

c93cc23

Merge branch 'apache:main' into hexagon-instr-profiling

a34689f

Resolve test failure for the simulator run

514f8cf

joshherr-quic suggested changes Oct 20, 2022

View reviewed changes

jverma-quic and others added 3 commits October 20, 2022 10:28

Merge branch 'apache:main' into hexagon-instr-profiling

0960de6

Allow for the tests to provide .so name

80353b3

Merge branch 'apache:main' into hexagon-instr-profiling

e7f708c

joshherr-quic approved these changes Oct 24, 2022

View reviewed changes

Merge branch 'apache:main' into hexagon-instr-profiling

774f61b

kparzysz-quic merged commit 23c2909 into apache:main Oct 25, 2022

leandron mentioned this pull request Feb 1, 2023

TVM v0.11.0 Release Candidate Notes #13899

Closed

		@@ -0,0 +1,152 @@
		<!--- Licensed to the Apache Software Foundation (ASF) under one -->

	set(PROFILER_DIR "${TVM_SOURCE_DIR}/src/runtime/hexagon/profiler")
	set(HEXAGON_PROFILER_DIR "${TVM_SOURCE_DIR}/src/runtime/hexagon/profiler")

[Hexagon] Add support for instrumentation based profiling for Hexagon #12971

[Hexagon] Add support for instrumentation based profiling for Hexagon #12971

Uh oh!

Conversation

jverma-quic commented Oct 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmoreau89 commented Oct 3, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmoreau89 commented Oct 5, 2022

Uh oh!

tmoreau89 commented Oct 5, 2022

Uh oh!

tkonolige left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jverma-quic commented Oct 5, 2022

Uh oh!

jverma-quic commented Oct 13, 2022

Uh oh!

tkonolige left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jverma-quic commented Oct 14, 2022

Uh oh!

jverma-quic commented Oct 14, 2022

Uh oh!

joshherr-quic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tvm-bot commented Oct 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jverma-quic commented Oct 3, 2022 •

edited

Loading