CUDA graph phase N - explicit graph construction

[leofang]

Instead of relying on stream capturing, which is considered an implementation detail (that in the future we could allow users to opt in or out), our graph builder APIs were designed to allow an alternative implementation based on explicitly constructing a graph by adding nodes.

The use cases are unclear atm, so just keeping this in the backlog for now.

[Andy-Jost]

I'm picking this up and trying to nail down the scope. The issue describes an alternative GraphBuilder backend that uses explicit graph construction (cuGraphCreate + cuGraphAddNode) instead of stream capture.

Although the graph interface is mostly independent of the construction method, I see two conceptual leaks:

• The GraphBuilder.stream property is tightly coupled to capture. The entire usage pattern is kernel.launch(builder.stream, ...).
• begin_building(mode=...) takes a mode parameter that maps directly to CUstreamCaptureMode.

If I understand the proposal correctly, we want to retain the same user-facing API while changing the construction method internally. That makes this a large cross-cutting change: every capturable operation needs a dispatch branch. For example, kernel launch becomes something like:

class Kernel:
    def launch(self, stream, ...):
        graph_builder = _get_active_builder(stream)
        if graph_builder is not None:
            graph_builder._add_kernel_node(self, ...)
        else:
            cuLaunchKernel(...)

This would add overhead to the hot paths and would largely duplicate logic already handled by the driver's own graph capture machinery, in addition to creating new maintenance burdens.

Before starting the implementation, I'd like to understand the motivation better. What use cases or user stories would explicit construction enable that stream capture doesn't handle well today? What does a "win" look like here? A concrete example would help a lot.

[Andy-Jost]

After discussion the plan is:

1. Develop a private module for explicit graph construction (cuGraphCreate + cuGraphAddNode), not exposed as public API.
2. Test it in isolation.
3. Wire it into the public GraphBuilder interface in a follow-up.

Stream capture remains the public-facing mechanism for now. Explicit construction becomes an internal implementation detail that enables a unified interface later.