Skip to content

Native call stack support for VS Code Jupyter Debugger#9

Open
mattChrisP wants to merge 1 commit intobterwijn:mainfrom
mattChrisP:feat/vscode-jupyter-support
Open

Native call stack support for VS Code Jupyter Debugger#9
mattChrisP wants to merge 1 commit intobterwijn:mainfrom
mattChrisP:feat/vscode-jupyter-support

Conversation

@mattChrisP
Copy link
Copy Markdown

Feature: Native call stack support for VS Code Jupyter Debugger

This PR introduces call stack parsing for Jupyter Notebook environments, ensuring memory_graph can seamlessly visualize cell executions without blowing up or rendering kernel noise.

Demonstration

memory_graph_2x.mov

(Note: This demonstration uses a locally installed version of memory_graph containing these PR changes, alongside a background script that watches the generated .gv file and automatically renders it to .png.)

Feature Summary

  1. stack_multi_slice: A robust version of stack_slice that acts like an eraser across the entire stack sequence, rather than terminating early on the first match.
  2. is_vscode_jupyter_banned: A universal safety net that filters out IPython noise while utilizing a strict prefix bypass (__main__ / <ipython-input>) to mathematically guarantee user code is never accidentally dropped.
  3. Return Value Alignment: Intercepts __pydevd_ret_val_dict, strips out underlying Jupyter I/O stream returns (like OutStream.write), and ensures return values are visually mapped back to the originating function's frame dictionary.
  4. IDE Alias vscode_jupyter: Added mg.vscode_jupyter() as a top-level wrapper, keeping the API consistent with existing debuggers and allowing instant usage in the VS Code Watch panel.

Why stack_slice fails in this environment

The legacy approach (stack_slice) searches from the top (innermost) frame downward, finds the very first match (e.g., trace_dispatch), adds a static frame offset (e.g., + 2), and blindly traces everything beneath it.

(Note: The following stack slices are drawn from exact, raw call stacks (using save_call_stack() function) logged from a live Jupyter notebook under VS Code)

Example 1 (Two trace_dispatch frames)

  function:trace_dispatch           <-- 1st occurrence!
  function:__call__                 
  function:trace_dispatch           <-- 2nd occurrence!
  function:sum_of_squares           <-- VALID (User code)

Example 2 (One trace_dispatch frame)

  function:trace_dispatch           <-- 1st occurrence!
  function:square                   <-- VALID (User code)
  function:sum_of_squares           <-- VALID (User code)

Result: If we use stack_slice with an offset of 3 (calibrated to work for Example 1), running that exact same slice on Example 2 would blindly skip trace_dispatch, skip square, and skip sum_of_squares. It mathematically guarantees accidentally deleting user code or leaving debugger garbage in the graph.


✅ How stack_multi_slice fixes it

Instead of relying on a static offset to jump over internal debugger hooks, stack_multi_slice coupled with is_vscode_jupyter_banned evaluates the entire sequence of frames.

  1. stack_multi_slice scans the full stack and acts like an eraser. Every time it sees a trace_dispatch (offset 1), it deletes that frame. If it sees _do_wait_suspend (offset 2), it deletes that frame and its immediately bound companion frame.
  2. is_vscode_jupyter_banned loops through what's left and erases every frame originating from IPython, debugpy, tornado, asyncio, etc.

Clean Output from stack_multi_slice

Because it iterates and explicitly deletes garbage frames wherever they appear, it seamlessly handles Example 1 and Example 2 without dropping user variables.

- function:_do_wait_suspend           <-- REMOVED (drop_functions)
- function:do_wait_suspend            <-- REMOVED (drop_functions)
- function:do_wait_suspend            <-- REMOVED (drop_functions)
- function:trace_dispatch             <-- REMOVED (drop_functions)
- function:__call__                   <-- REMOVED (is_vscode_jupyter_banned)
- function:trace_dispatch             <-- REMOVED (drop_functions)
+ function:sum_of_squares             <-- KEPT (User Code)
+ function:<module>                   <-- KEPT (This is where end_functions stops processing)
- function:run_code                   <-- REMOVED (never reached by dict slice)
- ... (kernel tail)                   <-- REMOVED

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant