Skip to content

refactor(jit): complete modularization and de-godding of codegen pipeline#3

Merged
SSL-ACTX merged 1 commit intomainfrom
refactor/jit-codegen-submodules
Mar 19, 2026
Merged

refactor(jit): complete modularization and de-godding of codegen pipeline#3
SSL-ACTX merged 1 commit intomainfrom
refactor/jit-codegen-submodules

Conversation

@SSL-ACTX
Copy link
Copy Markdown
Owner

Finalized the architectural overhaul of the JIT codegen module by extracting monolithic logic into specialized submodules and decomposing massive functions into maintainable handlers.

  • Modularization:

    • Extracted zero-copy buffer management to buffer.rs.
    • Moved recursive AST evaluation and SIMD math logic to eval.rs and math.rs.
    • Relocated execution orchestration and PyO3 bridging to exec.rs.
    • Reduced mod.rs to a high-level router and glue file.
  • Function Decomposition (De-godding):

    • Refactored gen_expr.rs: Split the 800+ line gen_expr function into specific handlers for variables, binary ops, calls, and various loop types (AnyFor, AllFor, SumFor).
    • Refactored exec.rs: Replaced the monolithic execute_jit_func with a clean strategy-based router. Extracted logic into try_exec_* helpers and centralized vectorized unrolling in execute_views.
  • Performance & Safety Refinements:

    • Centralized shared unrolling logic to eliminate code duplication between profiled and generic vector paths.
    • Moved core JitEntry methods to jit_types.rs for better visibility and dependency flow.
    • Fixed various compilation issues including missing trait imports (Module, AsPyPointer) and silenced unused warnings in the quantum registry.

…line

Finalized the architectural overhaul of the JIT codegen module by extracting
monolithic logic into specialized submodules and decomposing massive
functions into maintainable handlers.

- Modularization:
  - Extracted zero-copy buffer management to `buffer.rs`.
  - Moved recursive AST evaluation and SIMD math logic to `eval.rs` and `math.rs`.
  - Relocated execution orchestration and PyO3 bridging to `exec.rs`.
  - Reduced `mod.rs` to a high-level router and glue file.

- Function Decomposition (De-godding):
  - Refactored `gen_expr.rs`: Split the 800+ line `gen_expr` function into
    specific handlers for variables, binary ops, calls, and various loop types
    (AnyFor, AllFor, SumFor).
  - Refactored `exec.rs`: Replaced the monolithic `execute_jit_func` with a
    clean strategy-based router. Extracted logic into `try_exec_*` helpers and
    centralized vectorized unrolling in `execute_views`.

- Performance & Safety Refinements:
  - Centralized shared unrolling logic to eliminate code duplication between
    profiled and generic vector paths.
  - Moved core `JitEntry` methods to `jit_types.rs` for better visibility and
    dependency flow.
  - Fixed various compilation issues including missing trait imports (`Module`,
    `AsPyPointer`) and silenced unused warnings in the quantum registry.

Co-authored-by: Iris Seravelle <iris.seravelle@gmail.com>
@SSL-ACTX SSL-ACTX self-assigned this Mar 19, 2026
@SSL-ACTX SSL-ACTX added the enhancement New feature or request label Mar 19, 2026
Copilot AI review requested due to automatic review settings March 19, 2026 03:30
@SSL-ACTX SSL-ACTX merged commit 5dad664 into main Mar 19, 2026
4 checks passed
@SSL-ACTX SSL-ACTX deleted the refactor/jit-codegen-submodules branch March 19, 2026 03:31
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes a major refactor of the Python JIT codegen subsystem by splitting previously monolithic logic into focused submodules (buffer handling, math approximations, lowered-eval, and execution) and simplifying codegen/mod.rs into a routing/re-export layer. This aligns the JIT pipeline with a more maintainable architecture while preserving the existing public entry points used by the py::jit layer.

Changes:

  • Introduced new internal codegen submodules (buffer, math, eval, exec) and updated codegen/mod.rs to act as a high-level router and re-export hub.
  • Moved JitEntry methods into jit_types.rs and relocated SIMD/unrolling + execution logic into exec.rs.
  • Refactored expression lowering (gen_expr.rs) into smaller handlers, including dedicated implementations for loop constructs and call handling.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/py/jit/codegen/quantum.rs Formatting and visibility tweaks; adds dead_code allowances around lifecycle/query helpers.
src/py/jit/codegen/mod.rs Converts module into a thin router with submodule declarations and re-exports; retains execute_registered_jit and adds codegen-focused tests.
src/py/jit/codegen/math.rs New module encapsulating SIMD math mode selection and fast trig approximations (scalar + aarch64 pair NEON).
src/py/jit/codegen/jit_types.rs Moves core JitEntry methods here and exposes strategy/unroll/math-mode helpers for execution.
src/py/jit/codegen/jit_module.rs Thread-local JIT module setup and type-profile storage; minor import/formatting adjustments.
src/py/jit/codegen/gen_expr.rs Decomposes the expression lowering pipeline into handlers (var/binop/unary/call/ternary/loops).
src/py/jit/codegen/exec.rs New execution engine for JIT entries (profiles, buffer vectorization, reductions, and shared unrolling logic).
src/py/jit/codegen/eval.rs New lowered-expression evaluator module (scalar + pair-eval helpers) used by vectorized lowered-kernel execution.
src/py/jit/codegen/compiler.rs Pulls cranelift imports into this module and aligns code with the modularized codegen layout.
src/py/jit/codegen/buffer.rs New module extracting zero-copy Python buffer acquisition and typed reads (BufferView, open_typed_buffer, read_buffer_f64).

You can also share your feedback on Copilot code review. Take the survey.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants