docs: update cuDNN sliding window attention support

[sbhavani]

Description

Update documentation to reflect that cuDNN now supports causal sliding window attention (SWA) starting from version 9.2+.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

Please list the changes introduced in this PR:

Changes:

• Updated backend support matrix table to show cuDNN supports SWA (cuDNN 9.2+, causal masks only)
• Added SWA comparison between flash-attention and cuDNN in section 1.3
• Added clarifying note in cp_ag_thd_dpa_jax_deep_dive.ipynb that cuDNN supports SWA but not all striping patterns for context parallelism

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[greptile-apps]

Greptile Summary

Documentation-only PR that updates the attention documentation to reflect cuDNN's sliding window attention (SWA) support starting from version 9.2+.

• Updated the backend support matrix table in attention.ipynb to show cuDNN supports SWA with the qualifier "cuDNN 9.2+, causal masks only" (previously listed as "No")
• Added a new bullet point in Section 1.3 comparing SWA support between flash-attention (full support) and cuDNN (causal-only, no dropout, no bias)
• Added a clarifying note in cp_ag_thd_dpa_jax_deep_dive.ipynb explaining that cuDNN supports SWA but not all striping patterns for context parallelism

All documented constraints (dropout=0.0, bias_type="no_bias", causal masks, cuDNN 9.2+ version requirement) are verified against the actual validation logic in fused_attn.cpp and utils.py.

Confidence Score: 5/5

• This PR is safe to merge — it contains only documentation updates with no code changes.
• All changes are documentation-only (Jupyter notebook markdown cells). The documented constraints (cuDNN 9.2+, causal masks, dropout=0.0, no_bias) are verified against the actual C++ and Python validation logic in the codebase. No code behavior is affected.
• No files require special attention.

Important Files Changed

Filename	Overview
docs/examples/attention/attention.ipynb	Updated support matrix table (SWA column from "No" to "Yes (cuDNN 9.2+, causal masks only)") and added SWA comparison bullet in Section 1.3. All claims verified against code.
docs/examples/attention/cp_ag_thd_dpa_jax_deep_dive.ipynb	Added a clarifying note that cuDNN supports SWA (9.2+, causal masks) but not all striping patterns for context parallelism. Accurate and helpful context.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SWA Request] --> B{cuDNN Version?}
    B -->|< 9.2| C[SWA Not Supported\nFallback to flash-attention]
    B -->|>= 9.2| D{Mask Type?}
    D -->|causal / causal_bottom_right| E{dropout == 0.0?}
    D -->|non-causal| C
    E -->|Yes| F{bias_type == no_bias?}
    E -->|No| C
    F -->|Yes| G[cuDNN SWA Enabled]
    F -->|No| C

Loading

_{Last reviewed commit: 12be7aa}

[greptile-apps]

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[pggPL]

LGTM