Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions src/transformers/modeling_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1946,8 +1946,25 @@ def get_correct_experts_implementation(self, requested_experts: str | None) -> s

@classmethod
def _can_set_attn_implementation(cls) -> bool:
"""Detect whether the class supports setting its attention implementation dynamically. It is an ugly check based on
opening the file, but avoids maintaining yet another property flag.
"""Detect whether the class supports setting its attention implementation dynamically.

This is a heuristic based on reading the module's source file as text and searching for
specific string patterns. It avoids maintaining a separate property flag on every model class,
but has the following known limitations:

- **False positives from comments or string literals**: The searched patterns
(``eager_attention_forward``, ``ALL_ATTENTION_FUNCTIONS.get_interface(``) may appear
inside docstrings, comments, or string constants rather than actual control-flow code,
causing this method to return ``True`` even when the model does not properly implement
the dynamic-attention interface.
- **Compiled modules**: If the model class is defined in a compiled extension (``.so`` /
``.pyd``), the source file is not readable as text. This case is already handled: the
``__file__`` guard returns ``False`` before attempting to open the file.
- **Non-standard import environments**: Environments such as PyInstaller bundles,
``zipimport``, or custom module loaders may not expose a readable ``__file__`` path.
These also fall through the ``__file__`` guard and return ``False`` safely.
- **Jupyter notebooks and REPLs**: Models defined interactively have no associated source
file. This is also caught by the ``__file__`` guard.
"""
class_module = sys.modules[cls.__module__]
# This can happen for a custom model in a jupyter notebook or repl for example - simply do not allow to set it then
Expand All @@ -1965,8 +1982,24 @@ def _can_set_attn_implementation(cls) -> bool:

@classmethod
def _can_set_experts_implementation(cls) -> bool:
"""Detect whether the class supports setting its experts implementation dynamically. It is an ugly check based on
opening the file, but avoids maintaining yet another property flag.
"""Detect whether the class supports setting its experts implementation dynamically.

This is a heuristic based on reading the module's source file as text and searching for
the ``@use_experts_implementation`` decorator string. It avoids maintaining a separate
property flag on every model class, but has the following known limitations:

- **False positives from comments or string literals**: The searched pattern
(``@use_experts_implementation``) may appear inside docstrings, comments, or string
constants rather than actual decorator usage, causing this method to return ``True``
even when the decorator is not applied to any function.
- **Compiled modules**: If the model class is defined in a compiled extension (``.so`` /
``.pyd``), the source file is not readable as text. This case is already handled: the
``__file__`` guard returns ``False`` before attempting to open the file.
- **Non-standard import environments**: Environments such as PyInstaller bundles,
``zipimport``, or custom module loaders may not expose a readable ``__file__`` path.
These also fall through the ``__file__`` guard and return ``False`` safely.
- **Jupyter notebooks and REPLs**: Models defined interactively have no associated source
file. This is also caught by the ``__file__`` guard.
"""
class_module = sys.modules[cls.__module__]
# This can happen for a custom model in a jupyter notebook or repl for example - simply do not allow to set it then
Expand Down
Loading