Skip to content

Fix torch only support for fast Processors#42824

Open
Blaizzy wants to merge 20 commits intohuggingface:mainfrom
Blaizzy:pc/fix-torch-only-support
Open

Fix torch only support for fast Processors#42824
Blaizzy wants to merge 20 commits intohuggingface:mainfrom
Blaizzy:pc/fix-torch-only-support

Conversation

@Blaizzy
Copy link
Copy Markdown

@Blaizzy Blaizzy commented Dec 11, 2025

What does this PR do?

Adds MLX tensor support to BatchFeature and removes the PyTorch-only restriction in fast image processing utilities.

Changes:

  • Added is_mlx_array export to transformers/utils/__init__.py
  • Implemented MLX tensor conversion logic in BatchFeature.convert_to_tensors() (feature_extraction_utils.py)
  • Removed the hardcoded validation that restricted fast image processors to PyTorch tensors only (image_processing_utils_fast.py)

This enables using return_tensors="mlx" with vision models and fast image processors on Apple Silicon, which previously raised:

ValueError: Only returning PyTorch tensors is currently supported.

Before submitting

Who can review?

@Blaizzy Blaizzy changed the title Fix torch only support Fix torch only support for Processors Dec 11, 2025
@Blaizzy Blaizzy changed the title Fix torch only support for Processors Fix torch only support for fast Processors Dec 11, 2025
@Rocketknight1
Copy link
Copy Markdown
Member

cc @yonigozlan @molbap but not sure who to tag for MLX!

@Blaizzy
Copy link
Copy Markdown
Author

Blaizzy commented Dec 12, 2025

@ArthurZucker has some context as well

Copy link
Copy Markdown
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @yonigozlan is it fine for you as well?

Copy link
Copy Markdown
Member

@yonigozlan yonigozlan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Blaizzy ! Sorry for the delay. I'm ok with merging this if it can improve compatibility with mlx! We don't have any mlx image processor in the library for now, but I'm guessing this is to use with custom image processors?
As discussed with Arthur, we'll move towards having different backends for image processors, so we could add one for mlx. However image processors are quite different across models, so we would need to have custom ones for each backend and models, as making image processing operation not library dependent seems difficult. Anyway these are issues for another PR ;)

Comment thread src/transformers/feature_extraction_utils.py Outdated


class Pop2PianoFeatureExtractionTester:
def __init__(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if these modifications are related to this PR or if they were added here by mistake?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were a mistake 😅. I was following the formatter feedback.

@Blaizzy
Copy link
Copy Markdown
Author

Blaizzy commented Jan 13, 2026

We don't have any mlx image processor in the library for now, but I'm guessing this is to use with custom image processors

Thank you very much @yonigozlan! Yes, that will be the path I'll take in the meantime. Because for majority of models they can return mlx tensors from transformers but a few use torch and torch vision explicitly, that forces torch as dep.

Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Comment on lines +163 to +166
if isinstance(value, np.ndarray):
return mx.array(value)
else:
return mx.array(value)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if isinstance(value, np.ndarray):
return mx.array(value)
else:
return mx.array(value)
return mx.array(value)

@github-actions
Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: pop2piano

Comment thread src/transformers/feature_extraction_utils.py Outdated
@github-actions
Copy link
Copy Markdown
Contributor

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42824&sha=4adaad

@yonigozlan yonigozlan enabled auto-merge (squash) February 11, 2026 21:03
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yonigozlan yonigozlan disabled auto-merge February 11, 2026 22:15
@yonigozlan
Copy link
Copy Markdown
Member

Hey @Blaizzy ! I fixed some tiny formatting errors and moved the test to test_feature_extraction_utils.py‎. It would be great to get this merged soon as we're making progress on the PR to introduce image processing backends ;) #43514

However, I can't use mlx on my machine, and we don't have mlx support in our CI. Would you mind running the test (RUN_SLOW=1 python3 -m pytest tests/utils/test_feature_extraction_utils.py -k test_batch_feature_mlx_conversion) to make sure it passes?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants