Skip to content

Fix: Add **kwargs to QEffLlama4VisionModel.forward() for transformers v4.57.3 compatibility#933

Merged
quic-rishinr merged 1 commit intoquic:mainfrom
sudheepm-wq:fix/llama4-vision-kwargs-compatibility
Apr 24, 2026
Merged

Fix: Add **kwargs to QEffLlama4VisionModel.forward() for transformers v4.57.3 compatibility#933
quic-rishinr merged 1 commit intoquic:mainfrom
sudheepm-wq:fix/llama4-vision-kwargs-compatibility

Conversation

@sudheepm-wq
Copy link
Copy Markdown
Contributor

Problem

After the transformers v4.57.3 rebase (commit 7acb860), the following error occurs:
TypeError: QEffLlama4VisionModel.forward() got an unexpected keyword argument 'vision_feature_layer'

Root Cause Analysis

  1. In transformers v4.57.3, Llama4ForConditionalGeneration.get_image_features()
    now passes additional parameters (vision_feature_layer and vision_feature_select_strategy)
    to the vision model's forward method via **kwargs.

  2. The call chain:

    • QEffLlama4EncoderWrapper.forward() calls self.model.get_image_features()
    • get_image_features() (from transformers library) calls self.vision_model(**kwargs)
    • QEffLlama4VisionModel.forward() was not accepting **kwargs
  3. Since QEffLlama4VisionModel overrides the forward() method but didn't accept **kwargs,
    it raised a TypeError when these unexpected arguments were passed.

Solution

Added **kwargs parameter to QEffLlama4VisionModel.forward() method signature.
This allows the method to accept vision_feature_layer and vision_feature_select_strategy
parameters from the parent class's get_image_features() method, even though they're
not used in the QEff implementation.

Impact

  • Backward compatible change
  • Fixes ONNX export failures for Llama4 vision models
  • Maintains compatibility with transformers v4.57.3 API

Tested with: examples/image_text_to_text/models/llama4/single_image.py

@sudheepm-wq sudheepm-wq force-pushed the fix/llama4-vision-kwargs-compatibility branch from 1d04918 to ef43dab Compare April 21, 2026 14:54
@vbaddi
Copy link
Copy Markdown
Contributor

vbaddi commented Apr 21, 2026

@quic-rishinr Check for CI and merge pls, thanks :)

@sudheepm-wq
Copy link
Copy Markdown
Contributor Author

@quic-dhirajku any update on the review of this patch?

… v4.57.3 compatibility

## Problem
After the transformers v4.57.3 rebase (commit 7acb860), the following error occurs:
TypeError: QEffLlama4VisionModel.forward() got an unexpected keyword argument 'vision_feature_layer'

## Root Cause Analysis
1. In transformers v4.57.3, Llama4ForConditionalGeneration.get_image_features()
   now passes additional parameters (vision_feature_layer and vision_feature_select_strategy)
   to the vision model's forward method via **kwargs.

2. The call chain:
   - QEffLlama4EncoderWrapper.forward() calls self.model.get_image_features()
   - get_image_features() (from transformers library) calls self.vision_model(**kwargs)
   - QEffLlama4VisionModel.forward() was not accepting **kwargs

3. Since QEffLlama4VisionModel overrides the forward() method but didn't accept **kwargs,
   it raised a TypeError when these unexpected arguments were passed.

## Solution
Added **kwargs parameter to QEffLlama4VisionModel.forward() method signature.
This allows the method to accept vision_feature_layer and vision_feature_select_strategy
parameters from the parent class's get_image_features() method, even though they're
not used in the QEff implementation.

## Impact
- Backward compatible change
- Fixes ONNX export failures for Llama4 vision models
- Maintains compatibility with transformers v4.57.3 API

Tested with: examples/image_text_to_text/models/llama4/single_image.py

Signed-off-by: sudheepm <sudheepm@qti.qualcomm.com>
@quic-rishinr quic-rishinr force-pushed the fix/llama4-vision-kwargs-compatibility branch from ef43dab to 04c9f57 Compare April 24, 2026 05:36
@quic-rishinr quic-rishinr merged commit 4940d3a into quic:main Apr 24, 2026
5 checks passed
@sudheepm-wq sudheepm-wq deleted the fix/llama4-vision-kwargs-compatibility branch April 24, 2026 06:58
quic-akuruvil pushed a commit that referenced this pull request Apr 27, 2026
… v4.57.3 compatibility (#933)

## Problem
After the transformers v4.57.3 rebase (commit 7acb860), the following
error occurs:
TypeError: QEffLlama4VisionModel.forward() got an unexpected keyword
argument 'vision_feature_layer'

## Root Cause Analysis
1. In transformers v4.57.3,
Llama4ForConditionalGeneration.get_image_features()
now passes additional parameters (vision_feature_layer and
vision_feature_select_strategy)
   to the vision model's forward method via **kwargs.

2. The call chain:
- QEffLlama4EncoderWrapper.forward() calls
self.model.get_image_features()
- get_image_features() (from transformers library) calls
self.vision_model(**kwargs)
   - QEffLlama4VisionModel.forward() was not accepting **kwargs

3. Since QEffLlama4VisionModel overrides the forward() method but didn't
accept **kwargs,
   it raised a TypeError when these unexpected arguments were passed.

## Solution
Added **kwargs parameter to QEffLlama4VisionModel.forward() method
signature.
This allows the method to accept vision_feature_layer and
vision_feature_select_strategy
parameters from the parent class's get_image_features() method, even
though they're
not used in the QEff implementation.

## Impact
- Backward compatible change
- Fixes ONNX export failures for Llama4 vision models
- Maintains compatibility with transformers v4.57.3 API

Tested with: examples/image_text_to_text/models/llama4/single_image.py

Signed-off-by: sudheepm <sudheepm@qti.qualcomm.com>
Co-authored-by: sudheepm <sudheepm@qti.qualcomm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants