adjust input and output texts for test_modeling_recurrent_gemma.py by kaixuanliu · Pull Request #39190 · huggingface/transformers

kaixuanliu · 2025-07-03T08:40:16Z

In original test case, there is no semantic association between input prompt and expected text. This PR changes the input and expected output according to different platforms.

kaixuanliu · 2025-07-03T08:41:22Z

@ydshieh , pls help review, thx!

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

yao-matrix · 2025-07-03T23:27:16Z

+                    "What is Deep learning ?\n\nWhat is Deep Learning\n\nYou will be hearing a lot about Deep learning in the coming time. Today, Deep learning is at the pinnacle and is making history"
+                ],
+                ("xpu", 8): [
+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence (AI) discipline that is rapidly being adopted across industries. The success of Deep"


I suppose you mean ("xpu", 3)

Oops, wrong input. Have updated the code.

yao-matrix · 2025-07-03T23:27:47Z

+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence (AI) discipline that is rapidly being adopted across industries. The success of Deep"
+                ],
+                ("cuda", 7): [
+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence-derived technique allowing machines to perform actions typically performed only by humans. Deep learning"


("cuda", 7) is T4, did you test on T4?

Have updated the code. Thx for reminder~

i suppose you should use ("cuda", None) here

Hi! When previous code doesn't use Expectation and we add it in a PR, let's use (None, None).
This is a bit hidden, but (None, None) is the value that will be used if there is not match (i.e. kind like a default value)

@kaixuanliu Could you change it to (None, None), and add an entry

("cuda", "8"): ['Where is Paris ?\n\nChoose the word or phrase that is closest in meaning to the word in capital letters.\n\nREDEEM\n(A) sort out\n(B) think over\n(C) turn in\n(D) take back\n\nWrite the correct word in the space next to each definition. Use each word only once.\n\nto badly damage\n\nOn the lines provided, rewrite each of the following sentences, correcting the unclear or inexact pronoun references. You may need to add or rearrange words to make the sentences clear and meaningful.\n\n<strong>Example</strong> 1. Nha shook hands with Stephen when he presented the award. $\\underline{\\']

@kaixuanliu , pls update per @ydshieh 's comment here to use (None, None)

@ydshieh , thx for your advice. Have updated the code.

yao-matrix · 2025-07-04T02:47:09Z

+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence (AI) discipline that is rapidly being adopted across industries. The success of Deep"
+                ],
+                ("cuda", 7): [
+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence-derived technique allowing machines to perform actions typically performed only by humans. Deep learning"


i suppose you should use ("cuda", None) here

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

ArthurZucker · 2025-07-07T12:42:52Z

cc @ydshieh

github-actions · 2025-07-07T13:03:15Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: recurrent_gemma

ydshieh · 2025-07-07T13:08:25Z

+                (None, None): [
+                    "What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence (AI) discipline that is rapidly being adopted across industries. The success of Deep"


don't know exactly on which hardware this value is obtained. But it's ok.

ydshieh · 2025-07-07T13:08:47Z

    @require_read_token
    def test_model_2b_8bit(self):
-        EXPECTED_TEXTS = ['Hello I am doing a project on the topic of "The impact of the internet on the society" and I am looking', "Hi today I'm going to show you how to make a simple and easy to make a simple and easy"]  # fmt: skip
+        EXPECTED_TEXTS = ['Hello I am doing a project on the topic of "The impact of social media on the society" and I am looking', "Hi today I'm going to show you how to make a simple and easy to make a 3D"]  # fmt: skip


i updated this too so this model will pass all integration tests.

ydshieh

Thank you

ydshieh · 2025-07-07T13:12:57Z

Merge now. If there is anything to update for XPU, we can do it in a separate PR

…uggingface#39190) * adjust input and output texts for test_modeling_recurrent_gemma.py Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com> * fix bug Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com> * adjust Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com> * update Expectation match Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com> * fix --------- Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com> Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

adjust input and output texts for test_modeling_recurrent_gemma.py

6819a61

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

yao-matrix suggested changes Jul 3, 2025

View reviewed changes

yao-matrix suggested changes Jul 4, 2025

View reviewed changes

kaixuanliu requested a review from yao-matrix July 4, 2025 04:52

kaixuanliu added 4 commits July 4, 2025 06:18

fix bug

1fc1ffc

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

adjust

0fc452b

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

update Expectation match

db4f8f5

Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>

Merge branch 'main' into recurrent_gemma-CI

7416e0e

ArthurZucker requested a review from ydshieh July 7, 2025 12:42

fix

29f9516

ydshieh reviewed Jul 7, 2025

View reviewed changes

ydshieh approved these changes Jul 7, 2025

View reviewed changes

ydshieh merged commit c4e39ee into huggingface:main Jul 7, 2025
17 checks passed

kaixuanliu deleted the recurrent_gemma-CI branch April 13, 2026 02:41

		(None, None): [
		"What is Deep learning ?\n\nDeep learning is the next frontier in computer vision. It is an Artificial Intelligence (AI) discipline that is rapidly being adopted across industries. The success of Deep"

Conversation

kaixuanliu commented Jul 3, 2025

Uh oh!

kaixuanliu commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yao-matrix Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArthurZucker commented Jul 7, 2025

Uh oh!

github-actions Bot commented Jul 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ydshieh left a comment

Choose a reason for hiding this comment

Uh oh!

ydshieh commented Jul 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kaixuanliu commented Jul 3, 2025 •

edited

Loading

yao-matrix Jul 3, 2025 •

edited

Loading