nan values appearing in Style Scores when using UnifiedReward-2.0-qwen3vl-32b

[Alisaleelee]

hello, thanks for your nice work.

I am encountering an issue where the model frequently outputs nan (Not a Number) for specific score metrics—specifically the Style Score—while other metrics like Alignment and Coherence function normally.

I am using the following model: CodeGoat24/UnifiedReward-2.0-qwen3vl-32b.

Alignment Score:
Image 1: 0.5134999752044678
Image 2: 0.48649999499320984

Coherence Score:
Image 1: 0.578499972820282
Image 2: 0.42149999737739563

Style Score:
Image 1: nan
Image 2: nan

[CodeGoat24]

Hi~

Could you share the inference script (or a minimal inference example) for reproduction?

[Alisaleelee]

Hi~

Could you share the inference script (or a minimal inference example) for reproduction?

These two pairs are flux-generated data.
I use your original code, just update the image path.
https://github.com/CodeGoat24/UnifiedReward/blob/main/inference_qwen/UnifiedReward-2.0-inference/pair_score_image_generation.py#L29

I am considering switching to CodeGoat24/UnifiedReward-Think-qwen3vl-32b. Given its Chain-of-Thought (CoT) approach, would this model be more reliable than the standard version for scoring?

Additionally, I am looking for a robust Reward Model to be used in the post-training phase of an image generation pipeline. Could you provide any recommendations or best practices for selecting a model for this specific use case?

Thx.

[CodeGoat24]

Thanks for the feedback!

We recommend employing UnifiedReward-Flex-based Pref-GRPO.