Why the output of siglip+resampler is different between torch and llama.cpp？

[Xwmiss]

Hi, thanks for your work firstly.
while I find that I use the same picture as input of torch_model and llama.cpp_model, the res_(1, 96, 4096) of siglip+resampler part is different, the Cosine Similarity is only 0.75.
I am confused now.

[Xwmiss]

Even I change the input_raw in the function "encode_image_with_clip" to make the input for torch and llama.cpp is must same, but the result of these two inference way is still different and the Cosine Similarity is still 0.75.

[Xwmiss]

And the another thing is need to say, while the result of siglip+resampler is different, but the final llm result for the image and question is right, which make me confused more....

[tc-mb]

@ALL
Hi, I don't always pay attention to the issue area in this fork code repo.

1. If this issue still needs to be answered, please raise an issue in the main repo with "llamacpp" label. I will respond very quickly.
2. If this issue no longer needs to be answered, I will close it this week.