Clip model from LLaVA is not using GPU, but LLaVA model is using GPU in the same piece of code.

[dcostea]

Congratulations for adding LLaVA support starting with version 0.11.1.
It works very well!

Still, clip model is very slow.
I have Cuda12 and llava-mistral model is using GPU, as expected, but the clip model is not, instead is using the CPU.

NativeLibraryConfig.Instance
.WithCuda(true)
.WithLogs(true)
.WithAvx(NativeLibraryConfig.AvxLevel.Avx512);

Should the clip model run on GPU as well?

[SanftMonster]

Thank you for reporting us for this problem! It's the same with #639, which is supposed to be fixed by #647. We will publish a patch release 0.11.2 soon.

[SanftMonster]

v0.11.2 has been released in #651, please take a try to see if this issue has been fixed. Thank you!

[dcostea]

wow, look here, less than half of a second!

isn't it beautiful?

Thank you!

[dcostea]

issue is fixed!