mtmd: fix --no-warmup

[ngxson]

Fix #17676

[ggerganov]

I find it a bit confusing though. Does it mean if you warmup with a small image, then using a larger image would fail?

[ngxson]

I find it a bit confusing though. Does it mean if you warmup with a small image, then using a larger image would fail?

Indeed, the bug was because buf_compute_meta was never allocated if warmup is never called, so it still need to be called at least once.

If the user use a bigger image later on, ggml_backend_sched_alloc_graph inside clip_image_batch_encode will be responsible for allocating a bigger device buffer.

---

There is also an alternative to fix this issue: to allocate buf_compute_meta upon initializing clip_ctx. But since warmup also does other stuff, like deciding the status of flash attn, log unsupported ops, it should still be called at least once.

Note sure if you have any other ideas?