Remove dependency of vLLM in hf_policy

[terrykong]

Currently we rely on a very small part of vLLM just to get device uuids. We should own this logic so we don't need the HF worker environment tied to vLLM.

We'd need to decouple if inevitably as we extend inference backends.