vLLM native OpenAI compatible server with weight syncing

[BjarniHaukur]

This PR adds a new vllm_serve_async.py script to verifiers. It adds:

• a fully featured OpenAI compatible endpoint
• that mirrors the weight syncing logic from vllm_serve.py
• while delegating endpoint complexity to vllm.entrypoints.openai.api_server

[CLAassistant]

All committers have signed the CLA.

[BjarniHaukur]

I commented out the data parallel tests. I'm baffled as to why they don't work here. I have an open PR to trl where they pass without fail (trl pr)

If this is something you want to merge, I can take another look.

[willccbb]

i ended up implementing my own dynamic batching server which builds around LLM() but exposes an async endpoint, i never found AsyncLLM to be reliable for weight-syncing in training runs and it seems like the general guidance is that isn't fully supported without a custom build config/container (e.g. how it's handled in veRL and related libraries)

at some point we'll migrate to AsyncLLM but the current solution works well for now (perf tests are fairly close in my measurements) and offers a bit more control for error handling.

[lewtun]

i never found AsyncLLM to be reliable for weight-syncing in training runs

Out of curiosity, what kind of errors did you hit with AsyncLLM?