Refactor: separate sync/async vllm

[yuki-97]

For now, vllm.py is a bit complex with both sync/async code in it.

It's better to separate them to improve the experience of user and developer.

step1:
after doing this, we can aware the code changes of vllm at main branch and not miss them when doing step2.

1. put vllm stuffs from nemo_rl/models/generation to nemo_rl/models/generation/vllm, so that it's easy for us to support other inference FW in the future.
2. split sync and async vllm worker to different files to make it clear.

step2:

1. tidy and remove duplicated / useless code.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[terrykong]

@yuki-97 is this issue resolved?

[yuki-97]

Not completely, only step1 is finished through #900.

For step2, I plan to do these things after the async-rl PR merged and after v0.4.

1. Change xxx_async function name to xxx in VllmAsyncGenerationWorker, so the function name will be the same as VllmGenerationWorker, so that we can remove lots of "Choose the appropriate method based on async_engine setting" in VllmGeneration.
2. Remove measuring batch logic in VllmAsyncGenerationWorker:generate since async engine only receive single request each time. But need to see how async-rl PR use it and make sure the refactor won't break it.
3. Move patches and some codes that not so related to vLLM to a proper place to make the code clear.

[euronymous-aithal]

@yuki-97 what is the latest status on this ?

[yuki-97]

@yuki-97 what is the latest status on this ?

@euronymous-aithal I'd like to discuss with folks during next week meeting, and will do it after v0.4.