Skip to content

nvtx markers for async llm engine #922

@terrykong

Description

@terrykong

currently only the sync vllm worker has nvtx markers. this issue tracks adding this to the async llm engine for parity

Metadata

Metadata

Labels

PerformanceRelated to improving performance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions