[AMD] Use vLLM framework for DeepSeek R1 on MI325 and MI355 hardware#104
Closed
omirosh wants to merge 6 commits intoSemiAnalysisAI:mainfrom
Closed
[AMD] Use vLLM framework for DeepSeek R1 on MI325 and MI355 hardware#104omirosh wants to merge 6 commits intoSemiAnalysisAI:mainfrom
omirosh wants to merge 6 commits intoSemiAnalysisAI:mainfrom
Conversation
Contributor
|
@cquil11 , @functionstackx, I don't have the permission to assign a reviewer, so just tagging you both :). |
Contributor
|
@qcolombet yes, we are looking into it @cquil11 is just trying to land an massive refactor PR first to reduce tech debt and then we can look into this one |
Contributor
Contributor
|
@omirosh closing as this is stale PR. happy to take a look at amd deepseek vllm in addition to amd's sglang deepseek configs if single node is an community vllm image that supports fp4 and fp8 |
cquil11
changed the title
cquil11
added a commit
that referenced
this pull request
Apr 28, 2026
…ression Upstream commit 52e697d (#108 "fix(nginx): raise file descriptor limit for nginx workers") prepends `ulimit -n 1048576 &&` to the nginx srun command. On clusters whose container inherits a sub-1M RLIMIT_NOFILE hard limit from slurmd/PAM, the bash builtin's setrlimit fails with EPERM (raising the hard rlimit needs CAP_SYS_RESOURCE in the init user namespace, which pyxis --container-remap-root does not grant). The `&&` short-circuits and nginx never starts — caught when re-running dsr1-fp4-gb200-dynamo-sglang. Pin back to 698590e ("feat(config): cluster-wide default_bash_preamble for ulimits and the like (#104)"), the immediately prior commit, where nginx runs without the chained ulimit. Bump forward once upstream softens the ulimit to `|| true` or makes it opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please consider changing framework for DeepSeek R1 to vLLM, it shows better performance over SGLang.
Here is also documentation for running DeepSeek with vLLM.