Skip to content

server : tuning tests#7388

Merged
ggerganov merged 4 commits intomasterfrom
gg/server-tests
May 20, 2024
Merged

server : tuning tests#7388
ggerganov merged 4 commits intomasterfrom
gg/server-tests

Conversation

@ggerganov
Copy link
Copy Markdown
Member

@ggerganov ggerganov commented May 19, 2024

  • Fix incorrect passing of the temperature as the string "0.8f"
  • Increase timeout to make the thread sanitizer workflow pass
  • Pass temperature explicitly where it matters

@github-actions github-actions Bot added examples python python script changes server labels May 19, 2024
Copy link
Copy Markdown
Contributor

@JohannesGaessler JohannesGaessler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore my previous comment, I had mixed up old and new.

@mofosyne mofosyne added refactoring Refactoring Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix labels May 20, 2024
@ggerganov ggerganov marked this pull request as ready for review May 20, 2024 07:16
@ggerganov ggerganov merged commit 1cc0155 into master May 20, 2024
@ggerganov ggerganov deleted the gg/server-tests branch May 20, 2024 07:16

Scenario Outline: consistent results with same seed
Given <n_slots> slots
And 0.0 temperature
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be set to temperature 1, not temperature 0. With temperature 0 you get greedy decoding in which case the seed is irrelevant. So this test can no longer detect issues with e.g. the RNG state being shared between slots.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, will push a fix now

Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* server : don't pass temperature as string

* server : increase timeout

* tests : fix the fix 0.8f -> 0.8

ggml-ci

* tests : set explicit temperature
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
* server : don't pass temperature as string

* server : increase timeout

* tests : fix the fix 0.8f -> 0.8

ggml-ci

* tests : set explicit temperature
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples python python script changes refactoring Refactoring Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants