Skip to content

[Mirror] server: fix memory reservations in populate_token_probs#81

Open
ngxson wants to merge 1 commit intongxson:masterfrom
l-austenfeld:master
Open

[Mirror] server: fix memory reservations in populate_token_probs#81
ngxson wants to merge 1 commit intongxson:masterfrom
l-austenfeld:master

Conversation

@ngxson
Copy link
Owner

@ngxson ngxson commented Jan 19, 2026

Mirror from upstream PR: ggml-org#18787

Note: @coderabbitai use my 'Mirror PR' preset for reviewing this.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed an issue where returned top probabilities could exceed the requested amount. The system now consistently respects both the requested number of probabilities and actual available candidates, ensuring accurate results.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Jan 19, 2026

📝 Walkthrough

Walkthrough

This change introduces an explicit alias for requested top-n probabilities and ensures the returned number of probabilities is capped by both the requested amount and available candidates in both post_sampling and non-post_sampling code paths, with updated loops and reserves accordingly.

Changes

Cohort / File(s) Summary
Probability capping refinement
tools/server/server-context.cpp
Introduced n_probs_request alias for requested probabilities and implemented consistent capping logic across both post_sampling and non-post_sampling branches using min(max_probs, n_probs_request) to limit returned top-n probabilities

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 With probabilities now capped just right,
The top-n values shine so bright!
No overflow, no excess here,
The balance perfect, crystal clear!

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: fixing memory reservations in the populate_token_probs function in the server code.
Description check ✅ Passed The description is minimal but sufficient for a mirror PR, providing the upstream source link and review instructions; however, it lacks detail about what the fix accomplishes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants