Skip to content

fix tensor collect#154

Merged
Yunnglin merged 3 commits intomainfrom
fix_tensor_collect
Apr 13, 2026
Merged

fix tensor collect#154
Yunnglin merged 3 commits intomainfrom
fix_tensor_collect

Conversation

@Yunnglin
Copy link
Copy Markdown
Collaborator

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

Write the detail information belongs to this PR.

Experiment results

Paste your experiment result here(if needed).

Copilot AI review requested due to automatic review settings April 13, 2026 09:24
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the tinker dependency to version 0.16.1 and refactors the model output processing logic in common.py to better handle pipeline parallel stages. Specifically, _tinker_build_output now correctly identifies and skips stages that produce no outputs, and _normalize_tensor_output has been updated to handle empty lists and tensor concatenation. Feedback was provided to improve memory efficiency by moving individual tensors to the CPU before concatenation to prevent potential GPU OOM issues.

Comment thread src/twinkle/server/model/backends/common.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes tensor collection/normalization for Tinker-compatible server backends (especially when some pipeline stages produce no logits/logps), and updates the Docker image to use a newer tinker version.

Changes:

  • Normalize logits/logps outputs in a single path and treat empty lists as “missing output” to avoid downstream collection issues.
  • Return an empty per-datum output list when neither logits nor logps are produced (e.g., non-last PP stage).
  • Bump tinker in the Dockerfile from 0.14.0 to 0.16.1.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/twinkle/server/model/backends/common.py Adjusts output extraction/normalization to handle None and empty-list outputs cleanly for distributed/pipeline scenarios.
Dockerfile Updates the container dependency pin for tinker to a newer version.

Comment thread src/twinkle/server/model/backends/common.py
Comment thread Dockerfile
@Yunnglin Yunnglin merged commit 1c71781 into main Apr 13, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants