Skip to content

LLfp4 qr cap for atom #1673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
azaidy merged 4 commits into from
Dec 19, 2025
Merged

LLfp4 qr cap for atom #1673

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cf53506
QR cap implemented to limit QR to prefill
amirumoAMD Dec 16, 2025
b1d691e
test git config
amirumoAMD Dec 17, 2025
a12324c
Fix to genericize qr comm cap
amirumoAMD Dec 17, 2025
b1aca3b
Incorrect cap number
amirumoAMD Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions aiter/dist/device_communicators/communicator_cuda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ def all_reduce(
qr_comm is not None
and not qr_comm.disabled
and qr_comm.should_quick_allreduce(input_)
and (input_.nelement() * input_.element_size()) >= 4*1024*1024 # input shape should be such that quick reduce will show benefits.
# input shape estimated at 2 * max concurrency for now. if performance issues, subject to change
):
out = qr_comm.quick_all_reduce(input_)
assert out is not None
Expand Down