-
Notifications
You must be signed in to change notification settings - Fork 23
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Description
Currently Eagle3DraftModel has t2d and dt2 (vocab mapping) tensors as required inputs to it's init function. This means if a model uses the full verifier vocab, users must create "no-op" mapping tensors and pass them to the drafter. Furthermore, these mapping tensors will still be saved in the model checkpoints during training and then subsequently used by vllm when running the model.
It would be better if we made these optional, and used the full verifier vocab when they aren't provided.
Todo:
- Make
t2danddt2tensors optional inputs toEagle3DraftModel - Verify that if one is provided, the other is as well
- Make sure all existing uses of the tensors in
Eagle3DraftModelcan handle non-existent/Nonevalues - Update
scripts/train.pyso thatt2dandd2tare not required inputs - Update
scripts/gen_and_train.pyif needed.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers