fix num_assistant_tokens with heuristic schedule

[jmamou]

What does this PR do?

We have defined 2 different num_assistant_tokens_schedule values:

• heuristic: When all speculative tokens are correct, increase num_assistant_tokens by 2 else reduce by 1. num_assistant_tokens value is persistent over multiple generation calls with the same assistant model.
• heuristic_transient: Same as "heuristic but num_assistant_tokens is reset to its initial value after each generation call.

Fixes # (issue)
#27979 (comment)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@gante @amyeroberts

[gante]

Thank you for the fix and the tests 💛

[ArthurZucker]

Thanks for the PR!

[amyeroberts]

Thanks for adding this!

For the code quality checks, you'll need to run make fixup and push the changes.

From the docstring, it mentions behaviour about `"heuristic_transient" being reset, but I don't see logic relating to it in this diff. Does this already happen?

[jmamou]

Thanks for adding this!

For the code quality checks, you'll need to run make fixup and push the changes.

From the docstring, it mentions behaviour about `"heuristic_transient" being reset, but I don't see logic relating to it in this diff. Does this already happen?

I just fixed docstring

[amyeroberts]

@jmamou Let's try and make the CI green :) You'll need to resolve the quality checks by running make fixup and pushing the changes.

For the other failing tests, you'll need to try rebasing on main.

[jmamou]

@amyeroberts
tests_torch failed.
Are you familiar with the error?

[amyeroberts]

@jmamou Tbh, I'm not sure what the cause of these failures are. I would first suggest rebasing on main to make sure you have all of the most recent commits. This will trigger a re-run of the CI too.

[jmamou]

@amyeroberts
same test still fails 👎

[gante]

@jmamou Our apologies, this PR fixed it. Could you try rebasing again please? 🤗

[gante]

@amyeroberts both CI failures seem unrelated to this PR, including check_repository_consistency 👀 do you have an idea of what might be causing it?

[amyeroberts]

@jmamou Apologies for all of the current issues you've been experiencing with unrelated failures on this PR.

The two current batches of failing tests should have been resolved with #29037, #29043

Could you try one (final 🤞) rebase to get this CI green 🟢 ?

[jmamou]

@amyeroberts
tests passed on CI 👍

[amyeroberts]

@jmamou Thanks for this contribution and your patience with our misbehaving CI!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.