TextIteratorStreamer & generate graceful interruption

[paulcjh]

What does this PR do?

This PR adds in functionality to stop the generate function for LLMs when used in conjunction with a streamer. The TextIteratorStreamer typically runs in a separate thread , per the example provided in its docstring, and if you need to terminate the generation there's currently no way to do this. This makes operating the models in an API environment challenging when things such as client disconnects occur.

note: the file was also formatted with black, happy to remove and just keep the new interruption changes

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante

[dmarx]

FYI: it looks like you're including a lot of changes to generation/utils.py that are just formatting changes. I'm guessing this sort of thing is discouraged since it makes it difficult to identify the substantive changes you are proposing, and also is likely to cause merge conflicts for others. Concretely: I was curious to see what changes you had made in utils.py, but between the formatting changes and collapsing all of your work into a single commit, your substantive changes in that file are a needle in a haystack.

[paulcjh]

@dmarx just removed all of the cleanup - the changes in utils.py are very light

[paulcjh]

@gante do you have any thoughts on this feature?

[gante]

Hi @paulcjh 👋 Thank you for opening the PR!

We're planning to rework streaming soon, so I'm avoiding adding short-lived changes. Have a look at the conversation with @dmarx on this PR, I think it covers the reasons to not add changes and the roadmap for streaming 🤗

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.