Can you support streaming when generating?

[wac81]

Can you support streaming when generating?
Thanks

[wac81]

Is it a structural limitation of this model?

[syncdoth]

Streaming is out of scope for this project for me; my goal was to make HF conpatible implementation. You may find any streaming implementation on huggingface models and it should be compatible with this!

Also, anyone is free to implelement it and send a PR, it should be pretty easy to do.