fix: spec compliant instantiation and termination of streams

[Lederstrumpf]

Description

This PR fixes #340.

With reference to the example response from localAI from #340 (under "To Reproduce"), here's the corresponding updated response after this PR's changes:

HTTP/1.1 200 OK
Date: Sun, 21 May 2023 10:56:07 GMT
Content-Type: text/event-stream
Vary: Origin
Access-Control-Allow-Origin: *
Cache-Control: no-cache
Connection: keep-alive
Transfer-Encoding: chunked

data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"role":"assistant"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":"1"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":" 2"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":" 3"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: {"model":"ggml-gpt4all-j","choices":[{"finish_reason":"stop"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

data: [DONE]

Diff

diff -u old-response new-response
--- old-response	2023-05-21 14:36:35
+++ new-response	2023-05-21 14:36:35
@@ -1,5 +1,5 @@
 HTTP/1.1 200 OK
-Date: Sun, 21 May 2023 11:01:33 GMT
+Date: Sun, 21 May 2023 10:56:07 GMT
 Content-Type: text/event-stream
 Vary: Origin
 Access-Control-Allow-Origin: *
@@ -7,21 +7,13 @@
 Connection: keep-alive
 Transfer-Encoding: chunked
 
+data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"role":"assistant"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
-event: data
-
-data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"role":"assistant","content":"1"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
+data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":"1"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
-
 
-event: data
-
-data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"role":"assistant","content":" 2"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
+data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":" 2"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
-
 
-event: data
-
-data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"role":"assistant","content":" 3"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
+data: {"object":"chat.completion.chunk","model":"ggml-gpt4all-j","choices":[{"delta":{"content":" 3"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
-
 
-event: data
-
 data: {"model":"ggml-gpt4all-j","choices":[{"finish_reason":"stop"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
-

+data: [DONE]

Diff finished.  Sun May 21 14:36:35 2023

Notes for Reviewers

Signed commits

• Yes, I signed my commits.

[mudler]

Tested locally - looks good here, thank you!