when _validate_request, remove request_id, or error will like Unexpec…#40516
Closed
sywangyi wants to merge 1 commit intohuggingface:mainfrom
Closed
when _validate_request, remove request_id, or error will like Unexpec…#40516sywangyi wants to merge 1 commit intohuggingface:mainfrom
sywangyi wants to merge 1 commit intohuggingface:mainfrom
Conversation
…ted keys in the request: {'request_id'} INFO: ::1:44814 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Contributor
Author
|
fixed by #40501 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ted keys in the request: {'request_id'} INFO: ::1:44814 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity
What does this PR do?
Fixes # (issue)
@gante please review the PR.
how to produce the issue
transformers serve --log_level debug
transformers chat Qwen/Qwen2.5-0.5B-Instruct do_sample=False max_new_tokens=10
send 1st request-> OK
send 2st request->crash. client log:
File "/home/ywan171/transformers/src/transformers/commands/chat.py", line 742, in _inner_run
model_output, request_id = await interface.stream_output(stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ywan171/transformers/src/transformers/commands/chat.py", line 130, in stream_output
async for token in await stream:
^^^^^^^^^^^^
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 963, in chat_completion
data = await self._inner_post(request_parameters, stream=stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 289, in _inner_post
raise error
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 275, in _inner_post
response.raise_for_status()
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1161, in raise_for_status
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 422, message='Unprocessable Entity', url='http://localhost:8000/v1/chat/completions'
server log:
INFO: ::1:51300 - "POST /v1/chat/completions HTTP/1.1" 200 OK
Validating request: {'model': 'Qwen/Qwen2.5-0.5B-Instruct@main', 'stream': True, 'request_id': 'req_0', 'generation_config': '{\n "bos_token_id": 151643,\n "eos_token_id": [\n 151645,\n 151643\n ],\n "max_new_tokens": 10,\n "pad_token_id": 151643,\n "repetition_penalty": 1.1,\n "temperature": 0.7,\n "top_k": 20,\n "top_p": 0.8,\n "transformers_version": "4.56.0.dev0"\n}\n', 'messages': [{'role': 'user', 'content': 'hi'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today? If'}, {'role': 'user', 'content': 'hi'}]}
Unexpected keys in the request: {'request_id'}
INFO: ::1:39042 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity