Skip to content

when _validate_request, remove request_id, or error will like Unexpec…#40516

Closed
sywangyi wants to merge 1 commit intohuggingface:mainfrom
sywangyi:chat_error
Closed

when _validate_request, remove request_id, or error will like Unexpec…#40516
sywangyi wants to merge 1 commit intohuggingface:mainfrom
sywangyi:chat_error

Conversation

@sywangyi
Copy link
Copy Markdown
Contributor

…ted keys in the request: {'request_id'} INFO: ::1:44814 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity

What does this PR do?

Fixes # (issue)

@gante please review the PR.

how to produce the issue
transformers serve --log_level debug

transformers chat Qwen/Qwen2.5-0.5B-Instruct do_sample=False max_new_tokens=10

send 1st request-> OK
send 2st request->crash. client log:

File "/home/ywan171/transformers/src/transformers/commands/chat.py", line 742, in _inner_run
model_output, request_id = await interface.stream_output(stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ywan171/transformers/src/transformers/commands/chat.py", line 130, in stream_output
async for token in await stream:
^^^^^^^^^^^^
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 963, in chat_completion
data = await self._inner_post(request_parameters, stream=stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 289, in _inner_post
raise error
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py", line 275, in _inner_post
response.raise_for_status()
File "/workspace/ywan171/miniforge3/envs/optimum-intel/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1161, in raise_for_status
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 422, message='Unprocessable Entity', url='http://localhost:8000/v1/chat/completions'

server log:
INFO: ::1:51300 - "POST /v1/chat/completions HTTP/1.1" 200 OK
Validating request: {'model': 'Qwen/Qwen2.5-0.5B-Instruct@main', 'stream': True, 'request_id': 'req_0', 'generation_config': '{\n "bos_token_id": 151643,\n "eos_token_id": [\n 151645,\n 151643\n ],\n "max_new_tokens": 10,\n "pad_token_id": 151643,\n "repetition_penalty": 1.1,\n "temperature": 0.7,\n "top_k": 20,\n "top_p": 0.8,\n "transformers_version": "4.56.0.dev0"\n}\n', 'messages': [{'role': 'user', 'content': 'hi'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today? If'}, {'role': 'user', 'content': 'hi'}]}
Unexpected keys in the request: {'request_id'}
INFO: ::1:39042 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity

…ted keys in the request: {'request_id'} INFO: ::1:44814 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
@sywangyi
Copy link
Copy Markdown
Contributor Author

fixed by #40501

@sywangyi sywangyi closed this Aug 29, 2025
@sywangyi sywangyi deleted the chat_error branch November 19, 2025 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant