graphql-ws 'error' packet's .payload field should be array, not object

[seriyps]

Describe the bug

$ echo 'query ($id: ID) {session(id:$id) { addresses {address}} }' | gql-cli -v --transport=websockets ws://localhost:8080/api/graphql/web-test-202201302Vr7R/websocket -V id:1 | jq 
INFO:gql.transport.websockets:>>> {"type": "connection_init", "payload": {}}
INFO:gql.transport.websockets:<<< {"type":"connection_ack"}
INFO:gql.transport.websockets:>>> {"id": "1", "type": "subscribe", "payload": {"query": "query ($id: ID) {\n  session(id: $id) {\n    addresses {\n      address\n    }\n  }\n}", "variables": {"id": 1}}}
INFO:gql.transport.websockets:<<< {"type":"error","payload":[{"path":["ROOT","session","id","id"],"message":"Type mismatch. The query document has a value/variable of type (ID) but the schema expectes type (ID!)","extensions":{"code":"type_mismatch"}}],"id":"1"}
Traceback (most recent call last):
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets.py", line 292, in _parse_answer_graphqlws
    raise ValueError("payload is not a dict")
ValueError: payload is not a dict

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/seriy/workspace/temp_email/venv/bin/gql-cli", line 27, in <module>
    exit_code = loop.run_until_complete(main_task)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/cli.py", line 394, in main
    async for result in session.subscribe(query, **execute_args):
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/client.py", line 639, in subscribe
    async for result in inner_generator:
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/client.py", line 549, in _subscribe
    async for result in inner_generator:
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets_base.py", line 387, in subscribe
    answer_type, execution_result = await listener.get()
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets_base.py", line 56, in get
    raise item
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets_base.py", line 309, in _receive_data_loop
    answer_type, answer_id, execution_result = self._parse_answer(
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets.py", line 414, in _parse_answer
    return self._parse_answer_graphqlws(json_answer)
  File "/home/seriy/workspace/temp_email/venv/lib/python3.9/site-packages/gql/transport/websockets.py", line 326, in _parse_answer_graphqlws
    raise TransportProtocolError(
gql.transport.exceptions.TransportProtocolError: Server did not return a GraphQL result: {'type': 'error', 'payload': [{'path': ['ROOT', 'session', 'id', 'id'], 'message': 'Type mismatch. The query document has a value/variable of type (ID) but the schema expectes type (ID!)', 'extensions': {'code': 'type_mismatch'}}], 'id': '1'}

It seems that the graphql-ws specification declares ErrorMessage.payload as array of GraphQLError objects, but gql assumes that it should be just a single error object:

gql/gql/transport/websockets.py

Lines 291 to 292 in 12fc895

	if not isinstance(payload, dict):
	raise ValueError("payload is not a dict")

Relevant test / fixtures:

gql/tests/test_websocket_exceptions.py

Lines 149 to 202 in 12fc895

	invalid_payload_server_answer = (
	'{"type":"error","id":"1","payload":{"message":"Must provide document"}}'
	)
	async def server_invalid_payload(ws, path):
	await WebSocketServerHelper.send_connection_ack(ws)
	result = await ws.recv()
	print(f"Server received: {result}")
	await ws.send(invalid_payload_server_answer)
	await WebSocketServerHelper.wait_connection_terminate(ws)
	await ws.wait_closed()
	@pytest.mark.asyncio
	@pytest.mark.parametrize("server", [server_invalid_payload], indirect=True)
	@pytest.mark.parametrize("query_str", [invalid_query_str])
	async def test_websocket_sending_invalid_payload(
	event_loop, client_and_server, query_str
	):
	session, server = client_and_server
	# Monkey patching the _send_query method to send an invalid payload
	async def monkey_patch_send_query(
	self, document, variable_values=None, operation_name=None,
	) -> int:
	query_id = self.next_query_id
	self.next_query_id += 1
	query_str = json.dumps(
	{"id": str(query_id), "type": "start", "payload": "BLAHBLAH"}
	)
	await self._send(query_str)
	return query_id
	session.transport._send_query = types.MethodType(
	monkey_patch_send_query, session.transport
	)
	query = gql(query_str)
	with pytest.raises(TransportQueryError) as exc_info:
	await session.execute(query)
	exception = exc_info.value
	assert isinstance(exception.errors, List)
	error = exception.errors[0]
	assert error["message"] == "Must provide document"

To Reproduce

Query graphql-ws server with gql in such a way, so it returns {type: error, payload: ...} packet (for example, query a field that does not exist).

Expected behavior

gql should not crash, but rather report the errors, generated by the server.

System info (please complete the following information):

• OS: Linux
• Python version: 3.9.7
• gql version: 3.0.0
• graphql-core version: ???

[leszekhanusz]

Thanks for the bug report. It really shows that the graphql-ws protocol is not tested enough... Sorry about that.

[leszekhanusz]

Could you please check if the PR #299 fixes the problem ?