ClientPayloadError

[gimler]

• Python version: 3.6.3
• Operating System: Ubuntu 1604
• aiohttp version:
  aiohttp (3.6.1)
  aiohttp-sse-client (0.1.6)

Description

sometime i get ClientPayloadError

Task exception was never retrieved
future: <Task finished coro=<main() done, defined at prog.python:39> exception=NameError("name 'ClientPayloadError' is not defined",)>
Traceback (most recent call last):
  File "prog.python", line 23, in get_stock
    async for event in event_source:
  File "/home/?/.local/lib/python3.6/site-packages/aiohttp_sse_client/client.py", line 146, in __anext__
    async for line_in_bytes in self._response.content:
  File "/home/?/.local/lib/python3.6/site-packages/aiohttp/streams.py", line 39, in __anext__
    rv = await self.read_func()
  File "/home/?/.local/lib/python3.6/site-packages/aiohttp/streams.py", line 328, in readline
    await self._wait('readline')
  File "/home/?/.local/lib/python3.6/site-packages/aiohttp/streams.py", line 296, in _wait
    await waiter
aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed

What I Did

from aiohttp import ClientSession, ClientTimeout, ClientPayloadError
from aiohttp_sse_client import client as sse_client
...
timeout = ClientTimeout(sock_read=0)
    session = ClientSession(timeout=timeout)

    async with sse_client.EventSource(url, session=session) as event_source:
        try:
            async for event in event_source:
                j = json.loads(event.data)
                if 'value' in j:
#                    print(json.dumps(j, indent=4, sort_keys=True))
                    print(label + " " + str(j["value"]))
        except ConnectionError:
            pass
        except ClientPayloadError:
            pass

is there a better way to handle this?

[awarecan]

This exception was thrown when the connection was closed but the payload is not parsed complete (depends on the content encoding).

Per aiohttp document:

Reading from the stream may raise
:exc:aiohttp.ClientPayloadError if the response object is
closed before response receives all data or in case if any
transfer encoding related errors like misformed chunked
encoding of broken compression data.

I think it may be caused by the unstable network and your solution (catch it) looks correct.

[awarecan]

I gave it another look, you set ClientTimeout(sock_read=0), which is really tough setting. No wonder you will met ClientPayloadError.

I am going to close this issue as 1) it seems your coding issue 2) we have not heard back from you over a year.

[nick-antisoc]

Sorry for posting here after all this time, but I have a question.
How should someone handle exceptions like 'ClientPayloadError ', if we want the sse_client to be always up and running?

[awarecan]

@nick-antisoc the solution provided in the original post is the right way in my opinion.

Our library handled the automatically reconnection, so it is always up and running by default.

[nick-antisoc]

@awarecan Thank you for the quick reply.
Sorry if I'm missing something here, but in the simple example below how is the reconnection being handled?

import asyncio
from aiohttp_sse_client import client as sse_client
from aiohttp import ClientPayloadError

async def stream_wikipedia():
    async with sse_client.EventSource('https://stream.wikimedia.org/v2/stream/recentchange', timeout=-1) as event_source:
        try:
            async for event in event_source:
                print(event)
        except ConnectionError:
            pass
        except ClientPayloadError:
            pass

loop = asyncio.get_event_loop()
loop.run_until_complete(stream_wikipedia())

After ClientPayloadError the loop stops. Isn't that the proper way to use this functionality?

[awarecan]

hmmm, I think you are right.

You could add try except on

aiohttp-sse-client/aiohttp_sse_client/client.py

Line 157 in e311075

async for line_in_bytes in self._response.content:

PR is welcoming