-
-
Notifications
You must be signed in to change notification settings - Fork 999
Description
Prompted by #1733
It appears that iter_bytes can currently sometimes return a zero-length chunk. This isn't strictly a broken behaviour, since we don't make any assertions about if that might occur or not, and it's still correct, but it'd be preferable from a UX perspective if it didn't even do this.
Originally posted by michalc July 4, 2021
The last chunk in the below is of length zero:
with httpx.stream('GET', 'https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-4.4-essentials_build.zip') as r:
for chunk in r.iter_bytes():
print(len(chunk))is this expected?
It also looks like iter_raw doesn't have a zero length chunk at the end:
with httpx.stream('GET', 'https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-4.4-essentials_build.zip') as r:
for chunk in r.iter_raw():
print(len(chunk))since in this case there is no content-encoding, I guess I expect that iter_bytes == iter_raw
Having a bit of a nose around the code, suspecting it's from
Line 1483 in 47d712c
| decoded = decoder.flush() |
decoder.flush() returns an empty bytes instance, this is passed to chunker.decode, and since chunk_size is None, a list containing this empty bytes instance is returned Line 172 in 47d712c
| return [content] |
and then each element of this list is yielded to the calling code
This is related to uktrade/stream-unzip#13, where the code in stream-unzip does not expect any zero length chunks. Potentially stream-unzip should be robust against this case... but wasn't sure if this was desirable in httpx, so mentioning it here in case it is an issue to be addressed.