workaround pypi empty responses by disabling cache control#4823
Conversation
| ) | ||
|
|
||
| self._cache_control_cache = FileCache(str(release_cache_dir / "_http")) | ||
| self._session = CacheControl( |
There was a problem hiding this comment.
I am not a 100% convinced this is the right solution for this. I'd much rather have e-tag used in the cache key.
Skipping cache for empty responses or invalid cache values, should be handled by gracefully handling failure - ie. clear the invalid cache and retry or fallback.
There was a problem hiding this comment.
I guess that's the naive approach here.
There was a problem hiding this comment.
That approach is at the request level and not really at the cache level. I think an intermediate approach (e.g. layered on top of CacheControl and factoring in ETAGS) is the way to go. The caching is an important optimization, and adding a retry system at the high-level HTTP (e.g. .get()) API seems more a hack than a fix.
|
fwiw, i haven't actually hit this in a while using latest poetry releases.. closing, fwiw i think the actual issue was solved on the pypi cdn side. |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
4 participants
Pull Request Check List
Resolves: #4821
Alternative to: #4822 and #4717
This is a work-around fix for pypi/warehouse#10387 . Thanks @j-martin for also looking at this.
This disables cache control for package metadata due to issues observed that pypi can return empty responses. The package artifact is still maintained. The diff is a little larger due to need to keeping type checker happy and removing tests assuming cache control usage.