Best approach to building trio ecosystem

[merrellb]

My async experience has largely been with the aio-lib ecosystem (particularly http, websockets, elasticsearch and kafka). I would love to have this functionality with trio and would happily contribute to help make it happen. A few thoughts/questions:

1. If I wanted to for example add http functionality (which I believe trio currently lacks) would it be best to focus on porting aiohttp or perhaps focus on h11 or hyper-h2 (which already seem to have some basic curio integration).
2. Are there any useful patterns to port synchronous libraries (eg the official Elasticsearch library) to trio?
3. Would it be worthwhile to add subprojects under python-trio for various popular APIs (similar to aio-lib)?

[njsmith]

Hi Brian, great to hear from you!

There's definitely lots to work on. Regarding http: the main immediate prerequisite inside trio itself is TLS support (#9), which is very doable, though we need to sort out what our idea of a "stream" looks like (part of #73).

We also have some todo items to split out our pytest and sphinx infrastructure into reusable packages (#27, #71), which should be relatively straightforward and is something that any downstream projects are going to want.

Regarding aiohttp vs. h11/h2: I may be biased, but I would definitely try to build something on top of h11 rather than trying to do a direct port of aiohttp. My impression is that aiohttp's internals are... unnecessarily complex, and would need significant reworking to match the structure that trio requires. There definitely is a fair amount of stuff that a full fledged http client needs (I assume you're thinking mostly about clients?) that isn't in h11, though.

Another possibility is to look at urllib3; @Lukasa is working on porting it to use h11: urllib3/urllib3#1067
In the long run the ideal situation would probably be for urllib3 itself to gain the ability to run in "trio mode", because there is a ton of complicated stuff that a really high-quality http client has to do and there's less wasted work across the ecosystem if we can all share one implementation of most of it. It's possible that this will take long enough to become viable that we want something else as a stop-gap in the mean time; that could be a minimalist library written from scratch, or a hacky fork of @Lukasa's branch with a bunch of extra awaits sprinkled about... I dunno. Just throwing it out there as something to think about.

Regarding porting synchronous libraries: I'm not sure I understand the question, I'm not familiar with the Elasticsearch library, and I've barely written any more trio code than you have ;-), so I'm not sure what to say. As a general rule of thumb, I think idiomatic trio APIs are going to look at lot like synchronous APIs, except with blocking operations becoming async and with most of the timeout= arguments stripped out.

Regarding the python-trio organization: that is one of the reasons I registered an organization, yes :-). We probably shouldn't overdo it with trying to drag every trio-using project in, but certainly for more central/official pieces it's a good place. (pytest-trio and sphinx-trio were the ones I immediately had in mind.)

[oremanj]

Closing this since I don't think there are action items left:

• we got SSL and generic Stream interfaces
• pytest-trio and sphinxcontrib-trio were split out
• the urllib3 work is being tracked elsewhere
• the need for docs on porting to trio is tracked elsewhere