Using trio with Qt's event loop?

[nicoddemus]

Hi,

Awesome library, I've been studying it and the care with its design and implementation is something to behold, congratulations to @njsmith and the team!

At work we use Qt to develop desktop applications and I was wondering if it is possible to use Qt's event loop instead of the internal trio event loop, something like quamash does for asyncio.

[njsmith]

Hello!

Awesome library, I've been studying it and the care with its design and implementation is something to behold, congratulations to @njsmith and the team!

Thanks! And sorry for the slow response...

At work we use Qt to develop desktop applications and I was wondering if it is possible to use Qt's event loop instead of the internal trio event loop, something like quamash does for asyncio.

Yeah, Trio+GUI is an important question! Also... kind of complicated. Right now there isn't anything implemented. Obviously this should change. I have very little experience writing GUI apps, so I'm not sure I know enough to recognize a good solution when I see it. So it's tricky.

Well, if you look around the issue tracker, you'll see that Trio follows a thinking-out-loud-driven development methodology, and we don't have a ticket yet for GUI stuff, so let's start with a brain dump of some things I've been thinking about :-).

First, if you just want to get something working, the shortest path to do that will be to run Qt in one thread and Trio in another. It's not too hard to build some infrastructure to call between them, similar to what trio.BlockingTrioPortal does for calling between Trio and a thread running synchronous code – I can give you some tips if you like. This isn't exactly ideal, basically because it involves threads and threads are never ideal. In particular, if you have actual logic running simultaneously in the Qt thread and the Trio thread, then you have to take all the usual care about them stepping on each other toes, which is too bad since one of the major advantages of cooperative scheduling is that it makes it easier to reason about interleavings.

That said, people do manage to write threaded programs all the time, and this probably works great for lots of cases, especially if you do things like push all the complex state manipulation onto the Trio side and restrict the Qt side to just dealing with the GUI and dispatching events to the Trio side. I'll also CC @matham here, who's been doing something similar with Kivy instead of Qt, and might be able to offer some comments. If you just want to get something working, stop here, this is definitely the path of least resistance.

---

Okay, that's an expeditious hack solution. What's a real solution look like? I'm... not sure. Popular libraries like Qt are built entirely around callbacks, while Trio's whole design (and a lot of its benefits) come from religiously avoiding callbacks. So what would a "Trio-style" GUI API even look like? I'm intrigued by some of the things I see coming out of the JS world, like Elm and Redux, that are designed to build GUIs without callback spaghetti. But this is obviously a case where practicality beats purity – desktop GUI APIs currently don't work this way, and we're not going to redesign Qt anytime soon! This makes me a bit nervous about committing to a particular solution right now (e.g. by including some half-baked qt support directly inside the trio core), but we gotta do something. So let's think about a possible medium term solution: smushing Trio and Qt together into the same thread (so at least we avoid threading issues), while keeping their existing architectures/APIs.

There are some practical challenges here: currently Trio doesn't have the infrastructure to use a pluggable event loop; in fact it doesn't expose an "event loop" or "reactor" concept at all. Internally, of course, it does have an API for this to handle different OSes (see the IOManager classes in trio/_core/), so we could imagine having a Qt backend. But there are two reasons this API is internal: (1) I'm reluctant to commit to a particular stable form; that adds a big chunk of concrete right in the middle of some of our most complicated code, (2) it actually doesn't have a single fixed API that abstracts over different OSes, since that ends up exposing some messy least-common-denominator thing; instead each backend is allowed to expose the full power of the current OS. But this approach really works best when you know you only have a fixed set of backends you care about. You could imagine coming up with some way for a Qt backend to expose the particular primitives that Qt gives you, though we don't have that now (currently the set of exposed primitives in trio.hazmat are determined at import time based on the OS). That's doable. What gets really nasty though is that the abstraction layers on top have to know about all the different backends – so like implementing trio.subprocess is on my todo list (#4), and it's hard enough getting good subprocess support on the big 3 OSes, but now Qt's native subprocess primitives are totally different again, so I guess I'd need a fourth implementation of trio.subprocess? Ick.

[This is where I paused writing this for like 4 days, while mulling over an idea :-)]

OK, though... so the two things we've identified as wanting to do are: (1) run all the trio and qt user code in the same thread, to reduce race conditions, (2) keep trio's existing backend interfaces, and ideally implementations. So... is it possible to use a threading hack to move just trio's I/O backend into its own thread?

I haven't thought this through fully, but I think there are two basic approaches that might work:

• keep basically the same IOManager classes that we do now, and the same run loop, but implement the run loop as a chain of Qt callbacks, where each time it runs everything runnable, then kicks off the io_manager.handle_io call in a worker thread, which then eventually calls back into the main thread to schedule another tick of the run loop. This way, we're either in handle_io, or in a trio task, but never both at the same time, so it's OK that they're both manipulating the same data structures (in particular, the trio task queue). However, this would mean that other Qt code could be running at the same time as handle_io, so we couldn't allow random Qt code executing outside of a trio task to casually invoke trio functions, like event.set().

  Basically in this approach, Qt↔Trio interactions would still need to go through the same kind of interface that they would if Qt and Trio were executing in different threads. But if they were both acting on shared data structures without locking, then that would be OK, because cooperative scheduling. (And Trio calling Qt methods would be fine – I'm not sure if Qt allows that from other threads or not. I think maybe signals are OK, but not random methods like, I dunno, QLabel.selected_text?)

• alternatively, something like that, but moving just the lowest-level blocking call into a thread – so like doing run_sync_in_worker_thread(epoll.wait), except of course we can't use run_sync_in_worker_thread itself because it assumes a working run loop. The tricky thing here is that I think we'd have to basically rewrite the IOManager classes, because you can't necessarily manipulate an epoll/kqueue/select from one thread while it's running, so whenever a new task calls wait_socket_writable or whatever we'd have to poke the I/O thread to wake up and start watching this new socket. Or maybe this wouldn't be necessary if we were careful to make sure that you couldn't call such functions from Qt context, for similar reasons to above? There are very few synchronous methods exposed on IOManagers now, basically just the "register an arbitrary kqueue watch" and "register an arbitrary IOCP watch" methods.

Hmm.

[matham]

Yes, I implemented what you describe in the first part, and that was indeed the quickest and simplest approach. Lemme first explain how kivy does its eventloop, as I'm not super familiar with qt. Broadly, it looks like this:

def run():
    while True:
        sleep(1 / frame_rate)
        process_all_callbacks()
        read_input_providers_and_dispatch()
        update_gl_graphics()
run()

qt has something similar in proceesEvents or something, I believe.

I could see of two ways to run async code with kivy, which I had originally planned to implement using the asyncio lib. (1) have kivy as an asyncio backend by implementing the kivy eventloop as an AbstractEventLoop that maps tasks to callbacks. I think this is what you were thinking off. The cons of this is that I'm not sure I trust kivy to run async task scheduling because conceptually, scheduling and flipping between async tasks is very different than just executing all the callbacks in sequence and waiting until the next frame (even if you allow priority switching etc). What you're optimizing for is very different between these two situations.

The second approach is to have asyncio run kivy as just another task and leave asyncio squarely in charge of scheduling as follows:

async def run():
    while True:
        await sleep(1 / frame_rate)
        process_all_callbacks()
        read_input_providers_and_dispatch()
        update_gl_graphics()

loop = asyncio.get_event_loop()
loop.run_until_complete(run)
loop.close()

After seeing trio and nurseries I became convinced that nurseries is the way to go for async and that only the second option makes sense. I also think now that with current GUIs, the best thing to do is let the GUI callbacks deal only with GUI stuff and leave all the user stuff run separately as async tasks like in option (2). I.e. use the GUI callbacks as little as possible and instead use await etc. So how would this work, especially with threads?

I can see a few configurations, listed below as well as some example functions/classes that I use.

async def run_in_kivy_thread(fn, *args):
    # create kivy callbacks that will exec fn
    # suspend task and wait for kivy to notify us it executed
    # the fn and then we'll reschedule the task
    ...

class TrioTrioPortal:

    def __init__(self, token=None):
        self.token = token or current_trio_token()

    async def trio_run_in_other_trio_thread(fn, *args):
        # schedule a call in the other trio loop running on another thread
        # suspend task and wait until the task is done and return the result
        ...

• One thread, like in option (2). This should work okay as it's all one thread.
• Two threads; one running the trio event loop and one the kivy event loop. Call run_in_kivy_thread from trio thread to safely execute stuff in kivy thread. For the reverse, you'd need to use queues or traditional multithreaded approaches because you cannot create/inject a task into the trio thread from kivy thread.
• Two threads; both running the trio event loop, with one also running the kivy event loop like in option 2. This way, either thread can execute stuff in the other using the TrioTrioPortal. In addition you can also use run_in_kivy_thread. This I think is the best, as it can let you await in the user thread while stuff that needs to execute in the main kivy thread is executed through the TrioTrioPortal.

The last approach is what I'm currently trying to see how feasible it is in a real application. You should be able to do something very similar with qt.

There's another convenience thing I use; I added a way to await in trio for an event (or property change) to occur in kivy in a thread safe manner. I.e. say there's a kivy on_release event for a button, you can do:

async for event in button.async_bind('on_release'):
    do_something()

This allows you to watch in trio for events and other things within the GUI without blocking. The only difficulty here is that it's not suitable for actual event handling that needs an immediate response, because trio will schedule the task, rather than executing immediately like a traditional event handler.

This approach requires pretty much no changes in trio.

[njsmith]

Ah, right, that's another option I forgot to mention: using Trio to implement a new Qt event loop backend. This is possible in theory, by implementing the QAbstractEventDispatcher interface. (That's basically how we're supporting asyncio in trio-asyncio.) But this would require a lot of messing with complicated Qt internals that are already very fast and mature and may make awkward assumptions, so I'm a bit dubious. (In particular, Qt allows for recursively re-entering the event loop, which Trio doesn't.) OTOH maybe it would work great! If someone wants to experiment with this, I will be fascinated to hear what you come up with :-).

Edit: here's an example: https://github.com/sjinks/qt_eventdispatcher_libuv

[njsmith]

@matham

you cannot create/inject a task into the trio thread from kivy thread.

Can you elaborate on what you mean here? Obviously at some level you can do this, that's how trio.BlockingTrioPortal and your TrioTrioPortal work, so I assume you mean something a little more nuanced...

[matham]

you cannot create/inject a task into the trio thread from kivy thread.

As you say, we literally can, but then where do you wait for it without blocking or abandoning it as it's being executed in the other thread. Even if you're executing short user code in the user thread while waiting in the kivy thread, you're still making the main thread unresponsive. But, if you also run a trio event loop in the kivy thread, then you can just do await execute_in_user_trio_thread() in it and the kivy thread is not blocked.

Of course you could also make the user thread schedule a callback in kivy when it's done rather than it waiting, but that's more annoying. I'm still not sure how much this will actually be useful though as there may not be much need to call into the user thread and execute user code there (or at least I hope so, because otherwise the user code will be called callback style, which is opposite of the goal here).

[njsmith]

Right, but that's just the inherent annoyingness of working with a callback API, right? :-)

If you have something short and synchronous that you want to do, the kind of thing that you'd normally just execute immediately in the Qt thread, except that it has to be done in the Trio thread instead... I think in this case it might actually OK to block the Qt thread while waiting for it. The thread switch adds some bookkeeping overhead, but it's still not going to block the Qt thread for much longer than it would take to execute it normally.

For long-running actions, the only way to do that in Qt/Kivy's native API is to schedule them and then get a callback when they're done – that's just a fundamental property of how Qt/Kivy work, not something that Trio created or can do anything about :-). It'd be nice if we had non-callback-based GUI libraries, but until then I think we just have to live with this.

And in lots of cases, you may be fine just scheduling something to run in the trio thread and then abandon it – for example, run_sync_soon_in_trio(q.put_nowait, value), where q is a trio.hazmat.UnboundedQueue. This is not the kind of API we like in Trio-land for all the obvious reasons, like, what if the call fails? But in normal operation UnboundedQueue.put_nowait never fails; and if something really abnormal goes wrong, and we lose the exception... well, that's the same way that everything works on the Qt side, so we're not really making things worse if there are a few operations in the Qt/Trio interface that have the limitations of Qt.

[nicoddemus]

Hi @njsmith and @matham,

Thanks a lot for the detailed discussion.

Full disclosure, I've never programmed with asyncio or trio besides some toy examples, so please excuse me if I'm saying something which is clearly wrong or impossible to accomplish. 😁

I don't know the internal details and consequences of integrating a separate event loop in Trio, so I'm still digesting what has been said here.

---

The use case I had in mind was to allow the user to execute asynchronous code in response to user events, something like:

button.clicked.connect(download_images)

# ...

async def download_images():
    button.setEnabled(False)
    progress_bar = QProgressBar()
    for image_path in paths:
        await download_image(image_path)
        progress_bar.advance()
    button.setEnabled(True)

(I understand that's not possible from a technical stand-point, I'm just trying to demonstrate the general idea).

Without getting into the issue of actually integrating into the Qt event loop, I believe we can get the above to work with an approach similar to:

button.clicked.connect(download_images_sync)

def download_images_sync():
    trio.run(download_images)

# ...

async def download_images():
    button.setEnabled(False)
    progress_bar = QProgressBar()
    for image_path in paths:
        await download_image(image_path)
        progress_bar.advance()
    button.setEnabled(True)

Of course the problem with this is that download_images will block Qt's own event loop, leaving the GUI unresponsive.

QApplication.processEvents() can be called periodically to avoid blocking the Qt's event loop. What if we schedule two tasks to run at the same time, one doing the original work of downloading the images and another one which just calls QApplication.processEvents() periodically?

button.clicked.connect(download_images_sync)

def download_images_sync():
    trio.run(run_with_process_events)

async def run_with_process_events():
    async with trio.open_nursery() as nursery:
        event = trio.Event()
        nursery.start_soon(download_images_monitored, event)
        nursery.start_soon(process_events_tick, event)

async def process_events_tick(event):
    while not event.is_set():
        qApp.processEvents()
        trio.sleep(1 / 20)

async def download_images_monitored(event):
    await download_images()
    event.set()

# ...

async def download_images():
    button.setEnabled(False)
    progress_bar = QProgressBar()
    for image_path in paths:
        await download_image(image_path)
        progress_bar.advance()
    button.setEnabled(True)

This looks like it would run download_images asynchronously and would keep the GUI responsive. Of course this can eventually be refactored to something nicer to use, for example:

button.clicked.connect(async_binder(download_images))

async def download_images():
    button.setEnabled(False)
    progress_bar = QProgressBar()
    for image_path in paths:
        await download_image(image_path)
        progress_bar.advance()
    button.setEnabled(True)

Where async_binder would be responsible to create the intermediate management tasks.

But there's the question of what would happen if another user event (another button for example) also wants to execute its asynchronous handler while we are downloading images in download_images? What will happen if we call trio.run while we are already processing another task?

---

Back to how integrate this into the Qt event loop, what if trio exposed a way schedule tasks, and the user could drive task execution themselves? For example (using a fictional API):

task_manager = trio.get_task_manager()

task_manager.schedule(coroutine1)
task_manager.schedule(coroutine2)
# runs whatever task should execute next, until we find an "await" call
task_manager.advance_until_await()  

IOW, trio "event loop" would be something like:

while task_manager.has_running_tasks():
    task_manager.advance_until_await()  

If the above is possible, then integrating it into Qt's event loop would be simple:

# normal Qt application:
qApp.exec_loop()

# integrated with trio:
task_manager = trio.get_task_manager()
while some_condition_to_quit_app():
    task_manager.advance_until_await()
    qApp.processEvents()

I'm not sure how this would play with trio's internals (specially with I/O), I thought I would throw the idea here and see where it lands.

---

As I said I'm still digesting what has been said in this thread so far, so I will come back with more comments if I have more to contribute in the next few days.

[nicoddemus]

Hi folks,

I did a quick experiment with my ideas above in this repository: nicoddemus/trio-qt-sandbox.

The idea of running two tasks at the same type, one running a loop which calls QApplication.processEvents periodically while the other executes an asynchronous function works partially, because we can't have two trio.run calls happen simultaneously.

But the idea of starting a task which calls periodically QApplication.processEvents and user actions are mapped to asynchronous functions which are then scheduled to run in the same nursery as the task with the loop, this idea seems to work.

Here's the full code for the latter idea:

import trio
from PyQt5.QtWidgets import QApplication, QWidget, QHBoxLayout, QPushButton


async def loop(app, event):
    while True:
        app.processEvents()
        with trio.move_on_after(0.01):
            await event.wait()
        if event.is_set():
            return


_nursery = None


def get_nursery():
    return _nursery


def async_bind(fn):
    def wrapper():
        get_nursery().start_soon(fn)


    return wrapper


async def say_hello():
    print('Hello')


async def main():
    app = QApplication([])

    widget = QWidget()
    layout = QHBoxLayout(widget)
    hello = QPushButton('Hello', widget)
    exit = QPushButton('Exit')
    layout.addWidget(hello)
    layout.addWidget(exit)
    widget.show()

    event = trio.Event()


    def on_exit():
        print('on_exit')
        event.set()


    hello.clicked.connect(async_bind(say_hello))
    exit.clicked.connect(on_exit)

    async with trio.open_nursery() as nursery:
        global _nursery
        _nursery = nursery
        nursery.start_soon(loop, app, event)


if __name__ == '__main__':
    trio.run(main)

This idea works, but I don't like very much of that move_after wait inside the loop function, it feels this will make the event loop not as responsive as it should. I will see if there's a better solution in the Qt API.

Thoughts?

[imrn]

Your loop's resolution is 0.01 secs. What is the % load of the pyton instance imposed on the cpu core for this case? Can you provide some values for 0.001 0.0001 etc.?

[imrn]

Of course this question is for an idle application, just to get an idea for the fixed overhead of your solution.

[nicoddemus]

@imrn actually my CPU sits at 0% flat regardless of which value I use for move_on_ after. I tried 0.001, 0.0001 and even 0.0 and I don't see any CPU usage spikes. Strange, I expected at least some overhead...

[imrn]

Can you verify that your application is NOT blocked at somewhere else? For example you may add another task printing something every 0.2 secs. Is it operational?

[nicoddemus]

Scheduled a new task which prints to the console every 0.2 secs:

@@ -32,6 +31,13 @@ async def say_hello():
     print('Hello')


+async def tick_tack(event):
+    while not event.is_set():
+        with trio.move_on_after(0.2):
+            await event.wait()
+        print('Tick tack')
+
+
 async def main():
     app = QApplication([])

@@ -58,6 +64,7 @@ async def main():
         global _nursery
         _nursery = nursery
         nursery.start_soon(loop, app, event)
+        nursery.start_soon(tick_tack, event)

Works pretty well, I see the periodic output in the console, clicking in the hello button also prints "hello", and CPU usage is a flat 0% even after I change move_on_after in loop to 0.0001.

[njsmith]

Hi guys,

Psst, not everyone here is a guy :-). Can you edit your message to "Hi all" or "Hi folks" or something similar? Thanks!

Works pretty well, I see the periodic output in the console, clicking in the hello button also prints "hello", and CPU usage is a flat 0% even after I change move_on_after in loop to 0.0001.

Huh, that's pretty cool. The main downside to this approach is that it prevents the CPU from entering deeper sleep modes, so it wastes power. (If you're on Linux, you can run powertop and see it complain about all the wakeups the app is doing.) I also expected it would add a bunch of latency to the UI, but if it doesn't, then cool. And if your apps are running on desktops maybe the power cost isn't a big deal – depends on your particular users and your tradeoffs. This kind of busy-polling is kind of a hack, but if it solves your problem, well, sometimes a hack is the right solution...

Backing up: the fundamental challenge here is exactly how to get Qt's I/O and Trio's I/O to play nicely with each other. If you use a busy-polling approach, then you can just poll for both kinds of I/O on each cycle, and integration is easy – but you have the problem with wasting power. To solve that, you need the app to actually go to sleep until something happens to wake it up, which means calling some low-level OS routine like select or epoll or GetQueuedCompletionStatusEx. The way these all work is that they give the OS a list of events we're interested in, and tell it that it doesn't have to wake us up again until one of them happens. But obviously, a single thread can only call one of these functions at a time – we can't have Qt calling one of these functions, and have Trio separately calling one of these functions, and have everything running everything in a single thread. Something has to give. So there are a few options:

• Get Qt and Trio to work together to build up a single combined list of events they're interested in. This is the "run Trio on the Qt mainloop" or "run Qt on the Trio mainloop" approach. The challenge here is that you have to somehow get a single object that understands both Trio and Qt's I/O abstractions at the same time. That's what this message is about: Using trio with Qt's event loop? #399 (comment)

• Run Qt and Trio in separate threads, so that Qt can tell the OS to wake it when an event Qt cares about happens, and Trio can tell the OS to wake it when an event Trio cares about happens. With this approach it's super easy to get something basic working – just make a threading.Thread and away you go. With only a little bit more work you can also implement some APIs to pass messages between the threads, see e.g.. The challenge here is that now you have two threads, so if you're running non-trivial code in both of them accessing shared state, then you have all the normal thread synchronization issues to worry about. This may or may not be a problem for you in practice.

• Somehow arrange for Qt and Trio to call their core block-and-wait-for-IO operation in separate threads, but then do their actual scheduling and running of user code in the same thread. This is the idea that I was exploring in my first message.

In the long run, my tentative guess is that either the "Qt on Trio" approach or the "use two threads for waiting for I/O, and one thread for executing code" approach are the ideal options. But the "polling on a regular basis" and "use one thread for Qt and one for Trio" approaches are both good practical options if you need something now.

[njsmith]

Oh, ugh, there's another complication in gluing these libraries together that I forgot about: Qt allows for re-entering the main loop, like by calling processEvents recursively inside processEvents. This is kind of a gross hack that IIRC is used in a few places where they decided doing things in a proper non-blocking way was just too annoying. Trio doesn't support this. I think this is a big problem for "Qt on Trio".