Integration with existing GUI thread

[derceg]

Hi, I'm looking at integrating your library into my application, since the functionality it offers would be very helpful when it comes to writing asynchronous code. The thing I'd find most useful would be to be able to call PostTaskAndReply from the existing GUI thread and have that work. Out of the box, it doesn't work, because as your documentation notes, that method has to be called from a thread with a task queue.

I can see a potential way I could work around that:

• Create a custom MessagePump. When QueuePendingTask is called, the pump would invoke PostMessage to notify the main thread, via the standard Windows message loop, that a task is available.
• Create a class analogous to the existing Thread class (something like MainThread) that holds a SingleThreadTaskRunner. That class would run any pending tasks once notified (via the PostMessage call).
• That class would also set up the ScopedSequenceIdSetter and SequencedTaskRunnerHandle instances for the main thread once at startup, to ensure that there's always a SequencedTaskRunnerHandle available on the main thread.

I'm hoping to get your feedback on whether the last point is reasonable. My understanding is that if the main thread were solely processing tasks (and message loop items were turned into tasks), things would work, but that's a larger change that I don't really want to go through with. And it doesn't look like this library is set up to support that anyway (e.g. the Thread class only processes tasks and has no handling for something like the Windows message loop).

The solution above also requires using a few things under base::detail - for example, base::detail::SequenceIdGenerator and base::detail::ScopedSequenceIdSetter, so I'm wondering whether that could be better supported in the library.

Even without the changes above, I'd still need to be able to either customize what the thread message loop is doing (e.g. a Windows COM thread can need to pump the windows message loop) or create a custom class that does what I need. In which case, I'd still need to refer to base::detail::SequenceIdGenerator to construct the task runner.

Thanks for any help.

[RippeR37]

Hi David!

Super happy that you find libbase helpful! You asked a lot of great questions, but could you give more more context on what GUI APIs/libraries/etc. you're using for your own threading, message/task processing, etc.? I'll try to check it out in your repo, but it could take me longer that way.

I'll also check out the rest (I haven't been touching this lib for a while, need to dust of few things) and get back to you on that.

[derceg]

Thanks for getting back to me!

You asked a lot of great questions, but could you give more more context on what GUI APIs/libraries/etc. you're using for your own threading, message/task processing, etc.?

The application uses the Windows API and the Windows common controls. At the moment, I'm using a lightly modified version of https://github.com/vit-vit/CTPL to run various tasks in the background. That works, but the experience isn't as nice as it could be. For instance, I don't really have much of a general abstraction around CTPL, so if I wanted to launch a new type of task in the background, I'd have to write something new and duplicate parts of what I've already done. I have thought about building a simple interface around CTPL to implement something like PostTaskAndReply, but there are also a few other things in your library that would be very useful, so l don't think doing just that would be enough.

The threaded code I have currently works a bit like what I describe in the original message. A piece of work is run on a background thread. When the work is finished, a message is sent to the main thread using PostMessage. I'm not sure how familiar you are with the Windows message loop, but PostMessage will dispatch a message to the specified window, with the message guaranteed to be processed on the thread that created the window (which will be the main thread). That way, I can have the background thread easily notify the main thread that work has been completed. You can see an example of that here.

What prompted me to look for a task library again recently is some code I wrote to change the directory when the current directory is deleted. There's a function to find the first parent folder that still exists (so that a navigation to that parent folder can be initiated) and that code is currently running on the main thread. It should really run on a background thread, but I don't have an easy-to-use abstraction to do that. I'd have to do it in a partly ad-hoc way. I think something like PostTaskAndReplyWithResult is a much more natural interface. So, one of the things I'd want to do, specifically, would be to make a call like the following on the main thread:

thread_pool_.GetTaskRunner()->PostTaskAndReplyWithResult(FROM_HERE, base::BindOnce(&GetClosestExistingParentFolder), base::BindOnce(&ShellBrowser::OnClosestExistingParentFolderFound, weak_ptr_factory_.GetWeakPtr()));

[RippeR37]

Okay, I brushed my Windows API knowledge etc. and I think you were close with your initial idea.

The idiomatic way to implement this in libbase would be something like this:

• Implement a custom MessagePump which will:
  • on QueuePendingTask will do windows PostMessage, and save the original 'task' somewhere (queue/hashmap/...) (it would have to be some new unique message type)
  • on GetNextPendingTask will query windows message loop with GetMessage() and will return either Windows API Message or libbase task, depending on the kind of message that was returned by GetMessage()
• Implement a custom MessageLoop that uses that new MessagePump and either runs libbase task or passes the Windows Message to some specified base::RepeatingCallback that will effectively be content of your main event loop right now.
• (optionally, preferred) Implement a sort-of WindowsThread or something like that that uses the above classes in a similar way base::Thread is working. You could also name it MainWindowsThread which wouldn't spawn new std::thread for its work, but start executing work by itself, and wait for some signal from Windows API (or other threads) to quit.

Of course implementations of these should behave and set global state in same way as current internals do, including setting things like ScopedSequenceIdSetter and SequencedTaskRunnerHandle.

I can implement this functionality and add a short example how to use it soon (this or next week?) if you'd like to wait, or you can implement and send me a patch, and I'd include it (possibly after some tweaks to make it more aligned with libbase design) in the project.

[RippeR37]

Hey @derceg,

Could you check out PR #40 and see if this solves your problem here and would work for you nicely? Any feedback would be grateful.

Some notes from implementing this:

• I decided to go with a sort-of "addon/attachment" to existing thread because this is something that should be most adaptable approach to different codebases and to existing projects and doesn't force anything on the user.
  • This is also why the class is templated so that you can pass your own custom unique Message ID that will be unique (and don't have to bake this as some define/constant).
• Initially I based this on "user data" tied to a Window, but I didn't want to complicate user's logic with handling that through libbase, so I worked around it. The only thing that libbase is overwriting right now is window's procedure function for given HWND handle and substitutes it with its own to process what is needed for message loop, and sends all other messages to original procedure.

I still have to check out few things with it, but it should be possible for you to take it for a spin and see if it matches your expectations.

[derceg]

Thanks for taking the time to look into this more fully, I appreciate it! I'll have a look over the PR and see if it does what I need.

I did also want to check whether there's any plans to support package managers? I use vcpkg, so I could look at creating a port for that if necessary. The issue at the moment is that I don't really have a clean way of integrating a purely cmake project. I did also very recently decide to integrate glog into my application, so I think that might cause issues if this library is using its own specific version from a submodule. I'm not too sure how package managers typically handle transitive dependencies, but it would be helpful to have something like a version range on the dependency.

[RippeR37]

@derceg - a late reply, but I finally had some time to sit more on this.

Integration for WinAPI and wxWidgets has landed. At some point I may consider adding something for Qt or other 3rd party libs, but at the moment this isn't strictly needed and with these two and base::RunLoop, I think it would be easy for someone to add their own custom integration for different libraries as well.

Since you mentioned vcpkg - I also recently added port of this library to that package manager under ripper37-libbase name (unfortunate). This version doesn't have all these new features yet, but soon I should release v1.1 with all these new things and update it on vcpkg repository as well, so if you'd like to use this soon, hopefully nothing will stop you anymore.

For now I'm closing this ticket, new ticket can be opened for other 3rd party libs/frameworks.