Inline "small" systems into the multithreaded executor

[james7132]

Objective

Minimize overhead from scheduling large numbers of systems with very little work to do. Partially address #7208.

Solution

Keep a exponential moving average for how long it takes to run each system. If a system has historically run in under 2 microseconds, run it inside the multithreaded executor.

The threshold is currently hard-coded and is rather conservative to avoid potentially inlining too many systems and forcing it large sections to run single threaded.

TODO:

• Defer inline execution until all non-inlinable system tasks are spawned to avoid blocking
• Add an escape hatch for cases where systems have highly kurtotic behavior.

Performance

TODO

[james7132]

With #11906 now running the multithreaded executor immediately after every system, we can achieve similar results without needing to inline the systems. I've also come to realize that running systems inline is likely an all around bad practice given that it potentially blocks starting new systems even without a strict before/after relationship, and that's a huge risk in any cooperative task scheduler. Closing this out.