[Feature]: Blocking data consumers

[Youssef1313]

Summary

Some extensions may not be implementable using the current extensibility model of MTP.
For example, an extension might want to start and complete a specific piece of work and guarantee that it happens before the test starts.

Currently, the only option is to implement IDataConsumer and process TestNodeUpdateMessage with in-progress state in ConsumeAsync. But, due to the async nature of message bus, there is no guarantee that the test isn't already started by the time ConsumeAsync is called.

Proposed Feature

1. Introduce a new IBlockingDataConsumer interface.

   public interface IBlockingDataConsumer : ITestHostExtension
   {
       Type[] DataTypesConsumed { get; }
   
       bool IsInterestingData(IData value);
   
       Task ConsumeAsync(IDataProducer dataProducer, IData value, CancellationToken cancellationToken);
   }

2. Add a new extensibility point to register blocking data consumers. Users should be able to do builder.TestHost.AddBlockingDataConsumer(serviceProvider => new MyBlockingDataConsumer(serviceProver))

3. The implementation of IMessageBus.PublishAsync should consider such data consumers. When the data is published, it needs to be immediately be consumed.

4. The IsInterestingData is a perf optimization. We don't want such blocking extension to slow us down for every data type being consumed (e.g, TestNodeUpdateMessage). Instead, only block when it's really interested in this specific TestNodeUpdateMessage (e.g, it has in-progress state)

5. If the data being published is not interesting for any registered IBlockingDataConsumer (or there are no registered IBlockingDataConsumers at all), we maintain today's behavior.

[Evangelink]

I don't think we need a special registration API, we should instead just special case the IBlockingDataConsumer. I would also make IBlockingDataConsumer implement IDataConsumer. I would also rename IsInterestingData into ShouldConsume.

I recall that we did considered the filter in past with @MarcoRossignoli and ended up not implementing it as the call was done anyways so it was better to simply let the ConsumeAsync callback return quickly.

Thinking a bit more about this design, I think this is similar here as we would anyways need to wait for ShouldConsume to say if that's good or not before calling ConsumeAsync so we should instead just call ConsumeAsync.

[Youssef1313]

@Evangelink So, I think that makes it simply that IBlockingDataConsumer is a marker interface.

[MarcoRossignoli]

I think we need to discuss more about it, we don't control the user code and it's legit that an adapter for performance reason to do something like(pseudo):

queue.Add(Task.Run(() => mtp.push));
...
queue.Wait();
return end session

to guarantee what you described using the current infrastructure means introduce a sync context that's not easy to digest for developers.

[nohwnd]

Do we have extensions or proposals of extensions that need this? It might be easier to design this around them.

[Youssef1313]

There is the VideoRecorder which is available for VSTest.

[Youssef1313]

@drognanar Maybe LUT as well?

[drognanar]

Both LUT and VideoRecorder need an ability to update properties on the received TestNodes.
VideoRecorder wants to be able to add a file attachment for the videos, while LUT wants to add the coverage data to the TestNode.

In my original draft the interface returned Task<IEnumerable<IProperty>> ConsumeNodeUpdatesAsync(...), as this would allow to achieve ability of additional attachments.

[svrooij]

What is the progress on this issue?

We got our developer community issue closed, because it apperantly needs the feature described here. https://developercommunity.visualstudio.com/t/Live-Unit-Testing-doesnt-work-with-TUni/11026866

I have no clue what testfx is, didn't know testing was a different repo. Anyway related to TUnit not working in live unit testing

[ememadegbola]

Another user of Live Unit Testing chiming in to say that progress on this would be much appreciated. At the moment we're having to pick between the benefits of MTP and LUT