Execute TypeManager timers in the correct runtime context

[ReubenBond]

In Silo.DoStart() we should be scheduling TypeManager.Initialize() on the TypeManager's context (it's a system target), since it starts a timer which potentially makes grain calls.

The timer started there also needs to invoke its callback on the TypeManager's context.

Otherwise errors (see stack):

[benjaminpetit]

Makes sense, but I am not sure why so many tests failures...

[ReubenBond]

Not sure why the tests are failing - I'll look into it.

[ReubenBond]

In ChangeDefaultVersionCompatibilityStrategy, if I insert a delay between starting the v2 silo and getting a grain reference, then the test passes:

await StartSiloV2();

await Task.Delay(TimeSpan.FromMinutes(2));

var grainV2 = Client.GetGrain<IVersionUpgradeTestGrain>(2);

Any idea why putting a delay in that particular location might help?

[benjaminpetit]

Any idea why putting a delay in that particular location might help?

When we start a new silo in an existing cluster, we wait TestCluster.GetLivenessStabilizationTime. It doesn't seems to be enough here. On my machine I am not able to reproduce this race condition?

[benjaminpetit]

@dotnet-bot test this please

[benjaminpetit]

Could you try to add some delay after we started a new silo? It really seems that the value returned by TestCluster.GetLivenessStabilizationTime is not enough.

But I don't understand why it was more reliable before.

[sergeybykov]

@dotnet-bot test netstandard-win-functional
@dotnet-bot test netfx-bvt
@dotnet-bot test netfx-functional

[ReubenBond]

@benjaminpetit I've added a 90s wait there now, maybe those tests will be happier.

[sergeybykov]

90s is an eternity. That shouldn't be required for running these tests correctly. Something must be wrong there.

[sergeybykov]

I submitted ReubenBond#7 for this PR, which I hope will fix the test failures.

[benjaminpetit]

@sergeybykov the problem with your commit is that it defeats the purpose if this test: if we change the default at runtime, are the silo that are joining the cluster taking this new default value?

@ReubenBond It seems that it is still a timing issue, but I don't understand why we didn't had this issue before

[sergeybykov]

Closing this one, as #3309 is a replacement for it.