Disable stfldstatic2_r, Box_Unbox_volatile on osx-arm64

[BruceForstall]

Tracking: #91647, #91648

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Tracking: #91647, #91648

Author:	BruceForstall
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[BruceForstall]

/azp run runtime-coreclr outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[BruceForstall]

@EgorBo @dotnet/jit-contrib PTAL

[trylek]

LGTM, thanks Bruce!

[BruceForstall]

@trylek @markples @jkoritzinsky @dotnet/runtime-infrastructure One of the big problems with our merged test system is that if one test fails and brings down the run (e.g., crashes, or hits a JIT/VM assert), all the tests after it don't run at all. This PR is an example of how problematic that is. There were failing tests. I submitted a PR to disable them and make outerloop (and many other pipelines) clean. But then I get a failure on one of the tests that previously didn't run.

I notice in previous runs the text "30/211 tests run". With this PR, it's "80/211 tests run". There's no telling how many times I would need to (1) add new test to exclude, (2) rerun outerloop testing, (3) see new failure, (4) repeat; before the pipeline is clean again. I don't have access to an osx-arm64 box to do this more quickly (nor should I need to have such access).

Even if these tests were marked as "Known Issues", it would mean losing coverage on many un-run tests.

Is there a plan to fix this problem?

[BruceForstall]

#91652 should fix the bug, so closing.

[agocke]

@BruceForstall I believe we have a command or attribute that can run a specific test out of process. Can we modify such tests to run out-of-proc temporarily?

[BruceForstall]

@BruceForstall I believe we have a command or attribute that can run a specific test out of process. Can we modify such tests to run out-of-proc temporarily?

That doesn't solve the problem, since I don't know which tests to mark (it's the same problem as knowing which tests to disable).

One option would be to (1) disable merged tests for all merged groups that had such a failure, (2) re-run the pipeline, (3) disable all tests that failed and re-enable merged groups.

This is not a viable option; I don't think we have the ability right now to "un-merge".

The real solution is certainly to ensure that all tests that should run, do run (and have their results reported).

[agocke]

Isn't that equivalent to saying that every test needs to run in its own process?

[BruceForstall]

Isn't that equivalent to saying that every test needs to run in its own process?

No.

One option that has been discussed: a "watcher process". If a merged process fails, the "watcher" notices where it failed, and restarts it, starting after the failing test.

fwiw, our "PMI" JIT testing tool has done this for many years.

[trylek]

The "watcher process" is what I also remember from the last time we discussed this. At some point it seemed to be part of the "merge on red" plan but IIRC I was later told it just didn't fit in its time schedule. I still believe it's the most straightforward way to fix this issue.