JIT Testing Strategies for .NET 7

[TIHan]

This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Summary

There are a variety of improvements to be made in how we troubleshoot and test the JIT. The goal is to be able to diagnose issues more efficiently, and to verify correctness of IL and its code-gen.

Planned for .NET 7

@TIHan

• Learning paper for a deep dive on optimization correctness testing strategies from other compilers

Future

• Delete IL verifier from the JIT #32648 (JIT IL verification removal)
  • Remove Verify macros in importer.cpp #64399
• InvalidProgramException is extremely unhelpful #63198 (ILVerify)

@BruceForstall

• JIT Test Monitor for daily pipeline monitoring

@EgorBo

• (User friendly JIT diagnostics)
• (Application level benchmarks)

SuperPMI improvements

Generally, SuperPMI will be improved on an "as needed" basis, and as resources are available to work on it. The following items are the most likely to get attention.

• (Bruce WIP) superpmi-diffs pipeline improvements #61483
• Create checked/release SuperPMI asm diffs AzDO pipeline #49949

Moved to .NET 8

• ILVerify fails with error [MissingMethod] if call to method uses a parameter with type from an embedded interop assembly #62732 (ILVerify)

We really should find time to implement:

• superpmi: add coreclr test run collection #47543
• Add (automated) collections for additional open source components, e.g. Roslyn and Roslyn tests, F# tests, Paint.NET
• superpmi: support Linux arm/arm64 benchmarks collection #59446
• We have ASP.NET collections, but they are manually done. Automate them. superpmi: automated TechEmpower collection #47545.

category:planning
theme:testing
skill-level:intermediate
cost:extra-large
impact:large

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Summary

There are a variety of improvements to be made in how we troubleshoot and test the JIT. The goal is to be able to diagnose issues more efficiently, and to verify correctness of IL and its code-gen.

Sections

@TIHan

• Delete IL verifier from the JIT #32648 (JIT IL verification removal)
  • Remove Verify macros in importer.cpp #64399
• ILVerify fails with error [MissingMethod] if call to method uses a parameter with type from an embedded interop assembly #62732 (ILVerify)
• InvalidProgramException is extremely unhelpful #63198 (ILVerify)
• Learning paper for a deep dive on testing strategies from other compilers (more info coming soon)

@EgorBo

• (User friendly JIT diagnostics)
• (Application level benchmarks)

@BruceForstall

• (SPMI improvements)

Author:	TIHan
Assignees:	TIHan
Labels:	area-CodeGen-coreclr, User Story
Milestone:	7.0.0

[bruce]

👋 Just dropping by to mention you've tagged the wrong Bruce on this issue.

[BruceForstall]

Closing since this is now old.