Post-2.1 plan of Intel hardware intrinsic

[fiigii]

1. Refine the API design of Intel hardware intrinsic
   • related issues/PRs: Improve Intel hardware intrinsic APIs coreclr#17637, https://github.com/dotnet/corefx/issues/29247
2. Implement remaining AVX2 intrinsic [rely on (1)]
   • related issues/PRs: https://github.com/dotnet/coreclr/issues/16653
3. Implement remaining SSE4.2 intrinsic [rely on (1)]
   • related issues/PRs: https://github.com/dotnet/coreclr/issues/16270
4. Enable containment analysis on more hardware intrinsic forms (e.g., imm, 1-arg, 3-arg, etc.)
5. Implement FMA intrinsic [rely on (4)]
   • FMA intrinsic codegen is different from other ISAs whose instruction selection depends on the operator's position (e.g., in registers or memory?)
6. Implement other ISA classes (Bmi1, Bmi2, Aes, and Pclmulqdq)
   • fully support all the Intel hardware intrinsic of existing APIs
7. Create non-trivial benchmarks for Intel hardware intrinsic
   • related issues/PRs: https://github.com/dotnet/coreclr/issues/17798
8. Improve the CQ of Intel hardware intrinsic base-on key scenarios [partially rely on (7)]
   • especially when intrinsics are used in combination or with things like Vectors, and identify areas that need work.
   • related issues/PRs: https://github.com/dotnet/coreclr/issues/17073, https://github.com/dotnet/coreclr/issues/17108
9. Investigate the JIT throughput impact from hardware intrinsic recognition [rely on (7)]
   • related discussion: Implement "IsSupported" for all ISA classes of Intel hardware intrinsics coreclr#14020 (comment)
10. Identify candidates that can be optimized using HW intrinsics and implement them using intrinsics (CoreFX, mscorlib, HPC, ML, etc.)
    • related issues/PRs: https://github.com/dotnet/coreclr/issues/15506, https://github.com/dotnet/corefx/issues/25386
11. more...

category:cq
theme:intrinsics
skill-level:intermediate
cost:extra-large

[fiigii]

cc @CarolEidt @AndyAyersMS @jkotas @tannergooding @mikedn @redknightlois

Please feel free to improve the above list.

[AndyAyersMS]

The goal should be to have HW intrinsics be in shape to be fully supported.

Seems like the overall priorities are good:

• finish off current implementations
• get mileage on the existing APIs to evaluate usability
• start looking at CQ, especially when intrinsics are used in combination or with things like Vectors, and identify areas that need work.

IIRC there were some not fully resolved design aspects; they should be on the list too.

[redknightlois]

There are a few pain points in the interface to deal with HW intrinsics, mostly on the StaticCast<T> front that probably could be handled through library implicit conversions and overloads.

[fiigii]

@AndyAyersMS Thank you so much for the suggestions, updated the list.

[fiigii]

mostly on the StaticCast front that probably could be handled through library implicit conversions and overloads.

Good point. But I am afraid and not sure that "implicit conversions" may confuse the front-end compiler (Roslyn) and IDE to determine which overload users actually want.
Moveover, StaticCast<T, U> is important to let HW intrinsic work with generic code (i.e., StaticCast<T, float>(Sse.Add(...)) to return a generic Vector128<T>). Does implicit conversion help to this case?

[redknightlois]

They will have to be done with care, that is either implicit and/or overloads could be the way to go. Currently StaticCast<T,U> clutters the code a bit too much. There must be some non-obvious compromise that could work.

[CarolEidt]

Although the first item captures the broad issue of API design and usability, it would be worth calling out that, beyond getting CQ data, we also want to use scenarios to drive usability. @redknightlois points out one concern, but we should also seek usability feedback more broadly.

[tannergooding]

The PR adding support for the x86 FMA intrinsics is up: dotnet/coreclr#18105

[4creators]

They will have to be done with care, that is either implicit and/or overloads could be the way to go. Currently StaticCast<T,U> clutters the code a bit too much. There must be some non-obvious compromise that could work.

One of the proposals to handle this is in https://github.com/dotnet/corefx/issues/27911#issuecomment-372684004

There are many other possible solutions which could be based on VectorNNN<T> as well.

[4creators]

@fiigii @tannergooding Whis ISA I could start working on now as I have some free cycles?

[fiigii]

IIRC, SSE2 still has a few issues (e.g. CQ, tests, etc.), it would be better to refine SSE2 code at first.

[zpodlovics]

@fiigii Could you please also add PAUSE as intrinsics? The PAUSE instruction target is high performance / low latency intra-thread communication, lot's of energy could be saved with a small latency increase if these communicating tight loops could use a single PAUSE instruction instead of busy spinning. This is critical as we have more and more cores and multiple numa nodes within a server.

Could you please also check the explicit cache control intrinsics (CLFLUSH, CLFLUSHOPT, CLWB, CLZERO(?), SFENCE) as persistent memory become mainstream.

Original issue:
https://github.com/dotnet/coreclr/issues/916#issuecomment-366562117

[svick]

@zpodlovics

Could you please also add PAUSE as intrinsics?

As far as I can tell, calling Thread.SpinWait(1) on x64 will invoke pause enough times to wait 1 ns. Is that not good enough for you?

[zpodlovics]

@svick Unfortunately not. Thread.SpinWait(1) still still doing lot's of unwanted calculation, branches and calls instead of a single (or a few, but still fully controlled) PAUSE instruction within in tight loop. The cpu micro-op cache also plays a critical role here. I have benchmarked / tested every wait method available, in almost every possible way. If you do non-blocking intra-thread communications every call, every branch and every instruction counts, and I am talking about > 100M msg/sec rate between threads. I would like to improve the energy consumption a bit by using a single (or a few but still fully controlled) PAUSE instruction as intrinsics within a loop, instead of doing full busy loop.

And almost every other intrinsics are available, why PAUSE is not?

[redknightlois]

@svick For very high throughput low latency operations nothing beats calling PAUSE. Not on that level yet, but it is important to have it when dealing with that kind of stuff.