Improve/cleanup RyuJIT's IR handling of operand lists

[mikedn]

Some IR nodes require a variable number of operands and the current implementation uses linked lists of GT_LIST (or similar) nodes.

IR nodes knows to use such lists:

• GT_CALL - has a dedicated this argument operand and a list of operands for all other arguments
• GT_PHI - this is a GTK_UNOP node and its sole operand is a list
• GT_HWINTRINSIC - this is a GTK_BINOP node but if needs 3 operands or more the first operand is a list and the second is null
• GT_FIELD_LIST - a GTK_BINOP that acts both like an actual IR node or like a list node
• Anything else?

From the above only GT_CALL is reasonable, though it could probably be improved as well. The rest suffer from various issues:

• GT_PHI - this looks similar to GT_CALL on the surface but it lacks the special handling of calls throughout the JIT. Its list nodes end up in the linear order and they are visited by tree walking facilities like GenTreeVisitor. This is nonsense - these list nodes are an internal implementation detail of the GT_PHI node and they should not appear anywhere. They have no type, no VN, no nothing. They simply do not exists as far as the IR is concerned.
• GT_HWINTRINSIC suffers from the same problem as GT_PHI. It just makes it worse by sometimes using lists and sometimes not. And unlike GT_PHI, this lacks special handling in gtDispTree and dumps its list nodes.
• GT_FIELD_LIST IMO it would be better have a GT_PACK node with a list of GT_FIELD_LIST nodes.

• Stop using GT_LIST in GT_PHI ([Feedback] Can not update to .NETCore 1.1.0 after upgrading to Microsoft.NETCore.UniversalWindowsPlatform v5.3.0 #20266)
• Stop using GT_LIST in GT_CALL (dotnet-runtime-2.0.0 installation fails in centos 7.3 #26392)
• Stop using GT_LIST in GT_FIELD_LIST (Calling 3rd party DLL that makes calls out to the internet from .Net core fails proxy authentication despite configuring or coding the required settings #26800)
• Stop using GT_LIST in GT_SIMD ([WIP] Stop using LIST nodes for SIMD operand lists #1141)
• Stop using GT_LIST in GT_HWINTRINSIC
• Remove all the remaining GT_LIST code

category:implementation
theme:ir
skill-level:expert
cost:extra-large

[mikedn]

At a minimum, we should treat such lists consistently - if call's list nodes aren't visited by GenTreeVisitor & co. then the same should happen for PHIs and others. Same for adding list nodes to the linear order.

And once you stop traversing list nodes the next question comes: do they need to be GenTree nodes? A simple list node would have 2 pointers - next and op - 16 bytes. GenTreeArgList has 80 bytes. So every PHI argument needs 80 * 2 bytes. Hrm.

[mikedn]

From the above only GT_CALL is reasonable

Actually it is not. fgSetTreeSeqHelper does add the GT_LIST nodes used by GT_CALL to the linear order. It's GenTreeVisitor that does not visit them.

[ArtBlnd]

I hope just adding GT_ZEXT, GT_SEXT than using cast and shifts for better optimization slots.
Its really hard to know that is extends or just an cast or shift.

[mikedn]

I hope just adding GT_ZEXT, GT_SEXT than using cast and shifts for better optimization slots.
Its really hard to know that is extends or just an cast or shift.

I'm not sure what that has to do with this issue. And while it's true that cast nodes are kind of cumbersome they're no where near as bad as some other "legacy" issues the JIT has. That is, it's unlikely that anyone would spend a lot of time attempting to change how casts are represented in the IR.

[mikedn]

And once you stop traversing list nodes the next question comes: do they need to be GenTree nodes? A simple list node would have 2 pointers - next and op - 16 bytes. GenTreeArgList has 80 bytes. So every PHI argument needs 80 * 2 bytes. Hrm.

Tried to change PHI to use its own linked list nodes instead of GT_LIST. Works fine, there's simply no need to use GT_LIST and waste both memory and time (to link list nodes in linear order).

Same for GT_FIELD_LIST. The only weird thing about it is that since the field list head is also an IR node the IR node type is the same as the first field's type. You'd think that the IR node type is struct.

It's very likely that calls also do not need to use GT_LIST nodes. There are a few list dependencies here and there but nothing that seems strictly necessary. Quite the contrary in fact.

HW intrinsics are a bit of a problem. Since their op1 is either a list or a real node it's more difficult to avoid using GT_LIST. But if it turns out that HW intrinsics are the only nodes that truly need GT_LIST then there must be a way to change that.

[ArtBlnd]

I'm not sure what that has to do with this issue. And while it's true that cast nodes are kind of cumbersome they're no where near as bad as some other "legacy" issues the JIT has. That is, it's unlikely that anyone would spend a lot of time attempting to change how casts are represented in the IR.

Maybe I'll do that? :> I can see really lots of issues with casting(in performance issues). In my opinion, it will be not hard as expected. the legacy shifts and casts will work fine. we can just add phase(or just in morph phase?) shift and cast to SEXT or ZEXT to know another phase that its a ZEXT or SEXT casting. in another way make some cast to generate SEXT or ZEXT. It means we don't need to make it harder, but we can still remove that casts much more easily.

[ArtBlnd]

And also, I really agree with GT_PHI issues. that makes really nonsense. maybe we can make it as BasicBlock. seems it much reasonable than its GenTree(in current GenTree). The SSA theory is fits on current BasicBlock more than GenTree Its something like needs to have body, head, latches. It will be more reasonable when its BasicBlock or something like GT_STMT type.

[mikedn]

I can see really lots of issues with casting(in performance issues)

If you're seeing specific issues with casts then feel free to open relevant issues. Some already exist. I also have a PR in progress to fix some. But I doubt that whatever issue you are seeing requires adding additional operators.

maybe we can make it as BasicBlock. seems it much reasonable than its GenTree(in current GenTree). The SSA theory is fits on current BasicBlock more than GenTree Its something like needs to have body, head, latches. It will be more reasonable when its BasicBlock or something like GT_STMT type.

I'm not really sure what you are trying to say. But this issue has nothing to do with SSA "theory", turning the RyuJIT IR model upside down, rewriting RyuJIT, copying LLVM or any other compiler and other stuff like that.

This issue is solely about some inconsistencies in the handling of GT_LIST and the possibility of removing it from the IR.

[ArtBlnd]

I meant, the GT_LIST can be replaceable with multiple GenTree with BasicBlock. and make the BasicBlock as like GT_PHI(for example) node.(or marked as list on the BBF) and by calling it with some another GenTree or etc. This seems more make sense than using GT_LIST for GT_PHI and also for SSA theory which to be GT_PHI works. seems much clean way to be work fine with it.

or just nevermind. (I think I misunderstood what you saying)
maybe we can just replace it as next GenTree something like gtNext, gtPrev. just next to GT_PHI, GT_CALL or something.

I think the best way is making SmallVector for it. which uses memory statically when its small enough so make it normally small when its 2. and expand it when its needed. and keep that GetOp1() or something else. and just make GetOp(unsigned int index). so keep that GT_UNOP, GT_OP. and make GT_OP_LIST seems reasonable.

[ArtBlnd]

If you're seeing specific issues with casts then feel free to open relevant issues. Some already exist. I also have a PR in progress to fix some. But I doubt that whatever issue you are seeing requires adding additional operators.

Thats almost all of IV Widening. anyways. I mean the cost to remove that cast will be really heavy. so giving them flags, and make it GT_ZEXT, GT_SEXT. so not to iterate over and over again to find it. it will be much easier to remove that cast. also for future optimization slots.

[mikedn]

I meant, the GT_LIST can be replaceable with multiple GenTree with BasicBlock

What you say doesn't make sense. GT_LIST is already a GenTree. A list is multiple GenTrees. None of this has anything to do with BasicBlock.

And in general it's not a problem replacing GenTree based lists with "normal" linked lists or arrays or anything suitable. The problem is what effect such a change will that have on JIT, what will break, what needs fixing, assuming that it's feasible to fix in the first place. For all I know, there could be some hard dependency hidden somewhere in the JIT that turns out to be difficult to remove.

This seems more make sense than using GT_LIST for GT_PHI and also for SSA theory which to be GT_PHI works. seems much clean way to be work fine with it.

This has nothing to do with SSA "theory". GT_LIST is just a technical detail and it's used by nodes other than PHIs.

Thats almost all of IV Widening. anyways. I mean the cost to remove that cast will be really heavy. so giving them flags, and make it GT_ZEXT, GT_SEXT. so not to iterate over and over again to find it. it will be much easier to remove that cast. also for future optimization slots.

I don't see how having GT_ZEXT/SEXT makes IV widening easier than having GT_CAST. One way or another, a cast is a single node. And again, this has nothing to do with GT_LIST.

[ArtBlnd]

What you say doesn't make sense. GT_LIST is already a GenTree. A list is multiple GenTrees. None of this has anything to do with BasicBlock.

And in general it's not a problem replacing GenTree based lists with "normal" linked lists or arrays or anything suitable. The problem is what effect such a change will that have on JIT, what will break, what needs fixing, assuming that it's feasible to fix in the first place. For all I know, there could be some hard dependency hidden somewhere in the JIT that turns out to be difficult to remove.

Yes I think I just misunderstood what you saying. sorry for that :<

I don't see how having GT_ZEXT/SEXT makes IV widening easier than having GT_CAST. One way or another, a cast is a single node. And again, this has nothing to do with GT_LIST.

Thats what I misunderstood.
I thought "Improve/cleanup RyJIT's IR handling of operand lists" as "handling lists of improve/cleanup RyuJIT's IR operands"(something like TODO list.) this mean. I just totally misunderstood. and again. sorry for make you feels bad.

And just btw. GT_CAST has with shifts. that the problem. only with GT_CAST doesn't really matters.
the problem is GT_CAST after shifting. that makes really hard remove shifts with that cast.

[mikedn]

HW intrinsics are a bit of a problem. Since their op1 is either a list or a real node it's more difficult to avoid using GT_LIST. But if it turns out that HW intrinsics are the only nodes that truly need GT_LIST then there must be a way to change that.

GT_HWINTRINSIC/GT_SIMD nodes need to be GTK_SPECIAL and have a list/array of operands. Having them be GTK_BINOP/GenTreeOp when they can have 1-32 operands doesn't help, quite the opposite. This basically subverts the meaning GTK_BINOP and requires all IR traversal utilities to add special casing to handle the GT_LIST operand. So if it's special then let's make it special.

Unlike GT_PHI and GT_FIELD_LIST, SIMD nodes tend to require random access to operands so it's probably best to use an array rather than a linked list of uses. Up to 3 operands fit an "inline" array, for more than 3 nodes we'll need a separate array.

The only question is what to do about GTF_REVERSE_OPS, it may be useful for it to still work on SIMD nodes having exactly 2 operands. But evaluation order is relevant for any number of operands higher than 1 so perhaps an alternative mechanism should be used. Anyway GTF_REVERSE_OPS is very much a legacy JIT32 thing (and a pretty horrible thing).

[tannergooding]

when they can have 1-32 operands doesn't help

I think we are in a slightly better position here now.

Previously, we had the SetVector128/256 methods that were treated as intrinsic and they would have one operand per element. That has since changed and those methods are implemented in software (generally by calling existing intrinsics like Insert). So, I believe we now only have GT_HWINTRINSIC nodes that are 1, 2, or 3 operands. Ah, and there is also the Avx2.GatherMaskVector128/256 intrinsic that takes 4 operands.

So I think (at least for the HW_INTRINSIC nodes) the only time we need more than 4 is if/when support for emitting constants (https://github.com/dotnet/coreclr/issues/17225) is added; and that would likely only be needed if we wanted to support inputs that were determined to be constant post importation (similar to https://github.com/dotnet/coreclr/issues/17108 -- but maybe someone has a good idea on how to handle this).

I don't think we can workaround having 3 operands (since things like FMA have three distinct operands), but we might be able to do something about the 4 operand GatherMaskVector case since the actual instruction ends up combing 3 of the operands into a Vector SIB Memory Address

[mikedn]

Ah, and there is also the Avx2.GatherMaskVector128/256 intrinsic that takes 4 operands.

Yes, and 4 is already more that I can fit in a small node.

similar to #9989 -- but maybe someone has a good idea on how to handle this

Hmm? But the constant problem is already solved in dotnet/coreclr#14393. We just need to extend that to 32 byte constants, that requires VM support to allocate 32 byte aligned data sections. Shouldn't be a big problem I guess. And perhaps we need actual constant SIMD nodes in the IR, though the cases where that would be useful are pretty rare today.

I don't think we can workaround having 3 operands (since things like FMA have three distinct operands), but we might be able to do something about the 4 operand GatherMaskVector case since the actual instruction ends up combing 3 of the operands into a Vector SIB Memory Address

Yes, but even if we get rid of gather there's also new Vector4(x, y, z, w), with or without constant support.