linker throws InvalidCastException when encountering extension user-defined compound assignment operators

[tannergooding]

From: #117997 (comment)

Somewhere in trimDataFlowAnalysis.InterproceduralAnalyze();, an InvalidCastException is thrown due to an attempt to cast NonErrorNamedTypeSymbol to IMethodSymbol.

This is encountered at least for a user-defined compound assignment operator extension:

extension<TScalar>(Tensor<TScalar> tensor)
{
    public void operator +=(in ReadOnlyTensorSpan<TScalar> other) => Add(tensor, other, tensor);
    public void operator +=(TScalar other) => Add(tensor, other, tensor);
}

I was not able to get a stack trace and root cause exactly where the cast failure was occurring.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/illink
See info in area-owners.md if you want to be subscribed.

[tannergooding]

This is specifically happening in TrimAnalysisVisitor.VisitParameterReference when creating the ParameterProxy which assumes it exists on a method.

The parameter is indeed from an extension member parameter, such as extension<T>(ref T value) { }, specifically it is returning the ref T value where the ContainingSymbol is an ITypeSymbol representing the extension.

As a workaround, my PR (#117997) currently has this change:

 public override MultiValue VisitParameterReference(IParameterReferenceOperation paramRef, StateValue state)
 {
     IParameterSymbol parameter = paramRef.Parameter;

+    if (parameter.ContainingSymbol is not IMethodSymbol)
+    {
+        // TODO: Extension members allows parameters to be on types, rather than methods.
+        // For example: `extension<T>(ref T value) { }` will enumerate `value` where
+        // the containing symbol is a `NonErrorNamedTypeSymbol`
+        return TopValue;
+    }

     // Reading from a parameter always returns the same annotated value. We don't track modifications.
     return GetParameterTargetValue(parameter);
 }

This isn't really an ideal change as the scenario should likely get handled properly. But this at least avoids the analyzer crashing right now. I'd appreciate weigh-in from the linker team on if there is a more appropriate workaround so the tensor operator work isn't block for the time being.