-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Translate COALESCE as ISNULL #34171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Translate COALESCE as ISNULL #34171
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -202,6 +202,53 @@ protected override Expression VisitValues(ValuesExpression valuesExpression) | |||||
| return valuesExpression; | ||||||
| } | ||||||
|
|
||||||
| /// <summary> | ||||||
| /// This is an internal API that supports the Entity Framework Core infrastructure and not subject to | ||||||
| /// the same compatibility standards as public APIs. It may be changed or removed without notice in | ||||||
| /// any release. You should only use it directly in your code with extreme caution and knowing that | ||||||
| /// doing so can result in application failures when updating to a new Entity Framework Core release. | ||||||
| /// </summary> | ||||||
| protected override Expression VisitSqlFunction(SqlFunctionExpression sqlFunctionExpression) | ||||||
| { | ||||||
| if (sqlFunctionExpression is { IsBuiltIn: true, Arguments: not null } | ||||||
| && string.Equals(sqlFunctionExpression.Name, "COALESCE", StringComparison.OrdinalIgnoreCase)) | ||||||
| { | ||||||
| var type = sqlFunctionExpression.Type; | ||||||
| var typeMapping = sqlFunctionExpression.TypeMapping; | ||||||
| var defaultTypeMapping = _typeMappingSource.FindMapping(type); | ||||||
|
|
||||||
| // ISNULL always return a value having the same type as its first | ||||||
| // argument. Ideally we would convert the argument to have the | ||||||
| // desired type and type mapping, but currently EFCore has some | ||||||
| // trouble in computing types of non-homogeneous expressions | ||||||
| // (tracked in https://github.com/dotnet/efcore/issues/15586). To | ||||||
| // stay on the safe side we only use ISNULL if: | ||||||
| // - all sub-expressions have the same type as the expression | ||||||
| // - all sub-expressions have the same type mapping as the expression | ||||||
| // - the expression is using the default type mapping (combined | ||||||
| // with the two above, this implies that all of the expressions | ||||||
| // are using the default type mapping of the type) | ||||||
|
cincuranet marked this conversation as resolved.
|
||||||
| if (defaultTypeMapping == typeMapping | ||||||
| && sqlFunctionExpression.Arguments.All(a => a.Type == type && a.TypeMapping == typeMapping)) { | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Related to the above comment on the CLR type, I think we could compare the store type instead of referencing-comparing the type mapping instances... There's no guarantee in EF that there's only one type mapping instance for a given type mapping, and unless I'm mistaken the only thing that matters here for ISNULL is the actual store type. So comparing the store type should be safe and possibly allow for some more scenarios.
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IIRC this would not recognize cases in which the mismatch is in one of the parameters (such as
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think StoreType must always be the full store type - including any facets. If that's not the case, we most probably have bugs somewhere else as well. Do you have a specific case in mind where this doesn't work?
I'm not sure... The point here is that in this specific context, the only thing we care about is the actual type in the database; in other words, various other details that distinguish type mappings from one another are irrelevant (value comparers, value converters...), since they affect EF client-side processing only. Assuming we have a 100% unambiguous representation of the database type - which is what StoreType is supposed to be - comparing those should be sufficient and allow for maximum coverage of this optimization, I think. A general IEquatable would need to also compare e.g. value comparers (since it's possible that in other contexts that's relevant), and so would needlessly exclude some valid cases... |
||||||
|
|
||||||
| var head = sqlFunctionExpression.Arguments[0]; | ||||||
| sqlFunctionExpression = (SqlFunctionExpression)sqlFunctionExpression | ||||||
| .Arguments | ||||||
| .Skip(1) | ||||||
| .Aggregate(head, (l, r) => new SqlFunctionExpression( | ||||||
| "ISNULL", | ||||||
| arguments: [l, r], | ||||||
| nullable: true, | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At least for the top-most ISNULL, shouldn't we be taking the original coalesce expression's nullability? For example, in the very common case of As this is done extremely late in the query pipeline (SQL generator), there's not going to be anything afterwards that cares about this (this node is immediately processed, generating SQL string data, and then discarded). But for correctness's sake (and in case this code is copy-pasted somewhere else), maybe we should do it?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Note that in this context we do not have access to the results of the nullability processor (the
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Right, not incorrect in the sense of incorrect results - just potentially SQL that's more convoluted (and less efficient) than it needs to be.
That's true, but the expression But anyway, given where this code is, I agree it's theoretical. If we do any extra work on this, I'd add a comment making that clear.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I believe we are not losing any information here:
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see, I wasn't aware of that... That's presumably because we have special handling of COALESCE in any case in SqlNullabilityProcessor, so IsNullable is maybe effectively unused? I'd prefer it IsNullable were accurate here (across the board, regardless of COALESCE vs. ISNULL) - you never know when someone might look at it - but I guess it's not super important at this point.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The nullability processor relies on
Your suggestion goes in the direction of tracking nullability in the expression, which might indeed prove very useful, for example when lowering SqlServer boolean expressions/predicates 😫, but I think is not specifically related to the change COALESCE->ISNULL in this PR |
||||||
| argumentsPropagateNullability: [false, false], | ||||||
| sqlFunctionExpression.Type, | ||||||
| sqlFunctionExpression.TypeMapping | ||||||
| )); | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| return base.VisitSqlFunction(sqlFunctionExpression); | ||||||
| } | ||||||
|
|
||||||
| /// <summary> | ||||||
| /// This is an internal API that supports the Entity Framework Core infrastructure and not subject to | ||||||
| /// the same compatibility standards as public APIs. It may be changed or removed without notice in | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one also seems a bit strict - if all sub-expressions have the same store type, shouldn't everything be fine and we can assume the overall expression has the same type? What's the scenario that could be problematic given a non-default store type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EFCore computes accurate store types only in some cases (column expressions, simple additions), but in many other cases it approximates them, for example a literal string is modeled as
varcharinstead ofvarchar(length-of-the-literal).The idea was that if all of the the expressions are using the default type mapping, the C#-oriented type compatibility matches the SQL one as closely as possible... but now I wonder if some cases are actually unsafe even under this assumption 🤔 .
I think an example that would break with only same type and same type-mapping is:
IIRC EFCore computes the type mapping of the LHS (and of the whole expression) as the same type mapping as
col1VarChar10.I think an example that would break with the current set of constraints is :
I will try to check this ASAP.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, the general logic is that type mappings are configured on columns, and then inferred from them to other, non-column expression types (e.g. in
b.MyCol == "some constant"), as well as bubbling up in the tree as necessary.But I'm still a bit confused: if we know that all sub-expressions have the same store type (and therefore so should the SqlFunctionExpression for the COALESCE call on top), how could we possibly have an issue, regardless of whether that type mapping happens to be a default or not? Is this because you think that with a default type mapping there's less of chance that EF got the type mapping wrong (or similar)? Would be good to see a concrete case for that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Coalesce_Correct_TypeMapping_DoubleandCoalesce_Correct_TypeMapping_Stringare a couple of examples in which EFCore computes the wrong (/inaccurate) type mapping for some of the expressions.Assuming that the implementation in this PR is wrong, I will add another test case that shows that the check is not strict enough.
Conversely, you are right that checking that the type is the default one is not required; its main purpose was to trust C# for the type computation, under the (presumably wrong) assumption that when the C#-computed type and the EFCore-computed type matched, that also meant a match for the SQL type.