-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Help compiler enforce nullability annotations #32090
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| public bool TryGetValue(T equalValue, out T actualValue) | ||
| { | ||
| int hashCode = _equalityComparer.GetHashCode(equalValue); | ||
| int hashCode = _equalityComparer.GetHashCode(equalValue!); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 should we remove the [DisallowNull] on that GetHashCode, or add [DisallowNull] here?
(same question below) #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general IEqualityComparer<T>.GetHashCode (e.g. StringComparer.GetHashCode throws ArgumentNullException for null) even if some implementations do, so we can't remove [DisallowNull]. We could consider adding [DisallowNull] to this T equalValue, but then we'd be saying that null wasn't allowed even if the developer provided a comparer implementation that did actually allow null. This is an example where we lack sufficient expressivity. I guess for now it'd be best to add [DisallowNull], and then see what kind of issues that causes; we can remove [DisallowNull] later on, but it'll be harder to add it later on. #Resolved
| { | ||
| multipleMatches = multipleExports; | ||
| // TODO2 singleMatch dould be null when returning true | ||
| singleMatch = null!; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)] annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please remove it. Thanks. #Resolved
| private static bool TryGetCastFunction(Type genericType, bool isOpenGeneric, Type[] arguments, [NotNullWhen(true)] out Func<Export, object>? castFunction) | ||
| { | ||
| castFunction = null; | ||
| castFunction = null!; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)] annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please remove it. The dangers of the compiler not having checked those annotations ;) #Resolved
| if (this.IsGeneric()) | ||
| { | ||
| singleMatch = null; | ||
| singleMatch = null!; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 the [NotNullWhen(...)] annotation on method seems incorrect #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto #Resolved
src/libraries/System.Linq.Expressions/src/System/Linq/Expressions/Interpreter/LightCompiler.cs
Show resolved
Hide resolved
| int IComparer<DebugInfo>.Compare([AllowNull] DebugInfo d1, [AllowNull] DebugInfo d2) | ||
| { | ||
| if (d1.Index > d2.Index) return 1; | ||
| if (d1!.Index > d2!.Index) return 1; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 IComparer declares that it can compare null inputs, but this implementation doesn't handle. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's only ever used with non-null values: https://source.dot.net/#System.Linq.Expressions/System/Linq/Expressions/Interpreter/LightCompiler.cs,233 #Resolved
| } | ||
| WriteAssert(stackTrace, message, detailMessage); | ||
| FailCore(stackTrace, message, detailMessage, "Assertion failed."); | ||
| #nullable disable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 without this suppression, the compiler points out that this method could return normally. Looking at the implementation of FailCore, I think it has the same problem.
We need to discuss what is the best way to suppress this warning. Maybe a more fined-grained suppression on a specific diagnostic would be better than the coarse #nullable disable I used... #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're nullable disabling/enabling around the closing brace?
Regardless, yes, it's possible for someone to implement Fail in a way that still returns, but that goes against the purpose of the method. If we held true to the possibility that someone did that, then we'd need to remove [DoesNotReturnIf(false)] from Debug.Assert and all such methods, which would be terrible. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we add DoesNotReturn to FailCore? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@safern FailCore has return statements, so the same warning "method should not return" would be produced there if we add [DoesNotReturn]. #Resolved
| public bool Equals([AllowNull] RoAssemblyName other) | ||
| { | ||
| if (Name != other.Name) | ||
| if (Name != other!.Name) // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 not sure what to do here #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did we need to make it nullable? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an internal type, and presumably it's not expecting Equals to be used with null, even though it implements an interface that allows it. Seems like the right change is to remove this ! and ? and suppress the warning on the Equals method signature. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To implement IEquatable you must be compatible with the signature bool Equals([AllowNull] T other); #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To implement IEquatable you must be compatible with the signature bool Equals([AllowNull] T other);
I understand that. So I'm saying use #pragma warning disable whatever to suppress the warning. #Resolved
src/libraries/System.Runtime.Extensions/ref/System.Runtime.Extensions.cs
Show resolved
Hide resolved
| { | ||
| OperationResult result; | ||
| int hashCode = origin.EqualityComparer.GetHashCode(item); | ||
| int hashCode = origin.EqualityComparer.GetHashCode(item!); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But now it seems like that would actually not just be for TryGetValue, but really all operations on the hash set, at which point we'd be better off using a notnull constraint? Maybe instead of changing the annotations, open an issue about it and keep the ! changes you have here. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(And actually, the right answer is probably to do a null check prior to calling GetHashCode, as we do in HashSet<T>.) #Resolved
| } | ||
|
|
||
| public bool Equals(T x, T y) | ||
| public bool Equals([AllowNull] T x, [AllowNull] T y) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since T is constraint to class, can we instead annotate T as T? ? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed. I'd missed that. Thanks #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum, it looks like this file is compiled into more than one project. One with nullability enabled and one without. Reverted to use the attribute. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use the annotation and just add #nullable enable at the top of the file as well? That's what we've done in other cases like this, where we've not yet gotten to every project that includes it. #Resolved
...braries/System.Linq.Parallel/src/System/Linq/Parallel/QueryOperators/Unary/ForAllOperator.cs
Show resolved
Hide resolved
| { | ||
| Add(comparer != null ? comparer.GetHashCode(value) : (value?.GetHashCode() ?? 0)); | ||
| // TODO2 | ||
| Add(comparer != null ? comparer.GetHashCode(value!) : (value?.GetHashCode() ?? 0)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As with the earlier examples with ImmutableHashSet, we should probably fix this implementation. e.g. today if you wrote code like:
#nullable enable
using System;
class Program
{
static void Main(string[] args)
{
HashCode c = default;
c.Add(null, StringComparer.Ordinal);
}
}it will blow up. Seems like this line should instead be:
Add(value is null ? 0 :
comparer != null ? comparer.GetHashCode(value) :
value.GetHashCode());or something like that. #Resolved
| TManager? localManager; // Use register for null comparison rather than byref | ||
| manager = localManager = memory.GetObjectStartLength(out _, out _) as TManager; | ||
| return localManager != null; | ||
| return localManager != null; // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's wrong here? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The compiler doesn't know that manager == localManager, so it looks like we may be returning with a null manager when this method returns true.
We could fix that by doing return manager != null;, but that defeats the purpose indicated by above comment ("Use register for null comparison rather than byref"). #Closed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, so this TODO was about improving the compiler :)
For the purposes of this change, just suppress the warning with whatever syntax is least obtrusive. #Resolved
| Debug.Assert(length >= 0); | ||
|
|
||
| if (localManager == null) | ||
| if (localManager == null) // TODO2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's wrong here? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto #Closed
|
|
||
| using System.Diagnostics.CodeAnalysis; | ||
| using System.Runtime.CompilerServices; | ||
| #nullable enable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: mind moving this above the usings? #Resolved
| public bool TryGetValue(T equalValue, out T actualValue) | ||
| { | ||
| int hashCode = _equalityComparer.GetHashCode(equalValue); | ||
| int hashCode = equalValue is object ? _equalityComparer.GetHashCode(equalValue) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've generally not used "is object" as a null check.
| int hashCode = equalValue is object ? _equalityComparer.GetHashCode(equalValue) : 0; | |
| int hashCode = equalValue != null ? _equalityComparer.GetHashCode(equalValue) : 0; | |
| ``` #Resolved |
| { | ||
| OperationResult result; | ||
| int hashCode = origin.EqualityComparer.GetHashCode(item); | ||
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| ``` #Resolved |
| { | ||
| var result = OperationResult.NoChangeRequired; | ||
| int hashCode = origin.EqualityComparer.GetHashCode(item); | ||
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| ``` #Resolved |
| private static bool Contains(T item, MutationInput origin) | ||
| { | ||
| int hashCode = origin.EqualityComparer.GetHashCode(item); | ||
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| int hashCode = item is object ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| int hashCode = item != null ? origin.EqualityComparer.GetHashCode(item) : 0; | |
| ``` #Resolved |
| { | ||
| if (second == null) | ||
| { | ||
| Debug.Assert(value is object); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do we know this assert is valid? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The two callers get their values from ComposablePartDefinition.TryGetExports. That method can return with
- both
singleMatchandmultipleMatchesbeing null (return false;, but thenFastAppendToListAllowNullswon't be called because of calling pattern, copied below), - or
multipleMatchesbeing set and returning true, - or
singleMatchbeing set and returning true.
singleMatch corresponds to parameter value and multipleMatches corresponds to parameter second here.
Calling pattern:
if (part.TryGetExports(definition, out Tuple<ComposablePartDefinition, ExportDefinition>? singleMatch, out IEnumerable<Tuple<ComposablePartDefinition, ExportDefinition>>? multipleMatches))
{
exports = exports.FastAppendToListAllowNulls(singleMatch, multipleMatches);
}In reply to: 378614963 [](ancestors = 378614963)
| foreach (T t in list) | ||
| { | ||
| h ^= (h << 5) ^ cmp.GetHashCode(t); | ||
| if (t is object) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| if (t is object) | |
| if (t != null) | |
| ``` #Resolved |
| int IComparer<DebugInfo>.Compare(DebugInfo d1, DebugInfo d2) | ||
| int IComparer<DebugInfo>.Compare(DebugInfo? d1, DebugInfo? d2) | ||
| { | ||
| Debug.Assert(d1 is object && d2 is object); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Debug.Assert(d1 is object && d2 is object); | |
| Debug.Assert(d1 != null && d2 != null); | |
| ``` #Resolved |
| Debug.Assert(_comparer != null); | ||
| return _comparer.GetHashCode(x.Value); | ||
| T value = x.Value; | ||
| return value is null ? 0 : _comparer.GetHashCode(value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| return value is null ? 0 : _comparer.GetHashCode(value); | |
| return value != null ? 0 : _comparer.GetHashCode(value); | |
| ``` #Resolved |
src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/MemoryMarshal.cs
Show resolved
Hide resolved
| throw new ArgumentException(SR.Format(SR.DeserializeWrongType, type, value.GetType())); | ||
| } | ||
|
|
||
| [DoesNotReturn] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am curious, how did you catch this typo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A new version of the compiler checks method bodies to see that they honor the nullability attributes that they declare in their API. This method has return statements, so it was flagged ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
|
I believe this is ready to merge/squash if I can get the required sign-off(s). Thanks |
|
Thanks, Julien. |
|
Thanks! |
TL;DR: the goal for this PR is to confirm that some upcoming compiler changes for VS 16.6 are okay.
This PR is intended for discussion but not to merge as-is :-)Tagging @stephentoub @safern for discussion.
Background:
I have been working on compiler changes which adds some warnings as a result of nullability attributes.
For instance:
[MaybeNull] string parameterwould start with a maybe-null state.[MaybeNull] override T M()cannot overridevirtual T M()[MaybeNullWhen(true)] string parameter[DoesNotReturn]indeed don't returnJared recommended that I test the change on the
runtimerepo, and it indeed produced more warnings. Here's a PR that addresses some of the issues flagged and tags some others.Issues identified so far:
GetHashCodemethods are marked as[DisallowNull]and that's causing problemsFailFastis marked with[DoesNotReturn], but it looks like it could return in some cases[Maybe/NotNullWhen(...)]attributes seem incorrectly implemented