StringComparison.OrdinalIgnoreCase does not consistently match ToUpperInvariant #67873
Description
Per the following two documents, comparing strings using StringComparison.OrdinalIgnoreCase is explicitly documented as equivalent to calling ToUpperInvariant on each string, then performing an ordinal comparison against the contents.
- https://docs.microsoft.com/dotnet/api/system.stringcomparer.ordinalignorecase#remarks
- https://docs.microsoft.com/dotnet/standard/base-types/best-practices-strings#ordinal-string-operations
These statements within the docs imply that ToUpperInvariant and ToLowerInvariant are ordinal case conversions ("simple case mapping"), not linguistic case conversions. However, it looks like we're not consistenly following this pattern.
string s1 = "s";
string s2 = "\u017f"; // Latin Sharp S, which uppercase-maps to a normal ASCII "S"
Console.WriteLine(s1.Equals(s2, StringComparison.OrdinalIgnoreCase)); // False
Console.WriteLine(s1.ToUpperInvariant() == s2.ToUpperInvariant()); // TrueThis has collateral impact. For example, recent PRs like #67758 assume that non-ASCII characters cannot case-map to ASCII characters, which is not a guarantee offered by Unicode, but which might be a guarantee we'd be willing to make separately within the runtime by munging the Unicode tables.
See also #30960 for further discussion on case mapping as a more general Unicode concept.
/cc @tarekgh, who had thoughts on this offline.