xunit warnings: Skipping test case with duplicate ID

[ViktorHofer]

We are currently hiding thousands of warnings which indicate duplicate test data in Theories that are ignored.

Switching from xunit.console to dotnet test/vstest displays those warnings by default. I think we should at least investigate if the warnings are valid and consider fixing them / removing the duplicate theory data.

Test branch with dotnet vstest for local validation: https://github.com/ViktorHofer/corefx/tree/DotNetTest
Related: dotnet/arcade#3333

Attaching the xunit log from System.Text.RegularExpressions.Tests: xunit-warnings.txt

cc @danmosemsft @safern

[stephentoub]

Isn't this a dup of https://github.com/dotnet/corefx/issues/39649?

[Gnbrkm41]

some of them might be analyzer bug:
xunit/xunit#1877

though, there are some actual duplicates 🙃, so I think it might be worth checking.

[ViktorHofer]

This isn't a dup, see issue title. This is about duplicate inline/member data.

Right, there are definitely duplicates but I don't know yet if some of those are false positives. Nevertheless, they need to be fixed or silenced before we can switch to VSTest.

[ViktorHofer]

Duplicate test cases are fully addressed. The remaining warnings are because of xunit/visualstudio.xunit#161. Closing in favor of that issue.

[stephentoub]

Duplicate test cases are fully addressed

Not quite. I suppressed the analyzer in a bunch of cases where it was wrong, in particular it looked like where it couldn't tell the difference between values like 0.0 and -0.0.

[ViktorHofer]

Can you please open an issue in xunit?

[stephentoub]

I don't think it's really an xunit issue. They're considered equal even though their binary representations differ.

[Gnbrkm41]

Not quite. I suppressed the analyzer in a bunch of cases where it was wrong, in particular it looked like where it couldn't tell the difference between values like 0.0 and -0.0.

Just noticed the 0.0 and -0.0 thing while passing by; Isn't that more of a edge case? the IEquatable<float>/IEquatable<double> implementations do actually compare positive/negative zeros equal although their binary representations differ. This is per IEEE 754 spec I believe.

Unless the analyzers are designed to special case those values (NaN, +/- zeroes), I would think that that behaviour is expected, although undesired.