Test failure: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked

[scalablecory]

Error here:

https://dev.azure.com/dnceng/public/_build/results?buildId=452125

  Starting:    System.Security.Cryptography.X509Certificates.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked(pkiOptions: IssuerRevocationViaOcsp | AllEndEntityRevocation) [FAIL]
      Assert.Equal() Failure
      Expected: X509ChainStatusFlags[] [NoError, NoError, NoError]
      Actual:   X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, NoError]
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1267,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1177,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1109,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(101,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<NothingRevoked>b__8_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1239,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(97,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked(PkiOptions pkiOptions)
  Finished:    System.Security.Cryptography.X509Certificates.Tests

[AriNuer]

Similar test System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.CheckEndEntityWithExpiredRevocation failed here runtime-libraries outerloop:20200406.2
Error message:

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, RevocationStatusUnknown | OfflineRevocation]
Actual: X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, RevocationStatusUnknown | OfflineRevocation]

Stack trace:

   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1263
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RunWithInconclusiveEndEntityRevocation(ChainHolder holder, X509Certificate2 endEntity) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1043
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<CheckEndEntityWithExpiredRevocation>b__27_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 917
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1234
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.CheckEndEntityWithExpiredRevocation(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 911

Details:
https://dev.azure.com/dnceng/public/_build/results?buildId=589983&view=ms.vss-test-web.build-test-results-tab&runId=18595078&resultId=104363&paneView=debug

[bartonjs]

I see one similar failure within the last 90 days. This may mean we need to nudge the maximum allowed time up a notch, the test machines may just be a bit overloaded. But it's so intermittent that it's hard to specifically diagnose. (The fact that it depends on network timing, even for localhost, is part of why it's [OuterLoop])

[v-haren]

failed again in job: runtime-libraries outerloop 20200804.2

failed test: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked(pkiOptions: OcspEverywhere | IssuerAuthorityHasDesignatedOcspResponder)

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, NoError]
Actual:   X509ChainStatusFlags[] [NoError, NoError, RevocationStatusUnknown | OfflineRevocation]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1272
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1187
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1128
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<NothingRevoked>b__6_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 86
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1243
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 82

[v-haren]

failed again in job: runtime-libraries-coreclr outerloop 20201011.1

failed test: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked_RootRevocationOffline(pkiOptions: CrlEverywhere)

net6.0-Windows_NT-Release-x64-CoreCLR_release-Windows.81.Amd64.Open

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, NoError]
Actual:   X509ChainStatusFlags[] [NoError, NoError, RevocationStatusUnknown | OfflineRevocation]

Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1415
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.NothingRevoked_RootRevocationOffline(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 782

[v-haren]

failed again in job: runtime-libraries-coreclr outerloop 20201223.2

failed test: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeIntermediateWithExpiredRevocation(pkiOptions: IssuerRevocationViaCrl | OcspEverywhere | IssuerAuthorityHasDesignatedOcspResponder | RootAuthorityHasDesignatedOcspResponder)

net6.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, NoError]
Actual:   X509ChainStatusFlags[] [NoError, NoError, RevocationStatusUnknown | OfflineRevocation]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1415
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1330
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1271
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeIntermediateWithExpiredRevocation>b__25_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 906
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1386
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeIntermediateWithExpiredRevocation(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 893

[VincentBu]

Similar test failed in runtime-libraries-coreclr outerloop 20210321.3

Failed test:

net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
 -System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(pkiOptions: IssuerRevocationViaOcsp | AllEndEntityRevocation | IssuerAuthorityHasDesignatedOcspResponder | RootAuthorityHasDesignatedOcspResponder)

Error message:

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, Revoked, RevocationStatusUnknown | OfflineRevocation]
Actual:   X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, RevocationStatusUnknown | OfflineRevocation]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1437
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1352
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1293
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__14_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 327
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1408
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 306

[VincentBu]

Failed again in runtime-libraries-coreclr outerloop 20210801.2

Failed test:

net6.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open@mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504

- System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(pkiOptions: IssuerRevocationViaOcsp | AllEndEntityRevocation | RootAuthorityHasDesignatedOcspResponder)

Error message:

System.AggregateException : One or more errors occurred. (Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, Revoked, RevocationStatusUnknown | OfflineRevocation]
Actual:   X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, Revoked]) (An X509Extension with OID '1.3.6.1.5.5.7.1.1' has already been specified.) (m_safeCertContext is an invalid handle.) (m_safeCertContext is an invalid handle.) (m_safeCertContext is an invalid handle.)
---- Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, Revoked, RevocationStatusUnknown | OfflineRevocation]
Actual:   X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, Revoked]
---- System.InvalidOperationException : An X509Extension with OID '1.3.6.1.5.5.7.1.1' has already been specified.
---- System.Security.Cryptography.CryptographicException : m_safeCertContext is an invalid handle.
---- System.Security.Cryptography.CryptographicException : m_safeCertContext is an invalid handle.
---- System.Security.Cryptography.CryptographicException : m_safeCertContext is an invalid handle.


Stack trace
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 48
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1544
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 335
----- Inner Stack Trace #1 (Xunit.Sdk.EqualException) -----
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1573
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1482
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1425
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__18_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 356
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass46_0.<SimpleTest>b__0() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1542
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 40
----- Inner Stack Trace #2 (System.InvalidOperationException) -----
   at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X500DistinguishedName issuerName, X509SignatureGenerator generator, DateTimeOffset notBefore, DateTimeOffset notAfter, ReadOnlySpan`1 serialNumber) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 670
   at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.RebuildRootWithRevocation(X509Extension cdpExtension, X509Extension aiaExtension) in /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 204
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__18_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 342
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass46_0.<SimpleTest>b__0() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1542
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 40
----- Inner Stack Trace #3 (System.Security.Cryptography.CryptographicException) -----
   at System.Security.Cryptography.X509Certificates.X509Certificate.ThrowIfInvalid() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate.cs:line 641
   at System.Security.Cryptography.X509Certificates.X509Certificate2.get_SubjectName() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs:line 363
   at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.RebuildRootWithRevocation(X509Extension cdpExtension, X509Extension aiaExtension) in /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 204
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__18_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 342
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass46_0.<SimpleTest>b__0() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1542
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 40
----- Inner Stack Trace #4 (System.Security.Cryptography.CryptographicException) -----
   at System.Security.Cryptography.X509Certificates.X509Certificate.ThrowIfInvalid() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate.cs:line 641
   at System.Security.Cryptography.X509Certificates.X509Certificate2.get_SubjectName() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs:line 363
   at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.RebuildRootWithRevocation(X509Extension cdpExtension, X509Extension aiaExtension) in /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 204
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__18_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 342
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass46_0.<SimpleTest>b__0() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1542
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 40
----- Inner Stack Trace #5 (System.Security.Cryptography.CryptographicException) -----
   at System.Security.Cryptography.X509Certificates.X509Certificate.ThrowIfInvalid() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate.cs:line 641
   at System.Security.Cryptography.X509Certificates.X509Certificate2.get_SubjectName() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs:line 363
   at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.RebuildRootWithRevocation(X509Extension cdpExtension, X509Extension aiaExtension) in /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 204
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEverything>b__18_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder, RevocationResponder responder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 342
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c__DisplayClass46_0.<SimpleTest>b__0() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1542
   at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen) in /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs:line 40

[noahfalk]

Another hit on RevokeEverything:
https://github.com/dotnet/runtime/pull/56654/checks?check_run_id=3211463815
https://helix.dot.net/api/2019-06-17/jobs/93f87f3a-8c7d-4bc1-86c9-cff9d459bf96/workitems/System.Security.Cryptography.X509Certificates.Tests/console

Console log: 'System.Security.Cryptography.X509Certificates.Tests' from job 93f87f3a-8c7d-4bc1-86c9-cff9d459bf96 (windows.10.amd64.serverrs5.open.rt) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:nanoserver-1809-helix-amd64-08e8e40-20200107182504 on a004SNZ
running %HELIX_CORRELATION_PAYLOAD%\scripts\9dd8f448ee8e4195b6a6184a791d5016\execute.cmd in C:\h\w\B66509FF\w\9A3F089A\e max 900 seconds

Output:
[BEGIN EXECUTION]

C:\Work>cd /d C:\helix\work\workitem 

C:\helix\work\workitem>python -m pip install -r C:\helix\scripts\runtime_python_requirements.txt 
Collecting certifi==2020.6.20
  Downloading https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl (156kB)
Requirement already satisfied: ndg-httpsclient==0.5.1 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 2)) (0.5.1)
Collecting psutil==5.8.0
  Downloading https://files.pythonhosted.org/packages/71/ce/35107e81e7eae55c847313f872d4258a71d2640fa04f57c5520fc81473ce/psutil-5.8.0-cp37-cp37m-win_amd64.whl (244kB)
Requirement already satisfied: pyasn1==0.4.5 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 4)) (0.4.5)
Requirement already satisfied: pyopenssl==19.0.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 5)) (19.0.0)
Requirement already satisfied: six>=1.11.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 6)) (1.13.0)
Requirement already satisfied: cryptography<=3.4.6,>=2.8 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 8)) (2.8)
Requirement already satisfied: asn1crypto==0.24.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 9)) (0.24.0)
Requirement already satisfied: idna==2.8 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 10)) (2.8)
Collecting cffi==1.14.5
  Downloading https://files.pythonhosted.org/packages/c6/92/9188a1dff4934d0bd18df7f92dc481324b3342814cea34849b393ea2d232/cffi-1.14.5-cp37-cp37m-win_amd64.whl (178kB)
Requirement already satisfied: pycparser==2.19 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 12)) (2.19)
Requirement already satisfied: docker==3.7.2 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 14)) (3.7.2)
Requirement already satisfied: docker-pycreds==0.4.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 15)) (0.4.0)
Requirement already satisfied: websocket-client==0.56.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 16)) (0.56.0)
Requirement already satisfied: requests>=2.21.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 18)) (2.22.0)
Collecting urllib3==1.24.3
  Downloading https://files.pythonhosted.org/packages/01/11/525b02e4acc0c747de8b6ccdab376331597c569c42ea66ab0a1dbd36eca2/urllib3-1.24.3-py2.py3-none-any.whl (118kB)
Collecting azure-storage-blob==12.5.0
  Downloading https://files.pythonhosted.org/packages/b8/84/7e51b3e1156bcb89a20b9ec641d4fced4800aa79daac3a403898c32046be/azure_storage_blob-12.5.0-py2.py3-none-any.whl (326kB)
Collecting azure-core==1.7.0
  Downloading https://files.pythonhosted.org/packages/8b/00/efb68e2dda82139d732090fc3b7ff47fe6f34724ea7ba31e518a854b15c1/azure_core-1.7.0-py2.py3-none-any.whl (121kB)
Collecting azure-devops==5.0.0b9
  Downloading https://files.pythonhosted.org/packages/3d/e2/5e75a901aa79e8dc215c59b5beabce6d644d084fd8ec4b29f1584c29676e/azure_devops-5.0.0b9-py3-none-any.whl (1.3MB)
Requirement already satisfied: msrest==0.6.10 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 24)) (0.6.10)
Requirement already satisfied: isodate==0.6.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 25)) (0.6.0)
Requirement already satisfied: requests-oauthlib==1.2.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 26)) (1.2.0)
Requirement already satisfied: oauthlib==3.1.0 in c:\python\lib\site-packages (from -r C:\helix\scripts\runtime_python_requirements.txt (line 27)) (3.1.0)
Requirement already satisfied: pypiwin32==223; sys_platform == "win32" and python_version >= "3.6" in c:\python\lib\site-packages (from docker==3.7.2->-r C:\helix\scripts\runtime_python_requirements.txt (line 14)) (223)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\python\lib\site-packages (from requests>=2.21.0->-r C:\helix\scripts\runtime_python_requirements.txt (line 18)) (3.0.4)
Requirement already satisfied: pywin32>=223 in c:\python\lib\site-packages (from pypiwin32==223; sys_platform == "win32" and python_version >= "3.6"->docker==3.7.2->-r C:\helix\scripts\runtime_python_requirements.txt (line 14)) (227)
ERROR: helix-scripts 20191204.1 has requirement azure-core==1.0.0, but you'll have azure-core 1.7.0 which is incompatible.
ERROR: helix-scripts 20191204.1 has requirement azure-storage-blob==12.0.0, but you'll have azure-storage-blob 12.5.0 which is incompatible.
ERROR: helix-scripts 20191204.1 has requirement certifi==2018.11.29, but you'll have certifi 2020.6.20 which is incompatible.
ERROR: helix-scripts 20191204.1 has requirement cffi==1.12.3, but you'll have cffi 1.14.5 which is incompatible.
ERROR: helix-scripts 20191204.1 has requirement psutil==5.6.3, but you'll have psutil 5.8.0 which is incompatible.
Installing collected packages: certifi, psutil, cffi, urllib3, azure-core, azure-storage-blob, azure-devops
  Found existing installation: certifi 2018.11.29
    Uninstalling certifi-2018.11.29:
      Successfully uninstalled certifi-2018.11.29
  Found existing installation: psutil 5.6.3
    Uninstalling psutil-5.6.3:
      Successfully uninstalled psutil-5.6.3
  Found existing installation: cffi 1.12.3
    Uninstalling cffi-1.12.3:
      Successfully uninstalled cffi-1.12.3
  Found existing installation: urllib3 1.25.7
    Uninstalling urllib3-1.25.7:
      Successfully uninstalled urllib3-1.25.7
  Found existing installation: azure-core 1.0.0
    Uninstalling azure-core-1.0.0:
      Successfully uninstalled azure-core-1.0.0
  Found existing installation: azure-storage-blob 12.0.0
    Uninstalling azure-storage-blob-12.0.0:
      Successfully uninstalled azure-storage-blob-12.0.0
Successfully installed azure-core-1.7.0 azure-devops-5.0.0b9 azure-storage-blob-12.5.0 certifi-2020.6.20 cffi-1.14.5 psutil-5.8.0 urllib3-1.24.3
WARNING: You are using pip version 19.3.1; however, version 21.2.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

C:\helix\work\workitem>set PYTHONPATH=C:\Python\Lib;C:\Python\DLLs;;C:\helix\scripts 

C:\helix\work\workitem>C:\helix\work\correlation\scripts\9dd8f448ee8e4195b6a6184a791d5016\execute.cmd

C:\helix\work\workitem>taskkill.exe /f /im corerun.exe 
'taskkill.exe' is not recognized as an internal or external command,
operable program or batch file.

C:\helix\work\workitem>call RunTests.cmd --runtime-path C:\helix\work\correlation 
----- start Sun 08/01/2021  1:29:11.00 ===============  To repro directly: ===================================================== 
pushd C:\helix\work\workitem\
"C:\helix\work\correlation\dotnet.exe" exec --runtimeconfig System.Security.Cryptography.X509Certificates.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.X509Certificates.Tests.deps.json xunit.console.dll System.Security.Cryptography.X509Certificates.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing 
popd
===========================================================================================================

C:\helix\work\workitem>"C:\helix\work\correlation\dotnet.exe" exec --runtimeconfig System.Security.Cryptography.X509Certificates.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.X509Certificates.Tests.deps.json xunit.console.dll System.Security.Cryptography.X509Certificates.Tests.dll -xml testResults.xml -nologo -nocolor -trait category=OuterLoop -notrait category=IgnoreForCI -notrait category=failing  
  Discovering: System.Security.Cryptography.X509Certificates.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Security.Cryptography.X509Certificates.Tests (found 35 of 681 test cases)
  Starting:    System.Security.Cryptography.X509Certificates.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(pkiOptions: OcspEverywhere | RootAuthorityHasDesignatedOcspResponder) [FAIL]
      Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Unknown error (0xc0000001)
      Stack Trace:
        /_/src/libraries/Common/src/Internal/Cryptography/CngCommon.SignVerify.cs(41,0): at Internal.Cryptography.CngCommon.SignHash(SafeNCryptKeyHandle keyHandle, ReadOnlySpan`1 hash, AsymmetricPaddingMode paddingMode, Void* pPaddingInfo, Int32 estimatedSize)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.SignVerify.cs(81,0): at System.Security.Cryptography.RSACng.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs(179,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs(157,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs(22,0): at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs(692,0): at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X500DistinguishedName issuerName, X509SignatureGenerator generator, DateTimeOffset notBefore, DateTimeOffset notAfter, ReadOnlySpan`1 serialNumber)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs(531,0): at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X509Certificate2 issuerCertificate, DateTimeOffset notBefore, DateTimeOffset notAfter, ReadOnlySpan`1 serialNumber)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs(388,0): at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X509Certificate2 issuerCertificate, DateTimeOffset notBefore, DateTimeOffset notAfter, Byte[] serialNumber)
        /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs(287,0): at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.CreateCertificate(String subject, RSA publicKey, TimeSpan nestingBuffer, X509ExtensionCollection extensions, Boolean ocspResponder)
        /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs(170,0): at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.CreateEndEntity(String subject, RSA publicKey, X509ExtensionCollection extensions)
        /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs(918,0): at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.BuildPrivatePki(PkiOptions pkiOptions, RevocationResponder& responder, CertificateAuthority& rootAuthority, CertificateAuthority[]& intermediateAuthorities, X509Certificate2& endEntityCert, Int32 intermediateAuthorityCount, String testName, Boolean registerAuthorities, Boolean pkiOptionsInSubject, String subjectName, Int32 keySize, X509ExtensionCollection extensions)
        /_/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs(951,0): at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.BuildPrivatePki(PkiOptions pkiOptions, RevocationResponder& responder, CertificateAuthority& rootAuthority, CertificateAuthority& intermediateAuthority, X509Certificate2& endEntityCert, String testName, Boolean registerAuthorities, Boolean pkiOptionsInSubject, String subjectName, Int32 keySize, X509ExtensionCollection extensions)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1595,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.BuildPrivatePki(PkiOptions pkiOptions, RevocationResponder& responder, CertificateAuthority& rootAuthority, CertificateAuthority& intermediateAuthority, X509Certificate2& endEntityCert, String testName, Boolean registerAuthorities, Boolean pkiOptionsInSubject)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(1492,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName)
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs(335,0): at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEverything(PkiOptions pkiOptions)
  Finished:    System.Security.Cryptography.X509Certificates.Tests
=== TEST EXECUTION SUMMARY ===
   System.Security.Cryptography.X509Certificates.Tests  Total: 503, Errors: 0, Failed: 1, Skipped: 0, Time: 158.186s
----- end Sun 08/01/2021  1:31:50.41 ----- exit code 1 ----------------------------------------------------------
 Sun 08/01/2021- 1:31:50.42
The system cannot find the file specified.
The system cannot find the file specified.
Using base prefix 'C:\\Python'
New python executable in C:\Users\ContainerAdministrator\.azdo-env-tmp\Scripts\python.exe
Installing setuptools, pip, wheel...
done.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'azure'
Collecting azure-devops==5.0.0b9
  Using cached azure_devops-5.0.0b9-py3-none-any.whl (1.3 MB)
Collecting msrest<0.7.0,>=0.6.0
  Downloading msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting isodate>=0.6.0
  Downloading isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting requests-oauthlib>=0.5.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting requests~=2.16
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, six, requests, oauthlib, requests-oauthlib, isodate, msrest, azure-devops
Successfully installed azure-devops-5.0.0b9 certifi-2021.5.30 charset-normalizer-2.0.4 idna-3.2 isodate-0.6.0 msrest-0.6.21 oauthlib-3.1.1 requests-2.26.0 requests-oauthlib-1.3.0 six-1.16.0 urllib3-1.26.6
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'future'
Collecting future==0.17.1
  Downloading future-0.17.1.tar.gz (829 kB)
Building wheels for collected packages: future
  Building wheel for future (setup.py): started
  Building wheel for future (setup.py): finished with status 'done'
  Created wheel for future: filename=future-0.17.1-py3-none-any.whl size=488741 sha256=08fb635c78eb9bcdff1e98f703a69dc00c0ec9321dbeb123e66338a1a72c90f8
  Stored in directory: c:\helix\work\workitem\pip\cache\wheels\16\4c\84\8a6161d44282ede60ed233d090156c6109a7ab865e49c1c9f6
Successfully built future
Installing collected packages: future
Successfully installed future-0.17.1
 Sun 08/01/2021- 1:32:16.44
2021-08-01 01:32:16,769: INFO: 4388: run(141): main: Main thread starting 10 workers
Searching 'C:\helix\work\workitem\..' for log files
Generated log list: 
Searching 'C:\helix\work\workitem' for test results files
Found results file C:\helix\work\workitem\testResults.xml with format xunit
Searching 'C:\helix\work\workitem\uploads' for test results files
 Sun 08/01/2021- 1:32:22.74

[END EXECUTION]
Exit Code:0