AssociatePersistedKey_CAPIviaCNG_RSA failing with WindowsCryptographicException

[ahsonkhan]

From dotnet/corefx#38469

https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F38469~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190624.21/workItem/System.Security.Cryptography.X509Certificates.Tests/analysis/xunit/System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests~2FAssociatePersistedKey_CAPIviaCNG_RSA(provType:%201,%20keyNumber:%20Exchange)

System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests/AssociatePersistedKey_CAPIviaCNG_RSA(provType: 1, keyNumber: Exchange)
Windows.10.Amd64.ClientRS4.ES.Open-x86-Release

English exception message:
One of the devices connected to the system does not work.

Unhandled Exception of Type Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException
Message :
Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : Uno de los dispositivos conectados al sistema no funciona.
Stack Trace :
   at Internal.NativeCrypto.CapiHelper.SignValue(SafeProvHandle hProv, SafeKeyHandle hKey, Int32 keyNumber, Int32 calgKey, Int32 calgHash, Byte[] hash) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/CapiHelper.Windows.cs:line 1214
   at System.Security.Cryptography.RSACryptoServiceProvider.SignHash(Byte[] rgbHash, Int32 calgHash) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 507
   at System.Security.Cryptography.RSACryptoServiceProvider.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 702
   at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 175
   at System.Security.Cryptography.RSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 153
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm) in /_/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 24
   at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests.RSASha1Pkcs1SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm) in /_/src/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/PrivateKeyAssociationTests.cs:line 573
   at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X500DistinguishedName issuerName, X509SignatureGenerator generator, DateTimeOffset notBefore, DateTimeOffset notAfter, Byte[] serialNumber) in /_/src/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 595
   at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests.AssociatePersistedKey_CAPIviaCNG_RSA(Int32 provType, KeyNumber keyNumber) in /_/src/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/PrivateKeyAssociationTests.cs:line 114

[bartonjs]

"A device attached to the system is not functioning." (ERROR_GEN_FAILURE)

[bartonjs]

(If we ended up losing data in the message (or the CAPI/CNG layer), this could have been NTE_BAD_KEYSET_PARAM (0x8009001f vs 0x1f).

[bartonjs]

I don't see anything in either CAPI or CNG that would produce that code (though I looked at 1903, not 1803), so it's unclear what went wrong. The key handle is still open, so it shouldn't have been finalized out from under us... the "how big is this signature" call (CapiHelpers.Windows.cs:1203) succeeded, then the one for actually signing failed (1214)... so this seems like CAPI got into an internal hiccup state.

Without a Time Travel trace or a viable repro it's hard to be actionable, moving to Future for now.

[jkotas]

Hit in #31991:

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : A device attached to the system is not functioning.


Stack trace
   at Internal.NativeCrypto.CapiHelper.SignValue(SafeProvHandle hProv, SafeKeyHandle hKey, Int32 keyNumber, Int32 calgKey, Int32 calgHash, Byte[] hash) in /_/src/libraries/System.Security.Cryptography.Csp/src/System/Security/Cryptography/CapiHelper.Windows.cs:line 1200
   at System.Security.Cryptography.RSACryptoServiceProvider.SignHash(Byte[] rgbHash, Int32 calgHash) in /_/src/libraries/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 507
   at System.Security.Cryptography.RSACryptoServiceProvider.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/libraries/System.Security.Cryptography.Csp/src/System/Security/Cryptography/RSACryptoServiceProvider.Windows.cs:line 686
   at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 175
   at System.Security.Cryptography.RSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 153
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 24
   at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests.RSASha1Pkcs1SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/PrivateKeyAssociationTests.cs:line 573
   at System.Security.Cryptography.X509Certificates.CertificateRequest.Create(X500DistinguishedName issuerName, X509SignatureGenerator generator, DateTimeOffset notBefore, DateTimeOffset notAfter, Byte[] serialNumber) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 595
   at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.PrivateKeyAssociationTests.AssociatePersistedKey_CAPIviaCNG_RSA(Int32 provType, KeyNumber keyNumber) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/PrivateKeyAssociationTests.cs:line 114

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

[dotnet-policy-service]

This issue will now be closed since it had been marked no-recent-activity but received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.