Variation of EnvelopedCmsTests.Tests.DecryptTests.Decrypt_SignedWithinEnveloped fails cross platform

[krwq]

This is a variation of EnvelopedCmsTests.Tests.DecryptTests.Decrypt_SignedWithinEnveloped test which checks roundtripping between Windows 10 and Ubuntu 16.04 (very likely any Windows vs any non-Windows).

Full repro in point 2, point 1 shows how was the doc generated in case it is useful. Guessing that some of the encryption parameters might not be the same (i.e. different default IV) but requires more investigation.

1. Generate encrypted document on Windows and Ubuntu

Expand to see code and outputs

            byte[] content =
                 ("3082032506092a864886f70d010702a082031630820312020101310b300906052b0e03021a0500301206092a864886f70d01"
                + "0701a0050403010203a08202103082020c30820179a00302010202105d2ffff863babc9b4d3c80ab178a4cca300906052b0e"
                + "03021d0500301e311c301a060355040313135253414b65795472616e736665724361706931301e170d313530343135303730"
                + "3030305a170d3235303431353037303030305a301e311c301a060355040313135253414b65795472616e7366657243617069"
                + "3130819f300d06092a864886f70d010101050003818d0030818902818100aa272700586c0cc41b05c65c7d846f5a2bc27b03"
                + "e301c37d9bff6d75b6eb6671ba9596c5c63ba2b1af5c318d9ca39e7400d10c238ac72630579211b86570d1a1d44ec86aa8f6"
                + "c9d2b4e283ea3535923f398a312a23eaeacd8d34faaca965cd910b37da4093ef76c13b337c1afab7d1d07e317b41a336baa4"
                + "111299f99424408d0203010001a3533051304f0603551d0104483046801015432db116b35d07e4ba89edb2469d7aa120301e"
                + "311c301a060355040313135253414b65795472616e73666572436170693182105d2ffff863babc9b4d3c80ab178a4cca3009"
                + "06052b0e03021d05000381810081e5535d8eceef265acbc82f6c5f8bc9d84319265f3ccf23369fa533c8dc1938952c593166"
                + "2d9ecd8b1e7b81749e48468167e2fce3d019fa70d54646975b6dc2a3ba72d5a5274c1866da6d7a5df47938e034a075d11957"
                + "d653b5c78e5291e4401045576f6d4eda81bef3c369af56121e49a083c8d1adb09f291822e99a4296463181d73081d4020101"
                + "3032301e311c301a060355040313135253414b65795472616e73666572436170693102105d2ffff863babc9b4d3c80ab178a"
                + "4cca300906052b0e03021a0500300d06092a864886f70d010101050004818031a718ea1483c88494661e1d3dedfea0a3d97e"
                + "eb64c3e093a628b257c0cfc183ecf11697ac84f2af882b8de0c793572af38dc15d1b6f3d8f2392ba1cc71210e177c146fd16"
                + "b77a583b6411e801d7a2640d612f2fe99d87e9718e0e505a7ab9536d71dbde329da21816ce7da1416a74a3e0a112b86b33af"
                + "336a2ba6ae2443d0ab").HexToByteArray();

            ContentInfo contentInfo = new ContentInfo(new Oid(Oids.Pkcs7Signed), content);
            CertLoader certLoader = Certificates.RSAKeyTransferCapi1;
            string algorithmOidValue = Oids.Aes256;
            SubjectIdentifierType type = SubjectIdentifierType.IssuerAndSerialNumber;
            string certSubjectName;
            byte[] encodedMessage;
            using (X509Certificate2 certificate = certLoader.GetCertificate())
            {
                certSubjectName = certificate.Subject;
                AlgorithmIdentifier alg = new AlgorithmIdentifier(new Oid(algorithmOidValue));
                EnvelopedCms ecms = new EnvelopedCms(contentInfo, alg);
                CmsRecipient cmsRecipient = new CmsRecipient(type, certificate);
                ecms.Encrypt(cmsRecipient);
                encodedMessage = ecms.Encode();
            }

            Console.WriteLine(Convert.ToBase64String(encodedMessage, Base64FormattingOptions.InsertLineBreaks));

Example Ubuntu output:

MIIERwYJKoZIhvcNAQcDoIIEODCCBDQCAQAxgcwwgckCAQAwMjAeMRwwGgYDVQQDExNSU0FLZXlU
cmFuc2ZlckNhcGkxAhBdL//4Y7q8m008gKsXikzKMA0GCSqGSIb3DQEBAQUABIGAWSteC3DpgylV
MTLUdUy6Yz4dnHALkKQxB3i10K/LHi6nTRu9IBfyZoukCIV6mHpC9y3s1YVkTgoEa1t8BfOw/3Eu
D1TmdELJk+i9g341YL/M6B2JYFBui5ZYOxiOT/MiqiMLXOj58OS/qgkulJJ0/LEgr4dzXIk4jC00
ONtNEIgwggNeBgkqhkiG9w0BBwIwHQYJYIZIAWUDBAEqBBCCiF/uBuUUWB18cGguTMnJgIIDMOuR
dWReQHAc7T3nTKzh5vOAuLc7+SWGUQUXOCszjEPaSRWRwpi95kvQHEaskC+rQYm1Y806Tl9PnY1g
ip+N0dAVgYyRFpe5Mm6V1EjgvvA+G5JPrhF1ux/uRw2pAKtJgPWKU3FyO/S5Agd1SBhXSDfbi5AT
FwkZuNiDT2tohWxgVX9ysv3TsyXCaRersm4v+i4Az44HdHXg6mroK2N9cmd2UV1MhDF3paAROifl
dsVFqLMhR384lPYTgw/EHE/H4ygb/wRx/PxKCuFl1bkbkZau925CPi3kBhqj0Ev3dizeYOdABr0Q
6T8Q/eiTWawiT1LeJzAUdanu523voMK3n4SS/mABd7kFeCFsI0EAAEyydby8ftf7vmE0e1dLBzCW
Z/3aXRrAIuQFl89OrSlQu2OaasKus2q4MCWQjm4X7r+F2xZfQdccSImFChiH2yspOOIlKVmPLnrC
DxFZLkJ2oBL77vbutqYBJ3XeOHUt6e/r/PT1gAXuMt4XMw1y1lWxmE0pLG08hF23nTD00GEZl+jm
8KnDqsSc6cwPv8vu3LIYOHqyGX5pahcOhAIG/hLzPPBzveqj+FWDR/b/WCgoDW8GXK+uwz99Ebdg
+Onym0Xh/PahYLdVprAqtekn0qR6zABXDqVsNG1GzxAhPkrm7ZWuCRFZxL8InBmnC0oUS8z4+5JH
JQoG661wT/o1HUbm0FWWHG6dz12urMSMEFktKqGCu9VfUrf56P43+mErAYz9VWl7h0BV8kt/WOcZ
rmHuBHLliUanfKDGgF3JBl9Zlw/SJPuryeFikx7jj3FQm3B0e/1I+d2izPbANzBZH8s+/ayy+aLC
sNQNZhKJCbSKUL8RiUc8w1YOS2yauHJNC2cM+p2e7//jouzTegpOBZlTNYn6hif/FVj45I3SMYZj
J947GiPR8tPa8RUByOqmel4g6yOpAsREQ1oQ/JwBS5JefdJ1EfZvXhqvOFkaJzov2mAnmDFwq8jA
T5VYDXNCCOmQOlP9OHpORTW4ZX7fSwxIJtiBWs/7HVxgniRI93v+gNg2doe80K0yJb5IB3kh7krQ
bo5Du6/623f7IzCOyVHXDg==

Example Windows output:

MIIERwYJKoZIhvcNAQcDoIIEODCCBDQCAQAxgcwwgckCAQAwMjAeMRwwGgYDVQQDExNSU0FLZXlU
cmFuc2ZlckNhcGkxAhBdL//4Y7q8m008gKsXikzKMA0GCSqGSIb3DQEBAQUABIGApbrDSzyaaJWl
cBGSA9iGchDZ3GcdHtGNwC80ov2mBZ5QApcIeD+QBa0kfhKbOGWVjZuxMeiWKQLumTRLjlYMWyGG
Vr1vH2vogWylJzOHgl4a1SzBU512e0kJAj4X/+rxTc3J1BA++zqQwKc2ZMNwBpzwHAr8R9f/0Yqi
6g4cF3cwggNeBgkqhkiG9w0BBwIwHQYJYIZIAWUDBAEqBBCYvXFZhIoVjFhstITvI1eogIIDMDpN
UrvypCXSpPMd0NAsgphvXCnQHnipOKak/hZNA0epyN+HK/YtX6h7YrdTsR1lpbV5oZJGbkKdYuC2
nepkedh1acwNbxa5PbeRJOgpeM/OB4Q/SEANUd00+fn+AteuBvmWTV6EBUQJMpnKlaiAM0rJrdir
8wRyDB/hWQaU4c26yKoYNBzXCp2q6ml/DtPF1i9W9w7VTe3gPjHVTWejqeZHAJ/5CHrL86TImSdU
Cf22W2g9jG2VEoFw3gRg6sliDAdDfazBI7koXyqu8kRBPJHPfh5c30UGkFmfUREWhVfI/uldWF7p
3lmHbg7wZKBjbOKZs7lGwRomx7NKp252WHg8mzG1vNZeCKDweKU4lZMBn7YyYfmt3AYFs+dDYVAn
3UUylBEPNHXJCYkDeQNyPcemD15IsKEKOAOnmFNphu3Ra/0SBZKhopqeY+g3BEXJrA8VmNh1Yx3o
oOk2ruW94oPD4gLvMYgqelezuYXAZ6/r3vvNvgWTDiaDrjnf9C6hbn381iAjEThooFaRhK0c3TY7
svahOLQ6aNrp88yoaTLeD9eT9l/ZMyMgvyZxTxsdN5320tM015CTuObsVusKK721ZeMWOsc407xu
l02vLbOU/hejdoZu7XCe0VdzjWmmoTZrbFVZQyJHYmmC3+aWIvrIUSsuxIrATnUkWfpRBqEpAvLu
hELYX6oFN7Xej9pcX3JQVWYMF1pbeV755ckSNqapYLCct0Yjedd8AI5pKolok9q2Be0ablIfRSL/
7vhMyB7nGAdm9iEAnz7CGgLMayE5fApl5Z9MmgOQKB5N8XvHuzhdvypBXoV4WigAiarRMBOhhnMz
eEOak8tsXag/lWsyvqr05Khl5SwCVEr/OQafAhkuB//r9LrUOJAy1KTiAlRRckB1wyC4buidS/wI
FUuZ83Zc9dLnJhWGEvCb4QvSgnVFxfgetDOf56HlDwk6Jkna/ox0PhqMgZCORK4BGg9L7WENSuL8
1Dh33SoT4L6BEwF3Zrdu4f475KxcUyv34vaa2w4hcNVl/iYYcKW6L7V9wE9xVtfxSecfBK7khD1U
V19xndleslZ5fSjhrv4s/Q==

2. Try decrypt document on the platform which it was not generated on

Test cases (one of those test case passes on any platform, the other one fails)

        [Fact]
        public static void ReproWindows()
        {
            byte[] expectedContent =
                 ("3082032506092a864886f70d010702a082031630820312020101310b300906052b0e03021a0500301206092a864886f70d01"
                + "0701a0050403010203a08202103082020c30820179a00302010202105d2ffff863babc9b4d3c80ab178a4cca300906052b0e"
                + "03021d0500301e311c301a060355040313135253414b65795472616e736665724361706931301e170d313530343135303730"
                + "3030305a170d3235303431353037303030305a301e311c301a060355040313135253414b65795472616e7366657243617069"
                + "3130819f300d06092a864886f70d010101050003818d0030818902818100aa272700586c0cc41b05c65c7d846f5a2bc27b03"
                + "e301c37d9bff6d75b6eb6671ba9596c5c63ba2b1af5c318d9ca39e7400d10c238ac72630579211b86570d1a1d44ec86aa8f6"
                + "c9d2b4e283ea3535923f398a312a23eaeacd8d34faaca965cd910b37da4093ef76c13b337c1afab7d1d07e317b41a336baa4"
                + "111299f99424408d0203010001a3533051304f0603551d0104483046801015432db116b35d07e4ba89edb2469d7aa120301e"
                + "311c301a060355040313135253414b65795472616e73666572436170693182105d2ffff863babc9b4d3c80ab178a4cca3009"
                + "06052b0e03021d05000381810081e5535d8eceef265acbc82f6c5f8bc9d84319265f3ccf23369fa533c8dc1938952c593166"
                + "2d9ecd8b1e7b81749e48468167e2fce3d019fa70d54646975b6dc2a3ba72d5a5274c1866da6d7a5df47938e034a075d11957"
                + "d653b5c78e5291e4401045576f6d4eda81bef3c369af56121e49a083c8d1adb09f291822e99a4296463181d73081d4020101"
                + "3032301e311c301a060355040313135253414b65795472616e73666572436170693102105d2ffff863babc9b4d3c80ab178a"
                + "4cca300906052b0e03021a0500300d06092a864886f70d010101050004818031a718ea1483c88494661e1d3dedfea0a3d97e"
                + "eb64c3e093a628b257c0cfc183ecf11697ac84f2af882b8de0c793572af38dc15d1b6f3d8f2392ba1cc71210e177c146fd16"
                + "b77a583b6411e801d7a2640d612f2fe99d87e9718e0e505a7ab9536d71dbde329da21816ce7da1416a74a3e0a112b86b33af"
                + "336a2ba6ae2443d0ab").HexToByteArray();

            ContentInfo expectedContentInfo = new ContentInfo(new Oid(Oids.Pkcs7Signed), expectedContent);

            byte[] encodedMessage = Convert.FromBase64String(
                "MIIERwYJKoZIhvcNAQcDoIIEODCCBDQCAQAxgcwwgckCAQAwMjAeMRwwGgYDVQQDExNSU0FLZXlU" +
                "cmFuc2ZlckNhcGkxAhBdL//4Y7q8m008gKsXikzKMA0GCSqGSIb3DQEBAQUABIGAWSteC3DpgylV" +
                "MTLUdUy6Yz4dnHALkKQxB3i10K/LHi6nTRu9IBfyZoukCIV6mHpC9y3s1YVkTgoEa1t8BfOw/3Eu" +
                "D1TmdELJk+i9g341YL/M6B2JYFBui5ZYOxiOT/MiqiMLXOj58OS/qgkulJJ0/LEgr4dzXIk4jC00" +
                "ONtNEIgwggNeBgkqhkiG9w0BBwIwHQYJYIZIAWUDBAEqBBCCiF/uBuUUWB18cGguTMnJgIIDMOuR" +
                "dWReQHAc7T3nTKzh5vOAuLc7+SWGUQUXOCszjEPaSRWRwpi95kvQHEaskC+rQYm1Y806Tl9PnY1g" +
                "ip+N0dAVgYyRFpe5Mm6V1EjgvvA+G5JPrhF1ux/uRw2pAKtJgPWKU3FyO/S5Agd1SBhXSDfbi5AT" +
                "FwkZuNiDT2tohWxgVX9ysv3TsyXCaRersm4v+i4Az44HdHXg6mroK2N9cmd2UV1MhDF3paAROifl" +
                "dsVFqLMhR384lPYTgw/EHE/H4ygb/wRx/PxKCuFl1bkbkZau925CPi3kBhqj0Ev3dizeYOdABr0Q" +
                "6T8Q/eiTWawiT1LeJzAUdanu523voMK3n4SS/mABd7kFeCFsI0EAAEyydby8ftf7vmE0e1dLBzCW" +
                "Z/3aXRrAIuQFl89OrSlQu2OaasKus2q4MCWQjm4X7r+F2xZfQdccSImFChiH2yspOOIlKVmPLnrC" +
                "DxFZLkJ2oBL77vbutqYBJ3XeOHUt6e/r/PT1gAXuMt4XMw1y1lWxmE0pLG08hF23nTD00GEZl+jm" +
                "8KnDqsSc6cwPv8vu3LIYOHqyGX5pahcOhAIG/hLzPPBzveqj+FWDR/b/WCgoDW8GXK+uwz99Ebdg" +
                "+Onym0Xh/PahYLdVprAqtekn0qR6zABXDqVsNG1GzxAhPkrm7ZWuCRFZxL8InBmnC0oUS8z4+5JH" +
                "JQoG661wT/o1HUbm0FWWHG6dz12urMSMEFktKqGCu9VfUrf56P43+mErAYz9VWl7h0BV8kt/WOcZ" +
                "rmHuBHLliUanfKDGgF3JBl9Zlw/SJPuryeFikx7jj3FQm3B0e/1I+d2izPbANzBZH8s+/ayy+aLC" +
                "sNQNZhKJCbSKUL8RiUc8w1YOS2yauHJNC2cM+p2e7//jouzTegpOBZlTNYn6hif/FVj45I3SMYZj" +
                "J947GiPR8tPa8RUByOqmel4g6yOpAsREQ1oQ/JwBS5JefdJ1EfZvXhqvOFkaJzov2mAnmDFwq8jA" +
                "T5VYDXNCCOmQOlP9OHpORTW4ZX7fSwxIJtiBWs/7HVxgniRI93v+gNg2doe80K0yJb5IB3kh7krQ" +
                "bo5Du6/623f7IzCOyVHXDg==");

            CertLoader certLoader = Certificates.RSAKeyTransferCapi1;
            VerifySimpleDecrypt(encodedMessage, certLoader, expectedContentInfo);
        }

        [Fact]
        public static void ReproUbuntu()
        {
            byte[] expectedContent =
                 ("3082032506092a864886f70d010702a082031630820312020101310b300906052b0e03021a0500301206092a864886f70d01"
                + "0701a0050403010203a08202103082020c30820179a00302010202105d2ffff863babc9b4d3c80ab178a4cca300906052b0e"
                + "03021d0500301e311c301a060355040313135253414b65795472616e736665724361706931301e170d313530343135303730"
                + "3030305a170d3235303431353037303030305a301e311c301a060355040313135253414b65795472616e7366657243617069"
                + "3130819f300d06092a864886f70d010101050003818d0030818902818100aa272700586c0cc41b05c65c7d846f5a2bc27b03"
                + "e301c37d9bff6d75b6eb6671ba9596c5c63ba2b1af5c318d9ca39e7400d10c238ac72630579211b86570d1a1d44ec86aa8f6"
                + "c9d2b4e283ea3535923f398a312a23eaeacd8d34faaca965cd910b37da4093ef76c13b337c1afab7d1d07e317b41a336baa4"
                + "111299f99424408d0203010001a3533051304f0603551d0104483046801015432db116b35d07e4ba89edb2469d7aa120301e"
                + "311c301a060355040313135253414b65795472616e73666572436170693182105d2ffff863babc9b4d3c80ab178a4cca3009"
                + "06052b0e03021d05000381810081e5535d8eceef265acbc82f6c5f8bc9d84319265f3ccf23369fa533c8dc1938952c593166"
                + "2d9ecd8b1e7b81749e48468167e2fce3d019fa70d54646975b6dc2a3ba72d5a5274c1866da6d7a5df47938e034a075d11957"
                + "d653b5c78e5291e4401045576f6d4eda81bef3c369af56121e49a083c8d1adb09f291822e99a4296463181d73081d4020101"
                + "3032301e311c301a060355040313135253414b65795472616e73666572436170693102105d2ffff863babc9b4d3c80ab178a"
                + "4cca300906052b0e03021a0500300d06092a864886f70d010101050004818031a718ea1483c88494661e1d3dedfea0a3d97e"
                + "eb64c3e093a628b257c0cfc183ecf11697ac84f2af882b8de0c793572af38dc15d1b6f3d8f2392ba1cc71210e177c146fd16"
                + "b77a583b6411e801d7a2640d612f2fe99d87e9718e0e505a7ab9536d71dbde329da21816ce7da1416a74a3e0a112b86b33af"
                + "336a2ba6ae2443d0ab").HexToByteArray();

            ContentInfo expectedContentInfo = new ContentInfo(new Oid(Oids.Pkcs7Signed), expectedContent);

            byte[] encodedMessage = Convert.FromBase64String(
                "MIIERwYJKoZIhvcNAQcDoIIEODCCBDQCAQAxgcwwgckCAQAwMjAeMRwwGgYDVQQDExNSU0FLZXlU" +
                "cmFuc2ZlckNhcGkxAhBdL//4Y7q8m008gKsXikzKMA0GCSqGSIb3DQEBAQUABIGApbrDSzyaaJWl" +
                "cBGSA9iGchDZ3GcdHtGNwC80ov2mBZ5QApcIeD+QBa0kfhKbOGWVjZuxMeiWKQLumTRLjlYMWyGG" +
                "Vr1vH2vogWylJzOHgl4a1SzBU512e0kJAj4X/+rxTc3J1BA++zqQwKc2ZMNwBpzwHAr8R9f/0Yqi" +
                "6g4cF3cwggNeBgkqhkiG9w0BBwIwHQYJYIZIAWUDBAEqBBCYvXFZhIoVjFhstITvI1eogIIDMDpN" +
                "UrvypCXSpPMd0NAsgphvXCnQHnipOKak/hZNA0epyN+HK/YtX6h7YrdTsR1lpbV5oZJGbkKdYuC2" +
                "nepkedh1acwNbxa5PbeRJOgpeM/OB4Q/SEANUd00+fn+AteuBvmWTV6EBUQJMpnKlaiAM0rJrdir" +
                "8wRyDB/hWQaU4c26yKoYNBzXCp2q6ml/DtPF1i9W9w7VTe3gPjHVTWejqeZHAJ/5CHrL86TImSdU" +
                "Cf22W2g9jG2VEoFw3gRg6sliDAdDfazBI7koXyqu8kRBPJHPfh5c30UGkFmfUREWhVfI/uldWF7p" +
                "3lmHbg7wZKBjbOKZs7lGwRomx7NKp252WHg8mzG1vNZeCKDweKU4lZMBn7YyYfmt3AYFs+dDYVAn" +
                "3UUylBEPNHXJCYkDeQNyPcemD15IsKEKOAOnmFNphu3Ra/0SBZKhopqeY+g3BEXJrA8VmNh1Yx3o" +
                "oOk2ruW94oPD4gLvMYgqelezuYXAZ6/r3vvNvgWTDiaDrjnf9C6hbn381iAjEThooFaRhK0c3TY7" +
                "svahOLQ6aNrp88yoaTLeD9eT9l/ZMyMgvyZxTxsdN5320tM015CTuObsVusKK721ZeMWOsc407xu" +
                "l02vLbOU/hejdoZu7XCe0VdzjWmmoTZrbFVZQyJHYmmC3+aWIvrIUSsuxIrATnUkWfpRBqEpAvLu" +
                "hELYX6oFN7Xej9pcX3JQVWYMF1pbeV755ckSNqapYLCct0Yjedd8AI5pKolok9q2Be0ablIfRSL/" +
                "7vhMyB7nGAdm9iEAnz7CGgLMayE5fApl5Z9MmgOQKB5N8XvHuzhdvypBXoV4WigAiarRMBOhhnMz" +
                "eEOak8tsXag/lWsyvqr05Khl5SwCVEr/OQafAhkuB//r9LrUOJAy1KTiAlRRckB1wyC4buidS/wI" +
                "FUuZ83Zc9dLnJhWGEvCb4QvSgnVFxfgetDOf56HlDwk6Jkna/ox0PhqMgZCORK4BGg9L7WENSuL8" +
                "1Dh33SoT4L6BEwF3Zrdu4f475KxcUyv34vaa2w4hcNVl/iYYcKW6L7V9wE9xVtfxSecfBK7khD1U" +
                "V19xndleslZ5fSjhrv4s/Q==");

            CertLoader certLoader = Certificates.RSAKeyTransferCapi1;
            VerifySimpleDecrypt(encodedMessage, certLoader, expectedContentInfo);
        }

Error on Windows:

   System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.ReproWindows [FAIL]
      Assert.Equal() Failure
      Expected: Byte[] [48, 130, 3, 37, 6, ...]
      Actual:   Byte[] [48, 130, 3, 41, 48, ...]
      Stack Trace:
         F:\src\corefx4\src\System.Security.Cryptography.Pkcs\tests\EnvelopedCms\DecryptTests.cs(692,0): at System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.VerifySimpleDecrypt(Byte[] encodedMessage, CertLoader certLoader, ContentInfo expectedContent)
         F:\src\corefx4\src\System.Security.Cryptography.Pkcs\tests\EnvelopedCms\DecryptTests.cs(84,0): at System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.ReproWindows()

Error on Ubuntu:

     System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.ReproUbuntu [FAIL]
        Assert.Equal() Failure
        Expected: Byte[] [48, 130, 3, 37, 6, ...]
        Actual:   Byte[] [6, 9, 42, 134, 72, ...]
        Stack Trace:
           /home/krwq/src/corefx/src/System.Security.Cryptography.Pkcs/tests/EnvelopedCms/DecryptTests.cs(717,0): at System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.VerifySimpleDecrypt(Byte[] encodedMessage, CertLoader certLoader, ContentInfo expectedContent)
           /home/krwq/src/corefx/src/System.Security.Cryptography.Pkcs/tests/EnvelopedCms/DecryptTests.cs(248,0): at System.Security.Cryptography.Pkcs.EnvelopedCmsTests.Tests.DecryptTests.ReproUbuntu()

cc: @bartonjs @AlexGhiondea @joshfree

EDIT: Also seeing similar issue with: ContentInfo expectedContentInfo = new ContentInfo(new Oid(Oids.Pkcs7SignedEnveloped), content); (EnvelopedWithinEnveloped)

[krwq]

My current understanding is following:
Issue happens always when ContentInfo is not Oids.Pkcs7Data, which is any of the following:

        public const string Pkcs7Signed = "1.2.840.113549.1.7.2";
        public const string Pkcs7Enveloped = "1.2.840.113549.1.7.3";
        public const string Pkcs7SignedEnveloped = "1.2.840.113549.1.7.4";
        public const string Pkcs7Hashed = "1.2.840.113549.1.7.5";
        public const string Pkcs7Encrypted = "1.2.840.113549.1.7.6";

Quick look at the output the diff seems to be that Windows has additional DER prefix (Sequence Tag with length).

I've debugged through Windows decryption code (.NET Core) and in fact there is a code path which adds DER Sequence prefix when content type Oid is not 1.2.840.113549.1.7.1.

Spec is mentioning to use inner content: https://tools.ietf.org/html/rfc2315#section-7 (Notes, pt. 2).

I got initial fix ready to match Windows behavior on other platforms but will read spec more carefully before submitting a patch to make sure Windows correctly implements it.

Assuming Windows behavior is correct means that making this fix would mean that any document generated on non-Windows platform using ContentInfo not equal to Pkcs7Data is invalid and decrypted content will not be exactly correct (it will have additional DER Sequence prefix)

[krwq]

Ok reading spec a little bit more it seems that we should always encrypt only inner content (no identifier or length octets). Adding Sequence prefix seems to be related to the fact that all PKCS7 types are defined as sequence and this way we can roundtrip. In case of Pkcs7Data user has passed non ASN.1 encoded data which means we should strip all prefixes in order to give user the data back

[karelz]

Fixed in 2.1 servicing via PR dotnet/corefx#30432. Will be part of 2.1.3 release.