[release/6.0-staging] Disable implicit rejection for RSA PKCS#1 v1.5

[github-actions]

Backport of #95157 to release/6.0-staging

/cc @vcsjones @bartonjs

Customer Impact

OpenSSL introduced a behavior change in EVP_PKEY_decrypt where RSA decryption with PKCS#1 v1.5 padding no longer produces an error with invalid padding. Instead, it returns “random” data.

Customers using affected versions of OpenSSL will no longer get consistent and documented behavior from RSA.Decrypt.

This change is reacting to a breaking behavioral change in a dependency, OpenSSL, so that RSA.Decrypt continues to function as it is documented to, and to be consistent with Windows and other platforms.

Testing

This behavior change was identified with a test in #95115. Additional tests were added.

Risk

Low. This is reacting to a change in OpenSSL by settings a particular flag. Existing tests cover RSA.Decrypt’s behavior.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Backport of #95157 to release/6.0-staging

/cc @vcsjones

Customer Impact

Testing

Risk

IMPORTANT: If this backport is for a servicing release, please verify that:

• The PR target branch is release/X.0-staging, not release/X.0.

• If the change touches code that ships in a NuGet package, you have added the necessary package authoring and gotten it explicitly reviewed.

Author:	github-actions[bot]
Assignees:	-
Labels:	area-System.Security
Milestone:	-

[neverpanic]

Just to make sure that this doesn't end up being merged just because nobody saw the original comment: This re-enables a Bleichenbacher timing oracle attack against PKCS#1v1.5 decryption. See https://people.redhat.com/~hkario/marvin/ for details and #95157 (comment) for a comment by the researcher who published the vulnerability and proposed the change in OpenSSL.

I don't think this should be merged.

[carlossanlop]

@GrabYourPitchforks addressed the above feedback in detail in the other backport PR. We are good to merge.