PBKDF2 one-shot by vcsjones · Pull Request #48107 · dotnet/runtime

vcsjones · 2021-02-10T14:27:22Z

Work left to be done:

Wait for API approval and feedback (and modify PR accordingly)
- XML documentation (TBD after approval)
Handle other platforms accordingly (Android, Browser)
Comprehensive tests and edge cases
- Zero length values (empty password bytes, etc)
- Overlapping buffers
Perf numbers on throughput and allocations.
We should use the throwing (non-substituting) version of UTF-8 when performing the chars -> bytes
Use BCryptKeyDerivation for Windows 8+.

Implementation notes:

In Improve performance of Rfc2898DeriveBytes #24897 I (incorrectly) determined that OpenSSL 1.0 did not have PKCS5_PBKDF2_HMAC. This was a PEBCAK error while reading documentation. So no shimming required.
MacOS, Linux, and Windows 7 should not allocate, assuming you are not using one of the allocating helpers. Windows 8+ will mostly be non-allocating. They will allocate a handle, though it can be removed. I am doing some research to determine if removing the handle allocation makes any material impact on real world use cases.

ghost · 2021-02-10T14:27:27Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

ghost · 2021-02-10T14:27:28Z

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

This is a draft for PBKDF2 one-shots (an API that is yet-to-be-approved). Opening as a draft to get some feedback from the CI system.

Work left to be done:

Wait for API approval and feedback (and modify PR accordingly)
Handle other platforms accordingly (Android, Browser)
XML documentation
Comprehensive tests and edge cases
- Zero length values (empty password bytes, etc)
- Overlapping buffers
Perf numbers on throughput and allocations.

Implementation notes:

In Improve performance of Rfc2898DeriveBytes #24897 I (incorrectly) determined that OpenSSL 1.0 did not have PKCS5_PBKDF2_HMAC. This was a PEBCAK error while reading documentation. So no shimming required.
MacOS, Linux, and Windows 10 should not allocate. Windows <= 10 will mostly be non-allocating. They will use cached algorithm handles.

Author:	vcsjones
Assignees:	-
Labels:	`area-System.Security`, `new-api-needs-documentation`
Milestone:	-

vcsjones · 2021-02-10T14:29:56Z

Unrelated but exciting: All of the macOS work was done on a M1 Mac Mini without Rosetta (running as ARM64). Very cool to see dotnet/runtime in good shape for Apple Silicon.

vcsjones · 2021-02-10T17:37:40Z

Hm. Trying to use the pseudo handles in Windows 7 actually crashes the process with an access violation, so we can't even get an NTSTATUS back saying it failed. Do we just do an OS version check?

Reorder to keep alphabetic consistency. Add comment documentation. Tighten validation between password and salt inputs and their length. NULL input requires a zero length.

While here, use a stack allocation for small password inputs, which seems likely.

vcsjones

@bartonjs @GrabYourPitchforks I think this is at a point where it can be looked at in more scrutiny and might nearly resemble something complete, barring some self-review questions.

vcsjones · 2021-02-16T23:46:36Z

Apology in advance for changing the PR title and breaking email inbox threading.

GrabYourPitchforks

Reviewed the Windows implementation and skimmed the non-Windows implementations. Generally LGTM. Some open questions about whether avoiding the allocation is worth it in the Windows case.

Checks that inputs are valid before doing any allocations.

vcsjones added 3 commits February 9, 2021 19:18

Basic implementation of one-shot PBKDF2 for all platforms.

5f15fdf

Use HashAlgorithmName to PAL

c77b4e7

Use psuedo handles if available on the Windows platform.

1bf9d93

ghost added area-System.Security new-api-needs-documentation labels Feb 10, 2021

vcsjones commented Feb 10, 2021

View reviewed changes

Comment thread ...aries/System.Security.Cryptography.Algorithms/ref/System.Security.Cryptography.Algorithms.cs

vcsjones added 2 commits February 10, 2021 09:54

Fix compilation

22ead64

Undo inadvertent change in ref file.

abe4c36

bartonjs reviewed Feb 10, 2021

View reviewed changes

Comment thread ...urity.Cryptography.Algorithms/src/System/Security/Cryptography/Rfc2898DeriveBytes.OneShot.cs Outdated

bartonjs reviewed Feb 10, 2021

View reviewed changes

Comment thread ...urity.Cryptography.Algorithms/src/System/Security/Cryptography/Rfc2898DeriveBytes.OneShot.cs Outdated

bartonjs reviewed Feb 10, 2021

View reviewed changes

Comment thread src/libraries/Native/Unix/System.Security.Cryptography.Native/pal_evp.h Outdated

bartonjs reviewed Feb 10, 2021

View reviewed changes

Comment thread src/libraries/System.Security.Cryptography.Algorithms/tests/Rfc2898OneShotTests.cs Outdated

vcsjones added 3 commits February 10, 2021 13:02

Detect Windows 7 instead of relying on the NTSTATUS.

859112e

Loosen restrictions and add deeper tests

62d0ec2

Fix empty password on macOS.

d8e5c2d

vcsjones commented Feb 10, 2021

View reviewed changes

Comment thread ...urity.Cryptography.Algorithms/src/System/Security/Cryptography/Rfc2898DeriveBytes.OneShot.cs Outdated

vcsjones commented Feb 11, 2021

View reviewed changes

Comment thread ...urity.Cryptography.Algorithms/src/System/Security/Cryptography/Rfc2898DeriveBytes.OneShot.cs Outdated

vcsjones added 4 commits February 10, 2021 21:14

Permit empty salts

5ef000d

Fix OpenSSL 1.0 with empty values

d1c34a3

Rename exported function

a873c76

Rename Apple exported function

eed78a4

bartonjs reviewed Feb 11, 2021

View reviewed changes

Comment thread src/libraries/Native/Unix/System.Security.Cryptography.Native/entrypoints.c Outdated

vcsjones added 3 commits February 11, 2021 13:55

Cleanup native PALs.

51b77c1

Reorder to keep alphabetic consistency. Add comment documentation. Tighten validation between password and salt inputs and their length. NULL input requires a zero length.

Merge remote-tracking branch 'ms/master' into 24897-pbkdf2-one-shot

61140da

Fix issue with not slicing the rented buffer.

0210441

While here, use a stack allocation for small password inputs, which seems likely.

runfoapp Bot mentioned this pull request Feb 12, 2021

Test Failure : System.Net.Security.Tests.SslStreamNetworkStreamTest.SslStream_ClientCertificate_SendsChain #48091

Closed

vcsjones added 4 commits February 16, 2021 14:57

Tests for overlapping input / output buffers.

ddfd960

Add tests around HMAC block size boundaries.

5a2c91a

Fix block sizes.

b82528e

XML documentation.

dc72dc3

vcsjones marked this pull request as ready for review February 16, 2021 23:31

vcsjones changed the title ~~[WIP] PBKDF2 one-shot~~ PBKDF2 one-shot Feb 16, 2021

vcsjones commented Feb 16, 2021

View reviewed changes

GrabYourPitchforks reviewed Feb 17, 2021

View reviewed changes

vcsjones added 3 commits February 16, 2021 21:15

Code review feedback.

3ec9a55

Merge remote-tracking branch 'ms/master' into 24897-pbkdf2-one-shot

be3ca75

Move high iterations test to OuterLoop.

19faa41

bartonjs reviewed Feb 17, 2021

View reviewed changes

Comment thread src/libraries/Common/src/Interop/Windows/BCrypt/Interop.Blobs.cs Outdated

bartonjs reviewed Feb 17, 2021

View reviewed changes

Comment thread ...m.Security.Cryptography.Algorithms/src/Internal/Cryptography/Pbkdf2Implementation.Windows.cs

bartonjs reviewed Feb 17, 2021

View reviewed changes

Comment thread ...m.Security.Cryptography.Algorithms/src/Internal/Cryptography/Pbkdf2Implementation.Windows.cs

bartonjs reviewed Feb 17, 2021

View reviewed changes

Comment thread ...m.Security.Cryptography.Algorithms/src/Internal/Cryptography/Pbkdf2Implementation.Windows.cs

bartonjs reviewed Feb 17, 2021

View reviewed changes

Comment thread ...urity.Cryptography.Algorithms/src/System/Security/Cryptography/Rfc2898DeriveBytes.OneShot.cs Outdated

vcsjones added 4 commits February 17, 2021 13:38

Dispose of handles in error conditions

b98a679

Rename buffer descriptors with clearer prefix

bc02881

Refactor input validation.

8f00cf6

Checks that inputs are valid before doing any allocations.

Test for large passwords and salts.

b2199b1

vcsjones requested a review from bartonjs February 18, 2021 20:02

bartonjs reviewed Feb 19, 2021

View reviewed changes

Comment thread ...m.Security.Cryptography.Algorithms/src/Internal/Cryptography/Pbkdf2Implementation.Windows.cs

vcsjones added 2 commits February 19, 2021 12:35

Merge remote-tracking branch 'ms/master' into 24897-pbkdf2-one-shot

c78ca1a

Use asserts to make what should be pre-validated paths clearer.

b881b3a

bartonjs approved these changes Feb 19, 2021

View reviewed changes

bartonjs merged commit 0c47471 into dotnet:master Feb 22, 2021

vcsjones deleted the 24897-pbkdf2-one-shot branch February 22, 2021 20:55

omajid mentioned this pull request Mar 15, 2021

Use the framework implementation of PBKDF2/RFC2898 dotnet/aspnetcore#30941

Closed

ghost locked as resolved and limited conversation to collaborators Mar 24, 2021

Conversation

vcsjones commented Feb 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Feb 10, 2021

Uh oh!

ghost commented Feb 10, 2021

Uh oh!

vcsjones commented Feb 10, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vcsjones commented Feb 10, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vcsjones left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vcsjones commented Feb 16, 2021

Uh oh!

GrabYourPitchforks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vcsjones commented Feb 10, 2021 •

edited

Loading