[RFC/Experimental] NIZK Mutual Authentication Flow

[Radeonares32]

Overview
This PR introduces an experimental Non-Interactive Zero-Knowledge (NIZK) mutual authentication layer to the BoringTun handshake. It allows peers to prove knowledge of their static private keys without transmitting them in any form (encrypted or otherwise) during the Noise handshake, utilizing the Schnorr Identification Protocol transformed via the Fiat-Shamir heuristic.

Motivation
Standard WireGuard/Noise handshakes provide strong authentication, but adding a Zero-Knowledge layer opens the door for:

Anonymous Group Membership: Proving participation in a set without revealing a specific identity.
Enhanced Deniability: Strengthening the deniable nature of the handshake by using proofs instead of direct key-exchange-based authentication.
Privacy-Preserving Extensions: A foundation for future features like post-quantum signatures or OPAQUE-like credential systems.
Technical Implementation
Cryptographic Primitive: Implemented a Schnorr Proof-of-Knowledge (PoK) over the Edwards25519 group.
x25519 Compatibility: Since BoringTun uses x25519 (Montgomery), we implement "Edwards-lifting" with a sign-bit check during verification to ensure compatibility with existing static keys.
Clamping Aware: The implementation correctly handles x25519 bit-clamping to ensure the PoK remains valid for derived public keys.
Transcript Binding: Proofs are bound to the peer's static public key and the session context to prevent replay attacks.
Random Oracle: Uses Blake2s (already in the codebase) for the Fiat-Shamir challenge calculation.
Impact & Safety
Isolation: All changes are gated behind the experimental-nizk feature flag. The core BoringTun performance and protocol remain untouched for standard users.
Packet Size: Adds a 64-byte NIZK proof to both

HandshakeInit
and

HandshakeResponse
packets (Total +128 bytes per full handshake).
Security Audit: This is an RFC (Request for Comments). The implementation is experimental and has not been formally audited.
Verification
Full coverage unit tests in

noise/nizk.rs
.
End-to-end mutual authentication integration test in

noise/mod.rs
(

test_nizk_handshake_full_flow
).
Verified that all standard unit tests pass without the feature flag.

[Radeonares32]

Thanks for taking a look at this experimental PR.

After further review, I realize that some parts of the construction
(particularly the relationship between x25519 keys and the Edwards
representation used in the Schnorr proof) need a clearer formal
definition and stronger transcript binding.

This PR should be considered an experimental research prototype
Rather than a production-ready construction.

I'm happy to iterate on the design or move this work into a separate
research branch if that is more appropriate for the project