Rewrite BigInteger internals to use native-width (nuint) limbs

[stephentoub]

This rewrites the internal implementation of System.Numerics.BigInteger to use native-width (nuint) limbs instead of uint limbs. On 64-bit platforms, this halves the number of limbs needed to represent a value, improving throughput of all multi-limb arithmetic. The public API surface remains unchanged.

Core Representation Change

• Internal _bits array changed from uint[] to nuint[], with all arithmetic primitives updated accordingly.
• On 64-bit platforms, each limb holds 64 bits instead of 32, halving iteration counts for schoolbook algorithms.
• _sign remains int to avoid regressions on small inline values.
• Parse/ToString updated to work with nuint limbs while maintaining exact formatting behavior.

Algorithmic Improvements

Montgomery Multiplication for ModPow

Added Montgomery multiplication with REDC for modular exponentiation when the modulus is odd. This replaces the Barrett reduction path for odd moduli, eliminating expensive division-based reduction in the inner loop.

Sliding Window Exponentiation

ModPow now uses left-to-right sliding window exponentiation (window size chosen based on exponent bit length) instead of simple square-and-multiply, reducing the number of modular multiplications.

Fused Two's Complement for Bitwise Operations

Bitwise AND, OR, and XOR on negative values now fuse the two's complement conversion with the logical operation in a single pass, avoiding separate allocation and negation steps.

Cached Powers-of-10 Table

The PowersOf1e9 table used by divide-and-conquer ToString/Parse is now cached and reused across calls, avoiding repeated expensive computation for large number formatting.

Unrolled Single-Limb Primitives

Added Mul1, MulAdd1, and SubMul1 primitives that handle the common case of multiplying a multi-limb number by a single limb. These are used in the inner loops of schoolbook multiply and division.

GCD Optimizations

• LehmerCore rewritten to avoid Int128/UInt128 overhead, using direct ulong arithmetic.
• Eliminated unnecessary array copy when one GCD operand is zero.

Division Tuning

• Burnikel-Ziegler threshold retuned for 64-bit limbs.
• DivRem helpers optimized for the wider limb size.

Hardware Intrinsics

BigMul, DivRem, and AddWithCarry primitives use BMI2 (mulx) and ADX (adcx/adox) intrinsics when available, with fallback to Math.BigMul/UInt128 arithmetic.

ModPow Small-Modulus Optimization

When the modulus fits in a uint (common for single-limb moduli on 64-bit), the inner loop uses ulong arithmetic instead of UInt128, avoiding unnecessary widening.

Bug Fixes

• Fixed SubtractSelf callers to restore borrow == 0 postcondition, fixing incorrect results in Barrett reduction (FastReducer.SubMod) when the Barrett quotient overshoots, Toom-3/Karatsuba signed subtraction, and Montgomery reduction overflow handling.
• Fixed SubWithBorrow sign-extension issue that would produce incorrect results on 32-bit platforms.
• Fixed BitwiseAnd sign handling for specific negative operand combinations.
• Fixed ToString regression for numbers near limb boundaries by processing 32-bit halves in the naive conversion loop.

Test Coverage

• Added ~1,000 lines of property-based validation tests covering arithmetic identity laws, bitwise consistency, and round-trip conversions.
• Added edge-case tests for sign combinations, SIMD vector boundaries, and Barrett/FastReducer paths with even large moduli.
• All 3,031 existing tests pass in both Debug and Release configurations.

Fixes #97780
Fixes #111708
Fixes #41495

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This PR modernizes System.Numerics.BigInteger internals by switching the limb representation from uint to native-width nuint, and updates core arithmetic/modular algorithms (notably ModPow) to better exploit 64-bit platforms while keeping the public API unchanged.

Changes:

• Reworked BigInteger arithmetic plumbing to operate on nuint limbs (32-bit on x86, 64-bit on x64), including shift/rotate, add/sub, division, GCD, and reduction helpers.
• Improved modular exponentiation for odd moduli via a Montgomery + sliding-window path, plus supporting primitives (AddWithCarry, SubWithBorrow, Mul1, MulAdd1, SubMul1, etc.).
• Updated and expanded tests to be limb-width aware and added a large new property/edge-case test suite.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/libraries/System.Runtime.Numerics/tests/System.Runtime.Numerics.Tests.csproj	Adds the new BigInteger property/edge-case test file to the test project.
src/libraries/System.Runtime.Numerics/tests/BigIntegerTests.GenericMath.cs	Updates generic math tests to be native-limb-width aware (counts/rotates/byte counts/write-bytes).
src/libraries/System.Runtime.Numerics/tests/BigInteger/op_rightshift.cs	Adjusts shift tests for native limb width and nuint-based internals.
src/libraries/System.Runtime.Numerics/tests/BigInteger/Rotate.cs	Updates rotate tests for limb-width variability; refactors theory data generation.
src/libraries/System.Runtime.Numerics/tests/BigInteger/MyBigInt.cs	Updates helper logic that assumed 4-byte limb alignment to use nint.Size.
src/libraries/System.Runtime.Numerics/tests/BigInteger/DebuggerDisplayTests.cs	Updates debugger display tests for nuint[] internal constructor and 64-bit formatting differences.
src/libraries/System.Runtime.Numerics/tests/BigInteger/BigIntegerPropertyTests.cs	Adds extensive property-based + edge-case coverage for BigInteger across sizes/thresholds/sign combos.
src/libraries/System.Runtime.Numerics/tests/BigInteger/BigInteger.SubtractTests.cs	Adds new subtract cases around 64-bit carry/borrow boundaries and large power-of-two transitions.
src/libraries/System.Runtime.Numerics/tests/BigInteger/BigInteger.AddTests.cs	Adds new add cases around 64-bit boundaries and multi-limb carry propagation patterns.
src/libraries/System.Runtime.Numerics/src/System/Numerics/NumericsHelpers.cs	Updates complement helpers to Span<nuint> and adapts integer helpers for the new limb width.
src/libraries/System.Runtime.Numerics/src/System/Numerics/Complex.cs	Minor modernizations (target-typed new, inline out vars) unrelated to BigInteger logic.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.Utils.cs	Introduces key native-limb primitives and constants (BitsPerLimb, carry/borrow, widening ops, scalar mul helpers).
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.ShiftRot.cs	Ports rotate/shift routines to nuint limbs with SIMD paths.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.PowMod.cs	Reworks modular exponentiation, including Montgomery + sliding window for odd moduli and updated single-limb paths.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.GcdInv.cs	Updates GCD/Lehmer core to handle native-limb representation and avoids unnecessary overhead on some paths.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.FastReducer.cs	Ports Barrett/FastReducer logic to nuint limbs.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.DivRem.cs	Ports and retunes division (incl. Burnikel–Ziegler threshold) for nuint limbs and new helpers.
src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.AddSub.cs	Ports add/subtract to nuint limbs using new carry/borrow primitives and bounds-check elimination patterns.
src/libraries/System.Runtime.Numerics/src/System/Number.Polyfill.cs	Removes BOM and small formatting/style adjustments.
src/libraries/Common/src/System/Number.Parsing.Common.cs	Small refactor to pattern matching for readability.
src/libraries/Common/src/System/Number.NumberBuffer.cs	Minor debug assert/style updates.
src/libraries/Common/src/System/Number.Formatting.Common.cs	Minor debug assert/style updates (pattern matching).

Comments suppressed due to low confidence (3)

src/libraries/System.Runtime.Numerics/tests/BigIntegerTests.GenericMath.cs:482

• The final destination.ToArray() assertion assumes the entire 24-byte buffer has deterministic contents. On 32-bit, the last 4 bytes were never written by any successful call in this test (max write size is 20), so those bytes can contain stale stack data. Either clear destination up front / clear the tail after each write, or only assert over the bytes that are known to have been written.

            Assert.False(BinaryIntegerHelper<BigInteger>.TryWriteBigEndian(default, Span<byte>.Empty, out bytesWritten));
            Assert.Equal(0, bytesWritten);
            Assert.Equal(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }, destination.ToArray());

src/libraries/System.Runtime.Numerics/tests/BigIntegerTests.GenericMath.cs:493

• destination is stackalloc'd as 24 bytes for all platforms, but on 32-bit the largest successful writes in this test only write 20 bytes. Any later assertions that compare the full buffer (rather than bytesWritten) can observe uninitialized stack data. Consider allocating destination with nint.Size == 8 ? 24 : 20, or clearing the span before asserting full-buffer contents.

            Span<byte> destination = stackalloc byte[24];
            int bytesWritten = 0;

            Assert.True(BinaryIntegerHelper<BigInteger>.TryWriteLittleEndian(Zero, destination, out bytesWritten));
            Assert.Equal(nint.Size, bytesWritten);
            Assert.Equal(new byte[nint.Size], destination.Slice(0, nint.Size).ToArray());

src/libraries/System.Runtime.Numerics/tests/BigIntegerTests.GenericMath.cs:549

• The final destination.ToArray() assertion in the Little Endian test compares all 24 bytes. On 32-bit, the last 4 bytes are not guaranteed to have been written by any prior successful call (max write is 20), so the expected trailing zeros can be unstable. Clear destination (or at least the tail) before the last assertion, or assert only over the bytes known to be written.

            Assert.False(BinaryIntegerHelper<BigInteger>.TryWriteLittleEndian(default, Span<byte>.Empty, out bytesWritten));
            Assert.Equal(0, bytesWritten);
            Assert.Equal(nint.Size == 8 ? new byte[] { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF } : new byte[] { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00 }, destination.ToArray());

[stephentoub]

@MihuBot benchmark BigInteger -long

[tannergooding]

Just noting to myself that I finished going through BigInteger.cs and need to resume review here.

and noting I didn't post any of the repetitive comments around nint.Size or other nits, but if we do fix those then we should fix it consistently.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/libraries/System.Runtime.Numerics/src/System/Numerics/BigIntegerCalculator.AddSub.cs:229

• The private Subtract helper doesn't assert that the final borrow is 0. Given other subtraction helpers (e.g., SubtractSelf) now enforce the borrow == 0 postcondition, consider adding a Debug.Assert(borrow == 0) at the end here as well so underflow bugs are caught early in Debug builds.

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        private static void Subtract(ReadOnlySpan<nuint> left, Span<nuint> bits, int startIndex, nuint initialBorrow)
        {
            // Executes the subtraction for one big and one single-limb integer.

            int i = startIndex;
            nuint borrow = initialBorrow;

            if (left.Length != 0)
            {
                _ = bits[left.Length - 1];
            }

            if (left.Length <= CopyToThreshold)
            {
                for (; i < left.Length; i++)
                {
                    nuint val = left[i];
                    nuint diff = val - borrow;
                    borrow = (diff > val) ? 1 : (nuint)0;
                    bits[i] = diff;
                }
            }
            else
            {
                for (; i < left.Length;)
                {
                    nuint val = left[i];
                    nuint diff = val - borrow;
                    borrow = (diff > val) ? 1 : (nuint)0;
                    bits[i] = diff;
                    i++;

                    // Once borrow is set to 0 it can not be 1 anymore.
                    // So the tail of the loop is just the movement of argument values to result span.
                    if (borrow == 0)
                    {
                        break;
                    }
                }

                if (i < left.Length)
                {
                    CopyTail(left, bits, i);
                }
            }
        }

[stephentoub]

@EgorBot -linux_amd -osx_arm64

using System.Numerics;
using System.Text;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    [Benchmark]
    [Arguments(3000)]
    public StringBuilder PiDigits(int n) => PiDigitsRunner.Run(n);
}

public class PiDigitsRunner
{
    BigInteger q, r, s, t, u, v, w;
    int i;
    StringBuilder strBuf = new StringBuilder(40), lastBuf = null;
    int n;

    PiDigitsRunner(int n) { this.n = n; }

    void compose_r(int bq, int br, int bs, int bt)
    {
        u = r * bs;
        r *= bq;
        v = t * br;
        r += v;
        t *= bt;
        t += u;
        s *= bt;
        u = q * bs;
        s += u;
        q *= bq;
    }

    void compose_l(int bq, int br, int bs, int bt)
    {
        r *= bt;
        u = q * br;
        r += u;
        u = t * bs;
        t *= bt;
        v = s * br;
        t += v;
        s *= bq;
        s += u;
        q *= bq;
    }

    int extract(int j)
    {
        u = q * j;
        u += r;
        v = s * j;
        v += t;
        w = u / v;
        return (int)w;
    }

    bool prdigit(int y)
    {
        strBuf.Append(y);
        if (++i % 10 == 0 || i == n)
        {
            if (i % 10 != 0)
                for (int j = 10 - (i % 10); j > 0; j--)
                    strBuf.Append(" ");
            strBuf.Append("\t:");
            strBuf.Append(i);
            lastBuf = strBuf;
            strBuf = new StringBuilder(40);
        }
        return i == n;
    }

    void RunInner()
    {
        int k = 1;
        i = 0;
        q = 1; r = 0; s = 0; t = 1;
        for (;;)
        {
            int y = extract(3);
            if (y == extract(4))
            {
                if (prdigit(y)) return;
                compose_r(10, -10 * y, 0, 1);
            }
            else
            {
                compose_l(k, 4 * k + 2, 0, 2 * k + 1);
                k++;
            }
        }
    }

    public static StringBuilder Run(int n)
    {
        var m = new PiDigitsRunner(n);
        m.RunInner();
        return m.lastBuf;
    }
}