From 7d11670936d95ce7f0eaef28721c8f9b1c8d2832 Mon Sep 17 00:00:00 2001 From: thomhurst <30480171_thomhurst@users.noreply.github.com> Date: Thu, 19 Sep 2024 14:33:54 +0000 Subject: [PATCH] Update README.md --- README.md | 126 +++++++++++++++++++++++++++--------------------------- 1 file changed, 63 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index da0bcc515c..2fc0717a3a 100644 --- a/README.md +++ b/README.md @@ -187,6 +187,27 @@ It aims to build upon the useful features of both while trying to address any pa ### Scenario: Building the test project +#### macos-latest + +``` + +BenchmarkDotNet v0.14.0, macOS Sonoma 14.6.1 (23G93) [Darwin 23.6.0] +Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores +.NET SDK 8.0.401 + [Host] : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD + DefaultJob : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD + + +``` +| Method | Mean | Error | StdDev | +|------------- |---------:|---------:|---------:| +| Build_TUnit | 787.0 ms | 11.19 ms | 16.75 ms | +| Build_NUnit | 710.3 ms | 13.55 ms | 15.60 ms | +| Build_xUnit | 704.4 ms | 9.88 ms | 9.24 ms | +| Build_MSTest | 754.4 ms | 12.89 ms | 12.06 ms | + + + #### ubuntu-latest ``` @@ -201,10 +222,10 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` | Method | Mean | Error | StdDev | |------------- |--------:|---------:|---------:| -| Build_TUnit | 1.535 s | 0.0296 s | 0.0290 s | -| Build_NUnit | 1.399 s | 0.0198 s | 0.0154 s | -| Build_xUnit | 1.412 s | 0.0217 s | 0.0181 s | -| Build_MSTest | 1.427 s | 0.0228 s | 0.0214 s | +| Build_TUnit | 1.517 s | 0.0230 s | 0.0216 s | +| Build_NUnit | 1.383 s | 0.0267 s | 0.0237 s | +| Build_xUnit | 1.400 s | 0.0198 s | 0.0185 s | +| Build_MSTest | 1.418 s | 0.0216 s | 0.0202 s | @@ -222,31 +243,10 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` | Method | Mean | Error | StdDev | |------------- |--------:|---------:|---------:| -| Build_TUnit | 1.401 s | 0.0252 s | 0.0223 s | -| Build_NUnit | 1.328 s | 0.0261 s | 0.0321 s | -| Build_xUnit | 1.310 s | 0.0129 s | 0.0121 s | -| Build_MSTest | 1.322 s | 0.0131 s | 0.0116 s | - - - -#### macos-latest - -``` - -BenchmarkDotNet v0.14.0, macOS Sonoma 14.6.1 (23G93) [Darwin 23.6.0] -Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores -.NET SDK 8.0.401 - [Host] : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD - DefaultJob : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD - - -``` -| Method | Mean | Error | StdDev | Median | -|------------- |---------:|---------:|----------:|---------:| -| Build_TUnit | 969.3 ms | 41.27 ms | 119.08 ms | 939.0 ms | -| Build_NUnit | 712.8 ms | 13.30 ms | 25.62 ms | 702.8 ms | -| Build_xUnit | 728.1 ms | 14.48 ms | 34.14 ms | 712.7 ms | -| Build_MSTest | 761.2 ms | 14.10 ms | 12.50 ms | 757.8 ms | +| Build_TUnit | 1.410 s | 0.0275 s | 0.0376 s | +| Build_NUnit | 1.290 s | 0.0106 s | 0.0083 s | +| Build_xUnit | 1.307 s | 0.0256 s | 0.0252 s | +| Build_MSTest | 1.322 s | 0.0108 s | 0.0084 s | ### Scenario: A single test that completes instantly (including spawning a new process and initialising the test framework) @@ -265,11 +265,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` | Method | Mean | Error | StdDev | |---------- |------------:|----------:|----------:| -| TUnit_AOT | 43.59 ms | 1.181 ms | 3.481 ms | -| TUnit | 786.41 ms | 15.349 ms | 18.850 ms | -| NUnit | 1,413.53 ms | 19.386 ms | 18.133 ms | -| xUnit | 1,380.03 ms | 26.413 ms | 24.706 ms | -| MSTest | 1,249.64 ms | 21.385 ms | 20.004 ms | +| TUnit_AOT | 45.52 ms | 1.123 ms | 3.312 ms | +| TUnit | 731.72 ms | 14.486 ms | 19.338 ms | +| NUnit | 1,311.58 ms | 11.782 ms | 9.839 ms | +| xUnit | 1,298.54 ms | 14.451 ms | 13.518 ms | +| MSTest | 1,191.60 ms | 14.747 ms | 13.794 ms | @@ -287,11 +287,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` | Method | Mean | Error | StdDev | |---------- |------------:|----------:|----------:| -| TUnit_AOT | 78.11 ms | 1.557 ms | 1.793 ms | -| TUnit | 749.18 ms | 14.896 ms | 19.369 ms | -| NUnit | 1,303.82 ms | 8.412 ms | 7.869 ms | -| xUnit | 1,280.55 ms | 9.084 ms | 8.052 ms | -| MSTest | 1,162.80 ms | 11.494 ms | 10.752 ms | +| TUnit_AOT | 79.57 ms | 1.582 ms | 1.758 ms | +| TUnit | 760.44 ms | 15.071 ms | 20.120 ms | +| NUnit | 1,319.92 ms | 15.638 ms | 14.628 ms | +| xUnit | 1,308.98 ms | 22.597 ms | 21.138 ms | +| MSTest | 1,185.25 ms | 16.841 ms | 15.753 ms | @@ -307,13 +307,13 @@ Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores ``` -| Method | Mean | Error | StdDev | -|---------- |---------:|---------:|---------:| -| TUnit_AOT | 125.2 ms | 9.10 ms | 26.55 ms | -| TUnit | 553.5 ms | 11.04 ms | 26.03 ms | -| NUnit | 937.9 ms | 26.71 ms | 77.06 ms | -| xUnit | 859.0 ms | 20.05 ms | 58.80 ms | -| MSTest | 798.7 ms | 18.06 ms | 52.40 ms | +| Method | Mean | Error | StdDev | Median | +|---------- |----------:|----------:|-----------:|----------:| +| TUnit_AOT | 93.11 ms | 5.657 ms | 16.502 ms | 86.33 ms | +| TUnit | 421.62 ms | 7.932 ms | 20.616 ms | 412.31 ms | +| NUnit | 689.12 ms | 7.113 ms | 6.654 ms | 689.90 ms | +| xUnit | 887.74 ms | 39.790 ms | 115.438 ms | 924.08 ms | +| MSTest | 664.78 ms | 12.946 ms | 19.770 ms | 667.13 ms | ### Scenario: A test that takes 50ms to execute, repeated 100 times (including spawning a new process and initialising the test framework) @@ -332,11 +332,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` | Method | Mean | Error | StdDev | |---------- |------------:|----------:|----------:| -| TUnit_AOT | 89.17 ms | 1.806 ms | 5.326 ms | -| TUnit | 816.96 ms | 15.746 ms | 20.475 ms | -| NUnit | 6,302.98 ms | 9.893 ms | 9.254 ms | -| xUnit | 6,353.12 ms | 14.962 ms | 13.996 ms | -| MSTest | 6,278.93 ms | 6.041 ms | 5.650 ms | +| TUnit_AOT | 91.40 ms | 1.821 ms | 5.047 ms | +| TUnit | 831.20 ms | 16.107 ms | 26.011 ms | +| NUnit | 6,343.54 ms | 8.060 ms | 7.539 ms | +| xUnit | 6,424.20 ms | 14.931 ms | 13.236 ms | +| MSTest | 6,336.65 ms | 21.178 ms | 19.810 ms | @@ -352,13 +352,13 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores ``` -| Method | Mean | Error | StdDev | -|---------- |-----------:|----------:|----------:| -| TUnit_AOT | 136.6 ms | 2.72 ms | 6.52 ms | -| TUnit | 846.1 ms | 16.13 ms | 20.40 ms | -| NUnit | 8,760.6 ms | 171.45 ms | 291.13 ms | -| xUnit | 8,719.2 ms | 170.39 ms | 315.84 ms | -| MSTest | 8,651.6 ms | 168.87 ms | 341.12 ms | +| Method | Mean | Error | StdDev | +|---------- |-----------:|---------:|---------:| +| TUnit_AOT | 134.3 ms | 2.68 ms | 6.52 ms | +| TUnit | 827.7 ms | 16.36 ms | 20.09 ms | +| NUnit | 7,494.2 ms | 18.08 ms | 16.91 ms | +| xUnit | 7,478.1 ms | 18.89 ms | 16.74 ms | +| MSTest | 7,455.6 ms | 22.37 ms | 20.93 ms | @@ -376,11 +376,11 @@ Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores ``` | Method | Mean | Error | StdDev | |---------- |------------:|----------:|----------:| -| TUnit_AOT | 243.2 ms | 17.98 ms | 52.75 ms | -| TUnit | 604.9 ms | 22.67 ms | 66.13 ms | -| NUnit | 14,064.7 ms | 276.93 ms | 492.24 ms | -| xUnit | 14,659.0 ms | 293.11 ms | 564.72 ms | -| MSTest | 14,443.4 ms | 286.97 ms | 531.92 ms | +| TUnit_AOT | 251.0 ms | 15.49 ms | 45.66 ms | +| TUnit | 673.0 ms | 21.68 ms | 63.93 ms | +| NUnit | 14,295.6 ms | 284.12 ms | 526.64 ms | +| xUnit | 14,482.0 ms | 285.39 ms | 460.85 ms | +| MSTest | 14,455.9 ms | 286.31 ms | 537.77 ms |