Skip to content

Update ReadMe #545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thomhurst merged 1 commit into from
Sep 19, 2024
Merged

Update ReadMe #545

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 63 additions & 63 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,27 @@ It aims to build upon the useful features of both while trying to address any pa

### Scenario: Building the test project

#### macos-latest

```

BenchmarkDotNet v0.14.0, macOS Sonoma 14.6.1 (23G93) [Darwin 23.6.0]
Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
.NET SDK 8.0.401
[Host] : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD
DefaultJob : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD


```
| Method | Mean | Error | StdDev |
|------------- |---------:|---------:|---------:|
| Build_TUnit | 787.0 ms | 11.19 ms | 16.75 ms |
| Build_NUnit | 710.3 ms | 13.55 ms | 15.60 ms |
| Build_xUnit | 704.4 ms | 9.88 ms | 9.24 ms |
| Build_MSTest | 754.4 ms | 12.89 ms | 12.06 ms |



#### ubuntu-latest

```
Expand All @@ -201,10 +222,10 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
```
| Method | Mean | Error | StdDev |
|------------- |--------:|---------:|---------:|
| Build_TUnit | 1.535 s | 0.0296 s | 0.0290 s |
| Build_NUnit | 1.399 s | 0.0198 s | 0.0154 s |
| Build_xUnit | 1.412 s | 0.0217 s | 0.0181 s |
| Build_MSTest | 1.427 s | 0.0228 s | 0.0214 s |
| Build_TUnit | 1.517 s | 0.0230 s | 0.0216 s |
| Build_NUnit | 1.383 s | 0.0267 s | 0.0237 s |
| Build_xUnit | 1.400 s | 0.0198 s | 0.0185 s |
| Build_MSTest | 1.418 s | 0.0216 s | 0.0202 s |



Expand All @@ -222,31 +243,10 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
```
| Method | Mean | Error | StdDev |
|------------- |--------:|---------:|---------:|
| Build_TUnit | 1.401 s | 0.0252 s | 0.0223 s |
| Build_NUnit | 1.328 s | 0.0261 s | 0.0321 s |
| Build_xUnit | 1.310 s | 0.0129 s | 0.0121 s |
| Build_MSTest | 1.322 s | 0.0131 s | 0.0116 s |



#### macos-latest

```

BenchmarkDotNet v0.14.0, macOS Sonoma 14.6.1 (23G93) [Darwin 23.6.0]
Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
.NET SDK 8.0.401
[Host] : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD
DefaultJob : .NET 8.0.8 (8.0.824.36612), Arm64 RyuJIT AdvSIMD


```
| Method | Mean | Error | StdDev | Median |
|------------- |---------:|---------:|----------:|---------:|
| Build_TUnit | 969.3 ms | 41.27 ms | 119.08 ms | 939.0 ms |
| Build_NUnit | 712.8 ms | 13.30 ms | 25.62 ms | 702.8 ms |
| Build_xUnit | 728.1 ms | 14.48 ms | 34.14 ms | 712.7 ms |
| Build_MSTest | 761.2 ms | 14.10 ms | 12.50 ms | 757.8 ms |
| Build_TUnit | 1.410 s | 0.0275 s | 0.0376 s |
| Build_NUnit | 1.290 s | 0.0106 s | 0.0083 s |
| Build_xUnit | 1.307 s | 0.0256 s | 0.0252 s |
| Build_MSTest | 1.322 s | 0.0108 s | 0.0084 s |


### Scenario: A single test that completes instantly (including spawning a new process and initialising the test framework)
Expand All @@ -265,11 +265,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
```
| Method | Mean | Error | StdDev |
|---------- |------------:|----------:|----------:|
| TUnit_AOT | 43.59 ms | 1.181 ms | 3.481 ms |
| TUnit | 786.41 ms | 15.349 ms | 18.850 ms |
| NUnit | 1,413.53 ms | 19.386 ms | 18.133 ms |
| xUnit | 1,380.03 ms | 26.413 ms | 24.706 ms |
| MSTest | 1,249.64 ms | 21.385 ms | 20.004 ms |
| TUnit_AOT | 45.52 ms | 1.123 ms | 3.312 ms |
| TUnit | 731.72 ms | 14.486 ms | 19.338 ms |
| NUnit | 1,311.58 ms | 11.782 ms | 9.839 ms |
| xUnit | 1,298.54 ms | 14.451 ms | 13.518 ms |
| MSTest | 1,191.60 ms | 14.747 ms | 13.794 ms |



Expand All @@ -287,11 +287,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
```
| Method | Mean | Error | StdDev |
|---------- |------------:|----------:|----------:|
| TUnit_AOT | 78.11 ms | 1.557 ms | 1.793 ms |
| TUnit | 749.18 ms | 14.896 ms | 19.369 ms |
| NUnit | 1,303.82 ms | 8.412 ms | 7.869 ms |
| xUnit | 1,280.55 ms | 9.084 ms | 8.052 ms |
| MSTest | 1,162.80 ms | 11.494 ms | 10.752 ms |
| TUnit_AOT | 79.57 ms | 1.582 ms | 1.758 ms |
| TUnit | 760.44 ms | 15.071 ms | 20.120 ms |
| NUnit | 1,319.92 ms | 15.638 ms | 14.628 ms |
| xUnit | 1,308.98 ms | 22.597 ms | 21.138 ms |
| MSTest | 1,185.25 ms | 16.841 ms | 15.753 ms |



Expand All @@ -307,13 +307,13 @@ Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores


```
| Method | Mean | Error | StdDev |
|---------- |---------:|---------:|---------:|
| TUnit_AOT | 125.2 ms | 9.10 ms | 26.55 ms |
| TUnit | 553.5 ms | 11.04 ms | 26.03 ms |
| NUnit | 937.9 ms | 26.71 ms | 77.06 ms |
| xUnit | 859.0 ms | 20.05 ms | 58.80 ms |
| MSTest | 798.7 ms | 18.06 ms | 52.40 ms |
| Method | Mean | Error | StdDev | Median |
|---------- |----------:|----------:|-----------:|----------:|
| TUnit_AOT | 93.11 ms | 5.657 ms | 16.502 ms | 86.33 ms |
| TUnit | 421.62 ms | 7.932 ms | 20.616 ms | 412.31 ms |
| NUnit | 689.12 ms | 7.113 ms | 6.654 ms | 689.90 ms |
| xUnit | 887.74 ms | 39.790 ms | 115.438 ms | 924.08 ms |
| MSTest | 664.78 ms | 12.946 ms | 19.770 ms | 667.13 ms |


### Scenario: A test that takes 50ms to execute, repeated 100 times (including spawning a new process and initialising the test framework)
Expand All @@ -332,11 +332,11 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
```
| Method | Mean | Error | StdDev |
|---------- |------------:|----------:|----------:|
| TUnit_AOT | 89.17 ms | 1.806 ms | 5.326 ms |
| TUnit | 816.96 ms | 15.746 ms | 20.475 ms |
| NUnit | 6,302.98 ms | 9.893 ms | 9.254 ms |
| xUnit | 6,353.12 ms | 14.962 ms | 13.996 ms |
| MSTest | 6,278.93 ms | 6.041 ms | 5.650 ms |
| TUnit_AOT | 91.40 ms | 1.821 ms | 5.047 ms |
| TUnit | 831.20 ms | 16.107 ms | 26.011 ms |
| NUnit | 6,343.54 ms | 8.060 ms | 7.539 ms |
| xUnit | 6,424.20 ms | 14.931 ms | 13.236 ms |
| MSTest | 6,336.65 ms | 21.178 ms | 19.810 ms |



Expand All @@ -352,13 +352,13 @@ AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores


```
| Method | Mean | Error | StdDev |
|---------- |-----------:|----------:|----------:|
| TUnit_AOT | 136.6 ms | 2.72 ms | 6.52 ms |
| TUnit | 846.1 ms | 16.13 ms | 20.40 ms |
| NUnit | 8,760.6 ms | 171.45 ms | 291.13 ms |
| xUnit | 8,719.2 ms | 170.39 ms | 315.84 ms |
| MSTest | 8,651.6 ms | 168.87 ms | 341.12 ms |
| Method | Mean | Error | StdDev |
|---------- |-----------:|---------:|---------:|
| TUnit_AOT | 134.3 ms | 2.68 ms | 6.52 ms |
| TUnit | 827.7 ms | 16.36 ms | 20.09 ms |
| NUnit | 7,494.2 ms | 18.08 ms | 16.91 ms |
| xUnit | 7,478.1 ms | 18.89 ms | 16.74 ms |
| MSTest | 7,455.6 ms | 22.37 ms | 20.93 ms |



Expand All @@ -376,11 +376,11 @@ Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
```
| Method | Mean | Error | StdDev |
|---------- |------------:|----------:|----------:|
| TUnit_AOT | 243.2 ms | 17.98 ms | 52.75 ms |
| TUnit | 604.9 ms | 22.67 ms | 66.13 ms |
| NUnit | 14,064.7 ms | 276.93 ms | 492.24 ms |
| xUnit | 14,659.0 ms | 293.11 ms | 564.72 ms |
| MSTest | 14,443.4 ms | 286.97 ms | 531.92 ms |
| TUnit_AOT | 251.0 ms | 15.49 ms | 45.66 ms |
| TUnit | 673.0 ms | 21.68 ms | 63.93 ms |
| NUnit | 14,295.6 ms | 284.12 ms | 526.64 ms |
| xUnit | 14,482.0 ms | 285.39 ms | 460.85 ms |
| MSTest | 14,455.9 ms | 286.31 ms | 537.77 ms |