Skip to content

fix: enhance error messages for collection element expectations#953

Merged
vbreuss merged 1 commit into
mainfrom
topic/improve-collection-element-error-message
May 16, 2026
Merged

fix: enhance error messages for collection element expectations#953
vbreuss merged 1 commit into
mainfrom
topic/improve-collection-element-error-message

Conversation

@vbreuss
Copy link
Copy Markdown
Member

@vbreuss vbreuss commented May 16, 2026

This PR updates the collection-element ComplyWith expectations to enrich failure output with additional diagnostic contexts (notably the evaluated collection and, for some quantifiers, matching/non-matching item lists), and updates the corresponding tests to assert the new message shapes.

Changes:

  • Add Collection context to ComplyWith failures for sync/async enumerable element quantifiers.
  • Add quantifier-dependent contexts (Not matching items / Matching items) to improve failure diagnostics for All-style quantifiers.
  • Update unit tests and internal tests to match the revised failure messages (including AsPrefix() where the message now has additional suffix contexts).

@vbreuss vbreuss self-assigned this May 16, 2026
Copilot AI review requested due to automatic review settings May 16, 2026 18:11
@vbreuss vbreuss added the bug Something isn't working label May 16, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the collection-element ComplyWith expectations to enrich failure output with additional diagnostic contexts (notably the evaluated collection and, for some quantifiers, matching/non-matching item lists), and updates the corresponding tests to assert the new message shapes.

Changes:

  • Add Collection context to ComplyWith failures for sync/async enumerable element quantifiers.
  • Add quantifier-dependent contexts (Not matching items / Matching items) to improve failure diagnostics for All-style quantifiers.
  • Update unit tests and internal tests to match the revised failure messages (including AsPrefix() where the message now has additional suffix contexts).

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
Tests/aweXpect.Tests/Collections/ThatEnumerable.Exactly.ComplyWith.Tests.cs Updates expected failure messages to include Collection: context.
Tests/aweXpect.Tests/Collections/ThatEnumerable.AtLeast.ComplyWith.Tests.cs Updates expected failure message to include Collection: context.
Tests/aweXpect.Tests/Collections/ThatEnumerable.All.ComplyWith.Tests.cs Updates expected failure messages to include Not matching items: and Collection: contexts (including incomplete markers).
Tests/aweXpect.Tests/Collections/ThatEnumerable.All.ComplyWith.ImmutableTests.cs Mirrors new Not matching items: + Collection: contexts for immutable collection scenarios.
Tests/aweXpect.Tests/Collections/ThatEnumerable.All.ComplyWith.EnumerableTests.cs Mirrors new contexts for enumerable scenarios and truncation/incompleteness cases.
Tests/aweXpect.Tests/Collections/ThatAsyncEnumerable.All.Tests.cs Switches some message assertions to prefix matching (AsPrefix()) due to additional appended contexts; adjusts one count-based message.
Tests/aweXpect.Tests/Collections/ThatAsyncEnumerable.All.ComplyWith.Tests.cs Updates async comply-with failure expectations to include Not matching items: and Collection: contexts.
Tests/aweXpect.Internal.Tests/ThatTests/Collections/QuantifiableCollectionItems.AreEquivalentTests.cs Updates internal expectations for comply-with+equivalency to the new context-driven message shape.
Source/aweXpect/That/Collections/ThatEnumerable.Elements.ComplyWith.cs Implements the new contexts (Collection, matching/non-matching) for sync enumerable ComplyWith constraints.
Source/aweXpect/That/Collections/ThatAsyncEnumerable.Elements.ComplyWith.cs Implements the new contexts (Collection, matching/non-matching) for async enumerable ComplyWith constraints.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

Test Results

     23 files   - 27       23 suites   - 27   7m 21s ⏱️ -44s
 19 813 tests  - 22   19 812 ✅  - 22  1 💤 ±0  0 ❌ ±0 
102 380 runs   - 81  102 379 ✅  - 81  1 💤 ±0  0 ❌ ±0 

Results for commit 5d7b86e. ± Comparison against base commit 9e78cad.

This pull request removes 3180 and adds 3158 tests. Note that renamed tests count towards both.
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message3a94e8c0-fc86-4b06-ba15-58856e66c605")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message5a409f39-09d6-4c2e-b776-49b760f84c49")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message86094e5d-ba57-4607-92c7-21598e3ccabf")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message89bdd6e3-8ff8-4a5c-a327-7160cd316149")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messageb56a274a-b929-4bd7-9da7-d99f67965529")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messageb724d639-d5f6-4507-ba14-2f983cbf6652")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message04635c34-392d-478a-ba63-7a11f6161e7c")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message1bb78b57-1c82-4345-a132-708c99cbb215")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message3bfe8eaa-a3a8-4285-835e-ce5509b400ea")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message7bf671a3-e5db-44ec-bc82-c29c81718d21")
…
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message226ff8da-88eb-43ef-8e43-630fd5cd511e")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message276bc33e-32dd-4f5b-9ead-b642043e4efb")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message34ce027f-316b-439b-88c4-abeea7602a52")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "message81dc77f5-b5ba-41c9-8777-d478e95c5b17")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messagea13408f4-6771-4a2d-a7a5-646d1a5440a9")
aweXpect.Core.Tests.Core.Exceptions.FailExceptionTests ‑ Message_ShouldBeSet(message: "messaged0bedcfc-8948-4c37-b7cf-0f35dd5be847")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message2604b2a0-4d38-42ae-b780-23c5a864d0fb")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message68738217-9b66-4f91-9bf2-b4d2a4942d33")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "message70c6d5a3-4158-4d19-a102-5579f3961b9e")
aweXpect.Core.Tests.Core.Exceptions.SkipExceptionTests ‑ Message_ShouldBeSet(message: "messageabf5ff4c-5def-4b4b-966e-d4ddfb34c472")
…

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.4 LTS (Noble Numbat)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.300
[Host] : .NET 8.0.27 (8.0.27, 8.0.2726.22922), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method Mean Error StdDev Gen0 Gen1 Allocated
Bool_aweXpect 304.0 ns 0.73 ns 0.61 ns 0.0277 - 696 B
Bool_FluentAssertions 319.3 ns 2.48 ns 2.20 ns 0.0377 - 952 B
Equivalency_aweXpect 322,284.4 ns 526.01 ns 492.03 ns 13.1836 0.4883 335444 B
Equivalency_FluentAssertions 2,705,195.3 ns 10,578.94 ns 9,377.96 ns 191.4063 31.2500 4804887 B
Int_GreaterThan_aweXpect 324.3 ns 1.56 ns 1.46 ns 0.0343 - 864 B
Int_GreaterThan_FluentAssertions 349.6 ns 2.21 ns 2.07 ns 0.0486 - 1224 B
ItemsCount_AtLeast_aweXpect 576.5 ns 2.23 ns 1.97 ns 0.0534 - 1360 B
ItemsCount_AtLeast_FluentAssertions 614.5 ns 5.04 ns 4.46 ns 0.0792 - 2008 B
String_aweXpect 553.6 ns 2.53 ns 2.36 ns 0.0448 - 1128 B
String_FluentAssertions 1,483.7 ns 8.72 ns 8.15 ns 0.1564 - 3944 B
StringArray_aweXpect 1,914.4 ns 3.94 ns 3.69 ns 0.1030 - 2624 B
StringArray_FluentAssertions 1,600.2 ns 12.79 ns 11.96 ns 0.1640 - 4152 B
StringArrayInAnyOrder_aweXpect 2,339.8 ns 7.75 ns 7.25 ns 0.1106 - 2816 B
StringArrayInAnyOrder_FluentAssertions 72,971.7 ns 284.87 ns 237.88 ns 1.9531 - 57952 B

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

👽 Mutation Results

Mutation testing badge

aweXpect

Details
File Score Killed Survived Timeout No Coverage Ignored Compile Errors Total Detected Total Undetected Total Mutants
That/Collections/ThatAsyncEnumerable.Elements.ComplyWith.cs 76.47% 37 1 2 11 18 10 39 12 79
That/Collections/ThatEnumerable.Elements.ComplyWith.cs 67.63% 117 20 0 36 48 40 117 56 261

The final mutation score is 69.64%

Coverage Thresholds: high:80 low:60 break:0

aweXpect.Core

Details
File Score Killed Survived Timeout No Coverage Ignored Compile Errors Total Detected Total Undetected Total Mutants

The final mutation score is NaN%

Coverage Thresholds: high:80 low:60 break:0

@vbreuss vbreuss force-pushed the topic/improve-collection-element-error-message branch from ca91469 to 5d7b86e Compare May 16, 2026 18:48
@sonarqubecloud
Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
35.7% Duplication on New Code (required ≤ 30%)

See analysis details on SonarQube Cloud

@vbreuss vbreuss merged commit 891b1b7 into main May 16, 2026
12 of 13 checks passed
@vbreuss vbreuss deleted the topic/improve-collection-element-error-message branch May 16, 2026 21:07
github-actions Bot added a commit that referenced this pull request May 16, 2026
…ion element expectations (#953) by Valentin Breuß
github-actions Bot added a commit that referenced this pull request May 16, 2026
…ion element expectations (#953) by Valentin Breuß
@github-actions
Copy link
Copy Markdown
Contributor

This is addressed in release v2.34.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working state: released The issue is released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants