Improve the accuracy of immediate processing failure counting in scaled-out scenarios by utilizing the ApproximateReceiveCount attribute

[danielmarbach]

Closes #2163

This PR improves the reliability of receive count tracking in the Amazon SQS transport by combining the existing local LRU cache with the service-provided ApproximateReceiveCount.
Rather than replacing the local cache or relying solely on the potentially unreliable service count, we now take the maximum of the two values.

Rationale

• The current implementation tracks receive counts per message using an endpoint-level LRU cache. However, this is:
  • Flaky in scale-out scenarios (multiple endpoints don't share the cache).
  • Unreliable on endpoint restarts (cache is wiped).
• AWS ApproximateReceiveCount is also not always accurate, and can under-report in certain cases. See:
  • https://zaccharles.medium.com/i-actually-thought-you-were-going-to-say-approximatereceivecount-when-i-asked-how-youd-keep-track-282e8f51af17
• By taking Max(LRU Cache, ApproximateReceiveCount):
  • We hedge against both under-reporting by AWS and cache loss during endpoint restarts.
  • We improve reliability without blindly trusting either source.

Edge Case Considered

An unusual edge case was considered:

• A message is received once and tracked locally.
• Then it's processed via another competing consumer.
• The service-provided receive count increases, but the local cache remains lower.
• Using the max value ensures we reflect the higher count and prevent regressions.

Trade-offs

• ✅ Improved accuracy without a major rework.
• ✅ Safe fallback behavior in edge cases.
• ⚠️ Still requires maintaining the LRU cache (storage pressure, eviction policies, GC implications) but those are already existing design restrictions today.

Additional context

Discussed with @mauroservienti and we concluded, adding a test might just lead to another flaky test. An acceptance test would have to start an endpoint, handle the message once, stop the scenario, run another scenario and then try to assert that the number of retries is an exact match. It is challenging to abort an acceptance test scenarios in that exact manner.

I have also looked at a transport test, but unfortunately the transport test infrastructure is so rigid that it doesn't really allow you to properly restart pumps and wipe out the previous state. It also has built in assumptions around extracting the stack frame on the StartPump. StartPump fails if it is not the first await statement in a test, and restarting the pump would require to call StartPump multiple times.

This PR also includes a change to the MockClients that changes the underlying collection types to be concurrency safe because we got flaky tests due to the same list elements being overwritten concurrently

[danielmarbach]

Pulling this in for now because it also will address some of the flakyness on the other PR