System.Threading.ThreadPoolWorkQueue.EnsureThreadRequested() hotspot

[Rodrigo-Andrade]

This is more like a loose question, feel free to close it if its too off-topic.

For sometime i always see System.Threading.ThreadPoolWorkQueue.EnsureThreadRequested() as a hotspot when profiling my code (a reverse proxy, we do plan to move to YARP once its more mature).

Its always something like:

I see this on a 8 core VM, both in Windows and Linux. No thread starvation or something of the sort.
Since that method seems to call to native code, i have no clue to why this overhead is so big.

Anything on the matter will as always will be greatly appreciated.

[EgorBo]

Most likely unrelated, but I noticed that EnsureThreadRequested has a few minor codegen issues, here is a simplified repro:

private int field;

bool EnsureThreadRequested(int count)
{
    int prev = Interlocked.CompareExchange(ref field, count + 1, count);
    return prev == count;
}

Current codegen (RyuJIT, Windows x64):

cmp      dword ptr [rcx], ecx  ; <-- redundant nullcheck, see #44087
add      rcx, 8
lea      r8d, [rdx+1]
mov      eax, edx
lock     
cmpxchg  dword ptr [rcx], r8d ; <-- cmpxchg modifes EFLAGS so we can get rid of the following cmp
cmp      eax, edx
sete     al
movzx    rax, al
ret 

Codegen on Mono-LLVM (Linux x64 ABI, AT&T):

push   %rax
mov    %esi,%eax
lea    0x1(%rax),%ecx
lock cmpxchg %ecx,0x10(%rdi)
sete   %al
pop    %rcx
retq

[mangod9]

fyi @kouvel

[kouvel]

In short bursty workloads the thread pool currently releases too many threads too quickly. It's a known issue, fixing it would involve some tradeoffs but probably would be a better tradeoff. I'm hoping to look into this for .NET 6.

[Rodrigo-Andrade]

@kouvel any workarounds for now?

[kouvel]

Are you seeing this in 3.1 or 5? In .NET 5 there has been some improvement to time spent there in async socket operations, though it doesn't fix the root cause. I'm not aware of any other workarounds that would reduce the CPU time there.