Several allocation reductions in SocketsHttpHandler

[stephentoub]

For a request like this (which I created just by looking at what headers my browser is sending):

var m = new HttpRequestMessage(HttpMethod.Get, uri);
AddHeader(m, "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9");
AddHeader(m, "accept-encoding", "gzip, deflate, br");
AddHeader(m, "accept-language", "en-US,en;q=0.9");
AddHeader(m, "cache-control", "no-cache");
AddHeader(m, "pragma", "no-cache");
AddHeader(m, "sec-fetch-dest", "document");
AddHeader(m, "sec-fetch-mode", "navigate");
AddHeader(m, "sec-fetch-site", "none");
AddHeader(m, "sec-fetch-user", "?1");
AddHeader(m, "upgrade-insecure-requests", "1");
AddHeader(m, "user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4090.0 Safari/537.36 Edg/83.0.467.0");

using (var resp = await c.SendAsync(m, default))
using (var respStream = await resp.Content.ReadAsStreamAsync())
{
    await respStream.CopyToAsync(Stream.Null);
}

to a default web api application, this PR reduces allocation by ~20%. This is a benchmark.net run of doing the above 100K times in parallel per iteration (and with the server on the same machine, so grain of salt on the timings... the most interesting numbers are the GC/allocation-related ones):

Method	Toolchain	Mean	Ratio	Gen 0	Gen 1	Allocated
Request	\master\corerun.exe	1.653 s	1.00	60000.0000	20000.0000	351.92 MB
Request	\pr\corerun.exe	1.631 s	0.99	46000.0000	15000.0000	272.61 MB

Commit 1: Just manually inlines a small async method. This removes its state machine allocation in the common case, but it also really didn't need to be its own method.

Commit 2: Today whenever a header is added to a header collection with TryAddWithoutValidation, it's wrapped in a HeaderStoreItemInfo, even though we could just store the raw string directly. This PR changes it so that we store the raw string, and only upgrade it to being wrapped when necessary, e.g. whenever we actually parse it, if we need to. I'm interested in feedback as to whether folks think it makes the code too complicated / whether it's worthwhile.

Commit 3: Caches the Date and Server header values on a connection. Since Server should generally never change for all requests on a connection, and Date should only change once a second-ish, we can avoid those string allocations in the typical high-throughput case. Again, I'm interested in feedback on whether folks see a better way to do this.

I tried this with the ASP.NET HttpClient benchmark, and it had a small but repeatable positive impact of around 1-2% on RPS.

Depends on #34667
cc: @scalablecory, @davidsh, @dotnet/ncl

[ghost]

Tagging @dotnet/ncl as an area owner

[wfurt]

looks suspicious.

System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http11.GetAsync_EmptyResponseHeader_Success [FAIL]
      System.InvalidOperationException : Collection was modified; enumeration operation may not execute.
      Stack Trace:
        /_/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/Dictionary.cs(1201,0): at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
        /_/src/libraries/System.Net.Http/src/System/Net/Http/Headers/HttpHeaders.cs(315,0): at System.Net.Http.Headers.HttpHeaders.GetEnumeratorCore()+MoveNext()
        /_/src/libraries/System.Linq/src/System/Linq/Count.cs(39,0): at System.Linq.Enumerable.Count[TSource](IEnumerable`1 source)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Headers.cs(157,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Headers.<>c__DisplayClass6_0.

[wfurt]

hmm. I see the linked issue. that make sense.

[stephentoub]

/azp list

[azure-pipelines]

CI/CD Pipelines for this repository:

• coreclr-gc-longrunning
• coreclr-gc-simulator
• runtime-coreclr outerloop
• runtime-coreclr crossgen2
• runtime-libraries outerloop
• runtime-libraries outerloop-windows
• runtime-libraries outerloop-linux
• runtime-libraries outerloop-osx
• runtime
• runtime-libraries enterprise-linux
• runtime-libraries stress-http
• runtime-libraries stress-ssl
• runtime-live-build
• runtime-coreclr crossgen2 outerloop
• coreclr-release-outerloop-nightly
• sync-runtime-to-mono
• dotnet-runtime-perf
• runtime-coreclr crossgen2-composite
• runtime-jit-experimental

[stephentoub]

/azp run runtime-libraries outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[benaadams]

Can always pick up BytesOrdinalEqualsStringAndAscii from aspnetcore's ServerInfrastructure/StringUtilities

https://github.com/dotnet/aspnetcore/blob/c16d3364d4f83307ac3c6f74de15f8ca96b89759/src/Shared/ServerInfrastructure/StringUtilities.cs#L313-L463

[stephentoub]

Presumably @GrabYourPitchforks will have
https://github.com/dotnet/runtime/pull/34724/files/9d35e5ea0dbf79c474e97c9521f4cb4038beee7b#diff-5643273c2df7053c5ef81c7d9ec49e7cR11-R12
done soon and this can/should just use that.