Fortio numthreads field is inaccurate

[nareddyt]

In reference to the discussion in #168

The field should probably be set as the number of concurrent streams (governed by --max-active-requests), multiplied by the number specified in concurrency. However, --concurrency can be set to auto in the CLI, making it difficult for the output transform code to read it.

The output transform code shouldn't use the number of CPUs on the machine when concurrency == 'auto' because it may run on a different machine.

As a prerequisite: We need to offer an "effective concurrency" field in the NH output proto. Then the output transform code will not need to infer the concurrency.

[oschaaf]

I had a look at this in #226; we can derive the effective number of workers from the output proto, so that part is solvable. Some thoughts

There are some related things in flight in both Envoy and Nighthawk, and I think we may want to separate connections and stream concurrency in Nighthawk. So what I proposed in #226 is that we truthfully set connections to the amount of connection we use with Nighthawk/h2(c), which is one per worker. Then when we land support for multiple h2 connections per worker, we can update that.

With Fortio, what's not clear from the output is if h2 was involved in a load test, as that will get negotiated when using https and the server supports it. And subsequently what happens with regard to stream concurrency when h2 is involved.

Out of scope here, but later on, I feel ideally NH would someday also have a way to cap as well as show the number of concurrent requests per connection when using h2, (possibly more or less equatable to a max pipeline depth with h1 in a UI).

One other notable thing is that I observed that Fortio may lower the configured concurrency automatically if qps is low relative to the concurrency setting. Not sure yet if Fortio reflects that back in its json output. We can't rely on the socket count in there as that only gets set when the FastClient is involved.