Use unmanaged memory directly in e.g. MemoryPool2 instead of pinned/fixed across the source

[nietras]

It seems to me that the memory pool really should be based on unmanaged memory since it is pinned/fixed anyway when used with libuv or similar. Changing this should not only reduce the cross-cutting duplication of pinned/fixed statements but perhaps also improve perf.

Asking in a different way, why insist on using managed byte arrays when these are used in a fixed manner?

[benaadams]

The memory pool is pinned in the LoH so not in regular generational memory. They have only one pinned statement for the lifetime of the application.

On the other hand using unmanaged memory would require marshalling to and from they byte arrays for regular operations and also not allow the use of Vectorized operations on them, currently.

[benaadams]

As a comparison the RIO C# server shown in the benchmark results (also this PR aspnet/Benchmarks#12) is entirely managed memory.

[nietras]

Yes while I do understand that it is allocated on the LoH and therefore pinning isn't a huge issue. I have seen some Pin() statements in several places and also fixed statements. Couldn't the regular operations on byte arrays be replaced?

There is no need for marshalling if this change (perhaps huge) would be cross-cutting throughout. Libuv expects IntPtr pointers and fixed memory. In addition Vector operations ARE possible right now using https://github.com/DotNetCross/Memory.Unsafe and these should in fact become available in the runtime at some point. I think they said it would have to wait after RC2 but now that this has been moved this could still be put into it. And wouldn't unsafe Vector read/write help with perf too?

[nietras]

As far as I understand RIO has a similar API surface as libuv in that it expects fixed/pinned memory so my assumption would be it would help in this case too. E.g. lowering CPU load. And yes I am thinking of the stuff going on in e.g. MemoryPoolIterator2 and related code.

[aL3891]

If you're interested i've got a managed rio library going at https://github.com/aL3891/RioSharp that does use an unmanaged memory pool. It also has an experimental web server and the beginnings of an asp.net core host, but its o hold right now because of the current lack of VS tooling for the cli. I do plan to pick up the asp.net host once RC2 is out

[benaadams]

Pretty much all of the fixed statements can be removed from operations on the MemoryBlock2s though it requires some minor refactoring of the data type to directly expose the pointer and well as some of the code paths.

They sill exist because at one time the MemoryBlocks could accept non-pooled memory; as you can see by looking at their Pin and Unpin methods

[nietras]

fixed statements can be removed from operations on the MemoryBlock2

Yeah I was wondering about why these where even there, when the pool is pinned. Since these seem to incur a bounds check on the array due to indexing there would be some minor improvements there. The same bounds checks for Vector could be removed as well if using pointers directly.

Is there any existing easy way to run kestrel with profiler to do testing of this on a single PC?

Yes, it is the Pin/Unpin stuff that seemed out of place or inefficient at the least too me. However, since the pool is pinned and memory block is only supposed to use pooled memory this could be removed, right?

[benaadams]

The same bounds checks for Vector could be removed as well if using pointers directly.

Need to wait for https://github.com/dotnet/corefx/issues/5474

However, since the pool is pinned and memory block is only supposed to use pooled memory this could be removed, right?

Yes, can do something like this benaadams@686ad9d

Is there any existing easy way to run kestrel with profiler to do testing of this on a single PC?

Alas; while its easy to get memory profiles, with various tools like loadtest - you need to use wrk on a different machine with a > 1 GBps connection between the two to generate a high enough load.

For my testing I use two Azure G4 machines; one Linux; one Windows and they generate enough load between them (can get > 12GBps with large responses with this set up)

[benaadams]

Gap in the market for a good WIndows http perf tool that is multi-threaded, fast loopback and does pipelining ;-)

[nietras]

Need to wait for dotnet/corefx#5474

You can use Unsafe from the open source package https://www.nuget.org/packages/DotNetCross.Memory.Unsafe.dll/ right now. Any reason why this cannot be used?

Alas; while its easy to get memory profiles, with various tools like loadtest - you need to use wrk

Hmm, can micro-benchmarking the pool be used or are the load patterns very different so these won't necessarily show this?

There doesn't seem to be any perf tests as part of the kestrel source, wouldn't these add some value and allow for easier contribution?

Yeah, out-of-box experience for these things in dotnet are not good ;) In fact out-of-box for the aspnet benchmark repo is pretty bad too, it just doesn't work right now. A .NET Core perf tool would be great.

[benaadams]

The more general tests are https://github.com/aspnet/Performance

For specific micro-benchmarking I recommend https://github.com/PerfDotNet/BenchmarkDotNet

@mattwarren gives some good insight in to recent features http://mattwarren.github.io/2016/02/17/adventures-in-benchmarking-memory-allocations/

[benaadams]

Any reason why this cannot be used?

I'd assume issues around taking a dependency around an unknown, low contributor and not well supported package; which then MS has to take the onus on provide security and support for - how would they provide emergency patching for it in case of a security alert if you were on holiday so they couldn't update the source that the dependency was based on? Would be messy...

There doesn't seem to be any perf tests as part of the kestrel source, wouldn't these add some value and allow for easier contribution? Yeah, out-of-box experience for these things in dotnet are not good

Again I assume this would be an area that would be improved post RTM; as Kestrel performance is good, and there is more need for stabilising features currently.

As an aside, normally when I make a performance improvement PR to make it easier to evaluate, I'll include a gist of the micro-benchmark and the measurements from it, for example #498 (comment) and #634 (comment) (e.g. 2nd one non-included); if they show good results then they can be investigated further.

If its a system wide-change then I'll do a before and after wrk test #494 (comment) and for memory reductions I'd provide rational and before and after analysis #608 (comment)

[nietras]

Thanks for the reply again.

I know and have used BenchmarkDotNet, and it is definitely useful for determining the performance characteristics of different pieces of code in isolation. It is a work in progress, though, I have had issues with this not working with referenced assemblies etc. Also it is very conservative so for a quick feedback loop it is too cumbersome in my view, but once some different approaches need a definite answer then yes it is great. Just hoping it will have a better OOB experience too.

Yes, I have seen your micro-benchmark and read them with great interest to see the tricks used. Which is why I am also following the development of kestrel. Its great to have this out in the open.

I was rather looking for some good and easy out-of-the-box "F5" experience with regard to kestrel perf work, e.g. run dotnet restore, dotnet build, open solution press F5, and then run a simple command available via nuget that has been restored automatically to do some quick load testing. Nothing to fancy or something that gives absolute performance numbers, but something that can be used for profiling and seeing where most work or GC pressure comes from.

OOBE for dotnet repos is terrible as far as I can tell. I have not been able to run build.cmd on many that actually work i.e. have no errors or test failures (this includes the benchmark, performance, kestrel repos ... so I guess it points more and more to a problem on my machine). Anyway, this pretty much discourages contribution. I don't want to spent hours trying to get stuff working or installing a linux load testing tool in a virtual box etc.

Sorry, that was a rant.

issues around taking a dependency around an unknown, low contributor and not well supported package; which then MS has to take the onus on provide security and support for

I understand that very well, but before RTM and before Unsafe is added to the BCL this could be used in the mean time. If there is a performance benefit, this would also put pressure on actually getting Unsafe included in e.g. RC2.

[benaadams]

OOBE for dotnet repos is terrible as far as I can tell.

Its generally good - its just a bad time; the move to the dotnet cli has made things quite unstable - so much so that the release date for RC2 has moved to TBD https://github.com/aspnet/Home/wiki/Roadmap

It should be better again when that settles down.

[aL3891]

Gap in the market for a good WIndows http perf tool that is multi-threaded, fast loopback and does pipelining ;-)

Fyi Riosharp has one, it has the same cli interface as wrk, its still very limited though, it can only do gets and only count the number of requests but handy if you want to generate a lot of load on windows