System.ArgumentOutOfRangeException: Non-negative number required

[jruckert]

With the latest CI DEV package, we are getting the following when calling the a simple Web API controller.

An unhandled exception was thrown by the application.
System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: count
at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 dstOffset, Int32 count)
at Microsoft.AspNet.Server.Kestrel.Infrastructure.MemoryPoolIterator2.CopyFrom(Byte[] data, Int32 offset, Int32 count)
at Microsoft.AspNet.Server.Kestrel.Infrastructure.MemoryPoolIterator2.CopyFrom(ArraySegment1 buffer) at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.WriteAsync(ArraySegment1 buffer, Boolean immediate, Boolean chunk, Boolean socketShutdownSend, Boolean socketDisconnect)
at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.Microsoft.AspNet.Server.Kestrel.Http.ISocketOutput.Write(ArraySegment1 buffer, Boolean immediate, Boolean chunk) at Microsoft.AspNet.Server.Kestrel.Http.Frame.WriteChunked(ArraySegment1 data)
at Microsoft.AspNet.Server.Kestrel.Http.Frame.Write(ArraySegment1 data) at Microsoft.AspNet.Server.Kestrel.Http.FrameResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.Dispose(Boolean disposing) at System.IO.TextWriter.Dispose() at Microsoft.AspNet.Mvc.Infrastructure.JsonResultExecutor.ExecuteAsync(ActionContext context, JsonResult result) at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultAsync>d__44.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultFilterAsync>d__43.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAllResultFiltersAsync>d__42.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResourceFilterAsync>d__37.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAsync>d__32.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<InvokeActionAsync>d__8.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Builder.RouterMiddleware.<Invoke>d__4.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Cors.Infrastructure.CorsMiddleware.<Invoke>d__7.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.IISPlatformHandler.IISPlatformHandlerMiddleware.<Invoke>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Server.Kestrel.Http.Frame1.d__3.MoveNext()

[cesarblum]

Can you provide a repro?

[dougbu]

@halter73 and I discussed the same issue earlier today. Can be reproduced pretty easily using a load tester against an MVC server that's slowed down. One way to slow the server down is to collect memory allocations in dot Trace.

Problem goes away if you revert the caaf9d4 commit.

[dougbu]

More detailed stack trace with line numbers:

Hosting environment: Production
Now listening on: http://+:5000
Application started. Press Ctrl+C to shut down.
fail: Microsoft.AspNet.Server.Kestrel[13]
      An unhandled exception was thrown by the application.
      System.ArgumentOutOfRangeException: Non-negative number required.
      Parameter name: count
         at System.Buffer.BlockCopy(Array src, Int32 srcOffset, Array dst, Int32 dstOffset, Int32 count)
         at Microsoft.AspNet.Server.Kestrel.Infrastructure.MemoryPoolIterator2.CopyFrom(Byte[] data, Int32 offset, Int32 count) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Infrastructure\MemoryPoolIterator2.cs:line 757
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.WriteAsync(ArraySegment`1 buffer, Boolean immediate, Boolean chunk, Boolean socketShutdownSend, Boolean socketDisconnect) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 99
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.Microsoft.AspNet.Server.Kestrel.Http.ISocketOutput.Write(ArraySegment`1 buffer, Boolean immediate, Boolean chunk) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 377
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.Write(ArraySegment`1 data) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 424
         at Microsoft.AspNet.Server.Kestrel.Http.FrameResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameResponseStream.cs:line 71
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 334
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.Dispose(Boolean disposing) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 294
         at System.IO.TextWriter.Dispose()
         at Microsoft.AspNet.Mvc.ViewFeatures.ViewExecutor.<ExecuteAsync>d__17.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewFeatures\ViewExecutor.cs:line 185
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewResult.cs:line 75
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultAsync>d__45.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 884
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultFilterAsync>d__44.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 853
      --- End of stack trace from previous location where exception was thrown ---
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAllResultFiltersAsync>d__43.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 746
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResourceFilterAsync>d__38.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 466
      --- End of stack trace from previous location where exception was thrown ---
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAsync>d__33.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 180
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<InvokeActionAsync>d__8.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Infrastructure\MvcRouteHandler.cs:line 102
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
         at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.<RequestProcessingAsync>d__3.MoveNext() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameOfT.cs:line 86

[dougbu]

And stack trace for same scenario when running a Debug configuration (dnx --configuration Debug web):

Hosting environment: Production
Now listening on: http://+:5000
Application started. Press Ctrl+C to shut down.
warn: Microsoft.AspNet.Server.Kestrel[0]
      Connection processing ended abnormally
      System.IO.IOException: Error -4074 EFAULT bad address in system call argument ---> Microsoft.AspNet.Server.Kestrel.Networking.UvException: Error -4074 EFAULT bad address in system call argument
         --- End of inner exception stack trace ---
         at Microsoft.AspNet.Server.Kestrel.Http.SocketInput.GetResult() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketInput.cs:line 241
         at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.<RequestProcessingAsync>d__3.MoveNext() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameOfT.cs:line 51
fail: Microsoft.AspNet.Server.Kestrel[13]
      An unhandled exception was thrown by the application.
      System.Diagnostics.Debug+DebugAssertException:

         at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
         at System.Environment.get_StackTrace()
         at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage)
         at System.Diagnostics.Debug.Assert(Boolean condition)
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.ProducingStart() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 192
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.WriteAsync(ArraySegment`1 buffer, Boolean immediate, Boolean chunk, Boolean socketShutdownSend, Boolean socketDisconnect) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 93
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.Microsoft.AspNet.Server.Kestrel.Http.ISocketOutput.Write(ArraySegment`1 buffer, Boolean immediate, Boolean chunk) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 377
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.WriteChunked(ArraySegment`1 data) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 473
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.Write(ArraySegment`1 data) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 424
         at Microsoft.AspNet.Server.Kestrel.Http.FrameResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameResponseStream.cs:line 71
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 334
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.Dispose(Boolean disposing) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 294
         at System.IO.TextWriter.Dispose()
         at Microsoft.AspNet.Mvc.ViewFeatures.ViewExecutor.<ExecuteAsync>d__17.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewFeatures\ViewExecutor.cs:line 185
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.ViewFeatures.ViewExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewDataDictionary viewData, ITempDataDictionary tempData, String contentType, Nullable`1 statusCode)
         at Microsoft.AspNet.Mvc.ViewFeatures.ViewResultExecutor.ExecuteAsync(ActionContext actionContext, IView view, ViewResult viewResult) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewFeatures\ViewResultExecutor.cs:line 165

         at Microsoft.AspNet.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewResult.cs:line 75
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.ViewResult.ExecuteResultAsync(ActionContext context)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultAsync>d__45.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 884
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeResultAsync(IActionResult result)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultFilterAsync>d__44.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 853
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeResultFilterAsync()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultFilterAsync>d__44.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 835
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeResultFilterAsync()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAllResultFiltersAsync>d__43.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 738
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeAllResultFiltersAsync(IActionResult result)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResourceFilterAsync>d__38.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 466
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeResourceFilterAsync()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResourceFilterAsync>d__38.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 399
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeResourceFilterAsync()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeAllResourceFiltersAsync() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 313
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAsync>d__33.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 156
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.InvokeAsync()
         at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<InvokeActionAsync>d__8.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Infrastructure\MvcRouteHandler.cs:line 102
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.InvokeActionAsync(HttpContext httpContext, ActionDescriptor actionDescriptor)
         at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<>c__DisplayClass7_0.<RouteAsync>b__0(HttpContext c) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Infrastructure\MvcRouteHandler.cs:line 71
         at Microsoft.AspNet.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
         at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.Invoke(HttpContext httpContext)
         at Microsoft.AspNet.Hosting.Internal.HostingApplication.ProcessRequestAsync(Context context)
         at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.<RequestProcessingAsync>d__3.MoveNext() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameOfT.cs:line 86
         at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
         at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.RequestProcessingAsync()
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.<>c.<Start>b__109_0(Object o) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 272
         at System.Threading.Tasks.Task`1.InnerInvoke()
         at System.Threading.Tasks.Task.Execute()
         at System.Threading.Tasks.Task.ExecutionContextCallback(Object obj)
         at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
         at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)
         at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
         at System.Threading.ThreadPoolWorkQueue.Dispatch()
         at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
         at System.Diagnostics.Debug.WindowsDebugLogger.ShowAssertDialog(String stackTrace, String message, String detailMessage)
         at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage)
         at System.Diagnostics.Debug.Assert(Boolean condition)
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.ProducingStart() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 192
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.WriteAsync(ArraySegment`1 buffer, Boolean immediate, Boolean chunk, Boolean socketShutdownSend, Boolean socketDisconnect) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 93
         at Microsoft.AspNet.Server.Kestrel.Http.SocketOutput.Microsoft.AspNet.Server.Kestrel.Http.ISocketOutput.Write(ArraySegment`1 buffer, Boolean immediate, Boolean chunk) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\SocketOutput.cs:line 377
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.WriteChunked(ArraySegment`1 data) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 473
         at Microsoft.AspNet.Server.Kestrel.Http.Frame.Write(ArraySegment`1 data) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\Frame.cs:line 424
         at Microsoft.AspNet.Server.Kestrel.Http.FrameResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count) in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameResponseStream.cs:line 71
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.FlushInternal(Boolean flushEncoder) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 334
         at Microsoft.AspNet.Mvc.HttpResponseStreamWriter.Dispose(Boolean disposing) in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\HttpResponseStreamWriter.cs:line 294
         at System.IO.TextWriter.Dispose()
         at Microsoft.AspNet.Mvc.ViewFeatures.ViewExecutor.<ExecuteAsync>d__17.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewFeatures\ViewExecutor.cs:line 185
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
         at Microsoft.AspNet.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.ViewFeatures\ViewResult.cs:line 75
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultAsync>d__45.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 884
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResultFilterAsync>d__44.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 853
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAllResultFiltersAsync>d__43.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 746
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeResourceFilterAsync>d__38.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 466
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAsync>d__33.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Controllers\FilterActionInvoker.cs:line 180
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
         at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<InvokeActionAsync>d__8.MoveNext() in C:\dd\dnx\MVC\src\Microsoft.AspNet.Mvc.Core\Infrastructure\MvcRouteHandler.cs:line 102
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at Microsoft.AspNet.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
         at Microsoft.AspNet.Server.Kestrel.Http.Frame`1.<RequestProcessingAsync>d__3.MoveNext() in C:\dd\dnx\Universe\KestrelHttpServer\src\Microsoft.AspNet.Server.Kestrel\Http\FrameOfT.cs:line 86

[davidfowl]

@halter73 We should revert the commit if it turns out this is actually a regression caused by it (Which it seems to be).

[benaadams]

On "detailed stack trace with numbers" does the Universe build add any code? As
src\Microsoft.AspNet.Server.Kestrel\Infrastructure\MemoryPoolIterator2.cs:line 757 isn't related to a BlockCopy

[muratg]

https://github.com/aspnet/KestrelHttpServer/blob/dev/src/Microsoft.AspNet.Server.Kestrel/Infrastructure/MemoryPoolIterator2.cs#L757

[benaadams]

D'oh - thank you, was looking at the PR

[benaadams]

Writing some extra tests in this area (large multi-block, multi-write, chunked+unchunked)

[muratg]

Awesome, thanks!

[halter73]

Thanks @benaadams. Extra tests around this would be be great. I reverted the change since it was blocking @dougbu.

[halter73]

@benaadams I found the proximate cause of the CopyFrom bug. It was pretty tricky.

Basically we didn't set the BlockEndOffset property when the MemoryPoolBlock2 finalizer returned a copy of itself to the pool:

KestrelHttpServer/src/Microsoft.AspNet.Server.Kestrel/Infrastructure/MemoryPoolBlock2.cs

Line 95 in ce61282

Pool.Return(new MemoryPoolBlock2

We should also look into why blocks aren't properly being returned in the first place.