crash caused by an exception throw in SNIMarsHandle::Dispose()

[oldzhu]

we saw several crash caused by an exception throw in SNIMarsHandle::Dispose() in .net core 2.2 release as the below:

(lldb) pe 00007F27B92B7920
Exception object: 00007f27b92b7920
Exception type: System.Net.Sockets.SocketException
Message: Broken pipe
InnerException:
StackTrace (generated):
SP IP Function
00007F2786FFC190 00007F27DFF30C7B System.Net.Sockets.dll!System.Net.Sockets.NetworkStream.Write(Byte[], Int32, Int32)+0xcb

StackTraceString:
HResult: 80004005
There are nested exceptions on this thread. Run with -nested for details
(lldb) clrstack
OS Thread Id: 0x1cf3f (18)
Child SP IP Call Site
00007F2786FFAE98 00007f2858aa0b5a [HelperMethodFrame: 00007f2786ffae98]
00007F2786FFAFE0 00007F27E1C5D9A6 System.Data.SqlClient.SNI.SNIMarsHandle.Dispose() [//src/System.Data.SqlClient/src/System/Data/SqlClient/SNI/SNIMarsHandle.cs @ 49]
00007F2786FFC1D0 00007f2857325a57 [HelperMethodFrame: 00007f2786ffc1d0]
00007F2786FFC350 00007F27DE01A9E3 System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken) [/root/coreclr/src/mscorlib/src/System/Threading/Tasks/Task.cs @ 2788]
00007F2786FFC380 00007F27E1C440F2 System.Data.SqlClient.SNI.SslOverTdsStream.Write(Byte[], Int32, Int32)
00007F2786FFC3A0 00007F27DE8F8077 System.Net.Security.SslStreamInternal.WriteSingleChunk[[System.Net.Security.SslStreamInternal+SslWriteSync, System.Net.Security]](SslWriteSync, System.ReadOnlyMemory1<Byte>) [/root/corefx-2599009/src/System.Net.Security/src/System/Net/Security/SslStreamInternal.cs @ 415] 00007F2786FFC490 00007F27DE8F70BE System.Net.Security.SslStreamInternal.WriteAsyncInternal[[System.Net.Security.SslStreamInternal+SslWriteSync, System.Net.Security]](SslWriteSync, System.ReadOnlyMemory1) [/root/corefx-2599009/src/System.Net.Security/src/System/Net/Security/SslStreamInternal.cs @ 343]
00007F2786FFC550 00007F27DE8F5EDA System.Net.Security.SslStreamInternal.Write(Byte[], Int32, Int32) [/root/corefx-2599009/src/System.Net.Security/src/System/Net/Security/SslStreamInternal.cs @ 140]
00007F2786FFC5F0 00007F27DE90028C System.Net.Security.SslStream.Write(Byte[], Int32, Int32) [/root/corefx-2599009/src/System.Net.Security/src/System/Net/Security/SslStream.cs @ 701]
00007F2786FFC620 00007F27E1C3D969 System.Data.SqlClient.SNI.SNITCPHandle.Send(System.Data.SqlClient.SNI.SNIPacket) [//src/System.Data.SqlClient/src/System/Data/SqlClient/SNI/SNITcpHandle.cs @ 428]
00007F2786FFC660 00007F27E1C5BE31 System.Data.SqlClient.SNI.SNIMarsConnection.Send(System.Data.SqlClient.SNI.SNIPacket) [//src/System.Data.SqlClient/src/System/Data/SqlClient/SNI/SNIMarsConnection.cs @ 83]
00007F2786FFC690 00007F27E1C5BC48 System.Data.SqlClient.SNI.SNIMarsHandle.SendControlPacket(System.Data.SqlClient.SNI.SNISMUXFlags) [//src/System.Data.SqlClient/src/System/Data/SqlClient/SNI/SNIMarsHandle.cs @ 88]
00007F2786FFC6E0 00007F27E1C5D978 System.Data.SqlClient.SNI.SNIMarsHandle.Dispose() [//src/System.Data.SqlClient/src/System/Data/SqlClient/SNI/SNIMarsHandle.cs @ 49]
00007F2786FFC700 00007F27E1C5D86D System.Data.SqlClient.SNI.TdsParserStateObjectManaged.Dispose() [//src/System.Data.SqlClient/src/System/Data/SqlClient/TdsParserStateObjectManaged.cs @ 88]
00007F2786FFC720 00007F27E1C5D65E System.Data.SqlClient.TdsParserSessionPool.Dispose() [//src/System.Data.SqlClient/src/System/Data/SqlClient/TdsParserSessionPool.cs @ 85]
00007F2786FFC780 00007F27E1C5C31D System.Data.SqlClient.TdsParser.Disconnect() [//src/System.Data.SqlClient/src/System/Data/SqlClient/TdsParser.cs @ 943]
00007F2786FFC7B0 00007F27E40353B7 System.Data.SqlClient.SqlInternalConnectionTds.Dispose() [//src/System.Data.SqlClient/src/System/Data/SqlClient/SqlInternalConnectionTds.cs @ 599]
00007F2786FFC7D0 00007F27E4035313 System.Data.ProviderBase.DbConnectionPool.DestroyObject(System.Data.ProviderBase.DbConnectionInternal) [//src/System.Data.SqlClient/src/System/Data/ProviderBase/DbConnectionPool.cs @ 912]
00007F2786FFC810 00007F27E40235CF System.Data.ProviderBase.DbConnectionPool.CleanupCallback(System.Object) [/_/src/System.Data.SqlClient/src/System/Data/ProviderBase/DbConnectionPool.cs @ 582]
00007F2786FFC870 00007F27DDC89716 System.Threading.TimerQueueTimer.CallCallback() [/root/coreclr/src/mscorlib/src/System/Threading/Timer.cs @ 610]
00007F2786FFC8B0 00007F27DDC8953E System.Threading.TimerQueueTimer.Fire() [/root/coreclr/src/mscorlib/src/System/Threading/Timer.cs @ 569]
00007F2786FFC900 00007F27DDCABBAB System.Threading.TimerQueue.FireNextTimers() [/root/coreclr/src/mscorlib/src/System/Threading/Timer.cs @ 339]
00007F2786FFCCB0 00007f285733717f [DebuggerU2MCatchHandlerFrame: 00007f2786ffccb0]
(lldb) bt

• thread We need better way of handling inner- and outer- loop tests #13851, stop reason = signal SIGABRT
  • frame #0: 0x00007f2858aa0b5a libpthread.so.0__waitpid + 106 frame dotnet/corefx#1: libcoreclr.so::PROCCreateCrashDumpIfEnabled() at process.cpp:3242
    frame Update build tools reference to 1.0.11. corefx#2: libcoreclr.sosigsegv_handler(code=11, siginfo=0x00007f27b4a3fbf0, context=0x00007f27b4a3fac0) at signal.cpp:432 frame dotnet/corefx#3: 0x00007f2858aa10e0 libpthread.so.0___lldb_unnamed_symbol1$$libpthread.so.0 + 1
    frame Adding more details for how to build and test for the corefx repo corefx#4: 0x00007f2857e88529 libc.so.6abort + 617 frame dotnet/corefx#5: libcoreclr.so::PROCAbort() at process.cpp:3271
    frame Get XML building in VS by switching to standard .resx corefx#6: libcoreclr.soPROCEndProcess(hProcess=<unavailable>, uExitCode=<unavailable>, bTerminateUnconditionally=<unavailable>) at process.cpp:1439 frame dotnet/corefx#7: libcoreclr.soUnwindManagedExceptionPass1(ex=, frameContext=) at exceptionhandling.cpp:4681
    frame Remove internal package source from NuGet.config corefx#8: libcoreclr.soDispatchManagedException(ex=<unavailable>, isHardwareException=<unavailable>) at exceptionhandling.cpp:4752 frame dotnet/corefx#9: libcoreclr.soIL_Rethrow() at jithelpers.cpp:4894
    frame More enumeration perf fixes corefx#10: 0x00007f27e1c5d9a6
    ....

and also saw throw was removed by the following commit change several days ago:

remove throw in exception handling of dispose (#42457)
dotnet/corefx@790785a

Please port the change back to the .NET core 2.2 to fix the crash...

[oldzhu]

Actually the issue exists in all .NET core release (including 3.0/3.1/2.2), it would be better to port the change to all releases...

[cheenamalhotra]

PRs waiting to be merged:

• [release/2.1] Remove throw in exception handling of dispose corefx#42489 [.NET Core 2.1]
• [release/3.0] Remove throw in exception handling of dispose corefx#42486 [.NET Core 3.0]

Since .NET Core 2.2 will be out of support in a month, we're only proposing fix in 3.0 and 2.1 branches.

[johnsimons]

Hi @cheenamalhotra,
I just want to confirm that even though you are not planning to patch NET Core 2.2, we will be able to get this fix if our software is running under .NET Core 2.2?
We are referencing https://www.nuget.org/packages/System.Data.SqlClient/4.7.0, am I right in assuming a new version of this package will come out soon and we can update our reference to this new version and run it under the .NET 2.2 runtime?

[danmoseley]

@johnsimons if it is approved for 2.1 it will automatically flow to 2.2. Same for 3.0 and 3.1.

[johnsimons]

Is there any nuget feed I can get an alpha version of System.Data.SqlClient with this patch, I have tried https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json, but not sure what version to try?

[KennyChen0909]

@danmosemsft @cheenamalhotra Can we have an ETA for this release?

[danmoseley]

@KennyChen0909 which version do you run on?

Generally, it takes 6-8 weeks from the time the change is merged. Because 2.1 is being held to a high bar for changes right now, and 2.2/3.0 will soon be out of support, it may go only into 3.1 right now. That would probably be in 3.1.1. in Jan.

[danmoseley]

@KennyChen0909 which version do you run on?

Generally, it takes 6-8 weeks from the time the change is merged. Because 2.1 is being held to a high bar for changes right now, and 2.2/3.0 will soon be out of support, it may go only into 3.1 right now. That would probably be in 3.1.1. in Jan.

[KennyChen0909]

@danmosemsft I'm asking regarding @johnsimons . Their application is running under .NET Core 2.2. Would like to know the ETA for a patch release. Thanks.

[cheenamalhotra]

PR dotnet/corefx#42489 has been merged and would be part of .NET Core 2.1.16 release due soon, closing this issue on that note.