QUIC: QuicListener not reporting address in use correctly

[JamesNK]

Description

When two listeners are started on the same port, I get a generic error message instead of the address in use.

Note: this is on my local dev machine. I've seen S.N.Q report the right error message, e.g. dotnet/aspnetcore#42967

Reproduction Steps

See the unit test BindAsync_ListenersSharePort_ThrowAddressInUse added in dotnet/aspnetcore#42983

Expected behavior

QuicException has a status of AddressInUse and a friendly error message.

Actual behavior

ListenAsync throws this error: System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

Someone who gets that error message will have no idea what the problem is or how to fix it.

Full logs:

 Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log
   Source: QuicConnectionListenerTests.cs line 149
   Duration: 63 ms

  Message: 
System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

  Stack Trace: 
ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
QuicListener.ctor(QuicListenerOptions options)
QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
QuicConnectionListener.CreateListenerAsync() line 122
QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) line 64
QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) line 63
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
--- End of stack trace from previous location ---

  Standard Output: 
| [0.006s] TestLifetime Information: Starting test BindAsync_AddressInUse_Log at 2022-07-29T04:39:45
| [0.038s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:0.
| [0.062s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:65152.
| [0.072s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests Error: Test threw an exception.
| System.Net.Quic.QuicException: An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE
|    at System.Net.Quic.ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
|    at System.Net.Quic.QuicListener..ctor(QuicListenerOptions options)
|    at System.Net.Quic.QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Internal.QuicConnectionListener.CreateListenerAsync() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\Internal\QuicConnectionListener.cs:line 122
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\QuicTransportFactory.cs:line 64
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicTestHelpers.cs:line 63
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 264
| --- End of stack trace from previous location ---
|    at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
|    at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
| [0.074s] TestLifetime Information: Finished test BindAsync_AddressInUse_Log in 0.0711134s

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When two listeners are started on the same port, I get a generic error message instead of the address in use.

Note: this is on my local dev machine. I've seen S.N.Q report the right error message, e.g. dotnet/aspnetcore#42967

Reproduction Steps

See the unit test BindAsync_ListenersSharePort_ThrowAddressInUse added in dotnet/aspnetcore#42983

Expected behavior

QuicException has a status of AddressInUse and a friendly error message.

Actual behavior

ListenAsync throws this error: System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

Someone who gets that error message will have no idea what the problem is or how to fix it.

Full logs:

 Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log
   Source: QuicConnectionListenerTests.cs line 149
   Duration: 63 ms

  Message: 
System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

  Stack Trace: 
ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
QuicListener.ctor(QuicListenerOptions options)
QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
QuicConnectionListener.CreateListenerAsync() line 122
QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) line 64
QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) line 63
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
--- End of stack trace from previous location ---

  Standard Output: 
| [0.006s] TestLifetime Information: Starting test BindAsync_AddressInUse_Log at 2022-07-29T04:39:45
| [0.038s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:0.
| [0.062s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:65152.
| [0.072s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests Error: Test threw an exception.
| System.Net.Quic.QuicException: An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE
|    at System.Net.Quic.ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
|    at System.Net.Quic.QuicListener..ctor(QuicListenerOptions options)
|    at System.Net.Quic.QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Internal.QuicConnectionListener.CreateListenerAsync() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\Internal\QuicConnectionListener.cs:line 122
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\QuicTransportFactory.cs:line 64
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicTestHelpers.cs:line 63
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 264
| --- End of stack trace from previous location ---
|    at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
|    at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
| [0.074s] TestLifetime Information: Finished test BindAsync_AddressInUse_Log in 0.0711134s

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

Author:	JamesNK
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	-

[rzikm]

The error originates from MsQuic from here:

https://github.com/microsoft/msquic/blob/main/src/core/listener.c#L338

We should probably check with MsQuic team.

[JamesNK]

BindAsync_ListenersSharePort_ThrowAddressInUse failing on build server: https://dev.azure.com/dnceng/public/_build/results?buildId=1910981&view=ms.vss-test-web.build-test-results-tab&runId=49596722&resultId=109366&paneView=debug

[rzikm]

The issue seems to be only in cases where you try to start two QUIC listeners on the same port with intersecting ALPN sets inside the same process. When the desired port is occupied by a different process or by a Socket instance, the call throws with QuicError.AddressInUse as expected.

The problem with two QuicListeners in the same process is that MsQuic allows binding two listeners on the same port as long as they don't have conflicting ALPNs, so returning AddressInUse might be confusing because creating the listener with different ALPN would succeed.

From a brief conversation with MsQuic team, some possible solutions are:

• Map the error to AddressInUse at the System.Net.Quic layer (not recommended by nibanks)
• Surface the error as "Address in use" at the HTTP layer (but other protocols implemented on top of S.N.Q will face the same issue)
• introduce another status code/ QuicError: "ALPN in use" which would make more clear what is going on.

[JamesNK]

Adding QuicError.AlpnInUse to S.N.Q sounds the best. It would be nice if the exception message says which ALPN is in use, but it's not critical.

In Kestrel, we'll probably still surface the error to the user that the port is already in use. With HTTP/3, the ALPN is always h3.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When two listeners are started on the same port, I get a generic error message instead of the address in use.

Note: this is on my local dev machine. I've seen S.N.Q report the right error message, e.g. dotnet/aspnetcore#42967

Reproduction Steps

See the unit test BindAsync_ListenersSharePort_ThrowAddressInUse added in dotnet/aspnetcore#42983

Expected behavior

QuicException has a status of AddressInUse and a friendly error message.

Actual behavior

ListenAsync throws this error: System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

Someone who gets that error message will have no idea what the problem is or how to fix it.

Full logs:

 Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log
   Source: QuicConnectionListenerTests.cs line 149
   Duration: 63 ms

  Message: 
System.Net.Quic.QuicException : An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE

  Stack Trace: 
ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
QuicListener.ctor(QuicListenerOptions options)
QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
QuicConnectionListener.CreateListenerAsync() line 122
QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) line 64
QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) line 63
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
QuicConnectionListenerTests.BindAsync_AddressInUse_Log() line 156
--- End of stack trace from previous location ---

  Standard Output: 
| [0.006s] TestLifetime Information: Starting test BindAsync_AddressInUse_Log at 2022-07-29T04:39:45
| [0.038s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:0.
| [0.062s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: QUIC listener starting with configured endpoint 127.0.0.1:65152.
| [0.072s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests Error: Test threw an exception.
| System.Net.Quic.QuicException: An internal error has occurred. ListenerStart failed: QUIC_STATUS_INVALID_STATE
|    at System.Net.Quic.ThrowHelper.ThrowIfMsQuicError(Int32 status, String message)
|    at System.Net.Quic.QuicListener..ctor(QuicListenerOptions options)
|    at System.Net.Quic.QuicListener.ListenAsync(QuicListenerOptions options, CancellationToken cancellationToken)
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Internal.QuicConnectionListener.CreateListenerAsync() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\Internal\QuicConnectionListener.cs:line 122
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.QuicTransportFactory.BindAsync(EndPoint endpoint, IFeatureCollection features, CancellationToken cancellationToken) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\src\QuicTransportFactory.cs:line 64
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicTestHelpers.CreateConnectionListenerFactory(ILoggerFactory loggerFactory, ISystemClock systemClock, Boolean clientCertificateRequired, Int64 defaultCloseErrorCode, Int32 port) in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicTestHelpers.cs:line 63
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Microsoft.AspNetCore.Server.Kestrel.Transport.Quic.Tests.QuicConnectionListenerTests.BindAsync_AddressInUse_Log() in C:\Development\Source\aspnetcore\src\Servers\Kestrel\Transport.Quic\test\QuicConnectionListenerTests.cs:line 156
|    at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 264
| --- End of stack trace from previous location ---
|    at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
|    at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
| [0.074s] TestLifetime Information: Finished test BindAsync_AddressInUse_Log in 0.0711134s

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

Author:	JamesNK
Assignees:	-
Labels:	untriaged, area-System.Net.Quic
Milestone:	-

[karelz]

Triage: Limited to corner-case scenario - 2 QuicListener using the same port in the same process. Moving to Future.