It seems that the Cosmos Provider won't work with RBAC, because it unconditionally calls CreateContainerStreamAsync(), and management plane operations aren't supported under RBAC in CosmosDB. See https://aka.ms/cosmos-native-rbac.
.
> Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(System.Net.Http.HttpResponseMessage responseMessage, Newtonsoft.Json.JsonSerializerSettings serializerSettings, Microsoft.Azure.Documents.DocumentServiceRequest request) Line 121 C#
Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(Microsoft.Azure.Documents.DocumentServiceRequest request, Microsoft.Azure.Documents.ResourceType resourceType, System.Uri physicalAddress, System.Threading.CancellationToken cancellationToken) Line 48 C#
[Resuming Async Method]
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.GatewayStoreClient.<InvokeAsync>d__5>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<Microsoft.Azure.Documents.DocumentServiceResponse>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.GatewayStoreClient.<InvokeAsync>d__5>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.GatewayStoreClient.<InvokeAsync>d__5>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(System.Func<System.Threading.Tasks.ValueTask<System.Net.Http.HttpRequestMessage>> createRequestMessageAsync, Microsoft.Azure.Documents.ResourceType resourceType, Microsoft.Azure.Cosmos.HttpTimeoutPolicy timeoutPolicy, Microsoft.Azure.Documents.IClientSideRequestStatistics clientSideRequestStatistics, System.Threading.CancellationToken cancellationToken) Line 434 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<SendHttpHelperAsync>d__15>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<SendHttpHelperAsync>d__15>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<SendHttpHelperAsync>d__15>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(System.Net.Http.HttpRequestMessage requestMessage, Microsoft.Azure.Documents.ResourceType resourceType, System.Threading.CancellationToken cancellationToken) Line 493 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<ExecuteHttpHelperAsync>d__17>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<ExecuteHttpHelperAsync>d__17>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<Microsoft.Azure.Cosmos.CosmosHttpClientCore.<ExecuteHttpHelperAsync>d__17>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.HttpClient.SendAsync.__Core|83_0(System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationTokenSource cts, bool disposeCts, System.Threading.CancellationTokenSource pendingRequestsCts, System.Threading.CancellationToken originalCancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.RedirectHandler.SendAsync(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.RedirectHandler.<SendAsync>d__4>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.Metrics.MetricsHandler.SendAsyncWithMetrics(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.Metrics.MetricsHandler.<SendAsyncWithMetrics>d__5>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.Metrics.MetricsHandler.<SendAsyncWithMetrics>d__5>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.Metrics.MetricsHandler.<SendAsyncWithMetrics>d__5>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.DiagnosticsHandler.SendAsyncCore(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(System.Net.Http.HttpRequestMessage request, bool async, bool doRequestAuth, System.Threading.CancellationToken cancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.SetExistingTaskResult(System.Threading.Tasks.Task<System.__Canon> task, System.__Canon result) Line 494 C#
[Completed] System.Net.Http.dll!System.Net.Http.HttpConnection.SendAsync(System.Net.Http.HttpRequestMessage request, bool async, System.Threading.CancellationToken cancellationToken) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<SendAsync>d__57>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Net.Http.HttpResponseMessage>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<SendAsync>d__57>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.__Canon>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<SendAsync>d__57>.MoveNext() Line 346 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Line 273 C#
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Line 743 C#
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Line 3463 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.SetExistingTaskResult(System.Threading.Tasks.Task<System.Threading.Tasks.VoidTaskResult> task, System.Threading.Tasks.VoidTaskResult result) Line 490 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder.SetResult() Line 49 C#
[Completed] System.Net.Http.dll!System.Net.Http.HttpConnection.InitialFillAsync(bool async) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<InitialFillAsync>d__82>.ExecutionContextCallback(object s) Line 289 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<InitialFillAsync>d__82>.MoveNext(System.Threading.Thread threadPoolThread) Line 368 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Threading.Tasks.VoidTaskResult>.AsyncStateMachineBox<System.Net.Http.HttpConnection.<InitialFillAsync>d__82>.MoveNext() Line 346 C#
System.Net.Security.dll!System.Net.Security.SslStream.ReadAsyncInternal<System.Net.Security.AsyncReadWriteAdapter>(System.Memory<byte> buffer, System.Threading.CancellationToken cancellationToken) Unknown
[Resuming Async Method]
System.Private.CoreLib.dll!System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder<int>.StateMachineBox<System.Net.Security.SslStream.<ReadAsyncInternal>d__163<System.Net.Security.AsyncReadWriteAdapter>>.ExecutionContextCallback(object s) Line 393 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder<int>.StateMachineBox<System.Net.Security.SslStream.<ReadAsyncInternal>d__163<System.Net.Security.AsyncReadWriteAdapter>>.MoveNext() Line 415 C#
System.Net.Security.dll!System.Net.Security.SslStream.EnsureFullTlsFrameAsync<System.Net.Security.AsyncReadWriteAdapter>(System.Threading.CancellationToken cancellationToken, int estimatedSize) Unknown
[Resuming Async Method]
System.Private.CoreLib.dll!System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder<int>.StateMachineBox<System.Net.Security.SslStream.<EnsureFullTlsFrameAsync>d__161<System.Net.Security.AsyncReadWriteAdapter>>.ExecutionContextCallback(object s) Line 393 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Line 179 C#
System.Private.CoreLib.dll!System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder<int>.StateMachineBox<System.Net.Security.SslStream.<EnsureFullTlsFrameAsync>d__161<System.Net.Security.AsyncReadWriteAdapter>>.MoveNext() Line 415 C#
System.Net.Sockets.dll!System.Net.Sockets.SocketAsyncEventArgs..cctor.AnonymousMethod__173_0(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped) Unknown
System.Private.CoreLib.dll!System.Threading.ThreadPoolTypedWorkItemQueue<System.Threading.PortableThreadPool.IOCompletionPoller.Event, System.Threading.PortableThreadPool.IOCompletionPoller.Callback>.System.Threading.IThreadPoolWorkItem.Execute() Line 1137 C#
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 918 C#
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Line 102 C#
[Async Call Stack]
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(Microsoft.Azure.Documents.DocumentServiceRequest request, System.Threading.CancellationToken cancellationToken) Line 78 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 122 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 33 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.RouterHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 42 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.RequestHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 59 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.AbstractRetryHandler.ExecuteHttpRequestAsync(System.Func<System.Threading.Tasks.Task<Microsoft.Azure.Cosmos.ResponseMessage>> callbackMethod, System.Func<Microsoft.Azure.Cosmos.ResponseMessage, System.Threading.CancellationToken, System.Threading.Tasks.Task<Microsoft.Azure.Documents.ShouldRetryResult>> callShouldRetry, System.Func<System.Exception, System.Threading.CancellationToken, System.Threading.Tasks.Task<Microsoft.Azure.Documents.ShouldRetryResult>> callShouldRetryException, System.Threading.CancellationToken cancellationToken) Line 75 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.AbstractRetryHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 28 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.RequestHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 59 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.TelemetryHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 28 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.RequestHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 59 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 26 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.RequestHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 59 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Microsoft.Azure.Cosmos.RequestMessage request, System.Threading.CancellationToken cancellationToken) Line 77 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(string resourceUriString, Microsoft.Azure.Documents.ResourceType resourceType, Microsoft.Azure.Documents.OperationType operationType, Microsoft.Azure.Cosmos.RequestOptions requestOptions, Microsoft.Azure.Cosmos.ContainerInternal cosmosContainerCore, Microsoft.Azure.Cosmos.FeedRange feedRange, System.IO.Stream streamPayload, System.Action<Microsoft.Azure.Cosmos.RequestMessage> requestEnricher, Microsoft.Azure.Cosmos.Tracing.ITrace trace, System.Threading.CancellationToken cancellationToken) Line 305 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync<Microsoft.Azure.Cosmos.ResponseMessage>(string containerName, string databaseName, Microsoft.Azure.Documents.OperationType operationType, Microsoft.Azure.Cosmos.Tracing.ITrace trace, System.Func<Microsoft.Azure.Cosmos.Tracing.ITrace, System.Threading.Tasks.Task<Microsoft.Azure.Cosmos.ResponseMessage>> task, System.Func<Microsoft.Azure.Cosmos.ResponseMessage, Microsoft.Azure.Cosmos.Telemetry.OpenTelemetryAttributes> openTelemetry, string operationName, Microsoft.Azure.Cosmos.RequestOptions requestOptions) Line 504 C#
[Async] Microsoft.Azure.Cosmos.Client.dll!Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync<Microsoft.Azure.Cosmos.ResponseMessage>(string operationName, string containerName, string databaseName, Microsoft.Azure.Documents.OperationType operationType, Microsoft.Azure.Cosmos.RequestOptions requestOptions, System.Func<Microsoft.Azure.Cosmos.Tracing.ITrace, System.Threading.Tasks.Task<Microsoft.Azure.Cosmos.ResponseMessage>> task, System.Func<Microsoft.Azure.Cosmos.ResponseMessage, Microsoft.Azure.Cosmos.Telemetry.OpenTelemetryAttributes> openTelemetry, Microsoft.Azure.Cosmos.Tracing.TraceComponent traceComponent, Microsoft.Azure.Cosmos.Tracing.TraceLevel traceLevel) Line 253 C#
[Async] Microsoft.EntityFrameworkCore.Cosmos.dll!Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateContainerIfNotExistsOnceAsync(Microsoft.EntityFrameworkCore.DbContext _, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties Parameters, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper Wrapper) parametersTuple, System.Threading.CancellationToken cancellationToken) Line 189 C#
[Async] Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync.AnonymousMethod__0(Microsoft.EntityFrameworkCore.DbContext context, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper) state, System.Threading.CancellationToken cancellationToken) Line 311 C#
[Async] Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync<(Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), bool>(System.Func<Microsoft.EntityFrameworkCore.DbContext, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), System.Threading.CancellationToken, System.Threading.Tasks.Task<Microsoft.EntityFrameworkCore.Storage.ExecutionResult<bool>>> operation, System.Func<Microsoft.EntityFrameworkCore.DbContext, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), System.Threading.CancellationToken, System.Threading.Tasks.Task<Microsoft.EntityFrameworkCore.Storage.ExecutionResult<bool>>> verifySucceeded, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper) state, System.Threading.CancellationToken cancellationToken) Line 334 C#
[Async] Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteAsync<(Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), bool>((Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper) state, System.Func<Microsoft.EntityFrameworkCore.DbContext, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), System.Threading.CancellationToken, System.Threading.Tasks.Task<bool>> operation, System.Func<Microsoft.EntityFrameworkCore.DbContext, (Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.ContainerProperties, Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper), System.Threading.CancellationToken, System.Threading.Tasks.Task<Microsoft.EntityFrameworkCore.Storage.ExecutionResult<bool>>> verifySucceeded, System.Threading.CancellationToken cancellationToken) Line 310 C#
[Async] Microsoft.EntityFrameworkCore.Cosmos.dll!Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosDatabaseCreator.EnsureCreatedAsync(System.Threading.CancellationToken cancellationToken) Line 77 C#
[Async] CatalogDb.dll!CatalogDb.CatalogDbInitializer.SeedAsync(CatalogDb.CatalogDbContext dbContext, System.Threading.CancellationToken cancellationToken) Line 75 C#
[Async] CatalogDb.dll!CatalogDb.CatalogDbInitializer.ExecuteAsync(System.Threading.CancellationToken cancellationToken) Line 20 C#
[Async] Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(Microsoft.Extensions.Hosting.BackgroundService backgroundService) Line 197 C#
Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: Forbidden (403); Substatus: 5300; ActivityId: 71a867d4-e5ba-4e5c-a059-3cdcbef9b78f; Reason: (Request blocked by Auth <account> : The given request [POST /dbs/catalogdb/colls] cannot be authorized by AAD token in data plane. Learn more: https://aka.ms/cosmos-native-rbac.
ActivityId: 71a867d4-e5ba-4e5c-a059-3cdcbef9b78f, Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, Windows/10.0.22621 cosmos-netstandard-sdk/3.31.5);
at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)
at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
--- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(403, 5300)":1}},"name":"CreateContainerStreamAsync","start datetime":"2023-10-31T20:22:45.186Z","duration in milliseconds":117.7414,"data":{"Client Configuration":{"Client Created Time Utc":"2023-10-31T20:22:35.9044174Z","MachineId":"vmId:56ff0cad-cee7-4263-a50e-c0b11c03e369","VM Region":"eastus","NumberOfClientsCreated":1,"NumberOfActiveClients":1,"ConnectionMode":"Direct","User Agent":"cosmos-netstandard-sdk/3.36.0P|1|X64|Microsoft Windows 10.0.22621|.NET 8.0.0-rtm.23511.16|N| Microsoft.EntityFrameworkCore.Cosmos/8.0.0-rtm.23512.13","ConnectionConfig":{"gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:False, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":16},"DistributedTraceId":"24c1d4e147fa66f78ec82e110106300d"},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":93.8052,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":93.585,"data":{"System Info":{"systemHistory":[{"dateUtc":"2023-10-31T20:22:44.2657449Z","cpu":6.703,"memory":36892060.000,"threadInfo":{"isThreadStarving":"no info","availableThreads":32765,"minThreads":16,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":93.5632,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":93.5128,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":86.6403,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":86.6357,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":83.1871,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2023-10-31T20:22:45.2138560Z","DurationInMs":70.5362,"RequestUri":"[https://<account>-westus.documents.azure.com/dbs/catalogdb/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"71a867d4-e5ba-4e5c-a059-3cdcbef9b78f","StatusCode":"Forbidden","ReasonPhrase":"Forbidden"](https://<account>-westus.documents.azure.com/dbs/catalogdb/colls%22,%22ResourceType%22:%22Collection%22,%22HttpMethod%22:%22POST%22,%22ActivityId%22:%2271a867d4-e5ba-4e5c-a059-3cdcbef9b78f%22,%22StatusCode%22:%22Forbidden%22,%22ReasonPhrase%22:%22Forbidden%22);}]},"AuthProvider LifeSpan InSec":9.6557683,"Point Operation Statistics":{"Id":"PointOperationStatistics","ActivityId":"71a867d4-e5ba-4e5c-a059-3cdcbef9b78f","ResponseTimeUtc":"2023-10-31T20:22:45.2945474Z","StatusCode":403,"SubStatusCode":5300,"RequestCharge":0,"RequestUri":"dbs/catalogdb","ErrorMessage":"Microsoft.Azure.Documents.DocumentClientException: Request blocked by Auth <account> : The given request [POST /dbs/catalogdb/colls] cannot be authorized by AAD token in data plane. Learn more: https://aka.ms/cosmos-native-rbac.\r\nActivityId: 71a867d4-e5ba-4e5c-a059-3cdcbef9b78f, Microsoft.Azure.Documents.Common/2.14.0, Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, Windows/10.0.22621 cosmos-netstandard-sdk/3.31.5\r\n at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)\r\n at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)\r\n at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\r\n at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\r\n at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)\r\n at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)","RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}
It seems that the Cosmos Provider won't work with RBAC, because it unconditionally calls CreateContainerStreamAsync(), and management plane operations aren't supported under RBAC in CosmosDB. See https://aka.ms/cosmos-native-rbac.
The issue is probably at
efcore/src/EFCore.Cosmos/Storage/Internal/CosmosClientWrapper.cs
Line 189 in 0740f1c
Should probably be doing should be using
CreateIfNotExistsor do a Read instead of a Create + Conflict.Include stack traces
Include verbose output
Include provider and version information
EF Core version: 8.0.0-rtm.23512.13
Database provider: Microsoft.EntityFrameworkCore.Cosmos
Target framework: .NET 8.0
Operating system: Windows
IDE: Visual Studio 2022 17.9 Preview 1 (Internal)