.Net: Behavior of collection deletion

[markwallace-microsoft]

DeleteCollectionAsync seems to behave differently across store engines, when a collection doesn’t exist.

In some cases no error (expected behavior), in other instances an exception “the collection doesn’t exist” is thrown.

[roji]

/cc @adamsitnik this is exactly the kind of thing we should be covering in the specification/conformance tests.

[westey-m]

I think we should standardize on not throwing if it already does not exist. After all, the desired state has already been reached.

[adamsitnik]

I've hit that yesterday, when I decided to write a set of conformance tests when providing test coverage for the Pinecone connector.

https://github.com/adamsitnik/semantic-kernel/blob/9911ee546273b64eb722e86e65e29d6383eea2bd/dotnet/src/VectorDataIntegrationTests/VectorDataIntegrationTests/CRUD/CollectionConformanceTests.cs#L14-L22

I've already fixed Redis and included that in #10788. I am going to make sure all the connector behave the same.

@westey-m @roji to be 100% sure:

• deleting non existing collection should not throw and the same goes for records (both DeleteAsync and DeleteBatchAsync)
• CreateCollectionIfNotExistsAsync should not throw when the collection exists, even if there is a TOCTOU issue (we check if it does not exist, if so then we call a method that fails because somebody has created it in a meantime)
• CreateCollectionAsync should throw VectorStoreOperationException when it already exists

[westey-m]

@westey-m @roji to be 100% sure:

@adamsitnik, yes, those behaviors all make sense to me. Thanks for helping to unify these.
We can update the interface remarks for these methods and the specification document as well to indicate what the behavior will/should be.

[dluc]

InMemoryVectorStoreRecordCollection.CreateCollectionAsync does not throw if a collection already exists

semantic-kernel/dotnet/src/Connectors/Connectors.Memory.InMemory/InMemoryVectorStoreRecordCollection.cs

Line 114 in 5e90a43

public Task CreateCollectionAsync(CancellationToken cancellationToken = default)

[dluc]

@westey-m FYI with Qdrant, when a collection doesn't exist, collection deletion fails with an exception:

Qdrant.Client.QdrantException: Collection 'test123' could not be deleted
at Qdrant.Client.QdrantClient.DeleteCollectionAsync(String collectionName, Nullable1 timeout, CancellationToken cancellationToken) at Microsoft.SemanticKernel.Connectors.Qdrant.QdrantVectorStoreRecordCollection1.RunOperationAsync(String operationName, Func`1 operation)