Skip to content

Cosmos: Complex properties Query#37577

Merged
roji merged 59 commits intodotnet:mainfrom
JoasE:feature/complex-properties-query
Mar 13, 2026
Merged

Cosmos: Complex properties Query#37577
roji merged 59 commits intodotnet:mainfrom
JoasE:feature/complex-properties-query

Conversation

@JoasE
Copy link
Copy Markdown
Contributor

@JoasE JoasE commented Jan 28, 2026

Adds Cosmos Db query pipeline support for complex properties
Part of: #31253
Replace IEntityType and IProperty with ITypeBase and IPropertyBase in appropriate places. Along with a couple of class and variable renames
Adds ComplexProperty binding for StructuralTypeProjectionExpression
Do not translate distinct in TranslateDistinct with projections of child documents, related: #34067
Translate Nullable<>.HasValue to != null and Nullable<>.Value to direct property access
Move structural equality to partial class
Translate complex equality by direct comparison
Use CollectionResultExpression for complex property collections
Add assosiations, query and adhoc tests
list.Contains(complexType) not supported as per: 36468
TypeIs not supported, as per: 31250

  • I've read the guidelines for contributing and seen the walkthrough
  • I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
  • The code builds and tests pass locally (also verified by our automated build checks)
  • Commit messages follow this format:
        Summary of the changes
        - Detail 1
        - Detail 2

        Fixes #bugnumber
  • Tests for the changes have been added (for bug fixes / features)
  • Code follows the same patterns and style as existing code in this repo

JoasE

This comment was marked as resolved.

@JoasE JoasE force-pushed the feature/complex-properties-query branch from c4dea47 to 3b21620 Compare February 9, 2026 09:47
@JoasE JoasE marked this pull request as ready for review February 9, 2026 10:03
@JoasE JoasE requested a review from a team as a code owner February 9, 2026 10:03
Copilot AI review requested due to automatic review settings February 9, 2026 10:03
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Cosmos DB query pipeline support for complex/structural types (complex properties), expanding translation capabilities and adding Cosmos-specific test coverage to validate expected SQL and failure behaviors.

Changes:

  • Generalizes Cosmos projection/binding from entity-only to structural types via StructuralTypeProjectionExpression, adding complex-property binding support.
  • Introduces structural equality translation for complex types (including parameter extraction support) and adds a Distinct translation guard when client projection is involved.
  • Adds/updates Cosmos functional tests for complex type queries and complex-property association scenarios (projection, equality, set operations, collections, primitive collections), plus a few ad-hoc helpers/fixtures.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
test/EFCore.Specification.Tests/Query/AdHocComplexTypeQueryTestBase.cs Adjusts ad-hoc complex-type test seeding and exposes context for provider-specific derivations.
test/EFCore.Cosmos.FunctionalTests/Query/NorthwindAggregateOperatorsQueryCosmosTest.cs Updates Distinct-related baselines and replaces CI skip attribute.
test/EFCore.Cosmos.FunctionalTests/Query/ComplexTypeQueryCosmosTest.cs New Cosmos-specific complex type query test baselines and translation-failure expectations.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/OwnedNavigations/OwnedNavigationsProjectionCosmosTest.cs Adds a Distinct-over-owned-associate translation-failure test.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesStructuralEqualityCosmosTest.cs New Cosmos tests validating complex/associate structural equality translation behavior.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesSetOperationsCosmosTest.cs New Cosmos tests for set operations over complex-property association scenarios.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesProjectionCosmosTest.cs Enables previously-skipped projection tests and updates baselines accordingly.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesPrimitiveCollectionCosmosTest.cs New Cosmos tests for primitive-collection operations under complex properties.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesMiscellaneousCosmosTest.cs New Cosmos tests for miscellaneous complex-property predicate/value-type translations.
test/EFCore.Cosmos.FunctionalTests/Query/Associations/ComplexProperties/ComplexPropertiesCollectionCosmosTest.cs New Cosmos tests for complex-property collection operators (Count/Where/Distinct/Index/etc.).
test/EFCore.Cosmos.FunctionalTests/Query/AdHocCosmosTestHelpers.cs Adds a helper to configure deterministic int key generation for Cosmos ad-hoc tests.
test/EFCore.Cosmos.FunctionalTests/Query/AdHocComplexTypeQueryCosmosTest.cs New Cosmos ad-hoc complex-type test with Cosmos SQL baselines and Cosmos-specific initialization.
test/EFCore.Cosmos.FunctionalTests/CosmosComplexTypesTrackingTest.cs Changes one tracking test to skip sync rather than silently completing, and runs async path.
src/EFCore.Cosmos/Query/Internal/SqlExpressionVisitor.cs Updates visitor dispatch to the new structural projection expression type.
src/EFCore.Cosmos/Query/Internal/Expressions/StructuralTypeProjectionExpression.cs Renames/generalizes entity projection to structural projection and adds complex-property binding.
src/EFCore.Cosmos/Query/Internal/Expressions/SelectExpression.cs Tracks whether client projection is used (for later translation decisions like Distinct).
src/EFCore.Cosmos/Query/Internal/Expressions/ObjectReferenceExpression.cs Generalizes object references from IEntityType to ITypeBase.
src/EFCore.Cosmos/Query/Internal/Expressions/ObjectArrayAccessExpression.cs Extends array-access expressions to support complex-property collections.
src/EFCore.Cosmos/Query/Internal/Expressions/ObjectAccessExpression.cs Extends object-access expressions to support complex properties and structural types.
src/EFCore.Cosmos/Query/Internal/CosmosSqlTranslatingExpressionVisitor.cs Refactors to partial class and routes equality rewriting to structural-type-aware logic.
src/EFCore.Cosmos/Query/Internal/CosmosSqlTranslatingExpressionVisitor.StructuralEquality.cs New structural equality implementation for entity/complex comparisons and parameter extraction.
src/EFCore.Cosmos/Query/Internal/CosmosShapedQueryCompilingExpressionVisitor.CosmosProjectionBindingRemovingExpressionVisitorBase.cs Updates projection binding removal to work with structural projections and new access expressions.
src/EFCore.Cosmos/Query/Internal/CosmosQueryableMethodTranslatingExpressionVisitor.cs Enables complex-property collection translation paths and prevents Distinct in client-projection scenarios.
src/EFCore.Cosmos/Query/Internal/CosmosQuerySqlGenerator.cs Updates SQL generator to use the new structural projection expression.
src/EFCore.Cosmos/Query/Internal/CosmosProjectionBindingExpressionVisitor.cs Marks selects that fall back to client projection, impacting later translation decisions.
src/EFCore.Cosmos/Query/Internal/CosmosAliasManager.cs Updates alias rewriting for structural object references.
src/EFCore.Cosmos/Extensions/Internal/CosmosShapedQueryExpressionExtensions.cs Updates projection extraction logic for the renamed structural projection expression.
Comments suppressed due to low confidence (3)

src/EFCore.Cosmos/Query/Internal/Expressions/StructuralTypeProjectionExpression.cs:186

  • BindComplexProperty throws UnableToBindMemberToEntityProjection with memberType set to "navigation". This looks like a copy/paste error and will produce a misleading error message for complex properties; use a distinct memberType (e.g. "complexProperty") like the relational structural projection does.
    src/EFCore.Cosmos/Query/Internal/Expressions/StructuralTypeProjectionExpression.cs:287
  • UpdateEntityType should only be callable when StructuralType is an IEntityType. As written, it uses derivedType.GetAllBaseTypes().Contains(StructuralType) which mixes ITypeBase with the entity-type hierarchy and can misbehave (or fail to compile, depending on the GetAllBaseTypes() return type). Align this with the relational implementation by first asserting/casting StructuralType to IEntityType and then performing the base-type check against that entity type.
    src/EFCore.Cosmos/Query/Internal/Expressions/StructuralTypeProjectionExpression.cs:337
  • ToString() still returns the prefix "EntityProjectionExpression" even though the type was renamed to StructuralTypeProjectionExpression, which makes debugging/logging confusing. Update the string to reflect the new type name.

Comment thread test/EFCore.Cosmos.FunctionalTests/Query/AdHocCosmosTestHelpers.cs Outdated
Comment thread test/EFCore.Cosmos.FunctionalTests/Query/AdHocCosmosTestHelpers.cs Outdated
@roji
Copy link
Copy Markdown
Member

roji commented Feb 9, 2026

@JoasE thanks for this. I'll hold off reviewing until you resolve the Copilot comments above, just to save on the effort (we can do this in future PRs as well). Feel free to ping me once everything's ready for a human review.

Copy link
Copy Markdown
Contributor Author

@JoasE JoasE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roji
I made some changes based on your feedback. I think it's good to take a look at the discussion around CollectionResultExpression before doing a complete review

Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/SelectExpression.cs
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/CollectionResultExpression.cs Outdated
@JoasE JoasE requested a review from roji March 11, 2026 13:39
Copy link
Copy Markdown
Member

@roji roji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @JoasE, I gave this a full review and it looks very good. See my comments which are mainly nits and test adjustments - we should be able to merge this soon.

Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/ObjectAccessExpression.cs Outdated
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/ObjectArrayAccessExpression.cs Outdated
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/SelectExpression.cs
Comment thread test/EFCore.Specification.Tests/Query/AdHocComplexTypeQueryTestBase.cs Outdated
Comment thread test/EFCore.Cosmos.FunctionalTests/Query/AdHocCosmosTestHelpers.cs Outdated
Comment thread test/EFCore.Cosmos.FunctionalTests/CosmosComplexTypesTrackingTest.cs Outdated
Copilot AI review requested due to automatic review settings March 12, 2026 08:46
Copy link
Copy Markdown
Contributor Author

@JoasE JoasE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roji There is still a couple of comments open on differences with relational my review from 6th of march. Did you see all of them? Can I resolve those?

Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/ObjectArrayAccessExpression.cs Outdated
Comment thread test/EFCore.Specification.Tests/Query/AdHocComplexTypeQueryTestBase.cs Outdated
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/CollectionResultExpression.cs Outdated
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/ObjectAccessExpression.cs Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.


You can also share your feedback on Copilot code review. Take the survey.

Comment thread test/EFCore.Cosmos.FunctionalTests/Query/AdHocCosmosTestHelpers.cs Outdated
@JoasE JoasE requested a review from roji March 12, 2026 10:31
Copy link
Copy Markdown
Member

@roji roji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answered all comments, I think we're getting really close. When everything is addressed/answered let me know, I'll do a final pass.

Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/SelectExpression.cs
Comment thread src/EFCore.Cosmos/Query/Internal/Expressions/ObjectArrayAccessExpression.cs Outdated
Comment thread test/EFCore.Specification.Tests/Query/AdHocComplexTypeQueryTestBase.cs Outdated
Copilot AI review requested due to automatic review settings March 13, 2026 10:10
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/EFCore.Cosmos/Query/Internal/CosmosShapedQueryCompilingExpressionVisitor.CosmosProjectionBindingRemovingExpressionVisitorBase.cs:1

  • The code unconditionally casts objectAccessExpression.StructuralProperty.DeclaringType to IEntityType. Since ObjectAccessExpression now supports IComplexProperty as well, DeclaringType can be an IComplexType, making this cast invalid (runtime InvalidCastException). Guard this mapping so it only runs for owned navigations (e.g., when StructuralProperty is INavigation), and skip or handle complex properties explicitly.
// Licensed to the .NET Foundation under one or more agreements.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/EFCore.Cosmos/Query/Internal/CosmosSerializationUtilities.cs
Comment thread src/EFCore.Cosmos/Query/Internal/CosmosSerializationUtilities.cs
@JoasE JoasE requested a review from roji March 13, 2026 11:14
Copy link
Copy Markdown
Member

@roji roji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @JoasE, LGTM - approved. Let me know if there's anything else you want to do here, otherwise I'll hit merge.

Thanks again for all the patient work here! Not an easy task in a codebase that's unfortunately a bit messy and with quite a bit of legacy designs.

Comment thread src/EFCore.Cosmos/Query/Internal/CosmosSerializationUtilities.cs
/// any release. You should only use it directly in your code with extreme caution and knowing that
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </summary>
public static object? ConvertToProviderValue(this IProperty property, object? value)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK to me, especially considering it's all internal.

@AndriySvyryd PTAL, we can revert this part if you have objections.

@roji
Copy link
Copy Markdown
Member

roji commented Mar 13, 2026

BTW @JoasE can you drop a line to my email address, listed on my github profile?

@JoasE
Copy link
Copy Markdown
Contributor Author

JoasE commented Mar 13, 2026

@roji All done on my side! so you can go ahead and merge.
I'm happy we were able to navigate this together to bring this to a success, and thanks for your feedback and time aswell, I learned a lot.

@roji roji merged commit 0cdfa6e into dotnet:main Mar 13, 2026
13 of 14 checks passed
@roji roji linked an issue Mar 13, 2026 that may be closed by this pull request
3 tasks
@JoasE JoasE deleted the feature/complex-properties-query branch April 15, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Cosmos support for complex types

4 participants