Replace CTE approach with CROSS APPLY / CROSS JOIN LATERAL for reused block-body locals

[Copilot]

Drops the CTE-based deduplication strategy in favour of SQL CROSS APPLY (SQL Server) / CROSS JOIN LATERAL (PostgreSQL) inline subqueries for reused local variables in block-bodied [Projectable] methods. Extends support to EF Core 8 and 9, and uses the original C# variable name as the column alias inside the subquery.

Renamed / removed

• Deleted dead CTE artefacts: CteDeduplicatingRewriter, NormalizedSelectExpressionComparer, CteTableExpression, CteAwareQuerySqlGenerator, CteAwareQuerySqlGeneratorFactory
• Replaced with InlineSubqueryExpression + ProjectablesQuerySqlGenerator / ProjectablesQuerySqlGeneratorFactory

SQL generation

• SQL Server: emits CROSS APPLY (SELECT <expr> AS [varName]) AS [v]
• PostgreSQL: emits CROSS JOIN LATERAL (SELECT <expr> AS "varName") AS "v"
• Original C# variable name is preserved as the projected column alias
• Table alias uses v / v0 / v1 … (EF Core constraint on alias format)

PostgreSQL detection

UsesLateralJoin now checks the SqlGenerationHelper assembly name first (explicitly excludes SQL Server) with the double-quote delimiter as a secondary heuristic — more robust than delimiter-only detection.

[Projectable(AllowBlockBody = true)]
public static decimal GetDiscount(this Order o)
{
    var baseDiscount = o.Subtotal * 0.1m;   // reused → CROSS APPLY
    return baseDiscount + baseDiscount;
}
// SELECT [v].[baseDiscount] + [v].[baseDiscount]
// FROM [Orders] AS [o]
// CROSS APPLY (SELECT [o].[Subtotal] * 0.1 AS [baseDiscount]) AS [v]

EF Core 8 / 9 compatibility

ProjectablesParameterBasedSqlProcessorFactory (active on net8.0 and net9.0 only) decorates IRelationalParameterBasedSqlProcessorFactory. Its Optimize override temporarily removes CrossApplyExpression(InlineSubqueryExpression) entries from _tables around the provider's nullability pass (which throws on unknown TableExpressionBase subtypes in EF 8/9), then restores them so the SQL generator can still emit the CROSS APPLY clauses.

⚠️ EF8/9 CROSS APPLY tests are still failing — the Optimize decorator is wired correctly (confirmed in stack traces) but inner.Optimize still encounters InlineSubqueryExpression via a secondary visit path not yet identified.

Tests

• LocalVariableReuseTests — 12 scenarios: single/multi variable reuse, reuse in WHERE, chained projectable calls, conditional expressions (SQL Server)
• LocalVariableReuseLateralTests — mirrors key scenarios for PostgreSQL CROSS JOIN LATERAL
• Snapshot files updated for net8.0, net9.0, net10.0

Original prompt

Add CTE Support for Local Variable Deduplication in EF Core SQL Translation

Background

The project currently rewrites C# expression trees (including local variable declarations in blocks) into EF Core-compatible projections. The BlockStatementConverter in src/EntityFrameworkCore.Projectables.Generator/SyntaxRewriters/BlockStatementConverter.cs handles local variables declared in projectable expressions — but when a local variable is referenced multiple times, its defining sub-expression gets copied/duplicated in the resulting SQL expression tree.

This causes the same sub-query (or complex SQL fragment) to be emitted multiple times in the generated SQL, instead of being factored out into a SQL Common Table Expression (CTE) for reuse.

Goal

Introduce CTE-based deduplication for local variables that are referenced more than once in the translated SQL tree. When a local variable's defining expression would be inlined more than once, it should instead be:

1. Defined once as a WITH cteName AS (...) clause
2. Referenced by name in all subsequent uses

What Needs to Be Done

1. Custom CteTableExpression (new file)

Create a new TableExpressionBase subclass representing a CTE definition. It should:

• Extend TableExpressionBase (from Microsoft.EntityFrameworkCore.Query.SqlExpressions)
• Hold a SelectExpression as the CTE body (Inner)
• Expose the CTE name via Alias / CteName
• Implement all required abstract members: Clone, WithAlias, Quote (can throw NotSupportedException for now), Print, VisitChildren

Suggested location: src/EntityFrameworkCore.Projectables/Query/SqlExpressions/CteTableExpression.cs

2. Custom CteAwareQuerySqlGenerator (new file)

Override QuerySqlGenerator to handle CteTableExpression nodes:

• Override GenerateRootCommand to emit the WITH cteName AS (...) preamble before the main SELECT
• Collect all CteTableExpressions in tree-walk order (depth-first, so nested CTEs appear before their consumers)
• Override VisitExtension to handle CteTableExpression as a FROM clause reference (emitting just the name + alias, not the body again)

Suggested location: src/EntityFrameworkCore.Projectables/Query/CteAwareQuerySqlGenerator.cs

3. Custom CteAwareQuerySqlGeneratorFactory (new file)

Implement IQuerySqlGeneratorFactory to instantiate CteAwareQuerySqlGenerator.

Suggested location: src/EntityFrameworkCore.Projectables/Query/CteAwareQuerySqlGeneratorFactory.cs

4. Registration

Register the custom factory via ReplaceService<IQuerySqlGeneratorFactory, CteAwareQuerySqlGeneratorFactory>() in the existing EF Core service registration extension (wherever ProjectablesExtension or similar options extension hooks into IDbContextOptionsBuilder).

5. BlockStatementConverter — CTE candidate tracking

In src/EntityFrameworkCore.Projectables.Generator/SyntaxRewriters/BlockStatementConverter.cs, update the local variable handling logic so that:

• When a local variable declaration is encountered (e.g., var x = someExpression;), the defining expression is not immediately inlined everywhere.
• Instead, track the variable as a CTE candidate.
• When generating the final translated expression, if the variable is referenced more than once, wrap its defining SelectExpression in a CteTableExpression and replace all reference sites with ColumnExpressions pointing at the CTE.
• If referenced only once, fall back to the existing inline behavior (no CTE overhead).

6. CteDeduplicatingRewriter (optional utility, new file)

Optionally add a reusable ExpressionVisitor subclass that:

• Does a first pass to count occurrences of each SelectExpression subtree by structural equality (using ExpressionEqualityComparer.Instance)
• Does a second pass to replace duplicates with CteTableExpression wrappers + ColumnExpression references

Suggested location: src/EntityFrameworkCore.Projectables/Query/CteDeduplicatingRewriter.cs

Key Technical Notes

• TableExpressionBase is in Microsoft.EntityFrameworkCore.Query.SqlExpressions namespace
• QuerySqlGenerator is in Microsoft.EntityFrameworkCore.Query namespace
• IQuerySqlGeneratorFactory is in Microsoft.EntityFrameworkCore.Query namespace
• ExpressionEqualityComparer.Instance (already in EF Core) handles structural equality of SQL expression trees
• The WITH CTE syntax is supported by SQL Server, PostgreSQL, SQLite (3.8.3+), MySQL 8+, and MariaDB 10.2+
• QuerySqlGenerator.CheckComposableSql already accepts WITH as a valid composable SQL start
• ColumnExpression references into the CTE must use TableAlias == cteTable.Alias (the outer alias, not the inner SelectExpression's alias)
• CTE generation must happen after EF Core's SQL expression tree is finalized (immutable)
• Quote() can throw NotSupportedException initially — precompiled query support can be added later

Reference

The approach was designed in coll...

This pull request was created from Copilot chat.

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.