Skip to content

Fix failed Spark SQL tests due to shuffle enabled #231

@viirya

Description

@viirya

Describe the bug

#210 enables Comet shuffle in Spark SQL tests. However, there are a few tests failed due to that. We need to investigate the cause and fix the tests.

  • DataFrameSuite.scala: SPARK-35886: PromotePrecision should be subexpr replaced
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require StatefulOpClusteredDistribution
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require ClusteredDistribution from children if the query starts from checkpoint in 3.2.x - without initial state
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require ClusteredDistribution from children if the query starts from checkpoint in prior to 3.2
  • FlatMapGroupsWithStateWithInitialStateSuite.scala: flatMapGroupsWithState - initial state - initial state has flatMapGroupsWithState
  • FlatMapGroupsWithStateSuite.scala
  • hive/AggregationQuerySuite.scala: single distinct multiple columns set

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions