Skip to content

[CORE] Refine the test with specified spark version#9274

Merged
zhztheplayer merged 1 commit intoapache:mainfrom
yikf:simple-testWithSpecifiedSparkVersion
Apr 10, 2025
Merged

[CORE] Refine the test with specified spark version#9274
zhztheplayer merged 1 commit intoapache:mainfrom
yikf:simple-testWithSpecifiedSparkVersion

Conversation

@yikf
Copy link
Copy Markdown
Contributor

@yikf yikf commented Apr 9, 2025

What changes were proposed in this pull request?

This PR aims to refine the test with specified spark version.

for example, before, testWithSpecifiedSparkVersion(" XXX ", Some (" 3.3 ")) can let dev think only in 3.3 test, but the reality is above 3.3 is tested.

How was this patch tested?

GA

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2025

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2025

Run Gluten Clickhouse CI on x86

Copy link
Copy Markdown
Contributor

@Yohahaha Yohahaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice refactoring, thank you!

@Yohahaha
Copy link
Copy Markdown
Contributor

Yohahaha commented Apr 9, 2025

may need to verify the total count of passed UT does not decreased.

@yikf
Copy link
Copy Markdown
Contributor Author

yikf commented Apr 9, 2025

@Yohahaha Thanks for your review. I checked the,

  1. The number UTs of the JsonFunctionsValidateSuite at Spark 3.5, for testAtLeastSparkVersion since it is most likely to cause decrease ut.
  2. Reviewed the refactor function and call sides.

It works fine as expected.

@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from acdfab1 to f14772d Compare April 9, 2025 12:58
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 9, 2025

Run Gluten Clickhouse CI on x86

Copy link
Copy Markdown
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % CH CI is failing

Comment thread gluten-hudi/src/test/scala/org/apache/gluten/execution/HudiSuite.scala Outdated
Comment thread gluten-substrait/src/test/scala/org/apache/spark/sql/GlutenQueryTest.scala Outdated
Comment thread gluten-iceberg/src/test/scala/org/apache/gluten/execution/IcebergSuite.scala Outdated
Comment thread gluten-iceberg/src/test/scala/org/apache/gluten/execution/IcebergSuite.scala Outdated
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from f14772d to ceb368a Compare April 10, 2025 02:39
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from ceb368a to 00a9939 Compare April 10, 2025 02:52
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86


testWithSpecifiedSparkVersion(
"iceberg bucketed join partition value not exists partial cluster", Array("3.4", "3.5")) {
"iceberg bucketed join partition value not exists partial cluster", "3.4", "3.5") {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can also be changed to testWithMinSparkVersion

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, is should be all.
image


testWithSpecifiedSparkVersion("iceberg bucketed join partition value not exists",
Array("3.4", "3.5")) {
"3.4", "3.5") {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can also be changed to testWithMinSparkVersion

@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from 00a9939 to f567653 Compare April 10, 2025 06:07
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from f567653 to 4ebf0c4 Compare April 10, 2025 10:57
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@zhztheplayer zhztheplayer merged commit 118b114 into apache:main Apr 10, 2025
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants