Skip to content

MINOR: Exclude '**/*Suite.class' from test, unitTest and integrationTest#8381

Merged
ijuma merged 2 commits intoapache:trunkfrom
chia7712:fix_avoid_duplicate_tests
Mar 30, 2020
Merged

MINOR: Exclude '**/*Suite.class' from test, unitTest and integrationTest#8381
ijuma merged 2 commits intoapache:trunkfrom
chia7712:fix_avoid_duplicate_tests

Conversation

@chia7712
Copy link
Copy Markdown
Member

./gradlew unitTest integrationTest \
    --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@" \
    || { echo 'Test steps failed'; exit 1; }

The tasks unitTest and integrationTest used to run tests don't exclude the **/*Suite so the tests included by Suite class are executed two times. For example:

11:42:25 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState STARTED
11:42:26 
11:42:26 org.apache.kafka.streams.integration.GlobalKTableIntegrationTest > shouldKStreamGlobalKTableJoin PASSED
11:42:30 
11:42:30 org.apache.kafka.streams.integration.StoreQuerySuite > org.apache.kafka.streams.integration.QueryableStateIntegrationTest.shouldBeAbleToQueryMapValuesState PASSED
...
11:48:42 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState STARTED
11:48:46 
11:48:46 org.apache.kafka.streams.integration.QueryableStateIntegrationTest > shouldBeAbleToQueryMapValuesState PASSED

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@chia7712
Copy link
Copy Markdown
Member Author

@ijuma FYI

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Mar 28, 2020

Good catch, we should do the same for the test block, right? @guozhangwang @mjsax why do we have these suite classes in the first place?

@chia7712
Copy link
Copy Markdown
Member Author

we should do the same for the test block, right?

It was addressed by #5527 (the rule is added to streams module only since the Suite.class is used by streams module only)

PR
https://github.com/apache/kafka/pull/5527/files#diff-c197962302397baf3a4cc36463dce5eaR1228
current code
https://github.com/apache/kafka/blob/trunk/build.gradle#L1283

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Mar 28, 2020

I think we should do it consistently at the top level

@chia7712
Copy link
Copy Markdown
Member Author

I think we should do it consistently at the top level

done

@chia7712 chia7712 force-pushed the fix_avoid_duplicate_tests branch from 207f791 to 3db50fd Compare March 28, 2020 17:01
@mjsax mjsax added streams tests Test fixes (including flaky tests) labels Mar 28, 2020
@mjsax
Copy link
Copy Markdown
Member

mjsax commented Mar 28, 2020

why do we have these suite classes in the first place?

@vvcephei Suggested to add them and I think they are useful if one works on a specific feature (like suppress(), FK-joins, EOS): you can run all relevant test at once in IntelliJ in a convenient way.

I agree that we should not run the test twice in the build though.

@vvcephei
Copy link
Copy Markdown
Contributor

vvcephei commented Mar 28, 2020

Ah, my mistake with the incorrect scoping. Thanks for the fix.

FWIW, I do find that putting together a Suite is very useful while doing feature development, but having the suites checked in is less useful.

Coming back to features I’ve made suites for, I’ve often had to add or remove tests from the suites to cover the logic I’m currently working on.

Now, I’m wondering if we should just delete the Suite files and not allow them to be checked in, but only use them for local dev.

Copy link
Copy Markdown
Member

@ijuma ijuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ijuma
Copy link
Copy Markdown
Member

ijuma commented Mar 28, 2020

@vvcephei I would agree, but I'll merge this PR as is since it solves the issue and doesn't cause harm even if we remove the Suite classes. Feel free to submit a separate PR to remove the Suite classes if there is consensus on that.

@chia7712
Copy link
Copy Markdown
Member Author

QueryableStateIntegrationTest.concurrentAccesses seems to be a flaky.

@chia7712
Copy link
Copy Markdown
Member Author

retest this please

@ijuma ijuma changed the title Minor: exclude '**/*Suite.class' from unitTest and integrationTest MINOR: Exclude '**/*Suite.class' from test, unitTest and integrationTest Mar 30, 2020
@ijuma ijuma merged commit 923f1ec into apache:trunk Mar 30, 2020
@chia7712 chia7712 deleted the fix_avoid_duplicate_tests branch March 25, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

streams tests Test fixes (including flaky tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants