Skip to content

[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad #17487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Technoboy- merged 4 commits into from
Sep 8, 2022
Merged

[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad #17487

Technoboy- merged 4 commits into from
Sep 8, 2022

Conversation

@poorbarcode
Copy link
Contributor

@poorbarcode poorbarcode commented Sep 6, 2022 •
edited
Loading

Fixes #17386

Motivation

In this test, we expect it works like this:

  1. persistent broker data to ZK
  2. receive notice: broker data changes, and refresh bundle data in memory
  3. persistent bundle data which already updated in memory to ZK
  4. verify bundle data persist correctly
  5. delete namespace
  6. verify bundle data deleted

But the flow works like this:

  1. persistent broker data to ZK
  2. (High light) race condition
    2-x. receive notice: broker data changes, and refresh bundle data in memory
    2-x. persistent bundle data to ZK ( If the data in memory is not updated, it will not be written to ZK )
  3. ......

Modifications

  • guarantee step-3 is always executed after step-2
  • In this test, the original codes mock ModularLoadManagerWrapper to make isCentralized always return true. But this mock is completely unnecessary because that's ModularLoadManagerWrapper.isCentralized was exactly always returned true, so remove the mock.

Documentation

  • doc-required

  • doc-not-needed

  • doc

  • doc-complete

@poorbarcode
Copy link
Contributor Author

poorbarcode commented Sep 6, 2022 •
edited
Loading

This PR should merge into these branches (because this test was appended in branch-2.11):

  • branch-2.11
  • master

@codelipenghui codelipenghui added this to the 2.12.0 milestone Sep 6, 2022
@codelipenghui
Copy link
Contributor

codelipenghui commented Sep 6, 2022

@heesung-sn Please help review this PR, thanks.

gaoran10
gaoran10 reviewed Sep 6, 2022
View reviewed changes
pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java Outdated
}

private void waitResourceDataUpdateToZK(
LoadManager loadManager, Predicate<LoadData> utilChecker) throws Exception {
Copy link
Contributor

@gaoran10 gaoran10 Sep 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the param utilChecker is useless.

Copy link
Contributor Author

@poorbarcode poorbarcode Sep 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was left over from the previous solution and I forgot to delete it. Already deleted, Thanks

gaoran10
gaoran10 approved these changes Sep 6, 2022
View reviewed changes
Copy link
Contributor

@gaoran10 gaoran10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! LGTM

Demogorgon314
Demogorgon314 approved these changes Sep 6, 2022
View reviewed changes
Copy link
Member

@Demogorgon314 Demogorgon314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actions
Copy link

github-actions bot commented Sep 6, 2022

@poorbarcode Please provide a correct documentation label for your PR.
Instructions see Pulsar Documentation Label Guide.

@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Sep 7, 2022
@Technoboy- Technoboy- removed the doc-not-needed Your PR changes do not impact docs label Sep 7, 2022
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 7, 2022
Technoboy-
Technoboy- approved these changes Sep 7, 2022
View reviewed changes
Copy link
Contributor

@Technoboy- Technoboy- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add some comments for line 809~814

@poorbarcode
Copy link
Contributor Author

poorbarcode commented Sep 7, 2022

Hi @Technoboy-

Better to add some comments for line 809~814

Already add code-comment, thanks

@poorbarcode
Copy link
Contributor Author

poorbarcode commented Sep 7, 2022

/pulsarbot rerun-failure-checks

2 similar comments
@poorbarcode
Copy link
Contributor Author

poorbarcode commented Sep 7, 2022

/pulsarbot rerun-failure-checks

@poorbarcode
Copy link
Contributor Author

poorbarcode commented Sep 7, 2022

/pulsarbot rerun-failure-checks

@Technoboy- Technoboy- merged commit 5c67ded into apache:master Sep 8, 2022
Technoboy- pushed a commit that referenced this pull request Sep 8, 2022
@poorbarcode @Technoboy-
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
4cef40c 
…oveBundleAndLoad (#17487)
@heesung-sohn
Copy link
Contributor

heesung-sohn commented Sep 8, 2022

LGTM

@poorbarcode poorbarcode deleted the flaky/testModularLoadManagerRemoveBundleAndLoad_ branch September 17, 2022 02:44
@heesung-sohn
Copy link
Contributor

heesung-sohn commented Mar 4, 2023 •
edited
Loading 

  // Wait until "ModularLoadManager" completes processing the ZK notification.
        ModularLoadManagerWrapper modularLoadManagerWrapper = (ModularLoadManagerWrapper) loadManager;
        ModularLoadManagerImpl modularLoadManager = (ModularLoadManagerImpl) modularLoadManagerWrapper.getLoadManager();
        ScheduledExecutorService scheduler = Whitebox.getInternalState(modularLoadManager, "scheduler");

We don't have the scheduler anymore in ModularLoadManagerImpl. I think this test is failing now, caused by #19656

@heesung-sohn heesung-sohn mentioned this pull request Mar 4, 2023
Merged
5 tasks
@AnonHxy AnonHxy mentioned this pull request Mar 4, 2023
Merged
5 tasks
liangyepianzhou pushed a commit that referenced this pull request Jul 6, 2023
@poorbarcode @liangyepianzhou
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
44c033c 
…oveBundleAndLoad (#17487)

(cherry picked from commit 5c67ded)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jul 17, 2023
@poorbarcode @nicoloboschi
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
14b3f0a 
…oveBundleAndLoad (apache#17487)

(cherry picked from commit 5c67ded)
(cherry picked from commit 44c033c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Technoboy- Technoboy- Technoboy- approved these changes

@gaoran10 gaoran10 gaoran10 approved these changes

@Demogorgon314 Demogorgon314 Demogorgon314 approved these changes

Assignees

@poorbarcode poorbarcode

Labels

cherry-picked/branch-2.10 cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs release/2.10.5 release/2.11.1

Projects

None yet

Milestone

3.0.0

Development

Successfully merging this pull request may close these issues.

Flaky-test: NamespaceServiceTest. testModularLoadManagerRemoveBundleAndLoad

7 participants

@poorbarcode @codelipenghui @heesung-sohn @Technoboy- @gaoran10 @Demogorgon314 @liangyepianzhou