[Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more #13751

momo-jun · 2022-01-14T01:53:12Z

PR#12403: restructure and optimize content for message chunking.
PR#13108: add description for how to disable threshold check for parameter “loadBalancerNamespaceBundleMaxSessions”.
PR#13043: add 4 new parameters for both Broker and Standalone; add a paragraph for configuring ZK batching operations and benchmark results in the ZK admin topic.
Bug fix: remove the duplicate parameter “maxNumPartitionsPerPartitionedTopic” from the Standalone section of the Pulsar Configuration topic.
Original content: https://pulsar.apache.org/docs/en/next/reference-configuration/#standalone
Bug fix: fix the missing parameter table for global ZK (configuration store) and incomplete parameter table for local ZK by referencing the one in ZK section of the Pulsar Configuration topic.
Original content: https://pulsar.apache.org/docs/en/next/administration-zk-bk/#configuration-store

doc

github-actions · 2022-01-14T01:53:38Z

@momo-jun:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

Anonymitaet · 2022-01-14T04:10:16Z

@momo-jun before submitting this PR, did you preview the changes in your local environment? If so, can you add some screenshots of the preview in the PR description? So that reviewers can understand the changes quickly and clearly, thanks

momo-jun · 2022-01-14T07:15:26Z

@momo-jun before submitting this PR, did you preview the changes in your local environment? If so, can you add some screenshots of the preview in the PR description? So that reviewers can understand the changes quickly and clearly, thanks

Thanks Yu. I've attached the screenshots of detailed changes, and added the links of each PR.

Anonymitaet · 2022-01-17T02:19:21Z

@momo-jun

as we discussed just now, for PIP-45: Allow to configure metadata store URL in broker.conf #13077, we need to add explanations for "backend services (Rocksdb and Etcd)"

References: PIP-45: Allow to configure metadata store URL in broker.conf #13077 and search "rocksdb" in the PR page and find related info, e.g. https://github.com/apache/pulsar/pulls?q=rocksdb
can you modify the PR title? It should be the summary of your PR changes, thanks
can you @ related engineers to review this PR? (for example, @RobertIndie could you please review this PR? Thanks)

Anonymitaet · 2022-01-17T02:27:36Z

site2/docs/reference-configuration.md

-|loadBalancerNamespaceMaximumBundles| maximum number of bundles in a namespace  |128|
+|loadBalancerAutoBundleSplitEnabled|  Enable/disable namespace bundle auto split  |false|
+|loadBalancerNamespaceBundleMaxTopics|  Maximum topics in a bundle, otherwise bundle split will be triggered  |1000|
+|loadBalancerNamespaceBundleMaxSessions|  Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br>To disable the threshold check, set the value to -1.  |1000|


Suggested change

|loadBalancerNamespaceBundleMaxSessions| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br>To disable the threshold check, set the value to -1. |1000|

|loadBalancerNamespaceBundleMaxSessions| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br />To disable the threshold check, set the value to -1. |1000|

instructions: https://docs.google.com/document/d/1IV35SI_F8G8cL-Vuzknc6RTGLK9_edRMpZpnrHvAWNs/edit#bookmark=id.wcmnciyvvq6p

Thanks Yu. Will update.

Anonymitaet · 2022-01-17T02:28:44Z

site2/docs/reference-configuration.md

 | loadBalancerAutoUnloadSplitBundlesEnabled | Enable/Disable automatic unloading of split bundles. | true |
 |loadBalancerNamespaceBundleMaxTopics|    |1000|
-|loadBalancerNamespaceBundleMaxSessions|    |1000|
+|loadBalancerNamespaceBundleMaxSessions|  Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br>To disable the threshold check, set the value to -1.  |1000|


Suggested change

|loadBalancerNamespaceBundleMaxSessions| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br>To disable the threshold check, set the value to -1. |1000|

|loadBalancerNamespaceBundleMaxSessions| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br />To disable the threshold check, set the value to -1. |1000|

Thanks Yu. Will update.

momo-jun · 2022-01-18T09:55:40Z

Hi guys @RobertIndie @codelipenghui @rdhabalia @merlimat , can you please review this PR for doc udpates? Thanks.

momo-jun · 2022-01-18T09:57:26Z

@momo-jun

as we discussed just now, for PIP-45: Allow to configure metadata store URL in broker.conf #13077, we need to add explanations for "backend services (Rocksdb and Etcd)"
References: PIP-45: Allow to configure metadata store URL in broker.conf #13077 and search "rocksdb" in the PR page and find related info, e.g. https://github.com/apache/pulsar/pulls?q=rocksdb

can you modify the PR title? It should be the summary of your PR changes, thanks

can you @ related engineers to review this PR? (for example, @RobertIndie could you please review this PR? Thanks)

Thanks Yu. I've modified the title and invited reviewers. For item1, I will open a new PR.

RobertIndie

@momo-jun Thanks for your PR!
It looks good to me for the chunked message part.
But this PR involves too many parts of changes. It would be better to split this big PR into multiple PRs. I suggest that the next time you submit a PR, you could make one PR do only one thing.

momo-jun · 2022-01-25T02:13:45Z

@momo-jun Thanks for your PR! It looks good to me for the chunked message part. But this PR involves too many parts of changes. It would be better to split this big PR into multiple PRs. I suggest that the next time you submit a PR, you could make one PR do only one thing.

Thanks for your suggestion. I will definitely do that next time. :)

1. PR#12403: restructure and optimize content for message chunking. 2. PR#13108: add description for how to disable threshold check for parameter “loadBalancerNamespaceBundleMaxSessions”. 3. PR#13043: add 4 new parameters for both Broker and Standalone; add a paragraph for configuring ZK batching operations and benchmark results in the ZK admin topic. 4. Bug fix: remove the duplicate parameter “maxNumPartitionsPerPartitionedTopic” from the Standalone section of the Pulsar Configuration topic. 5. Bug fix: fix the missing parameter table for global ZK (configuration store) and incomplete parameter table for local ZK by referencing the one in ZK section of the Pulsar Configuration topic.

…o recent changes.

…rameter.

…erations; Improve content for chunking; Bug fixes and more (apache#13751)

github-actions bot assigned momo-jun Jan 14, 2022

github-actions bot added the doc-label-missing label Jan 14, 2022

Anonymitaet added doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. and removed doc-label-missing labels Jan 14, 2022

Anonymitaet added this to the 2.10.0 milestone Jan 14, 2022

Anonymitaet reviewed Jan 17, 2022

View reviewed changes

momo-jun changed the title ~~[Docs] Doc updates for PR#12403, PR#13108 and PR#13043~~ [Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more Jan 18, 2022

RobertIndie approved these changes Jan 21, 2022

View reviewed changes

momo-jun added 2 commits January 28, 2022 10:22

Fix doc bugs about style syntax

28c530c

momo-jun force-pushed the 2022-1-13 branch from ec8de81 to 28c530c Compare January 28, 2022 08:34

momo-jun and others added 3 commits January 29, 2022 17:21

Remove default value for expireTimeOfIncompleteChunkedMessage due t…

23f81a8

…o recent changes.

Update default value of the expireTimeOfIncompleteChunkedMessage pa…

59f61d8

…rameter.

Merge branch 'master' into 2022-1-13

0331aa3

Anonymitaet approved these changes Feb 1, 2022

View reviewed changes

Anonymitaet merged commit a65c887 into apache:master Feb 1, 2022

Nicklee007 pushed a commit to Nicklee007/pulsar that referenced this pull request Apr 20, 2022

[Doc] Add 4 metadatastore parameters and a section for ZK batching op…

9c2a094

…erations; Improve content for chunking; Bug fixes and more (apache#13751)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more #13751

[Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more #13751

Uh oh!

momo-jun commented Jan 14, 2022 •

edited

Loading

Uh oh!

github-actions bot commented Jan 14, 2022

Uh oh!

Anonymitaet commented Jan 14, 2022

Uh oh!

momo-jun commented Jan 14, 2022

Uh oh!

Anonymitaet commented Jan 17, 2022 •

edited

Loading

Uh oh!

Anonymitaet Jan 17, 2022

Uh oh!

momo-jun Jan 18, 2022

Uh oh!

Anonymitaet Jan 17, 2022

Uh oh!

momo-jun Jan 18, 2022

Uh oh!

momo-jun commented Jan 18, 2022 •

edited

Loading

Uh oh!

momo-jun commented Jan 18, 2022

Uh oh!

RobertIndie left a comment

Uh oh!

momo-jun commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	\|loadBalancerNamespaceBundleMaxSessions\| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br>To disable the threshold check, set the value to -1. \|1000\|
	\|loadBalancerNamespaceBundleMaxSessions\| Maximum sessions (producers + consumers) in a bundle, otherwise bundle split will be triggered. <br />To disable the threshold check, set the value to -1. \|1000\|

[Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more #13751

[Doc] Add 4 metadatastore parameters and a section for ZK batching operations; Improve content for chunking; Bug fixes and more #13751

Uh oh!

Conversation

momo-jun commented Jan 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 14, 2022

Uh oh!

Anonymitaet commented Jan 14, 2022

Uh oh!

momo-jun commented Jan 14, 2022

Uh oh!

Anonymitaet commented Jan 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Anonymitaet Jan 17, 2022

Choose a reason for hiding this comment

Uh oh!

momo-jun Jan 18, 2022

Choose a reason for hiding this comment

Uh oh!

Anonymitaet Jan 17, 2022

Choose a reason for hiding this comment

Uh oh!

momo-jun Jan 18, 2022

Choose a reason for hiding this comment

Uh oh!

momo-jun commented Jan 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

momo-jun commented Jan 18, 2022

Uh oh!

RobertIndie left a comment

Choose a reason for hiding this comment

Uh oh!

momo-jun commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

momo-jun commented Jan 14, 2022 •

edited

Loading

Anonymitaet commented Jan 17, 2022 •

edited

Loading

momo-jun commented Jan 18, 2022 •

edited

Loading