[feat][cli] Add command line option for configuring the memory limit

[sunheyi6]

Fixes #15912

Master Issue: #xyz

PIP: #xyz

Motivation

Modifications

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[github-actions]

@shy-share Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

[lhotari]

It's pretty unconvenient to specify megabytes as bytes on the command line. Do we already have a parser that can accept typical units like M = megabyte etc. .

[sunheyi6]

Do you think it should be changed to this ？

@parameter(names = { "-M", "--Memory", },

[tisonkun]

@shy-share IIUC it means you can specify -m 41M means configuring memory as 41MB.

[sunheyi6]

@tisonkun I know what you mean but I don't know how to write this. like this

    protected static final double MB = 1024 * 1024;
    @Parameter(names = { "-m", "--memory", }, description = "Configure the Pulsar client memory limit{MB}")
    Long memory=0*MB;

[tisonkun]

@lhotari @nicoloboschi Does our CLI tools already provide such functionality? I can read broker.conf options like:

# For Amazon S3 ledger offload, Max block size in bytes. (64MB by default, 5MB minimum)
s3ManagedLedgerOffloadMaxBlockSizeInBytes=67108864

# For Amazon S3 ledger offload, Read buffer size in bytes (1MB by default)
s3ManagedLedgerOffloadReadBufferSizeInBytes=1048576

[github-actions]

The pr had no activity for 30 days, mark with Stale label.

[tisonkun]

I notice that we do have some utils to implement this future. See #20646 (review).

Closing as stale...

But anyone can take over this work as it's clear to follow now.