Add schema, offload and compaction commands of namespaces for pulsasrctl #64

zymap · 2019-09-17T09:42:12Z

Command List

set-schema-validation-enforced
get-schema-validation-enforced
set-schema-autoupdate-strategy
get-schema-autoupdate-strategy
set-offload-threshold
get-offload-threshold
set-offload-deletion-lag
get-offload-deletion-lag
clear-offload-deletion-lag
set-compaction-threshold
get-compaction-threshold

set-schema-validation-enforced

USED FOR:
    This command is used for setting the schema whether open schema validation enforced.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Enable schema validation enforced
    pulsarctl namespace set-schema-validation-enforced <namespace-name>

    #Disable schema validation enforced
    pulsarctl namespace set-schema-validation-enforced --disable <namespace-name>

OUTPUT:
    #normal output
    Enable/Disable schema validation enforced

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces set-schema-validation-enforced [flags]

get-schema-validation-enforced

USED FOR:
    This command is used for getting the schema validation enforced.

REQUIRED PERMISSION:
    This command requires super-user and tenant admin permissions.

EXAMPLES:
    #Get schema validation status
    pulsarctl namespace get-schema-validation-enforced <namespace-name>

OUTPUT:
    #normal output
    Schema validation enforced is enabled/disabled

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces get-schema-validation-enforced [flags]

set-schema-autoupdate-strategy

USED FOR:
    This command is used for setting the schema auto-update strategy for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Set the schema auto-update strategy <strategy>
    pulsarctl namespace set-schema-autoupdate-strategy --compatibility <strategy> <namespace-name>

OUTPUT:
    #normal output
    Set the schema auto-update strategy <strategy> for a namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces set-schema-autoupdate-strategy [flags]

Schema Auto Update Strategy flags:
  -c, --compatibility string   Compatibility level required for new schemas created via a Producer. Possible values (AutoUpdateDisabled, Backward, Forward, Full, AlwaysCompatible, BackwardTransitive, ForwardTransitive, FullTransitive)

get-schema-autoupdate-strategy

USED FOR:
    This command is used for getting the schema auto-update strategy for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Get the schema auto-update strategy for a namespace <namespace-name>
    pulsarctl namespace get-schema-autoupdate-strategy <namespace-name>

OUTPUT:
    #normal output
    The schema auto-update strategy is <strategy> for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces get-schema-autoupdate-strategy [flags]

set-offload-threshold

USED FOR:
    This command is used for setting offload threshold for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Set offload threshold <size> for a namespace <namespace-name>
    pulsarctl namespace set-offload-threshold --size <size> <namespace-name>

OUTPUT:
    #normal output
    Set the offload threshold <threshold> for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces set-offload-threshold [flags]

Offload Threshold flags:
      --size string   Maximum number of bytes stored in the pulsar cluster for a topic before data will  start being automatically offloaded to longterm  storage (e.g. 10m, 16g, 3t, 100)
                      Negative values disable automatic offload.
                      0 triggers offloading as soon as possible. (default "-1")

get-offload-threshold

USED FOR:
    This command is used for getting offload threshold for a namespace.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Get offload threshold for a namespace <namespace-name>
    pulsarctl namespace get-offload-threshold <namespace-name>

OUTPUT:
    #normal output
    The offload threshold is <size> byte(s) for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces get-offload-threshold [flags]

set-offload-deletion-lag

USED FOR:
    This command is used for setting offload deletion for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Set offload deletion <duration> for a namespace <namespace-name>
    pulsarctl namespace set-offload-deletion-lag --lag <duration> <namespace-name>

OUTPUT:
    #normal output
    Set offload deletion lag <duration> for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces set-offload-deletion-lag [flags]

get-offload-deletion-lag

USED FOR:
    This command is used for getting offload deletion for a namespace.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Get offload deletion for a namespace <namespace-name>
    pulsarctl namespace get-offload-deletion-lag <namespace-name>

OUTPUT:
    #normal output
    The offload deletion lag is <n> minute(s) for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces get-offload-deletion-lag [flags]

clear-offload-deletion-lag

USED FOR:
    This command is used for clearing offload deletion for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Clear offload deletion for a namespace <namespace-name>
    pulsarctl namespace clear-offload-deletion-lag <namespace-name>

OUTPUT:
    #normal output
    Clear offload deletion lag for the namespace <namespace-name> successfully

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces clear-offload-deletion-lag [flags]

set-compaction-threshold

USED FOR:
    This command is used for setting compaction threshold for a namespace.

REQUIRED PERMISSION:
    This command requires super-user permissions and broker has write policies permission.

EXAMPLES:
    #Set compaction threshold <size> for a namespace <namespace-name>
    pulsarctl namespace set-compaction-threshold --size <size> <namespace-name>

OUTPUT:
    #normal output
    Set the compaction threshold <threshold> for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces set-compaction-threshold [flags]

Compaction Threshold flags:
      --size string   Maximum number of bytes in a topic backlog before compaction is triggered (e.g. 10M, 16G, 3T). 0 disable automatic compaction (default "0")

get-compaction-threshold

USED FOR:
    This command is used for getting compaction threshold for a namespace.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Get compaction threshold for a namespace <namespace-name>
    pulsarctl namespace get-compaction-threshold <namespace-name>

OUTPUT:
    #normal output
    The compaction threshold is <size> byte(s) for the namespace <namespace-name>

    #the namespace name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified namespace name does not exist
    [✖]  code: 404 reason: Namespace does not exist

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl namespaces get-compaction-threshold [flags]

pkg/ctl/namespace/compaction_threshold_get.go

sijie · 2019-09-17T12:13:39Z

pkg/ctl/namespace/compaction_threshold_set.go

+func SetCompactionThresholdCmd(vc *cmdutils.VerbCmd) {
+	var desc LongDescription
+	desc.CommandUsedFor = "This command is used for setting compaction threshold for a namespace."
+	desc.CommandPermission = "This command requires super-user permissions and broker has write policies permission."


why super-user? and what does "broker has write policies permission" mean?

protected void internalSetCompactionThreshold(long newThreshold) { validateSuperUserAccess(); validatePoliciesReadOnlyAccess();

broker has write policies permission

It will check the broker is allowed to read-write operations in the global zookeeper.

I don't think it requires super-user access. so please document this in the spreadsheet and we need to improve it.

broker has write policies permission

Okay we need to document this. Because not everyone will know what does that mean. So pleasee improve the message here.

pkg/ctl/namespace/compaction_threshold_set.go

pkg/ctl/namespace/schema_autoupdate_strategy_set.go

pkg/ctl/namespace/schema_validation_enforce_set.go

pkg/pulsar/namespace.go

wolfstudy · 2019-09-17T13:18:53Z

@sijie Please merge #66 first

wolfstudy · 2019-09-19T02:10:41Z

@zymap Please rebase master code

zymap · 2019-09-19T02:16:10Z

@wolfstudy ok

sijie · 2019-10-03T04:37:12Z

please rebase to master and fix CI

zymap · 2019-10-11T05:44:25Z

@sijie PTAL. Thanks

sijie · 2019-10-12T08:43:45Z

Please rebase to latest master

…ctl (streamnative/pulsarctl#64) Command List set-schema-validation-enforced get-schema-validation-enforced set-schema-autoupdate-strategy get-schema-autoupdate-strategy set-offload-threshold get-offload-threshold set-offload-deletion-lag get-offload-deletion-lag clear-offload-deletion-lag set-compaction-threshold get-compaction-threshold

zymap added 3 commits September 17, 2019 17:20

Add some namespace commands

ffd5092

Fix shorthand of the flag

154507b

Fix test error

3034bfc

zymap requested review from sijie and wolfstudy September 17, 2019 11:06

zymap self-assigned this Sep 17, 2019

zymap added the area/pulsar/namespaces label Sep 17, 2019

Remove unused code

63dca32

sijie requested changes Sep 17, 2019

View reviewed changes

wolfstudy changed the title ~~Add some namespace commands~~ Add schema, offload and compaction commands of namespaces for pulsasrctl Sep 17, 2019

wolfstudy changed the title ~~Add schema, offload and compaction commands of namespaces for pulsasrctl~~ [WIP]Add schema, offload and compaction commands of namespaces for pulsasrctl Sep 17, 2019

sijie mentioned this pull request Sep 17, 2019

pulsarctl MVP #2

Closed

29 tasks

Merge branch 'master' into namespace-operation

3ce55c0

zymap changed the title ~~[WIP]Add schema, offload and compaction commands of namespaces for pulsasrctl~~ Add schema, offload and compaction commands of namespaces for pulsasrctl Sep 19, 2019

zymap added 4 commits October 8, 2019 16:37

Merge branch 'master' into namespace-operation

f7a8c8b

Address comments

521cc51

Rename file

d80ed09

Rename

f9e98ba

sijie approved these changes Oct 12, 2019

View reviewed changes

sijie added this to the 0.0.1 milestone Oct 12, 2019

zymap added 2 commits October 12, 2019 17:16

Merge branch 'master' into namespace-operation

0a00c94

Fix build errors

42c77cd

zymap merged commit b072674 into master Oct 12, 2019

zymap deleted the namespace-operation branch October 12, 2019 11:47

Add schema, offload and compaction commands of namespaces for pulsasrctl #64

Add schema, offload and compaction commands of namespaces for pulsasrctl #64

Uh oh!

Conversation

zymap commented Sep 17, 2019

Command List

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sijie Sep 17, 2019

Choose a reason for hiding this comment

Uh oh!

zymap Sep 17, 2019

Choose a reason for hiding this comment

Uh oh!

sijie Sep 17, 2019

Choose a reason for hiding this comment

Uh oh!

zymap Sep 17, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wolfstudy commented Sep 17, 2019

Uh oh!

wolfstudy commented Sep 19, 2019

Uh oh!

zymap commented Sep 19, 2019

Uh oh!

sijie commented Oct 3, 2019

Uh oh!

zymap commented Oct 11, 2019

Uh oh!

sijie commented Oct 12, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants