Skip to content

Conversation

@zymap
Copy link
Member

@zymap zymap commented Sep 24, 2019

Motivation

  • Add command offload
  • Add command offload-status

offload

USED FOR:
    This command is used for triggering offloading the data from a topic to long-term storage (e.g. Amazon S3)

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Trigger offloading the data from a topic <topic-name> to a long-term storage and keep the configured amount of data in BookKeeper only (e.g. 10M, 5G, default is byte)
    pulsarctl topic offload <topic-name> <threshold>

OUTPUT:
    #normal output
    Offload trigger for <topic-name> for messages before <message-id>

    #noting to offload
    Nothing to offload

    #the topic name is not specified or the offload threshold is not specified
    [✖]  only two argument is allowed to be used as names

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload [flags]

offload-status

USED FOR:
    This command is used for checking the status of data offloading from a persistent topic to long-term storage.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Check the status of data offloading from a topic <persistent-topic-name> to long-term storage
    pulsarctl topic offload-status <persistent-topic-name>

    #Wait for offloading to complete
    pulsarctl topic offload-status --wait <persistent-topic-name>

OUTPUT:
    #normal output
    Offloading topic <topic-name> data is done successfully

    #Offloading topic is not running
    Offloading topic <topic-name> data is not running

    #Offloading topic is running
    Offloading topic <topic-name> data is running

    #Offloading topic with error
    Offloading topic <topic-name> data is done with error <error-msg>

    #the topic name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload-status [flags]

OffloadStatus flags:
  -w, --wait   Wait for offloading to complete

*Motivation*

- Add command `offload`
- Add command `offload-status`
@zymap zymap requested a review from sijie September 24, 2019 09:52
@zymap zymap self-assigned this Sep 24, 2019
@zymap
Copy link
Member Author

zymap commented Oct 9, 2019

@sijie PTAL. Thanks.

@sijie sijie added this to the 0.0.1 milestone Oct 10, 2019
@sijie sijie merged commit 165ccc4 into master Oct 10, 2019
@sijie sijie deleted the topic_offload branch October 10, 2019 10:50
@sijie sijie mentioned this pull request Oct 11, 2019
29 tasks
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this pull request Aug 15, 2023
…ctl#74)

*Motivation*

- Add command `offload`
- Add command `offload-status`

*offload*
```
USED FOR:
    This command is used for triggering offloading the data from a topic to long-term storage (e.g. Amazon S3)

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Trigger offloading the data from a topic <topic-name> to a long-term storage and keep the configured amount of data in BookKeeper only (e.g. 10M, 5G, default is byte)
    pulsarctl topic offload <topic-name> <threshold>

OUTPUT:
    #normal output
    Offload trigger for <topic-name> for messages before <message-id>

    #noting to offload
    Nothing to offload

    #the topic name is not specified or the offload threshold is not specified
    [✖]  only two argument is allowed to be used as names

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload [flags]
```

*offload-status*
```
USED FOR:
    This command is used for checking the status of data offloading from a persistent topic to long-term storage.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Check the status of data offloading from a topic <persistent-topic-name> to long-term storage
    pulsarctl topic offload-status <persistent-topic-name>

    #Wait for offloading to complete
    pulsarctl topic offload-status --wait <persistent-topic-name>

OUTPUT:
    #normal output
    Offloading topic <topic-name> data is done successfully

    #Offloading topic is not running
    Offloading topic <topic-name> data is not running

    #Offloading topic is running
    Offloading topic <topic-name> data is running

    #Offloading topic with error
    Offloading topic <topic-name> data is done with error <error-msg>

    #the topic name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload-status [flags]

OffloadStatus flags:
  -w, --wait   Wait for offloading to complete

```
tisonkun pushed a commit to apache/pulsar-client-go that referenced this pull request Aug 16, 2023
…ctl#74)

*Motivation*

- Add command `offload`
- Add command `offload-status`

*offload*
```
USED FOR:
    This command is used for triggering offloading the data from a topic to long-term storage (e.g. Amazon S3)

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Trigger offloading the data from a topic <topic-name> to a long-term storage and keep the configured amount of data in BookKeeper only (e.g. 10M, 5G, default is byte)
    pulsarctl topic offload <topic-name> <threshold>

OUTPUT:
    #normal output
    Offload trigger for <topic-name> for messages before <message-id>

    #noting to offload
    Nothing to offload

    #the topic name is not specified or the offload threshold is not specified
    [✖]  only two argument is allowed to be used as names

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload [flags]
```

*offload-status*
```
USED FOR:
    This command is used for checking the status of data offloading from a persistent topic to long-term storage.

REQUIRED PERMISSION:
    This command requires tenant admin permissions.

EXAMPLES:
    #Check the status of data offloading from a topic <persistent-topic-name> to long-term storage
    pulsarctl topic offload-status <persistent-topic-name>

    #Wait for offloading to complete
    pulsarctl topic offload-status --wait <persistent-topic-name>

OUTPUT:
    #normal output
    Offloading topic <topic-name> data is done successfully

    #Offloading topic is not running
    Offloading topic <topic-name> data is not running

    #Offloading topic is running
    Offloading topic <topic-name> data is running

    #Offloading topic with error
    Offloading topic <topic-name> data is done with error <error-msg>

    #the topic name is not specified
    [✖]  only one argument is allowed to be used as a name

    #the specified topic does not found
    [✖]  code: 404 reason: Topic not found

    #the topic name is not in the format of <tenant>/<namespace>/<topic> or <topic>
    [✖]  Invalid short topic name '<topic-name>', it should be in the format of <tenant>/<namespace>/<topic> or <topic>

    #the topic name is not in the format of <domain>://<tenant>/<namespace>/<topic>
    [✖]  Invalid complete topic name '<topic-name>', it should be in the format of <domain>://<tenant>/<namespace>/<topic>

    #the topic name is not in the format of <tenant>/<namespace>/<topic>
    [✖]  Invalid topic name '<topic-name>', it should be in the format of<tenant>/<namespace>/<topic>

    #the namespace name is not in the format of <tenant>/<namespace>
    [✖]  The complete name of namespace is invalid. complete name : <namespace-complete-name>

    #the tenant name and(or) namespace name is empty
    [✖]  Invalid tenant or namespace. [<tenant>/<namespace>]

    #the tenant name contains unsupported special chars. the alphanumeric (a-zA-Z0-9) and the special chars (-=:.%)  is allowed
    [✖]  Tenant name include unsupported special chars. tenant : [<namespace>]

    #the namespace name contains unsupported special chars. the  alphanumeric (a-zA-Z0-9) and the special chars (-=:.%) is allowed
    [✖]  Namespace name include unsupported special chars. namespace : [<namespace>]

Usage: pulsarctl topics offload-status [flags]

OffloadStatus flags:
  -w, --wait   Wait for offloading to complete

```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants