Skip to content

Conversation

@ethqunzhong
Copy link
Contributor

@ethqunzhong ethqunzhong commented May 31, 2022

Motivation

add new featrue to support tiered storage by tencent cloud cos

Modifications

Tencent Cloud COS is compatible with the S3 API.
official document as below:
China: https://cloud.tencent.com/document/product/436/41284
International: https://intl.cloud.tencent.com/document/product/436/34688

Verifying this change

  • test passed at standalone
  • test passed at test env
  • test passed at product env

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (yes)
  • The admin cli options: (yes)
  • Anything that affects deployment: (no)

Documentation

  • doc-not-needed

i will commit a new PR for adding documents.

@github-actions
Copy link

@ethqunzhong:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@github-actions
Copy link

@ethqunzhong:Thanks for providing doc info!

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label May 31, 2022
@ethqunzhong
Copy link
Contributor Author

@ethqunzhong:Thanks for your contribution. For this PR, do we need to update docs? (The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

i will commit a new PR for adding documents independent.

@ethqunzhong
Copy link
Contributor Author

/pulsarbot run-failure-checks

Copy link
Member

@zymap zymap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this implementation already covered all the S3-compatible storage services?

@ethqunzhong
Copy link
Contributor Author

ethqunzhong commented May 31, 2022

I think this implementation already covered all the S3-compatible storage services?

great! i will test this implementation for tencent cloud cos, because tencent cloud cos also compatible with the S3 API.

@ethqunzhong
Copy link
Contributor Author

I think this implementation already covered all the S3-compatible storage services?

great! i will test this implementation for tencent cloud cos, because tencent cloud cos also compatible with the S3 API.

@zymap hi, I use this implementation to offload data into tencent cloud cos failed because cos is use virtual host-style as default and s3 provider default is path-style.

and i think we cloud add a "s3-compatible-storage" offloader to offload data into all these s3-compatible-storages.

I commit a new PR to solve this problem. PTAL #15893

@zymap
Copy link
Member

zymap commented Jun 6, 2022

You can specify it using the jclouds properties. https://github.com/apache/pulsar/pull/15710/files#diff-5fa8cee1e48758e8c2ace441539e0fd696aef158af2da84341532c3e492efc61R333.

The property is jclouds.s3.virtual-host-buckets, you can change it by the environment variable or the command properties

@github-actions
Copy link

github-actions bot commented Jul 6, 2022

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Jul 6, 2022
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@lhotari lhotari modified the milestones: 4.0.0, 4.1.0 Oct 14, 2024
@coderzc coderzc modified the milestones: 4.1.0, 4.2.0 Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants