[improve] [broker] [break change] Do not create partitioned DLQ/Retry topic automatically

[poorbarcode]

Discussion: https://lists.apache.org/thread/t756oy8881vqf9fz2zzgk3y04zsyykbh

Motivation

After you set defaultNumPartitions to 16, you will get 16 partitions per topic, and 16*16 DLQ partitions per subscription, you will get a huge number of DLQs if you have more than one subscription under one topic. For example:

• create topic t1 with 16 partitions
• you will get t1-partition-0, t1-partition-1...tp-partition-15.
• after you enable DLQ, you will get the following 16*16 DLQs per subscription
  • t1-partition-0-{subscription}-DLQ-partition-0
  • t1-partition-0-{subscription}-DLQ-partition-1
  • ...
  • t1-partition-15-{subscription}-DLQ-partition-15
  • t1-partition-1-{subscription}-DLQ-partition-0
  • t1-partition-1-{subscription}-DLQ-partition-1
  • ...
  • ...
  • ...
  • t1-partition-15-{subscription}-partition-15

Issue both partitioned DLQ and non-partitioned DLQ were created automatically.

• Background
  • Broker settings: {allowAutoTopicCreation=true, defaultNumPartitions=1, allowAutoTopicCreationType=partitioned}
  • client version < 3.1
• Flows of creating DLQ.
  • Trigger the DLQ creation.
  • Get partitioned metadata of DLQ: partitions: 1.
    • Broker created the partitioned metadata at this step.
  • Before 3.1, the client mistakenly assumed {tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ-partition-0 is {tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ due to the bug that was fixed by [fix] [broker] Fix wrong logic of method TopicName.getPartition(int index) #19841.
  • Start a dead-letter producer under {tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ
    • Triggered a non-partitioned topic creation.

Eventually, you will get both partitioned and non-partitioned DLQ.

> bin/pulsar-admin topics list-partitioned-topics {tenant}/{namespace}
persistent://{tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ

> bin/pulsar-admin topics list {tenant}/{namespace}                  
persistent://{tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ

Modifications

• Do not create partitioned DLQs/Retry topics automatically.
• Users can also create partitioned DLQ manually if they need it.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: x

[mattisonchao]

cc @shibd, I remember we discussed this place.

[poorbarcode]

/pulsarbot rerun-failure-checks

dismiss the approve, need more discuss

[dao-jun]

I sent a comment to the mail list, PTAL https://lists.apache.org/thread/x5x5kyxwypothwsmrb7x3fflhklrdzb9