[WIP]Record unused schema ledgers to ZK for partitioned topics #8258
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
When creating schema for a partitioned topic, e.g.
my-topic, only one z-node/schemas/public/default/my-topicwould be created to record the schema's metadata. However, multiple ledgers would be created because each single partition would try to create the schema: first append a ledger, then write the same z-node.#2959 and #6683 solved the race condition of accessing the same z-node by calling
putSchema()again. However, the unused ledgers of other partitions were created and were not recorded in the z-node. So, there's no way to locate these ledgers from ZK, which means there's no way to delete a topic's all associated schema ledgers.Modifications
/schemas/public/default/my-topic/unusedLedgers/0for topicmy-topicand ledger 0.unusedLedgersz-node has these ledgers' id and these ledgers exist, after a partitioned topic's schema was created.BookkeeperSchemaStorage.delete()delete ledgers found from ZK, instead of recording ledgers in memory.Verifying this change
This change added tests and can be verified as follows:
Run test
PartitionedTopicsSchemaTest.testUnusedLedgers.