This repository was archived by the owner on Jan 24, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 142
Split the message storage logic into independent classes #925
Merged
BewareMyPower
merged 15 commits into
streamnative:master
from
Demogorgon314:split-storage
Nov 28, 2021
Merged
Split the message storage logic into independent classes #925
BewareMyPower
merged 15 commits into
streamnative:master
from
Demogorgon314:split-storage
Nov 28, 2021
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/KafkaProtocolHandler.java
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/KafkaRequestHandler.java
Outdated
Show resolved
Hide resolved
Collaborator
|
NOTE: I'll go out for a while and the review is not completed. You can address these comments first. |
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/KafkaRequestHandler.java
Outdated
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Outdated
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/utils/KopTopic.java
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Outdated
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Outdated
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Outdated
Show resolved
Hide resolved
kafka-impl/src/main/java/io/streamnative/pulsar/handlers/kop/storage/PartitionLog.java
Outdated
Show resolved
Hide resolved
Collaborator
|
LGTM but left some minor comments. |
BewareMyPower
approved these changes
Nov 28, 2021
BewareMyPower
pushed a commit
that referenced
this pull request
Dec 3, 2021
### Motivation Currently, the KoP store message logic is in `KafkaRequestHandler`, to avoid duplicate code and easier to support idempotent produce, we must refactor the message storage logic. ### Modifications * Refactor message storage logic. * The `ReplicaManager` is a public interface to store messages, we should always use `ReplicaManager` to store the message in any situation. * `PartitionLog` mapping to Kafka is `Log`, the reason for using this name is to avoid naming conflict, like: `log.debug()`. * `PartitionLogManager` hold all `PartitionLog`, key is full partition name, value is `PartitionLog`. ### TODO * Add fetch operation
BewareMyPower
pushed a commit
that referenced
this pull request
Dec 3, 2021
Currently, the KoP store message logic is in `KafkaRequestHandler`, to avoid duplicate code and easier to support idempotent produce, we must refactor the message storage logic. * Refactor message storage logic. * The `ReplicaManager` is a public interface to store messages, we should always use `ReplicaManager` to store the message in any situation. * `PartitionLog` mapping to Kafka is `Log`, the reason for using this name is to avoid naming conflict, like: `log.debug()`. * `PartitionLogManager` hold all `PartitionLog`, key is full partition name, value is `PartitionLog`. * Add fetch operation Fix conflicts made by #819. The `path` method was removed from master. Keep this method in branch-2.8.1.
eolivelli
pushed a commit
to eolivelli/kop
that referenced
this pull request
Dec 17, 2021
…e#925) Currently, the KoP store message logic is in `KafkaRequestHandler`, to avoid duplicate code and easier to support idempotent produce, we must refactor the message storage logic. * Refactor message storage logic. * The `ReplicaManager` is a public interface to store messages, we should always use `ReplicaManager` to store the message in any situation. * `PartitionLog` mapping to Kafka is `Log`, the reason for using this name is to avoid naming conflict, like: `log.debug()`. * `PartitionLogManager` hold all `PartitionLog`, key is full partition name, value is `PartitionLog`. * Add fetch operation (cherry picked from commit 1c07bd5)
Hongten
reviewed
Feb 16, 2023
|
|
||
| authorize(AclOperation.WRITE, Resource.of(ResourceType.TOPIC, fullPartitionName)) | ||
| .whenComplete((isAuthorized, ex) -> { | ||
| if (ex != null) { |
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems those two if can combine together. Like
if (ex != null || !isAuthorized) {
if (ex != null) {
log.error("Write topic authorize failed, topic - {}. {}",
fullPartitionName, ex.getMessage());
}
unauthorizedTopicResponsesMap.put(topicPartition,
new ProduceResponse.PartitionResponse(Errors.TOPIC_AUTHORIZATION_FAILED));
completeOne.run();
return;
}
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Currently, the KoP store message logic is in
KafkaRequestHandler, to avoid duplicate code and easier to support idempotent produce, we must refactor the message storage logic.Modifications
Refactor message storage logic.
The
ReplicaManageris a public interface to store messages, we should always useReplicaManagerto store the message in any situation.PartitionLogmapping to Kafka isLog, the reason for using this name is to avoid naming conflict, like:log.debug().PartitionLogManagerhold allPartitionLog, key is full partition name, value isPartitionLog.TODO