Skip to content

PIP-267: Support multi-topic messageId deserialization to ack messages #20221

@rdhabalia

Description

@rdhabalia

Motivation

Pulsar api provides MessageId interface which is generally used by producer and consumer applications to manage topic offset. Sometimes, these applications would like to serialize and deserialize messageIds specifically consumer app which would like to persist messageId and ack with those messageIds by deserializing them. However, right now Pulsar doesn't support correct deserialization of multi-topic or partitioned-topic because of that 1acknowledge` API call fails for those topics with below error

org.apache.pulsar.client.api.PulsarClientException$NotAllowedException: Only TopicMessageId is allowed to acknowledge for a multi-topics consumer, while messageId is org.apache.pulsar.client.impl.BatchMessageIdImpl
	at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1083)
	at org.apache.pulsar.client.impl.ConsumerBase.acknowledge(ConsumerBase.java:418)
	at org.apache.pulsar.client.api.MultiTopicsConsumerTest.testMultiTopicAckWithByteMessageId(MultiTopicsConsumerTest.java:409)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

MessageIdImpl stores id metadata into MessageIdData which is doesn't contain context about topic name to find out which topic belongs to this MessageID. Thefore, we need to add topic-name into MessageIdData and allow multi-topic/partitioned topic to deserialize message correctly so, API like acknowledge can perform as expected.

Goal

Fix MessageId serialization/deserialization for multi-topic and partitioned-topics so, various apis such as acknowledge can work without any error.

This issue is already implemented in this PR: #19944

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions