Skip to content

Conversation

@rdhabalia
Copy link
Contributor

Motivation

Incrementing partition for the topic requires creating new partitions with existing subscriptions. Sometimes, partitions of the topic are distributed to multiple brokers, and the creation of few partitions may fail due to various reasons. However, retrying update-partition command for such topic fails with the below error

Caused by: org.apache.pulsar.client.admin.PulsarAdminException$ConflictException: Subscription already exists for topic
	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:219) ~[classes/:?]
	at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:129) ~[classes/:?]
	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:839) ~[jersey-client-2.31.jar:?]
	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:820) ~[jersey-client-2.31.jar:?]
	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[jersey-client-2.31.jar:?]
	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[jersey-client-2.31.jar:?]
	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[jersey-client-2.31.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[jersey-common-2.31.jar:?]
	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[jersey-client-2.31.jar:?]
	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$0(AsyncHttpConnector.java:212) ~[classes/:?]

We frequently face this issue and need to manually update znode to fix the issue and increment partitions of the topic.

Modification

Handle Subscription already exists for topic error and create failed partitions to successfully increment partitions. Provide force flag to handle the error and successfully create partially created partitons.

@rdhabalia rdhabalia added this to the 2.8.0 milestone Apr 26, 2021
@rdhabalia rdhabalia self-assigned this Apr 26, 2021
@rdhabalia rdhabalia force-pushed the part_create branch 2 times, most recently from 491af3f to 1ee8b4b Compare May 11, 2021 22:00
@rdhabalia
Copy link
Contributor Author

/pulsarbot run-failure-checks

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall LGTM
but we are breaking the Admin API.

we should not remove an existing method, but only add a new one

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli eolivelli modified the milestones: 2.9.0, 2.10.0 Oct 6, 2021
@rdhabalia rdhabalia merged commit bfa2b29 into apache:master Oct 6, 2021
@rdhabalia rdhabalia deleted the part_create branch October 6, 2021 21:17
@eolivelli eolivelli modified the milestones: 2.10.0, 2.9.0 Oct 8, 2021
eolivelli pushed a commit that referenced this pull request Oct 8, 2021
* [pulsar-broker] Fix: handle failed partitions topic creation

* fix test
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
…10374)

* [pulsar-broker] Fix: handle failed partitions topic creation

* fix test
AnonHxy pushed a commit that referenced this pull request Aug 28, 2022
…ception: subscription already exists for topic (#17251)

### Motivation
#10374 wants to handle the existing subscriptions when updating partition with `force` command parameter. However, it could not update the partition metadata if the `Subscription already exists for topic` exception comes. Because it miss updating when handling such exception.
Pomelongan pushed a commit to Pomelongan/pulsar that referenced this pull request Aug 28, 2022
…ception: subscription already exists for topic (apache#17251)

### Motivation
apache#10374 wants to handle the existing subscriptions when updating partition with `force` command parameter. However, it could not update the partition metadata if the `Subscription already exists for topic` exception comes. Because it miss updating when handling such exception.
aloyszhang pushed a commit to aloyszhang/pulsar that referenced this pull request Nov 30, 2022
…10374)

* [pulsar-broker] Fix: handle failed partitions topic creation

* fix test
Technoboy- pushed a commit that referenced this pull request Mar 6, 2023
…ception: subscription already exists for topic (#17251)

### Motivation
#10374 wants to handle the existing subscriptions when updating partition with `force` command parameter. However, it could not update the partition metadata if the `Subscription already exists for topic` exception comes. Because it miss updating when handling such exception.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants