Allow to enable/disable delayed delivery for messages on namespace #5915

wolfstudy · 2019-12-23T03:32:03Z

Signed-off-by: xiaolong.ran rxl@apache.org

Fixes: #4080

Modifications

add set-delayed-delivery policy for namespace
add set-delayed-delivery-time policy for namespace
add test case
update admin cli docs

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui

I think use a single command to enable or disable delay delivery is more convenient.

pulsar-admin namespaces set-delayed-delivery --enable --tick_time_mills 2000 public/default
pulsar-admin namespaces set-delayed-delivery --disable public/default

And we need to add method for get delayed-delivery

pulsar-admin namespaces get-delayed-delivery public/default

{
"enable": true,
"tick_time_mills": 2000
}

BTW, how do we handle exists messages already push to delayed message tracker when disable delayed delivery, send them to consumer immediately?

codelipenghui · 2019-12-24T09:18:44Z

pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/Policies.java

    @SuppressWarnings("checkstyle:MemberName")
+    public boolean delayed_delivery = true;
+    @SuppressWarnings("checkstyle:MemberName")
+    public long delayed_delivery_time = 0;


It should be delayed_delivery_tick_time

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy · 2019-12-26T13:58:15Z

@codelipenghui PTAL again, thanks.

Get

$ ./bin/pulsar-admin namespaces get-delayed-delivery public/default

{
  "tickTime" : 1000,
  "active" : true
}

Set

$ ./bin/pulsar-admin namespaces set-delayed-delivery public/default --enable --time 2s

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy · 2019-12-27T12:19:19Z

ping @codelipenghui PTAL again, thanks.

sijie · 2020-01-01T01:14:06Z

@codelipenghui can you help review this pull request again?

congbobo184 · 2020-01-03T08:02:56Z

pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/Namespaces.java

+     *     "tickTime" : 1000, // Enable or disable delayed delivery for messages on a namespace
+     *     "active" : true,   // The tick time for when retrying on delayed delivery messages


are the two notes reversed?

Signed-off-by: xiaolong.ran <rxl@apache.org>

sijie · 2020-01-06T03:29:30Z

@codelipenghui can you review this change?

codelipenghui · 2020-01-06T11:54:43Z

.../java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java

    @Override
    public boolean trackDelayedDelivery(long ledgerId, long entryId, MessageMetadata msgMetadata) {
-        if (!isDelayedDeliveryEnabled) {
+        if (!isDelayedDeliveryEnabled || !topic.checkActiveDelayedDelivery()) {


The isDelayedDeliveryEnabled in the dispatcher is for indicating the dispatcher is enable or disable delayed delivery. It's better to update this field while the delayed delivery policy changed because it's more economic than get policy from a hashmap for each message.

codelipenghui · 2020-01-06T11:58:39Z

.../java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java

                        .of(topic.getBrokerService().getDelayedDeliveryTrackerFactory().newTracker(this));
            }

+            delayedDeliveryTracker.get().resetTickTime(topic.checkTickTimeDelayedDelivery());


Same as above comment, reset tick time when the namespace policy changed is more economic than check need to reset for every message

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy · 2020-01-19T12:58:06Z

Sorry, i missed the message.

@codelipenghui fixed done, please take a look again, thanks.

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui · 2020-01-20T02:55:54Z

.../java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java

    @Override
    public boolean trackDelayedDelivery(long ledgerId, long entryId, MessageMetadata msgMetadata) {
-        if (!isDelayedDeliveryEnabled) {
+        if (!isDelayedDeliveryEnabled || !topic.delayedDeliveryEnabled) {


I think you can remove the isDelayedDeliveryEnabled in dispatcher, maybe we just need it on topic level on subscription level

codelipenghui · 2020-01-20T02:57:57Z

pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java

+
+

remove these 2 lines

codelipenghui

Looks good to me, just left some minor comments, please take a look

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy · 2020-01-20T06:16:18Z

retest this please

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy · 2020-01-21T13:00:58Z

run java8 tests
run cpp tests

wolfstudy · 2020-01-21T13:43:46Z

run java8 tests

tuteng · 2020-03-21T03:12:56Z

Add label release-2.5.1, due #6183 dependency

…pache#5915) * Allow to enable/disable delyed delivery for messages on namespace Signed-off-by: xiaolong.ran <rxl@apache.org> * add isDelayedDeliveryEnabled function Signed-off-by: xiaolong.ran <rxl@apache.org> * add delayed_delivery_time process logic Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case Signed-off-by: xiaolong.ran <rxl@apache.org> * update admin cli docs Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * update import lib Signed-off-by: xiaolong.ran <rxl@apache.org> * avoid import * Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * remove unuse code Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case for delayed delivery messages Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> (cherry picked from commit f0d339e)

…5915) * Allow to enable/disable delyed delivery for messages on namespace Signed-off-by: xiaolong.ran <rxl@apache.org> * add isDelayedDeliveryEnabled function Signed-off-by: xiaolong.ran <rxl@apache.org> * add delayed_delivery_time process logic Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case Signed-off-by: xiaolong.ran <rxl@apache.org> * update admin cli docs Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * update import lib Signed-off-by: xiaolong.ran <rxl@apache.org> * avoid import * Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * remove unuse code Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case for delayed delivery messages Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> (cherry picked from commit f0d339e)

…pache#5915) * Allow to enable/disable delyed delivery for messages on namespace Signed-off-by: xiaolong.ran <rxl@apache.org> * add isDelayedDeliveryEnabled function Signed-off-by: xiaolong.ran <rxl@apache.org> * add delayed_delivery_time process logic Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case Signed-off-by: xiaolong.ran <rxl@apache.org> * update admin cli docs Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * update import lib Signed-off-by: xiaolong.ran <rxl@apache.org> * avoid import * Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * remove unuse code Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case for delayed delivery messages Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> (cherry picked from commit f0d339e)

…pache#5915) * Allow to enable/disable delyed delivery for messages on namespace Signed-off-by: xiaolong.ran <rxl@apache.org> * add isDelayedDeliveryEnabled function Signed-off-by: xiaolong.ran <rxl@apache.org> * add delayed_delivery_time process logic Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case Signed-off-by: xiaolong.ran <rxl@apache.org> * update admin cli docs Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * update import lib Signed-off-by: xiaolong.ran <rxl@apache.org> * avoid import * Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * remove unuse code Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * add test case for delayed delivery messages Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org> * fix comments Signed-off-by: xiaolong.ran <rxl@apache.org>

Allow to enable/disable delyed delivery for messages on namespace

d70c283

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy self-assigned this Dec 23, 2019

wolfstudy requested review from jiazhai, merlimat and sijie December 23, 2019 03:32

wolfstudy added the type/feature The PR added a new feature or issue requested a new feature label Dec 23, 2019

wolfstudy added this to the 2.6.0 milestone Dec 23, 2019

wolfstudy changed the title ~~Allow to enable/disable delayed delivery for messages on namespace~~ [WIP]Allow to enable/disable delayed delivery for messages on namespace Dec 23, 2019

wolfstudy added 3 commits December 23, 2019 14:39

add isDelayedDeliveryEnabled function

bb5d4bb

Signed-off-by: xiaolong.ran <rxl@apache.org>

add delayed_delivery_time process logic

ed80499

Signed-off-by: xiaolong.ran <rxl@apache.org>

add test case

fb7fca8

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy changed the title ~~[WIP]Allow to enable/disable delayed delivery for messages on namespace~~ Allow to enable/disable delayed delivery for messages on namespace Dec 23, 2019

wolfstudy requested a review from codelipenghui December 23, 2019 12:02

update admin cli docs

404d0fc

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui reviewed Dec 24, 2019

View reviewed changes

wolfstudy added 3 commits December 26, 2019 16:42

fix comments

af92cfe

Signed-off-by: xiaolong.ran <rxl@apache.org>

fix comments

f8aa453

Signed-off-by: xiaolong.ran <rxl@apache.org>

fix comments

5c49513

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy added 2 commits December 26, 2019 22:01

update import lib

59a2454

Signed-off-by: xiaolong.ran <rxl@apache.org>

avoid import *

d6f2b7d

Signed-off-by: xiaolong.ran <rxl@apache.org>

congbobo184 approved these changes Jan 3, 2020

View reviewed changes

fix comments

1bd4dc0

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui reviewed Jan 6, 2020

View reviewed changes

fix comments

41a74f0

Signed-off-by: xiaolong.ran <rxl@apache.org>

remove unuse code

b475f80

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui reviewed Jan 20, 2020

View reviewed changes

fix comments

5acfa56

Signed-off-by: xiaolong.ran <rxl@apache.org>

codelipenghui approved these changes Jan 20, 2020

View reviewed changes

add test case for delayed delivery messages

3b57384

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy added 3 commits January 20, 2020 16:11

fix comments

425b886

Signed-off-by: xiaolong.ran <rxl@apache.org>

Merge branch 'master' into xiaolong/add-delayed-deliver

71c0fb1

fix comments

1d1df2f

Signed-off-by: xiaolong.ran <rxl@apache.org>

wolfstudy merged commit f0d339e into apache:master Jan 21, 2020

tuteng added the release/2.5.1 label Mar 21, 2020

		* "tickTime" : 1000, // Enable or disable delayed delivery for messages on a namespace
		* "active" : true, // The tick time for when retrying on delayed delivery messages

Allow to enable/disable delayed delivery for messages on namespace #5915

Allow to enable/disable delayed delivery for messages on namespace #5915

Uh oh!

Conversation

wolfstudy commented Dec 23, 2019 • edited by sijie Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Modifications

Uh oh!

codelipenghui left a comment

Choose a reason for hiding this comment

Uh oh!

codelipenghui Dec 24, 2019

Choose a reason for hiding this comment

Uh oh!

wolfstudy commented Dec 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Get

Set

Uh oh!

wolfstudy commented Dec 27, 2019

Uh oh!

sijie commented Jan 1, 2020

Uh oh!

congbobo184 Jan 3, 2020

Choose a reason for hiding this comment

Uh oh!

sijie commented Jan 6, 2020

Uh oh!

codelipenghui Jan 6, 2020

Choose a reason for hiding this comment

Uh oh!

codelipenghui Jan 6, 2020

Choose a reason for hiding this comment

Uh oh!

wolfstudy commented Jan 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codelipenghui Jan 20, 2020

Choose a reason for hiding this comment

Uh oh!

codelipenghui Jan 20, 2020

Choose a reason for hiding this comment

Uh oh!

codelipenghui left a comment

Choose a reason for hiding this comment

Uh oh!

wolfstudy commented Jan 20, 2020

Uh oh!

wolfstudy commented Jan 21, 2020

Uh oh!

wolfstudy commented Jan 21, 2020

Uh oh!

tuteng commented Mar 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

wolfstudy commented Dec 23, 2019 •

edited by sijie

Loading

wolfstudy commented Dec 26, 2019 •

edited

Loading

wolfstudy commented Jan 19, 2020 •

edited

Loading