RocketMQ transport binding by zongtanghu · Pull Request #352 · cloudevents/spec

zongtanghu · 2018-11-23T14:04:43Z

I commited the rocketmq-transport-binding.md for CloudEvents.The issues is #351.

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

duglin · 2018-11-24T13:00:02Z

@zongtanghu can you please add this md file to the "verify" step in the Makefile so the verify-specs.sh script is run on it?

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu · 2018-11-24T14:32:14Z

Hi @duglin ,I have already add this md file to the "verify" step in the Makefile.

duglin · 2018-11-24T15:13:49Z

Thanks

zongtanghu · 2018-11-24T15:34:58Z

You're welcome.Please help to review RocketMQ transport binding document.Thank you very much.

duglin · 2018-11-24T19:18:39Z

 	@tools/verify-specs.sh -v spec.md documented-extensions.md json-format.md \
 		http-transport-binding.md http-webhook.md mqtt-transport-binding.md \
-		nats-transport-binding.md protobuf-format.md
+		nats-transport-binding.md protobuf-format.md rocketmq-transport-binding.md


can you wrap this at 80 cols?

Okay I will fix it.

duglin · 2018-11-24T19:18:49Z

+
+## Abstract
+
+The [RocketMQ][RocketMQ] Transport Binding for CloudEvents defines how events are mapped to RocketMQ messages.


please wrap this doc at 80 cols.

Okay,I will fix it in a new commit.

duglin · 2018-11-24T19:20:21Z

+
+### 1.3. Content Modes
+
+The specification defines two content modes for transferring events:


This feels a bit odd to say we're going to define 2 modes but then say only one of them is supported. I'd prefer to not confuse people by only talking about the one we do support.

Okay,I will rewrite this paragraph.

duglin · 2018-11-24T19:24:12Z

+The receiver of the event can distinguish between the two content modes by inspecting 
+the `CE_contentType` property of the RocketMQ message. If the value is prefixed with the 
+CloudEvents media type `application/cloudevents`, indicating the use of a known event format, 
+the receiver uses structured mode, otherwise it defaults to binary mode.


Again, let's drop the notion of binary since we don't support it. Then we can just mandate, with a MUST, what CE_contentType's value needs to be.

duglin · 2018-11-24T19:24:59Z

+
+Examples:
+
+    * `eventTime` maps to `CE_eventTime`


These need to use the new names we just adopted (time, id, specversion)

duglin · 2018-11-24T19:25:22Z

+that it cannot handle, for instance `application/cloudevents+avro`, it MAY still treat the event 
+as binary and forward it to another party as-is .
+
+### 3.1. Binary Content Mode


I'd drop this entire section since we're not supporting it.

Binary mode is used in rocketmq message header.So,I write the "Binary Content Mode" Here.

duglin · 2018-11-24T19:36:40Z

+
+## 3. RocketMQ Message Mapping
+The receiver of the event can distinguish between the two content modes by inspecting 
+the `CE_contentType` property of the RocketMQ message. If the value is prefixed with the 


Is CE_contentType the cloudevent property or one defined by RocketMQ? If it's the CE one then I don't think it is used to determine binary vs structured.

Again,binary mode is used in rocketmq message header.Structured content mode is used for RocketMQ message body.So,we put the cloudevent property into the message body to determine the message contentType.

From looking at: https://github.com/cloudevents/spec/pull/352/files#diff-b1ac390f402e79182f0aa7a46737eb90R198 something doesn't feel right. On line 198 you've pulled out one of the CE properties (contenttype) as special and placed it in the "userproperties" section but I don't think that's correct. "contenttype" is meant to describe the format of "data" not of the "payload" which is how you've used it. So, let me ask again... is "CE_contenttype" the cloudevents property or one defined by RocketMQ? I believe it should be defined by RocketMQ and NOT the CloudEvents one since at that point in the processing we don't even know we have a cloudEvent at all, so prefixing a property with "CE_" feels premature.

@duglin We need a 'CE_contenttype' field to ensure the receiver of the event can distinguish between the two content modes by inspecting the 'CE_contenttype' property of the RocketMQ message. and this is a property can be set according to the data format of CloudEvents.

duglin · 2018-11-24T19:38:19Z

+
+[CE]: ./spec.md
+[JSON-format]: ./json-format.md
+[RocketMQ]: http://rocketmq.apache.org/


Can you make this a direct pointer to the messaging format and not the project? I can't seem to find the format itself in the docs.

Hi,duglin,you can see our messaging format in github.The link is https://github.com/apache/rocketmq/blob/master/common/src/main/java/org/apache/rocketmq/common/message/Message.java.

Later,we will put the detailed messaging format on the direct pointer Link which is on our RocketMQ website.

duglin · 2018-11-24T19:39:31Z

+Example for the JSON format:
+
+```
+content-type: application/cloudevents+json; charset=UTF-8


the key doesn't match what's specified on line 168

Thanks,I will fix it.

duglin · 2018-11-24T19:42:25Z

+
+------------------ user properties -------------------
+
+CE_contentType: application/cloudevents+json; charset=UTF-8


Isn't there some RocketMQ defined header that should hold this value?

Year,I think Clound Event properties value hold this values more suitable rather than RocketMQ defined header.

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

duglin · 2018-11-26T14:15:21Z

+specification MUST support the [JSON event format][JSON-format].
+
+### 1.5. Security
+This specification does not introduce any new security features for RocketMQ, or mandate specific existing features to be used.


please wrap at 80. There are other lines down below too.

Okay,I will fix these issues.

clemensv · 2018-11-28T15:27:46Z

RocketMQ is a project-proprietary transport, is it not?

Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu · 2018-11-28T16:08:48Z

RocketMQ is a project-proprietary transport, is it not?

@clemensv Apache RocketMQ™ is an open source distributed messaging and streaming data platform.

duhengforever · 2018-11-28T16:13:13Z

@clemensv RocketMQ is a top level project in Apache Foundation, and it has been adopted by hundreds of companies, it‘s official website：https://rocketmq.apache.org/

duglin · 2018-11-29T19:33:37Z

@zongtanghu we discussed this on today's call and there was some concern that RocketMQ might not meet the minimum bar we've defined for new protocols. In particular, if you look in the primer you'll see this section:

The CloudEvents effort shall not become a vehicle to even implicitly endorse or promote project- or product-proprietary protocols, because that would be counterproductive towards CloudEvents' original goals.

For a protocol or encoding to qualify for a core CloudEvents event format or protocol binding, it must belong to either one of the following categories:

The protocol has a formal status as a standard with a widely-recognized multi-vendor protocol standardization body (e.g. W3C, IETF, OASIS, ISO)
The protocol has a "de-facto standard" status for its ecosystem category, which means it is used so widely that it is considered a standard for a given application. Practically, we would like to see at least one open source implementation under the umbrella of a vendor-neutral open-source organization (e.g. Apache, Eclipse, CNCF, .NET Foundation) and at least a dozen independent vendors using it in their products/services.

Can you comment on whether you think RocketMQ satisfies either one of those bullets? Since I don't think RocketMQ has been taken to an SDO, it's really the 2nd bullet that we'd like to hear your thoughts on. There was some concern that if there is just one implementation of this protocol then the interoperability aspect of our goals might not be met.

duhengforever · 2018-11-30T01:39:26Z

@duglin thanks for your comment and we are very grateful for your work.

But firstly, RocketMQ not only is a TLP of Apache Foundation, but also IMHO, it has been to be a de-facto standard in the field of e-commerce and finance， especially in China，it has been used by hundreds of companies in their production environment，its popularity has exceeded some standards in the messaging field that you are talking about.

Secondly, RocketMQ is the first implementation of OpenMessaing which is a vendor-neutral specification in the messaging field and it has been supported by a dozen independent vendors or cloud vendors.

So we think we meet the second bar list above, and we do the binding for the purpose of promoting the serverless ecology and the messaging ecology, and hope to describing event data in a common way in the messaging field, and provide more support for the severless ecology, so we think that these bars don’t make much sense. and we also don't need to make a binding with CE to enhance our brand because it doesn't make sense, and in my opinion, bar should be used to help with the ecological construction, rather than simply raising the threshold, after all, these have great subjective factors.

duglin · 2018-12-06T20:03:18Z

@zongtanghu on today's call we chatting about this and there were mixed feeling on it. However, @JemDay did have an interesting idea of creating a home for protocols that are in a similar situation. A home for them to be hosted and developed, but not yet made into an official CloudEvents approved specification. Not unlike how we have our "documented extensions". It could also be a good testing ground to see how much community interest there is in it. Anyway, @rachelmyers agreed to take a first pass at creating a proposal to see how the group feels about that direction. It is still possible that this PR get adopted "as is", or we may choose to direct it toward this other category of transport docs - we just don't know yet, but we wanted to explore this path before we made a final decision.

@JemDay ping @rachelmyers if you're interested in collaborating on the proposal.

zongtanghu · 2018-12-10T01:03:06Z

@duglin Thanks for your help.

duhenglucky · 2018-12-17T04:59:04Z

@JemDay @rachelmyers Welcome to help us complete this pr, and also explore a way for other projects to make the ecosystem of cloudevents more perfect.

duhenglucky · 2019-01-12T04:33:42Z

@duglin Could we push this PR move on together :)

duglin · 2019-01-15T14:01:08Z

@duhenglucky we're waiting for @rachelmyers's PR before we move forward on this one. @rachelmyers any update or ETA?

duhenglucky · 2019-01-28T12:14:05Z

@rachelmyers Hi, Could we work together to move forward on this PR? I think it's a good start for CloudEvents ecosystem.

clemensv · 2019-01-29T13:31:08Z

See #351 (comment)

duglin · 2019-03-16T01:08:10Z

With #380 now resolved, I believe that the WG will probably request that this PR be modified to add a link to this spec from the new proprietary-specs doc and that it be hosted elsewhere. I will add this to the next call's agenda for confirmation.

duglin · 2019-03-16T01:10:44Z

If you have an opinion on this topic (see previous comment) please either join the next weekly call or add a comment in this PR prior to the call to ensure your thoughts are taken into account.

duhenglucky · 2019-03-17T09:27:55Z

@duglin @clemensv Thanks for your work, we believe that this decision is fair and sustainable. and we also support the decision, so @zongtanghu could we work together to modify this PR to provide a link?

duglin · 2019-03-21T18:00:57Z

On today's call we agreed that this specification should follow the new process we've defined for proprietary specifications - plus the originators of this PR agreed with that direction. So, with that, I'm going to close this PR. @zongtanghu please open up another PR to modify the proprietary-specs.md document so it points to your hosted specification.

zongtanghu mentioned this pull request Nov 23, 2018

RocketMQ transport binding #351

Closed

zongtanghu force-pushed the master branch from a5f49c0 to 88c9b4d Compare November 23, 2018 14:10

zongtanghu changed the title ~~I commited the rocketmq-transport-binding.md for CloudEvents.~~ RocketMQ transport binding Nov 23, 2018

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

ba0463e

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 88c9b4d to ba0463e Compare November 24, 2018 05:45

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

74a3713

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from bd1141d to 74a3713 Compare November 24, 2018 05:56

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

d9a9882

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 64820d6 to d9a9882 Compare November 24, 2018 06:08

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

b5b1d9f

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 3059e7b to b5b1d9f Compare November 24, 2018 06:12

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

1d345f3

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 7c15944 to 1d345f3 Compare November 24, 2018 14:04

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

1f2c9ea

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 345c118 to 1f2c9ea Compare November 24, 2018 14:21

duglin reviewed Nov 24, 2018

View reviewed changes

I commited the rocketmq-transport-binding.md for CloudEvents,similar …

4dee749

…to the proposed NATS, MQTT, AMQP bindings. Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from 9bf3d0d to 4dee749 Compare November 26, 2018 12:47

duglin reviewed Nov 26, 2018

View reviewed changes

duglin mentioned this pull request Nov 26, 2018

Rename "contenttype" to "datatype" #354

Closed

I fix some issues for suggestion.

7546f2e

Signed-off-by: huzongtang <huzongtang@cmss.chinamobile.com>

zongtanghu force-pushed the master branch from ed48ec8 to 7546f2e Compare November 28, 2018 16:07

duglin closed this Mar 21, 2019

duhenglucky mentioned this pull request Mar 29, 2019

Add Apache RocketMQ proprietary transport binding with cloudevents #412

Merged


		## Abstract

		The [RocketMQ][RocketMQ] Transport Binding for CloudEvents defines how events are mapped to RocketMQ messages.


		### 1.3. Content Modes

		The specification defines two content modes for transferring events:


		------------------ user properties -------------------

		CE_contentType: application/cloudevents+json; charset=UTF-8

Conversation

zongtanghu commented Nov 23, 2018

Uh oh!

duglin commented Nov 24, 2018

Uh oh!

zongtanghu commented Nov 24, 2018

Uh oh!

duglin commented Nov 24, 2018

Uh oh!

zongtanghu commented Nov 24, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zongtanghu Nov 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clemensv commented Nov 28, 2018

Uh oh!

zongtanghu commented Nov 28, 2018

Uh oh!

duhengforever commented Nov 28, 2018

Uh oh!

duglin commented Nov 29, 2018

Uh oh!

duhengforever commented Nov 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

duglin commented Dec 6, 2018

Uh oh!

zongtanghu commented Dec 10, 2018

Uh oh!

duhenglucky commented Dec 17, 2018

Uh oh!

duhenglucky commented Jan 12, 2019

Uh oh!

duglin commented Jan 15, 2019

Uh oh!

zongtanghu Nov 26, 2018 •

edited

Loading

duhengforever commented Nov 30, 2018 •

edited

Loading