Skip to content

Move extension serialization into other specs & rename extensions.md #277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
duglin merged 3 commits into from
Aug 30, 2018
Merged

Move extension serialization into other specs & rename extensions.md #277

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1124c61
Move extension serialization into other specs & rename extensions.md
Jul 24, 2018
9285e38
More editorial tweaks
Aug 10, 2018
5cd8c06
More tweaks
Aug 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ verify:
@# Use "-x" if you want to skip external links
@tools/verify-links.sh -v .
@echo Running the spec phrase checker:
@tools/verify-specs.sh -v spec.md extensions.md json-format.md \
@tools/verify-specs.sh -v spec.md documented-extensions.md json-format.md \
http-transport-binding.md http-webhook.md mqtt-transport-binding.md \
nats-transport-binding.md
@echo Running the doc phrase checker:
Expand Down
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ The following specifications are available:
| **AMQP Event Format** | - | [master](https://github.com/cloudevents/spec/blob/master/amqp-format.md) |
| **AMQP Transport Binding** | - | [master](https://github.com/cloudevents/spec/blob/master/amqp-transport-binding.md) |

There is also the [CloudEvents Extension Attributes](https://github.com/cloudevents/spec/blob/master/extensions.md)
There is also the
[CloudEvents documented extension attributes](https://github.com/cloudevents/spec/blob/master/documented-extensions.md)
Copy link
Copy Markdown

@cathyhongzhang cathyhongzhang Aug 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link "https://github.com/cloudevents/spec/blob/master/documented-extensions.md" is not a valid link.

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it will be fixed once the PR is merged. For now its just within the scope of the branch/PR.

document and a [CloudEvents Primer](primer.md) to help understand some of
the history and design goals of the working group.

Expand Down
4 changes: 2 additions & 2 deletions amqp-format.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ interpreted as described in [RFC2119][RFC2119].

This section defines how CloudEvents attributes are mapped to the AMQP
type-system. This specification does not explicitly map each attribute, but
provides a generic mapping model that applies to all current and future Cloud
Event attributes.
provides a generic mapping model that applies to all current and future
CloudEvents attributes, including extensions.

### 2.1. Base Type System

Expand Down
0 extensions.md → documented-extensions.md
View file
Open in desktop
File renamed without changes.
22 changes: 13 additions & 9 deletions json-format.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,10 @@ interpreted as described in [RFC2119][RFC2119].
## 2. Attributes

This section defines how CloudEvents attributes are mapped to JSON. This
specification does not explicitly map each attribute, but provides a generic
mapping model that applies to all current and future CloudEvents attributes.
specification does not explicitly map each attribute, but
provides a generic mapping model that applies to all current and future
CloudEvents attributes, including extensions.


### 2.1. Base Type System

Expand Down Expand Up @@ -86,7 +88,6 @@ The following table shows exemplary mappings:
| eventID | String | "1234-1234-1234"
| eventTime | Timestamp| "2018-04-05T17:31:00Z"
| contentType | String | "application/json"
| extensions | Map | { "extA" : "vA", "extB", "vB" }
| data | String | "<much wow=\"xml\"/>"
| data | Binary | "Q2xvdWRFdmVudHM="
| data | Map | { "objA" : "vA", "objB", "vB" }
Expand Down Expand Up @@ -146,8 +147,9 @@ Example event with `String`-valued `data`:
"source" : "/mycontext",
"eventID" : "A234-1234-1234",
"eventTime" : "2018-04-05T17:31:00Z",
"extensions" : {
"comExampleExtension" : "value"
"comExampleExtension1" : "value",
"comExampleExtension2" : {
"otherValue": 5
},
"contentType" : "text/xml",
"data" : "<much wow=\"xml\"/>"
Expand All @@ -163,8 +165,9 @@ Example event with `Binary`-valued data
"source" : "/mycontext",
"eventID" : "B234-1234-1234",
"eventTime" : "2018-04-05T17:31:00Z",
"extensions" : {
"comExampleExtension" : "value"
"comExampleExtension1" : "value",
"comExampleExtension2" : {
"otherValue": 5
},
"contentType" : "application/vnd.apache.thrift.binary",
"data" : "... base64 encoded string ..."
Expand All @@ -181,8 +184,9 @@ a `Map` or [JSON data](#31-special-handling-of-the-data-attribute) data:
"source" : "/mycontext",
"eventID" : "C234-1234-1234",
"eventTime" : "2018-04-05T17:31:00Z",
"extensions" : {
"comExampleExtension" : "value"
"comExampleExtension1" : "value",
"comExampleExtension2" : {
"otherValue": 5
},
"contentType" : "application/json",
"data" : {
Expand Down
34 changes: 28 additions & 6 deletions primer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,14 @@ either the event producer or event consumer.

CloudEvents, at its core, defines a set of metadata, called attributes, about
the event being transferred between systems, and how those pieces of metadata
should appear in that message. The metadata is not meant to duplicate any of
the application data of the event itself, rather it is just the minimal
information needed to route the request to the proper component that will
process the event.
should appear in that message. This metadata is meant to be the minimal
set of information needed to route the request to the proper component
and to facilitate proper processing of the event by that component.
So, while this might mean that some of the application
data of the event itself might be duplicated as part of the CloudEvent's set
of attributes, this is to be done solely for the purpose of proper delivery,
and processing, of the message. Data that is not intended for that purpose
should instead be placed within the event (the `data` attribute) itself.

Along with the definition of these attributes, there will also be
specifications of how to serialize the event in different formats and
Expand Down Expand Up @@ -136,8 +140,9 @@ in these cases will be defined within the specfication itself.
When the working group determines that an attribute is not common enough to
fall into those two categories but would still benefit from the level of
interoperability that comes from being well-defined, then they will be placed
into the "extensions" category and put into the (extensions)[extensions.md]
document. The specification defines how these extension attributes will
into the "extensions" category and put into
(documented extensions)[documented-extensions.md].
The specification defines how these extension attributes will
appear within a CloudEvent.

In determining which category a proposed attribute belongs, or even if it
Expand All @@ -146,6 +151,23 @@ user-stories to explain the rationale and need for them. This supporting
information will be added to the [Prior Art](#prior-art) section of this
document.

Extension attributes to the CloudEvent specification are meant to
be additional metadata that needs to be included to help ensure proper
routing and processing of the CloudEvent. Additional metadata for other
purposes, that is related to the event itself and not needed in the
Copy link
Copy Markdown

@cathyhongzhang cathyhongzhang Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Again, why do we restrict the metadata purpose to be just "transportation of the CloudEvent"?
  2. Could you clarify what is meant by "extensibility points of the event itself."?

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some text to try to clear it up.

transportation or processing of the CloudEvent, should instead be placed
within the proper extensibility points of the event (the `data` attribute)
itself.

The specification places no restrictions on the type of the extension
Copy link
Copy Markdown

@cathyhongzhang cathyhongzhang Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metadata in the format of property/classification bag is not exactly the same as structured data format since we can only define the name of the bag and format of the bag, but have to leave the key-value pairs inside the bag to be customizable and added by the event producer. Suggest to reword the text as follows to avoid confusion

Meaning, they may be simple types (e.g. string, integers), or structured data, or property bags.

This applies to metadata defined in both the main spec and the extensions spec.

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some text for this.

attributes. Meaning, they may be simple types (e.g. strings, integers),
complex (e.g. structured) or undefined collection of attributes.

If an extension becomes popular then the specification authors might
consider moving it into the specification as a core attribute. This means
that the extension mechanism/process can be used as a way to vet new
attributes prior to formally adding them to the specification.

## Qualifying Protocols and Encodings

The explicit goal of the CloudEvents effort, as expressed in the specification,
Expand Down
56 changes: 33 additions & 23 deletions spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,29 @@ attributes and the event data will be materialized. For example, in the case
of a JSON serialization, the context attributes and the event data might
both appear within the same JSON object.
Copy link
Copy Markdown

@cathyhongzhang cathyhongzhang Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to add something like the following to the "Overview" section or another appropriate place:
"If a metadata in the format of a property bag will be used by a large number of use cases, then a consistent name will be given to the property bag and its semantics should be defined in the main spec.md "

Rationale for this:
•A consistent name is needed to prevent different event producers from giving different/random property names resulting in many property names for the same property bag.
•Simplify the Event consumers’ implementation. Instead of having to parse different property names from different event producers for the same type of information, the consumer just needs to parse one name.
•There are other standard protocols that define the names for property bags

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some more text to the primer, at the end of the extension section, to address this.


### Extension Attributes
CloudEvent producers MAY include additional extension attributes within the
event. This enables event producers, or middleware, to include additional
metadata that might be used for any purpose in the processing of the
Copy link
Copy Markdown
Contributor

@deissnerk deissnerk Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that middleware can change or add extension attributes of an event while passing it on?

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO yes. To me, middleware can be viewed as first a consumer, and then a producer. How it generates the outgoing event is up to it. If people don't like what it produces they won't use it. :-)

Copy link
Copy Markdown
Contributor

@deissnerk deissnerk Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duglin In my opinion, consuming and then producing again should lead to a new event with a new eventID. I realize that this question does not directly touch the main issue addressed by this PR. Shall I open a new issue raising the question of immutability of an event?

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deissnerk yea, please open a new issue for this topic since I agree with you that its an important, but different, topic since this would be a question for us regardless of whether we have a bag or not.

Copy link
Copy Markdown
Contributor

@deissnerk deissnerk Aug 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duglin
done -> #290

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

CloudEvent, such as identifying or correlating event sources. See
Copy link
Copy Markdown

@cathyhongzhang cathyhongzhang Aug 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the identityLabels property will be used by a large number of use cases and thus we might need to define it in the main spec.md as an optional attribute instead of defining it in the extensions.md. Suggest to discuss this in the next week's meeting.

Copy link
Copy Markdown
Collaborator Author

@duglin duglin Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a separate PR to discuss that - let's tackle that after this one is resolved.

[CloudEvent Attributes Extensions](primer.md#cloudevent-attribute-extensions)
for additional information concerning the use and definition of
extensions.

This specification places no restriction on the type or semantics of the
extension attributes. Each definition of an extensions SHOULD fully
define all aspects of the attribute - e.g. its name, semantic meaning
and possible values or even to indicate that it places no restrictions on
its values. New extension definitions SHOULD use a name that is
descriptive enough to reduce the chances of name collisions with other
extensions. In particular, extension authors SHOULD check the
[documented extensions](documented-extensions.md) document for the
set of known extensions - not just for possible name conflicts but
for extensions that might be of interest.

Each specification that defines how to serialize a CloudEvent will
define how extension attributes will appear.

### eventType
* Type: `String`
* Description: Type of occurrence which has happened. Often this
Expand Down Expand Up @@ -207,38 +230,24 @@ Incompatible changes to the schema SHOULD be reflected by a different URL.
`contentType` values are defined in the event format specifications; for
example, the JSON event format defines the relationship in
[section 3.1](./json-format.md#31-special-handling-of-the-data-attribute).
When this attribute is omitted, the "data" attribute simply follows the
event format's encoding rules. For the JSON event format, the "data"
attribute value can therefore be a JSON object, array, or value.

When this attribute is omitted, the "data" attribute simply follows the
event format's encoding rules. For the JSON event format, the "data"
attribute value can therefore be a JSON object, array, or value.

For the binary mode of some of the CloudEvents transport bindings,
where the "data" content is immediately mapped into the payload of the
transport frame, this field is directly mapped to the respective transport
or application protocol's content-type metadata property. Normative rules
for the binary mode and the content-type metadata mapping can be found
for the binary mode and the content-type metadata mapping can be found
in the respective transport mapping specifications.

* Constraints:
* OPTIONAL
* If present, MUST adhere to the format specified in
[RFC 2046](https://tools.ietf.org/html/rfc2046)
* For Media Type examples see [IANA Media Types](http://www.iana.org/assignments/media-types/media-types.xhtml)

### extensions
* Type: `Map`
* Description: This is for additional metadata and this does not have a
mandated structure. This enables a place for custom fields a producer or
middleware might want to include and provides a place to test metadata before
adding them to the CloudEvents specification.
See the [Extensions](extensions.md) document for a list of possible
attributes.
* Constraints:
* OPTIONAL
* If present, MUST contain at least one entry
* Examples:
* authorization data

## Data Attribute

As defined by the term [Data](#data), CloudEvents MAY include domain-specific
Expand All @@ -264,8 +273,9 @@ The following example shows a CloudEvent serialized as JSON:
"source" : "/mycontext",
"eventID" : "A234-1234-1234",
"eventTime" : "2018-04-05T17:31:00Z",
"extensions" : {
"comExampleExtension" : "value"
"comExampleExtension1" : "value",
"comExampleExtension2" : {
"otherValue": 5
},
"contentType" : "text/xml",
"data" : "<much wow=\"xml\"/>"
Expand Down