From 1124c6104546c21486b75d637caa4dde4787637f Mon Sep 17 00:00:00 2001 From: Doug Davis Date: Mon, 23 Jul 2018 18:20:57 -0700 Subject: [PATCH 1/3] Move extension serialization into other specs & rename extensions.md Replacement for PR #225 Made spec.md into more of an infoSet spec that just defines the properties of a CloudEvent and abstractly talks about how extension properties can be defined by others. Added some text to the primer to help clarify when extensions should be used and when extra properties are meant to be part of the event (`data`) itself. Renamed "extensions.md" to "experimental.md" because there was some confusion that CE extensions were only allowed to come from that doc. Hopefully, renaming it will make it clear that extensions can come from anythere - not just that doc. Left each serialization/transport-binding spec to deal with extensions in their own way. Most say nothing special about them, which means they will be serialized just like any other CE attribute. Based on the discussions in: https://docs.google.com/document/d/1h4A4ys3x6OmfEIv1gqlOt0kroVRfRO3JewlLunlxTIg/edit# , while it wasn't unanimous it did seem there was more people leaning towards serializing extensions like all other attributes due to the pain involved in: dealing with promotion of extension to 1st class properties, the desire to pass along all extensions (top-level or nested ones) to backend components, and support for future attributes added to the spec. See the doc for more details. Signed-off-by: Doug Davis --- Makefile | 2 +- README.md | 3 +- amqp-format.md | 4 +- extensions.md => documented-extensions.md | 0 json-format.md | 19 +++----- primer.md | 23 +++++++--- spec.md | 54 +++++++++++++---------- 7 files changed, 59 insertions(+), 46 deletions(-) rename extensions.md => documented-extensions.md (100%) diff --git a/Makefile b/Makefile index cc302e2ed..b8d183f73 100644 --- a/Makefile +++ b/Makefile @@ -5,7 +5,7 @@ verify: @# Use "-x" if you want to skip external links @tools/verify-links.sh -v . @echo Running the spec phrase checker: - @tools/verify-specs.sh -v spec.md extensions.md json-format.md \ + @tools/verify-specs.sh -v spec.md documented-extensions.md json-format.md \ http-transport-binding.md http-webhook.md mqtt-transport-binding.md \ nats-transport-binding.md @echo Running the doc phrase checker: diff --git a/README.md b/README.md index 820996fb6..d72c50eb3 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,8 @@ The following specifications are available: | **AMQP Event Format** | - | [master](https://github.com/cloudevents/spec/blob/master/amqp-format.md) | | **AMQP Transport Binding** | - | [master](https://github.com/cloudevents/spec/blob/master/amqp-transport-binding.md) | -There is also the [CloudEvents Extension Attributes](https://github.com/cloudevents/spec/blob/master/extensions.md) +There is also the +[CloudEvents documented extension attributes](https://github.com/cloudevents/spec/blob/master/documented-extensions.md) document and a [CloudEvents Primer](primer.md) to help understand some of the history and design goals of the working group. diff --git a/amqp-format.md b/amqp-format.md index 78dfdf9fa..69574442f 100644 --- a/amqp-format.md +++ b/amqp-format.md @@ -40,8 +40,8 @@ interpreted as described in [RFC2119][RFC2119]. This section defines how CloudEvents attributes are mapped to the AMQP type-system. This specification does not explicitly map each attribute, but -provides a generic mapping model that applies to all current and future Cloud -Event attributes. +provides a generic mapping model that applies to all current and future +CloudEvents attributes, including extensions. ### 2.1. Base Type System diff --git a/extensions.md b/documented-extensions.md similarity index 100% rename from extensions.md rename to documented-extensions.md diff --git a/json-format.md b/json-format.md index dca4054a1..a154ed596 100644 --- a/json-format.md +++ b/json-format.md @@ -39,8 +39,10 @@ interpreted as described in [RFC2119][RFC2119]. ## 2. Attributes This section defines how CloudEvents attributes are mapped to JSON. This -specification does not explicitly map each attribute, but provides a generic -mapping model that applies to all current and future CloudEvents attributes. +specification does not explicitly map each attribute, but +provides a generic mapping model that applies to all current and future +CloudEvents attributes, including extensions. + ### 2.1. Base Type System @@ -86,7 +88,6 @@ The following table shows exemplary mappings: | eventID | String | "1234-1234-1234" | eventTime | Timestamp| "2018-04-05T17:31:00Z" | contentType | String | "application/json" -| extensions | Map | { "extA" : "vA", "extB", "vB" } | data | String | "" | data | Binary | "Q2xvdWRFdmVudHM=" | data | Map | { "objA" : "vA", "objB", "vB" } @@ -146,9 +147,7 @@ Example event with `String`-valued `data`: "source" : "/mycontext", "eventID" : "A234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "extensions" : { - "comExampleExtension" : "value" - }, + "comExampleExtension" : "value", "contentType" : "text/xml", "data" : "" } @@ -163,9 +162,7 @@ Example event with `Binary`-valued data "source" : "/mycontext", "eventID" : "B234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "extensions" : { - "comExampleExtension" : "value" - }, + "comExampleExtension" : "value", "contentType" : "application/vnd.apache.thrift.binary", "data" : "... base64 encoded string ..." } @@ -181,9 +178,7 @@ a `Map` or [JSON data](#31-special-handling-of-the-data-attribute) data: "source" : "/mycontext", "eventID" : "C234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "extensions" : { - "comExampleExtension" : "value" - }, + "comExampleExtension" : "value", "contentType" : "application/json", "data" : { "appinfoA" : "abc", diff --git a/primer.md b/primer.md index d5c0c7fee..15d2383c2 100644 --- a/primer.md +++ b/primer.md @@ -103,10 +103,13 @@ either the event producer or event consumer. CloudEvents, at its core, defines a set of metadata, called attributes, about the event being transferred between systems, and how those pieces of metadata -should appear in that message. The metadata is not meant to duplicate any of -the application data of the event itself, rather it is just the minimal -information needed to route the request to the proper component that will -process the event. +should appear in that message. This metadata is meant to be the minimal +set of information needed to route the request to the proper component that +will process the event. So, while this might mean that some of the application +data of the event itself might be duplicated as part of the CloudEvent's set +of properties, this is to be done solely for the purpose of proper delivery +of the message. Data that is meant strictly for use by the component +processing the event should be within the event. Along with the definition of these attributes, there will also be specifications of how to serialize the event in different formats and @@ -136,8 +139,9 @@ in these cases will be defined within the specfication itself. When the working group determines that an attribute is not common enough to fall into those two categories but would still benefit from the level of interoperability that comes from being well-defined, then they will be placed -into the "extensions" category and put into the (extensions)[extensions.md] -document. The specification defines how these extension attributes will +into the "extensions" category and put into +(documented extensions)[documented-extensions.md]. +The specification defines how these extension attributes will appear within a CloudEvent. In determining which category a proposed attribute belongs, or even if it @@ -146,6 +150,13 @@ user-stories to explain the rationale and need for them. This supporting information will be added to the [Prior Art](#prior-art) section of this document. +Extension attributes to the CloudEvent specification are meant to +be additional metadata that needs to be included to help ensure proper +routing and processing of the CloudEvent. Additional metadata for other +purposes, that is related to the event itself and not needed in the +transportation of the CloudEvent, should instead be placed within the proper +extensibility points of the event itself. + ## Qualifying Protocols and Encodings The explicit goal of the CloudEvents effort, as expressed in the specification, diff --git a/spec.md b/spec.md index d6760ae9b..cc0035816 100644 --- a/spec.md +++ b/spec.md @@ -138,6 +138,28 @@ attributes and the event data will be materialized. For example, in the case of a JSON serialization, the context attributes and the event data might both appear within the same JSON object. +### Extension Attributes +CloudEvent producers MAY include additional extension attributes within the +event. This enables event producers, or middleware, to include additional +metadata that might be used for any purpose in the processing of the +CloudEvent, such as identifying or correlating event sources. See +[CloudEvent Attributes Extensions](primer.md#cloudevent-attribute-extensions) +for additional information concerning the use and definition of +extensions. + +This specification places no restriction on the type or semantics of the +extension attributes. Each definition of an extensions SHOULD fully +define all aspects of the attribute - e.g. its name, semantic meaning +and possible values. New extension definitions SHOULD use a name that is +descriptive enough to reduce the chances of name collisions with other +extensions. + +Each specification that defines how to serialize a CloudEvent will +define how extension attributes will appear. + +See [documented extensions](documented-extensions.md) for a list +of extension attributes that might be of interest. + ### eventType * Type: `String` * Description: Type of occurrence which has happened. Often this @@ -207,38 +229,24 @@ Incompatible changes to the schema SHOULD be reflected by a different URL. `contentType` values are defined in the event format specifications; for example, the JSON event format defines the relationship in [section 3.1](./json-format.md#31-special-handling-of-the-data-attribute). - - When this attribute is omitted, the "data" attribute simply follows the - event format's encoding rules. For the JSON event format, the "data" - attribute value can therefore be a JSON object, array, or value. - + + When this attribute is omitted, the "data" attribute simply follows the + event format's encoding rules. For the JSON event format, the "data" + attribute value can therefore be a JSON object, array, or value. + For the binary mode of some of the CloudEvents transport bindings, where the "data" content is immediately mapped into the payload of the transport frame, this field is directly mapped to the respective transport or application protocol's content-type metadata property. Normative rules - for the binary mode and the content-type metadata mapping can be found + for the binary mode and the content-type metadata mapping can be found in the respective transport mapping specifications. - + * Constraints: * OPTIONAL * If present, MUST adhere to the format specified in [RFC 2046](https://tools.ietf.org/html/rfc2046) * For Media Type examples see [IANA Media Types](http://www.iana.org/assignments/media-types/media-types.xhtml) -### extensions -* Type: `Map` -* Description: This is for additional metadata and this does not have a - mandated structure. This enables a place for custom fields a producer or - middleware might want to include and provides a place to test metadata before - adding them to the CloudEvents specification. - See the [Extensions](extensions.md) document for a list of possible - attributes. -* Constraints: - * OPTIONAL - * If present, MUST contain at least one entry -* Examples: - * authorization data - ## Data Attribute As defined by the term [Data](#data), CloudEvents MAY include domain-specific @@ -264,9 +272,7 @@ The following example shows a CloudEvent serialized as JSON: "source" : "/mycontext", "eventID" : "A234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "extensions" : { - "comExampleExtension" : "value" - }, + "comExampleExtension" : "value", "contentType" : "text/xml", "data" : "" } From 9285e380f0e87cad46f77ed905c5846a9278b60a Mon Sep 17 00:00:00 2001 From: Doug Davis Date: Thu, 9 Aug 2018 17:15:57 -0700 Subject: [PATCH 2/3] More editorial tweaks Signed-off-by: Doug Davis --- json-format.md | 15 ++++++++++++--- primer.md | 25 ++++++++++++++++++------- spec.md | 16 ++++++++++------ 3 files changed, 40 insertions(+), 16 deletions(-) diff --git a/json-format.md b/json-format.md index a154ed596..b5de02c0c 100644 --- a/json-format.md +++ b/json-format.md @@ -147,7 +147,10 @@ Example event with `String`-valued `data`: "source" : "/mycontext", "eventID" : "A234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "comExampleExtension" : "value", + "comExampleExtension1" : "value", + "comExampleExtension2" : { + "otherValue": 5 + }, "contentType" : "text/xml", "data" : "" } @@ -162,7 +165,10 @@ Example event with `Binary`-valued data "source" : "/mycontext", "eventID" : "B234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "comExampleExtension" : "value", + "comExampleExtension1" : "value", + "comExampleExtension2" : { + "otherValue": 5 + }, "contentType" : "application/vnd.apache.thrift.binary", "data" : "... base64 encoded string ..." } @@ -178,7 +184,10 @@ a `Map` or [JSON data](#31-special-handling-of-the-data-attribute) data: "source" : "/mycontext", "eventID" : "C234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "comExampleExtension" : "value", + "comExampleExtension1" : "value", + "comExampleExtension2" : { + "otherValue": 5 + }, "contentType" : "application/json", "data" : { "appinfoA" : "abc", diff --git a/primer.md b/primer.md index 15d2383c2..18b5350ac 100644 --- a/primer.md +++ b/primer.md @@ -104,12 +104,13 @@ either the event producer or event consumer. CloudEvents, at its core, defines a set of metadata, called attributes, about the event being transferred between systems, and how those pieces of metadata should appear in that message. This metadata is meant to be the minimal -set of information needed to route the request to the proper component that -will process the event. So, while this might mean that some of the application +set of information needed to route the request to the proper component +and to facilitate proper processing of the event by that component. +So, while this might mean that some of the application data of the event itself might be duplicated as part of the CloudEvent's set -of properties, this is to be done solely for the purpose of proper delivery -of the message. Data that is meant strictly for use by the component -processing the event should be within the event. +of attributes, this is to be done solely for the purpose of proper delivery, +and processing, of the message. Data that is not intended for that purpose +should instead be placed within the event (the `data` attribute) itself. Along with the definition of these attributes, there will also be specifications of how to serialize the event in different formats and @@ -154,8 +155,18 @@ Extension attributes to the CloudEvent specification are meant to be additional metadata that needs to be included to help ensure proper routing and processing of the CloudEvent. Additional metadata for other purposes, that is related to the event itself and not needed in the -transportation of the CloudEvent, should instead be placed within the proper -extensibility points of the event itself. +transportation or processing of the CloudEvent, should instead be placed +within the proper extensibility points of the event (the `data` attribute) +itself. + +The specification places no restrictions on the type of the extension +attributes. Meaning, they may be simple types (e.g. strings, integers) +or they can be complex types of structured data. + +If an extension becomes popular then the specification authors might +consider moving it into the specification as a core attribute. This means +that the extension mechanism/process can be used as a way to vet new +attributes prior to formally adding them to the specification. ## Qualifying Protocols and Encodings diff --git a/spec.md b/spec.md index cc0035816..7e0a43013 100644 --- a/spec.md +++ b/spec.md @@ -150,16 +150,17 @@ extensions. This specification places no restriction on the type or semantics of the extension attributes. Each definition of an extensions SHOULD fully define all aspects of the attribute - e.g. its name, semantic meaning -and possible values. New extension definitions SHOULD use a name that is +and possible values or even to indicate that it places no restrictions on +its values. New extension definitions SHOULD use a name that is descriptive enough to reduce the chances of name collisions with other -extensions. +extensions. In particular, extension authors SHOULD check the +[documented extensions](documented-extensions.md) document for the +set of known extensions - not just for possible name conflicts but +for extensions that might be of interest. Each specification that defines how to serialize a CloudEvent will define how extension attributes will appear. -See [documented extensions](documented-extensions.md) for a list -of extension attributes that might be of interest. - ### eventType * Type: `String` * Description: Type of occurrence which has happened. Often this @@ -272,7 +273,10 @@ The following example shows a CloudEvent serialized as JSON: "source" : "/mycontext", "eventID" : "A234-1234-1234", "eventTime" : "2018-04-05T17:31:00Z", - "comExampleExtension" : "value", + "comExampleExtension1" : "value", + "comExampleExtension2" : { + "otherValue": 5 + }, "contentType" : "text/xml", "data" : "" } From 5cd8c0679b6c29f9b9bcd0eaa0ce4d00982475a1 Mon Sep 17 00:00:00 2001 From: Doug Davis Date: Thu, 9 Aug 2018 17:57:47 -0700 Subject: [PATCH 3/3] More tweaks Signed-off-by: Doug Davis --- primer.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/primer.md b/primer.md index 18b5350ac..0e7f53f0d 100644 --- a/primer.md +++ b/primer.md @@ -160,8 +160,8 @@ within the proper extensibility points of the event (the `data` attribute) itself. The specification places no restrictions on the type of the extension -attributes. Meaning, they may be simple types (e.g. strings, integers) -or they can be complex types of structured data. +attributes. Meaning, they may be simple types (e.g. strings, integers), +complex (e.g. structured) or undefined collection of attributes. If an extension becomes popular then the specification authors might consider moving it into the specification as a core attribute. This means