Channel spec: iteration 1

docs/spec/README.md

@@ -14,6 +14,7 @@ Docs in this directory:
 - [Resource type overview](overview.md)
 - [Interface contracts](interfaces.md)
 - [Object model specification](spec.md)
+- [Channel specification](channel.md) and [older channel (0.6.0) spec](channel_060.md)
 
 <!-- TODO(n3wscott): * [Error conditions and reporting](errors.md) -->
 <!-- TODO(n3wscott): * [Sample API usage](normative_examples.md) -->

docs/spec/channel.md

@@ -0,0 +1,140 @@
+# Channel Spec (IN PROGRESS)
+
+## Background
+
+Starting with Version 0.7.0 all the different Channel CRDs (e.g. `InMemoryChannel` or `KafkaChannel`) are living in the `messaging.knative.dev/v1alpha1` API Group.
+
+A channel logically receives events on its input domain and forwards them to its subscribers. Below is a specification for the generic parts of each _Channel_.
+
+A typical channel consists of a _Controller_ and a _Dispatcher_ pod.
+
+## Channel Spec Parts
+
+### Control Plane
+
+Each Channel implementation is backed by its own CRD (e.g. `InMemoryChannel` or `KafkaChannel`). Unlike in earlier versions there is no concept for a _default channel_, using the different CRDs. Below is an example for a `KafkaChannel` object:
+
+```
+apiVersion: messaging.knative.dev/v1alpha1
+kind: KafkaChannel
+metadata:
+  name: kafka-channel
+spec:
+  numPartitions: 3
+  replicationFactor: 1
+```
+
+A different example for the `InMemoryChannel`:
+
+```
+apiVersion: messaging.knative.dev/v1alpha1
+  kind: InMemoryChannel
+  metadata:
+    name: my-channel
+```
+
+Each _Channel Controller_ ensures the required tasks on the backing technology are applied. In this case a Kafka topic with the desired configuration is being created, backing all messages from the channel.
+
+#### Aggregated Channelable ClusterRole
+
+Every CRD must create a corresponding ClusterRole, that will be aggregated into the `channelable-manipulator` ClusterRole. This ClusterRole must include permissions to create, read, patch, and update the CRD's custom objects. Below is an example for the `KafkaChannel`:
+
+```
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kafka-channelable-manipulator
+  labels:
+    duck.knative.dev/channelable: "true"
+# Do not use this role directly. These rules will be added to the "channelable-manipulator" role.
+rules:
+  - apiGroups:
+      - messaging.knative.dev
+    resources:
+      - kafkachannels
+      - kafkachannels/status
+    verbs:
+      - create
+      - get
+      - list
+      - watch
+      - update
+      - patch
+```
+
+Each channel MUST have the `duck.knative.dev/channelable: "true"` label on its `channelable-manipulator` CR file.
+
+#### CustomResourceDefinition per Channel
+
+For each channel implementation a `CustomResourceDefinition` is created, like:
+
+```
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: kafkachannels.messaging.knative.dev
+ labels:
+    knative.dev/crd-install: "true"
+    messaging.knative.dev/subscribable: "true"
+spec:
+  group: messaging.knative.dev
+  version: v1alpha1
+  names:
+    kind: KafkaChannel
+    plural: kafkachannels
+    singular: kafkachannel
+    categories:
+    - all
+    - knative
+    - messaging
+    - channel
+    shortNames:
+    - kc
+  scope: Namespaced
+...
+```
+
+Each channel is _namespaced_ and MUST have the following:
+
+* label of `messaging.knative.dev/subscribable: "true"`
+* The category `channel`.
+
+#### Broker and Triggers
+
+TODO
+
+### Data Plane
+
+The data plane describes the input and output flow of a _Channel_. All Channels exclusively communicate using CloudEvents.
+
+#### Input
+
+Every Channel must expose either an HTTP or HTTPS endpoint. It MAY expose both. The endpoint(s) MUST conform to [HTTP Transport Binding for CloudEvents - Version 0.2](https://github.com/cloudevents/spec/blob/v0.2/http-transport-binding.md). It MUST support both Binary Content mode and Structured Content mode. The HTTP(S) endpoint MAY be on any port, not just the standard 80 and 443. Channels MAY expose other, non-HTTP endpoints in addition to HTTP at their discretion (e.g. expose a gRPC endpoint to accept events).
+
+##### Generic
+
+If a Channel receives an event queueing request and is unable to parse a valid CloudEvent, then it MUST reject the request.
+
+Every event queueing request to the Channel will come with a bearer token, likely a JWT. The bearer token MUST be validated before any other work is done on the request. The specifics of how and what to validate will be identical to Broker ingress verification, which is being [defined](https://github.com/knative/eventing/issues/705#issuecomment-496722527).
+
+The Channel MUST pass through all tracing information as CloudEvents attributes. In particular, it MUST translate any incoming OpenTracing or B3 headers to the [Distributed Tracing Extension](https://github.com/cloudevents/spec/blob/v0.2/extensions/distributed-tracing.md). The Channel SHOULD sample and write traces to the location specified in [`config-tracing`](https://github.com/cloudevents/spec/blob/v0.2/extensions/distributed-tracing.md).
+
+##### HTTP
+
+Channels MUST reject all HTTP event queueing requests with a method other than POST responding with HTTP status code `405 Method Not Supported`. Non-event queueing requests (e.g. health checks) are not constrained.
+
+The HTTP event queueing request's URL MUST correspond to a single, unique Channel at any given moment in time. This MAY be done via the host, path, query string, or any combination of these. This mapping is handled exclusively by the Channel implementation, exposed via the Channel's `status.address`. If an HTTP event queueing request's URL does not correspond to an existing Channel, then the Channel MUST respond with `404 Not Found`.
+
+The Channel MUST respond with `202 Accepted` if the event queueing request is accepted by the server.
+
+If a Channel receives an event queueing request and is unable to parse a valid CloudEvent, then it MUST respond with `400 Bad Request`.
+
+If a Channel receives an event queueing request without a bearer token or the provided bearer token fails to validate, then the Channel must respond with `403 Forbidden`.
+
+#### Output
+
+Channels MUST output CloudEvents. The output MUST be via a binding specified in the [CloudEvents specification](https://github.com/cloudevents/spec/tree/v0.2#cloudevents-documents). Every Channel MUST support sending events via Structured Content Mode HTTP Transport Binding.
+
+Channels MUST NOT alter an event that goes through them. All CloudEvent attributes, including the data attribute, MUST be received at the subscriber identical to how they were received by the Channel. The only exception is the [Distributed Tracing Extension Attribute](https://github.com/cloudevents/spec/blob/v0.2/extensions/distributed-tracing.md), which is expected to change as the span id will be altered at every network hop.
+
+Channels MUST attach a bearer token to all outgoing requests, likely in the form of a JWT. This bearer token MUST use an identity associated with the Channel, not the individual Subscription.

docs/spec/channel_060.md

@@ -0,0 +1,137 @@
+# DEPRECATED Channel Spec (version 0.6.0)
+
+## Background
+
+The current system consists of two major APIs, the `Channel` and their `ClusterChannelProvisioner`. Both are living in the `eventing.knative.dev/v1alpha1` API Group.
+
+### The Channel
+
+A `kind: Channel` logically receives events on its input domain and forwards them to its subscribers. Below is a specification for the different parts of the `kind: Channel`.
+
+### The ClusterChannelProvisioner
+
+Describes an abstract configuration of a Source system which produces events or a Channel system that receives and delivers events.
+
+## Channel Spec Parts
+
+### Control Plane
+
+There exists a single Channel CRD. Users and the defaulting webhook are allowed to specify `spec.provisioner`. The only programmatic interactions made with the Channel are via the Channelable duck type, which revolves around the injection of `spec.subscribers`, and its `status.conditions[?(.type == "Ready")].status`.
+
+#### Default Channel setup
+
+The notion of the used default channel is configured in the `default-channel-webhook` Config map, and can be changed to a given `ClusterChannelProvisioner`:
+
+```
+kind: ConfigMap
+metadata:
+  name: default-channel-webhook
+  namespace: knative-eventing
+data:
+  default-channel-config: |
+    clusterdefault:
+      apiversion: eventing.knative.dev/v1alpha1
+      kind: ClusterChannelProvisioner
+      name: in-memory
+    namespacedefaults:
+      some-namespace:
+        apiversion: eventing.knative.dev/v1alpha1
+        kind: ClusterChannelProvisioner
+        name: some-other-provisioner
+```
+
+Example of a channel, backed by the default config (`in-memory` CCP by default):
+
+```
+apiVersion: eventing.knative.dev/v1alpha1
+kind: Channel
+metadata:
+  name: my-channel
+```
+
+#### Setup of a non-default Channel
+
+With the explicit reference to a `spec.provisioner`, a channel is backed by its available `ClusterChannelProvisioner`. Below is an example for a _Kafka Channel_:
+
+```
+apiVersion: eventing.knative.dev/v1alpha1
+kind: Channel
+metadata:
+  name: my-kafka-channel
+spec:
+  provisioner:
+    apiVersion: eventing.knative.dev/v1alpha1
+    kind: ClusterChannelProvisioner
+    name: kafka
+```
+
+#### Broker and Triggers
+
+With the usage of the `knative-eventing-injection: enabled` label a default broker is configured, which uses the default Channel's CCP:
+
+```
+apiVersion: eventing.knative.dev/v1alpha1
+kind: Broker
+metadata:
+  name: default
+```
+
+##### Overriding the channel of a Broker
+
+With the usage of the `spec.channelTemplate`, a broker instance can change its backing Channel:
+
+```
+apiVersion: eventing.knative.dev/v1alpha1
+kind: Broker
+metadata:
+  name: default
+spec:
+  channelTemplate:
+    provisioner:
+      apiVersion: eventing.knative.dev/v1alpha1
+      kind: ClusterChannelProvisioner
+      name: gcp-pubsub
+```
+
+Details about Broker and Trigger can be found [here](../broker/README.md).
+
+### Data Plane
+
+The data plane describes the input and output flow of a _Channel_:
+
+#### Input
+
+The input container must have an HTTP endpoint running on port 80. However, it might be possible to run on a non-standard port, depending on how well calling pieces handle the Addressable contract. It might be possible to support HTTPS. The HTTP request MUST be a POST to the path '/', where anything else is rejected with a `405 Method Not Supported` and `404 Not Found` respectively.
+
+The HTTP body is passed through and all HTTP headers that match an allowed list (e.g. x-b3-*) are passed through (TODO: document them). It is used exclusively in CloudEvents 0.2 HTTP binary content mode.
+
+Currently there is no request validation is performed.  For example:
+ * All conforming CloudEvents must specify a Type, but a Channel will happily pass along a request that does not have a Type.
+ * Channels will send a body of size zero and no headers, essentially a totally blank message.
+
+HTTP status codes that are used:
+ * `202 Accepted` - The request was received and sent to the middleware successfully.
+ * `404 Not Found`:
+    * The request was sent to path other than '/'.
+    * The request was sent to a Channel that the dispatcher did not know about.
+ * `405 Method Not Supported` - The request was not a POST.
+ * `500 Server Error` - All other errors. Such as:
+    * Could not extract the Channel name from the host.
+    * Could not read the entire HTTP body.
+    * Problem sending to the middleware.
+
+   *NOTE:   * There is no authentication or authorization performed. If a request is received, then it is allowed.
+
+
+#### Output
+
+An HTTP POST is made to the URI that is injected into the Channel's spec field. It is normally just HTTP, to a hostname, with a path of '/', because that is what Addressable currently supports. It can be an arbitrary URI. However, it almost certainly needs to be HTTP(S), due to the usage of the `http.Client` package in current implementations.
+
+The request is made by taking the original HTTP body combining it with all the passed through headers. The request is made effectively as a CloudEvents 0.2 HTTP binary content mode request.
+
+Different Channels handle failures in different ways:
+ * gcp-pubsub - Exponential backoff, up to five minutes. Will continue retrying forever.
+ * kafka - Immediate retry, no backoff. Will continue retrying forever.
+    * Due to head of line blocking, nothing else get through this topic's partition.
+ * natss - Immediate retry, no backoff. Will continue retrying forever.
+ * in-memory - No retries. An error message is logged and the event is discarded.