Adding docs/spec shell (v2). by n3wscott · Pull Request #286 · knative/eventing

n3wscott · 2018-07-25T17:01:47Z

replaces #147

…eds more work.

google-prow-robot · 2018-07-25T17:01:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: n3wscott
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: evankanderson

If they are not already assigned, you can assign the PR to them by writing /assign @evankanderson in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

n3wscott · 2018-07-25T17:05:55Z

/assign @evankanderson

evankanderson · 2018-07-25T17:09:15Z

@@ -0,0 +1,32 @@
+// This is the overview graph for object refrences in the control plane.


This graph does not include Flows at all.

I added a Flow, but flow is flexible so it is hard to represent in a single graph

evankanderson · 2018-07-25T17:10:29Z

+
+This directory contains the specification of the Knative Eventing API, which is
+implemented in [`channels.knative.dev`](/pkg/apis/channels/v1alpha1) and
+[`feeds.knative.dev`](/pkg/apis/feeds/v1alpha1) and verified via [the e2e


This doesn't mention Flows at all.

evankanderson · 2018-07-25T17:16:29Z

@@ -0,0 +1,80 @@
+# Resource Groups


Feel free to steal from https://github.com/knative/docs/blob/master/eventing/README.md#architecture if it makes sense.

I linked to it. That doc has the wrong abstraction. We did not abstract the bus, we abstracted the channel.

Or we have the wrong API group names. I could be convinced either way.

evankanderson · 2018-07-25T17:16:42Z

@@ -0,0 +1,80 @@
+# Resource Groups
+
+Knative Eventing API is grouped into _Channels_ and _Feeds_:


No mention of Flows.

evankanderson · 2018-07-25T17:17:05Z

+
+* _Feeds_ bridge the event source into the eventing framework.
+
+* _Flows_ are the higher order abstractions for eventing that use _Channels_


I'd prefer to talk about what Flows gives the user, rather than where it sits in relation to other objects.

I am at a loss for the correct words for this

the desired path from an external Source of events to a destination that will react to the events

?

not strictly "external", e.g. the k8sevents EventSource

I am trying: * _Flows_ abstract the path of an event from a source takes to reach an event consumer.

evankanderson · 2018-07-25T17:19:10Z

+
+## EventType
+
+TODO


Can we fill this out, with at least some justification?

I have pointed it to the Architecture page from docs.

evankanderson · 2018-07-25T17:22:39Z

+
+# Eventing
+
+TODO an overview of how these objects are created and perhaps the persona


Do we need this section in this doc? If so, it seems like the overview should be at the top rather than near the bottom. (I don't quite understand what should be in this section.)

evankanderson · 2018-07-25T17:35:12Z

+
+**Updates to this spec should include a corresponding change to the
+API implementation for [channels](/pkg/apis/channels/v1alpha1) or
+in [feeds](/pkg/apis/feeds/v1alpha1) and [the e2e test](/test/e2e).**


This may be the right place, but it seems like it would be useful to include a section on either "Known issues" with the API and/or patterns that we've agreed upon. In particular, the use of containers/images for extension of Buses and Sources is probably worth highlighting somewhere.

The Parameters/ParametersFrom pattern for passing args to the images is probably also worth documenting.

I think that is a good TODO but I don't have the context to fully author those. I added a todo for you :D

n3wscott

Updated, PTAL. Thanks for the review!

n3wscott · 2018-07-25T18:05:36Z

+
+**Updates to this spec should include a corresponding change to the
+API implementation for [channels](/pkg/apis/channels/v1alpha1) or
+in [feeds](/pkg/apis/feeds/v1alpha1) and [the e2e test](/test/e2e).**


I think that is a good TODO but I don't have the context to fully author those. I added a todo for you :D

n3wscott · 2018-07-25T18:10:46Z

@@ -0,0 +1,32 @@
+// This is the overview graph for object refrences in the control plane.


I added a Flow, but flow is flexible so it is hard to represent in a single graph

n3wscott · 2018-07-25T18:29:03Z

@@ -0,0 +1,80 @@
+# Resource Groups


I linked to it. That doc has the wrong abstraction. We did not abstract the bus, we abstracted the channel.

n3wscott · 2018-07-25T18:54:44Z

+
+* _Feeds_ bridge the event source into the eventing framework.
+
+* _Flows_ are the higher order abstractions for eventing that use _Channels_


I am at a loss for the correct words for this

n3wscott · 2018-07-25T18:55:01Z

+
+## EventType
+
+TODO


I have pointed it to the Architecture page from docs.

evankanderson

A few more comments -- thanks for writing this up and pinging me about actually getting back to the review.

evankanderson · 2018-08-02T20:44:36Z

+
+* [Motivation and goals](motivation.md)
+* [Resource type overview](overview.md)
+<!-- * [Knative Eventing API spec](spec.md) -->


Mark these as TODO?

evankanderson · 2018-08-02T20:47:40Z

+    subgraph cluster_B {
+      Channel
+      Subscription;
+      label = "Bus";


I'm not sure a "Bus" contains a Channel or Subscription in the same way that a Flow "contains" a Feed or Subscription. In the former case, the implementation of the Channel or Subscription is done by the Bus's controller, but in the latter case, the Flow literally describes the spec of the Feed and Subscription objects in some way. It feels more like a "Bus" is referenced by a Channel (and indirectly via Subscription).

Updated the graph to have Channel point to Bus, but had to giveup a little formatting OCD because graphviz does what it wants sometimes.

evankanderson · 2018-08-02T21:03:48Z

@@ -0,0 +1,17 @@
+# Motivation
+
+Knative Eventing implements the common components for an event delivery


I'd turn this around (this talks about what we did, not why):

The goal of Knative Eventing is to define common, composable primitives to enable late-binding event sources and event consumers.

[Why late-binding]

Alternately, you could steal the following principles:

Services are loosely coupled during development and deployed independently on a variety of platforms (Kubernetes, VMs, SaaS or FaaS).

A producer can generate events before a consumer is listening, and a consumer can express an interest in an event or class of events that is not yet being produced.

Services can be connected to create new applications
a. without modifying producer or consumer.
b. with the ability to select a specific subset of events from a particular producer

From there, I think it's fine to reference CloudEvents and Kubernetes event-processing or lack thereof.

evankanderson · 2018-08-02T21:06:29Z

+The primary resources in the Knative Eventing _Feeds_ API are EventSource,
+EventType and Feed:
+
+* **EventSource**, an archetype of an event producer.


This doesn't seem entirely clear; maybe give an example, like github.com or storage.googleapis.com. It might also be worth mentioning whether alternate environments/implementations of the API are separate Sources or a parameter in the Feed. (e.g. do you have different EventSources for each AWS region, or is that a Feed status in some way?)

I added an example to each. See if that helps.

evankanderson · 2018-08-02T21:07:34Z

+
+* **EventSource**, an archetype of an event producer.
+
+* **EventType**, the schema for an event.


It's worth calling out that there may be multiple EventTypes for a single Source. Again, a worked example might be illuminating, even if it is aspirational rather than factual.

You might also want to say something about EventType implying some sort of more-specific schema for the payload (data) of the event.

I think this should go into the spec part that is still TODO. This is suppose to be the overview.

evankanderson · 2018-08-02T21:14:02Z

+The primary resources in the Knative Eventing _Channels_ API are Channel,
+Subscription and Bus:
+
+* **Channel**, a named endpoint which accepts and forwards events.


Again, it might be useful to mention the cardinality here -- a Bus may support many Channels, and a Channel may support multiple Subscriptions.

I think these are details of our choice of implementation?

No, I think these are reasons for these to be different objects (like Pod, ReplicaSet, and Deployment).

evankanderson · 2018-08-02T21:14:52Z

+
+* **Channel**, a named endpoint which accepts and forwards events.
+
+* **Bus**, an implementation of an event delivery mechanism.


I'd highlight that a Bus implements some sort of store-and-forward fanout strategy (whether that is Kafka, GCP PubSub, or in-memory).

Again, I think these are details of our choice of implementation?

I think the role of the Bus is to provide the store-and-forward delivery implementation. Some may do that more durably than others, but a Bus object describes a particular implementation strategy for the store and forward contract.

evankanderson · 2018-08-02T21:15:53Z

+
+* **Bus**, an implementation of an event delivery mechanism.
+
+* **Subscription**, an expressed interest in events to be delivered to a


... events from a channel to be delivered to an HTTP endpoint. (Which might be a Knative service, or any other HTTP implementation.)

evankanderson · 2018-08-02T21:17:45Z

+* **Subscription**, an expressed interest in events to be delivered to a
+  service.
+
+The primary resources in the Knative Eventing _Flows_ API are Flow:


"resource" (no "s")

It might make more sense to say this as The _Flows_ API implements a single resource, Flow:

evankanderson · 2018-08-02T21:19:08Z

+
+The primary resources in the Knative Eventing _Flows_ API are Flow:
+
+* **Flow**, the connection between an event producer to the event consumer.


Can you express that this is a high-level intent which abstracts both Flow and Subscription (as well as most of the other resources)?

https://github.com/knative/serving/blob/master/docs/spec/overview.md#service may have some useful verbiage.

evankanderson

This is looking better, but I'm still concerned that the object model doesn't clearly justify the choices and objects.

Thanks for starting this; I'm happy to add the remaining docs over time, but would like the initial object model to explain the decomposition of work and value added.

/approve

evankanderson · 2018-08-06T15:55:04Z

+
+* **Channel**, a named endpoint which accepts and forwards events.
+
+* **Bus**, an implementation of an event delivery mechanism.


I think the role of the Bus is to provide the store-and-forward delivery implementation. Some may do that more durably than others, but a Bus object describes a particular implementation strategy for the store and forward contract.

evankanderson · 2018-08-06T16:30:45Z

+* **Subscription**, an expressed interest in events from a _Channel_ to be
+  delivered to an HTTP endpoint.
+
+An example _Channels_ setup: A _Channel_ has declared it is implemented by a


I feel like the example here doesn't elucidate much. I'd rather see something like "cluster bus X is implemented via NATS, and has 2 channels: 'pulls' and 'analysis'. 'pulls' has one subscription, and 'analysis' has none at the moment.

evankanderson · 2018-08-06T16:32:39Z

+  the event consumer leveraging _Feed_s and _Channels_.
+
+
+An example _Flows_ setup: The _Flow_ describes the same properties as the _Feed_


Maybe it would be better to give an example of buffering the GitHub notifications if the target is not up yet (to contrast with Feed, which would not).

evankanderson · 2018-08-06T16:33:44Z

+![Object Model](images/overview-reference.png)
+
+See the [Knative Eventing Docs
+Architecture](https://github.com/knative/docs/blob/master/eventing/README.md#architecture)


What does "architecture" mean as a modifier to docs?

knative-prow-robot · 2018-08-06T16:37:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: evankanderson, n3wscott

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [evankanderson]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

n3wscott · 2018-08-07T15:20:50Z

@grantr for lgtm?

grantr · 2018-08-07T17:00:31Z

/lgtm
Thanks for writing this @n3wscott. I think it's sufficient to describe the what for now and add the why in future PRs.

Scott Nichols added 8 commits June 28, 2018 17:02

Adding base spec dir with shell of files.

d9471db

Adding words to Motivation.

18af19e

First crack at an overview, but expanded definitions are needed.

ee428b3

REword based on feedback and copied some of the glossary terms in. Ne…

88d7500

…eds more work.

Adding a simple image for object model.

acbf960

Merge remote-tracking branch 'upstream/master' into spec

8950a8f

Clean up the docs, take in feedback from the PR.

93ee630

fix wording, remove bind.

82a3ba7

google-prow-robot requested review from inlined and vaikas July 25, 2018 17:01

google-prow-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 25, 2018

google-prow-robot assigned evankanderson Jul 25, 2018

evankanderson reviewed Jul 25, 2018

View reviewed changes

Updated based on feedback.

dc6501e

n3wscott commented Jul 25, 2018

View reviewed changes

Scott Nichols added 2 commits July 26, 2018 12:36

Update what a Flow is.

3daacf0

better wording.

a25a237

evankanderson reviewed Aug 2, 2018

View reviewed changes

Clarify the roles of the resources a bit more.

4fd65ee

n3wscott force-pushed the spec branch from b12225c to 7a83f56 Compare August 3, 2018 17:34

Make the a. b. list work.

ff90955

n3wscott force-pushed the spec branch from 7a83f56 to ff90955 Compare August 3, 2018 17:36

evankanderson reviewed Aug 6, 2018

View reviewed changes

knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 6, 2018

knative-prow-robot assigned grantr Aug 7, 2018

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 7, 2018

knative-prow-robot merged commit 291c121 into knative:master Aug 7, 2018

		@@ -0,0 +1,32 @@
		// This is the overview graph for object refrences in the control plane.

		@@ -0,0 +1,80 @@
		# Resource Groups

		Knative Eventing API is grouped into _Channels_ and _Feeds_:


		* _Feeds_ bridge the event source into the eventing framework.

		* _Flows_ are the higher order abstractions for eventing that use _Channels_


		# Eventing

		TODO an overview of how these objects are created and perhaps the persona

		@@ -0,0 +1,17 @@
		# Motivation

		Knative Eventing implements the common components for an event delivery


		* EventSource, an archetype of an event producer.

		* EventType, the schema for an event.


		* Channel, a named endpoint which accepts and forwards events.

		* Bus, an implementation of an event delivery mechanism.


		* Bus, an implementation of an event delivery mechanism.

		* Subscription, an expressed interest in events to be delivered to a


		The primary resources in the Knative Eventing _Flows_ API are Flow:

		* Flow, the connection between an event producer to the event consumer.

		the event consumer leveraging _Feed_s and _Channels_.


		An example _Flows_ setup: The _Flow_ describes the same properties as the _Feed_

Conversation

n3wscott commented Jul 25, 2018

Uh oh!

google-prow-robot commented Jul 25, 2018

Uh oh!

n3wscott commented Jul 25, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n3wscott Jul 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

n3wscott left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evankanderson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

n3wscott Jul 26, 2018 •

edited

Loading