[BUG] Channel and Subscription Readiness

[Harwayne]

Describe the bug
Creating a Channel and adding a new Subscription to that Channel leave both the Channel and Subscription Ready=true, before events sent to the Channel will be sent to the Subscriber.

Expected behavior
If both the Channel and Subscription are Ready=true, then the control plane should expect any event sent to the Channel will be sent to the Subscriber. Data plane readiness may lag slightly, but should also be minimized.

To Reproduce

1. Create a Channel.
2. Wait for it to be Ready=true.
3. Create a Subscription on that Channel.
4. Observe that the Channel and Subscription are both Ready=true before the Channel has a chance to notice and reconcile the new spec.subscriber entry.

Knative release version
0.5.

Additional context
This was found in #1076 (comment).

[Harwayne]

Proposed Solution
After talking with @grantr, we propose the following:

• Channel readiness represents that the Channel is ready to receive events. It makes no guarantees about which subscribers will receive those events.
• Subscription readiness represents that events sent to the Channel will be sent to the subscriber.

But this introduces something odd, the Subscription controller needs to wait for some signal from the Channel controller that the Subscription is ready. As of today we don't have a good way to send this signal. Some possibilities are:

• Channel controller directly sets a status field/condition on the Subscription object.
  • This feels like we will run into race conditions. It also is odd to have multiple controllers set the status of a single object.
• Channel adds a new status field that contains the status of every injected subscriber. These per subscriber statuses will need to include something like the observed generation, so the Subscription controller can differentiate between 'this is correctly synced now' vs 'this is correctly synced with an older subscriber'. Something like:

type ChannelStatus struct {
  Subscribers []Subscriber `json:"subscribers,omitempty"`
  // All the already existing fields.
}
type Subscriber struct {
  UID types.UID `json:"uid,omitempty"`
  ObservedGeneration int64 `json:"observedGeneration,omitempty"`
  Ready corev1.ConditionStatus `json:"ready,omitempty"
  Message string `json:"message,omitempty"
}

[n3wscott]

What should the channel do for events when half their subscribers are ready? Are they ready?

[Harwayne]

What should the channel do for events when half their subscribers are ready? Are they ready?

The proposal says yes, the Channel is ready. The Subscriptions that the Channel has not yet said are ready, would not be ready.

This is a horribly mangled version of what the YAML would look like.

Channel.Status:
  Ready: true
  Subscribers:
    - UID: 123
      Ready: true
    - UID: 456
      Ready: false
    - UID: 789
      Ready: false
---
Subscription(uid = 123).status:
  Ready: true
---
Subscription(uid = 456).status:
  Ready: false
---
Subscription(uid = 456).status:
  Ready: false

[akashrv]

/unkind bug

[akashrv]

/remove kind/bug

[akashrv]

/kind feature-request

[akashrv]

/remove-kind bug

[akashrv]

/milestone v0.7.0

[akashrv]

/assign @akashrv