[Broker/Trigger] Make it easier to consume events directly from specific sources

[lionelvillard]

Problem
The current solution involves setting the right attributes on both the source and the triggers. For example:

apiVersion: sources.knative.dev/v1alpha2
kind: PingSource
metadata:
  name: ping-source-1
spec:
  ceOverrides:
    extensions:
      sourceid: ping-source-1
  schedule: "*/1 * * * *"
  jsonData: '{"message": "Hello world! from 1"}'
  sink:
    ref:
      apiVersion: eventing.knative.dev/v1
      kind: Broker
      name: broker

apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: trigger-1
spec:
  broker: broker
  filter:
    attributes:
      sourceid: ping-source-1
  subscriber:
    ref:
      apiVersion: v1
      kind: Service
      name: event-display-1

While this solution allows for a trigger to specify that they want events from a specific source, it is quite brittle as there is no way to statically check that the CE extension attribute sourceid matches.

This is a common scenario and Knative Eventing should have a good support for it.

Persona:
Which persona is this feature for?

Exit Criteria
A measurable (binary) test that would indicate that the problem has been resolved.

Time Estimate (optional):
How many developer-days do you think this may take to resolve?
1-2 days?

Additional context (optional)
Add any other context about the feature request here.

If you really want: #1381

Possible solutions:

• From 1381: well-behaved source populate sourceid to be metadata.uid. Trigger supports refing sources.
• "brokered direct event delivery": the event source delivers events via a broker

[vaikas]

Also as discussed in the triage, how does this fit in with trigger-dependency-annotation.

[lionelvillard]

@vaikas is the trigger-dependency-annotation feature documented somewhere? Or at least an example?

[lionelvillard]

The knative.dev/dependency annotation solves only half of this problem: it marks a trigger ready only when the dependency is ready.

The second half is about filtering out events not coming from the dependency.

The example above shows how this can be done when the dependency is a source. All what we really need to make it less brittle is to let Knative managing the Trigger filter and the ceOverrides.extensions.sourceid fields for us. Since we only want to enable this for some triggers, we need an annotation enabling filtering to receive events coming from the dependency only. Let's call it eventing.knative.dev/filter with two possible values, dependency or spec. When set to dependency, the sugar controller (which seems to the right place to implement this logic) manages the 2 fields mentioned above.

When the dependency is not a source, the eventing validation webhook rejects the trigger when eventing.knative.dev/filter is set.

The example above becomes:

apiVersion: sources.knative.dev/v1alpha2
kind: PingSource
metadata:
  name: ping-source-1
spec:
  schedule: "*/1 * * * *"
  jsonData: '{"message": "Hello world! from 1"}'
  sink:
    ref:
      apiVersion: eventing.knative.dev/v1
      kind: Broker
      name: broker

apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: trigger-1
  annotations:
    knative.dev/dependency: '{"kind":"PingSource", "name":"ping-source-1","apiVersion":"sources.knative.dev/v1alpha2"}'
    eventing.knative.dev/filter: dependency
spec:
  broker: broker
  subscriber:
    ref:
      apiVersion: v1
      kind: Service
      name: event-display-1

@vaikas @n3wscott @dug Comments?

[lionelvillard]

Pinging the client team as they might have looked into this scenario. @rhuss @maximilien

[rhuss]

When the dependency is not a source, the eventing validation webhook rejects the trigger when eventing.knative.dev/filter is set.

How can you check if a dependency is a "source" in the case the dependency is not available yet ? Inspecting the CRD for a 'sink' field ? or base it on naming conventions ?

[rhuss]

If this backend feature would be available, the client could offer an option --source to kn trigger create, which would add those annotations, using the same naming conventions for the value as we do for a --sink specification.

[lionelvillard]

How can you check if a dependency is a "source" in the case the dependency is not available yet ? Inspecting the CRD for a 'sink' field ? or base it on naming conventions ?

When the dependency is not available, the trigger is marked as not ready, reason DependencyDoesNotExist. I should have read my comment before answering, sorry about that.

That's a good point. I don't have a better answer right now to just say that the annotation is then just ignored.