Brokers persist messages unnecessarily

[grantr]

Problem
The current design of the broker delivers all messages to an internal channel. This is intended to allow stateless publishers to transfer responsibility for delivering the message to the Broker.

Several sources (Kafka, Pubsub, GitHub, etc), are capable of handling their own persistence, buffering, and retry. Using a Broker with these sources incurs unnecessary resource usage (and cost) because messages are persisted in the source and in the Broker.

The Broker should allow operators to leverage the sources' capabilities to eliminate unnecessary resource usage.

Persona:
Operator

Exit Criteria
A message originating from a Kafka or Pubsub topic is delivered without intermediate persistence.

Time Estimate (optional):

Additional context (optional)
Related to #508 and #2050

[matzew]

This would mean we introduce some level of "guarantee" on the sources? Based on configuration and guarantees, "double persistence" would be off ?

[grantr]

/area performance

[grantr]

/milestone clear

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.