Implement Advanced Filtering proposal#1152
Closed
grantr wants to merge 10 commits into
Closed
Conversation
The Attributes filter is a map of attribute keys to values to compare for equality with the corresponding event context attributes, including extensions. This replaces the SourceAndType filter which is now deprecated. The Expression filter is a string expression that is evaluated by the CEL runtime for each event's context and data. Only one filter can be specified per Trigger. The Trigger will be rejected as invalid if multiple filters are specified. If no filters are specified, that's currently interpreted as a filter matching everything.
Add the CEL runtime to filter CEL expressions. CEL expressions are compiled into programs and cached in an LRU cache by expression string. Currently the cache size is fixed at 100 entries, though that should eventually be based on the number of Triggers. Expressions are inspected to determine if they rely on parsing event data. Event data is not parsed when the expression doesn't need it. The SourceAndType and Attributes filters are implemented by translation to CEL expression.
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: grantr The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This is an error today, but it may not be in the future, depending on how strict users want to be with types.
The receiver now records an overall event received count per trigger, with accept/reject result tags, and a filtering time histogram with pass/fail/error result tags.
Same setup as the broker ingress pod: OpenCensus Prometheus exporter listening on :9090.
Contributor
Author
|
The receiver now instruments filtering. This is ready for review. |
|
The following is the coverage report on pkg/.
|
Contributor
Author
|
This is on hold for now; see #1047 (comment). |
Contributor
Author
|
Attributes filters made it in via #1643. |
matzew
added a commit
to matzew/eventing
that referenced
this pull request
Apr 1, 2021
…tive#1152) Co-authored-by: Yi Zhang <cathyzhyi@google.com> Co-authored-by: Knative Prow Robot <knative-prow-robot@google.com> Co-authored-by: Yi Zhang <cathyzhyi@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #1047 at 76554b8.
Proposed Changes
attributesfilter to the Trigger spec accepting a map of attribute keys and values to compare to the event context attributes, including extensions. ReplacessourceAndType.expressionfilter to the Trigger spec accepting a CEL expression to be evaluated with respect to the event context and data.sourceAndTypefilter. This filter may be removed or translated into anattributesfilter in a future release.Details
Only one filter can be specified per Trigger. The Trigger will be rejected as invalid if multiple filters are specified. If no filters are specified, that's currently interpreted as a filter matching everything.
CEL expressions are compiled into programs and cached in an LRU cache by expression string. Currently the cache size is fixed at 100 entries, though that should eventually be based on the number of Triggers.
Expressions are inspected to determine if they rely on parsing event data. Event data is not parsed when the expression doesn't need it.
The SourceAndType and Attributes filters are implemented by translation to CEL expression.
Attribute filter keys containing special characters such as
.are supported.Attribute filters do not support comparing non-string values. Attribute filters containing non-string values will be TODO. Non-string attribute values compared to attribute filters will generate an evaluation error.
Evaluation errors cause events to be rejected.
Remaining work
Currently evaluation errors are only surfaced in dispatcher logs. This should be surfaced in a form more visible to the user.
Instrumentation is missing and should be added before release.Release Note
/hold
Holding for Docs PR and #1047 merge.
Fixes #930.