Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
176 commits
Select commit Hold shift + click to select a range
d7ed39b
Initial API outline for Broker.
Harwayne Jan 28, 2019
080f5eb
Metadata is exported
Harwayne Jan 28, 2019
f20a2df
Initial Trigger API.
Harwayne Jan 28, 2019
163474d
Remove old comments.
Harwayne Jan 28, 2019
5f53e05
Register the types.
Harwayne Jan 28, 2019
2f6a03f
Initial work on the Broker controller.
Harwayne Jan 28, 2019
3ee0b8b
Initial work on the Trigger controller.
Harwayne Jan 31, 2019
da9fe36
Initial work on the new model and its corresponding broker controller.
Harwayne Feb 4, 2019
f8cca2e
Initial work on the Trigger controller.
Harwayne Feb 5, 2019
d7ebdcb
Add simple mains.
Harwayne Feb 5, 2019
c5f7a83
Small fixes, still not working.
Harwayne Feb 5, 2019
054c10d
Add the Istio injection annotation.
Harwayne Feb 5, 2019
942b979
Resolve subscriber in the Trigger controller.
Harwayne Feb 5, 2019
afd71af
Standardize on 'Any'.
Harwayne Feb 5, 2019
7a1f69b
Make Broker and Trigger generational
Harwayne Feb 5, 2019
2be56bd
Happy control path.
Harwayne Feb 5, 2019
10acfa3
TargetPort is 8080
Harwayne Feb 5, 2019
2108d83
Watch channels.
Harwayne Feb 5, 2019
6aa7086
Custom service account for filter (needs trigger watch).
Harwayne Feb 5, 2019
3cd236f
Increase logging in the Filter to debug level.
Harwayne Feb 5, 2019
be350a0
Use the default channel provisioner.
Harwayne Feb 6, 2019
0068232
Adding filtering using k8s label selectors
nachocano Feb 6, 2019
b4e2358
Watch namespaces and create a default Broker.
Harwayne Feb 6, 2019
3a9fd94
Merge remote-tracking branch 'broker/broker-new-model' into broker-ne…
nachocano Feb 6, 2019
5ca1b9f
Updating trigger example with filters
nachocano Feb 6, 2019
bd53cda
Broker changes cause the namespace watcher to reconcile.
Harwayne Feb 6, 2019
17889fb
Merge remote-tracking branch 'broker/broker-new-model' into broker-ne…
nachocano Feb 7, 2019
582eb4c
Merge branch 'master' into broker-new-model
Harwayne Feb 7, 2019
4f803a2
Move the Logging package import and format.
Harwayne Feb 7, 2019
eec0f8d
Updating after review comments. Only doing exact header matching.
nachocano Feb 8, 2019
ac3587e
Merge remote-tracking branch 'broker/broker-new-model' into broker-ne…
nachocano Feb 8, 2019
34d42e0
Adding filtering expressions. Currently using LabelSelectors without
nachocano Feb 8, 2019
1788a9b
Merge remote-tracking branch 'upstream/master' into broker-new
nachocano Feb 11, 2019
ad4e6f5
Changes to compile
nachocano Feb 11, 2019
bc519d8
moving filter
nachocano Feb 11, 2019
3f47238
Moving cmds to broker
nachocano Feb 11, 2019
3070157
updating controller
nachocano Feb 11, 2019
0957fda
Moving provider and reconciler to reconciler folder, and merging them…
nachocano Feb 11, 2019
ea53702
Merge branch 'master' into broker-new-model
Harwayne Feb 11, 2019
f0106aa
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 11, 2019
509c8a1
Adding verbs to dispatcher
nachocano Feb 12, 2019
d33308b
Moving back mains to cmd
nachocano Feb 12, 2019
af83baa
Updating config
nachocano Feb 12, 2019
8df699c
Moving resources to reconcilers
nachocano Feb 12, 2019
bd6a8fa
Moving broker folder back where it belongs
nachocano Feb 12, 2019
d2011e6
Merge pull request #5 from nachocano/broker-new
Harwayne Feb 12, 2019
d567412
Merge branch 'broker-new-model' into broker-new
nachocano Feb 12, 2019
3b84c91
Revert "Merge branch 'broker-new-model' into broker-new"
nachocano Feb 12, 2019
1d6c759
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 12, 2019
0e43a7f
Merge remote-tracking branch 'broker/broker-new-model' into filter-exp
nachocano Feb 12, 2019
2df6bdc
Updating headers and headerExpressions to attributes and attributeExp…
nachocano Feb 12, 2019
bfe61c1
Merge branch 'master' into broker-clean-up
Harwayne Feb 12, 2019
22efbe0
Reconcilers notice when mroe things change.
Harwayne Feb 13, 2019
3a3c13c
PR comment.
Harwayne Feb 13, 2019
f834f89
Remove redundant watch.
Harwayne Feb 13, 2019
698a40c
Merge pull request #6 from Harwayne/broker-clean-up
Harwayne Feb 13, 2019
6b817ef
Unit test scaffold.
Harwayne Feb 13, 2019
451b607
Merge pull request #7 from Harwayne/broker-unit-test-scaffold
Harwayne Feb 13, 2019
4097d0a
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 13, 2019
97e642d
Tests for the namespace reconciler.
Harwayne Feb 13, 2019
5e0319f
Merge pull request #8 from Harwayne/ns-test
Harwayne Feb 13, 2019
4bb7fda
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 13, 2019
da32bae
Exact matching for filters without using k8s selectors-based syntax.
nachocano Feb 13, 2019
7bc8b7a
Merge remote-tracking branch 'broker/broker-new-model' into filter-exp
nachocano Feb 13, 2019
8ccfcca
Removing t3 as we don't have set expressions
nachocano Feb 13, 2019
29acf36
Updates after code review
nachocano Feb 13, 2019
e75141c
Attempt to reconcile broker in trigger controller
nachocano Feb 13, 2019
e79b8f7
Broker controller unit tests.
Harwayne Feb 13, 2019
ae665bd
Merge pull request #9 from Harwayne/broker-test
Harwayne Feb 13, 2019
af2f475
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 13, 2019
dbe650a
Merge pull request #4 from nachocano/filter-exp
Harwayne Feb 14, 2019
30f1c74
Deleting and re-creating subscription object as the backing channel
nachocano Feb 14, 2019
f363e02
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 14, 2019
e45d19a
Adding comment
nachocano Feb 14, 2019
1628333
Rename Trigger.Spec.Filter.ExactMatch to Trigger.Spec.Filter.SourceAn…
Harwayne Feb 14, 2019
a6d2650
Merge pull request #11 from Harwayne/filter-test
Harwayne Feb 14, 2019
434bf9d
Merge remote-tracking branch 'broker/broker-new-model' into broker-new
nachocano Feb 14, 2019
cbbd066
Adding event messages
nachocano Feb 14, 2019
c33c569
Using the broker's namespacedNamed as key to the triggers map.
nachocano Feb 14, 2019
66f6f44
Merge pull request #10 from nachocano/broker-new
Harwayne Feb 14, 2019
40ca8da
Adding some tests to trigger
nachocano Feb 14, 2019
650ef47
More UTs
nachocano Feb 15, 2019
d2c1ef8
More UTs
nachocano Feb 15, 2019
ae04007
More UTs
nachocano Feb 15, 2019
6d6c488
Merge pull request #12 from nachocano/broker-new-trigger-ut
Harwayne Feb 15, 2019
9c63dbc
Namespace reconciler automatically creates the Broker Filter's Servic…
Harwayne Feb 15, 2019
3ad193f
Remove no longer needed label.
Harwayne Feb 15, 2019
d18acab
Broker and trigger types UTs
nachocano Feb 15, 2019
2a02ba5
WIP early E2E test
grantr Feb 15, 2019
54cdb3c
Changes after code review. Adding trigger defaults and validation tests
nachocano Feb 19, 2019
11a1734
Cleaner trigger validation
nachocano Feb 19, 2019
d713eeb
Adding dummy tests for broker validation...
nachocano Feb 19, 2019
89cdd58
Merge pull request #14 from nachocano/broker-new-trigger-ut
Harwayne Feb 19, 2019
bd4345a
Merge remote-tracking branch 'grantr/broker-new-model-e2e' into broke…
nachocano Feb 19, 2019
49fd1a0
Compiling and moving things around
nachocano Feb 19, 2019
4cdfd58
Updating test
nachocano Feb 20, 2019
b42d359
More updates
nachocano Feb 20, 2019
29be515
Waiting for potentially multiple contents.
nachocano Feb 20, 2019
1edbf0a
Compiling
nachocano Feb 20, 2019
9c415e6
Fixing compilation
nachocano Feb 20, 2019
ac9b485
Fixing compilation errors. Adding AnnotateNamespace function.
nachocano Feb 20, 2019
2b39168
Adding ns
nachocano Feb 20, 2019
ed9fcbe
Adding logs. Changing to lowercase any otherwise the pod name is invalid
nachocano Feb 20, 2019
4848952
Removing namespace when creating trigger subscriber spec.
nachocano Feb 20, 2019
311e174
Checking if all triggers are ready
nachocano Feb 20, 2019
d361afb
Updated logs
nachocano Feb 20, 2019
d1198a9
Working
nachocano Feb 20, 2019
6bd659c
Adding logs... Still not receiving the events.
nachocano Feb 21, 2019
0d80f9c
More logs
nachocano Feb 21, 2019
cd862ff
Adding build constraint
nachocano Feb 21, 2019
2ea7aa3
Removing unnecessary stuff
nachocano Feb 21, 2019
aa60a8a
Removing ugly structs
nachocano Feb 21, 2019
6471b4f
More logs
nachocano Feb 21, 2019
6bc7f64
Merge pull request #13 from Harwayne/ns-rbac
Harwayne Feb 21, 2019
56ad43c
Merge remote-tracking branch 'broker/broker-new-model' into broker-e2e
nachocano Feb 21, 2019
c4959e2
Removing quotes
nachocano Feb 21, 2019
69ebf18
More logs
nachocano Feb 21, 2019
1dccad8
Adding delay
nachocano Feb 21, 2019
2b67eb8
Listing triggers in receiver when we create it, so that we don't miss
nachocano Feb 22, 2019
a44d6d2
Adding delay to sender pod
nachocano Feb 22, 2019
41ab0a8
Removing withDelay method and just sleep for a while
nachocano Feb 22, 2019
6a832ea
Improve log...
nachocano Feb 22, 2019
446022a
Updates after code review.
nachocano Feb 23, 2019
7f24f14
Adding some more logs and trailing dots.
nachocano Feb 23, 2019
2a3eb19
Merge branch 'master' into broker-controller-runtime
Harwayne Feb 25, 2019
9debaf2
Switch import order.
Harwayne Feb 25, 2019
abf3427
Merge pull request #16 from Harwayne/broker-controller-runtime
Harwayne Feb 25, 2019
a83ca82
Updating comments.
nachocano Feb 25, 2019
fda6550
Updating comments.
nachocano Feb 25, 2019
3ee6343
Merge remote-tracking branch 'broker/broker-new-model' into broker-ge…
nachocano Feb 25, 2019
2aae7eb
Replace the bad errgroup usage with the runnableServer.
Harwayne Feb 25, 2019
e15431d
Merge pull request #18 from Harwayne/codegen
Harwayne Feb 25, 2019
5f201fa
Namespace scoped the Broker Filter's client.
Harwayne Feb 25, 2019
ba726b3
Fix unit tests.
Harwayne Feb 25, 2019
e7ff5b2
Fix yaml
Harwayne Feb 25, 2019
d640c7a
Merge pull request #19 from Harwayne/namespace-rolebinding
Harwayne Feb 25, 2019
dc91427
Merge remote-tracking branch 'broker/broker-new-model' into broker-e2e
nachocano Feb 25, 2019
0ea0402
Merge branch 'broker-e2e' of github.com:nachocano/eventing into broke…
nachocano Feb 25, 2019
dcb8774
Setting source to source not type.
nachocano Feb 25, 2019
df88eab
Merge remote-tracking branch 'broker/broker-new-model' into broker-ge…
nachocano Feb 25, 2019
86f7330
Merge pull request #15 from nachocano/broker-e2e
Harwayne Feb 26, 2019
b6fe6bd
Merge pull request #17 from nachocano/broker-gen-pass
Harwayne Feb 26, 2019
71c5250
Switch from annotating the namespace to labeling it, to match Istio.
Harwayne Feb 26, 2019
b12da7e
Merge pull request #20 from Harwayne/ns-eventing-label
Harwayne Feb 26, 2019
8d6df67
General clean up.
Harwayne Feb 27, 2019
c27d283
Merge pull request #21 from Harwayne/broker-cleanup
Harwayne Feb 27, 2019
d9a607c
Merge branch 'master' into broker-merge-master
Harwayne Feb 27, 2019
77a69f9
Merge pull request #22 from Harwayne/broker-merge-master
Harwayne Feb 27, 2019
fef7628
Create an ingress channel for Brokers.
Harwayne Feb 28, 2019
6dd814f
adding cloudevents/sdk-go
Harwayne Mar 2, 2019
f8d606d
Fix unit tests.
Harwayne Mar 4, 2019
a3b7728
Delete old recevier.
Harwayne Mar 4, 2019
80005e8
Change assumption about response to only being a CloudEvent if it can…
Harwayne Mar 4, 2019
bf335bc
Merge branch 'master' into broker-ingress-cloudevents
Harwayne Mar 18, 2019
a106553
Clean up from the merge.
Harwayne Mar 18, 2019
bc05fdf
More merge clean up.
Harwayne Mar 18, 2019
134198d
Update cloudevents/sdk-go to 0.4.0.
Harwayne Mar 18, 2019
0bdac35
Move to the newer, idiomatic CloudEvents library useage.
Harwayne Mar 18, 2019
2e50060
tctx.URI is actually just the path...
Harwayne Mar 18, 2019
f27c63a
Partial update to the unit tests.
Harwayne Mar 18, 2019
747e290
Unit tests working.
Harwayne Mar 19, 2019
2b9a2e0
Merge branch 'master' into broker-ingress-cloudevents
Harwayne Mar 19, 2019
f3121ae
Increase broker_types.go code coverage.
Harwayne Mar 19, 2019
50aa925
hack/update-deps.sh
Harwayne Mar 19, 2019
c63f23e
Increase receiver.go code coverage.
Harwayne Mar 19, 2019
ad9bf7c
Increase trigger.go code coverage.
Harwayne Mar 19, 2019
b314abd
Constraint, not override.
Harwayne Mar 19, 2019
a9d7111
broker.go unit tests.
Harwayne Mar 19, 2019
bd47d55
Merge branch 'master' into broker-ingress-cloudevents
Harwayne Mar 19, 2019
36ef2a4
Better receiver shutdown logic.
Harwayne Mar 20, 2019
c66f21e
cloudevents/sdk-go 0.4.1
Harwayne Mar 20, 2019
f94d5a8
Forward tracing headers.
Harwayne Mar 20, 2019
a6f6bd4
Update comment.
Harwayne Mar 20, 2019
44e4575
Merge branch 'master' into keep-tracing
Harwayne Mar 20, 2019
591fd60
Relearning why message_dispatcher was written the way it was.
Harwayne Mar 20, 2019
f6ea5ac
Add a TODO for where to make the change for response tracing too.
Harwayne Mar 20, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ required = [
name = "github.com/nats-io/nats-streaming-server"
version = "0.11.0"

# The latest release as of March 12, 2019.
# The latest release as of March 20, 2019.
[[constraint]]
name = "github.com/cloudevents/sdk-go"
version = "=0.4.0"
version = "=0.4.1"
Comment thread
Harwayne marked this conversation as resolved.
52 changes: 50 additions & 2 deletions pkg/broker/receiver.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"errors"
"net/http"
"net/url"
"strings"
"time"

"github.com/cloudevents/sdk-go/pkg/cloudevents"
Expand All @@ -31,6 +32,7 @@ import (
"github.com/knative/eventing/pkg/provisioners"
"go.uber.org/zap"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/sets"
"sigs.k8s.io/controller-runtime/pkg/client"
)

Expand All @@ -40,6 +42,22 @@ const (
writeTimeout = 1 * time.Minute
)

var (
// These MUST be lowercase strings, as they will be compared against lowercase strings.
forwardHeaders = sets.NewString(
// tracing
"x-request-id",
)
// These MUST be lowercase strings, as they will be compared against lowercase strings.
forwardPrefixes = []string{
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use sets or an array, not both? why both?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use a set for the first one so that I can easily do forwardHeaders.Has(n). The second, I iterate over, so slice and set are the same to me.

// knative
"knative-",
// tracing
"x-b3-",
"x-ot-",
}
)

// Receiver parses Cloud Events, determines if they pass a filter, and sends them to a subscriber.
type Receiver struct {
logger *zap.Logger
Expand Down Expand Up @@ -142,18 +160,22 @@ func (r *Receiver) serveHTTP(ctx context.Context, event cloudevents.Event, resp

r.logger.Debug("Received message", zap.Any("triggerRef", triggerRef))

responseEvent, err := r.sendEvent(ctx, triggerRef, &event)
responseEvent, err := r.sendEvent(ctx, tctx, triggerRef, &event)
if err != nil {
r.logger.Error("Error sending the event", zap.Error(err))
return err
}
resp.Status = http.StatusAccepted
resp.Event = responseEvent

// TODO Add filtered headers (mostly tracing) to the response. We are waiting for CloudEvents
// SDK to allow this.

return nil
}

// sendEvent sends an event to a subscriber if the trigger filter passes.
func (r *Receiver) sendEvent(ctx context.Context, trigger provisioners.ChannelReference, event *cloudevents.Event) (*cloudevents.Event, error) {
func (r *Receiver) sendEvent(ctx context.Context, tctx cehttp.TransportContext, trigger provisioners.ChannelReference, event *cloudevents.Event) (*cloudevents.Event, error) {
t, err := r.getTrigger(ctx, trigger)
if err != nil {
r.logger.Info("Unable to get the Trigger", zap.Error(err), zap.Any("triggerRef", trigger))
Expand All @@ -179,6 +201,7 @@ func (r *Receiver) sendEvent(ctx context.Context, trigger provisioners.ChannelRe
}

sendingCtx := cecontext.WithTarget(ctx, subscriberURI.String())
sendingCtx = addFilteredHeaders(sendingCtx, tctx)
return r.ceHttp.Send(sendingCtx, *event)
}

Expand Down Expand Up @@ -214,3 +237,28 @@ func (r *Receiver) shouldSendMessage(ts *eventingv1alpha1.TriggerSpec, event *cl
}
return true
}

func addFilteredHeaders(ctx context.Context, tctx cehttp.TransportContext) context.Context {
// Helper function that adds the header name and all its values.
addHeader := func(ctx context.Context, n string, v []string) context.Context {
for _, iv := range v {
ctx = cehttp.ContextWithHeader(ctx, n, iv)
}
return ctx
}

for n, v := range tctx.Header {
lower := strings.ToLower(n)
if forwardHeaders.Has(lower) {
ctx = addHeader(ctx, n, v)
continue
}
for _, prefix := range forwardPrefixes {
if strings.HasPrefix(lower, prefix) {
ctx = addHeader(ctx, n, v)
break
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you break here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this will only add one header?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks out of the 'range over forwardPrefixes' for loop, not the 'range over all headers from the context' for loop.

}
}
}
return ctx
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading