Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
14a4842
Initial work on the GCP PubSub Channel Controller.
Harwayne Nov 12, 2018
f99bf21
Switch to the more standard model, a single global dispatcher.
Harwayne Nov 13, 2018
72bb3eb
Args are now defaulted on the controller, removed the ability to cust…
Harwayne Nov 13, 2018
b3b9275
YAML and scaffold for the dispatcher.
Harwayne Nov 13, 2018
03c69e6
Created the 'receive' side of the dispatcher. The dispatch side is to…
Harwayne Nov 14, 2018
c1d1074
Initial work on the dispatcher.
Harwayne Nov 14, 2018
ac685a2
More work on the dispatcher.
Harwayne Nov 14, 2018
a6ff217
Dispatcher mostly working.
Harwayne Nov 14, 2018
8f5756c
Handle the reconcileChan
Harwayne Nov 14, 2018
7660fe3
Merge branch 'master' into pb-channel
Harwayne Nov 14, 2018
43d6e06
Minor improvements.
Harwayne Nov 14, 2018
7bb74b0
Touch ups.
Harwayne Nov 15, 2018
9032baf
Merge branch 'master' into pb-channel
Harwayne Nov 15, 2018
85c3aca
Rename Run->Start.
Harwayne Nov 15, 2018
08e0320
Fake PubSub.
Harwayne Nov 15, 2018
7acb792
Util unit tests.
Harwayne Nov 16, 2018
c05f46b
Unit tests for the channel controller.
Harwayne Nov 16, 2018
c32b5ab
Beginning unit tests for dispatcher.
Harwayne Nov 16, 2018
667ba1a
update-deps
Harwayne Nov 16, 2018
9542a6d
Unit tests for dispatcher/reconcile.go
Harwayne Nov 19, 2018
6901012
Unit test dispatcher/receiver.
Harwayne Nov 19, 2018
abaa872
Add a README.
Harwayne Nov 19, 2018
ba2e849
DeepCopy the sub
Harwayne Nov 19, 2018
5e0ce22
Merge branch 'master' into pb-channel
Harwayne Nov 20, 2018
e67a304
Add the ServiceEntry to the Channel's config.
Harwayne Nov 20, 2018
4cff962
Typo
Harwayne Nov 20, 2018
5b77a10
PR comments
Harwayne Nov 28, 2018
d59873d
Merge branch 'master' into pb-channel
Harwayne Nov 28, 2018
3906810
buses -> provisioners
Harwayne Nov 28, 2018
0b73e4c
Merge remote-tracking branch 'origin/pb-channel' into pb-channel
Harwayne Nov 28, 2018
15784c9
Switch to the common utils.
Harwayne Nov 29, 2018
864c20d
Add VirtualService.Update RBAC now that VirtualServices are updated i…
Harwayne Nov 29, 2018
2e8bf1d
PR comments
Harwayne Nov 30, 2018
b2d0d9f
Move the contoller packages inside the controller folder.
Harwayne Nov 30, 2018
5a38f65
Merge branch 'master' into pb-channel
Harwayne Dec 3, 2018
a3ea34d
PR comments (mostly log levels).
Harwayne Dec 3, 2018
49f47a3
Desugar to get the fields right.
Harwayne Dec 3, 2018
74a1b6f
Switch service name, to match #626.
Harwayne Dec 3, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 106 additions & 8 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

91 changes: 91 additions & 0 deletions config/provisioners/gcppubsub/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# GCP PubSub Channels

GCP PubSub channels are production-quality Channels that are backed by
[GCP PubSub](https://cloud.google.com/pubsub/).

They offer:
* Persistence
- If the Channel's Pod goes down, all events already ACKed by the Channel will persist and be
retransmitted when the Pod restarts.
* Redelivery attempts
- If downstream rejects an event, that request is attempted again.
Comment thread
Harwayne marked this conversation as resolved.

They do not offer:
* Ordering guarantees
- Events seen downstream may not occur in the same order they were inserted into the Channel.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this particular formatting of a list of titles plus descriptions tough to take in at a glance. I prefer something more like this:

  • Bold title. Normal description, all in a single list item.
  • Another bold title. Another description.


### Deployment Steps

#### Prerequisites

1. Create a [Google Cloud Project](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
1. Enable the 'Cloud Pub/Sub API' on that project.

```shell
gcloud services enable pubsub.googleapis.com
```

1. Create a GCP [Service Account](https://console.cloud.google.com/iam-admin/serviceaccounts/project).
1. Determine the Service Account to use, or create a new one.
1. Give that Service Account the 'Pub/Sub Editor' role on your GCP project.
1. Download a new JSON private key for that Service Account.
1. Create a secret for the downloaded key:

```shell
kubectl -n knative-sources create secret generic gcppubsub-channel-key --from-file=key.json=PATH_TO_KEY_FILE.json
```

1. Setup [Knative Eventing](../../../DEVELOPMENT.md).
Comment thread
Harwayne marked this conversation as resolved.

#### Deployment

1. Set the shell variable with the correct value:

```shell
export GCP_PROJECT=REPLACE_ME
```

1. Apply `gcppubsub.yaml`.

```shell
sed "s/REPLACE_WITH_GCP_PROJECT/$GCP_PROJECT/" config/provisioners/gcppubsub/gcppubsub.yaml | ko apply -f -
```

1. Create Channels that reference the `gcp-pubsub` Channel.

```yaml
apiVersion: eventing.knative.dev/v1alpha1
kind: Channel
metadata:
name: foo
spec:
provisioner:
apiVersion: eventing.knative.dev/v1alpha1
kind: ClusterChannelProvisioner
name: gcp-pubsub
```

### Components

The major components are:
* [Channel Controller](../../../pkg/provisioners/gcppubsub/controller)
- [ClusterChannelProvisioner Controller](../../../pkg/provisioners/gcppubsub/clusterchannelprovisioner)
- [Channel Controller](../../../pkg/provisioners/gcppubsub/channel)
* [Channel Dispatcher](../../../pkg/provisioners/gcppubsub/dispatcher/cmd)
- [Dispatcher](../../../pkg/provisioners/gcppubsub/dispatcher/dispatcher)
- [Receiver](../../../pkg/provisioners/gcppubsub/dispatcher/receiver)

The `Channel Controller` controls all the Kubernetes resources and creates `Topic`s and
`Subscription`s in GCP PubSub. It runs in the Deployment:

```shell
kubectl get deployment -n knative-eventing gcp-pubsub-channel-controller
```

The `Channel Dispatcher` handles all the data plane portions of the `Channel`. It receives events
from the cluster and writes them to GCP PubSub `Topic`s. It also polls `Subscriptions`s and sends
events back into the cluster. It runs in the Deployment:

```shell
kubectl get deployment -n knative-eventing gcp-pubsub-channel-dispatcher
```
Loading