diff --git a/docs/eventing/samples/README.md b/docs/eventing/samples/README.md deleted file mode 100644 index de8ae6360a4..00000000000 --- a/docs/eventing/samples/README.md +++ /dev/null @@ -1,6 +0,0 @@ - -Use the following code samples to help you understand the various use cases for -Knative Eventing and Event Sources. -[Learn more about Knative Eventing and Eventing Sources](../README.md). - -[**See all Knative code samples**](../../samples.md) diff --git a/docs/eventing/samples/_index.md b/docs/eventing/samples/_index.md index 2d58899ef44..1a5edfd5806 100644 --- a/docs/eventing/samples/_index.md +++ b/docs/eventing/samples/_index.md @@ -6,4 +6,8 @@ type: "docs" showlandingtoc: "true" --- -{{% readfile file="README.md" %}} +Use the following code samples to help you understand the various use cases for +Knative Eventing and Event Sources. +[Learn more about Knative Eventing and Eventing Sources](../README.md). + +[**See all Knative code samples**](../../samples.md) diff --git a/docs/eventing/samples/cloud-audit-logs-source/README.md b/docs/eventing/samples/cloud-audit-logs-source/README.md deleted file mode 100644 index d016cf9d7ca..00000000000 --- a/docs/eventing/samples/cloud-audit-logs-source/README.md +++ /dev/null @@ -1 +0,0 @@ -Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudauditlogssource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-audit-logs-source/index.md b/docs/eventing/samples/cloud-audit-logs-source/index.md index 044b26d85e8..756c1c120d4 100644 --- a/docs/eventing/samples/cloud-audit-logs-source/index.md +++ b/docs/eventing/samples/cloud-audit-logs-source/index.md @@ -5,4 +5,4 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudauditlogssource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-pubsub-source/README.md b/docs/eventing/samples/cloud-pubsub-source/README.md deleted file mode 100644 index 471c3d56c6e..00000000000 --- a/docs/eventing/samples/cloud-pubsub-source/README.md +++ /dev/null @@ -1 +0,0 @@ -Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudpubsubsource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-pubsub-source/index.md b/docs/eventing/samples/cloud-pubsub-source/index.md index d0806a4667b..c95f5e5f4bd 100644 --- a/docs/eventing/samples/cloud-pubsub-source/index.md +++ b/docs/eventing/samples/cloud-pubsub-source/index.md @@ -7,4 +7,4 @@ aliases: - /docs/eventing/samples/gcp-pubsub-source/README.md --- -{{% readfile file="README.md" %}} +Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudpubsubsource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-scheduler-source/README.md b/docs/eventing/samples/cloud-scheduler-source/README.md deleted file mode 100644 index 4adfe737d2e..00000000000 --- a/docs/eventing/samples/cloud-scheduler-source/README.md +++ /dev/null @@ -1 +0,0 @@ -Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudschedulersource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-scheduler-source/index.md b/docs/eventing/samples/cloud-scheduler-source/index.md index 5d44a69ef9c..cf0096ea6c6 100644 --- a/docs/eventing/samples/cloud-scheduler-source/index.md +++ b/docs/eventing/samples/cloud-scheduler-source/index.md @@ -5,4 +5,4 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudschedulersource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-storage-source/README.md b/docs/eventing/samples/cloud-storage-source/README.md deleted file mode 100644 index 15a524b5a5d..00000000000 --- a/docs/eventing/samples/cloud-storage-source/README.md +++ /dev/null @@ -1 +0,0 @@ -Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudstoragesource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/cloud-storage-source/index.md b/docs/eventing/samples/cloud-storage-source/index.md index 5e6dbe4c8fe..f69c977e13e 100644 --- a/docs/eventing/samples/cloud-storage-source/index.md +++ b/docs/eventing/samples/cloud-storage-source/index.md @@ -5,4 +5,4 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +Please refer to the [example](https://github.com/google/knative-gcp/blob/master/docs/examples/cloudstoragesource/README.md) in knative-gcp. diff --git a/docs/eventing/samples/container-source/README.md b/docs/eventing/samples/container-source/README.md deleted file mode 100644 index 2efa15d9ff4..00000000000 --- a/docs/eventing/samples/container-source/README.md +++ /dev/null @@ -1,171 +0,0 @@ -ContainerSource will start a container image which will generate events under -certain situations and send messages to a sink URI. It also can be an easy way -to support your own event sources in Knative. This guide shows how to configure -ContainerSource as an event source for functions and summarizes guidelines for -creating your own event source as a ContainerSource. - -## Create a heartbeats ContainerSource - -### Prerequisites - -1. Setup [Knative Serving](../../../serving). -1. Setup [Knative Eventing and Sources](../../../eventing). - -### Prepare the heartbeats image - -Knative [event-sources](https://github.com/knative/eventing-contrib) has a -sample of heartbeats event source. You could clone the source code by - -``` -git clone -b "{{< branch >}}" https://github.com/knative/eventing-contrib.git -``` - -And then build a heartbeats image and publish to your image repo with - -``` -ko publish knative.dev/eventing-contrib/cmd/heartbeats -``` - -**Note**: `ko publish` requires: - -- [`KO_DOCKER_REPO`](https://github.com/knative/serving/blob/main/DEVELOPMENT.md#environment-setup) - to be set. (e.g. `gcr.io/[gcloud-project]` or `docker.io/`) -- you to be authenticated with your `KO_DOCKER_REPO` -- [`docker`](https://docs.docker.com/install/) to be installed - -### Create a Knative Service - -In order to verify `ContainerSource` is working, we will create a Event Display -Service that dumps incoming messages to its log. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: event-display -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Use following command to create the service from `service.yaml`: - -```shell -kubectl apply --filename service.yaml -``` - -The status of the created service can be seen using: - -```shell -kubectl get ksvc - -NAME URL LATESTCREATED LATESTREADY READY REASON -event-display http://event-display.default.1.2.3.4.xip.io event-display-gqjbw event-display-gqjbw True -``` - -### Create a ContainerSource using the heartbeats image - -In order to run the heartbeats container as an event source, you have to create -a concrete ContainerSource with specific arguments and environment settings. Be -sure to replace `heartbeats_image_uri` with a valid uri for your heartbeats -image in your image repo in [heartbeats-source.yaml](./heartbeats-source.yaml) -file. Note that arguments and environment variables are set and will be passed -to the container. - -```yaml -apiVersion: sources.knative.dev/v1 -kind: ContainerSource -metadata: - name: test-heartbeats -spec: - template: - spec: - containers: - - image: - name: heartbeats - args: - - --period=1 - env: - - name: POD_NAME - value: "mypod" - - name: POD_NAMESPACE - value: "event-test" - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display -``` - -Use the following command to create the event source from -`heartbeats-source.yaml`: - -```shell -kubectl apply --filename heartbeats-source.yaml -``` - -### Verify - -We will verify that the message was sent to the Knative eventing system by -looking at event-display service logs. - -```shell -kubectl logs -l serving.knative.dev/service=event-display -c user-container --since=10m -``` - -You should see log lines showing the request headers and body of the event -message sent by the heartbeats source to the display function: - -``` -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.eventing.samples.heartbeat - source: https://knative.dev/eventing-contrib/cmd/heartbeats/#event-test/mypod - id: 2b72d7bf-c38f-4a98-a433-608fbcdd2596 - time: 2019-10-18T15:23:20.809775386Z - contenttype: application/json -Extensions, - beats: true - heart: yes - the: 42 -Data, - { - "id": 2, - "label": "" - } -``` - -## Create a new event source using ContainerSource - -In order to create a new event source using ContainerSource, you will create a -container image at first, and then create a ContainerSource with the image uri -and specify the values of parameters. - -### Develop, build and publish a container image - -The container image can be developed with any language, build and publish with -any tools you like. Here are some basic guidelines: - -- The container image must have a `main` method to start with. -- The `main` method will accept parameters from arguments and environment - variables. -- Two environments variables will be injected by the `ContainerSource` controller, -`K_SINK` and `K_CE_OVERRIDES`, resolved from `spec.sink` and `spec.ceOverrides` respectively. -- The event messages shall be sent to the sink URI specified in `K_SINK`. The message can be any - format. - [CloudEvents](https://github.com/cloudevents/spec/blob/master/spec.md#design-goals) - format is recommended. - -[heartbeats](https://github.com/knative/eventing-contrib/blob/main/cmd/heartbeats/main.go) -event source is a sample for your reference. - -### Create the ContainerSource using this container image - -When the container image is ready, a YAML file will be used to create a concrete -`ContainerSource`. Use [heartbeats-source.yaml](./heartbeats-source.yaml) as a -sample for reference. [Learn more about the ContainerSource -specification](../../../eventing#containersource). diff --git a/docs/eventing/samples/container-source/index.md b/docs/eventing/samples/container-source/index.md index 45f91706f7e..21285761657 100644 --- a/docs/eventing/samples/container-source/index.md +++ b/docs/eventing/samples/container-source/index.md @@ -5,4 +5,174 @@ weight: 10 type: "docs" --- -{{% readfile file="README.md" %}} +ContainerSource will start a container image which will generate events under +certain situations and send messages to a sink URI. It also can be an easy way +to support your own event sources in Knative. This guide shows how to configure +ContainerSource as an event source for functions and summarizes guidelines for +creating your own event source as a ContainerSource. + +## Create a heartbeats ContainerSource + +### Prerequisites + +1. Setup [Knative Serving](../../../serving). +1. Setup [Knative Eventing and Sources](../../../eventing). + +### Prepare the heartbeats image + +Knative [event-sources](https://github.com/knative/eventing-contrib) has a +sample of heartbeats event source. You could clone the source code by + +``` +git clone -b "{{< branch >}}" https://github.com/knative/eventing-contrib.git +``` + +And then build a heartbeats image and publish to your image repo with + +``` +ko publish knative.dev/eventing-contrib/cmd/heartbeats +``` + +**Note**: `ko publish` requires: + +- [`KO_DOCKER_REPO`](https://github.com/knative/serving/blob/main/DEVELOPMENT.md#environment-setup) + to be set. (e.g. `gcr.io/[gcloud-project]` or `docker.io/`) +- you to be authenticated with your `KO_DOCKER_REPO` +- [`docker`](https://docs.docker.com/install/) to be installed + +### Create a Knative Service + +In order to verify `ContainerSource` is working, we will create a Event Display +Service that dumps incoming messages to its log. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: event-display +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Use following command to create the service from `service.yaml`: + +```shell +kubectl apply --filename service.yaml +``` + +The status of the created service can be seen using: + +```shell +kubectl get ksvc + +NAME URL LATESTCREATED LATESTREADY READY REASON +event-display http://event-display.default.1.2.3.4.xip.io event-display-gqjbw event-display-gqjbw True +``` + +### Create a ContainerSource using the heartbeats image + +In order to run the heartbeats container as an event source, you have to create +a concrete ContainerSource with specific arguments and environment settings. Be +sure to replace `heartbeats_image_uri` with a valid uri for your heartbeats +image in your image repo in [heartbeats-source.yaml](./heartbeats-source.yaml) +file. Note that arguments and environment variables are set and will be passed +to the container. + +```yaml +apiVersion: sources.knative.dev/v1 +kind: ContainerSource +metadata: + name: test-heartbeats +spec: + template: + spec: + containers: + - image: + name: heartbeats + args: + - --period=1 + env: + - name: POD_NAME + value: "mypod" + - name: POD_NAMESPACE + value: "event-test" + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display +``` + +Use the following command to create the event source from +`heartbeats-source.yaml`: + +```shell +kubectl apply --filename heartbeats-source.yaml +``` + +### Verify + +We will verify that the message was sent to the Knative eventing system by +looking at event-display service logs. + +```shell +kubectl logs -l serving.knative.dev/service=event-display -c user-container --since=10m +``` + +You should see log lines showing the request headers and body of the event +message sent by the heartbeats source to the display function: + +``` +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.eventing.samples.heartbeat + source: https://knative.dev/eventing-contrib/cmd/heartbeats/#event-test/mypod + id: 2b72d7bf-c38f-4a98-a433-608fbcdd2596 + time: 2019-10-18T15:23:20.809775386Z + contenttype: application/json +Extensions, + beats: true + heart: yes + the: 42 +Data, + { + "id": 2, + "label": "" + } +``` + +## Create a new event source using ContainerSource + +In order to create a new event source using ContainerSource, you will create a +container image at first, and then create a ContainerSource with the image uri +and specify the values of parameters. + +### Develop, build and publish a container image + +The container image can be developed with any language, build and publish with +any tools you like. Here are some basic guidelines: + +- The container image must have a `main` method to start with. +- The `main` method will accept parameters from arguments and environment + variables. +- Two environments variables will be injected by the `ContainerSource` controller, +`K_SINK` and `K_CE_OVERRIDES`, resolved from `spec.sink` and `spec.ceOverrides` respectively. +- The event messages shall be sent to the sink URI specified in `K_SINK`. The message can be any + format. + [CloudEvents](https://github.com/cloudevents/spec/blob/master/spec.md#design-goals) + format is recommended. + +[heartbeats](https://github.com/knative/eventing-contrib/blob/main/cmd/heartbeats/main.go) +event source is a sample for your reference. + +### Create the ContainerSource using this container image + +When the container image is ready, a YAML file will be used to create a concrete +`ContainerSource`. Use [heartbeats-source.yaml](./heartbeats-source.yaml) as a +sample for reference. [Learn more about the ContainerSource +specification](../../../eventing#containersource). diff --git a/docs/eventing/samples/github-source/README.md b/docs/eventing/samples/github-source/README.md deleted file mode 100644 index f1e56276e46..00000000000 --- a/docs/eventing/samples/github-source/README.md +++ /dev/null @@ -1,184 +0,0 @@ -GitHub Source example shows how to wire GitHub events for consumption -by a Knative Service. - -## Before you begin - -1. Set up [Knative Serving](../../../serving). -1. Ensure Knative Serving is [configured with a domain - name](../../../serving/using-a-custom-domain.md) - that allows GitHub to call into the cluster. -1. If you're using GKE, you'll also want to [assign a static IP address](../../../serving/gke-assigning-static-ip-address.md). -1. Set up [Knative Eventing](../../../eventing) with the GitHub source. - -### Create a Knative Service - -To verify the GitHub source is working, create a simple Knative -Service that dumps incoming messages to its log. The `service.yaml` file -defines this basic Service. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: github-message-dumper -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Enter the following command to create the service from `service.yaml`: - -```shell -kubectl --namespace default apply --filename service.yaml -``` - -### Create GitHub Tokens - -Create a [personal access token](https://github.com/settings/tokens) -for GitHub that the GitHub source can use to register webhooks with -the GitHub API. Also decide on a secret token that your code will use -to authenticate the incoming webhooks from GitHub (_secretToken_). - -The token can be named anything you find convenient. The Source -requires `repo:public_repo` and `admin:repo_hook`, to let it fire -events from your public repositories and to create webhooks for those -repositories. Copy and save this token; GitHub will force you to -generate it again if misplaced. - -Here's an example for a token named "GitHubSource Sample" with the -recommended scopes: - -![GitHub UI](personal_access_token.png "GitHub personal access token screenshot") - -Update `githubsecret.yaml` with those values. If your generated access -token is `'personal_access_token_value'` and you choose your _secretToken_ -as `'asdfasfdsaf'`, you'd modify `githubsecret.yaml` like so: - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: githubsecret -type: Opaque -stringData: - accessToken: personal_access_token_value - secretToken: asdfasfdsaf -``` - -Hint: you can makeup a random _secretToken_ with: - -```shell -head -c 8 /dev/urandom | base64 -``` - -Then, apply the githubsecret using `kubectl`: - -```shell -kubectl --namespace default apply --filename githubsecret.yaml -``` - -### Create Event Source for GitHub Events - -In order to receive GitHub events, you have to create a concrete Event -Source for a specific namespace. Be sure to replace the -`ownerAndRepository` value with a valid GitHub public repository owned -by your GitHub user. - -If using GitHub enterprise you will need to add an additional githubAPIURL -field to the spec specifying your GitHub enterprise API endpoint, see -[here](../../README.md#githubsource) - -```yaml -apiVersion: sources.knative.dev/v1alpha1 -kind: GitHubSource -metadata: - name: githubsourcesample -spec: - eventTypes: - - pull_request - ownerAndRepository: / - accessToken: - secretKeyRef: - name: githubsecret - key: accessToken - secretToken: - secretKeyRef: - name: githubsecret - key: secretToken - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: github-message-dumper -``` - -Then, apply that yaml using `kubectl`: - -```shell -kubectl --namespace default apply --filename github-source.yaml -``` - -### Verify - -Verify the GitHub webhook was created by looking at the list of -webhooks under the Settings tab in your GitHub repository. A hook -should be listed that points to your Knative cluster with a green -check mark to the left of the hook URL, as shown below. - -![GitHub Webhook](webhook_created.png "GitHub webhook screenshot") - -### Create Events - -Create a pull request in your GitHub repository. We will verify -that the GitHub events were sent into the Knative eventing system -by looking at our message dumper function logs. - -```shell -kubectl --namespace default get pods -kubectl --namespace default logs github-event-display-XXXX user-container -``` - -You should log lines similar to: - -``` -2018/11/08 18:25:34 Message Dumper received a message: POST / HTTP/1.1 -Host: github-event-display.knative-demo.svc.cluster.local -Accept-Encoding: gzip -Ce-Cloudeventsversion: 0.1 -Ce-Eventid: a8d4cf20-e383-11e8-8069-46e3c8ad2b4d -Ce-Eventtime: 2018-11-08T18:25:32.819548012Z -Ce-Eventtype: dev.knative.source.github.pull_request -Ce-Source: https://github.com/someuser/somerepo/pull/1 -Content-Length: 21060 -Content-Type: application/json -User-Agent: Go-http-client/1.1 -X-B3-Parentspanid: b2e514c3dbe94c03 -X-B3-Sampled: 1 -X-B3-Spanid: c85e346d89c8be4e -X-B3-Traceid: abf6292d458fb8e7 -X-Envoy-Expected-Rq-Timeout-Ms: 60000 -X-Envoy-Internal: true -X-Forwarded-For: 127.0.0.1, 127.0.0.1 -X-Forwarded-Proto: http -X-Request-Id: 8a2201af-5075-9447-b593-ec3a243aff52 - -{"action":"opened","number":1,"pull_request": ...} -``` - -### Cleanup - -You can remove the Github webhook by deleting the Github source: - -```shell -kubectl --namespace default delete --filename github-source.yaml -``` - -Similarly, you can remove the Service and Secret via: - -```shell -kubectl --namespace default delete --filename service.yaml -kubectl --namespace default delete --filename githubsecret.yaml - -``` diff --git a/docs/eventing/samples/github-source/index.md b/docs/eventing/samples/github-source/index.md index 532c401e7ad..f7c3aa3cd5c 100644 --- a/docs/eventing/samples/github-source/index.md +++ b/docs/eventing/samples/github-source/index.md @@ -5,4 +5,187 @@ weight: 30 type: "docs" --- -{{% readfile file="README.md" %}} +GitHub Source example shows how to wire GitHub events for consumption +by a Knative Service. + +## Before you begin + +1. Set up [Knative Serving](../../../serving). +1. Ensure Knative Serving is [configured with a domain + name](../../../serving/using-a-custom-domain.md) + that allows GitHub to call into the cluster. +1. If you're using GKE, you'll also want to [assign a static IP address](../../../serving/gke-assigning-static-ip-address.md). +1. Set up [Knative Eventing](../../../eventing) with the GitHub source. + +### Create a Knative Service + +To verify the GitHub source is working, create a simple Knative +Service that dumps incoming messages to its log. The `service.yaml` file +defines this basic Service. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: github-message-dumper +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Enter the following command to create the service from `service.yaml`: + +```shell +kubectl --namespace default apply --filename service.yaml +``` + +### Create GitHub Tokens + +Create a [personal access token](https://github.com/settings/tokens) +for GitHub that the GitHub source can use to register webhooks with +the GitHub API. Also decide on a secret token that your code will use +to authenticate the incoming webhooks from GitHub (_secretToken_). + +The token can be named anything you find convenient. The Source +requires `repo:public_repo` and `admin:repo_hook`, to let it fire +events from your public repositories and to create webhooks for those +repositories. Copy and save this token; GitHub will force you to +generate it again if misplaced. + +Here's an example for a token named "GitHubSource Sample" with the +recommended scopes: + +![GitHub UI](personal_access_token.png "GitHub personal access token screenshot") + +Update `githubsecret.yaml` with those values. If your generated access +token is `'personal_access_token_value'` and you choose your _secretToken_ +as `'asdfasfdsaf'`, you'd modify `githubsecret.yaml` like so: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: githubsecret +type: Opaque +stringData: + accessToken: personal_access_token_value + secretToken: asdfasfdsaf +``` + +Hint: you can makeup a random _secretToken_ with: + +```shell +head -c 8 /dev/urandom | base64 +``` + +Then, apply the githubsecret using `kubectl`: + +```shell +kubectl --namespace default apply --filename githubsecret.yaml +``` + +### Create Event Source for GitHub Events + +In order to receive GitHub events, you have to create a concrete Event +Source for a specific namespace. Be sure to replace the +`ownerAndRepository` value with a valid GitHub public repository owned +by your GitHub user. + +If using GitHub enterprise you will need to add an additional githubAPIURL +field to the spec specifying your GitHub enterprise API endpoint, see +[here](../../README.md#githubsource) + +```yaml +apiVersion: sources.knative.dev/v1alpha1 +kind: GitHubSource +metadata: + name: githubsourcesample +spec: + eventTypes: + - pull_request + ownerAndRepository: / + accessToken: + secretKeyRef: + name: githubsecret + key: accessToken + secretToken: + secretKeyRef: + name: githubsecret + key: secretToken + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: github-message-dumper +``` + +Then, apply that yaml using `kubectl`: + +```shell +kubectl --namespace default apply --filename github-source.yaml +``` + +### Verify + +Verify the GitHub webhook was created by looking at the list of +webhooks under the Settings tab in your GitHub repository. A hook +should be listed that points to your Knative cluster with a green +check mark to the left of the hook URL, as shown below. + +![GitHub Webhook](webhook_created.png "GitHub webhook screenshot") + +### Create Events + +Create a pull request in your GitHub repository. We will verify +that the GitHub events were sent into the Knative eventing system +by looking at our message dumper function logs. + +```shell +kubectl --namespace default get pods +kubectl --namespace default logs github-event-display-XXXX user-container +``` + +You should log lines similar to: + +``` +2018/11/08 18:25:34 Message Dumper received a message: POST / HTTP/1.1 +Host: github-event-display.knative-demo.svc.cluster.local +Accept-Encoding: gzip +Ce-Cloudeventsversion: 0.1 +Ce-Eventid: a8d4cf20-e383-11e8-8069-46e3c8ad2b4d +Ce-Eventtime: 2018-11-08T18:25:32.819548012Z +Ce-Eventtype: dev.knative.source.github.pull_request +Ce-Source: https://github.com/someuser/somerepo/pull/1 +Content-Length: 21060 +Content-Type: application/json +User-Agent: Go-http-client/1.1 +X-B3-Parentspanid: b2e514c3dbe94c03 +X-B3-Sampled: 1 +X-B3-Spanid: c85e346d89c8be4e +X-B3-Traceid: abf6292d458fb8e7 +X-Envoy-Expected-Rq-Timeout-Ms: 60000 +X-Envoy-Internal: true +X-Forwarded-For: 127.0.0.1, 127.0.0.1 +X-Forwarded-Proto: http +X-Request-Id: 8a2201af-5075-9447-b593-ec3a243aff52 + +{"action":"opened","number":1,"pull_request": ...} +``` + +### Cleanup + +You can remove the Github webhook by deleting the Github source: + +```shell +kubectl --namespace default delete --filename github-source.yaml +``` + +Similarly, you can remove the Service and Secret via: + +```shell +kubectl --namespace default delete --filename service.yaml +kubectl --namespace default delete --filename githubsecret.yaml + +``` diff --git a/docs/eventing/samples/gitlab-source/README.md b/docs/eventing/samples/gitlab-source/README.md deleted file mode 100644 index 3fa521ea777..00000000000 --- a/docs/eventing/samples/gitlab-source/README.md +++ /dev/null @@ -1,194 +0,0 @@ -GitLab Source example shows how to wire GitLab events for consumption by a -Knative Service. - -## Gitlab source deployment - -### Prerequisites - -You will need: - -1. An internet-accessible Kubernetes cluster with Knative Serving installed. - Follow the [installation instructions](../../../install/README.md) if you - need to create one. -1. Ensure Knative Serving is - [configured with a domain name](../../../serving/using-a-custom-domain.md) - that allows GitLab to call into the cluster. -1. If you're using GKE, you'll also want to - [assign a static IP address](../../../serving/gke-assigning-static-ip-address.md). -1. Install [Knative Eventing](../../../eventing). - -### Install GitLab Event Source - -GitLab Event source lives in the [knative/eventing-contrib](https://github.com/knative/eventing-contrib). Head to the releases page, find the latest release with `gitlab.yaml` -artifact and replace the `` with version tag: - -```shell -kubectl apply -f https://github.com/knative/eventing-contrib/releases/download//gitlab.yaml -``` - -Check that the manager is running: - -```shell -kubectl -n knative-sources get pods --selector control-plane=gitlab-controller-manager -``` - -With the controller running you can now move on to a user persona and setup a -GitLab webhook as well as a function that will consume GitLab events. - -## Using the GitLab Event Source - -You are now ready to use the Event Source and trigger functions based on GitLab -projects events. - -We will: - -- Create a Knative service which will receive the events. To keep things simple - this service will simply dump the events to `stdout`, this is the so-called: - _event_display_ -- Create a GitLab access token and a random secret token used to secure the - webhooks -- Create the event source by posting a GitLab source object manifest to - Kubernetes - -### Create a Knative Service - -The `event-display.yaml` file shown below defines the basic service which will -receive events from the GitLab source. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: gitlab-event-display -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Create the service: - -```shell -kubectl -n default apply -f event-display.yaml -``` - -### Create GitLab Tokens - -1. Create a - [personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) - which the GitLab source will use to register webhooks with the GitLab API. - The token must have an "api" access scope in order to create repository - webhooks. Also decide on a secret token that your source will use to - authenticate the incoming webhooks from GitLab. - -1. Update a secret values in `secret.yaml` defined below: - - `accessToken` is the personal access token created in step 1 and - `secretToken` is any token of your choosing. - - Hint: you can generate a random _secretToken_ with: - - ```shell - head -c 8 /dev/urandom | base64 - ``` - - `secret.yaml`: - - ```yaml - apiVersion: v1 - kind: Secret - metadata: - name: gitlabsecret - type: Opaque - stringData: - accessToken: - secretToken: - ``` - -1. Create the secret using `kubectl`. - - ```shell - kubectl -n default apply -f secret.yaml - ``` - -### Create Event Source for GitLab Events - -1. In order to receive GitLab events, you have to create a concrete Event Source - for a specific namespace. Replace the `projectUrl` value in the - `gitlabsource.yaml` file with your GitLab project URL, for example - `https://gitlab.com/knative-examples/functions`. - - `gitlabsource.yaml`: - - ```yaml - apiVersion: sources.knative.dev/v1alpha1 - kind: GitLabSource - metadata: - name: gitlabsource-sample - spec: - eventTypes: - - push_events - - issues_events - projectUrl: - accessToken: - secretKeyRef: - name: gitlabsecret - key: accessToken - secretToken: - secretKeyRef: - name: gitlabsecret - key: secretToken - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: gitlab-event-display - ``` - -1. Apply the yaml file using `kubectl`: - - ```shell - kubectl -n default apply -f gitlabsource.yaml - ``` - -### Verify - -Verify that GitLab webhook was created by looking at the list of webhooks under -**Settings >> Integrations** in your GitLab project. A hook should be listed -that points to your Knative cluster. - -Create a push event and check the logs of the Pod backing the -`gitlab-event-display` knative service. You will see the event: - -``` -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 0.3 - type: dev.knative.sources.gitlabsource.Push Hook - source: https://gitlab.com// - id: f83c080f-c2af-48ff-8d8b-fd5b21c5938e - time: 2020-03-12T11:08:41.414572482Z - datacontenttype: application/json -Data, - { - - } -``` - -### Cleanup - -You can remove the GitLab webhook by deleting the GitLab source: - -```shell -kubectl --namespace default delete --filename gitlabsource.yaml -``` - -Similarly, you can remove the Service and Secret via: - -```shell -kubectl --namespace default delete --filename event-display.yaml -kubectl --namespace default delete --filename secret.yaml - -``` diff --git a/docs/eventing/samples/gitlab-source/index.md b/docs/eventing/samples/gitlab-source/index.md index 25262202788..c2aa04a98eb 100644 --- a/docs/eventing/samples/gitlab-source/index.md +++ b/docs/eventing/samples/gitlab-source/index.md @@ -5,4 +5,197 @@ weight: 30 type: "docs" --- -{{% readfile file="README.md" %}} +GitLab Source example shows how to wire GitLab events for consumption by a +Knative Service. + +## Gitlab source deployment + +### Prerequisites + +You will need: + +1. An internet-accessible Kubernetes cluster with Knative Serving installed. + Follow the [installation instructions](../../../install/README.md) if you + need to create one. +1. Ensure Knative Serving is + [configured with a domain name](../../../serving/using-a-custom-domain.md) + that allows GitLab to call into the cluster. +1. If you're using GKE, you'll also want to + [assign a static IP address](../../../serving/gke-assigning-static-ip-address.md). +1. Install [Knative Eventing](../../../eventing). + +### Install GitLab Event Source + +GitLab Event source lives in the [knative/eventing-contrib](https://github.com/knative/eventing-contrib). Head to the releases page, find the latest release with `gitlab.yaml` +artifact and replace the `` with version tag: + +```shell +kubectl apply -f https://github.com/knative/eventing-contrib/releases/download//gitlab.yaml +``` + +Check that the manager is running: + +```shell +kubectl -n knative-sources get pods --selector control-plane=gitlab-controller-manager +``` + +With the controller running you can now move on to a user persona and setup a +GitLab webhook as well as a function that will consume GitLab events. + +## Using the GitLab Event Source + +You are now ready to use the Event Source and trigger functions based on GitLab +projects events. + +We will: + +- Create a Knative service which will receive the events. To keep things simple + this service will simply dump the events to `stdout`, this is the so-called: + _event_display_ +- Create a GitLab access token and a random secret token used to secure the + webhooks +- Create the event source by posting a GitLab source object manifest to + Kubernetes + +### Create a Knative Service + +The `event-display.yaml` file shown below defines the basic service which will +receive events from the GitLab source. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: gitlab-event-display +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Create the service: + +```shell +kubectl -n default apply -f event-display.yaml +``` + +### Create GitLab Tokens + +1. Create a + [personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) + which the GitLab source will use to register webhooks with the GitLab API. + The token must have an "api" access scope in order to create repository + webhooks. Also decide on a secret token that your source will use to + authenticate the incoming webhooks from GitLab. + +1. Update a secret values in `secret.yaml` defined below: + + `accessToken` is the personal access token created in step 1 and + `secretToken` is any token of your choosing. + + Hint: you can generate a random _secretToken_ with: + + ```shell + head -c 8 /dev/urandom | base64 + ``` + + `secret.yaml`: + + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: gitlabsecret + type: Opaque + stringData: + accessToken: + secretToken: + ``` + +1. Create the secret using `kubectl`. + + ```shell + kubectl -n default apply -f secret.yaml + ``` + +### Create Event Source for GitLab Events + +1. In order to receive GitLab events, you have to create a concrete Event Source + for a specific namespace. Replace the `projectUrl` value in the + `gitlabsource.yaml` file with your GitLab project URL, for example + `https://gitlab.com/knative-examples/functions`. + + `gitlabsource.yaml`: + + ```yaml + apiVersion: sources.knative.dev/v1alpha1 + kind: GitLabSource + metadata: + name: gitlabsource-sample + spec: + eventTypes: + - push_events + - issues_events + projectUrl: + accessToken: + secretKeyRef: + name: gitlabsecret + key: accessToken + secretToken: + secretKeyRef: + name: gitlabsecret + key: secretToken + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: gitlab-event-display + ``` + +1. Apply the yaml file using `kubectl`: + + ```shell + kubectl -n default apply -f gitlabsource.yaml + ``` + +### Verify + +Verify that GitLab webhook was created by looking at the list of webhooks under +**Settings >> Integrations** in your GitLab project. A hook should be listed +that points to your Knative cluster. + +Create a push event and check the logs of the Pod backing the +`gitlab-event-display` knative service. You will see the event: + +``` +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 0.3 + type: dev.knative.sources.gitlabsource.Push Hook + source: https://gitlab.com// + id: f83c080f-c2af-48ff-8d8b-fd5b21c5938e + time: 2020-03-12T11:08:41.414572482Z + datacontenttype: application/json +Data, + { + + } +``` + +### Cleanup + +You can remove the GitLab webhook by deleting the GitLab source: + +```shell +kubectl --namespace default delete --filename gitlabsource.yaml +``` + +Similarly, you can remove the Service and Secret via: + +```shell +kubectl --namespace default delete --filename event-display.yaml +kubectl --namespace default delete --filename secret.yaml + +``` diff --git a/docs/eventing/samples/helloworld/README.md b/docs/eventing/samples/helloworld/README.md deleted file mode 100644 index efd99bf490b..00000000000 --- a/docs/eventing/samples/helloworld/README.md +++ /dev/null @@ -1,18 +0,0 @@ -Following examples include a simple web app written in the language of your choice that you can -use to test knative eventing. It shows how to consume a [CloudEvent](https://cloudevents.io/) -in Knative eventing, and optionally how to respond back with another CloudEvent in the HTTP response. - -We will deploy the app as a -[Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) -along with a -[Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). -However, you can also deploy the app as a [Knative Serving Service](../../../serving/README.md) - -## Prerequisites - -- A Kubernetes cluster with [Knative Eventing](../../getting-started.md#installing-knative-eventing) - installed. - - If you decide to deploy the app as a Knative Serving Service then you will have to install - [Knative Serving](../../../install/README.md). -- [Docker](https://www.docker.com) installed and running on your local machine, - and a Docker Hub account configured (we'll use it for a container registry). diff --git a/docs/eventing/samples/helloworld/_index.md b/docs/eventing/samples/helloworld/_index.md index cf33a8fbff0..329afa5a118 100644 --- a/docs/eventing/samples/helloworld/_index.md +++ b/docs/eventing/samples/helloworld/_index.md @@ -5,4 +5,21 @@ weight: 10 type: "docs" --- -{{% readfile file="README.md" %}} +Following examples include a simple web app written in the language of your choice that you can +use to test knative eventing. It shows how to consume a [CloudEvent](https://cloudevents.io/) +in Knative eventing, and optionally how to respond back with another CloudEvent in the HTTP response. + +We will deploy the app as a +[Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) +along with a +[Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). +However, you can also deploy the app as a [Knative Serving Service](../../../serving/README.md) + +## Prerequisites + +- A Kubernetes cluster with [Knative Eventing](../../getting-started.md#installing-knative-eventing) + installed. + - If you decide to deploy the app as a Knative Serving Service then you will have to install + [Knative Serving](../../../install/README.md). +- [Docker](https://www.docker.com) installed and running on your local machine, + and a Docker Hub account configured (we'll use it for a container registry). diff --git a/docs/eventing/samples/helloworld/helloworld-go/README.md b/docs/eventing/samples/helloworld/helloworld-go/README.md deleted file mode 100644 index a46e8e0f0e4..00000000000 --- a/docs/eventing/samples/helloworld/helloworld-go/README.md +++ /dev/null @@ -1,457 +0,0 @@ -A simple web app written in Go that you can use to test knative eventing. It -shows how to consume a [CloudEvent](https://cloudevents.io/) in Knative -eventing, and optionally how to respond back with another CloudEvent in the http -response, using the -[Go SDK for CloudEvents](https://github.com/cloudevents/sdk-go) - -We will deploy the app as a -[Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) -along with a -[Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). -However, you can also deploy the app as a -[Knative Serving Service](../../../../serving/README.md). - -Follow the steps below to create the sample code and then deploy the app to your -cluster. You can also download a working copy of the sample, by running the -following commands: - -```shell -git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs -cd knative-docs/docs/eventing/samples/helloworld/helloworld-go -``` - -## Before you begin - -- A Kubernetes cluster with - [Knative Eventing](../../../getting-started.md#installing-knative-eventing) - installed. -- [Docker](https://www.docker.com) installed and running on your local machine, - and a Docker Hub account configured (we'll use it for a container registry). - -## Recreating the sample code - -1. Create a new file named `helloworld.go` and paste the following code. This - code creates a basic web server which listens on port 8080: - - ```go - import ( - "context" - "log" - - cloudevents "github.com/cloudevents/sdk-go/v2" - "github.com/google/uuid" - ) - - func receive(ctx context.Context, event cloudevents.Event) (*cloudevents.Event, cloudevents.Result) { - // Here is where your code to process the event will go. - // In this example we will log the event msg - log.Printf("Event received. \n%s\n", event) - data := &HelloWorld{} - if err := event.DataAs(data); err != nil { - log.Printf("Error while extracting cloudevent Data: %s\n", err.Error()) - return nil, cloudevents.NewHTTPResult(400, "failed to convert data: %s", err) - } - log.Printf("Hello World Message from received event %q", data.Msg) - - // Respond with another event (optional) - // This is optional and is intended to show how to respond back with another event after processing. - // The response will go back into the knative eventing system just like any other event - newEvent := cloudevents.NewEvent() - newEvent.SetID(uuid.New().String()) - newEvent.SetSource("knative/eventing/samples/hello-world") - newEvent.SetType("dev.knative.samples.hifromknative") - if err := newEvent.SetData(cloudevents.ApplicationJSON, HiFromKnative{Msg: "Hi from helloworld-go app!"}); err != nil { - return nil, cloudevents.NewHTTPResult(500, "failed to set response data: %s", err) - } - log.Printf("Responding with event\n%s\n", newEvent) - return &newEvent, nil - } - - func main() { - log.Print("Hello world sample started.") - c, err := cloudevents.NewDefaultClient() - if err != nil { - log.Fatalf("failed to create client, %v", err) - } - log.Fatal(c.StartReceiver(context.Background(), receive)) - } - ``` - -1. Create a new file named `eventschemas.go` and paste the following code. This - defines the data schema of the CloudEvents. - - ```go - package main - - // HelloWorld defines the Data of CloudEvent with type=dev.knative.samples.helloworld - type HelloWorld struct { - // Msg holds the message from the event - Msg string `json:"msg,omitempty,string"` - } - - // HiFromKnative defines the Data of CloudEvent with type=dev.knative.samples.hifromknative - type HiFromKnative struct { - // Msg holds the message from the event - Msg string `json:"msg,omitempty,string"` - } - ``` - -1. In your project directory, create a file named `Dockerfile` and copy the code - block below into it. For detailed instructions on dockerizing a Go app, see - [Deploying Go servers with Docker](https://blog.golang.org/docker). - - ```docker - # Use the official Golang image to create a build artifact. - # This is based on Debian and sets the GOPATH to /go. - # https://hub.docker.com/_/golang - FROM golang:1.14 as builder - - # Copy local code to the container image. - WORKDIR /app - - # Retrieve application dependencies using go modules. - # Allows container builds to reuse downloaded dependencies. - COPY go.* ./ - RUN go mod download - - # Copy local code to the container image. - COPY . ./ - - # Build the binary. - # -mod=readonly ensures immutable go.mod and go.sum in container builds. - RUN CGO_ENABLED=0 GOOS=linux go build -mod=readonly -v -o helloworld - - # Use a Docker multi-stage build to create a lean production image. - # https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds - FROM alpine:3 - RUN apk add --no-cache ca-certificates - - # Copy the binary to the production image from the builder stage. - COPY --from=builder /app/helloworld /helloworld - - # Run the web service on container startup. - CMD ["/helloworld"] - ``` - -1. Create a new file, `sample-app.yaml` and copy the following service - definition into the file. Make sure to replace `{username}` with your Docker - Hub username. - - ```yaml - # Namespace for sample application - apiVersion: v1 - kind: Namespace - metadata: - name: knative-samples - --- - # A default broker - apiVersion: eventing.knative.dev/v1 - kind: Broker - metadata: - name: default - namespace: knative-samples - spec: {} - --- - # Helloworld-go app deploment - apiVersion: apps/v1 - kind: Deployment - metadata: - name: helloworld-go - namespace: knative-samples - spec: - replicas: 1 - selector: - matchLabels: &labels - app: helloworld-go - template: - metadata: - labels: *labels - spec: - containers: - - name: helloworld-go - image: docker.io/{username}/helloworld-go - - --- - # Service that exposes helloworld-go app. - # This will be the subscriber for the Trigger - kind: Service - apiVersion: v1 - metadata: - name: helloworld-go - namespace: knative-samples - spec: - selector: - app: helloworld-go - ports: - - protocol: TCP - port: 80 - targetPort: 8080 - --- - # Knative Eventing Trigger to trigger the helloworld-go service - apiVersion: eventing.knative.dev/v1 - kind: Trigger - metadata: - name: helloworld-go - namespace: knative-samples - spec: - broker: default - filter: - attributes: - type: dev.knative.samples.helloworld - source: dev.knative.samples/helloworldsource - subscriber: - ref: - apiVersion: v1 - kind: Service - name: helloworld-go - ``` - -1. Use the go tool to create a - [`go.mod`](https://github.com/golang/go/wiki/Modules#gomod) manifest. - - ```shell - go mod init github.com/knative/docs/docs/serving/samples/hello-world/helloworld-go - ``` - -## Building and deploying the sample - -Once you have recreated the sample code files (or used the files in the sample -folder) you're ready to build and deploy the sample app. - -1. Use Docker to build the sample code into a container. To build and push with - Docker Hub, run these commands replacing `{username}` with your Docker Hub - username: - - ```shell - # Build the container on your local machine - docker build -t {username}/helloworld-go . - - # Push the container to docker registry - docker push {username}/helloworld-go - ``` - -1. After the build has completed and the container is pushed to docker hub, you - can deploy the sample application into your cluster. Ensure that the - container image value in `sample-app.yaml` matches the container you built in - the previous step. Apply the configuration using `kubectl`: - - ```shell - kubectl apply --filename sample-app.yaml - ``` - - 1. Above command created a namespace `knative-samples` and create a default - Broker it. Verify using the following command: - - ```shell - kubectl get broker --namespace knative-samples - ``` - - **Note:** you can also use injection based on labels with the - Eventing sugar controller. - For how to install the Eventing sugar controller, see - [Install optional Eventing extensions](../../../../install/install-extensions.md#install-optional-eventing-extensions). - - 1. It deployed the helloworld-go app as a K8s Deployment and created a K8s - service names helloworld-go. Verify using the following command. - - ```shell - kubectl --namespace knative-samples get deployments helloworld-go - - kubectl --namespace knative-samples get svc helloworld-go - ``` - - 1. It created a Knative Eventing Trigger to route certain events to the - helloworld-go application. Make sure that Ready=true - ```shell - kubectl --namespace knative-samples get trigger helloworld-go - ``` - -## Send and verify CloudEvents - -Once you have deployed the application and verified that the namespace, sample -application and trigger are ready, let's send a CloudEvent. - -### Send CloudEvent to the Broker - -We can send an http request directly to the [Broker](../../../broker/README.md) -with correct CloudEvent headers set. - -1. Deploy a curl pod and SSH into it - ```shell - kubectl --namespace knative-samples run curl --image=radial/busyboxplus:curl -it - ``` -1. Get the Broker URL - ```shell - kubectl --namespace knative-samples get broker default - ``` -1. Run the following in the SSH terminal. Please replace the URL with the URL of - the default broker. - - ```shell - curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/knative-samples/default" \ - -X POST \ - -H "Ce-Id: 536808d3-88be-4077-9d7a-a3f162705f79" \ - -H "Ce-Specversion: 1.0" \ - -H "Ce-Type: dev.knative.samples.helloworld" \ - -H "Ce-Source: dev.knative.samples/helloworldsource" \ - -H "Content-Type: application/json" \ - -d '{"msg":"Hello World from the curl pod."}' - - exit - ``` - -### Verify that event is received by helloworld-go app - -Helloworld-go app logs the context and the msg of the above event, and replies -back with another event. - -1. Display helloworld-go app logs - - ```shell - kubectl --namespace knative-samples logs -l app=helloworld-go --tail=50 - ``` - - You should see something similar to: - - ```shell - Event received. - Validation: valid - Context Attributes, - specversion: 1.0 - type: dev.knative.samples.helloworld - source: dev.knative.samples/helloworldsource - id: 536808d3-88be-4077-9d7a-a3f162705f79 - time: 2019-10-04T22:35:26.05871736Z - datacontenttype: application/json - Extensions, - knativearrivaltime: 2019-10-04T22:35:26Z - knativehistory: default-kn2-trigger-kn-channel.knative-samples.svc.cluster.local - traceparent: 00-971d4644229653483d38c46e92a959c7-92c66312e4bb39be-00 - Data, - {"msg":"Hello World from the curl pod."} - - Hello World Message "Hello World from the curl pod." - Responded with event - Validation: valid - Context Attributes, - specversion: 1.0 - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - id: 37458d77-01f5-411e-a243-a459bbf79682 - datacontenttype: application/json - Data, - {"msg":"Hi from Knative!"} - - ``` - - Play around with the CloudEvent attributes in the curl command and the - trigger specification to understand how - [Triggers work](../../../broker/README.md#trigger). - -## Verify reply from helloworld-go app - -`helloworld-go` app replies back with an event of -`type= dev.knative.samples.hifromknative`, and -`source=knative/eventing/samples/hello-world`. This event enters the eventing -mesh via the Broker and can be delivered to other services using a Trigger - -1. Deploy a pod that receives any CloudEvent and logs the event to its output. - - ```shell - kubectl --namespace knative-samples apply --filename - << END - # event-display app deploment - apiVersion: apps/v1 - kind: Deployment - metadata: - name: event-display - namespace: knative-samples - spec: - replicas: 1 - selector: - matchLabels: &labels - app: event-display - template: - metadata: - labels: *labels - spec: - containers: - - name: helloworld-go - # Source code: https://github.com/knative/eventing-contrib/tree/main/cmd/event_display - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display - --- - # Service that exposes event-display app. - # This will be the subscriber for the Trigger - kind: Service - apiVersion: v1 - metadata: - name: event-display - namespace: knative-samples - spec: - selector: - app: event-display - ports: - - protocol: TCP - port: 80 - targetPort: 8080 - END - ``` - -1. Create a trigger to deliver the event to the above service - - ```shell - kubectl --namespace knative-samples apply --filename - << END - apiVersion: eventing.knative.dev/v1 - kind: Trigger - metadata: - name: event-display - namespace: knative-samples - spec: - broker: default - filter: - attributes: - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - subscriber: - ref: - apiVersion: v1 - kind: Service - name: event-display - END - ``` - -1. [Send a CloudEvent to the Broker](###Send-CloudEvent-to-the-Broker) - -1. Check the logs of event-display service - ```shell - kubectl --namespace knative-samples logs -l app=event-display --tail=50 - ``` - You should see something similar to: - ```shell - cloudevents.Event - Validation: valid - Context Attributes, - specversion: 1.0 - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - id: 8a7384b9-8bbe-4634-bf0f-ead07e450b2a - time: 2019-10-04T22:53:39.844943931Z - datacontenttype: application/json - Extensions, - knativearrivaltime: 2019-10-04T22:53:39Z - knativehistory: default-kn2-ingress-kn-channel.knative-samples.svc.cluster.local - traceparent: 00-4b01db030b9ea04bb150b77c8fa86509-2740816590a7604f-00 - Data, - { - "msg": "Hi from helloworld-go app!" - } - ``` - -**Note: You could use the above approach to test your applications too.** - -## Removing the sample app deployment - -To remove the sample app from your cluster, delete the service record: - -```shell -kubectl delete --filename sample-app.yaml -``` diff --git a/docs/eventing/samples/helloworld/helloworld-go/_index.md b/docs/eventing/samples/helloworld/helloworld-go/_index.md index b9ae8b20abf..b2381562af3 100644 --- a/docs/eventing/samples/helloworld/helloworld-go/_index.md +++ b/docs/eventing/samples/helloworld/helloworld-go/_index.md @@ -5,4 +5,460 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +A simple web app written in Go that you can use to test knative eventing. It +shows how to consume a [CloudEvent](https://cloudevents.io/) in Knative +eventing, and optionally how to respond back with another CloudEvent in the http +response, using the +[Go SDK for CloudEvents](https://github.com/cloudevents/sdk-go) + +We will deploy the app as a +[Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) +along with a +[Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). +However, you can also deploy the app as a +[Knative Serving Service](../../../../serving/README.md). + +Follow the steps below to create the sample code and then deploy the app to your +cluster. You can also download a working copy of the sample, by running the +following commands: + +```shell +git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs +cd knative-docs/docs/eventing/samples/helloworld/helloworld-go +``` + +## Before you begin + +- A Kubernetes cluster with + [Knative Eventing](../../../getting-started.md#installing-knative-eventing) + installed. +- [Docker](https://www.docker.com) installed and running on your local machine, + and a Docker Hub account configured (we'll use it for a container registry). + +## Recreating the sample code + +1. Create a new file named `helloworld.go` and paste the following code. This + code creates a basic web server which listens on port 8080: + + ```go + import ( + "context" + "log" + + cloudevents "github.com/cloudevents/sdk-go/v2" + "github.com/google/uuid" + ) + + func receive(ctx context.Context, event cloudevents.Event) (*cloudevents.Event, cloudevents.Result) { + // Here is where your code to process the event will go. + // In this example we will log the event msg + log.Printf("Event received. \n%s\n", event) + data := &HelloWorld{} + if err := event.DataAs(data); err != nil { + log.Printf("Error while extracting cloudevent Data: %s\n", err.Error()) + return nil, cloudevents.NewHTTPResult(400, "failed to convert data: %s", err) + } + log.Printf("Hello World Message from received event %q", data.Msg) + + // Respond with another event (optional) + // This is optional and is intended to show how to respond back with another event after processing. + // The response will go back into the knative eventing system just like any other event + newEvent := cloudevents.NewEvent() + newEvent.SetID(uuid.New().String()) + newEvent.SetSource("knative/eventing/samples/hello-world") + newEvent.SetType("dev.knative.samples.hifromknative") + if err := newEvent.SetData(cloudevents.ApplicationJSON, HiFromKnative{Msg: "Hi from helloworld-go app!"}); err != nil { + return nil, cloudevents.NewHTTPResult(500, "failed to set response data: %s", err) + } + log.Printf("Responding with event\n%s\n", newEvent) + return &newEvent, nil + } + + func main() { + log.Print("Hello world sample started.") + c, err := cloudevents.NewDefaultClient() + if err != nil { + log.Fatalf("failed to create client, %v", err) + } + log.Fatal(c.StartReceiver(context.Background(), receive)) + } + ``` + +1. Create a new file named `eventschemas.go` and paste the following code. This + defines the data schema of the CloudEvents. + + ```go + package main + + // HelloWorld defines the Data of CloudEvent with type=dev.knative.samples.helloworld + type HelloWorld struct { + // Msg holds the message from the event + Msg string `json:"msg,omitempty,string"` + } + + // HiFromKnative defines the Data of CloudEvent with type=dev.knative.samples.hifromknative + type HiFromKnative struct { + // Msg holds the message from the event + Msg string `json:"msg,omitempty,string"` + } + ``` + +1. In your project directory, create a file named `Dockerfile` and copy the code + block below into it. For detailed instructions on dockerizing a Go app, see + [Deploying Go servers with Docker](https://blog.golang.org/docker). + + ```docker + # Use the official Golang image to create a build artifact. + # This is based on Debian and sets the GOPATH to /go. + # https://hub.docker.com/_/golang + FROM golang:1.14 as builder + + # Copy local code to the container image. + WORKDIR /app + + # Retrieve application dependencies using go modules. + # Allows container builds to reuse downloaded dependencies. + COPY go.* ./ + RUN go mod download + + # Copy local code to the container image. + COPY . ./ + + # Build the binary. + # -mod=readonly ensures immutable go.mod and go.sum in container builds. + RUN CGO_ENABLED=0 GOOS=linux go build -mod=readonly -v -o helloworld + + # Use a Docker multi-stage build to create a lean production image. + # https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds + FROM alpine:3 + RUN apk add --no-cache ca-certificates + + # Copy the binary to the production image from the builder stage. + COPY --from=builder /app/helloworld /helloworld + + # Run the web service on container startup. + CMD ["/helloworld"] + ``` + +1. Create a new file, `sample-app.yaml` and copy the following service + definition into the file. Make sure to replace `{username}` with your Docker + Hub username. + + ```yaml + # Namespace for sample application + apiVersion: v1 + kind: Namespace + metadata: + name: knative-samples + --- + # A default broker + apiVersion: eventing.knative.dev/v1 + kind: Broker + metadata: + name: default + namespace: knative-samples + spec: {} + --- + # Helloworld-go app deploment + apiVersion: apps/v1 + kind: Deployment + metadata: + name: helloworld-go + namespace: knative-samples + spec: + replicas: 1 + selector: + matchLabels: &labels + app: helloworld-go + template: + metadata: + labels: *labels + spec: + containers: + - name: helloworld-go + image: docker.io/{username}/helloworld-go + + --- + # Service that exposes helloworld-go app. + # This will be the subscriber for the Trigger + kind: Service + apiVersion: v1 + metadata: + name: helloworld-go + namespace: knative-samples + spec: + selector: + app: helloworld-go + ports: + - protocol: TCP + port: 80 + targetPort: 8080 + --- + # Knative Eventing Trigger to trigger the helloworld-go service + apiVersion: eventing.knative.dev/v1 + kind: Trigger + metadata: + name: helloworld-go + namespace: knative-samples + spec: + broker: default + filter: + attributes: + type: dev.knative.samples.helloworld + source: dev.knative.samples/helloworldsource + subscriber: + ref: + apiVersion: v1 + kind: Service + name: helloworld-go + ``` + +1. Use the go tool to create a + [`go.mod`](https://github.com/golang/go/wiki/Modules#gomod) manifest. + + ```shell + go mod init github.com/knative/docs/docs/serving/samples/hello-world/helloworld-go + ``` + +## Building and deploying the sample + +Once you have recreated the sample code files (or used the files in the sample +folder) you're ready to build and deploy the sample app. + +1. Use Docker to build the sample code into a container. To build and push with + Docker Hub, run these commands replacing `{username}` with your Docker Hub + username: + + ```shell + # Build the container on your local machine + docker build -t {username}/helloworld-go . + + # Push the container to docker registry + docker push {username}/helloworld-go + ``` + +1. After the build has completed and the container is pushed to docker hub, you + can deploy the sample application into your cluster. Ensure that the + container image value in `sample-app.yaml` matches the container you built in + the previous step. Apply the configuration using `kubectl`: + + ```shell + kubectl apply --filename sample-app.yaml + ``` + + 1. Above command created a namespace `knative-samples` and create a default + Broker it. Verify using the following command: + + ```shell + kubectl get broker --namespace knative-samples + ``` + + **Note:** you can also use injection based on labels with the + Eventing sugar controller. + For how to install the Eventing sugar controller, see + [Install optional Eventing extensions](../../../../install/install-extensions.md#install-optional-eventing-extensions). + + 1. It deployed the helloworld-go app as a K8s Deployment and created a K8s + service names helloworld-go. Verify using the following command. + + ```shell + kubectl --namespace knative-samples get deployments helloworld-go + + kubectl --namespace knative-samples get svc helloworld-go + ``` + + 1. It created a Knative Eventing Trigger to route certain events to the + helloworld-go application. Make sure that Ready=true + ```shell + kubectl --namespace knative-samples get trigger helloworld-go + ``` + +## Send and verify CloudEvents + +Once you have deployed the application and verified that the namespace, sample +application and trigger are ready, let's send a CloudEvent. + +### Send CloudEvent to the Broker + +We can send an http request directly to the [Broker](../../../broker/README.md) +with correct CloudEvent headers set. + +1. Deploy a curl pod and SSH into it + ```shell + kubectl --namespace knative-samples run curl --image=radial/busyboxplus:curl -it + ``` +1. Get the Broker URL + ```shell + kubectl --namespace knative-samples get broker default + ``` +1. Run the following in the SSH terminal. Please replace the URL with the URL of + the default broker. + + ```shell + curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/knative-samples/default" \ + -X POST \ + -H "Ce-Id: 536808d3-88be-4077-9d7a-a3f162705f79" \ + -H "Ce-Specversion: 1.0" \ + -H "Ce-Type: dev.knative.samples.helloworld" \ + -H "Ce-Source: dev.knative.samples/helloworldsource" \ + -H "Content-Type: application/json" \ + -d '{"msg":"Hello World from the curl pod."}' + + exit + ``` + +### Verify that event is received by helloworld-go app + +Helloworld-go app logs the context and the msg of the above event, and replies +back with another event. + +1. Display helloworld-go app logs + + ```shell + kubectl --namespace knative-samples logs -l app=helloworld-go --tail=50 + ``` + + You should see something similar to: + + ```shell + Event received. + Validation: valid + Context Attributes, + specversion: 1.0 + type: dev.knative.samples.helloworld + source: dev.knative.samples/helloworldsource + id: 536808d3-88be-4077-9d7a-a3f162705f79 + time: 2019-10-04T22:35:26.05871736Z + datacontenttype: application/json + Extensions, + knativearrivaltime: 2019-10-04T22:35:26Z + knativehistory: default-kn2-trigger-kn-channel.knative-samples.svc.cluster.local + traceparent: 00-971d4644229653483d38c46e92a959c7-92c66312e4bb39be-00 + Data, + {"msg":"Hello World from the curl pod."} + + Hello World Message "Hello World from the curl pod." + Responded with event + Validation: valid + Context Attributes, + specversion: 1.0 + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + id: 37458d77-01f5-411e-a243-a459bbf79682 + datacontenttype: application/json + Data, + {"msg":"Hi from Knative!"} + + ``` + + Play around with the CloudEvent attributes in the curl command and the + trigger specification to understand how + [Triggers work](../../../broker/README.md#trigger). + +## Verify reply from helloworld-go app + +`helloworld-go` app replies back with an event of +`type= dev.knative.samples.hifromknative`, and +`source=knative/eventing/samples/hello-world`. This event enters the eventing +mesh via the Broker and can be delivered to other services using a Trigger + +1. Deploy a pod that receives any CloudEvent and logs the event to its output. + + ```shell + kubectl --namespace knative-samples apply --filename - << END + # event-display app deploment + apiVersion: apps/v1 + kind: Deployment + metadata: + name: event-display + namespace: knative-samples + spec: + replicas: 1 + selector: + matchLabels: &labels + app: event-display + template: + metadata: + labels: *labels + spec: + containers: + - name: helloworld-go + # Source code: https://github.com/knative/eventing-contrib/tree/main/cmd/event_display + image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display + --- + # Service that exposes event-display app. + # This will be the subscriber for the Trigger + kind: Service + apiVersion: v1 + metadata: + name: event-display + namespace: knative-samples + spec: + selector: + app: event-display + ports: + - protocol: TCP + port: 80 + targetPort: 8080 + END + ``` + +1. Create a trigger to deliver the event to the above service + + ```shell + kubectl --namespace knative-samples apply --filename - << END + apiVersion: eventing.knative.dev/v1 + kind: Trigger + metadata: + name: event-display + namespace: knative-samples + spec: + broker: default + filter: + attributes: + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + subscriber: + ref: + apiVersion: v1 + kind: Service + name: event-display + END + ``` + +1. [Send a CloudEvent to the Broker](###Send-CloudEvent-to-the-Broker) + +1. Check the logs of event-display service + ```shell + kubectl --namespace knative-samples logs -l app=event-display --tail=50 + ``` + You should see something similar to: + ```shell + cloudevents.Event + Validation: valid + Context Attributes, + specversion: 1.0 + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + id: 8a7384b9-8bbe-4634-bf0f-ead07e450b2a + time: 2019-10-04T22:53:39.844943931Z + datacontenttype: application/json + Extensions, + knativearrivaltime: 2019-10-04T22:53:39Z + knativehistory: default-kn2-ingress-kn-channel.knative-samples.svc.cluster.local + traceparent: 00-4b01db030b9ea04bb150b77c8fa86509-2740816590a7604f-00 + Data, + { + "msg": "Hi from helloworld-go app!" + } + ``` + +**Note: You could use the above approach to test your applications too.** + +## Removing the sample app deployment + +To remove the sample app from your cluster, delete the service record: + +```shell +kubectl delete --filename sample-app.yaml +``` diff --git a/docs/eventing/samples/helloworld/helloworld-python/README.md b/docs/eventing/samples/helloworld/helloworld-python/README.md deleted file mode 100644 index 5c57208b203..00000000000 --- a/docs/eventing/samples/helloworld/helloworld-python/README.md +++ /dev/null @@ -1,338 +0,0 @@ -A simple web app written in Python that you can use to test knative eventing. It shows how to consume a [CloudEvent](https://cloudevents.io/) in Knative eventing, and optionally how to respond back with another CloudEvent in the http response, by adding the Cloud Eventing headers outlined in the Cloud Events standard definition. - -We will deploy the app as a [Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) along with a [Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). -However, you can also deploy the app as a [Knative Serving Service](../../../../serving/README.md). - -Follow the steps below to create the sample code and then deploy the app to your -cluster. You can also download a working copy of the sample, by running the -following commands: - -```shell -# Clone the relevant branch version such as "release-0.13" -git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs -cd knative-docs/docs/eventing/samples/helloworld/helloworld-python -``` - -## Before you begin - -- A Kubernetes cluster with [Knative Eventing](../../../getting-started.md#installing-knative-eventing) installed. -- [Docker](https://www.docker.com) installed and running on your local machine, - and a Docker Hub account configured (we'll use it for a container registry). - -## Recreating the sample code - -1. Create a new file named `helloworld.py` and paste the following code. This - code creates a basic web server which listens on port 8080: - - ```python - from flask import Flask, request, make_response - import uuid - - app = Flask(__name__) - - @app.route('/', methods=['POST']) - def hello_world(): - app.logger.warning(request.data) - # Respond with another event (optional) - response = make_response({ - "msg": "Hi from helloworld-python app!" - }) - response.headers["Ce-Id"] = str(uuid.uuid4()) - response.headers["Ce-specversion"] = "0.3" - response.headers["Ce-Source"] = "knative/eventing/samples/hello-world" - response.headers["Ce-Type"] = "dev.knative.samples.hifromknative" - return response - - if __name__ == '__main__': - app.run(debug=True, host='0.0.0.0', port=8080) - - - ``` - -1. Add a requirements.txt file containing the following contents: - - ```bash - Flask==1.1.1 - ``` -1. In your project directory, create a file named `Dockerfile` and copy the code - block below into it. For detailed instructions on dockerizing a Go app, see - [Deploying Go servers with Docker](https://blog.golang.org/docker). - - ```docker - FROM python:alpine3.7 - - COPY . /app - - WORKDIR /app - - RUN pip install -r requirements.txt - - EXPOSE 8080 - - ENTRYPOINT [ "python" ] - - CMD [ "helloworld.py" ] - - ``` - -1. Create a new file, `sample-app.yaml` and copy the following service definition - into the file. Make sure to replace `{username}` with your Docker Hub - username. - - ```yaml - # Namespace for sample application with eventing enabled - apiVersion: v1 - kind: Namespace - metadata: - name: knative-samples - labels: - eventing.knative.dev/injection: enabled - --- - # Helloworld-python app deploment - apiVersion: apps/v1 - kind: Deployment - metadata: - name: helloworld-python - namespace: knative-samples - spec: - replicas: 1 - selector: - matchLabels: &labels - app: helloworld-python - template: - metadata: - labels: *labels - spec: - containers: - - name: helloworld-python - image: docker.io/{username}/helloworld-python - imagePullPolicy: IfNotPresent - --- - # Service that exposes helloworld-python app. - # This will be the subscriber for the Trigger - kind: Service - apiVersion: v1 - metadata: - name: helloworld-python - namespace: knative-samples - spec: - selector: - app: helloworld-python - ports: - - protocol: TCP - port: 80 - targetPort: 8080 - --- - # Knative Eventing Trigger to trigger the helloworld-python service - apiVersion: eventing.knative.dev/v1 - kind: Trigger - metadata: - name: helloworld-python - namespace: knative-samples - spec: - broker: default - filter: - attributes: - type: dev.knative.samples.helloworld - source: dev.knative.samples/helloworldsource - subscriber: - ref: - apiVersion: v1 - kind: Service - name: helloworld-python - ``` - -## Building and deploying the sample - -Once you have recreated the sample code files (or used the files in the sample -folder) you're ready to build and deploy the sample app. - -1. Use Docker to build the sample code into a container. To build and push with - Docker Hub, run these commands replacing `{username}` with your Docker Hub - username: - - ```shell - # Build the container on your local machine - docker build -t {username}/helloworld-python . - - # Push the container to docker registry - docker push {username}/helloworld-python - ``` - -1. After the build has completed and the container is pushed to Docker Hub, you - can deploy the sample application into your cluster. Ensure that the container image value - in `sample-app.yaml` matches the container you built in the previous step. Apply - the configuration using `kubectl`: - - ```shell - kubectl apply --filename sample-app.yaml - ``` - 1. Above command created a namespace `knative-samples` and labelled it with `knative-eventing-injection=enabled`, to enable eventing in the namespace. Verify using the following command: - ```shell - kubectl get ns knative-samples --show-labels - ``` - 1. It deployed the helloworld-python app as a K8s Deployment and created a K8s service names helloworld-python. Verify using the following command. - ```shell - kubectl --namespace knative-samples get deployments helloworld-python - kubectl --namespace knative-samples get svc helloworld-python - ``` - 1. It created a Knative Eventing Trigger to route certain events to the helloworld-python application. Make sure that Ready=true - ```shell - kubectl --namespace knative-samples get trigger helloworld-python - ``` -## Send and verify CloudEvents -After you have deployed the application, and have verified that the namespace, sample application and trigger are ready, you can send a CloudEvent. - -### Send CloudEvent to the Broker -You can send an HTTP request directly to the Knative [broker](../../../broker-trigger.md) if the correct CloudEvent headers are set. - - 1. Deploy a curl pod and SSH into it - ```shell - kubectl --namespace knative-samples run curl --image=radial/busyboxplus:curl -it - ``` - 1. Run the following in the SSH terminal - ```shell - curl -v "default-broker.knative-samples.svc.cluster.local" \ - -X POST \ - -H "Ce-Id: 536808d3-88be-4077-9d7a-a3f162705f79" \ - -H "Ce-specversion: 0.3" \ - -H "Ce-Type: dev.knative.samples.helloworld" \ - -H "Ce-Source: dev.knative.samples/helloworldsource" \ - -H "Content-Type: application/json" \ - -d '{"msg":"Hello World from the curl pod."}' - - exit - ``` -### Verify that event is received by helloworld-python app -Helloworld-python app logs the context and the msg of the above event, and replies back with another event. - 1. Display helloworld-python app logs - ```shell - kubectl --namespace knative-samples logs -l app=helloworld-python --tail=50 - ``` - You should see something similar to: - ```shell - Event received. Context: Context Attributes, - specversion: 0.3 - type: dev.knative.samples.helloworld - source: dev.knative.samples/helloworldsource - id: 536808d3-88be-4077-9d7a-a3f162705f79 - time: 2019-10-04T22:35:26.05871736Z - datacontenttype: application/json - Extensions, - knativearrivaltime: 2019-10-04T22:35:26Z - knativehistory: default-kn2-trigger-kn-channel.knative-samples.svc.cluster.local - traceparent: 00-971d4644229653483d38c46e92a959c7-92c66312e4bb39be-00 - - Hello World Message "Hello World from the curl pod." - Responded with event Validation: valid - Context Attributes, - specversion: 0.2 - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - id: 37458d77-01f5-411e-a243-a459bbf79682 - Data, - {"msg":"Hi from Knative!"} - - ``` - Try the CloudEvent attributes in the curl command and the trigger specification to understand how [triggers](../../../broker-trigger.md#trigger) work. - -## Verify reply from helloworld-python app -The `helloworld-python` app replies with an event type `type= dev.knative.samples.hifromknative`, and source `source=knative/eventing/samples/hello-world`. The event enters the eventing mesh through the broker, and can be delivered to event sinks using a trigger - - 1. Deploy a pod that receives any CloudEvent and logs the event to its output. - ```shell - kubectl --namespace knative-samples apply --filename - << END - # event-display app deploment - apiVersion: apps/v1 - kind: Deployment - metadata: - name: event-display - namespace: knative-samples - spec: - replicas: 1 - selector: - matchLabels: &labels - app: event-display - template: - metadata: - labels: *labels - spec: - containers: - - name: helloworld-python - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display - --- - # Service that exposes event-display app. - # This will be the subscriber for the Trigger - kind: Service - apiVersion: v1 - metadata: - name: event-display - namespace: knative-samples - spec: - selector: - app: event-display - ports: - - protocol: TCP - port: 80 - targetPort: 8080 - END - ``` - 1. Create a trigger to deliver the event to the above service - ```shell - kubectl --namespace knative-samples apply --filename - << END - apiVersion: eventing.knative.dev/v1 - kind: Trigger - metadata: - name: event-display - namespace: knative-samples - spec: - broker: default - filter: - attributes: - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - subscriber: - ref: - apiVersion: v1 - kind: Service - name: event-display - END - ``` - - 1. [Send a CloudEvent to the Broker](###Send-CloudEvent-to-the-Broker) - - 1. Check the logs of event-display service - ```shell - kubectl --namespace knative-samples logs -l app=event-display --tail=50 - ``` - You should see something similar to: - ```shell - cloudevents.Event - Validation: valid - Context Attributes, - specversion: 0.3 - type: dev.knative.samples.hifromknative - source: knative/eventing/samples/hello-world - id: 8a7384b9-8bbe-4634-bf0f-ead07e450b2a - time: 2019-10-04T22:53:39.844943931Z - datacontenttype: application/json - Extensions, - knativearrivaltime: 2019-10-04T22:53:39Z - knativehistory: default-kn2-ingress-kn-channel.knative-samples.svc.cluster.local - traceparent: 00-4b01db030b9ea04bb150b77c8fa86509-2740816590a7604f-00 - Data, - { - "msg": "Hi from helloworld- app!" - } - ``` - - **Note: You could use the above approach to test your applications too.** - - -## Removing the sample app deployment - -To remove the sample app from your cluster, delete the service record: - -```shell -kubectl delete --filename sample-app.yaml -``` diff --git a/docs/eventing/samples/helloworld/helloworld-python/_index.md b/docs/eventing/samples/helloworld/helloworld-python/_index.md index e5be21926fc..618919dd02e 100644 --- a/docs/eventing/samples/helloworld/helloworld-python/_index.md +++ b/docs/eventing/samples/helloworld/helloworld-python/_index.md @@ -5,4 +5,341 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +A simple web app written in Python that you can use to test knative eventing. It shows how to consume a [CloudEvent](https://cloudevents.io/) in Knative eventing, and optionally how to respond back with another CloudEvent in the http response, by adding the Cloud Eventing headers outlined in the Cloud Events standard definition. + +We will deploy the app as a [Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) along with a [Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/). +However, you can also deploy the app as a [Knative Serving Service](../../../../serving/README.md). + +Follow the steps below to create the sample code and then deploy the app to your +cluster. You can also download a working copy of the sample, by running the +following commands: + +```shell +# Clone the relevant branch version such as "release-0.13" +git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs +cd knative-docs/docs/eventing/samples/helloworld/helloworld-python +``` + +## Before you begin + +- A Kubernetes cluster with [Knative Eventing](../../../getting-started.md#installing-knative-eventing) installed. +- [Docker](https://www.docker.com) installed and running on your local machine, + and a Docker Hub account configured (we'll use it for a container registry). + +## Recreating the sample code + +1. Create a new file named `helloworld.py` and paste the following code. This + code creates a basic web server which listens on port 8080: + + ```python + from flask import Flask, request, make_response + import uuid + + app = Flask(__name__) + + @app.route('/', methods=['POST']) + def hello_world(): + app.logger.warning(request.data) + # Respond with another event (optional) + response = make_response({ + "msg": "Hi from helloworld-python app!" + }) + response.headers["Ce-Id"] = str(uuid.uuid4()) + response.headers["Ce-specversion"] = "0.3" + response.headers["Ce-Source"] = "knative/eventing/samples/hello-world" + response.headers["Ce-Type"] = "dev.knative.samples.hifromknative" + return response + + if __name__ == '__main__': + app.run(debug=True, host='0.0.0.0', port=8080) + + + ``` + +1. Add a requirements.txt file containing the following contents: + + ```bash + Flask==1.1.1 + ``` +1. In your project directory, create a file named `Dockerfile` and copy the code + block below into it. For detailed instructions on dockerizing a Go app, see + [Deploying Go servers with Docker](https://blog.golang.org/docker). + + ```docker + FROM python:alpine3.7 + + COPY . /app + + WORKDIR /app + + RUN pip install -r requirements.txt + + EXPOSE 8080 + + ENTRYPOINT [ "python" ] + + CMD [ "helloworld.py" ] + + ``` + +1. Create a new file, `sample-app.yaml` and copy the following service definition + into the file. Make sure to replace `{username}` with your Docker Hub + username. + + ```yaml + # Namespace for sample application with eventing enabled + apiVersion: v1 + kind: Namespace + metadata: + name: knative-samples + labels: + eventing.knative.dev/injection: enabled + --- + # Helloworld-python app deploment + apiVersion: apps/v1 + kind: Deployment + metadata: + name: helloworld-python + namespace: knative-samples + spec: + replicas: 1 + selector: + matchLabels: &labels + app: helloworld-python + template: + metadata: + labels: *labels + spec: + containers: + - name: helloworld-python + image: docker.io/{username}/helloworld-python + imagePullPolicy: IfNotPresent + --- + # Service that exposes helloworld-python app. + # This will be the subscriber for the Trigger + kind: Service + apiVersion: v1 + metadata: + name: helloworld-python + namespace: knative-samples + spec: + selector: + app: helloworld-python + ports: + - protocol: TCP + port: 80 + targetPort: 8080 + --- + # Knative Eventing Trigger to trigger the helloworld-python service + apiVersion: eventing.knative.dev/v1 + kind: Trigger + metadata: + name: helloworld-python + namespace: knative-samples + spec: + broker: default + filter: + attributes: + type: dev.knative.samples.helloworld + source: dev.knative.samples/helloworldsource + subscriber: + ref: + apiVersion: v1 + kind: Service + name: helloworld-python + ``` + +## Building and deploying the sample + +Once you have recreated the sample code files (or used the files in the sample +folder) you're ready to build and deploy the sample app. + +1. Use Docker to build the sample code into a container. To build and push with + Docker Hub, run these commands replacing `{username}` with your Docker Hub + username: + + ```shell + # Build the container on your local machine + docker build -t {username}/helloworld-python . + + # Push the container to docker registry + docker push {username}/helloworld-python + ``` + +1. After the build has completed and the container is pushed to Docker Hub, you + can deploy the sample application into your cluster. Ensure that the container image value + in `sample-app.yaml` matches the container you built in the previous step. Apply + the configuration using `kubectl`: + + ```shell + kubectl apply --filename sample-app.yaml + ``` + 1. Above command created a namespace `knative-samples` and labelled it with `knative-eventing-injection=enabled`, to enable eventing in the namespace. Verify using the following command: + ```shell + kubectl get ns knative-samples --show-labels + ``` + 1. It deployed the helloworld-python app as a K8s Deployment and created a K8s service names helloworld-python. Verify using the following command. + ```shell + kubectl --namespace knative-samples get deployments helloworld-python + kubectl --namespace knative-samples get svc helloworld-python + ``` + 1. It created a Knative Eventing Trigger to route certain events to the helloworld-python application. Make sure that Ready=true + ```shell + kubectl --namespace knative-samples get trigger helloworld-python + ``` +## Send and verify CloudEvents +After you have deployed the application, and have verified that the namespace, sample application and trigger are ready, you can send a CloudEvent. + +### Send CloudEvent to the Broker +You can send an HTTP request directly to the Knative [broker](../../../broker-trigger.md) if the correct CloudEvent headers are set. + + 1. Deploy a curl pod and SSH into it + ```shell + kubectl --namespace knative-samples run curl --image=radial/busyboxplus:curl -it + ``` + 1. Run the following in the SSH terminal + ```shell + curl -v "default-broker.knative-samples.svc.cluster.local" \ + -X POST \ + -H "Ce-Id: 536808d3-88be-4077-9d7a-a3f162705f79" \ + -H "Ce-specversion: 0.3" \ + -H "Ce-Type: dev.knative.samples.helloworld" \ + -H "Ce-Source: dev.knative.samples/helloworldsource" \ + -H "Content-Type: application/json" \ + -d '{"msg":"Hello World from the curl pod."}' + + exit + ``` +### Verify that event is received by helloworld-python app +Helloworld-python app logs the context and the msg of the above event, and replies back with another event. + 1. Display helloworld-python app logs + ```shell + kubectl --namespace knative-samples logs -l app=helloworld-python --tail=50 + ``` + You should see something similar to: + ```shell + Event received. Context: Context Attributes, + specversion: 0.3 + type: dev.knative.samples.helloworld + source: dev.knative.samples/helloworldsource + id: 536808d3-88be-4077-9d7a-a3f162705f79 + time: 2019-10-04T22:35:26.05871736Z + datacontenttype: application/json + Extensions, + knativearrivaltime: 2019-10-04T22:35:26Z + knativehistory: default-kn2-trigger-kn-channel.knative-samples.svc.cluster.local + traceparent: 00-971d4644229653483d38c46e92a959c7-92c66312e4bb39be-00 + + Hello World Message "Hello World from the curl pod." + Responded with event Validation: valid + Context Attributes, + specversion: 0.2 + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + id: 37458d77-01f5-411e-a243-a459bbf79682 + Data, + {"msg":"Hi from Knative!"} + + ``` + Try the CloudEvent attributes in the curl command and the trigger specification to understand how [triggers](../../../broker-trigger.md#trigger) work. + +## Verify reply from helloworld-python app +The `helloworld-python` app replies with an event type `type= dev.knative.samples.hifromknative`, and source `source=knative/eventing/samples/hello-world`. The event enters the eventing mesh through the broker, and can be delivered to event sinks using a trigger + + 1. Deploy a pod that receives any CloudEvent and logs the event to its output. + ```shell + kubectl --namespace knative-samples apply --filename - << END + # event-display app deploment + apiVersion: apps/v1 + kind: Deployment + metadata: + name: event-display + namespace: knative-samples + spec: + replicas: 1 + selector: + matchLabels: &labels + app: event-display + template: + metadata: + labels: *labels + spec: + containers: + - name: helloworld-python + image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display + --- + # Service that exposes event-display app. + # This will be the subscriber for the Trigger + kind: Service + apiVersion: v1 + metadata: + name: event-display + namespace: knative-samples + spec: + selector: + app: event-display + ports: + - protocol: TCP + port: 80 + targetPort: 8080 + END + ``` + 1. Create a trigger to deliver the event to the above service + ```shell + kubectl --namespace knative-samples apply --filename - << END + apiVersion: eventing.knative.dev/v1 + kind: Trigger + metadata: + name: event-display + namespace: knative-samples + spec: + broker: default + filter: + attributes: + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + subscriber: + ref: + apiVersion: v1 + kind: Service + name: event-display + END + ``` + + 1. [Send a CloudEvent to the Broker](###Send-CloudEvent-to-the-Broker) + + 1. Check the logs of event-display service + ```shell + kubectl --namespace knative-samples logs -l app=event-display --tail=50 + ``` + You should see something similar to: + ```shell + cloudevents.Event + Validation: valid + Context Attributes, + specversion: 0.3 + type: dev.knative.samples.hifromknative + source: knative/eventing/samples/hello-world + id: 8a7384b9-8bbe-4634-bf0f-ead07e450b2a + time: 2019-10-04T22:53:39.844943931Z + datacontenttype: application/json + Extensions, + knativearrivaltime: 2019-10-04T22:53:39Z + knativehistory: default-kn2-ingress-kn-channel.knative-samples.svc.cluster.local + traceparent: 00-4b01db030b9ea04bb150b77c8fa86509-2740816590a7604f-00 + Data, + { + "msg": "Hi from helloworld- app!" + } + ``` + + **Note: You could use the above approach to test your applications too.** + + +## Removing the sample app deployment + +To remove the sample app from your cluster, delete the service record: + +```shell +kubectl delete --filename sample-app.yaml +``` diff --git a/docs/eventing/samples/iot-core/README.md b/docs/eventing/samples/iot-core/README.md deleted file mode 100644 index a49a0e8ed23..00000000000 --- a/docs/eventing/samples/iot-core/README.md +++ /dev/null @@ -1,216 +0,0 @@ -This sample shows how to bind a running service to an -[IoT core](https://cloud.google.com/iot-core/) using -[GCP PubSub](https://cloud.google.com/pubsub/) as the event source. With minor -modifications, it can be used to bind a running service to anything that sends -events via GCP PubSub. - -``` -Note: All commands are given relative to the root of this repository. -``` - -## Deployment Steps - -### Environment Variables - -To make the following commands easier, we are going to set the various variables -here and use them later. - -#### Variables you must Change - -```shell -export IOTCORE_PROJECT="s9-demo" -``` - -#### Variables you may Change - -```shell -export IOTCORE_REGISTRY="iot-demo" -export IOTCORE_DEVICE="iot-demo-client" -export IOTCORE_REGION="us-central1" -export IOTCORE_TOPIC_DATA="iot-demo-pubsub-topic" -export IOTCORE_TOPIC_DEVICE="iot-demo-device-pubsub-topic" -``` - -### Prerequisites - -#### Kubernetes - -1. Have a running Kubernetes cluster with `kubectl` pointing at it. - -#### GCP - -1. Create a - [Google Cloud Project](https://cloud.google.com/resource-manager/docs/creating-managing-projects). - -1. Have [gcloud](https://cloud.google.com/sdk/gcloud/) installed and pointing - at that project. - -1. Enable the `Cloud Pub/Sub API` on that project. - - ```shell - gcloud services enable pubsub.googleapis.com - ``` - -1. Create the two GCP PubSub `topic`s. - - ```shell - gcloud pubsub topics create $IOTCORE_TOPIC_DATA - gcloud pubsub topics create $IOTCORE_TOPIC_DEVICE - ``` - -1. Setup [Knative Eventing](../../../eventing). - -#### GCP PubSub Source - -1. Create a GCP - [Service Account](https://console.cloud.google.com/iam-admin/serviceaccounts/project). - - 1. Determine the Service Account to use, or create a new one. - 1. Give that Service Account the 'Pub/Sub Editor' role on your GCP project. - 1. Download a new JSON private key for that Service Account. - 1. Create two secrets with the downloaded key (one for the Source, one for - the Receive Adapter): - - ```shell - kubectl --namespace knative-sources create secret generic gcppubsub-source-key --from-file=key.json=PATH_TO_KEY_FILE.json - kubectl --namespace default create secret generic google-cloud-key --from-file=key.json=PATH_TO_KEY_FILE.json - ``` - -1. Deploy the `GcpPubSubSource` controller as part of eventing-source's - controller. - - ```shell - kubectl apply --filename https://github.com/knative/eventing-contrib/releases/download/v0.8.2/gcppubsub.yaml - ``` - -### Deploying - -#### Broker - -1. Install the default `Broker`. - - ```shell - kubectl create -f - < ./root-ca.pem - ``` - -1. Register a device using the generated certificates. - - ```shell - gcloud iot devices create $IOTCORE_DEVICE \ - --project=$IOTCORE_PROJECT \ - --region=$IOTCORE_REGION \ - --registry=$IOTCORE_REGISTRY \ - --public-key path=./device.crt.pem,type=rsa-x509-pem - ``` - -### Running - -We now have everything installed and ready to go. We will generate events and -see them in the subscriber. - -1. Run the following program to generate events: - - ```shell - go run github.com/knative/docs/docs/eventing/samples/iot-core/generator \ - -project $IOTCORE_PROJECT \ - -region $IOTCORE_REGION \ - -registry $IOTCORE_REGISTRY \ - -device $IOTCORE_DEVICE \ - -ca "$PWD/root-ca.pem" \ - -key "$PWD/device.key.pem" \ - -src "iot-core demo" \ - -events 10 - ``` - -1. Inspect the logs of the subscriber: - - ```shell - kubectl logs --selector serving.knative.dev/service=event-display -c user-container - ``` - - You should see something along the similar to: - - ```shell - {"ID":"481014114648052","Data":"eyJzb3VyY2VfaWQiOiJpb3QtY29yZSBkZW1vIiwiZXZlbnRfaWQiOiJlaWQtMzI3MjJiMzItZWU5Mi00YzZlLWEzOTgtNDlmYjRkYWYyNGE1IiwiZXZlbnRfdHMiOjE1NTM3MTczOTYsIm1ldHJpYyI6MC4xMzY1MjI5OH0=","Attributes":{"deviceId":"iot-demo-client","deviceNumId":"2754785852315736","deviceRegistryId":"iot-demo","deviceRegistryLocation":"us-central1","projectId":"s9-demo","subFolder":""},"PublishTime":"2019-03-27T20:09:56.685Z"} - ``` - -### Cleanup - -To cleanup the knative resources: - -1. Remove the `GcpPubSubSource`: - - ```shell - sed -e "s/PROJECT_ID/$IOTCORE_PROJECT/" \ - -e "s/TOPIC_NAME/$IOTCORE_TOPIC_DATA/" \ - docs/eventing/samples/iot-core/gcp-pubsub-source.yaml | - kubectl delete --filename - - ``` - -1. Remove the Trigger: - - ```shell - kubectl delete --filename docs/eventing/samples/iot-core/trigger.yaml - ``` - -1. Remove the `GcpPubSubSource` controller: - - ```shell - kubectl delete --filename https://github.com/knative/eventing-contrib/releases/download/v0.8.2/gcppubsub.yaml - ``` diff --git a/docs/eventing/samples/iot-core/index.md b/docs/eventing/samples/iot-core/index.md index 1730a67f102..9619c0ea80b 100644 --- a/docs/eventing/samples/iot-core/index.md +++ b/docs/eventing/samples/iot-core/index.md @@ -5,4 +5,219 @@ weight: 40 type: "docs" --- -{{% readfile file="README.md" %}} +This sample shows how to bind a running service to an +[IoT core](https://cloud.google.com/iot-core/) using +[GCP PubSub](https://cloud.google.com/pubsub/) as the event source. With minor +modifications, it can be used to bind a running service to anything that sends +events via GCP PubSub. + +``` +Note: All commands are given relative to the root of this repository. +``` + +## Deployment Steps + +### Environment Variables + +To make the following commands easier, we are going to set the various variables +here and use them later. + +#### Variables you must Change + +```shell +export IOTCORE_PROJECT="s9-demo" +``` + +#### Variables you may Change + +```shell +export IOTCORE_REGISTRY="iot-demo" +export IOTCORE_DEVICE="iot-demo-client" +export IOTCORE_REGION="us-central1" +export IOTCORE_TOPIC_DATA="iot-demo-pubsub-topic" +export IOTCORE_TOPIC_DEVICE="iot-demo-device-pubsub-topic" +``` + +### Prerequisites + +#### Kubernetes + +1. Have a running Kubernetes cluster with `kubectl` pointing at it. + +#### GCP + +1. Create a + [Google Cloud Project](https://cloud.google.com/resource-manager/docs/creating-managing-projects). + +1. Have [gcloud](https://cloud.google.com/sdk/gcloud/) installed and pointing + at that project. + +1. Enable the `Cloud Pub/Sub API` on that project. + + ```shell + gcloud services enable pubsub.googleapis.com + ``` + +1. Create the two GCP PubSub `topic`s. + + ```shell + gcloud pubsub topics create $IOTCORE_TOPIC_DATA + gcloud pubsub topics create $IOTCORE_TOPIC_DEVICE + ``` + +1. Setup [Knative Eventing](../../../eventing). + +#### GCP PubSub Source + +1. Create a GCP + [Service Account](https://console.cloud.google.com/iam-admin/serviceaccounts/project). + + 1. Determine the Service Account to use, or create a new one. + 1. Give that Service Account the 'Pub/Sub Editor' role on your GCP project. + 1. Download a new JSON private key for that Service Account. + 1. Create two secrets with the downloaded key (one for the Source, one for + the Receive Adapter): + + ```shell + kubectl --namespace knative-sources create secret generic gcppubsub-source-key --from-file=key.json=PATH_TO_KEY_FILE.json + kubectl --namespace default create secret generic google-cloud-key --from-file=key.json=PATH_TO_KEY_FILE.json + ``` + +1. Deploy the `GcpPubSubSource` controller as part of eventing-source's + controller. + + ```shell + kubectl apply --filename https://github.com/knative/eventing-contrib/releases/download/v0.8.2/gcppubsub.yaml + ``` + +### Deploying + +#### Broker + +1. Install the default `Broker`. + + ```shell + kubectl create -f - < ./root-ca.pem + ``` + +1. Register a device using the generated certificates. + + ```shell + gcloud iot devices create $IOTCORE_DEVICE \ + --project=$IOTCORE_PROJECT \ + --region=$IOTCORE_REGION \ + --registry=$IOTCORE_REGISTRY \ + --public-key path=./device.crt.pem,type=rsa-x509-pem + ``` + +### Running + +We now have everything installed and ready to go. We will generate events and +see them in the subscriber. + +1. Run the following program to generate events: + + ```shell + go run github.com/knative/docs/docs/eventing/samples/iot-core/generator \ + -project $IOTCORE_PROJECT \ + -region $IOTCORE_REGION \ + -registry $IOTCORE_REGISTRY \ + -device $IOTCORE_DEVICE \ + -ca "$PWD/root-ca.pem" \ + -key "$PWD/device.key.pem" \ + -src "iot-core demo" \ + -events 10 + ``` + +1. Inspect the logs of the subscriber: + + ```shell + kubectl logs --selector serving.knative.dev/service=event-display -c user-container + ``` + + You should see something along the similar to: + + ```shell + {"ID":"481014114648052","Data":"eyJzb3VyY2VfaWQiOiJpb3QtY29yZSBkZW1vIiwiZXZlbnRfaWQiOiJlaWQtMzI3MjJiMzItZWU5Mi00YzZlLWEzOTgtNDlmYjRkYWYyNGE1IiwiZXZlbnRfdHMiOjE1NTM3MTczOTYsIm1ldHJpYyI6MC4xMzY1MjI5OH0=","Attributes":{"deviceId":"iot-demo-client","deviceNumId":"2754785852315736","deviceRegistryId":"iot-demo","deviceRegistryLocation":"us-central1","projectId":"s9-demo","subFolder":""},"PublishTime":"2019-03-27T20:09:56.685Z"} + ``` + +### Cleanup + +To cleanup the knative resources: + +1. Remove the `GcpPubSubSource`: + + ```shell + sed -e "s/PROJECT_ID/$IOTCORE_PROJECT/" \ + -e "s/TOPIC_NAME/$IOTCORE_TOPIC_DATA/" \ + docs/eventing/samples/iot-core/gcp-pubsub-source.yaml | + kubectl delete --filename - + ``` + +1. Remove the Trigger: + + ```shell + kubectl delete --filename docs/eventing/samples/iot-core/trigger.yaml + ``` + +1. Remove the `GcpPubSubSource` controller: + + ```shell + kubectl delete --filename https://github.com/knative/eventing-contrib/releases/download/v0.8.2/gcppubsub.yaml + ``` diff --git a/docs/eventing/samples/kafka/README.md b/docs/eventing/samples/kafka/README.md deleted file mode 100644 index dc71a28225c..00000000000 --- a/docs/eventing/samples/kafka/README.md +++ /dev/null @@ -1,95 +0,0 @@ -The following examples will help you understand how to use the different Apache -Kafka components for Knative. - -## Prerequisites - -All examples require: - -- A Kubernetes cluster with - - Knative Eventing v0.9+ - - Knative Serving v0.9+ -- An Apache Kafka cluster - -### Setting up Apache Kafka - -If you want to run the Apache Kafka cluster on Kubernetes, the simplest option -is to install it by using [Strimzi](https://strimzi.io). - -1. Create a namespace for your Apache Kafka installation, like `kafka`: - ```shell - kubectl create namespace kafka - ``` -1. Install the Strimzi operator, like: - ```shell - curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.16.2/strimzi-cluster-operator-0.16.2.yaml" \ - | sed 's/namespace: .*/namespace: kafka/' \ - | kubectl -n kafka apply -f - - ``` -1. Describe the size of your Apache Kafka installation in `kafka.yaml`, like: - ```yaml - apiVersion: kafka.strimzi.io/v1beta1 - kind: Kafka - metadata: - name: my-cluster - spec: - kafka: - version: 2.4.0 - replicas: 1 - listeners: - plain: {} - tls: {} - config: - offsets.topic.replication.factor: 1 - transaction.state.log.replication.factor: 1 - transaction.state.log.min.isr: 1 - log.message.format.version: "2.4" - storage: - type: ephemeral - zookeeper: - replicas: 3 - storage: - type: ephemeral - entityOperator: - topicOperator: {} - userOperator: {} - ``` -1. Deploy the Apache Kafka cluster - ``` - $ kubectl apply -n kafka -f kafka.yaml - ``` - -This will install a small, non-production, cluster of Apache Kafka. To verify -your installation, check if the pods for Strimzi are all up, in the `kafka` -namespace: - -```shell -$ kubectl get pods -n kafka -NAME READY STATUS RESTARTS AGE -my-cluster-entity-operator-65995cf856-ld2zp 3/3 Running 0 102s -my-cluster-kafka-0 2/2 Running 0 2m8s -my-cluster-zookeeper-0 2/2 Running 0 2m39s -my-cluster-zookeeper-1 2/2 Running 0 2m49s -my-cluster-zookeeper-2 2/2 Running 0 2m59s -strimzi-cluster-operator-77555d4b69-sbrt4 1/1 Running 0 3m14s -``` - -> NOTE: For production ready installs check [Strimzi](https://strimzi.io). - -### Installation script - -If you want to install the latest version of Strimzi, in just one step, we have -a [script](./kafka_setup.sh) for your convenience, which does exactly the same -steps that are listed above: - -```shell -$ ./kafka_setup.sh -``` - -## Examples of Apache Kafka and Knative - -A number of different examples, showing the `KafkaSource`, `KafkaChannel` and -`KafkaBinding` can be found here: - -- [`KafkaSource` to `Service`](./source/README.md) -- [`KafkaChannel` and Broker](./channel/README.md) -- [`KafkaBinding`](./binding/README.md) diff --git a/docs/eventing/samples/kafka/_index.md b/docs/eventing/samples/kafka/_index.md index afaf74da5d3..4702fb6d20f 100644 --- a/docs/eventing/samples/kafka/_index.md +++ b/docs/eventing/samples/kafka/_index.md @@ -5,4 +5,98 @@ weight: 10 type: "docs" --- -{{% readfile file="README.md" %}} +The following examples will help you understand how to use the different Apache +Kafka components for Knative. + +## Prerequisites + +All examples require: + +- A Kubernetes cluster with + - Knative Eventing v0.9+ + - Knative Serving v0.9+ +- An Apache Kafka cluster + +### Setting up Apache Kafka + +If you want to run the Apache Kafka cluster on Kubernetes, the simplest option +is to install it by using [Strimzi](https://strimzi.io). + +1. Create a namespace for your Apache Kafka installation, like `kafka`: + ```shell + kubectl create namespace kafka + ``` +1. Install the Strimzi operator, like: + ```shell + curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.16.2/strimzi-cluster-operator-0.16.2.yaml" \ + | sed 's/namespace: .*/namespace: kafka/' \ + | kubectl -n kafka apply -f - + ``` +1. Describe the size of your Apache Kafka installation in `kafka.yaml`, like: + ```yaml + apiVersion: kafka.strimzi.io/v1beta1 + kind: Kafka + metadata: + name: my-cluster + spec: + kafka: + version: 2.4.0 + replicas: 1 + listeners: + plain: {} + tls: {} + config: + offsets.topic.replication.factor: 1 + transaction.state.log.replication.factor: 1 + transaction.state.log.min.isr: 1 + log.message.format.version: "2.4" + storage: + type: ephemeral + zookeeper: + replicas: 3 + storage: + type: ephemeral + entityOperator: + topicOperator: {} + userOperator: {} + ``` +1. Deploy the Apache Kafka cluster + ``` + $ kubectl apply -n kafka -f kafka.yaml + ``` + +This will install a small, non-production, cluster of Apache Kafka. To verify +your installation, check if the pods for Strimzi are all up, in the `kafka` +namespace: + +```shell +$ kubectl get pods -n kafka +NAME READY STATUS RESTARTS AGE +my-cluster-entity-operator-65995cf856-ld2zp 3/3 Running 0 102s +my-cluster-kafka-0 2/2 Running 0 2m8s +my-cluster-zookeeper-0 2/2 Running 0 2m39s +my-cluster-zookeeper-1 2/2 Running 0 2m49s +my-cluster-zookeeper-2 2/2 Running 0 2m59s +strimzi-cluster-operator-77555d4b69-sbrt4 1/1 Running 0 3m14s +``` + +> NOTE: For production ready installs check [Strimzi](https://strimzi.io). + +### Installation script + +If you want to install the latest version of Strimzi, in just one step, we have +a [script](./kafka_setup.sh) for your convenience, which does exactly the same +steps that are listed above: + +```shell +$ ./kafka_setup.sh +``` + +## Examples of Apache Kafka and Knative + +A number of different examples, showing the `KafkaSource`, `KafkaChannel` and +`KafkaBinding` can be found here: + +- [`KafkaSource` to `Service`](./source/README.md) +- [`KafkaChannel` and Broker](./channel/README.md) +- [`KafkaBinding`](./binding/README.md) diff --git a/docs/eventing/samples/kafka/binding/README.md b/docs/eventing/samples/kafka/binding/README.md deleted file mode 100644 index 4c72edd368a..00000000000 --- a/docs/eventing/samples/kafka/binding/README.md +++ /dev/null @@ -1,269 +0,0 @@ -KafkaBinding is responsible for injecting Kafka bootstrap connection information -into a Kubernetes resource that embed a PodSpec (as `spec.template.spec`). This -enables easy bootstrapping of a Kafka client. - -## Create a Job that uses KafkaBinding - -In the below example a Kubernetes Job will be using the KafkaBinding to produce -messages on a Kafka Topic, which will be received by the Event Display service -via Kafka Source - -### Prerequisites - -1. You must ensure that you meet the - [prerequisites listed in the Apache Kafka overview](../README.md). -2. This feature is available from Knative Eventing 0.15+ - -### Creating a `KafkaSource` source CRD - -1. Install the `KafkaSource` sub-component to your Knative cluster: - - ``` - kubectl apply -f https://storage.googleapis.com/knative-releases/eventing-contrib/latest/kafka-source.yaml - - ``` - -1. Check that the `kafka-controller-manager-0` pod is running. - ``` - kubectl get pods --namespace knative-sources - NAME READY STATUS RESTARTS AGE - kafka-controller-manager-0 1/1 Running 0 42m - ``` - -### Create the Event Display service - -1. (Optional) Source code for Event Display service - - Get the source code of Event Display container image from - [here](https://github.com/knative/eventing-contrib/blob/main/cmd/event_display/main.go) - -1. Deploy the Event Display Service via kubectl: - - ```yaml - apiVersion: serving.knative.dev/v1 - kind: Service - metadata: - name: event-display - spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display - ``` - - ``` - $ kubectl apply --filename event-display.yaml - ... - service.serving.knative.dev/event-display created - ``` - -1. (Optional) Deploy the Event Display Service via kn cli: - - Alternatively, you can create the knative service using the `kn` cli like - below - - ``` - kn service create event-display --image=gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display - ``` - -1. Ensure that the Service pod is running. The pod name will be prefixed with - `event-display`. - ``` - $ kubectl get pods - NAME READY STATUS RESTARTS AGE - event-display-00001-deployment-5d5df6c7-gv2j4 2/2 Running 0 72s - ... - ``` - -### Apache Kafka Event Source - -1. Modify `event-source.yaml` accordingly with bootstrap servers, topics, - etc...: - - ```yaml - apiVersion: sources.knative.dev/v1beta1 - kind: KafkaSource - metadata: - name: kafka-source - spec: - consumerGroup: knative-group - bootstrapServers: - - my-cluster-kafka-bootstrap.kafka:9092 #note the kafka namespace - topics: - - logs - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display - ``` - -1. Deploy the event source. - ``` - $ kubectl apply -f event-source.yaml - ... - kafkasource.sources.knative.dev/kafka-source created - ``` -1. Check that the event source pod is running. The pod name will be prefixed - with `kafka-source`. - ``` - $ kubectl get pods - NAME READY STATUS RESTARTS AGE - kafka-source-xlnhq-5544766765-dnl5s 1/1 Running 0 40m - ``` - -### Kafka Binding Resource - -Create the KafkaBinding that will inject kafka bootstrap information into select -`Jobs`: - -1. Modify `kafka-binding.yaml` accordingly with bootstrap servers etc...: - - ```yaml - apiVersion: bindings.knative.dev/v1beta1 - kind: KafkaBinding - metadata: - name: kafka-binding-test - spec: - subject: - apiVersion: batch/v1 - kind: Job - selector: - matchLabels: - kafka.topic: "logs" - bootstrapServers: - - my-cluster-kafka-bootstrap.kafka:9092 - ``` - -In this case, we will bind any `Job` with the labels `kafka.topic: "logs"`. - -### Create Kubernetes Job - -1. Source code for kafka-publisher service - - Get the source code of kafka-publisher container image from - [here](https://github.com/knative-sandbox/eventing-kafka/blob/main/test/test_images/kafka-publisher/main.go) - -1. Now we will use the kafka-publisher container to send events to kafka topic - when the Job runs. - - ```yaml - apiVersion: batch/v1 - kind: Job - metadata: - labels: - kafka.topic: "logs" - name: kafka-publisher-job - spec: - backoffLimit: 1 - completions: 1 - parallelism: 1 - template: - metadata: - annotations: - sidecar.istio.io/inject: "false" - spec: - restartPolicy: Never - containers: - - image: docker.io/murugappans/kafka-publisher-1974f83e2ff7c8994707b5e8731528e8@sha256:fd79490514053c643617dc72a43097251fed139c966fd5d131134a0e424882de - env: - - name: KAFKA_TOPIC - value: "logs" - - name: KAFKA_KEY - value: "0" - - name: KAFKA_HEADERS - value: "content-type:application/json" - - name: KAFKA_VALUE - value: '{"msg":"This is a test!"}' - name: kafka-publisher - ``` -1. Check that the Job has run successfully. - ``` - $ kubectl get jobs - NAME COMPLETIONS DURATION AGE - kafka-publisher-job 1/1 7s 7s - ``` - -### Verify - -1. Ensure the Event Display received the message sent to it by the Event Source. - - ``` - $ kubectl logs --selector='serving.knative.dev/service=event-display' -c user-container - - ☁️ cloudevents.Event - Validation: valid - Context Attributes, - specversion: 1.0 - type: dev.knative.kafka.event - source: /apis/v1/namespaces/default/kafkasources/kafka-source#logs - subject: partition:0#1 - id: partition:0/offset:1 - time: 2020-05-17T19:45:02.7Z - datacontenttype: application/json - Extensions, - kafkaheadercontenttype: application/json - key: 0 - traceparent: 00-f383b779f512358b24ffbf6556a6d6da-cacdbe78ef9b5ad3-00 - Data, - { - "msg": "This is a test!" - } - - ``` - -## Connecting to a TLS enabled Kafka broker - -The KafkaBinding supports TLS and SASL authentication methods. For injecting TLS -authentication, please have the below files - -- CA Certificate -- Client Certificate and Key - -These files are expected to be in pem format, if it is in other format like jks -, please convert to pem. - -1. Create the certificate files as secrets in the namespace where KafkaBinding - is going to be set up - - ``` - $ kubectl create secret generic cacert --from-file=caroot.pem - secret/cacert created - - $ kubectl create secret tls kafka-secret --cert=certificate.pem --key=key.pem - secret/key created - - ``` - -2. Apply the kafkabinding-tls.yaml, change bootstrapServers accordingly. - ```yaml - apiVersion: sources.knative.dev/v1beta1 - kind: KafkaBinding - metadata: - name: kafka-source-with-tls - spec: - subject: - apiVersion: batch/v1 - kind: Job - selector: - matchLabels: - kafka.topic: "logs" - net: - tls: - enable: true - cert: - secretKeyRef: - key: tls.crt - name: kafka-secret - key: - secretKeyRef: - key: tls.key - name: kafka-secret - caCert: - secretKeyRef: - key: caroot.pem - name: cacert - consumerGroup: knative-group - bootstrapServers: - - my-secure-kafka-bootstrap.kafka:443 - ``` diff --git a/docs/eventing/samples/kafka/binding/index.md b/docs/eventing/samples/kafka/binding/index.md index b84b2e5356f..cbc038954b0 100644 --- a/docs/eventing/samples/kafka/binding/index.md +++ b/docs/eventing/samples/kafka/binding/index.md @@ -5,4 +5,272 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +KafkaBinding is responsible for injecting Kafka bootstrap connection information +into a Kubernetes resource that embed a PodSpec (as `spec.template.spec`). This +enables easy bootstrapping of a Kafka client. + +## Create a Job that uses KafkaBinding + +In the below example a Kubernetes Job will be using the KafkaBinding to produce +messages on a Kafka Topic, which will be received by the Event Display service +via Kafka Source + +### Prerequisites + +1. You must ensure that you meet the + [prerequisites listed in the Apache Kafka overview](../README.md). +2. This feature is available from Knative Eventing 0.15+ + +### Creating a `KafkaSource` source CRD + +1. Install the `KafkaSource` sub-component to your Knative cluster: + + ``` + kubectl apply -f https://storage.googleapis.com/knative-releases/eventing-contrib/latest/kafka-source.yaml + + ``` + +1. Check that the `kafka-controller-manager-0` pod is running. + ``` + kubectl get pods --namespace knative-sources + NAME READY STATUS RESTARTS AGE + kafka-controller-manager-0 1/1 Running 0 42m + ``` + +### Create the Event Display service + +1. (Optional) Source code for Event Display service + + Get the source code of Event Display container image from + [here](https://github.com/knative/eventing-contrib/blob/main/cmd/event_display/main.go) + +1. Deploy the Event Display Service via kubectl: + + ```yaml + apiVersion: serving.knative.dev/v1 + kind: Service + metadata: + name: event-display + spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display + ``` + + ``` + $ kubectl apply --filename event-display.yaml + ... + service.serving.knative.dev/event-display created + ``` + +1. (Optional) Deploy the Event Display Service via kn cli: + + Alternatively, you can create the knative service using the `kn` cli like + below + + ``` + kn service create event-display --image=gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display + ``` + +1. Ensure that the Service pod is running. The pod name will be prefixed with + `event-display`. + ``` + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + event-display-00001-deployment-5d5df6c7-gv2j4 2/2 Running 0 72s + ... + ``` + +### Apache Kafka Event Source + +1. Modify `event-source.yaml` accordingly with bootstrap servers, topics, + etc...: + + ```yaml + apiVersion: sources.knative.dev/v1beta1 + kind: KafkaSource + metadata: + name: kafka-source + spec: + consumerGroup: knative-group + bootstrapServers: + - my-cluster-kafka-bootstrap.kafka:9092 #note the kafka namespace + topics: + - logs + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display + ``` + +1. Deploy the event source. + ``` + $ kubectl apply -f event-source.yaml + ... + kafkasource.sources.knative.dev/kafka-source created + ``` +1. Check that the event source pod is running. The pod name will be prefixed + with `kafka-source`. + ``` + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + kafka-source-xlnhq-5544766765-dnl5s 1/1 Running 0 40m + ``` + +### Kafka Binding Resource + +Create the KafkaBinding that will inject kafka bootstrap information into select +`Jobs`: + +1. Modify `kafka-binding.yaml` accordingly with bootstrap servers etc...: + + ```yaml + apiVersion: bindings.knative.dev/v1beta1 + kind: KafkaBinding + metadata: + name: kafka-binding-test + spec: + subject: + apiVersion: batch/v1 + kind: Job + selector: + matchLabels: + kafka.topic: "logs" + bootstrapServers: + - my-cluster-kafka-bootstrap.kafka:9092 + ``` + +In this case, we will bind any `Job` with the labels `kafka.topic: "logs"`. + +### Create Kubernetes Job + +1. Source code for kafka-publisher service + + Get the source code of kafka-publisher container image from + [here](https://github.com/knative-sandbox/eventing-kafka/blob/main/test/test_images/kafka-publisher/main.go) + +1. Now we will use the kafka-publisher container to send events to kafka topic + when the Job runs. + + ```yaml + apiVersion: batch/v1 + kind: Job + metadata: + labels: + kafka.topic: "logs" + name: kafka-publisher-job + spec: + backoffLimit: 1 + completions: 1 + parallelism: 1 + template: + metadata: + annotations: + sidecar.istio.io/inject: "false" + spec: + restartPolicy: Never + containers: + - image: docker.io/murugappans/kafka-publisher-1974f83e2ff7c8994707b5e8731528e8@sha256:fd79490514053c643617dc72a43097251fed139c966fd5d131134a0e424882de + env: + - name: KAFKA_TOPIC + value: "logs" + - name: KAFKA_KEY + value: "0" + - name: KAFKA_HEADERS + value: "content-type:application/json" + - name: KAFKA_VALUE + value: '{"msg":"This is a test!"}' + name: kafka-publisher + ``` +1. Check that the Job has run successfully. + ``` + $ kubectl get jobs + NAME COMPLETIONS DURATION AGE + kafka-publisher-job 1/1 7s 7s + ``` + +### Verify + +1. Ensure the Event Display received the message sent to it by the Event Source. + + ``` + $ kubectl logs --selector='serving.knative.dev/service=event-display' -c user-container + + ☁️ cloudevents.Event + Validation: valid + Context Attributes, + specversion: 1.0 + type: dev.knative.kafka.event + source: /apis/v1/namespaces/default/kafkasources/kafka-source#logs + subject: partition:0#1 + id: partition:0/offset:1 + time: 2020-05-17T19:45:02.7Z + datacontenttype: application/json + Extensions, + kafkaheadercontenttype: application/json + key: 0 + traceparent: 00-f383b779f512358b24ffbf6556a6d6da-cacdbe78ef9b5ad3-00 + Data, + { + "msg": "This is a test!" + } + + ``` + +## Connecting to a TLS enabled Kafka broker + +The KafkaBinding supports TLS and SASL authentication methods. For injecting TLS +authentication, please have the below files + +- CA Certificate +- Client Certificate and Key + +These files are expected to be in pem format, if it is in other format like jks +, please convert to pem. + +1. Create the certificate files as secrets in the namespace where KafkaBinding + is going to be set up + + ``` + $ kubectl create secret generic cacert --from-file=caroot.pem + secret/cacert created + + $ kubectl create secret tls kafka-secret --cert=certificate.pem --key=key.pem + secret/key created + + ``` + +2. Apply the kafkabinding-tls.yaml, change bootstrapServers accordingly. + ```yaml + apiVersion: sources.knative.dev/v1beta1 + kind: KafkaBinding + metadata: + name: kafka-source-with-tls + spec: + subject: + apiVersion: batch/v1 + kind: Job + selector: + matchLabels: + kafka.topic: "logs" + net: + tls: + enable: true + cert: + secretKeyRef: + key: tls.crt + name: kafka-secret + key: + secretKeyRef: + key: tls.key + name: kafka-secret + caCert: + secretKeyRef: + key: caroot.pem + name: cacert + consumerGroup: knative-group + bootstrapServers: + - my-secure-kafka-bootstrap.kafka:443 + ``` diff --git a/docs/eventing/samples/kafka/channel/README.md b/docs/eventing/samples/kafka/channel/README.md deleted file mode 100644 index d75a3a2e23d..00000000000 --- a/docs/eventing/samples/kafka/channel/README.md +++ /dev/null @@ -1,223 +0,0 @@ -You can install and configure the Apache Kafka CRD (`KafkaChannel`) as the -default channel configuration in Knative Eventing. - -## Prerequisites - -- Ensure that you meet the - [prerequisites listed in the Apache Kafka overview](../README.md). -- A Kubernetes cluster with - [Knative Kafka Channel installed](../../../../install/README.md). - -## Creating a `KafkaChannel` channel CRD - -Create a new object by configuring the YAML file as follows: - -``` -cat <<-EOF | kubectl apply -f - ---- -apiVersion: messaging.knative.dev/v1beta1 -kind: KafkaChannel -metadata: - name: my-kafka-channel -spec: - numPartitions: 3 - replicationFactor: 1 -EOF -``` - -## Specifying the default channel configuration - -To configure the usage of the `KafkaChannel` CRD as the -[default channel configuration](../../../channels/default-channels.md), edit the -`default-ch-webhook` ConfigMap as follows: - -``` -cat <<-EOF | kubectl apply -f - ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: default-ch-webhook - namespace: knative-eventing -data: - # Configuration for defaulting channels that do not specify CRD implementations. - default-ch-config: | - clusterDefault: - apiVersion: messaging.knative.dev/v1beta1 - kind: KafkaChannel - spec: - numPartitions: 3 - replicationFactor: 1 -EOF -``` - -## Creating an Apache Kafka channel using the default channel configuration - -Now that `KafkaChannel` is set as the default channel configuration, you can use -the `channels.messaging.knative.dev` CRD to create a new Apache Kafka channel, -using the generic `Channel`: - -``` -cat <<-EOF | kubectl apply -f - ---- -apiVersion: messaging.knative.dev/v1 -kind: Channel -metadata: - name: testchannel-one -EOF -``` - -Check Kafka for a `testchannel` topic. With Strimzi this can be done by using -the command: - -``` -kubectl -n kafka exec -it my-cluster-kafka-0 -- bin/kafka-topics.sh --zookeeper localhost:2181 --list -``` - -The result is: - -``` -... -knative-messaging-kafka.default.testchannel-one -... -``` - -The Apache Kafka topic that is created by the channel implementation is prefixed -with `knative-messaging-kafka`. This indicates it is an Apache Kafka channel -from Knative. It contains the name of the namespace, `default` in this example, -followed by the actual name of the channel. - -## Configuring the Knative broker for Apache Kafka channels - -To setup a broker that will use the new default Kafka channels, you must create -a new _default_ broker, using the command: - -```shell -kubectl create -f - < generic \ - --from-file=ca.crt=caroot.pem \ - --from-file=user.crt=certificate.pem \ - --from-file=user.key=key.pem - ``` - -*NOTE:* It is important to use the same keys (`ca.crt`, `user.crt` and `user.key`). - -Reference your secret and the namespace of the secret in the `config-kafka` ConfigMap: -``` -apiVersion: v1 -kind: ConfigMap -metadata: - name: config-kafka - namespace: knative-eventing -data: - bootstrapServers: - authSecretName: - authSecretNamespace: - ``` - -### SASL authentication - -To use SASL authentication, you will need the following information: - -* A username and password. -* The type of SASL mechanism you wish to use. For example; `PLAIN`, `SCRAM-SHA-256` or `SCRAM-SHA-512`. - -**NOTE:** It is recommended to also enable TLS. If you enable this, you will also need the `ca.crt` certificate as described in the previous section. - -1. Create the certificate files as secrets in your chosen namespace: -``` -$ kubectl create secret --namespace generic \ - --from-file=ca.crt=caroot.pem \ - --from-literal=password="SecretPassword" \ - --from-literal=saslType="SCRAM-SHA-512" \ - --from-literal=user="my-sasl-user" -``` - -*NOTE:* It is important to use the same keys; `user`, `password` and `saslType`. - -Reference your secret and the namespace of the secret in the `config-kafka` ConfigMap: -``` -apiVersion: v1 -kind: ConfigMap -metadata: - name: config-kafka - namespace: knative-eventing -data: - bootstrapServers: - authSecretName: - authSecretNamespace: -``` diff --git a/docs/eventing/samples/kafka/channel/index.md b/docs/eventing/samples/kafka/channel/index.md index dc2f1683de8..ace6f326361 100644 --- a/docs/eventing/samples/kafka/channel/index.md +++ b/docs/eventing/samples/kafka/channel/index.md @@ -5,4 +5,226 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +You can install and configure the Apache Kafka CRD (`KafkaChannel`) as the +default channel configuration in Knative Eventing. + +## Prerequisites + +- Ensure that you meet the + [prerequisites listed in the Apache Kafka overview](../README.md). +- A Kubernetes cluster with + [Knative Kafka Channel installed](../../../../install/README.md). + +## Creating a `KafkaChannel` channel CRD + +Create a new object by configuring the YAML file as follows: + +``` +cat <<-EOF | kubectl apply -f - +--- +apiVersion: messaging.knative.dev/v1beta1 +kind: KafkaChannel +metadata: + name: my-kafka-channel +spec: + numPartitions: 3 + replicationFactor: 1 +EOF +``` + +## Specifying the default channel configuration + +To configure the usage of the `KafkaChannel` CRD as the +[default channel configuration](../../../channels/default-channels.md), edit the +`default-ch-webhook` ConfigMap as follows: + +``` +cat <<-EOF | kubectl apply -f - +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: default-ch-webhook + namespace: knative-eventing +data: + # Configuration for defaulting channels that do not specify CRD implementations. + default-ch-config: | + clusterDefault: + apiVersion: messaging.knative.dev/v1beta1 + kind: KafkaChannel + spec: + numPartitions: 3 + replicationFactor: 1 +EOF +``` + +## Creating an Apache Kafka channel using the default channel configuration + +Now that `KafkaChannel` is set as the default channel configuration, you can use +the `channels.messaging.knative.dev` CRD to create a new Apache Kafka channel, +using the generic `Channel`: + +``` +cat <<-EOF | kubectl apply -f - +--- +apiVersion: messaging.knative.dev/v1 +kind: Channel +metadata: + name: testchannel-one +EOF +``` + +Check Kafka for a `testchannel` topic. With Strimzi this can be done by using +the command: + +``` +kubectl -n kafka exec -it my-cluster-kafka-0 -- bin/kafka-topics.sh --zookeeper localhost:2181 --list +``` + +The result is: + +``` +... +knative-messaging-kafka.default.testchannel-one +... +``` + +The Apache Kafka topic that is created by the channel implementation is prefixed +with `knative-messaging-kafka`. This indicates it is an Apache Kafka channel +from Knative. It contains the name of the namespace, `default` in this example, +followed by the actual name of the channel. + +## Configuring the Knative broker for Apache Kafka channels + +To setup a broker that will use the new default Kafka channels, you must create +a new _default_ broker, using the command: + +```shell +kubectl create -f - < generic \ + --from-file=ca.crt=caroot.pem \ + --from-file=user.crt=certificate.pem \ + --from-file=user.key=key.pem + ``` + +*NOTE:* It is important to use the same keys (`ca.crt`, `user.crt` and `user.key`). + +Reference your secret and the namespace of the secret in the `config-kafka` ConfigMap: +``` +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-kafka + namespace: knative-eventing +data: + bootstrapServers: + authSecretName: + authSecretNamespace: + ``` + +### SASL authentication + +To use SASL authentication, you will need the following information: + +* A username and password. +* The type of SASL mechanism you wish to use. For example; `PLAIN`, `SCRAM-SHA-256` or `SCRAM-SHA-512`. + +**NOTE:** It is recommended to also enable TLS. If you enable this, you will also need the `ca.crt` certificate as described in the previous section. + +1. Create the certificate files as secrets in your chosen namespace: +``` +$ kubectl create secret --namespace generic \ + --from-file=ca.crt=caroot.pem \ + --from-literal=password="SecretPassword" \ + --from-literal=saslType="SCRAM-SHA-512" \ + --from-literal=user="my-sasl-user" +``` + +*NOTE:* It is important to use the same keys; `user`, `password` and `saslType`. + +Reference your secret and the namespace of the secret in the `config-kafka` ConfigMap: +``` +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-kafka + namespace: knative-eventing +data: + bootstrapServers: + authSecretName: + authSecretNamespace: +``` diff --git a/docs/eventing/samples/kafka/source/README.md b/docs/eventing/samples/kafka/source/README.md deleted file mode 100644 index a1fffa39f7c..00000000000 --- a/docs/eventing/samples/kafka/source/README.md +++ /dev/null @@ -1,329 +0,0 @@ -Tutorial on how to build and deploy a `KafkaSource` [Eventing source](../../../sources/README.md) using a Knative Serving `Service`. - -## Prerequisites - -- Ensure that you meet the [prerequisites listed in the Apache Kafka overview](../README.md). -- A Kubernetes cluster with [Knative Kafka Source installed](../../../../install/README.md). - -## Apache Kafka Topic (Optional) - -1. If using Strimzi, you can set a topic modifying `source/kafka-topic.yaml` - with your desired: - -- Topic -- Cluster Name -- Partitions -- Replicas - - ```yaml - apiVersion: kafka.strimzi.io/v1beta1 - kind: KafkaTopic - metadata: - name: knative-demo-topic - namespace: kafka - labels: - strimzi.io/cluster: my-cluster - spec: - partitions: 3 - replicas: 1 - config: - retention.ms: 7200000 - segment.bytes: 1073741824 - ``` - -2. Deploy the `KafkaTopic` - - ```shell - $ kubectl apply -f strimzi-topic.yaml - kafkatopic.kafka.strimzi.io/knative-demo-topic created - ``` - -3. Ensure the `KafkaTopic` is running. - - ```shell - $ kubectl -n kafka get kafkatopics.kafka.strimzi.io - NAME AGE - knative-demo-topic 16s - ``` - -## Create the Event Display service - -1. Download a copy of the code: - - ```shell - git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs - cd knative-docs/docs/eventing/samples/kafka/source - ``` - -2. Build the Event Display Service (`event-display.yaml`) - - ```yaml - apiVersion: serving.knative.dev/v1 - kind: Service - metadata: - name: event-display - namespace: default - spec: - template: - spec: - containers: - - # This corresponds to - # https://github.com/knative/eventing-contrib/tree/main/cmd/event_display/main.go - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display - ``` - -1. Deploy the Event Display Service - - ``` - $ kubectl apply --filename event-display.yaml - ... - service.serving.knative.dev/event-display created - ``` - -1. Ensure that the Service pod is running. The pod name will be prefixed with - `event-display`. - ``` - $ kubectl get pods - NAME READY STATUS RESTARTS AGE - event-display-00001-deployment-5d5df6c7-gv2j4 2/2 Running 0 72s - ... - ``` - -### Apache Kafka Event Source - -1. Modify `source/event-source.yaml` accordingly with bootstrap servers, topics, - etc...: - - ```yaml - apiVersion: sources.knative.dev/v1beta1 - kind: KafkaSource - metadata: - name: kafka-source - spec: - consumerGroup: knative-group - bootstrapServers: - - my-cluster-kafka-bootstrap.kafka:9092 # note the kafka namespace - topics: - - knative-demo-topic - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display - ``` - -1. Deploy the event source. - ``` - $ kubectl apply -f event-source.yaml - ... - kafkasource.sources.knative.dev/kafka-source created - ``` -1. Check that the event source pod is running. The pod name will be prefixed - with `kafka-source`. - ``` - $ kubectl get pods - NAME READY STATUS RESTARTS AGE - kafka-source-xlnhq-5544766765-dnl5s 1/1 Running 0 40m - ``` -1. Ensure the Apache Kafka Event Source started with the necessary - configuration. - ``` - $ kubectl logs --selector='knative-eventing-source-name=kafka-source' - {"level":"info","ts":"2020-05-28T10:39:42.104Z","caller":"adapter/adapter.go:81","msg":"Starting with config: ","Topics":".","ConsumerGroup":"...","SinkURI":"...","Name":".","Namespace":"."} - ``` - -### Verify - -1. Produce a message (`{"msg": "This is a test!"}`) to the Apache Kafka topic, - like shown below: - ``` - kubectl -n kafka run kafka-producer -ti --image=strimzi/kafka:0.14.0-kafka-2.3.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic knative-demo-topic - If you don't see a command prompt, try pressing enter. - >{"msg": "This is a test!"} - ``` -1. Check that the Apache Kafka Event Source consumed the message and sent it to - its sink properly. Since these logs are captured in debug level, edit the key `level` of `config-logging` configmap in `knative-sources` namespace to look like this: - ``` - data: - loglevel.controller: info - loglevel.webhook: info - zap-logger-config: | - { - "level": "debug", - "development": false, - "outputPaths": ["stdout"], - "errorOutputPaths": ["stderr"], - "encoding": "json", - "encoderConfig": { - "timeKey": "ts", - "levelKey": "level", - "nameKey": "logger", - "callerKey": "caller", - "messageKey": "msg", - "stacktraceKey": "stacktrace", - "lineEnding": "", - "levelEncoder": "", - "timeEncoder": "iso8601", - "durationEncoder": "", - "callerEncoder": "" - } - } - - ``` - Now manually delete the kafkasource deployment and allow the `kafka-controller-manager` deployment running in `knative-sources` namespace to redeploy it. Debug level logs should be visible now. - - ``` - $ kubectl logs --selector='knative-eventing-source-name=kafka-source' - ... - - {"level":"debug","ts":"2020-05-28T10:40:29.400Z","caller":"kafka/consumer_handler.go:77","msg":"Message claimed","topic":".","value":"."} - {"level":"debug","ts":"2020-05-28T10:40:31.722Z","caller":"kafka/consumer_handler.go:89","msg":"Message marked","topic":".","value":"."} - ``` - -1. Ensure the Event Display received the message sent to it by the Event Source. - - ``` - $ kubectl logs --selector='serving.knative.dev/service=event-display' -c user-container - - ☁️ cloudevents.Event - Validation: valid - Context Attributes, - specversion: 1.0 - type: dev.knative.kafka.event - source: /apis/v1/namespaces/default/kafkasources/kafka-source#my-topic - subject: partition:0#564 - id: partition:0/offset:564 - time: 2020-02-10T18:10:23.861866615Z - datacontenttype: application/json - Extensions, - key: - Data, - { - "msg": "This is a test!" - } - ``` - -## Teardown Steps - -1. Remove the Apache Kafka Event Source - ``` - - $ kubectl delete -f source/source.yaml kafkasource.sources.knative.dev - "kafka-source" deleted - - ``` - 2. Remove the Event Display - ``` - - $ kubectl delete -f source/event-display.yaml service.serving.knative.dev - "event-display" deleted - - ``` - 3. Remove the Apache Kafka Event Controller - ``` - - $ kubectl delete -f https://storage.googleapis.com/knative-releases/eventing-contrib/latest/kafka-source.yaml - serviceaccount "kafka-controller-manager" deleted - clusterrole.rbac.authorization.k8s.io "eventing-sources-kafka-controller" - deleted clusterrolebinding.rbac.authorization.k8s.io - "eventing-sources-kafka-controller" deleted - customresourcedefinition.apiextensions.k8s.io "kafkasources.sources.knative.dev" - deleted service "kafka-controller" deleted statefulset.apps - "kafka-controller-manager" deleted - - ``` -4. (Optional) Remove the Apache Kafka Topic - - ```shell - $ kubectl delete -f kafka-topic.yaml - kafkatopic.kafka.strimzi.io "knative-demo-topic" deleted - ``` - -## (Optional) Specify the key deserializer - -When `KafkaSource` receives a message from Kafka, it dumps the key in the Event -extension called `Key` and dumps Kafka message headers in the extensions -starting with `kafkaheader`. - -You can specify the key deserializer among four types: - -* `string` (default) for UTF-8 encoded strings -* `int` for 32-bit & 64-bit signed integers -* `float` for 32-bit & 64-bit floating points -* `byte-array` for a Base64 encoded byte array - -To specify it, add the label `kafkasources.sources.knative.dev/key-type` to the `KafkaSource` definition like: - ```yaml - apiVersion: sources.knative.dev/v1beta1 - kind: KafkaSource - metadata: - name: kafka-source - labels: - kafkasources.sources.knative.dev/key-type: int - spec: - consumerGroup: knative-group - bootstrapServers: - - my-cluster-kafka-bootstrap.kafka:9092 # note the kafka namespace - topics: - - knative-demo-topic - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display - ``` - -## Connecting to a TLS enabled Kafka broker - -The KafkaSource supports TLS and SASL authentication methods. For enabling TLS authentication, please have the below files - -* CA Certificate -* Client Certificate and Key - -KafkaSource expects these files to be in pem format, if it is in other format like jks, please convert to pem. - -1. Create the certificate files as secrets in the namespace where KafkaSource is going to be set up - ``` - - $ kubectl create secret generic cacert --from-file=caroot.pem - secret/cacert created - - $ kubectl create secret tls kafka-secret --cert=certificate.pem --key=key.pem - secret/key created - - - ``` - -2. Apply the KafkaSource, change bootstrapServers and topics accordingly. -```yaml - apiVersion: sources.knative.dev/v1beta1 - kind: KafkaSource - metadata: - name: kafka-source-with-tls - spec: - net: - tls: - enable: true - cert: - secretKeyRef: - key: tls.crt - name: kafka-secret - key: - secretKeyRef: - key: tls.key - name: kafka-secret - caCert: - secretKeyRef: - key: caroot.pem - name: cacert - consumerGroup: knative-group - bootstrapServers: - - my-secure-kafka-bootstrap.kafka:443 - topics: - - knative-demo-topic - sink: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display - ``` diff --git a/docs/eventing/samples/kafka/source/index.md b/docs/eventing/samples/kafka/source/index.md index b5b8a888849..ee9e70dcbce 100644 --- a/docs/eventing/samples/kafka/source/index.md +++ b/docs/eventing/samples/kafka/source/index.md @@ -5,4 +5,332 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +Tutorial on how to build and deploy a `KafkaSource` [Eventing source](../../../sources/README.md) using a Knative Serving `Service`. + +## Prerequisites + +- Ensure that you meet the [prerequisites listed in the Apache Kafka overview](../README.md). +- A Kubernetes cluster with [Knative Kafka Source installed](../../../../install/README.md). + +## Apache Kafka Topic (Optional) + +1. If using Strimzi, you can set a topic modifying `source/kafka-topic.yaml` + with your desired: + +- Topic +- Cluster Name +- Partitions +- Replicas + + ```yaml + apiVersion: kafka.strimzi.io/v1beta1 + kind: KafkaTopic + metadata: + name: knative-demo-topic + namespace: kafka + labels: + strimzi.io/cluster: my-cluster + spec: + partitions: 3 + replicas: 1 + config: + retention.ms: 7200000 + segment.bytes: 1073741824 + ``` + +2. Deploy the `KafkaTopic` + + ```shell + $ kubectl apply -f strimzi-topic.yaml + kafkatopic.kafka.strimzi.io/knative-demo-topic created + ``` + +3. Ensure the `KafkaTopic` is running. + + ```shell + $ kubectl -n kafka get kafkatopics.kafka.strimzi.io + NAME AGE + knative-demo-topic 16s + ``` + +## Create the Event Display service + +1. Download a copy of the code: + + ```shell + git clone -b "{{< branch >}}" https://github.com/knative/docs knative-docs + cd knative-docs/docs/eventing/samples/kafka/source + ``` + +2. Build the Event Display Service (`event-display.yaml`) + + ```yaml + apiVersion: serving.knative.dev/v1 + kind: Service + metadata: + name: event-display + namespace: default + spec: + template: + spec: + containers: + - # This corresponds to + # https://github.com/knative/eventing-contrib/tree/main/cmd/event_display/main.go + image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display + ``` + +1. Deploy the Event Display Service + + ``` + $ kubectl apply --filename event-display.yaml + ... + service.serving.knative.dev/event-display created + ``` + +1. Ensure that the Service pod is running. The pod name will be prefixed with + `event-display`. + ``` + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + event-display-00001-deployment-5d5df6c7-gv2j4 2/2 Running 0 72s + ... + ``` + +### Apache Kafka Event Source + +1. Modify `source/event-source.yaml` accordingly with bootstrap servers, topics, + etc...: + + ```yaml + apiVersion: sources.knative.dev/v1beta1 + kind: KafkaSource + metadata: + name: kafka-source + spec: + consumerGroup: knative-group + bootstrapServers: + - my-cluster-kafka-bootstrap.kafka:9092 # note the kafka namespace + topics: + - knative-demo-topic + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display + ``` + +1. Deploy the event source. + ``` + $ kubectl apply -f event-source.yaml + ... + kafkasource.sources.knative.dev/kafka-source created + ``` +1. Check that the event source pod is running. The pod name will be prefixed + with `kafka-source`. + ``` + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + kafka-source-xlnhq-5544766765-dnl5s 1/1 Running 0 40m + ``` +1. Ensure the Apache Kafka Event Source started with the necessary + configuration. + ``` + $ kubectl logs --selector='knative-eventing-source-name=kafka-source' + {"level":"info","ts":"2020-05-28T10:39:42.104Z","caller":"adapter/adapter.go:81","msg":"Starting with config: ","Topics":".","ConsumerGroup":"...","SinkURI":"...","Name":".","Namespace":"."} + ``` + +### Verify + +1. Produce a message (`{"msg": "This is a test!"}`) to the Apache Kafka topic, + like shown below: + ``` + kubectl -n kafka run kafka-producer -ti --image=strimzi/kafka:0.14.0-kafka-2.3.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic knative-demo-topic + If you don't see a command prompt, try pressing enter. + >{"msg": "This is a test!"} + ``` +1. Check that the Apache Kafka Event Source consumed the message and sent it to + its sink properly. Since these logs are captured in debug level, edit the key `level` of `config-logging` configmap in `knative-sources` namespace to look like this: + ``` + data: + loglevel.controller: info + loglevel.webhook: info + zap-logger-config: | + { + "level": "debug", + "development": false, + "outputPaths": ["stdout"], + "errorOutputPaths": ["stderr"], + "encoding": "json", + "encoderConfig": { + "timeKey": "ts", + "levelKey": "level", + "nameKey": "logger", + "callerKey": "caller", + "messageKey": "msg", + "stacktraceKey": "stacktrace", + "lineEnding": "", + "levelEncoder": "", + "timeEncoder": "iso8601", + "durationEncoder": "", + "callerEncoder": "" + } + } + + ``` + Now manually delete the kafkasource deployment and allow the `kafka-controller-manager` deployment running in `knative-sources` namespace to redeploy it. Debug level logs should be visible now. + + ``` + $ kubectl logs --selector='knative-eventing-source-name=kafka-source' + ... + + {"level":"debug","ts":"2020-05-28T10:40:29.400Z","caller":"kafka/consumer_handler.go:77","msg":"Message claimed","topic":".","value":"."} + {"level":"debug","ts":"2020-05-28T10:40:31.722Z","caller":"kafka/consumer_handler.go:89","msg":"Message marked","topic":".","value":"."} + ``` + +1. Ensure the Event Display received the message sent to it by the Event Source. + + ``` + $ kubectl logs --selector='serving.knative.dev/service=event-display' -c user-container + + ☁️ cloudevents.Event + Validation: valid + Context Attributes, + specversion: 1.0 + type: dev.knative.kafka.event + source: /apis/v1/namespaces/default/kafkasources/kafka-source#my-topic + subject: partition:0#564 + id: partition:0/offset:564 + time: 2020-02-10T18:10:23.861866615Z + datacontenttype: application/json + Extensions, + key: + Data, + { + "msg": "This is a test!" + } + ``` + +## Teardown Steps + +1. Remove the Apache Kafka Event Source + ``` + + $ kubectl delete -f source/source.yaml kafkasource.sources.knative.dev + "kafka-source" deleted + + ``` + 2. Remove the Event Display + ``` + + $ kubectl delete -f source/event-display.yaml service.serving.knative.dev + "event-display" deleted + + ``` + 3. Remove the Apache Kafka Event Controller + ``` + + $ kubectl delete -f https://storage.googleapis.com/knative-releases/eventing-contrib/latest/kafka-source.yaml + serviceaccount "kafka-controller-manager" deleted + clusterrole.rbac.authorization.k8s.io "eventing-sources-kafka-controller" + deleted clusterrolebinding.rbac.authorization.k8s.io + "eventing-sources-kafka-controller" deleted + customresourcedefinition.apiextensions.k8s.io "kafkasources.sources.knative.dev" + deleted service "kafka-controller" deleted statefulset.apps + "kafka-controller-manager" deleted + + ``` +4. (Optional) Remove the Apache Kafka Topic + + ```shell + $ kubectl delete -f kafka-topic.yaml + kafkatopic.kafka.strimzi.io "knative-demo-topic" deleted + ``` + +## (Optional) Specify the key deserializer + +When `KafkaSource` receives a message from Kafka, it dumps the key in the Event +extension called `Key` and dumps Kafka message headers in the extensions +starting with `kafkaheader`. + +You can specify the key deserializer among four types: + +* `string` (default) for UTF-8 encoded strings +* `int` for 32-bit & 64-bit signed integers +* `float` for 32-bit & 64-bit floating points +* `byte-array` for a Base64 encoded byte array + +To specify it, add the label `kafkasources.sources.knative.dev/key-type` to the `KafkaSource` definition like: + ```yaml + apiVersion: sources.knative.dev/v1beta1 + kind: KafkaSource + metadata: + name: kafka-source + labels: + kafkasources.sources.knative.dev/key-type: int + spec: + consumerGroup: knative-group + bootstrapServers: + - my-cluster-kafka-bootstrap.kafka:9092 # note the kafka namespace + topics: + - knative-demo-topic + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display + ``` + +## Connecting to a TLS enabled Kafka broker + +The KafkaSource supports TLS and SASL authentication methods. For enabling TLS authentication, please have the below files + +* CA Certificate +* Client Certificate and Key + +KafkaSource expects these files to be in pem format, if it is in other format like jks, please convert to pem. + +1. Create the certificate files as secrets in the namespace where KafkaSource is going to be set up + ``` + + $ kubectl create secret generic cacert --from-file=caroot.pem + secret/cacert created + + $ kubectl create secret tls kafka-secret --cert=certificate.pem --key=key.pem + secret/key created + + + ``` + +2. Apply the KafkaSource, change bootstrapServers and topics accordingly. +```yaml + apiVersion: sources.knative.dev/v1beta1 + kind: KafkaSource + metadata: + name: kafka-source-with-tls + spec: + net: + tls: + enable: true + cert: + secretKeyRef: + key: tls.crt + name: kafka-secret + key: + secretKeyRef: + key: tls.key + name: kafka-secret + caCert: + secretKeyRef: + key: caroot.pem + name: cacert + consumerGroup: knative-group + bootstrapServers: + - my-secure-kafka-bootstrap.kafka:443 + topics: + - knative-demo-topic + sink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display + ``` diff --git a/docs/eventing/samples/parallel/README.md b/docs/eventing/samples/parallel/README.md deleted file mode 100644 index 40f12dcd9f1..00000000000 --- a/docs/eventing/samples/parallel/README.md +++ /dev/null @@ -1,27 +0,0 @@ -The following examples will help you understand how to use Parallel to describe -various flows. - -## Prerequisites - -All examples require: - -- A Kubernetes cluster with - - Knative Eventing - - Knative Serving - -All examples are using the -[default channel template](../../channels/default-channels.md). - -## Examples - -For each of these examples below, we'll use -[`PingSource`](../ping-source/README.md) as the source of events. - -We also use simple -[functions](https://github.com/lionelvillard/knative-functions) to perform -trivial filtering, transformation and routing of the incoming events. - -The examples are: - -- [Parallel with multiple branches and global reply](./multiple-branches/README.md) -- [Parallel with mutually exclusive cases](./mutual-exclusivity/README.md) diff --git a/docs/eventing/samples/parallel/_index.md b/docs/eventing/samples/parallel/_index.md index 152ee32b38f..fcc74e21f0c 100644 --- a/docs/eventing/samples/parallel/_index.md +++ b/docs/eventing/samples/parallel/_index.md @@ -5,4 +5,30 @@ weight: 10 type: "docs" --- -{{% readfile file="README.md" %}} +The following examples will help you understand how to use Parallel to describe +various flows. + +## Prerequisites + +All examples require: + +- A Kubernetes cluster with + - Knative Eventing + - Knative Serving + +All examples are using the +[default channel template](../../channels/default-channels.md). + +## Examples + +For each of these examples below, we'll use +[`PingSource`](../ping-source/README.md) as the source of events. + +We also use simple +[functions](https://github.com/lionelvillard/knative-functions) to perform +trivial filtering, transformation and routing of the incoming events. + +The examples are: + +- [Parallel with multiple branches and global reply](./multiple-branches/README.md) +- [Parallel with mutually exclusive cases](./mutual-exclusivity/README.md) diff --git a/docs/eventing/samples/parallel/multiple-branches/README.md b/docs/eventing/samples/parallel/multiple-branches/README.md deleted file mode 100644 index c8098976da9..00000000000 --- a/docs/eventing/samples/parallel/multiple-branches/README.md +++ /dev/null @@ -1,215 +0,0 @@ -We are going to create a Parallel with two branches: - -- the first branch accepts events with a time that is is even -- the second branch accepts events with a time that is is odd - -The events produced by each branch are then sent to the `event-display` service. - -## Prerequisites - -Please refer to the sample overview for the [prerequisites](../README.md). - -### Create the Knative Services - -Let's first create the filter and transformer services that we will use in our -Parallel. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: even-filter -spec: - template: - spec: - containers: - - image: villardl/filter-nodejs:0.1 - env: - - name: FILTER - value: | - Math.round(Date.parse(event.time) / 60000) % 2 === 0 ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: odd-filter -spec: - template: - spec: - containers: - - image: villardl/filter-nodejs:0.1 - env: - - name: FILTER - value: | - Math.round(Date.parse(event.time) / 60000) % 2 !== 0 ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: even-transformer -spec: - template: - spec: - containers: - - image: villardl/transformer-nodejs - env: - - name: TRANSFORMER - value: | - ({"message": "we are even!"}) - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: odd-transformer -spec: - template: - spec: - containers: - - image: villardl/transformer-nodejs:0.1 - env: - - name: TRANSFORMER - value: | - ({"message": "this is odd!"}) -. -``` - -```shell -kubectl create -f ./filters.yaml -f ./transformers.yaml -``` - -### Create the Service displaying the events created by Sequence - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: event-display -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -your resources to be created. - -```shell -kubectl -n default create -f ./event-display.yaml -``` - -### Create the Parallel - -The `parallel.yaml` file contains the specifications for creating the Parallel. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Parallel -metadata: - name: odd-even-parallel -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - branches: - - filter: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: even-filter - subscriber: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: even-transformer - - filter: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: odd-filter - subscriber: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: odd-transformer - reply: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: event-display -``` - -```shell -kubectl create -f ./parallel.yaml -``` - -### Create the PingSource targeting the Parallel - -This will create a PingSource which will send a CloudEvent with -`{"message": "Even or odd?"}` as the data payload every minute. - -```yaml -apiVersion: sources.knative.dev/v1beta2 -kind: PingSource -metadata: - name: ping-source -spec: - schedule: "*/1 * * * *" - contentType: "application/json" - data: '{"message": "Even or odd?"}' - sink: - ref: - apiVersion: flows.knative.dev/v1 - kind: Parallel - name: odd-even-parallel -``` - -```shell -kubectl create -f ./ping-source.yaml -``` - -### Inspecting the results - -You can now see the final output by inspecting the logs of the event-display -pods. Note that since we set the `PingSource` to emit every minute, it might -take some time for the events to show up in the logs. - -Let's look at the `event-display` log: - -```shell -kubectl logs -l serving.knative.dev/service=event-display --tail=30 -c user-container - -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.sources.ping - source: /apis/v1/namespaces/default/pingsources/ping-source - id: 015a4cf4-8a43-44a9-8702-3d4171d27ba5 - time: 2020-03-03T21:24:00.0007254Z - datacontenttype: application/json; charset=utf-8 -Extensions, - knativehistory: odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; odd-even-parallel-kn-parallel-0-kn-channel.default.svc.cluster.local - traceparent: 00-41a139bf073f3cfcba7bb7ce7f1488fc-68a891ace985221a-00 -Data, - { - "message": "we are even!" - } -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.sources.ping - source: /apis/v1/namespaces/default/pingsources/ping-source - id: 52e6b097-f914-4b5a-8539-165650e85bcd - time: 2020-03-03T21:23:00.0004662Z - datacontenttype: application/json; charset=utf-8 -Extensions, - knativehistory: odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; odd-even-parallel-kn-parallel-1-kn-channel.default.svc.cluster.local - traceparent: 00-58d371410d7daf2033be226860b4ee5d-05d686ee90c3226f-00 -Data, - { - "message": "this is odd!" - } -``` diff --git a/docs/eventing/samples/parallel/multiple-branches/_index.md b/docs/eventing/samples/parallel/multiple-branches/_index.md index a32a0f90c22..861c119c9b2 100644 --- a/docs/eventing/samples/parallel/multiple-branches/_index.md +++ b/docs/eventing/samples/parallel/multiple-branches/_index.md @@ -5,4 +5,218 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +We are going to create a Parallel with two branches: + +- the first branch accepts events with a time that is is even +- the second branch accepts events with a time that is is odd + +The events produced by each branch are then sent to the `event-display` service. + +## Prerequisites + +Please refer to the sample overview for the [prerequisites](../README.md). + +### Create the Knative Services + +Let's first create the filter and transformer services that we will use in our +Parallel. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: even-filter +spec: + template: + spec: + containers: + - image: villardl/filter-nodejs:0.1 + env: + - name: FILTER + value: | + Math.round(Date.parse(event.time) / 60000) % 2 === 0 +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: odd-filter +spec: + template: + spec: + containers: + - image: villardl/filter-nodejs:0.1 + env: + - name: FILTER + value: | + Math.round(Date.parse(event.time) / 60000) % 2 !== 0 +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: even-transformer +spec: + template: + spec: + containers: + - image: villardl/transformer-nodejs + env: + - name: TRANSFORMER + value: | + ({"message": "we are even!"}) + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: odd-transformer +spec: + template: + spec: + containers: + - image: villardl/transformer-nodejs:0.1 + env: + - name: TRANSFORMER + value: | + ({"message": "this is odd!"}) +. +``` + +```shell +kubectl create -f ./filters.yaml -f ./transformers.yaml +``` + +### Create the Service displaying the events created by Sequence + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: event-display +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +your resources to be created. + +```shell +kubectl -n default create -f ./event-display.yaml +``` + +### Create the Parallel + +The `parallel.yaml` file contains the specifications for creating the Parallel. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Parallel +metadata: + name: odd-even-parallel +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + branches: + - filter: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: even-filter + subscriber: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: even-transformer + - filter: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: odd-filter + subscriber: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: odd-transformer + reply: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: event-display +``` + +```shell +kubectl create -f ./parallel.yaml +``` + +### Create the PingSource targeting the Parallel + +This will create a PingSource which will send a CloudEvent with +`{"message": "Even or odd?"}` as the data payload every minute. + +```yaml +apiVersion: sources.knative.dev/v1beta2 +kind: PingSource +metadata: + name: ping-source +spec: + schedule: "*/1 * * * *" + contentType: "application/json" + data: '{"message": "Even or odd?"}' + sink: + ref: + apiVersion: flows.knative.dev/v1 + kind: Parallel + name: odd-even-parallel +``` + +```shell +kubectl create -f ./ping-source.yaml +``` + +### Inspecting the results + +You can now see the final output by inspecting the logs of the event-display +pods. Note that since we set the `PingSource` to emit every minute, it might +take some time for the events to show up in the logs. + +Let's look at the `event-display` log: + +```shell +kubectl logs -l serving.knative.dev/service=event-display --tail=30 -c user-container + +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.sources.ping + source: /apis/v1/namespaces/default/pingsources/ping-source + id: 015a4cf4-8a43-44a9-8702-3d4171d27ba5 + time: 2020-03-03T21:24:00.0007254Z + datacontenttype: application/json; charset=utf-8 +Extensions, + knativehistory: odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; odd-even-parallel-kn-parallel-0-kn-channel.default.svc.cluster.local + traceparent: 00-41a139bf073f3cfcba7bb7ce7f1488fc-68a891ace985221a-00 +Data, + { + "message": "we are even!" + } +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.sources.ping + source: /apis/v1/namespaces/default/pingsources/ping-source + id: 52e6b097-f914-4b5a-8539-165650e85bcd + time: 2020-03-03T21:23:00.0004662Z + datacontenttype: application/json; charset=utf-8 +Extensions, + knativehistory: odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; odd-even-parallel-kn-parallel-1-kn-channel.default.svc.cluster.local + traceparent: 00-58d371410d7daf2033be226860b4ee5d-05d686ee90c3226f-00 +Data, + { + "message": "this is odd!" + } +``` diff --git a/docs/eventing/samples/parallel/mutual-exclusivity/README.md b/docs/eventing/samples/parallel/mutual-exclusivity/README.md deleted file mode 100644 index e46f4a1c09d..00000000000 --- a/docs/eventing/samples/parallel/mutual-exclusivity/README.md +++ /dev/null @@ -1,198 +0,0 @@ -In this example, we are going to see how we can create a Parallel with mutually -exclusive branches. - -This example is the same as the -[multiple branches example](../multiple-branches/README.md) except that we are -now going to rely on the Knative -[switch](https://github.com/lionelvillard/knative-functions#switch) function to -provide a soft mutual exclusivity guarantee. - -NOTE: this example must be deployed in the default namespace. - -## Prerequisites - -Please refer to the sample overview for the [prerequisites](../README.md). - -### Create the Knative Services - -Let's first create the switcher and transformer services that we will use in our -Parallel. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: me-even-odd-switcher -spec: - template: - spec: - containers: - - image: villardl/switcher-nodejs:0.1 - env: - - name: EXPRESSION - value: Math.round(Date.parse(event.time) / 60000) % 2 - - name: CASES - value: '[0, 1]' ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: even-transformer -spec: - template: - spec: - containers: - - image: villardl/transformer-nodejs:0.1 - env: - - name: TRANSFORMER - value: | - ({"message": "we are even!"}) - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: odd-transformer -spec: - template: - spec: - containers: - - image: villardl/transformer-nodejs:0.1 - env: - - name: TRANSFORMER - value: | - ({"message": "this is odd!"}) -. -``` - -```shell -kubectl create -f ./switcher.yaml -f ./transformers.yaml -``` - -### Create the Service displaying the events created by Parallel - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: event-display -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -```shell -kubectl -n default create -f ./event-display.yaml -``` - -### Create the Parallel object - -The `parallel.yaml` file contains the specifications for creating the Parallel -object. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Parallel -metadata: - name: me-odd-even-parallel -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - branches: - - filter: - uri: "http://me-even-odd-switcher.default.svc.cluster.local/0" - subscriber: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: me-even-transformer - - filter: - uri: "http://me-even-odd-switcher.default.svc.cluster.local/1" - subscriber: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: me-odd-transformer - reply: - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: me-event-display -``` - -```shell -kubectl create -f ./parallel.yaml -``` - -### Create the PingSource targeting the Parallel object - -This will create a PingSource which will send a CloudEvent with -`{"message": "Even or odd?"}` as the data payload every minute. - -```yaml -apiVersion: sources.knative.dev/v1beta2 -kind: PingSource -metadata: - name: me-ping-source -spec: - schedule: "*/1 * * * *" - contentType: "application/json" - data: '{"message": "Even or odd?"}' - sink: - ref: - apiVersion: flows.knative.dev/v1 - kind: Parallel - name: me-odd-even-parallel -``` - -```shell -kubectl create -f ./ping-source.yaml -``` - -### Inspecting the results - -You can now see the final output by inspecting the logs of the -`me-event-display` pods. Note that since we set the `PingSource` to emit every -minute, it might take some time for the events to show up in the logs. - -Let's look at the `me-event-display` log: - -```shell -kubectl logs -l serving.knative.dev/service=me-event-display --tail=50 -c user-container - -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.sources.ping - source: /apis/v1/namespaces/default/pingsources/me-ping-source - id: 0b0db15c-9b36-4388-8eaa-8c23a9dc2707 - time: 2020-03-03T21:30:00.0007292Z - datacontenttype: application/json; charset=utf-8 -Extensions, - knativehistory: me-odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; me-odd-even-parallel-kn-parallel-0-kn-channel.default.svc.cluster.local - traceparent: 00-e8b17109cd21d4fa66a86633b5a614c7-ba96d220fe13211c-00 -Data, - { - "message": "we are even!" - } -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.sources.ping - source: /apis/v1/namespaces/default/pingsources/me-ping-source - id: 321170d1-dea7-4b18-9290-28adb1de089b - time: 2020-03-03T21:31:00.0007847Z - datacontenttype: application/json; charset=utf-8 -Extensions, - knativehistory: me-odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; me-odd-even-parallel-kn-parallel-1-kn-channel.default.svc.cluster.local - traceparent: 00-78d8b044d23c85789f0a13fd3679ac24-1d69ddde56d620c7-00 -Data, - { - "message": "this is odd!" - } -``` diff --git a/docs/eventing/samples/parallel/mutual-exclusivity/_index.md b/docs/eventing/samples/parallel/mutual-exclusivity/_index.md index a6702fc1f73..f330f89fbb8 100644 --- a/docs/eventing/samples/parallel/mutual-exclusivity/_index.md +++ b/docs/eventing/samples/parallel/mutual-exclusivity/_index.md @@ -5,4 +5,201 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +In this example, we are going to see how we can create a Parallel with mutually +exclusive branches. + +This example is the same as the +[multiple branches example](../multiple-branches/README.md) except that we are +now going to rely on the Knative +[switch](https://github.com/lionelvillard/knative-functions#switch) function to +provide a soft mutual exclusivity guarantee. + +NOTE: this example must be deployed in the default namespace. + +## Prerequisites + +Please refer to the sample overview for the [prerequisites](../README.md). + +### Create the Knative Services + +Let's first create the switcher and transformer services that we will use in our +Parallel. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: me-even-odd-switcher +spec: + template: + spec: + containers: + - image: villardl/switcher-nodejs:0.1 + env: + - name: EXPRESSION + value: Math.round(Date.parse(event.time) / 60000) % 2 + - name: CASES + value: '[0, 1]' +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: even-transformer +spec: + template: + spec: + containers: + - image: villardl/transformer-nodejs:0.1 + env: + - name: TRANSFORMER + value: | + ({"message": "we are even!"}) + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: odd-transformer +spec: + template: + spec: + containers: + - image: villardl/transformer-nodejs:0.1 + env: + - name: TRANSFORMER + value: | + ({"message": "this is odd!"}) +. +``` + +```shell +kubectl create -f ./switcher.yaml -f ./transformers.yaml +``` + +### Create the Service displaying the events created by Parallel + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: event-display +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +```shell +kubectl -n default create -f ./event-display.yaml +``` + +### Create the Parallel object + +The `parallel.yaml` file contains the specifications for creating the Parallel +object. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Parallel +metadata: + name: me-odd-even-parallel +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + branches: + - filter: + uri: "http://me-even-odd-switcher.default.svc.cluster.local/0" + subscriber: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: me-even-transformer + - filter: + uri: "http://me-even-odd-switcher.default.svc.cluster.local/1" + subscriber: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: me-odd-transformer + reply: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: me-event-display +``` + +```shell +kubectl create -f ./parallel.yaml +``` + +### Create the PingSource targeting the Parallel object + +This will create a PingSource which will send a CloudEvent with +`{"message": "Even or odd?"}` as the data payload every minute. + +```yaml +apiVersion: sources.knative.dev/v1beta2 +kind: PingSource +metadata: + name: me-ping-source +spec: + schedule: "*/1 * * * *" + contentType: "application/json" + data: '{"message": "Even or odd?"}' + sink: + ref: + apiVersion: flows.knative.dev/v1 + kind: Parallel + name: me-odd-even-parallel +``` + +```shell +kubectl create -f ./ping-source.yaml +``` + +### Inspecting the results + +You can now see the final output by inspecting the logs of the +`me-event-display` pods. Note that since we set the `PingSource` to emit every +minute, it might take some time for the events to show up in the logs. + +Let's look at the `me-event-display` log: + +```shell +kubectl logs -l serving.knative.dev/service=me-event-display --tail=50 -c user-container + +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.sources.ping + source: /apis/v1/namespaces/default/pingsources/me-ping-source + id: 0b0db15c-9b36-4388-8eaa-8c23a9dc2707 + time: 2020-03-03T21:30:00.0007292Z + datacontenttype: application/json; charset=utf-8 +Extensions, + knativehistory: me-odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; me-odd-even-parallel-kn-parallel-0-kn-channel.default.svc.cluster.local + traceparent: 00-e8b17109cd21d4fa66a86633b5a614c7-ba96d220fe13211c-00 +Data, + { + "message": "we are even!" + } +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.sources.ping + source: /apis/v1/namespaces/default/pingsources/me-ping-source + id: 321170d1-dea7-4b18-9290-28adb1de089b + time: 2020-03-03T21:31:00.0007847Z + datacontenttype: application/json; charset=utf-8 +Extensions, + knativehistory: me-odd-even-parallel-kn-parallel-kn-channel.default.svc.cluster.local; me-odd-even-parallel-kn-parallel-1-kn-channel.default.svc.cluster.local + traceparent: 00-78d8b044d23c85789f0a13fd3679ac24-1d69ddde56d620c7-00 +Data, + { + "message": "this is odd!" + } +``` diff --git a/docs/eventing/samples/sequence/sequence-reply-to-event-display/README.md b/docs/eventing/samples/sequence/sequence-reply-to-event-display/README.md deleted file mode 100644 index 18ae6c4eac4..00000000000 --- a/docs/eventing/samples/sequence/sequence-reply-to-event-display/README.md +++ /dev/null @@ -1,196 +0,0 @@ -We are going to create the following logical configuration. We create a -PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md), then -taking the output of that `Sequence` and displaying the resulting output. - -![Logical Configuration](./sequence-reply-to-event-display.png) - -The functions used in these examples live in -[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). - -## Prerequisites - -For this example, we'll assume you have set up an `InMemoryChannel` as well as -Knative Serving (for our functions). The examples use `default` namespace, -again, if you want to deploy to another Namespace, you will need to modify the -examples to reflect this. - -If you want to use different type of `Channel`, you will have to modify the -`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. - -## Setup - -### Create the Knative Services - -Change `default` below to create the steps in the Namespace where you want -resources created. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: first -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 0" - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: second -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 1" ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: third -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 2" ---- - -``` - -```shell -kubectl -n default create -f ./steps.yaml -``` - -### Create the Sequence - -The `sequence.yaml` file contains the specifications for creating the Sequence. -If you are using a different type of Channel, you need to change the -spec.channelTemplate to point to your desired Channel. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Sequence -metadata: - name: sequence -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - steps: - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: first - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: second - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: third - reply: - ref: - kind: Service - apiVersion: serving.knative.dev/v1 - name: event-display -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -the resources to be created. - -```shell -kubectl -n default create -f ./sequence.yaml -``` - -### Create the Service displaying the events created by Sequence - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: event-display -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -your resources to be created. - -```shell -kubectl -n default create -f ./event-display.yaml -``` - -### Create the PingSource targeting the Sequence - -This will create a PingSource which will send a CloudEvent with {"message": -"Hello world!"} as the data payload every 2 minutes. - -```yaml -apiVersion: sources.knative.dev/v1beta2 -kind: PingSource -metadata: - name: ping-source -spec: - schedule: "*/2 * * * *" - contentType: "application/json" - data: '{"message": "Hello world!"}' - sink: - ref: - apiVersion: flows.knative.dev/v1 - kind: Sequence - name: sequence -``` - -```shell -kubectl -n default create -f ./ping-source.yaml -``` - -### Inspecting the results - -You can now see the final output by inspecting the logs of the event-display -pods. - -```shell -kubectl -n default get pods -``` - -Wait a bit and then look at the logs for the event-display pod: - -```shell -kubectl -n default logs -l serving.knative.dev/service=event-display -c user-container --tail=-1 -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: samples.http.mode3 - source: /apis/v1/namespaces/default/pingsources/ping-source - id: e8fa7906-ab62-4e61-9c13-a9406e2130a9 - time: 2020-03-02T20:52:00.0004957Z - datacontenttype: application/json -Extensions, - knativehistory: sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; sequence-kn-sequence-2-kn-channel.default.svc.cluster.local - traceparent: 00-6e2947379387f35ddc933b9190af16ad-de3db0bc4e442394-00 -Data, - { - "id": 0, - "message": "Hello world! - Handled by 0 - Handled by 1 - Handled by 2" - } -``` - -And you can see that the initial PingSource message `("Hello World!")` has been -appended to it by each of the steps in the Sequence. diff --git a/docs/eventing/samples/sequence/sequence-reply-to-event-display/index.md b/docs/eventing/samples/sequence/sequence-reply-to-event-display/index.md index 0de60ff978c..95ded8107f6 100644 --- a/docs/eventing/samples/sequence/sequence-reply-to-event-display/index.md +++ b/docs/eventing/samples/sequence/sequence-reply-to-event-display/index.md @@ -5,4 +5,199 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +We are going to create the following logical configuration. We create a +PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md), then +taking the output of that `Sequence` and displaying the resulting output. + +![Logical Configuration](./sequence-reply-to-event-display.png) + +The functions used in these examples live in +[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). + +## Prerequisites + +For this example, we'll assume you have set up an `InMemoryChannel` as well as +Knative Serving (for our functions). The examples use `default` namespace, +again, if you want to deploy to another Namespace, you will need to modify the +examples to reflect this. + +If you want to use different type of `Channel`, you will have to modify the +`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. + +## Setup + +### Create the Knative Services + +Change `default` below to create the steps in the Namespace where you want +resources created. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: first +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 0" + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: second +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 1" +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: third +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 2" +--- + +``` + +```shell +kubectl -n default create -f ./steps.yaml +``` + +### Create the Sequence + +The `sequence.yaml` file contains the specifications for creating the Sequence. +If you are using a different type of Channel, you need to change the +spec.channelTemplate to point to your desired Channel. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Sequence +metadata: + name: sequence +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + steps: + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: first + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: second + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: third + reply: + ref: + kind: Service + apiVersion: serving.knative.dev/v1 + name: event-display +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +the resources to be created. + +```shell +kubectl -n default create -f ./sequence.yaml +``` + +### Create the Service displaying the events created by Sequence + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: event-display +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +your resources to be created. + +```shell +kubectl -n default create -f ./event-display.yaml +``` + +### Create the PingSource targeting the Sequence + +This will create a PingSource which will send a CloudEvent with {"message": +"Hello world!"} as the data payload every 2 minutes. + +```yaml +apiVersion: sources.knative.dev/v1beta2 +kind: PingSource +metadata: + name: ping-source +spec: + schedule: "*/2 * * * *" + contentType: "application/json" + data: '{"message": "Hello world!"}' + sink: + ref: + apiVersion: flows.knative.dev/v1 + kind: Sequence + name: sequence +``` + +```shell +kubectl -n default create -f ./ping-source.yaml +``` + +### Inspecting the results + +You can now see the final output by inspecting the logs of the event-display +pods. + +```shell +kubectl -n default get pods +``` + +Wait a bit and then look at the logs for the event-display pod: + +```shell +kubectl -n default logs -l serving.knative.dev/service=event-display -c user-container --tail=-1 +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: samples.http.mode3 + source: /apis/v1/namespaces/default/pingsources/ping-source + id: e8fa7906-ab62-4e61-9c13-a9406e2130a9 + time: 2020-03-02T20:52:00.0004957Z + datacontenttype: application/json +Extensions, + knativehistory: sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; sequence-kn-sequence-2-kn-channel.default.svc.cluster.local + traceparent: 00-6e2947379387f35ddc933b9190af16ad-de3db0bc4e442394-00 +Data, + { + "id": 0, + "message": "Hello world! - Handled by 0 - Handled by 1 - Handled by 2" + } +``` + +And you can see that the initial PingSource message `("Hello World!")` has been +appended to it by each of the steps in the Sequence. diff --git a/docs/eventing/samples/sequence/sequence-reply-to-sequence/README.md b/docs/eventing/samples/sequence/sequence-reply-to-sequence/README.md deleted file mode 100644 index ee8b1f45c66..00000000000 --- a/docs/eventing/samples/sequence/sequence-reply-to-sequence/README.md +++ /dev/null @@ -1,276 +0,0 @@ -We are going to create the following logical configuration. We create a -PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md), then -taking the output of that `Sequence` and sending it to a second `Sequence` and -finally displaying the resulting output. - -![Logical Configuration](./sequence-reply-to-sequence.png) - -The functions used in these examples live in -[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). - -## Prerequisites - -For this example, we'll assume you have set up an `InMemoryChannel` as well as -Knative Serving (for our functions). The examples use `default` namespace, -again, if you want to deploy to another namespace, you will need to modify the -examples to reflect this. - -If you want to use different type of `Channel`, you will have to modify the -`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. - -## Setup - -### Create the Knative Services - -Change `default` below to create the steps in the Namespace where you want -resources created. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: first -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 0" - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: second -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 1" ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: third -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 2" ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: fourth -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 3" - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: fifth -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 4" ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: sixth -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 5" ---- - -``` - -```shell -kubectl -n default create -f ./steps.yaml -``` - -### Create the first Sequence - -The `sequence1.yaml` file contains the specifications for creating the Sequence. -If you are using a different type of Channel, you need to change the -spec.channelTemplate to point to your desired Channel. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Sequence -metadata: - name: first-sequence -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - steps: - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: first - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: second - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: third - reply: - ref: - kind: Sequence - apiVersion: flows.knative.dev/v1 - name: second-sequence -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -your resources created. - -```shell -kubectl -n default create -f ./sequence1.yaml -``` - -### Create the second Sequence - -The `sequence2.yaml` file contains the specifications for creating the Sequence. -If you are using a different type of Channel, you need to change the -spec.channelTemplate to point to your desired Channel. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Sequence -metadata: - name: second-sequence -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - steps: - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: fourth - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: fifth - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: sixth - reply: - ref: - kind: Service - apiVersion: serving.knative.dev/v1 - name: event-display -``` - -```shell -kubectl -n default create -f ./sequence2.yaml -``` - -### Create the Service displaying the events created by Sequence - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: event-display -spec: - template: - spec: - containerers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -your resources created. - -```shell -kubectl -n default create -f ./event-display.yaml -``` - -### Create the PingSource targeting the first Sequence - -This will create a PingSource which will send a CloudEvent with -`{"message": "Hello world!"}` as the data payload every 2 minutes. - -```yaml -apiVersion: sources.knative.dev/v1beta2 -kind: PingSource -metadata: - name: ping-source -spec: - schedule: "*/2 * * * *" - contentType: "application/json" - data: '{"message": "Hello world!"}' - sink: - ref: - apiVersion: flows.knative.dev/v1 - kind: Sequence - name: first-sequence -``` - -```shell -kubectl -n default create -f ./ping-source.yaml -``` - -### Inspecting the results - -You can now see the final output by inspecting the logs of the event-display -pods. - -```shell -kubectl -n default get pods -``` - -Then look at the logs for the event-display pod: - -```shell -kubectl -n default logs -l serving.knative.dev/service=event-display -c user-container --tail=-1 -☁️ cloudevents.Event -Validation: valid -Context Attributes, - specversion: 1.0 - type: dev.knative.sources.ping - source: /apis/v1/namespaces/default/pingsources/ping-source - id: 29d531df-78d8-4d11-9ffd-ba24045241a9 - time: 2020-03-02T21:18:00.0011708Z - datacontenttype: application/json -Extensions, - knativehistory: first-sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; first-sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; first-sequence-kn-sequence-2-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-2-kn-channel.default.svc.cluster.local - traceparent: 00-e5abc9de525a89ead80560b8f328de5c-fc12b64a6296f541-00 -Data, - { - "id": 0, - "message": "Hello world! - Handled by 0 - Handled by 1 - Handled by 2 - Handled by 3 - Handled by 4 - Handled by 5" - } -``` - -And you can see that the initial PingSource message `("Hello World!")` has been -appended to it by each of the steps in the Sequence. diff --git a/docs/eventing/samples/sequence/sequence-reply-to-sequence/index.md b/docs/eventing/samples/sequence/sequence-reply-to-sequence/index.md index 7d5164e034f..460d6a8ba96 100644 --- a/docs/eventing/samples/sequence/sequence-reply-to-sequence/index.md +++ b/docs/eventing/samples/sequence/sequence-reply-to-sequence/index.md @@ -5,4 +5,279 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +We are going to create the following logical configuration. We create a +PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md), then +taking the output of that `Sequence` and sending it to a second `Sequence` and +finally displaying the resulting output. + +![Logical Configuration](./sequence-reply-to-sequence.png) + +The functions used in these examples live in +[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). + +## Prerequisites + +For this example, we'll assume you have set up an `InMemoryChannel` as well as +Knative Serving (for our functions). The examples use `default` namespace, +again, if you want to deploy to another namespace, you will need to modify the +examples to reflect this. + +If you want to use different type of `Channel`, you will have to modify the +`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. + +## Setup + +### Create the Knative Services + +Change `default` below to create the steps in the Namespace where you want +resources created. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: first +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 0" + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: second +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 1" +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: third +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 2" +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: fourth +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 3" + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: fifth +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 4" +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: sixth +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 5" +--- + +``` + +```shell +kubectl -n default create -f ./steps.yaml +``` + +### Create the first Sequence + +The `sequence1.yaml` file contains the specifications for creating the Sequence. +If you are using a different type of Channel, you need to change the +spec.channelTemplate to point to your desired Channel. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Sequence +metadata: + name: first-sequence +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + steps: + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: first + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: second + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: third + reply: + ref: + kind: Sequence + apiVersion: flows.knative.dev/v1 + name: second-sequence +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +your resources created. + +```shell +kubectl -n default create -f ./sequence1.yaml +``` + +### Create the second Sequence + +The `sequence2.yaml` file contains the specifications for creating the Sequence. +If you are using a different type of Channel, you need to change the +spec.channelTemplate to point to your desired Channel. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Sequence +metadata: + name: second-sequence +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + steps: + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: fourth + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: fifth + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: sixth + reply: + ref: + kind: Service + apiVersion: serving.knative.dev/v1 + name: event-display +``` + +```shell +kubectl -n default create -f ./sequence2.yaml +``` + +### Create the Service displaying the events created by Sequence + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: event-display +spec: + template: + spec: + containerers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +your resources created. + +```shell +kubectl -n default create -f ./event-display.yaml +``` + +### Create the PingSource targeting the first Sequence + +This will create a PingSource which will send a CloudEvent with +`{"message": "Hello world!"}` as the data payload every 2 minutes. + +```yaml +apiVersion: sources.knative.dev/v1beta2 +kind: PingSource +metadata: + name: ping-source +spec: + schedule: "*/2 * * * *" + contentType: "application/json" + data: '{"message": "Hello world!"}' + sink: + ref: + apiVersion: flows.knative.dev/v1 + kind: Sequence + name: first-sequence +``` + +```shell +kubectl -n default create -f ./ping-source.yaml +``` + +### Inspecting the results + +You can now see the final output by inspecting the logs of the event-display +pods. + +```shell +kubectl -n default get pods +``` + +Then look at the logs for the event-display pod: + +```shell +kubectl -n default logs -l serving.knative.dev/service=event-display -c user-container --tail=-1 +☁️ cloudevents.Event +Validation: valid +Context Attributes, + specversion: 1.0 + type: dev.knative.sources.ping + source: /apis/v1/namespaces/default/pingsources/ping-source + id: 29d531df-78d8-4d11-9ffd-ba24045241a9 + time: 2020-03-02T21:18:00.0011708Z + datacontenttype: application/json +Extensions, + knativehistory: first-sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; first-sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; first-sequence-kn-sequence-2-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-0-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-1-kn-channel.default.svc.cluster.local; second-sequence-kn-sequence-2-kn-channel.default.svc.cluster.local + traceparent: 00-e5abc9de525a89ead80560b8f328de5c-fc12b64a6296f541-00 +Data, + { + "id": 0, + "message": "Hello world! - Handled by 0 - Handled by 1 - Handled by 2 - Handled by 3 - Handled by 4 - Handled by 5" + } +``` + +And you can see that the initial PingSource message `("Hello World!")` has been +appended to it by each of the steps in the Sequence. diff --git a/docs/eventing/samples/sequence/sequence-terminal/README.md b/docs/eventing/samples/sequence/sequence-terminal/README.md deleted file mode 100644 index 51ca2e93b6a..00000000000 --- a/docs/eventing/samples/sequence/sequence-terminal/README.md +++ /dev/null @@ -1,191 +0,0 @@ -We are going to create the following logical configuration. We create a -PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md). -Sequence can then do either external work, or out of band create additional -events. - -![Logical Configuration](./sequence-terminal.png) - -The functions used in these examples live in -[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). - -## Prerequisites - -For this example, we'll assume you have set up an `InMemoryChannel` as well as -Knative Serving (for our functions). The examples use `default` namespace, -again, if you want to deploy to another Namespace, you will need to modify the -examples to reflect this. - -If you want to use different type of `Channel`, you will have to modify the -`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. - -## Setup - -### Create the Knative Services - -First create the 3 steps that will be referenced in the Steps. - -```yaml -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: first -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 0" - ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: second -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 1" ---- -apiVersion: serving.knative.dev/v1 -kind: Service -metadata: - name: third -spec: - template: - spec: - containers: - - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender - env: - - name: MESSAGE - value: " - Handled by 2" ---- - -``` - -```shell -kubectl -n default create -f ./steps.yaml -``` - -### Create the Sequence - -The `sequence.yaml` file contains the specifications for creating the Sequence. -If you are using a different type of Channel, you need to change the -spec.channelTemplate to point to your desired Channel. - -```yaml -apiVersion: flows.knative.dev/v1 -kind: Sequence -metadata: - name: sequence -spec: - channelTemplate: - apiVersion: messaging.knative.dev/v1 - kind: InMemoryChannel - steps: - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: first - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: second - - ref: - apiVersion: serving.knative.dev/v1 - kind: Service - name: third -``` - -Change `default` below to create the `Sequence` in the Namespace where you want -the resources to be created. - -Here, if you are using different type of Channel, you need to change the -spec.channelTemplate to point to your desired Channel. - -```shell -kubectl -n default create -f ./sequence.yaml -``` - -### Create the PingSource targeting the Sequence - -This will create a PingSource which will send a CloudEvent with -`{"message": "Hello world!"}` as the data payload every 2 minutes. - -```yaml -apiVersion: sources.knative.dev/v1beta2 -kind: PingSource -metadata: - name: ping-source -spec: - schedule: "*/2 * * * *" - contentType: "application/json" - data: '{"message": "Hello world!"}' - sink: - ref: - apiVersion: flows.knative.dev/v1 - kind: Sequence - name: sequence -``` - -```shell -kubectl -n default create -f ./ping-source.yaml -``` - -### Inspecting the results - -You can now see the final output by inspecting the logs of the event-display -pods. Note that since we set the `PingSource` to emit every 2 minutes, it might -take some time for the events to show up in the logs. - -```shell -kubectl -n default get pods -``` - -Let's look at the logs for the first `Step` in the `Sequence`: - -```shell -kubectl -n default logs -l serving.knative.dev/service=first -c user-container --tail=-1 - -2020/03/02 21:28:00 listening on 8080, appending " - Handled by 0" to events -2020/03/02 21:28:01 Received a new event: -2020/03/02 21:28:01 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world!} -2020/03/02 21:28:01 Transform the event to: -2020/03/02 21:28:01 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0} -``` -And you can see that the initial PingSource message `("Hello World!")` has now -been modified by the first step in the Sequence to include " - Handled by 0". -Exciting :) - -Then we can look at the output of the second Step in the `Sequence`: - -```shell -kubectl -n default logs -l serving.knative.dev/service=second -c user-container --tail=-1 - -2020/03/02 21:28:02 listening on 8080, appending " - Handled by 1" to events -2020/03/02 21:28:02 Received a new event: -2020/03/02 21:28:02 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0} -2020/03/02 21:28:02 Transform the event to: -2020/03/02 21:28:02 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1} -``` -And as expected it's now been handled by both the first and second Step as -reflected by the Message being now: "Hello world! - Handled by 0 - Handled by 1" - -Then we can look at the output of the last Step in the `Sequence`: - -```shell -kubectl -n default logs -l serving.knative.dev/service=third -c user-container --tail=-1 - -2020/03/02 21:28:03 listening on 8080, appending " - Handled by 2" to events -2020/03/02 21:28:03 Received a new event: -2020/03/02 21:28:03 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1} -2020/03/02 21:28:03 Transform the event to: -2020/03/02 21:28:03 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1 - Handled by 2} -``` - - diff --git a/docs/eventing/samples/sequence/sequence-terminal/index.md b/docs/eventing/samples/sequence/sequence-terminal/index.md index e2e106925d5..8dc05208a0b 100644 --- a/docs/eventing/samples/sequence/sequence-terminal/index.md +++ b/docs/eventing/samples/sequence/sequence-terminal/index.md @@ -5,4 +5,194 @@ weight: 20 type: "docs" --- -{{% readfile file="README.md" %}} +We are going to create the following logical configuration. We create a +PingSource, feeding events to a [`Sequence`](../../../flows/sequence.md). +Sequence can then do either external work, or out of band create additional +events. + +![Logical Configuration](./sequence-terminal.png) + +The functions used in these examples live in +[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). + +## Prerequisites + +For this example, we'll assume you have set up an `InMemoryChannel` as well as +Knative Serving (for our functions). The examples use `default` namespace, +again, if you want to deploy to another Namespace, you will need to modify the +examples to reflect this. + +If you want to use different type of `Channel`, you will have to modify the +`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. + +## Setup + +### Create the Knative Services + +First create the 3 steps that will be referenced in the Steps. + +```yaml +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: first +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 0" + +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: second +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 1" +--- +apiVersion: serving.knative.dev/v1 +kind: Service +metadata: + name: third +spec: + template: + spec: + containers: + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/appender + env: + - name: MESSAGE + value: " - Handled by 2" +--- + +``` + +```shell +kubectl -n default create -f ./steps.yaml +``` + +### Create the Sequence + +The `sequence.yaml` file contains the specifications for creating the Sequence. +If you are using a different type of Channel, you need to change the +spec.channelTemplate to point to your desired Channel. + +```yaml +apiVersion: flows.knative.dev/v1 +kind: Sequence +metadata: + name: sequence +spec: + channelTemplate: + apiVersion: messaging.knative.dev/v1 + kind: InMemoryChannel + steps: + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: first + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: second + - ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: third +``` + +Change `default` below to create the `Sequence` in the Namespace where you want +the resources to be created. + +Here, if you are using different type of Channel, you need to change the +spec.channelTemplate to point to your desired Channel. + +```shell +kubectl -n default create -f ./sequence.yaml +``` + +### Create the PingSource targeting the Sequence + +This will create a PingSource which will send a CloudEvent with +`{"message": "Hello world!"}` as the data payload every 2 minutes. + +```yaml +apiVersion: sources.knative.dev/v1beta2 +kind: PingSource +metadata: + name: ping-source +spec: + schedule: "*/2 * * * *" + contentType: "application/json" + data: '{"message": "Hello world!"}' + sink: + ref: + apiVersion: flows.knative.dev/v1 + kind: Sequence + name: sequence +``` + +```shell +kubectl -n default create -f ./ping-source.yaml +``` + +### Inspecting the results + +You can now see the final output by inspecting the logs of the event-display +pods. Note that since we set the `PingSource` to emit every 2 minutes, it might +take some time for the events to show up in the logs. + +```shell +kubectl -n default get pods +``` + +Let's look at the logs for the first `Step` in the `Sequence`: + +```shell +kubectl -n default logs -l serving.knative.dev/service=first -c user-container --tail=-1 + +2020/03/02 21:28:00 listening on 8080, appending " - Handled by 0" to events +2020/03/02 21:28:01 Received a new event: +2020/03/02 21:28:01 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world!} +2020/03/02 21:28:01 Transform the event to: +2020/03/02 21:28:01 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0} +``` +And you can see that the initial PingSource message `("Hello World!")` has now +been modified by the first step in the Sequence to include " - Handled by 0". +Exciting :) + +Then we can look at the output of the second Step in the `Sequence`: + +```shell +kubectl -n default logs -l serving.knative.dev/service=second -c user-container --tail=-1 + +2020/03/02 21:28:02 listening on 8080, appending " - Handled by 1" to events +2020/03/02 21:28:02 Received a new event: +2020/03/02 21:28:02 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0} +2020/03/02 21:28:02 Transform the event to: +2020/03/02 21:28:02 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1} +``` +And as expected it's now been handled by both the first and second Step as +reflected by the Message being now: "Hello world! - Handled by 0 - Handled by 1" + +Then we can look at the output of the last Step in the `Sequence`: + +```shell +kubectl -n default logs -l serving.knative.dev/service=third -c user-container --tail=-1 + +2020/03/02 21:28:03 listening on 8080, appending " - Handled by 2" to events +2020/03/02 21:28:03 Received a new event: +2020/03/02 21:28:03 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1} +2020/03/02 21:28:03 Transform the event to: +2020/03/02 21:28:03 [2020-03-02T21:28:00.0010247Z] /apis/v1/namespaces/default/pingsources/ping-source dev.knative.sources.ping: &{Sequence:0 Message:Hello world! - Handled by 0 - Handled by 1 - Handled by 2} +``` + + diff --git a/docs/eventing/samples/sequence/sequence-with-broker-trigger/README.md b/docs/eventing/samples/sequence/sequence-with-broker-trigger/README.md deleted file mode 100644 index 998400bfe0a..00000000000 --- a/docs/eventing/samples/sequence/sequence-with-broker-trigger/README.md +++ /dev/null @@ -1,263 +0,0 @@ -We are going to create the following logical configuration. We create a -PingSource, feeding events into the Broker, then we create a `Filter` that wires -those events into a [`Sequence`](../../../flows/sequence.md) consisting of 3 -steps. Then we take the end of the Sequence and feed newly minted events back -into the Broker and create another Trigger which will then display those events. - -## Prerequisites - -- Knative Serving -- `InMemoryChannel` - -**NOTE:** The examples use the `default` namespace. - -If you want to use different type of `Channel`, you will have to modify the -`Sequence.Spec.ChannelTemplate` to create the appropriate Channel resources. - -![Logical Configuration](./sequence-with-broker-trigger.png) - -The functions used in these examples live in -[https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go](https://github.com/knative/eventing-contrib/blob/main/cmd/appender/main.go). - -## Setup - -### Creating the Broker - -To create the cluster default Broker type: - -```shell -kubectl create -f - <