Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 46 additions & 6 deletions config/provisioners/kafka/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,35 @@

Deployment steps:
1. Setup [Knative Eventing](../../../DEVELOPMENT.md)
1. Install an Apache Kafka cluster. There are two choices:
1. If not done already, install an Apache Kafka cluster. There are two choices:
* Simple installation of [Apache Kafka](broker).
* A production grade installation using the [Strimzi Kafka Operator](strimzi).
Installation [guides](http://strimzi.io/quickstarts/) are provided for
kubernetes and Openshift.

1. Now that the Apache Kafka is installed, apply the 'Kafka' ClusterChannelProvisioner:
1. Now that Apache Kafka is installed, you need to configure the
`bootstrap_servers` value in the `kafka-channel-controller-config` ConfigMap,
located inside the `config/provisioners/kafka/kafka-provisioner.yaml` file:
```
...
apiVersion: v1
kind: ConfigMap
metadata:
name: kafka-channel-controller-config
namespace: knative-eventing
data:
# Broker URL's for the provisioner
bootstrap_servers: kafkabroker.kafka:9092
...
```
> Note: The `bootstrap_servers` needs to contain the address of at least
one broker of your Apache Kafka cluster. If you are using Strimzi, you need
to update the `bootstrap_servers` value to
`my-cluster-kafka-bootstrap.mynamespace:9092`.
1. Apply the 'Kafka' ClusterChannelProvisioner, Controller, and Dispatcher:
```
ko apply -f config/provisioners/kafka/kafka-provisioner.yaml
```
> Note: If you are using Strimzi, you need to update the `KAFKA_BOOTSTRAP_SERVERS` value in the `kafka-channel-controller-config` ConfigMap to `my-cluster-kafka-bootstrap.kafka.9092`.
1. Create Channels that reference the 'kafka-channel'.

```yaml
Expand All @@ -24,15 +44,35 @@ Deployment steps:
kind: ClusterChannelProvisioner
name: kafka-channel
```
1. (Optional) Install [Kail](https://github.com/boz/kail) - Kubernetes tail

## Components

The major components are:
* ClusterChannelProvisioner Controller
* Channel Controller Config Map
* Channel Controller
* Channel Controller Config Map.
* Channel Dispatcher
* Channel Dispatcher Config Map.

The ClusterChannelProvisioner Controller and the Channel Controller are colocated in one Pod.
The ClusterChannelProvisioner Controller and the Channel Controller are colocated
in one Pod:
```shell
kubectl get deployment -n knative-eventing kafka-channel-controller
```

The Channel Controller Config Map is used to configure the `bootstrap_servers`
of your Apache Kafka installation:
```shell
kubectl get configmap -n knative-eventing kafka-channel-dispatcher-config-map
```

The Channel Dispatcher receives and distributes all events:
```shell
kubectl get statefulset -n knative-eventing kafka-channel-dispatcher
```

The Channel Dispatcher Config Map is used to send information about Channels and
Subscriptions from the Channel Controller to the Channel Dispatcher:
```shell
kubectl get configmap -n knative-eventing kafka-channel-dispatcher-config-map
```
2 changes: 1 addition & 1 deletion config/provisioners/kafka/broker/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Apache Kafka - simple installation

1. For an installation of a simple Apache Kafka cluster, a setup is provided:
1. For an installation of a simple (**non production**) Apache Kafka cluster, a setup is provided:
```
kubectl create namespace kafka
kubectl apply -n kafka -f kafka-broker.yaml
Expand Down
31 changes: 0 additions & 31 deletions config/provisioners/kafka/strimzi/README.md

This file was deleted.

23 changes: 0 additions & 23 deletions config/provisioners/kafka/strimzi/kafka-ephemeral.yaml

This file was deleted.

27 changes: 0 additions & 27 deletions config/provisioners/kafka/strimzi/kafka-persistent.yaml

This file was deleted.