Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
*.tar.gz
*.rar

node_modules
package-lock.json

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*

Expand Down
76 changes: 74 additions & 2 deletions cloud/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,87 @@

## Programming Languages

This directory includes examples of Apache Pulsar client applications, showcasing producers and consumers, written in various programming languages. The README for each language walks through the necessary steps to run each example. When each client establishes a connection with the Pulsar cluster through OAuth2, it needs to obtain the specified options from the Pulsar cluster and OAuth2 services. [How to get OAuth2 options](#How to get OAuth2 options) explains how you get these options.
This directory includes examples of Apache Pulsar client applications, showcasing producers and consumers, written in various programming languages. The README for each language walks through the necessary steps to run each example. When each client establishes a connection with the Pulsar cluster through **OAuth2** or **Token**.

For the OAuth2, it needs to obtain the specified options from the Pulsar cluster and OAuth2 services. [How to get OAuth2 options](#How to get OAuth2 options) explains how you get these options.

Currently, we support the following three languages to connect through OAuth2:

- Java
- Go
- CPP

For clients in other languages, you can connect through token, reference to [here](https://pulsar.apache.org/docs/en/security-tls-transport/#client-configuration).
The following clients and Pulsar CLI tools are supported to connect to cluster through the Token. And [How to get Token options](#How to get Token options) explains how you get these options.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about the websocket client

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

next pull request, I need to investigate


- Java
- Go
- CPP
- Python
- CSharp(TODO)
- NodeJS
- pulsarctl
- pulsar-admin
- pulsar-client
- pulsar-perf

## How to get token options

When you use Token to connect to Pulsar cluster, you need to provide the following options:

- `SERVICE_URL`
- `WEB_SERVICE_URL`
- `AUTH_PARAMS`

For the `SERVICE_URL` field, you can get the **hostname** through the following command:

```shell script
$ snctl get pulsarclusters [CLUSTER_NAME] -n [NAMESPACE] -o json | jq '.spec.serviceEndpoints[0].dnsName'
```

Output:

```text
api.test.cloud.xxx.streamnative.dev
```

A `SERVICE_URL` is a combination of protocol, hostname and port, so an example of a complete `SERVICE_URL` is as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

port ID?



```text
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two service urls seem to be local. i don't know whether it is Ok.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is just an output example, users can get the real hostname through the above command

pulsar://api.test.cloud.xxx.streamnative.dev:6650

# For tls
pulsar+ssl://api.test.cloud.xxx.streamnative.dev:6651
```

For the `WEB_SERVICE_URL` field, you can get the **hostname** through the following command:

```shell script
$ snctl get pulsarclusters [CLUSTER_NAME] -n [NAMESPACE] -o json | jq '.spec.serviceEndpoints[0].dnsName'
```

Output:

```text
api.test.cloud.xxx.streamnative.dev
```

A `WEB_SERVICE_URL` is a combination of protocol, hostname and port, so an example of a complete `WEB_SERVICE_URL` is as follows:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

port ID?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?


```text
http://api.test.cloud.xxx.streamnative.dev:8080

# For tls
https://api.test.cloud.xxx.streamnative.dev:8443
```

For the `AUTH_PARAMS` field, you can get it through the following command:

```shell script
$ snctl auth get-token [INSTANCE] [flags]
```

> Tips: In code implementation, for safety and convenience, you can consider setting `AUTH_PARAMS` as an environment variable.

## How to get OAuth2 options

Expand Down
36 changes: 36 additions & 0 deletions cloud/cpp/ConnectByOAuth2.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@

/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
#include <iostream>
#include <pulsar/Client.h>

using namespace pulsar;

int main() {
ClientConfiguration config;
std::string params = R"({
"issuer_url": "https://dev-kt-aa9ne.us.auth0.com/oauth/token",
"private_key": "../../pulsar-broker/src/test/resources/authentication/token/cpp_credentials_file.json",
"audience": "https://dev-kt-aa9ne.us.auth0.com/api/v2/"})";

config.setAuth(pulsar::AuthOauth2::create(params));

Client client("pulsar+ssl://cluster.test.us-east4.streamnative.test.g.sn2.dev:6651", config);
client.close();
}
31 changes: 31 additions & 0 deletions cloud/cpp/ConnectByToken.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@

/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
#include <iostream>
#include <pulsar/Client.h>

using namespace pulsar;

int main() {
ClientConfiguration config;
config.setAuth(AuthToken::createWithToken("AUTH_PARAMS"));
Client client(SERVICE_URL, config);

client.close();
}
10 changes: 10 additions & 0 deletions cloud/cpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ Produce message to and consume message from a Pulsar cluster using [Apache pulsa

# Prerequisites

## Linux

Since 2.1.0 release, Pulsar ships pre-built RPM and Debian packages. You can download and install those packages directly.

For more information, refer to [here](https://pulsar.apache.org/docs/en/client-libraries-cpp/#supported-platforms).

## MacOS

Pulsar releases are available in the Homebrew core repository. You can install the C++ client library with the following command. The package is installed with the library and headers.

```shell script
Expand All @@ -16,6 +24,8 @@ In this example, the producer will publish data to the `topic-1` in your Pulsar
The content of each message payload is `content`.
The consumer will receive the message from the `topic-1` and `ack` the receipt of each message received.

> Tips: The following code example uses the OAuth2 connection method. If you want to connect to the Pulsar cluster using Token, please refer to the implementation of **connectByToken.cc**.

1. Run the consumer, and start to receiving the message from `topic-1`:

```bash
Expand Down
3 changes: 3 additions & 0 deletions cloud/csharp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Overview
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mode contents required?


This document describes how to produce message to and consume message from a Pulsar cluster using [Apache pulsar-dotpulsar](https://github.com/apache/pulsar-dotpulsar).
2 changes: 2 additions & 0 deletions cloud/go/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ In this example, the producer publishes data to the `topic-1` in your Pulsar clu
The content of each message payload is a combination of `hello-` and a digital (0-9) (e.g: `hello-0`).
The consumer receives the message from the `topic-1` and `acknowledges` each received message.

> Tips: The following code example uses the OAuth2 connection method. If you want to connect to the Pulsar cluster using Token, please refer to the implementation of **connectByToken.go**.

1. Run the consumer, and start to receiving the message from `topic-1`:

```bash
Expand Down
34 changes: 34 additions & 0 deletions cloud/go/connectByToken.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package main

import (
"github.com/apache/pulsar-client-go/pulsar"
"log"
)

func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: SERVICE_URL,
Authentication: pulsar.NewAuthenticationToken(AUTH_PARAMS),
})
if err != nil {
log.Fatal(err)
}
defer client.Close()
}
4 changes: 3 additions & 1 deletion cloud/java/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@ This document describes how to produce messages to and consume messages from a A
- Java: 1.8+
- Pulsar broker: 2.7.0-742fc5c9b+

> You can get this tarball from [bintray](https://bintray.com/streamnative/maven/org.apache.pulsar/2.7.0-742fc5c9b). When the 2.6.1 release is complete, you can also use the official 2.6.1 version.
> You can get this tarball from [bintray](https://bintray.com/streamnative/maven/org.apache.pulsar/2.7.0-742fc5c9b). When Pulsar 2.6.1 is released, you can also use the official 2.6.1 version.

# Example

In this example, the producer publishes messages to the `topic-1` in your Pulsar cluster.
The content of each message payload is a combination of `my-message-` and a digital (0-9) (e.g: `my-message-0`).
The consumer receives the message from the `topic-1` and `acknowledges` each received message.

> Tips: The following code example uses the OAuth2 connection method. If you want to connect to the Pulsar cluster using Token, please refer to the implementation of **connectByToken.java**.

1. Run the consumer.

```shell script
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
import org.apache.pulsar.client.impl.auth.oauth2.AuthenticationFactoryOAuth2;

public class ConnectByOauth2 {
public static void main(String[] args) throws Exception{
public static void main(String[] args) throws Exception {
String issuerUrl = "https://dev-kt-aa9ne.us.auth0.com/oauth/token";
String credentialsUrl = "file:///path/to/KeyFile.json";
String audience = "https://dev-kt-aa9ne.us.auth0.com/api/v2/";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package io.streamnative.examples.oauth2;

import org.apache.pulsar.client.api.AuthenticationFactory;
import org.apache.pulsar.client.api.PulsarClient;

public class ConnectByToken {
public static void main(String[] args) throws Exception {
String SERVICE_URL = "";
String AUTH_PARAMS = "";

PulsarClient client = PulsarClient.builder()
.serviceUrl(SERVICE_URL)
.authentication(AuthenticationFactory.token(AUTH_PARAMS))
.build();

client.close();
}
}
75 changes: 75 additions & 0 deletions cloud/node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Overview

This document describes how to produce message to and consume message from a Pulsar cluster using [Apache pulsar-client-node](https://github.com/apache/pulsar-client-node).

# Prerequisites

Pulsar Node.js client library is based on the C++ client library. Follow the instructions for
[C++ library](https://pulsar.apache.org/docs/en/client-libraries-cpp/) for installing the binaries through
[RPM](https://pulsar.apache.org/docs/en/client-libraries-cpp/#rpm),
[Deb](https://pulsar.apache.org/docs/en/client-libraries-cpp/#deb) or
[Homebrew packages](https://pulsar.apache.org/docs/en/client-libraries-cpp/#macos).

> #### Note
> You need to install both the pulsar-client library and the pulsar-client-dev library.

Also, this library works only in Node.js 10.x or later because it uses the
[node-addon-api](https://github.com/nodejs/node-addon-api) module to wrap the C++ library.

## Install

### Install pulsar-client in your project:

```shell
$ npm install pulsar-client
```

# Example

In this example, the producer publishes data to the `my-topic` in your Pulsar cluster.
The content of each message payload is a combination of `my-message-` and a digital (0-9) (e.g: `my-message-0`).
The consumer receives the message from the `my-topic` and `acknowledges` each received message.

1. Run the consumer, and start to receiving the message from `my-topic`:

```bash
$ cd cloud/node
$ node sample_consumer.js
```

Output:

```text
my-message-0
my-message-1
my-message-3
my-message-2
my-message-4
my-message-5
my-message-7
my-message-6
my-message-8
my-message-9
```

2. Run the producer and publish messages to the `my-topic`:

```bash
$ cd cloud/node
$ node sample_producer.js
```

Output:

```text
Sent message: my-message-0
Sent message: my-message-1
Sent message: my-message-2
Sent message: my-message-3
Sent message: my-message-4
Sent message: my-message-5
Sent message: my-message-6
Sent message: my-message-7
Sent message: my-message-8
Sent message: my-message-9
```
Loading