Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/i18n/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@
"node_modules/react/README": {
"title": "node_modules/react/README"
},
"obj/perspective-python": {
"title": "perspective-python API"
},
"obj/perspective-viewer": {
"title": "perspective-viewer API"
"obj/perspective-python": {
"title": "perspective-python API"
},
"obj/perspective-viewer": {
"title": "perspective-viewer API"
},
"obj/perspective": {
"title": "perspective API"
Expand Down
21 changes: 11 additions & 10 deletions docs/md/js.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ different visualization on load or transform the dataset, use the viewer's attri
```

For more details about the full attribute API, see the
[`<perspective-viewer>`](/js.html#setting--reading-viewer-configuration-via-attributes)section of this user guide.
[`<perspective-viewer>`](#setting--reading-viewer-configuration-via-attributes)section of this user guide.

## Module Structure

Expand Down Expand Up @@ -740,14 +740,10 @@ const fs = require("fs");
// module's directory.
const host = new WebSocketServer({assets: [__dirname], port: 8080});

// Read an arrow file from the file system and load it as a named table.
// Read an arrow file from the file system and host it as a named table.
const arr = fs.readFileSync(__dirname + "/superstore.arrow");
const tbl = table(arr);
host.host_table("table_one", tbl);

// Or host a view
const view = tbl.view({filter: [["State", "==", "Texas"]]});
host.host_view("view_one", view);
```

In the browser:
Expand All @@ -760,11 +756,16 @@ const websocket = perspective.websocket(window.location.origin.replace("http", "

// Bind the viewer to the preloaded data source. `table` and `view` objects
// live on the server.
elem.load(websocket.open_table("table_one"));
const server_table = websocket.open_table("table_one");
elem.load(server_table);

// Or load data from a view. The browser now also has a copy of this view in
// its own `table`, as well as its updates. Transfer uses Arrows.
elem.load(websocket.open_view("view_one"));
// Or load data from a table using a view. The browser now also has a copy of
// this view in its own `table`, as well as its updates transferred to the
// browser using Apache Arrow.
const worker = perspective.worker();
const server_view = server_table.view();
const client_table = worker.table(server_view);
elem.load(client_table);
```

`<perspective-viewer>` instances bound in this way are otherwise no different
Expand Down
134 changes: 73 additions & 61 deletions docs/md/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,31 +302,44 @@ of `PerspectiveManager` with a string name and the instance to be hosted:
```python
manager = PerspectiveManager()
table = Table(data)
view = table.view()
manager.host_table("data_source", table)
manager.host_view("view_1", view)
```

The `name` provided is important, as it enables Perspective in Javascript to look
up a `Table`/`View` and get a handle to it over the network. This enables
up a `Table` and get a handle to it over the network. This enables
several powerful server/client implementations of Perspective, as explained in
the next section.

### Using a hosted `Table` in Javascript
### Distributed Mode

Using Tornado and [`PerspectiveTornadoHandler`](/docs/md/python.html#perspectivetornadohandler),
as well as `Perspective`'s Javascript library, we can create a client/server
architecture that hosts and transforms _massive_ datasets with minimal
client resource usage.

Perspective's design allows a `table()` created in Javascript to _proxy_ its
operations to a `Table` created in Python, which executes the operations in
the Python kernel, and returns the results of the operation to the browser.
All of this is _enabled_ through `PerspectiveManager`, which handles messaging,
processing method calls, serializing outputs for the network, etc.

In Python, use `PerspectiveManager` and `PerspectiveTornadoHandler` to create
a websocket server that exposes a `Table`:
as well as `Perspective`'s Javascript library, we can set up "distributed"
Perspective instances that allows multiple browser `perspective-viewer`
clients to read from a common `perspective-python` server. In exchange for sending the
entire dataset to the client on initialization, server load is reduced and
client performance is not network-dependent.

[Example](https://github.com/finos/perspective/tree/master/examples/tornado-python)

This architecture works by maintaining two `Tables`—one on the server, and one
on the client that mirrors the server's `Table` automatically using `on_update`.
All updates to the table on the server are automatically applied to each client,
which makes this architecture a natural fit for streaming dashboards and other
distributed use-cases.

Because the `Table` is mirrored, the user gets all the performance benefits of
Perspective in WebAssembly, and can examine server-hosted datasets with zero
network lag on their interactions.

In conjunction with [Async Mode](#async-mode), distributed Perspective offers
consistently high performance over large numbers of clients and large datasets.
As server dataset sizes increase, the initial load time of a client will
increase, but once the data is loaded there is no network lag visible to the
user.

Using the [tornado-python](https://github.com/finos/perspective/tree/master/examples/tornado-python)
example, one can easily create a distributed Perspective server using
`server.py` and `index.html`:

_*server.py*_

Expand All @@ -335,7 +348,7 @@ from perspective import Table, PerspectiveManager, PerspectiveTornadoHandler

# Create an instance of PerspectiveManager, and host a Table
MANAGER = PerspectiveManager()
TABLE = Table(large_dataset)
TABLE = Table(data)

# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
MANAGER.host_table("data_source", TABLE)
Expand All @@ -352,13 +365,9 @@ loop = tornado.ioloop.IOLoop.current()
loop.start()
```

`PerspectiveTornadoHandler`, as outlined in the [docs](/docs/md/python.html#perspectivetornadohandler),
takes a `PerspectiveManager` instance exposes it over a websocket at the URL
specified. This allows a `table()` in Javascript to access the `Table` in
Python and read data from it.

Most importantly, the client code in Javascript does not require Webpack or any
bundler, and can be implemented in a single HTML file:
Instead of calling `load(server_table)`, create a `View` using `server_table`
and pass that into `viewer.load()`. This will automatically register an
`on_update` callback that synchronizes state between the server and the client.

_*index.html*_

Expand All @@ -371,58 +380,61 @@ _*index.html*_
// to accept connections at the specified URL.
const websocket = perspective.websocket("ws://localhost:8888/websocket");

/* `table` is a proxy for the `Table` we created on the server.
// Get a handle to the Table on the server
const server_table = websocket.open_table("data_source_one");

All operations that are possible through the Javascript API are possible
on the Python API as well, thus calling `view()`, `schema()`, `update()`
etc. on `const table` will pass those operations to the Python `Table`,
execute the commands, and return the result back to Javascript.*/
const table = websocket.open_table("data_source_one");
// Create a new view
const server_view = table.view();

// Load this in the `<perspective-viewer>`.
document.getElementById("viewer").load(table);
// Create a Table on the client using `perspective.worker()`
const worker = perspective.worker();
const client_table = worker.table(view);

// Load the client table in the `<perspective-viewer>`.
document.getElementById("viewer").load(client_table);
});
</script>
```

### Using a hosted `View` in Javascript
For a more complex example that offers distributed editing of the server
dataset, see [client_server_editing.html](https://github.com/finos/perspective/blob/master/examples/tornado-python/client_server_editing.html).

An alternative client/server architecture using `PerspectiveTornadoHandler` and
`PerspectiveManager` involves hosting a `View`, and creating a new `table()` in
Javascript on top of the Python `View`.
### Server Mode

When the `table()` is created in Javascript, it serializes the Python `View`'s
data into Arrow, transfers it into the Javascript `table()`, and sets up an
`on_update` callback to `update()` the Table whenever the Python `View`'s
`Table` updates.
An alternative architecture uses a single `Table` on the Python server, which
allows hosting of _massive_ datasets with minimal client resource usage. This
comes at the expense of client-side performance, as all operations must be
proxied over the network to the server.

Implementing the server in Python is extremely similar to the implementation
described in the last section.
The server setup is identical to [Distributed Mode](#distributed-mode) above,
but instead of creating a view, the client calls `load(server_table)`:
In Python, use `PerspectiveManager` and `PerspectiveTornadoHandler` to create
a websocket server that exposes a `Table`:

Replace `host_table` with `host_view`:
_*index.html*_

```python
# we have an instance of `PerspectiveManager`
TABLE = Table(data)
VIEW = TABLE.view()
MANAGER.host_view("view_one", VIEW)
```html
<perspective-viewer id="viewer" editable></perspective-viewer>

# Continue with Tornado setup
```
<script>
window.addEventListener("WebComponentsReady", async function () {
// Create a client that expects a Perspective server
// to accept connections at the specified URL.
const websocket = perspective.websocket("ws://localhost:8888/websocket");

Changes to the client code are also minimal. Use `open_view` instead of
`open_table`:
/* `table` is a proxy for the `Table` we created on the server.

```javascript
// const websocket has been defined already
const view = websocket.open_view("view_one");
const table = perspective.table(view);
// continue with loading the table into `<perspective-viewer>
```
All operations that are possible through the Javascript API are possible
on the Python API as well, thus calling `view()`, `schema()`, `update()`
etc. on `const table` will pass those operations to the Python `Table`,
execute the commands, and return the result back to Javascript.*/
const table = websocket.open_table("data_source_one");

The benefit of this design is that only new updates will be sent to the client,
efficiently serialized in the Apache Arrow format. In exchange for sending the
entire dataset to the client on initialization, it reduces load on the server.
// Load this in the `<perspective-viewer>`.
document.getElementById("viewer").load(table);
});
</script>
```

## `PerspectiveWidget`

Expand Down
3 changes: 2 additions & 1 deletion examples/git-history/chained.html
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@
window.addEventListener('WebComponentsReady', async function() {
var elem = document.getElementById('view1');
var client = perspective.websocket();
let view = client.open_view('data_source_one');
let table = client.open_table('data_source');
let view = table.view();
let arrow = await view.to_arrow();
elem.load(perspective.worker().table(arrow));
});
Expand Down
7 changes: 3 additions & 4 deletions examples/remote-express-typescript/assets/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,9 @@
const worker = perspective.worker();
worker.initialize_profile_thread();

// Get a proxy for a view named "data_source_one", registered on
// the server with a reciprocal call to `host_view()`.
// No data is transferred, `view` is a virtual handle for data on
// the server.
// Open a `Table` that is hosted on the server. All instructions
// will be proxied to the server `Table` - no calculations are
// done on the client.
const table = websocket.open_table('remote_table');

// Create a `table` from this, owned by the local WebWorker.
Expand Down
7 changes: 3 additions & 4 deletions examples/remote-express/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,9 @@
const worker = perspective.worker();
worker.initialize_profile_thread();

// Get a proxy for a view named "data_source_one", registered on
// the server with a reciprocal call to `host_view()`.
// No data is transferred, `view` is a virtual handle for data on
// the server.
// Open a `Table` that is hosted on the server. All instructions
// will be proxied to the server `Table` - no calculations are
// done on the client.
const table = websocket.open_table('remote_table');

// Create a `table` from this, owned by the local WebWorker.
Expand Down
2 changes: 1 addition & 1 deletion examples/remote-workspace/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ const perspective = require("@finos/perspective");
const {securities} = require("../datasources");

const host = new perspective.WebSocketServer();
securities().then(table => host.host_view("securities", table.view()));
securities().then(table => host.host_table("securities_table", table));
4 changes: 2 additions & 2 deletions examples/remote-workspace/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ import "./index.less";
window.addEventListener("load", () => {
const websocket = perspective.websocket("ws://localhost:8080");
const worker = perspective.shared_worker();
const view = websocket.open_view("securities");
const table = worker.table(view, {limit: 10000});
const server_table = websocket.open_table("securities_table");
const table = worker.table(server_table.view(), {limit: 10000});

const workspace = document.createElement("perspective-workspace");
document.body.appendChild(workspace);
Expand Down
15 changes: 9 additions & 6 deletions examples/tornado-python/client_server_editing.html
Original file line number Diff line number Diff line change
Expand Up @@ -55,17 +55,20 @@
const websocket = perspective.websocket("ws://localhost:8080/websocket");
const worker = perspective.shared_worker();

// Get handles to both the `Table` and `View`, as we manually set
// up the client/server editing.
// Get handles to the `Table` on the server, and create a
// `view()` on the server.
const server_table = websocket.open_table("data_source_one");
const server_view = websocket.open_view("view_one");
const server_view = server_table.view();

// Serialize the current state of the view to an arrow, and create
// a Table on the client that has the same index as the Table
// on the server. Client/server editing does not work on an
// unindexed Table.
// on the server using `get_index()`.
//
// Client/server editing does not work on an unindexed Table.
const arrow = await server_view.to_arrow();
const client_table = worker.table(arrow, {index: "Row ID"});
const client_table = worker.table(arrow, {
index: await server_table.get_index()
});
const client_view = client_table.view();// client -> server

await viewer.load(client_table);
Expand Down
24 changes: 18 additions & 6 deletions examples/tornado-python/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,23 @@
* state is automatically synchronized between the two.
*
* To create a Perspective client running in "distributed mode",
* host a `View` in the Python server and create a `Table` in the
* client using a proxy to the hosted `View`.
* host a `Table` in the Python server, create a `Table` on the
* client through a `View` on the server. This synchronizes the
* server's `Table` with the client's `Table`.
*/
const worker = perspective.worker();

/**
* `view` is a proxy to the hosted `View` in the Python server. All
* public API methods are available through this proxied view.
* `server_table` is a proxy to the hosted `Table` in the Python
* server. All public API methods are available through this
* proxied table.
*/
const view = websocket.open_view('view_one');
const server_table = websocket.open_table("data_source_one");

/**
* Create a `View` on the server.
*/
const server_view = server_table.view();

/**
* When a `View` is passed into `table()` to create a new `Table`,
Expand All @@ -74,11 +81,16 @@
* `Table` and its data (and associated viewer) are still fully
* functional, but will not be kept up-to-date with server state.
*
* Use `table.get_index()` in order to create a `Table` on the
* client that has the exact same settings as the server.
*
* For a more complex example of manually controlling state
* synchronization between server and client, see
* `client_server_editing.html`.
*/
const table = worker.table(view);
const table = worker.table(server_view, {
index: await server_table.get_index()
});

// Load the local table in the `<perspective-viewer>`.
document.getElementById('viewer').load(table);
Expand Down
1 change: 0 additions & 1 deletion examples/tornado-python/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ def perspective_thread(manager):
with open(file_path, mode="rb") as file:
table = Table(file.read(), index="Row ID")
manager.host_table("data_source_one", table)
manager.host_view("view_one", table.view())
psp_loop.start()


Expand Down
Loading