From b330154edb7f033464e17317fbc2c700f86f8a23 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Sat, 5 Sep 2020 22:45:51 -0400 Subject: [PATCH 1/9] add initial file descriptors document --- design/file-descriptors.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 design/file-descriptors.md diff --git a/design/file-descriptors.md b/design/file-descriptors.md new file mode 100644 index 000000000..eb408a232 --- /dev/null +++ b/design/file-descriptors.md @@ -0,0 +1,17 @@ +# File Descriptors +File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)) are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. + +# Null File Descriptor +A File Descriptor of 0 is a null file descriptor. It's used whenever there is *no* file descriptor. Passing this to a method is a no-no, and receiving one from a method must be handled. + +# The Console +When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `1`, `2`, and `3`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below: + +```wat +(module + ;; TODO: small hello world? +) +``` + +# Files +Files can be opened and closed with the method [`path_open`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-path_openfd-fd-dirflags-lookupflags-path-string-oflags-oflags-fs_rights_base-rights-fs_rights_inherting-rights-fdflags-fdflags---errno-fd), and simply drop the returned `fd` to close it. These commands will fail if the WASM environment did not give the program permission to execute these. From 2f89d3c580475a6a467e4055cd0f987a69026f2e Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Sun, 6 Sep 2020 04:12:09 -0400 Subject: [PATCH 2/9] Fix null/1/2/3 fd misinformation --- design/file-descriptors.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index eb408a232..4103dcb67 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -1,11 +1,8 @@ # File Descriptors File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)) are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. -# Null File Descriptor -A File Descriptor of 0 is a null file descriptor. It's used whenever there is *no* file descriptor. Passing this to a method is a no-no, and receiving one from a method must be handled. - # The Console -When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `1`, `2`, and `3`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below: +When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below: ```wat (module From 1ce90adb831aa51796f45ab7097c55fa35d6caab Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Sun, 6 Sep 2020 04:15:12 -0400 Subject: [PATCH 3/9] use less precise URL for referencing path_open --- design/file-descriptors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index 4103dcb67..46b0d1088 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -11,4 +11,4 @@ When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ``` # Files -Files can be opened and closed with the method [`path_open`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-path_openfd-fd-dirflags-lookupflags-path-string-oflags-oflags-fs_rights_base-rights-fs_rights_inherting-rights-fdflags-fdflags---errno-fd), and simply drop the returned `fd` to close it. These commands will fail if the WASM environment did not give the program permission to execute these. +Files can be opened and closed with the method [`path_open`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#path_open), and simply drop the returned `fd` to close it. These commands will fail if the WASM environment did not give the program permission to execute these. From e86b5cfdd60e00e8150de34824da7b4b6df9c793 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Tue, 22 Sep 2020 14:18:20 -0400 Subject: [PATCH 4/9] add hello world code from test suite --- design/file-descriptors.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index 46b0d1088..2c83f3863 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -2,11 +2,33 @@ File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)) are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. # The Console -When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below: +When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): ```wat (module - ;; TODO: small hello world? + (import "wasi_snapshot_preview1" "proc_exit" + (func $__wasi_proc_exit (param i32))) + (import "wasi_snapshot_preview1" "fd_write" + (func $__wasi_fd_write (param i32 i32 i32 i32) (result i32))) + (func $_start + (i32.store (i32.const 24) (i32.const 14)) + (i32.store (i32.const 20) (i32.const 0)) + (block + (br_if 0 + (call $__wasi_fd_write + (i32.const 1) + (i32.const 20) + (i32.const 1) + (i32.const 16))) + (br_if 0 (i32.ne (i32.load (i32.const 16)) (i32.const 14))) + (br 1) + ) + (call $__wasi_proc_exit (i32.const 1)) + ) + (memory 1) + (export "memory" (memory 0)) + (export "_start" (func $_start)) + (data (i32.const 0) "Hello, world!\0a") ) ``` From 416f04296ec2fc7426869ff5635a1eb5a75b98d5 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Tue, 22 Sep 2020 14:18:46 -0400 Subject: [PATCH 5/9] fix style nit --- design/file-descriptors.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index 2c83f3863..34afb0bfa 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -1,7 +1,7 @@ # File Descriptors File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)) are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. -# The Console +## The Console When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): ```wat @@ -32,5 +32,5 @@ When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ) ``` -# Files +## Files Files can be opened and closed with the method [`path_open`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#path_open), and simply drop the returned `fd` to close it. These commands will fail if the WASM environment did not give the program permission to execute these. From c7a287c45dbe00d2efdf96bf31df433e1593bfcc Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Tue, 22 Sep 2020 14:21:24 -0400 Subject: [PATCH 6/9] mention "file descriptor" being interchaneable with "handle" --- design/file-descriptors.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index 34afb0bfa..27d3114ab 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -1,5 +1,5 @@ -# File Descriptors -File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)) are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. +# File Descriptors (a.k.a handles) +File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)), often referred to as "handles" interchangeably, are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. ## The Console When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): From 7a713151fd4315f0b41f6f0ad4bc5201ce2186b1 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Tue, 22 Sep 2020 14:21:43 -0400 Subject: [PATCH 7/9] solve ordering of stdin, stdout, stderr. --- design/file-descriptors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index 27d3114ab..b611a0b86 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -2,7 +2,7 @@ File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)), often referred to as "handles" interchangeably, are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. ## The Console -When a program is initiated, `stdout`, `stdin`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): +When a program is initiated, `stdin`, `stdout`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): ```wat (module From b605ed75971af1bd275617d12ab3c0eb554c8674 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Tue, 11 May 2021 05:07:06 -0400 Subject: [PATCH 8/9] Be more specific about opening and closing files --- design/file-descriptors.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/design/file-descriptors.md b/design/file-descriptors.md index b611a0b86..b60f1ca73 100644 --- a/design/file-descriptors.md +++ b/design/file-descriptors.md @@ -33,4 +33,7 @@ When a program is initiated, `stdin`, `stdout`, and `stderr` (respectively `fd` ``` ## Files -Files can be opened and closed with the method [`path_open`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#path_open), and simply drop the returned `fd` to close it. These commands will fail if the WASM environment did not give the program permission to execute these. +Files can be opened with the method [`path_open`][path_open], and closed by calling [`fd_close`][fd_close]. These commands will fail if the WASM environment did not give the program permission to open files. + +[path_open]: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md#path_open +[fd_close]: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md#fd_close From 678c7d0a0bcb125e7417be76370240f8eae2e3e0 Mon Sep 17 00:00:00 2001 From: SirJosh3917 Date: Thu, 13 May 2021 01:03:16 -0400 Subject: [PATCH 9/9] Describe the Resources and Handles proposal --- design/file-descriptors.md | 39 -------------- design/resources-and-handles.md | 95 +++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 39 deletions(-) delete mode 100644 design/file-descriptors.md create mode 100644 design/resources-and-handles.md diff --git a/design/file-descriptors.md b/design/file-descriptors.md deleted file mode 100644 index b60f1ca73..000000000 --- a/design/file-descriptors.md +++ /dev/null @@ -1,39 +0,0 @@ -# File Descriptors (a.k.a handles) -File Descriptors ([witx type `fd`](https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-fd)), often referred to as "handles" interchangeably, are an abstract way of representing access to a resource. More often than not, they are processes (WIP), sockets (WIP), files, or even the console. - -## The Console -When a program is initiated, `stdin`, `stdout`, and `stderr` (respectively `fd` ids `0`, `1`, and `2`) are opened. To print to these, simply call `fd_write` with the corresponding arguments. An simple "Hello World!" program written in `wat` is shown below, [sourced from the wasmtime test suite](https://github.com/bytecodealliance/wasmtime/blob/main/tests/wasm/hello_wasi_snapshot1.wat): - -```wat -(module - (import "wasi_snapshot_preview1" "proc_exit" - (func $__wasi_proc_exit (param i32))) - (import "wasi_snapshot_preview1" "fd_write" - (func $__wasi_fd_write (param i32 i32 i32 i32) (result i32))) - (func $_start - (i32.store (i32.const 24) (i32.const 14)) - (i32.store (i32.const 20) (i32.const 0)) - (block - (br_if 0 - (call $__wasi_fd_write - (i32.const 1) - (i32.const 20) - (i32.const 1) - (i32.const 16))) - (br_if 0 (i32.ne (i32.load (i32.const 16)) (i32.const 14))) - (br 1) - ) - (call $__wasi_proc_exit (i32.const 1)) - ) - (memory 1) - (export "memory" (memory 0)) - (export "_start" (func $_start)) - (data (i32.const 0) "Hello, world!\0a") -) -``` - -## Files -Files can be opened with the method [`path_open`][path_open], and closed by calling [`fd_close`][fd_close]. These commands will fail if the WASM environment did not give the program permission to open files. - -[path_open]: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md#path_open -[fd_close]: https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md#fd_close diff --git a/design/resources-and-handles.md b/design/resources-and-handles.md new file mode 100644 index 000000000..90f416d2f --- /dev/null +++ b/design/resources-and-handles.md @@ -0,0 +1,95 @@ +# Resources and Handles +Resources and handles in WASI is the standard way to represent access to +OS-dependent resources, such as processes (WIP), threads (WIP), sockets (WIP), +and files (WIP). In addition, resources and handles can be used to represent +resources that you do not want to copy around willy nilly, such as large image +buffers. + +Handles are implemented as their own type, so they cannot be fudged with unlike +`int` handles returned in C (e.g. by adding/subtracting one). However, this +makes it harder to interact with APIs that may still use ints under the hood. +In WASI, the [Canonical ABI][canonincal-abi] is used as a stepping stone for code +that still needs to operate on integer handles. Handles for the Canonical ABI +are passed via regular integers, where that integer is an index into the +WebAssembly module's memory. At that position in memory should be a WebAssembly +handle. The primary intent for this is that existing users that expect integers +to pass around to functions as handles can transparently use the new handles as +they are rolled out, without having to modify their existing code. At the time +of writing, it may be a while before resources and handles come to fruition, so +it's a good way to ensure that critical features can get done now while +preparing for the future. + +An example visualization of a canonical reference can be seen below: + +``` +0 1 2 3 4 5 6 7 8 ++---+---+---+---+---+---+---+---+---+--- +| H | A | N | D | L | E | | | |... ++---+---+---+---+---+---+---+---+---+--- +^ the canonincal reference `file` points to 0 in memory at this position in + memory, the implementation-defined bytes for the handle are located. special + care is taken to ensure that you cannot fudge with the canonincal reference + to invoke bad behavior. + +;; pseudo-code, i can't write/read `wat` very well :p +i32 canonincal_reference = open_canonincal_resource("asdf") + +;; prints "0" +print(canonincal_reference) + +;; decrements the amount of +;; things pointing to the +;; resource, which may drop the +;; resource at this point +drop_canonincal_reference(canonincal_reference) +``` + +Handles cannot be copied by value, unlike other primitives. Instead, they must +be explicitly cloned with the `resource_clone_$resource` method, and explicitly +dropped with the `resource_drop_$resource` method. This makes managing the +lifetime of the underlying resource the handle points to easier. + +In WAT, one way one can define a resource with handles is by the following: + +```wat +;; NOTE: this is pseudo-code, i think i saw the code for htis somewhwere but i +;; can't seem to find where i saw it, so i'm just doing what my brain remembers +(export resource MyResource $R1) +(func resource_clone_MyResource (param (handle $R1))) +(func resource_drop_MyResource (param (handle $R1))) +(func make_resource (result (handle $R1))) +``` + +and can import and use the resource like so: + +```wat +(import "module.wasm" (resource $R1)) +(func use_resource ( + ;; i have no idea what i'm doing honestly lol + ((resource $R1) x (expr make_resource)) + (resource_drop_MyResource(x)) +)) +``` + +Resources and handles go hand in hand, and are widely used throughout the WASI +ecosystem. An example of resources and handles in practical use are in the +following areas: + +- [Files](#Files) +- [Network Sockets](#Network-Sockets) + +## Files + +Handles and resources can be found in use in the [wasi-filesystem][wasi-filesystem] +proposal. A simple example of opening and closing a file may be provided at a +later date, when the ABI stabilizes more. + +## Network Sockets + +Handles and resources can be found in use in networking, for reading and +writing data to a socket, as well as closing the socket when it is no longer in +use. A simple example of opening and closing a socket may be provided at a +later date, when the ABI stabilizes more. + +[canonincal-abi]: https://github.com/WebAssembly/interface-types/pull/132 +[wasi-filesystem]: https://github.com/WebAssembly/wasi-filesystem