From 3f0aafb7ec9591d6e9058046b012531a46b1c107 Mon Sep 17 00:00:00 2001 From: Julian Friedman Date: Wed, 2 Sep 2015 19:43:08 +0100 Subject: [PATCH 01/63] Add initial command line spec Signed-off-by: Julian Friedman --- runtime.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 runtime.md diff --git a/runtime.md b/runtime.md new file mode 100644 index 000000000..d2e07ee03 --- /dev/null +++ b/runtime.md @@ -0,0 +1,24 @@ +# Operations: + +The command line should support each operation listed below as its first argument. It operates by default on the 'config.json' in the current directory. + +## Start + +Starts a container from a bundle directory. + +*Flags:* none. +*Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process exit code. + +Example: +~~~~ + # in a bundle directory with a process that echos "hello" and exits 42 + $ funC start + hello + + $ echo $? + 42 +~~~~ + +## Stop + + ... From 577f9911b00873d67cb9135e4aed6dba21895e87 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:16:30 -0700 Subject: [PATCH 02/63] Wrap to one sentence per line To match the specs convention [1]. [1]: https://github.com/opencontainers/specs/blob/v0.1.1/README.md#markdown-style Signed-off-by: W. Trevor King --- runtime.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index d2e07ee03..2ad863f76 100644 --- a/runtime.md +++ b/runtime.md @@ -1,6 +1,7 @@ # Operations: -The command line should support each operation listed below as its first argument. It operates by default on the 'config.json' in the current directory. +The command line should support each operation listed below as its first argument. +It operates by default on the 'config.json' in the current directory. ## Start From ff5ae1d17813d4e2822ce627c2d3e4b5651a4ec4 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:17:38 -0700 Subject: [PATCH 03/63] Use a Markdown list for flags and output Otherwise they're rendered as a single paragraph. Signed-off-by: W. Trevor King --- runtime.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 2ad863f76..73f8f8091 100644 --- a/runtime.md +++ b/runtime.md @@ -7,8 +7,8 @@ It operates by default on the 'config.json' in the current directory. Starts a container from a bundle directory. -*Flags:* none. -*Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process exit code. +* *Flags:* none. +* *Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process exit code. Example: ~~~~ From 4dfe30b9f2e76c8f654f6c92f45cdedb150adf30 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:20:27 -0700 Subject: [PATCH 04/63] 'sh' syntax highlighting for fenced code blocks See [1]. 'sh' keyword comes from [2], and seemed more explicit to the POSIX semantics than 'shell'. [1]: https://help.github.com/articles/github-flavored-markdown/#syntax-highlighting [2]: https://github.com/github/linguist/blob/master/lib/linguist/languages.yml Signed-off-by: W. Trevor King --- runtime.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/runtime.md b/runtime.md index 73f8f8091..eec8f2909 100644 --- a/runtime.md +++ b/runtime.md @@ -11,14 +11,14 @@ Starts a container from a bundle directory. * *Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process exit code. Example: -~~~~ - # in a bundle directory with a process that echos "hello" and exits 42 - $ funC start - hello +```sh +# in a bundle directory with a process that echos "hello" and exits 42 +$ funC start +hello - $ echo $? - 42 -~~~~ +$ echo $? +42 +``` ## Stop From 259bbb7bcb528077519e226b896738f1660dfc1b Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:22:31 -0700 Subject: [PATCH 05/63] Drop colon from "Operations:" header It's pretty clear that the header describes the rest of the file's contents without the colon ;). Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index eec8f2909..761896be6 100644 --- a/runtime.md +++ b/runtime.md @@ -1,4 +1,4 @@ -# Operations: +# Operations The command line should support each operation listed below as its first argument. It operates by default on the 'config.json' in the current directory. From efaef2eb0f9f2298a246d2795fb23063d80e6937 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:23:50 -0700 Subject: [PATCH 06/63] Fix "process exit code" -> "process's exit code" I think that's the possessive form of 'process' ;). Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 761896be6..1820c4caf 100644 --- a/runtime.md +++ b/runtime.md @@ -8,7 +8,7 @@ It operates by default on the 'config.json' in the current directory. Starts a container from a bundle directory. * *Flags:* none. -* *Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process exit code. +* *Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process's exit code. Example: ```sh From 969c763edc870c542b63fcb96c00718fadfb7da9 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:27:48 -0700 Subject: [PATCH 07/63] Split standard streams and exit codes into their own entries To give space for more detail about the streams. I'm not sure what platform-agnostic language for this should look like, but on POSIX I'm expecting just exec'ing the application process and inheriting the file descriptors. Signed-off-by: W. Trevor King --- runtime.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 1820c4caf..0665fc15c 100644 --- a/runtime.md +++ b/runtime.md @@ -8,7 +8,8 @@ It operates by default on the 'config.json' in the current directory. Starts a container from a bundle directory. * *Flags:* none. -* *Output:* The process output is printed to stdout and stderr, and the process exits with the delegate process's exit code. +* *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. +* *Exit code:* The runtime must exit with the application process's exit code. Example: ```sh From 60cddc4ec5fd31b2a4461a9e136677c6b13f959b Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:32:37 -0700 Subject: [PATCH 08/63] Introduce the funC placeholder Signed-off-by: W. Trevor King --- runtime.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 0665fc15c..daa101892 100644 --- a/runtime.md +++ b/runtime.md @@ -1,6 +1,7 @@ # Operations -The command line should support each operation listed below as its first argument. +A conformant runtime should provide an executable (called `funC` in the following examples). +That executable should support each operation listed below as its first argument. It operates by default on the 'config.json' in the current directory. ## Start From a7f5aa925d32c01a4e905bdcd26a319ada712cc4 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 17:33:03 -0700 Subject: [PATCH 09/63] Link to runtime.json too Catch up with [1]. [1]: https://github.com/opencontainers/specs/pull/88 Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index daa101892..da2d237c7 100644 --- a/runtime.md +++ b/runtime.md @@ -2,7 +2,7 @@ A conformant runtime should provide an executable (called `funC` in the following examples). That executable should support each operation listed below as its first argument. -It operates by default on the 'config.json' in the current directory. +It operates by default on the `config.json` and `runtime.json` in the current directory. ## Start From 9d97d3953f6129763004972a0f118ad40e411ea9 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 19:45:31 -0700 Subject: [PATCH 10/63] Add --config and --runtime options Use GNU-style long options to avoid ambiguous, one-character options in the spec, while still allowing the runtime to support one-character options with packing. Signed-off-by: W. Trevor King --- runtime.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index da2d237c7..276a1cd67 100644 --- a/runtime.md +++ b/runtime.md @@ -8,7 +8,9 @@ It operates by default on the `config.json` and `runtime.json` in the current di Starts a container from a bundle directory. -* *Flags:* none. +* *Options* + * *`--config `* Override `config.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). + * *`--runtime `* Override `runtime.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). * *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. * *Exit code:* The runtime must exit with the application process's exit code. From 11da39702f8b2544d05e5e13eda84d0d0717b86d Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 19:58:11 -0700 Subject: [PATCH 11/63] Add an overall usage template and a section on global options Let folks do whatever they want as long as they don't clobber our command namespace. Signed-off-by: W. Trevor King --- runtime.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 276a1cd67..815a25d7d 100644 --- a/runtime.md +++ b/runtime.md @@ -3,8 +3,20 @@ A conformant runtime should provide an executable (called `funC` in the following examples). That executable should support each operation listed below as its first argument. It operates by default on the `config.json` and `runtime.json` in the current directory. +The template for supported commands is: -## Start +```sh +$ funC [global-options] [command-specific-options] +``` + +## Global options + +None are required, but the runtime may support options that start with at least one hyphen. +Global options may take positional arguments (e.g. `--log-level debug`), but the option parsing must be such that `funC ` is unambiguously an invocation of `` for any `` that does not start with a hyphen (including commands not specified in this document). + +## Commands + +### start Starts a container from a bundle directory. @@ -24,6 +36,6 @@ $ echo $? 42 ``` -## Stop +### stop ... From ffa124c191dfe624b6ff811d2519d2fc79a5511a Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 14 Sep 2015 20:13:26 -0700 Subject: [PATCH 12/63] Add a 'version' command Make it easy for a caller to report which runtime they're using. Signed-off-by: W. Trevor King --- runtime.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/runtime.md b/runtime.md index 815a25d7d..1ce16b3f9 100644 --- a/runtime.md +++ b/runtime.md @@ -16,6 +16,29 @@ Global options may take positional arguments (e.g. `--log-level debug`), but the ## Commands +### version + +* *Options* None are required, but the runtime may support options. +* *Standard streams* + * *stdin:* The runtime may not attempt to read from its stdin. + * *stdout:* The runtime must print its name, a space, and its version as the first line to its stdout. + The name may contain any Unicode characters except a control codes and newlines. + The runtime may print additional lines its stdout, and the format for those lines is not specified in this document. + * *stderr:* The runtime may print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* The runtime must exit with zero. + +Print the runtime version and exit. +The version + +Example: +```sh +$ funC version +funC 1.0.0 +Built for x86_64-pc-linux-gnu +$ echo $? +0 +``` + ### start Starts a container from a bundle directory. From 5033c59d722acc5a8811645944aadae4f7beb815 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 10:22:41 -0700 Subject: [PATCH 13/63] Add an --id option to 'start' Give the user a consistent way to pick their container ID. For example, they may want to use: --id $(dirname "$PWD") or they may want to use: --id $(uuidgen) or they may want to leave ID generation to the runtime. Signed-off-by: W. Trevor King --- runtime.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 1ce16b3f9..4404af3bc 100644 --- a/runtime.md +++ b/runtime.md @@ -44,6 +44,8 @@ $ echo $? Starts a container from a bundle directory. * *Options* + * *`--id `* Set the container ID when creating or joining a container. + If not set, the runtime is free to pick any ID that is not already in use. * *`--config `* Override `config.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). * *`--runtime `* Override `runtime.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). * *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. @@ -52,7 +54,7 @@ Starts a container from a bundle directory. Example: ```sh # in a bundle directory with a process that echos "hello" and exits 42 -$ funC start +$ funC start --id hello-1 hello $ echo $? From f3a1c0889acb705f5d6a707bf5042ce631d66903 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 10:36:23 -0700 Subject: [PATCH 14/63] Shift config.json and runtime.json into the 'start' section You shouldn't need these for any other operations. Signed-off-by: W. Trevor King --- runtime.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 4404af3bc..a1c0e279b 100644 --- a/runtime.md +++ b/runtime.md @@ -1,8 +1,6 @@ # Operations A conformant runtime should provide an executable (called `funC` in the following examples). -That executable should support each operation listed below as its first argument. -It operates by default on the `config.json` and `runtime.json` in the current directory. The template for supported commands is: ```sh @@ -42,6 +40,7 @@ $ echo $? ### start Starts a container from a bundle directory. +It operates by default on the `config.json` and `runtime.json` in the current directory. * *Options* * *`--id `* Set the container ID when creating or joining a container. From 9751a363bc86ffedf54ee71eadbcf9674e7680a7 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 10:38:27 -0700 Subject: [PATCH 15/63] Add an 'exec' command for launching a process in an existing container Signed-off-by: W. Trevor King --- runtime.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/runtime.md b/runtime.md index a1c0e279b..0d981c444 100644 --- a/runtime.md +++ b/runtime.md @@ -60,6 +60,26 @@ $ echo $? 42 ``` +### exec + +Runs a secondary process in the given container. + +* *Options* + * *`--process `* Override `process.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). +* *Arguments* + * *``* The container ID to join. +* *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. +* *Exit code:* The runtime must exit with the application process's exit code. + +Example: +```sh +# in a directory with a process.json that echos "goodbye" and exits 43 +$ funC exec hello-1 +goodbye +$ echo $? +43 +``` + ### stop ... From dd38d9d51ff81b01a58e52f417253ef937372690 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 10:47:27 -0700 Subject: [PATCH 16/63] Add a 'signal' command for signalling containers Mostly for stopping them, but also useful for any number of signals (e.g. many applications use HUP to reload their configuration). Signed-off-by: W. Trevor King --- runtime.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 0d981c444..eaac25113 100644 --- a/runtime.md +++ b/runtime.md @@ -80,6 +80,22 @@ $ echo $? 43 ``` -### stop +### signal - ... +Sends a signal to the container. + +* *Options* + * *`--signal `* The signal to send. + This must be one of the valid POSIX signals, although runtimes on non-POSIX systems must translate the POSIX name to their platorm's analogous signal. + Defaults to TERM. +* *Arguments* + * *``* The container ID to join. +* *Exit code:* 0 on success, non-zero on error. + A 0 exit status does not imply the process has exited (as it may have caught the signal). + +Example: +```sh +$ funC signal --signal KILL hello-1 +$ echo $? +0 +``` From 1261380b0d7063e7a7cfe4c619e16d9b60ae3eff Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 10:59:32 -0700 Subject: [PATCH 17/63] Add a link to lifecycle docs for cleanup if main process dies On IRC just now: 10:56 < crosbymichael> if the main process dies in the container, all other process are killed ... 10:58 < julz> crosbymichael: Im assuming what you mean is you kill everything in the cgroup -> everything in the container dies? 10:58 < crosbymichael> julz: yes, that is how its implemented ... 10:59 < crosbymichael> julz: we actually freeze first, send the KILL, then unfreeze so we don't have races We don't have lifecycle docs for that yet, but once we do, we can update this to link to them. Signed-off-by: W. Trevor King --- runtime.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/runtime.md b/runtime.md index eaac25113..747c7bdf4 100644 --- a/runtime.md +++ b/runtime.md @@ -71,6 +71,8 @@ Runs a secondary process in the given container. * *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. * *Exit code:* The runtime must exit with the application process's exit code. +If the main application (launched by `start`) dies, all other processes in its container will be killed [TODO: link to lifecycle docs explaining this]. + Example: ```sh # in a directory with a process.json that echos "goodbye" and exits 43 From 1c32342ccc66d0281c71b4256347bc42ea384ca2 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 15 Sep 2015 11:09:57 -0700 Subject: [PATCH 18/63] Add 'pause' and 'resume' commands These are useful for checkpointing, since getting a consistent checkpoint of a running container is hard ;). This doesn't handle the checkpoints themselves though, which are currently not specified. Checkpoint behavior will look something like: $ funC pause --wait container-id $ checkpoint ... $ funC resume container-id Signed-off-by: W. Trevor King --- runtime.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/runtime.md b/runtime.md index 747c7bdf4..a0a95e920 100644 --- a/runtime.md +++ b/runtime.md @@ -82,6 +82,42 @@ $ echo $? 43 ``` +### pause + +Pause all processes in a container. + +* *Options* + * *`--wait`* Block until the process is completely paused. + Otherwise return immediately after initiating the pause, which may happen before the pause is complete. +* *Arguments* + * *``* The container ID to join. +* *Exit code:* 0 on success, non-zero on error. + +Example: +```sh +$ funC pause --wait hello-1 +$ echo $? +0 +``` + +### resume + +Unpause all processes in a container. + +* *Options* + * *`--wait`* Block until the process is completely unpaused. + Otherwise return immediately after initiating the unpause, which may happen before the unpause is complete. +* *Arguments* + * *``* The container ID to join. +* *Exit code:* 0 on success, non-zero on error. + +Example: +```sh +$ funC resume hello-1 +$ echo $? +0 +``` + ### signal Sends a signal to the container. From d41ab073ac09731de3b41ff4de7ef1022df2852c Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 17 Sep 2015 09:29:06 -0700 Subject: [PATCH 19/63] Document LISTEN_FDS for passing open file descriptors This landed in runC with [1], but the bundle-author <-> runtime specs explicitly avoid talking about how this is set (since the bundle-author doesn't care about the runtime-caller <-> runtime interface) [2]. However, *this* spec is about the runtime-caller <-> runtime interface, so we need to document it here. I've left LISTEN_PID [3,4] out, since I don't see how the runtime-caller would choose anything other than 1 for its value. It seems like something that a process would have to set for itself (because guessing the PID of a child before spawning it seems racy ;). In any event, the runC implementation seems to set this to 1 regardless of what systemd passes to it [4]. I've borrowed Shishir's wording for the example [4]. [1]: https://github.com/opencontainers/runc/pull/231 [2]: https://github.com/opencontainers/specs/pull/113#issuecomment-135857404 [3]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html [4]: https://github.com/opencontainers/runc/pull/231#issuecomment-135469477 Signed-off-by: W. Trevor King --- runtime.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/runtime.md b/runtime.md index a0a95e920..c6b7f7036 100644 --- a/runtime.md +++ b/runtime.md @@ -48,6 +48,9 @@ It operates by default on the `config.json` and `runtime.json` in the current di * *`--config `* Override `config.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). * *`--runtime `* Override `runtime.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). * *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. +* *Environment variables* + * *`LISTEN_FDS`:* The number of file descriptors passed. + For example, `LISTEN_FDS=2` would mean passing 3 and 4 (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. * *Exit code:* The runtime must exit with the application process's exit code. Example: @@ -137,3 +140,6 @@ $ funC signal --signal KILL hello-1 $ echo $? 0 ``` + +[standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors +[systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html From 5afcaca3e94995000c6189bf78c15ce77f89bc02 Mon Sep 17 00:00:00 2001 From: Michael Crosby Date: Thu, 2 Jul 2015 13:56:14 -0700 Subject: [PATCH 20/63] Add license and DCO information for contributions Signed-off-by: Michael Crosby Cherry-picked from [1]. [1]: https://github.com/opencontainers/specs/commit/84707b0843db72c7ef4fe476f2bf641b3dac664c Signed-off-by: W. Trevor King --- LICENSE | 191 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 59 +++++++++++++++++ 2 files changed, 250 insertions(+) create mode 100644 LICENSE create mode 100644 README.md diff --git a/LICENSE b/LICENSE new file mode 100644 index 000000000..bdc403653 --- /dev/null +++ b/LICENSE @@ -0,0 +1,191 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + Copyright 2015 The Linux Foundation. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md new file mode 100644 index 000000000..de6303b2b --- /dev/null +++ b/README.md @@ -0,0 +1,59 @@ +# Contributing + +The specification and code is licensed under the Apache 2.0 license found in +the `LICENSE` file of this repository. + +### Sign your work + +The sign-off is a simple line at the end of the explanation for the +patch, which certifies that you wrote it or otherwise have the right to +pass it on as an open-source patch. The rules are pretty simple: if you +can certify the below (from +[developercertificate.org](http://developercertificate.org/)): + +``` +Developer Certificate of Origin +Version 1.1 + +Copyright (C) 2004, 2006 The Linux Foundation and its contributors. +660 York Street, Suite 102, +San Francisco, CA 94110 USA + +Everyone is permitted to copy and distribute verbatim copies of this +license document, but changing it is not allowed. + + +Developer's Certificate of Origin 1.1 + +By making a contribution to this project, I certify that: + +(a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + +(b) The contribution is based upon previous work that, to the best + of my knowledge, is covered under an appropriate open source + license and I have the right under that license to submit that + work with modifications, whether created in whole or in part + by me, under the same open source license (unless I am + permitted to submit under a different license), as indicated + in the file; or + +(c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified + it. + +(d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including all + personal information I submit with it, including my sign-off) is + maintained indefinitely and may be redistributed consistent with + this project or the open source license(s) involved. +``` + +then you just add a line to every git commit message: + + Signed-off-by: Joe Smith + +using your real name (sorry, no pseudonyms or anonymous contributions.) + +You can add the sign off when creating the git commit via `git commit -s`. From d00530d95a2d76b605c470c2fcb203fe6db31db7 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 11:11:34 -0800 Subject: [PATCH 21/63] CONTRIBUTING: Put contributing information where GitHub likes it See https://github.com/blog/1184-contributing-guidelines Also strip some trailing whitespace. Signed-off-by: W. Trevor King --- README.md => CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename README.md => CONTRIBUTING.md (97%) diff --git a/README.md b/CONTRIBUTING.md similarity index 97% rename from README.md rename to CONTRIBUTING.md index de6303b2b..9cf17a5c3 100644 --- a/README.md +++ b/CONTRIBUTING.md @@ -1,7 +1,7 @@ # Contributing -The specification and code is licensed under the Apache 2.0 license found in -the `LICENSE` file of this repository. +The specification and code is licensed under the Apache 2.0 license found in +the `LICENSE` file of this repository. ### Sign your work From ffdd7046744c671e8b14bc748600e4e258c055ff Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 11:25:47 -0800 Subject: [PATCH 22/63] README: Document the project and scope Signed-off-by: W. Trevor King --- README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 000000000..b34508031 --- /dev/null +++ b/README.md @@ -0,0 +1,14 @@ +# OCI command-line API + +The [OCI Specifications][specs] currently focus on the bundle-author ↔ runtime interface, but there is renewed interest in specifying a command-line API for the runtime-caller ↔ runtime interface. +A common command-line API would make it easier to build higher-level tools that are runtime-agnostic (e.g. conformance testers like [ocitools][ocitools-test]). +This repository contains initial work on that API, with more detailed discussion in [this thread][thread]. +The usual [development rules][rules] apply, and the legal stuff is spelled out [here](CONTRIBUTING.md). +The target for the inital design will be to match [the lifecycle pull request][lifecycle], keeping as much similarity with the existing [runC][] command-line as possible. + +[specs]: https://github.com/opencontainers/specs +[ocitools-test]: https://github.com/mrunalp/ocitools#testing-oci-runtimes +[thread]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/BIxya5eSNLo +[rules]: https://github.com/opencontainers/specs#contributing +[lifecycle]: https://github.com/opencontainers/specs/pull/231 +[runC]: https://github.com/opencontainers/runc From 3606bcf98b5aee6cf9eea778160b3f493b6a37f2 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 12:11:02 -0800 Subject: [PATCH 23/63] Punt to the operating system for character encodings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Without this, "may contain any Unicode characters" seemed too ambiguous. I wish there were cleaner references for the {language}.{encoding} locales like en_US.UTF-8 and UTF-8. But [1,2] seems too glib, and I can't find a more targetted UTF-8 link than just dropping folks into a Unicode chapter (which is what [1] does): The Unicode Standard, Version 6.0, §3.9 D92, §3.10 D95 (2011) With the current v8.0 (2015-06-17), it's still §3.9 D92 and §3.10 D95. The TR35 link is for: In addition, POSIX locales may also specify the character encoding, which requires the data to be transformed into that target encoding. and the POSIX §6.2 link is for: In other locales, the presence, meaning, and representation of any additional characters are locale-specific. [1]: https://en.wikipedia.org/wiki/UTF-8 [2]: https://en.wikipedia.org/wiki/Locale#POSIX_platforms Signed-off-by: W. Trevor King Reviewed-by: Jesse Butler --- runtime.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/runtime.md b/runtime.md index c6b7f7036..04d379649 100644 --- a/runtime.md +++ b/runtime.md @@ -12,6 +12,11 @@ $ funC [global-options] [command-specific-options] ` is unambiguously an invocation of `` for any `` that does not start with a hyphen (including commands not specified in this document). +## Character encodings + +This API specification does not cover character encodings, but runtimes should conform to their native operating system. +For example, POSIX systems define [`LANG` and related environment variables][posix-lang] for [declaring][posix-locale-encoding] [locale-specific character encodings][posix-encoding], so a runtime in an `en_US.UTF-8` locale should write its [version](#version) to stdout in [UTF-8][]. + ## Commands ### version @@ -141,5 +146,9 @@ $ echo $? 0 ``` +[posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 +[posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 +[posix-locale-encoding: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html +[UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From 1cbcac6bdb631a2a7d3b7a17bdc7cc82dfea8abc Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 16:24:25 -0800 Subject: [PATCH 24/63] Shift version description to top of section The other commands have the following layout: ### {command name} {one-line description} * *Options:* ... ... * *Exit code:* ... {additional notes} {example} so its good to follow that pattern for 'version'. This commit also removes a dangling "The version" which snuck in with ffa124c1 (Add a 'version' command, 2015-09-14). Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/runtime.md b/runtime.md index 04d379649..b7d078616 100644 --- a/runtime.md +++ b/runtime.md @@ -21,6 +21,8 @@ For example, POSIX systems define [`LANG` and related environment variables][pos ### version +Print the runtime version and exit. + * *Options* None are required, but the runtime may support options. * *Standard streams* * *stdin:* The runtime may not attempt to read from its stdin. @@ -30,9 +32,6 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *stderr:* The runtime may print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Exit code:* The runtime must exit with zero. -Print the runtime version and exit. -The version - Example: ```sh $ funC version From 933b55ddfdb3659dc4102eccd5f0311f12038dfd Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 16:37:32 -0800 Subject: [PATCH 25/63] Strip trailing whitespace With: $ sed -i 's/[[:space:]]*$//' * Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index b7d078616..411df9f45 100644 --- a/runtime.md +++ b/runtime.md @@ -43,7 +43,7 @@ $ echo $? ### start -Starts a container from a bundle directory. +Starts a container from a bundle directory. It operates by default on the `config.json` and `runtime.json` in the current directory. * *Options* @@ -62,7 +62,7 @@ Example: # in a bundle directory with a process that echos "hello" and exits 42 $ funC start --id hello-1 hello - + $ echo $? 42 ``` From b92273237f94737f50da1787c79a8884340bdd50 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 16:02:10 -0800 Subject: [PATCH 26/63] Drop exec, pause, resume, and signal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All of these require state information to be shared between funC invocations (to map from a container ID to the cgroups/namespaces/etc.), and after today's meeting we may be backing away from that [1,2]. Even if we keep a requirement for sharing state between funC invocations, we don't want to specify these IPC-requiring commands until we have more clarity on that requirement in the spec. On systems like Solaris, the kernel maintains a registry of container IDs directly, so they don't need an external registry [3]. But without a consensus around the minimal amount of inter-process state sharing, we don't want to require container ID → state lookups in the command-line spec. Once we have more clarity on a minimal required mechanism (e.g. Julz's --state-file [4,5,6]), we can add them back in with an API that all runtimes can easily support (although runtimes are of course free to provide more convenient APIs as additional extensions). Pause, resume, and signal are still in the current lifecycle pull request [7], but I've requested they be removed until we have more clarity around the basic lifecycle [8]. [1]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2015/opencontainers.2015-12-02-18.01.html [2]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2015/opencontainers.2015-12-02-18.01.log.html#l-79 [3]: https://github.com/wking/oci-command-line-api/pull/3#issuecomment-162079383 [4]: https://github.com/wking/oci-command-line-api/pull/3#issuecomment-162009033 [5]: https://github.com/wking/oci-command-line-api/pull/3#issuecomment-162041009 [6]: https://github.com/wking/oci-command-line-api/pull/3#issuecomment-162082556 [7]: https://github.com/mrunalp/specs/commit/bd549a2c9569fd5d775323f7f39c835baf825d0a#diff-b84a8d65d8ed53f4794cd2db7e8ea731R48 [8]: https://github.com/opencontainers/specs/pull/231/files#r45532412 Signed-off-by: W. Trevor King Reviewed-by: Mike Brown Reviewed-by: Jesse Butler Reviewed-by: Julian Friedman --- runtime.md | 78 ------------------------------------------------------ 1 file changed, 78 deletions(-) diff --git a/runtime.md b/runtime.md index 411df9f45..6c0ff6a1e 100644 --- a/runtime.md +++ b/runtime.md @@ -67,84 +67,6 @@ $ echo $? 42 ``` -### exec - -Runs a secondary process in the given container. - -* *Options* - * *`--process `* Override `process.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). -* *Arguments* - * *``* The container ID to join. -* *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. -* *Exit code:* The runtime must exit with the application process's exit code. - -If the main application (launched by `start`) dies, all other processes in its container will be killed [TODO: link to lifecycle docs explaining this]. - -Example: -```sh -# in a directory with a process.json that echos "goodbye" and exits 43 -$ funC exec hello-1 -goodbye -$ echo $? -43 -``` - -### pause - -Pause all processes in a container. - -* *Options* - * *`--wait`* Block until the process is completely paused. - Otherwise return immediately after initiating the pause, which may happen before the pause is complete. -* *Arguments* - * *``* The container ID to join. -* *Exit code:* 0 on success, non-zero on error. - -Example: -```sh -$ funC pause --wait hello-1 -$ echo $? -0 -``` - -### resume - -Unpause all processes in a container. - -* *Options* - * *`--wait`* Block until the process is completely unpaused. - Otherwise return immediately after initiating the unpause, which may happen before the unpause is complete. -* *Arguments* - * *``* The container ID to join. -* *Exit code:* 0 on success, non-zero on error. - -Example: -```sh -$ funC resume hello-1 -$ echo $? -0 -``` - -### signal - -Sends a signal to the container. - -* *Options* - * *`--signal `* The signal to send. - This must be one of the valid POSIX signals, although runtimes on non-POSIX systems must translate the POSIX name to their platorm's analogous signal. - Defaults to TERM. -* *Arguments* - * *``* The container ID to join. -* *Exit code:* 0 on success, non-zero on error. - A 0 exit status does not imply the process has exited (as it may have caught the signal). - -Example: -```sh -$ funC signal --signal KILL hello-1 -$ echo $? -0 -``` - [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup From 839b6ee12affb1ba5defab3ba471d09451e86a61 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Sat, 5 Dec 2015 10:59:27 -0800 Subject: [PATCH 27/63] Use imperative phrasing for command summaries This makes everything consistent with the version command's: Print the runtime version and exit. And follows the practice recommended by Python's PEP 257 [1]: The docstring is a phrase ending in a period. It prescribes the function or method's effect as a command ("Do this", "Return that"), not as a description; e.g. don't write "Returns the pathname ...". [1]: https://www.python.org/dev/peps/pep-0257/#one-line-docstrings Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 6c0ff6a1e..97acbceb1 100644 --- a/runtime.md +++ b/runtime.md @@ -43,7 +43,7 @@ $ echo $? ### start -Starts a container from a bundle directory. +Start a container from a bundle directory. It operates by default on the `config.json` and `runtime.json` in the current directory. * *Options* From 0d0c8dbc2ab433bd981588ce80f02a859b79f762 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Sat, 5 Dec 2015 10:48:58 -0800 Subject: [PATCH 28/63] start: Remove start-scoped config.json and runtime.json references The reference landed in the initial 3f0aafb7 (Add initial command line spec, 2015-09-02) in the operations section. In f3a1c088 (Shift config.json and runtime.json into the 'start' section, 2015-09-15) they moved to the 'start' command section, because other commands don't need them. But we talk about them in start's options section since 9d97d395 (Add --config and --runtime options, 2015-09-14), so there's no need for this command-scoped reference. Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 1 - 1 file changed, 1 deletion(-) diff --git a/runtime.md b/runtime.md index 97acbceb1..c46c91fd2 100644 --- a/runtime.md +++ b/runtime.md @@ -44,7 +44,6 @@ $ echo $? ### start Start a container from a bundle directory. -It operates by default on the `config.json` and `runtime.json` in the current directory. * *Options* * *`--id `* Set the container ID when creating or joining a container. From e9a6625e7621162e55c35b08e2b9f175ed6da16e Mon Sep 17 00:00:00 2001 From: Mike Brown Date: Mon, 7 Dec 2015 16:04:25 -0600 Subject: [PATCH 29/63] Clarity for commands vs global options While reviewing the `global options` section in `runtime.md`, it seemed additional clarity was needed for the `command` and `global options` requirements. Discussed and worked on wording with @wking via private IRC. Also adds uppercase RFC 2119 words for this section. With regard to the statement `Command names MUST not start with hyphens,` the rationale behind this decision is to "distinguish unrecognized commands from unrecognized options, and because we are "requiring runtimes to fail-fast for unrecognized commands" [1,2]. `[1]` https://github.com/wking/oci-command-line-api/pull/8/files#r46898167 `[2]` https://github.com/wking/oci-command-line-api/commit/527f3c629a7224144c1e5e41a8a197fe0eb5bc00#commitcomment-14835617 Signed-off-by: Mike Brown Reviewed-by: W. Trevor King --- runtime.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index c46c91fd2..b7141c5c2 100644 --- a/runtime.md +++ b/runtime.md @@ -9,8 +9,11 @@ $ funC [global-options] [command-specific-options] ` is unambiguously an invocation of `` for any `` that does not start with a hyphen (including commands not specified in this document). +None are required, but the runtime MAY support options that start with at least one hyphen. +Global options MAY take positional arguments (e.g. `--log-level debug`). +Command names MUST not start with hyphens. +The option parsing MUST be such that `funC ` is unambiguously an invocation of `` (even for commands not specified in this document). +If the runtime is invoked with an unrecognized command, it MUST exit with a nonzero exit code and MAY log a warning to stderr. ## Character encodings From 3f348cd4c48238c0266361048f36ed62846f2b37 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 9 Dec 2015 11:20:38 -0800 Subject: [PATCH 30/63] start: Change --config and --runtime to --bundle Personally I prefer a single config file [1]. I want folks to be able to pipe their config into the 'funC start' command (e.g. via a /dev/fd/3 pseudo-filesystem path) [2], and I have a working example that supports this without difficulty [3]. But since [4] landed on 2015-11-16, runC has replaced their --config-file and --runtime-file flags with --bundle, and the current goal of this repository is "keeping as much similarity with the existing runC command-line as possible", not "makes sense to Trevor" ;). [1]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/0QbyJDM9fWY Subject: Single, unified config file (i.e. rolling back specs#88) Date: Wed, 4 Nov 2015 09:53:20 -0800 Message-ID: <20151104175320.GC24652@odin.tremily.us> [2]: https://github.com/opencontainers/runc/issues/202 [3]: https://github.com/wking/ccon/tree/8ab5b535b5eca1a62e12b5e865735e24f1e1666d#configuration [4]: https://github.com/opencontainers/runc/pull/373 Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index b7141c5c2..8a0a645a6 100644 --- a/runtime.md +++ b/runtime.md @@ -51,8 +51,7 @@ Start a container from a bundle directory. * *Options* * *`--id `* Set the container ID when creating or joining a container. If not set, the runtime is free to pick any ID that is not already in use. - * *`--config `* Override `config.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). - * *`--runtime `* Override `runtime.json` with an alternative path. The path may not support seeking (e.g. `/dev/fd/3`). + * *`--bundle `* Override the path to the bundle directory (defaults to the current working directory). * *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. From e41ffc1678a200b8aef347cd5e0ff94aa9c15b65 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 2 Dec 2015 16:35:13 -0800 Subject: [PATCH 31/63] Use RFC 2119's keywords (MUST, MAY, ...) This approach has landed in opencontainers/specs [1,2]. You can find the RFC here [3]. [1]: https://github.com/opencontainers/specs/issues/256 [2]: https://github.com/opencontainers/specs/pull/260 [3]: http://tools.ietf.org/html/rfc2119 Signed-off-by: W. Trevor King Reviewed-by: Mike Brown --- runtime.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/runtime.md b/runtime.md index 8a0a645a6..d56042ece 100644 --- a/runtime.md +++ b/runtime.md @@ -1,7 +1,7 @@ # Operations -A conformant runtime should provide an executable (called `funC` in the following examples). -The template for supported commands is: +A conformant runtime MUST provide an executable (called `funC` in the following examples). +That executable MUST support commands with the following template: ```sh $ funC [global-options] [command-specific-options] @@ -26,14 +26,14 @@ For example, POSIX systems define [`LANG` and related environment variables][pos Print the runtime version and exit. -* *Options* None are required, but the runtime may support options. +* *Options* None are required, but the runtime MAY support options. * *Standard streams* - * *stdin:* The runtime may not attempt to read from its stdin. - * *stdout:* The runtime must print its name, a space, and its version as the first line to its stdout. - The name may contain any Unicode characters except a control codes and newlines. - The runtime may print additional lines its stdout, and the format for those lines is not specified in this document. - * *stderr:* The runtime may print diagnostic messages to stderr, and the format for those lines is not specified in this document. -* *Exit code:* The runtime must exit with zero. + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The runtime MUST print its name, a space, and its version as the first line to its stdout. + The name MAY contain any Unicode characters, but MUST NOT contain control codes or newlines. + The runtime MAY print additional lines to its stdout, and the format for those lines is not specified in this document. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* The runtime MUST exit with zero. Example: ```sh @@ -52,11 +52,11 @@ Start a container from a bundle directory. * *`--id `* Set the container ID when creating or joining a container. If not set, the runtime is free to pick any ID that is not already in use. * *`--bundle `* Override the path to the bundle directory (defaults to the current working directory). -* *Standard streams:* The runtime must attach its standard streams directly to the application process without inspection. +* *Standard streams:* The runtime MUST attach its standard streams directly to the application process without inspection. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. - For example, `LISTEN_FDS=2` would mean passing 3 and 4 (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. -* *Exit code:* The runtime must exit with the application process's exit code. + For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the application process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. +* *Exit code:* The runtime MUST exit with the application process's exit code. Example: ```sh From 2fc4e033dfd5207264a8f411aaff97585d5b74ed Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 22 Jun 2016 12:51:28 -0700 Subject: [PATCH 32/63] runtime: Fix closing bracket in posix-locale-encoding reference Fix a typo from 3606bcf9 (Punt to the operating system for character encodings, 2015-12-02). Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index d56042ece..bd8433927 100644 --- a/runtime.md +++ b/runtime.md @@ -70,7 +70,7 @@ $ echo $? [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 -[posix-locale-encoding: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup +[posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html [UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From 770453114b748d4746ba1614a1e9a5d9003803fe Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 22 Jun 2016 11:40:28 -0700 Subject: [PATCH 33/63] runtime: Drop 'sh' highlighting from fenced code blocks Because these are shell sessions, not scripts (e.g. we don't want 'for' in the version output highlighted as a keyword). Rolls back 4dfe30b9 ('sh' syntax highlighting for fenced code blocks, 2015-09-14). Signed-off-by: W. Trevor King --- runtime.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index bd8433927..7dbd66ac2 100644 --- a/runtime.md +++ b/runtime.md @@ -36,7 +36,8 @@ Print the runtime version and exit. * *Exit code:* The runtime MUST exit with zero. Example: -```sh + +``` $ funC version funC 1.0.0 Built for x86_64-pc-linux-gnu @@ -59,7 +60,8 @@ Start a container from a bundle directory. * *Exit code:* The runtime MUST exit with the application process's exit code. Example: -```sh + +``` # in a bundle directory with a process that echos "hello" and exits 42 $ funC start --id hello-1 hello From b6d22e869eae766add01cdfae6a3a0637a6da17c Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 22 Jun 2016 11:45:43 -0700 Subject: [PATCH 34/63] runtime: Use headers for examples To provide anchors for linking [1]. [1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc1/style.md#anchoring Signed-off-by: W. Trevor King --- runtime.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 7dbd66ac2..7c85e002b 100644 --- a/runtime.md +++ b/runtime.md @@ -35,7 +35,7 @@ Print the runtime version and exit. * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Exit code:* The runtime MUST exit with zero. -Example: +#### Example ``` $ funC version @@ -59,7 +59,7 @@ Start a container from a bundle directory. For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the application process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. * *Exit code:* The runtime MUST exit with the application process's exit code. -Example: +#### Example ``` # in a bundle directory with a process that echos "hello" and exits 42 From 60148f0fc04f8e60166d6381e26c0dcaa3d86b47 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 22 Jun 2016 12:36:52 -0700 Subject: [PATCH 35/63] runtime: Add a 'state' command Partially catch up with opencontainers/runtime-spec@7117ede7 (Expand on the definition of our ops, 2015-10-13, opencontainers/runtime-spec#225, v0.4.0). The state example is adapted from the current release [1], but we defer the actual definition of that JSON to runtime-spec. The encoding for the output JSON (and all standard-stream activity) is covered by the "Character encodings" section. In cases where the runtime ignores the SHOULD (still technically compliant), RFC 7159 makes encoding detection reasonably straightforward [2]. The obsolete RFC 4627 has some hints as well [3] (although these were dropped in RFC 7518 [4], probably as a result of removing the constraint that "JSON text" be an object or array [4]). The hints should still apply to the state output, because we know it will be an object. If that ends up being too dicey and we want to certify runtimes that do not respect their operating-system conventions, we can add an --encoding option later. [1]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc1/runtime.md#state [2]: https://tools.ietf.org/html/rfc7159#section-8.1 [3]: https://tools.ietf.org/html/rfc4627#section-3 [4]: https://tools.ietf.org/html/rfc7158#appendix-A Which is currently identical to 7519. Signed-off-by: W. Trevor King --- runtime.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/runtime.md b/runtime.md index 7c85e002b..09d26942d 100644 --- a/runtime.md +++ b/runtime.md @@ -70,6 +70,38 @@ $ echo $? 42 ``` +### state + +Request the container state. + +* *Arguments* + * *``* The container whose state is being requested. +* *Standard streams:* + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The runtime MUST print the state JSON to its stdout. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* Zero if the state was successfully written to stdout and non-zero on errors. + +#### Example + +``` +# in a bundle directory with a process that sleeps for several seconds +$ funC start --id sleeper-1 & +$ funC state sleeper-1 +{ + "ociVersion": "1.0.0-rc1", + "id": "sleeper-1", + "status": "running", + "pid": 4422, + "bundlePath": "/containers/sleeper", + "annotations" { + "myKey": "myValue" + } +} +$ echo $? +0 +``` + [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup From b7429c88fa7a835086eb640052714398ede1c98c Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 22 Jun 2016 13:18:52 -0700 Subject: [PATCH 36/63] runtime: RFC 2119 SHOULDs in the "Character encodings" section These slipped through e41ffc16 (Use RFC 2119's keywords (MUST, MAY, ...), 2015-12-02, #9). Signed-off-by: W. Trevor King --- runtime.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 09d26942d..aeb39e5ed 100644 --- a/runtime.md +++ b/runtime.md @@ -17,8 +17,8 @@ If the runtime is invoked with an unrecognized command, it MUST exit with a nonz ## Character encodings -This API specification does not cover character encodings, but runtimes should conform to their native operating system. -For example, POSIX systems define [`LANG` and related environment variables][posix-lang] for [declaring][posix-locale-encoding] [locale-specific character encodings][posix-encoding], so a runtime in an `en_US.UTF-8` locale should write its [version](#version) to stdout in [UTF-8][]. +This API specification does not cover character encodings, but runtimes SHOULD conform to their native operating system. +For example, POSIX systems define [`LANG` and related environment variables][posix-lang] for [declaring][posix-locale-encoding] [locale-specific character encodings][posix-encoding], so a runtime in an `en_US.UTF-8` locale SHOULD write its [version](#version) to stdout in [UTF-8][]. ## Commands From 7f11bd8531aa7304d41ef8dc8de950ebff4e3fb9 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:28:54 -0800 Subject: [PATCH 37/63] README: Update links and text This hadn't been touched since 2015-12-02, and was getting pretty stale ;). Signed-off-by: W. Trevor King --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index b34508031..62a65c4d5 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ # OCI command-line API -The [OCI Specifications][specs] currently focus on the bundle-author ↔ runtime interface, but there is renewed interest in specifying a command-line API for the runtime-caller ↔ runtime interface. -A common command-line API would make it easier to build higher-level tools that are runtime-agnostic (e.g. conformance testers like [ocitools][ocitools-test]). +The [OCI Runtime Specification][runtime-spec] currently focuses on the bundle-author ↔ runtime interface, but there is renewed interest in specifying a command-line API for the runtime-caller ↔ runtime interface. +A common command-line API would make it easier to build higher-level tools that are runtime-agnostic (e.g. compliance testers like [runtime-tools][runtime-tools-compliance]). This repository contains initial work on that API, with more detailed discussion in [this thread][thread]. The usual [development rules][rules] apply, and the legal stuff is spelled out [here](CONTRIBUTING.md). -The target for the inital design will be to match [the lifecycle pull request][lifecycle], keeping as much similarity with the existing [runC][] command-line as possible. +The target for the inital design will be to match [the specified lifecycle][lifecycle], keeping as much similarity with the existing [runC][] command-line as possible. -[specs]: https://github.com/opencontainers/specs -[ocitools-test]: https://github.com/mrunalp/ocitools#testing-oci-runtimes +[runtime-spec]: https://github.com/opencontainers/runtime-spec +[runtime-tools-compliance]: https://github.com/opencontainers/runtime-tools#testing-oci-runtimes [thread]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/BIxya5eSNLo -[rules]: https://github.com/opencontainers/specs#contributing -[lifecycle]: https://github.com/opencontainers/specs/pull/231 +[rules]: https://github.com/opencontainers/runtime-spec#contributing +[lifecycle]: https://github.com/opencontainers/runtime-spec/blob/master/runtime.md#lifecycle [runC]: https://github.com/opencontainers/runc From 57296956d9ae1e80183db7b13997b9ec7d7d6288 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:30:35 -0800 Subject: [PATCH 38/63] *: "OCI Runtime Command Line Interface" Pick a better title. Signed-off-by: W. Trevor King --- README.md | 2 +- runtime.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 62a65c4d5..e513349ea 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# OCI command-line API +# OCI Runtime Command Line Interface The [OCI Runtime Specification][runtime-spec] currently focuses on the bundle-author ↔ runtime interface, but there is renewed interest in specifying a command-line API for the runtime-caller ↔ runtime interface. A common command-line API would make it easier to build higher-level tools that are runtime-agnostic (e.g. compliance testers like [runtime-tools][runtime-tools-compliance]). diff --git a/runtime.md b/runtime.md index aeb39e5ed..1d4c74970 100644 --- a/runtime.md +++ b/runtime.md @@ -1,4 +1,4 @@ -# Operations +# OCI Runtime Command Line Interface A conformant runtime MUST provide an executable (called `funC` in the following examples). That executable MUST support commands with the following template: From c44520ec3019e3b9ead682474132a2ce585a9a6a Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:35:44 -0800 Subject: [PATCH 39/63] runtime: Semantically version this specification The command-line interface is largely orthogonal to the config format, and config authors and runtime callers may be entirely different sets of people. Zhang Wei called for more explicit versioning for the CLI [1]. Wedging multiple, independently versioned entities into a single repository can be awkward, but earlier proposals to put the CLI in its own repository [2] were unsuccessful because compliance testing requires both a CLI and a config specification [3]. I don't think that's a solid reason [4], but discussion along that line stalled out, before being revived in today's meeting. Unfortunately, now this CLI spec is destined for runtime-tools [5], so we may still have multiple, independently versioned entities in a single repository. Wherever this lands up, it's useful for this CLI spec to be clear about it's own versioning. [1]: https://github.com/opencontainers/runtime-spec/pull/513#issuecomment-257204979 [2]: https://github.com/opencontainers/runtime-spec/pull/513#r72818875 [3]: https://github.com/opencontainers/runtime-spec/pull/513#r72866653 [4]: https://github.com/opencontainers/runtime-spec/pull/513#r72868322 [5]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2017/opencontainers.2017-02-08-22.00.log.html#l-125 Signed-off-by: W. Trevor King --- runtime.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 1d4c74970..bbe9a6b5d 100644 --- a/runtime.md +++ b/runtime.md @@ -1,6 +1,16 @@ # OCI Runtime Command Line Interface -A conformant runtime MUST provide an executable (called `funC` in the following examples). +This section defines the OCI Runtime Command Line Interface version 1.0.0. + +## Versioning + +The command line interface is versioned with [SemVer v2.0.0][semver]. +The command line interface version is independent of the OCI Runtime Specification as a whole (which is tied to the [configuration format][runtime-spec-version]. +For example, if a caller is compliant with version 1.1 of the command line interface, they are compatible with all runtimes that support any 1.1 or later release of the command line interface, but are not compatible with a runtime that supports 1.0 and not 1.1. + +## Global usage + +The runtime MUST provide an executable (called `funC` in the following examples). That executable MUST support commands with the following template: ```sh @@ -105,6 +115,8 @@ $ echo $? [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup +[semver]: http://semver.org/spec/v2.0.0.html [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html +[runtime-spec-version]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#specification-version [UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From a65fe1e09910803afdfe3943b641e4483af667cd Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:42:21 -0800 Subject: [PATCH 40/63] runtime: Drop 'sh' highlighting from fenced code blocks Fix an instance missed by 77045311 (runtime: Drop 'sh' highlighting from fenced code blocks, 2016-06-22, wking/oci-command-line-api#15). Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index bbe9a6b5d..5bae8ad66 100644 --- a/runtime.md +++ b/runtime.md @@ -13,7 +13,7 @@ For example, if a caller is compliant with version 1.1 of the command line inter The runtime MUST provide an executable (called `funC` in the following examples). That executable MUST support commands with the following template: -```sh +``` $ funC [global-options] [command-specific-options] ``` From 5fc039d44d56e74c1fe58640eab05e151a44bc01 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:44:43 -0800 Subject: [PATCH 41/63] runtime: Fix "MUST not" -> "MUST NOT" Signed-off-by: W. Trevor King --- runtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index 5bae8ad66..db0281f5b 100644 --- a/runtime.md +++ b/runtime.md @@ -21,7 +21,7 @@ $ funC [global-options] [command-specific-options] ` is unambiguously an invocation of `` (even for commands not specified in this document). If the runtime is invoked with an unrecognized command, it MUST exit with a nonzero exit code and MAY log a warning to stderr. From 5d0e44cff46dc723a2b648876fb3ac3ce885db7f Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:46:03 -0800 Subject: [PATCH 42/63] runtime: Address unspecified commands and options Make it clear that runtimes are free to add additional commands and options as they see fit (although namespacing to avoid collision with future versions of this specification would be wise). Signed-off-by: W. Trevor King --- runtime.md | 1 + 1 file changed, 1 insertion(+) diff --git a/runtime.md b/runtime.md index db0281f5b..de4147179 100644 --- a/runtime.md +++ b/runtime.md @@ -24,6 +24,7 @@ Global options MAY take positional arguments (e.g. `--log-level debug`). Command names MUST NOT start with hyphens. The option parsing MUST be such that `funC ` is unambiguously an invocation of `` (even for commands not specified in this document). If the runtime is invoked with an unrecognized command, it MUST exit with a nonzero exit code and MAY log a warning to stderr. +Beyond the above rules, the behavior of the runtime in the presence of commands and options not specified in this document is unspecified. ## Character encodings From 4c7901cdf3762fe3871fda429da1d593e64b0c33 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:52:19 -0800 Subject: [PATCH 43/63] runtime: Drop the 'version' command Having a 'version' command to make it easy for a caller to report which runtime they're using. But we don't have a use-case that makes it strictly necessary for interop, so we're leaving it out for now [1]. [1]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-20-21.03.log.html#l-75 Signed-off-by: W. Trevor King --- runtime.md | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/runtime.md b/runtime.md index de4147179..b6d729901 100644 --- a/runtime.md +++ b/runtime.md @@ -29,33 +29,10 @@ Beyond the above rules, the behavior of the runtime in the presence of commands ## Character encodings This API specification does not cover character encodings, but runtimes SHOULD conform to their native operating system. -For example, POSIX systems define [`LANG` and related environment variables][posix-lang] for [declaring][posix-locale-encoding] [locale-specific character encodings][posix-encoding], so a runtime in an `en_US.UTF-8` locale SHOULD write its [version](#version) to stdout in [UTF-8][]. +For example, POSIX systems define [`LANG` and related environment variables][posix-lang] for [declaring][posix-locale-encoding] [locale-specific character encodings][posix-encoding], so a runtime in an `en_US.UTF-8` locale SHOULD write its [state](#state) to stdout in [UTF-8][]. ## Commands -### version - -Print the runtime version and exit. - -* *Options* None are required, but the runtime MAY support options. -* *Standard streams* - * *stdin:* The runtime MUST NOT attempt to read from its stdin. - * *stdout:* The runtime MUST print its name, a space, and its version as the first line to its stdout. - The name MAY contain any Unicode characters, but MUST NOT contain control codes or newlines. - The runtime MAY print additional lines to its stdout, and the format for those lines is not specified in this document. - * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. -* *Exit code:* The runtime MUST exit with zero. - -#### Example - -``` -$ funC version -funC 1.0.0 -Built for x86_64-pc-linux-gnu -$ echo $? -0 -``` - ### start Start a container from a bundle directory. From 6819bc90a3639415ac3ac829dee72f4e6d8321d4 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 15:55:11 -0800 Subject: [PATCH 44/63] runtime: Four-space indents for nested lists Following opencontainers/runtime-spec@7795661 (runtime.md: Fix sub-bullet indentation, 2016-06-08, opencontainers/runtime-spec#495). From [1]: List items may consist of multiple paragraphs. Each subsequent paragraph in a list item must be indented by either 4 spaces or one tab... I expect that's intended to be read with "block element" instead of "paragraph", in which case it applies to nested lists too. And while GitHub supports two-space indents [2]: You can create nested lists by indenting lines by two spaces. it seems that pandoc does not. [1]: http://daringfireball.net/projects/markdown/syntax#list [2]: https://help.github.com/articles/basic-writing-and-formatting-syntax/#lists Signed-off-by: W. Trevor King --- runtime.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/runtime.md b/runtime.md index b6d729901..83dea6d6d 100644 --- a/runtime.md +++ b/runtime.md @@ -38,13 +38,13 @@ For example, POSIX systems define [`LANG` and related environment variables][pos Start a container from a bundle directory. * *Options* - * *`--id `* Set the container ID when creating or joining a container. - If not set, the runtime is free to pick any ID that is not already in use. - * *`--bundle `* Override the path to the bundle directory (defaults to the current working directory). + * *`--id `* Set the container ID when creating or joining a container. + If not set, the runtime is free to pick any ID that is not already in use. + * *`--bundle `* Override the path to the bundle directory (defaults to the current working directory). * *Standard streams:* The runtime MUST attach its standard streams directly to the application process without inspection. * *Environment variables* - * *`LISTEN_FDS`:* The number of file descriptors passed. - For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the application process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. + * *`LISTEN_FDS`:* The number of file descriptors passed. + For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the application process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. * *Exit code:* The runtime MUST exit with the application process's exit code. #### Example @@ -63,11 +63,11 @@ $ echo $? Request the container state. * *Arguments* - * *``* The container whose state is being requested. + * *``* The container whose state is being requested. * *Standard streams:* - * *stdin:* The runtime MUST NOT attempt to read from its stdin. - * *stdout:* The runtime MUST print the state JSON to its stdout. - * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The runtime MUST print the state JSON to its stdout. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Exit code:* Zero if the state was successfully written to stdout and non-zero on errors. #### Example From 5c87cc6469f41fdbb5f0d22cdc16f9d585232fbc Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:02:01 -0800 Subject: [PATCH 45/63] runtime: Add a 'kill' command Similar to the 'signal' command removed in b9227323 (Drop exec, pause, resume, and signal, 2015-12-02, wking/oci-command-line-api#3). The runtime-spec gained a kill operation as part of opencontainers/runtime-spec@be594153 (Split create and start, 2016-04-01, opencontainers/runtime-spec#384). The interface is based on POSIX [1], util-linux [2], and GNU coreutils [3]. The TERM/KILL requirement is a minimum portability requirement for soft/hard stops. Windows lacks POSIX signals [4], and currently supports soft stops in Docker with whatever is behind hcsshim.ShutdownComputeSystem [5]. The docs we're landing here explicitly allow that sort of substitution, because we need to have soft/hard stop on those platforms but *can't* use POSIX signals. They borrow wording from opencontainers/runtime-spec@35b0e9ee (config: Clarify MUST for platform.os and .arch, 2016-05-19, opencontainers/runtime-spec#441) to recommend runtime authors document the alternative technology so bundle-authors can prepare (e.g. by installing the equivalent to a SIGTERM signal handler). [1]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/kill.html [2]: http://man7.org/linux/man-pages/man1/kill.1.html [3]: http://www.gnu.org/software/coreutils/manual/html_node/kill-invocation.html [4]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/PlGKu7QUwLE Subject: Fwd: Windows support for OCI stop/signal/kill (runtime-spec#356) Date: Thu, 26 May 2016 11:03:29 -0700 Message-ID: <20160526180329.GL17496@odin.tremily.us> [5]: https://github.com/docker/docker/pull/16997/files#diff-5d0b72cccc4809455d52aadc62329817R230 docker/docker@bc503ca8 (Windows: [TP4] docker kill handling, 2015-10-12, docker/docker#16997) Signed-off-by: W. Trevor King --- runtime.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/runtime.md b/runtime.md index 83dea6d6d..380ca9264 100644 --- a/runtime.md +++ b/runtime.md @@ -90,9 +90,45 @@ $ echo $? 0 ``` +### kill + +[Send a signal][kill] to the container process. + +* *Arguments* + * *``* The container being signaled. +* *Options* + * *`--signal `* The signal to send (defaults to `TERM`). + The runtime MUST support `TERM` and `KILL` signals with [the POSIX semantics][posix-signals]. + The runtime MAY support additional signal names. + On platforms that support [POSIX signals][posix-signals], the runtime MUST implement this command using POSIX signals. + On platforms that do not support POSIX signals, the runtime MAY implement this command with alternative technology as long as `TERM` and `KILL` retain their POSIX semantics. + Runtime authors on non-POSIX platforms SHOULD submit documentation for their TERM implementation to this specificiation, so runtime callers can configure the container process to gracefully handle the signals. +* *Standard streams:* + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The handling of stdout is unspecified. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* Zero if the signal was successfully sent to the container process and non-zero on errors. + Successfully sent does not mean that the signal was successfully received or handled by the container process. + +#### Example + +``` +# in a bundle directory with a process ignores TERM +$ funC start --id sleeper-1 & +$ funC kill sleeper-1 +$ echo $? +0 +$ funC kill --signal KILL sleeper-1 +$ echo $? +0 +``` + +[kill]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#kill +[kill.2]: http://man7.org/linux/man-pages/man2/kill.2.html [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup +[posix-signals]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/signal.h.html#tag_13_42_03 [semver]: http://semver.org/spec/v2.0.0.html [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html From 6375ad0143cc3bb20226ed98f232abcb3f25e75d Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:25:45 -0800 Subject: [PATCH 46/63] runtime: Link to the spec's 'bundle' docs Make life easy for folks not familiar with the term. Signed-off-by: W. Trevor King --- runtime.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/runtime.md b/runtime.md index 380ca9264..577ebc6f3 100644 --- a/runtime.md +++ b/runtime.md @@ -35,12 +35,12 @@ For example, POSIX systems define [`LANG` and related environment variables][pos ### start -Start a container from a bundle directory. +Start a container from a [bundle directory][bundle]. * *Options* * *`--id `* Set the container ID when creating or joining a container. If not set, the runtime is free to pick any ID that is not already in use. - * *`--bundle `* Override the path to the bundle directory (defaults to the current working directory). + * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). * *Standard streams:* The runtime MUST attach its standard streams directly to the application process without inspection. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. @@ -123,6 +123,7 @@ $ echo $? 0 ``` +[bundle]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/bundle.md [kill]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#kill [kill.2]: http://man7.org/linux/man-pages/man2/kill.2.html [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 From 3d50871641064c9e850cb6b5fab4255a35b7b693 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:27:45 -0800 Subject: [PATCH 47/63] runtime: Move start's --id to a positional parameter I added this as an option in 5033c59d (Add an --id option to 'start', 2015-09-15), because some callers might want to leave ID generation to the runtime. When there is a long-running host process waiting on the container process to perform cleanup, the runtime-caller may not need to know the container ID. However, runC has been requiring a user-specified ID since [1], and the coming create/start split will follow the early-exit 'create' from [2], so require an ID here. We can revisit this if we regain a long-running 'create' process. You can create a config that adds no isolation vs. the runtime namespace or completely joins another set of existing namespaces. It seems odd to call that a new "container", but the ID is really more of a process ID, and less of a container ID. The "container" phrasing is just a useful hint that there might be some isolation going on. And we're always creating a new "container process" with 'start' (which will become 'create'). [1]: https://github.com/opencontainers/runc/pull/541 opencontainers/runc@a7278cad (Require container id as arg1, 2016-02-08, opencontainers/runc#541) [2]: https://github.com/opencontainers/runc/pull/827 Summary: Implement create and start Signed-off-by: W. Trevor King --- runtime.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/runtime.md b/runtime.md index 577ebc6f3..100024941 100644 --- a/runtime.md +++ b/runtime.md @@ -37,9 +37,9 @@ For example, POSIX systems define [`LANG` and related environment variables][pos Start a container from a [bundle directory][bundle]. +* *Arguments* + * *``* Set the container ID to create. * *Options* - * *`--id `* Set the container ID when creating or joining a container. - If not set, the runtime is free to pick any ID that is not already in use. * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). * *Standard streams:* The runtime MUST attach its standard streams directly to the application process without inspection. * *Environment variables* @@ -51,7 +51,7 @@ Start a container from a [bundle directory][bundle]. ``` # in a bundle directory with a process that echos "hello" and exits 42 -$ funC start --id hello-1 +$ funC start hello-1 hello $ echo $? @@ -74,7 +74,7 @@ Request the container state. ``` # in a bundle directory with a process that sleeps for several seconds -$ funC start --id sleeper-1 & +$ funC start sleeper-1 & $ funC state sleeper-1 { "ociVersion": "1.0.0-rc1", @@ -114,7 +114,7 @@ $ echo $? ``` # in a bundle directory with a process ignores TERM -$ funC start --id sleeper-1 & +$ funC start sleeper-1 & $ funC kill sleeper-1 $ echo $? 0 From 56c906dbeb634c6533c0b6b61a5d22189d8058e7 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:35:12 -0800 Subject: [PATCH 48/63] runtime: "application" -> "container" Michael doesn't like "application" [1], and the runtime spec has standardized around "container process". [1]: https://github.com/opencontainers/runtime-spec/pull/107#issuecomment-145615479 Signed-off-by: W. Trevor King --- runtime.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/runtime.md b/runtime.md index 100024941..e4ca39988 100644 --- a/runtime.md +++ b/runtime.md @@ -41,11 +41,11 @@ Start a container from a [bundle directory][bundle]. * *``* Set the container ID to create. * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). -* *Standard streams:* The runtime MUST attach its standard streams directly to the application process without inspection. +* *Standard streams:* The runtime MUST attach its standard streams directly to the container process without inspection. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. - For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the application process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. -* *Exit code:* The runtime MUST exit with the application process's exit code. + For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. +* *Exit code:* The runtime MUST exit with the container process's exit code. #### Example From 5ace459890aa44c6123a31615d9746e0ddbfd36f Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:43:17 -0800 Subject: [PATCH 49/63] runtime: Split 'create' and 'start' (and add 'delete') Catch up with opencontainers/runtime-spec@be594153 (Split create and start, 2016-04-01, opencontainers/runtime-spec#384). One benefit of the early-exit 'create' is that the exit code does not conflate container process exits with "failed to setup the sandbox" exits. We can take advantage of that and use non-zero 'create' exits to allow stderr writing (so the runtime can log errors while dying without having to successfully connect to syslog or some such). I still likes the long-running 'create' API because it makes collecting the exit code easier. I've proposed an 'event' operation [1] which would provide a convenient created trigger. With 'event' in place, we don't need the 'create' process exit to serve as that trigger, and could have a long-running 'create' that collects the container process exit code using the portable waitid() family. But the consensus after the 2016-07-13 meeting was to table that while we land docs for the runC API [2], and runC has an early-exit create [3]. The "Callers MAY block..." wording is going to be hard to enforce, but with the runC model, clients rely on the command exits to trigger post-create and post-start activity. The longer the runtime hangs around after completing its action, the laggier those triggers will be. The "MUST NOT attempt to read from its stdin" means a generic caller can safely exec the command with a closed or null stdin, and not have to worry about the command blocking or crashing because of that. The stdout spec for start/delete is more lenient, because runtimes are unlikely to change their behavior because they are unable to write to stdout. If this assumption proves troublesome, we may have to tighten it up later. The ptrace idea in this commit is from Mrunal [4]. [1]: https://github.com/opencontainers/runtime-spec/pull/508 Subject: runtime: Add an 'event' operation for subscribing to pushes [2]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-13-17.03.log.html#l-15 [3]: https://github.com/opencontainers/runc/pull/827 Summary: Implement create and start [4]: http://ircbot.wl.linuxfoundation.org/eavesdrop/%23opencontainers/%23opencontainers.2016-07-13.log.html#t2016-07-13T18:58:54 Signed-off-by: W. Trevor King --- runtime.md | 86 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 74 insertions(+), 12 deletions(-) diff --git a/runtime.md b/runtime.md index e4ca39988..496bf0bf9 100644 --- a/runtime.md +++ b/runtime.md @@ -33,53 +33,91 @@ For example, POSIX systems define [`LANG` and related environment variables][pos ## Commands -### start +### create -Start a container from a [bundle directory][bundle]. +[Create][create] a container from a [bundle directory][bundle]. * *Arguments* * *``* Set the container ID to create. * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). -* *Standard streams:* The runtime MUST attach its standard streams directly to the container process without inspection. +* *Standard streams:* + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The handling of stdout is unspecified. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. -* *Exit code:* The runtime MUST exit with the container process's exit code. +* *Exit code:* Zero if the container was successfully created and non-zero on errors. + +Callers MAY block on this command's successful exit to trigger post-create activity. #### Example ``` # in a bundle directory with a process that echos "hello" and exits 42 +$ test -t 1 && echo 'stdout is a terminal' +stdout is a terminal +$ funC create hello-1 <&- >stdout 2>stderr +$ echo $? +0 +$ wc stdout +0 0 0 stdout $ funC start hello-1 +$ echo $? +0 +$ cat stdout hello - +$ block-on-exit-and-collect-exit-code hello-1 $ echo $? 42 +$ funC delete hello-1 +$ echo $? +0 ``` +#### Container process exit + +The [example's](#example) `block-on-exit-and-collect-exit-code` is platform-specific and is not specified in this document. +On Linux, it might involve an ancestor process which had set [`PR_SET_CHILD_SUBREAPER`][prctl.2] and collected the container PID [from the state][state], or a process that was [ptracing][ptrace.2] the container process for [`exit_group`][exit_group.2], although both of those race against the container process exiting before the watcher is monitoring. + +### start + +[Start][start] the user-specified code from [`process`][process]. + +* *Arguments* + * *``* The container to start. +* *Standard streams:* + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The handling of stdout is unspecified. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* Zero if the container was successfully started and non-zero on errors. + +Callers MAY block on this command's successful exit to trigger post-start activity. + +See [create](#example) for an example. + ### state -Request the container state. +[Request][state-request] the container [state][state]. * *Arguments* * *``* The container whose state is being requested. * *Standard streams:* * *stdin:* The runtime MUST NOT attempt to read from its stdin. - * *stdout:* The runtime MUST print the state JSON to its stdout. + * *stdout:* The runtime MUST print the [state JSON][state] to its stdout. * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Exit code:* Zero if the state was successfully written to stdout and non-zero on errors. #### Example ``` -# in a bundle directory with a process that sleeps for several seconds -$ funC start sleeper-1 & +$ funC create sleeper-1 $ funC state sleeper-1 { "ociVersion": "1.0.0-rc1", "id": "sleeper-1", - "status": "running", + "status": "created", "pid": 4422, "bundlePath": "/containers/sleeper", "annotations" { @@ -113,8 +151,9 @@ $ echo $? #### Example ``` -# in a bundle directory with a process ignores TERM -$ funC start sleeper-1 & +# in a bundle directory where the container process ignores TERM +$ funC create sleeper-1 +$ funC start sleeper-1 $ funC kill sleeper-1 $ echo $? 0 @@ -123,15 +162,38 @@ $ echo $? 0 ``` +### delete + +[Release](#delete) container resources after the container process has exited. + +* *Arguments* + * *``* The container to delete. +* *Standard streams:* + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The handling of stdout is unspecified. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. +* *Exit code:* Zero if the container was successfully deleted and non-zero on errors. + +See [create](#example) for an example. + [bundle]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/bundle.md +[create]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#create +[delete]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#delete +[exit_group.2]: http://man7.org/linux/man-pages/man2/exit_group.2.html [kill]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#kill [kill.2]: http://man7.org/linux/man-pages/man2/kill.2.html +[process]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#process [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup [posix-signals]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/signal.h.html#tag_13_42_03 +[prctl.2]: http://man7.org/linux/man-pages/man2/prctl.2.html +[ptrace.2]: http://man7.org/linux/man-pages/man2/ptrace.2.html [semver]: http://semver.org/spec/v2.0.0.html [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors +[start]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#start +[state]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#state +[state-request]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#query-state [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html [runtime-spec-version]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#specification-version [UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From 6f5d033a0530a8e9f1a71a7b1c5929e2d914648c Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:54:25 -0800 Subject: [PATCH 50/63] runtime: Add --pid-file to 'create' You can get the PID by calling 'state' [1], and container PIDs may not be portable [2]. But it's a common way for interfacing with init systems like systemd [3], and for this first pass at the command line API folks are ok with some Linux-centrism [4]. [1]: https://github.com/opencontainers/runtime-spec/pull/511#r70353376 Subject: Add initial pass at a cmd line spec [2]: https://github.com/opencontainers/runtime-spec/pull/459 Subject: [ Runtime ] Allow for excluding pid from state [3]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-20-21.03.log.html#l-69 [4]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-07-13-17.03.log.html#l-39 Signed-off-by: W. Trevor King --- runtime.md | 1 + 1 file changed, 1 insertion(+) diff --git a/runtime.md b/runtime.md index 496bf0bf9..b1d46b164 100644 --- a/runtime.md +++ b/runtime.md @@ -41,6 +41,7 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *``* Set the container ID to create. * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). + * *`--pid-file `* The runtime MUST write the container PID to this path. * *Standard streams:* * *stdin:* The runtime MUST NOT attempt to read from its stdin. * *stdout:* The handling of stdout is unspecified. From 394fa4fbd948f529d83105eea1aeb7854bc6764b Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 16:59:37 -0800 Subject: [PATCH 51/63] runtime: Add --console-socket for terminal handling Based on work by Aleksa in opencontainers/runC#1018, this commit makes the following choices: * SOCK_SEQPACKET instead of SOCK_STREAM, because this is a message-based protocol, so it seems more natural to use a message-oriented socket type. * A string 'type' field for all messages, so we can add additional message types in the future without breaking backwards compatibility (new console-socket servers will still support old clients). Aleksa favored splitting this identifier into an integer 'type' and 'version' fields [runc-socket-type-version], but I don't see the point if they're both opaque integers without internal structure. And I expect this protocol to be stable enough that it's not worth involving SemVer and its structured versioning. * Response messages, so the client can tell whether the request was received and processed successfully or not. That gives the client a way to bail out if, for example, the server does not support the 'terminal' message type. Signed-off-by: W. Trevor King --- runtime.md | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 4 deletions(-) diff --git a/runtime.md b/runtime.md index b1d46b164..cb60b8299 100644 --- a/runtime.md +++ b/runtime.md @@ -42,17 +42,62 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). * *`--pid-file `* The runtime MUST write the container PID to this path. + * *`--console-socket `* The runtime MUST pass the [pseudoterminal master][posix_openpt.3] through the socket at ``; the protocol is [described below](#console-socket). * *Standard streams:* - * *stdin:* The runtime MUST NOT attempt to read from its stdin. - * *stdout:* The handling of stdout is unspecified. - * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. + * If [`process.terminal`][process] is true: + * *stdin:* The runtime MUST NOT attempt to read from its stdin. + * *stdout:* The handling of stdout is unspecified. + * *stderr:* The runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. + * If [`process.terminal`][process] is not true: + * *stdin:* The runtime MUST pass its stdin file descriptor through to the container process without manipulation or modification. + "Without manipulation or modification" means that the runtime MUST not seek on the file descriptor, or close it, or read or write to it, or [`ioctl`][ioctl.3] it, or perform any other action on it besides passing it through to the container process. + * *stdout:* The runtime MUST pass its stdout file descriptor through to the container process without manipulation or modification. + * *stderr:* When `create` exists with a zero code, the runtime MUST pass its stderr file descriptor through to the container process without manipulation or modification. + When `create` exits with a non-zero code, the runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. - For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the [standard streams][standard-streams]) to support [socket activation][systemd-listen-fds]. + For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the standard streams) to support [socket activation][systemd-listen-fds]. * *Exit code:* Zero if the container was successfully created and non-zero on errors. Callers MAY block on this command's successful exit to trigger post-create activity. +#### Console socket + +The [`AF_UNIX`][unix-socket] used by [`--console-socket`](#create) handles request and response messages between a runtime and server. +The socket type MUST be [`SOCK_SEQPACKET`][socket-types]. +The server MUST send a single response for each runtime request. +The [normal data][socket-queue] ([`msghdr.msg_iov*`][socket.h]) of all messages MUST be [UTF-8][] [JSON](glossary.md#json). + +There are [JSON Schemas](schema/README.md) and [Go bindings](specs-go/socket/socket.go) for the messages specified in this section. + +##### Requests + +All requests MUST contain a **`type`** property whose value MUST one of the following strings: + +* `terminal`, if the request is passing a [pseudoterminal master][posix_openpt.3]. + When `type` is `terminal`, the request MUST also contain the following properties: + + * **`container`** (string, REQUIRED) The container ID, as set by [create](#create). + + The message's [ancillary data][socket-queue] (`msg_control*`) MUST contain at least one [`cmsghdr`][socket.h]). + The first `cmsghdr` MUST have: + + * `cmsg_type` set to [`SOL_SOCKET`][socket.h], + * `cmsg_level` set to [`SCM_RIGHTS`][socket.h], + * `cmsg_len` greater than or equal to `CMSG_LEN(sizeof(int))`, and + * `((int*)CMSG_DATA(cmsg))[0]` set to the pseudoterminal master file descriptor. + +##### Responses + +All responses MUST contain a **`type`** property whose value MUST be one of the following strings: + +* `success`, if the request was successfully processed. +* `error`, if the request was not successfully processed. + +In addition, responses MAY contain any of the following properties: + +* **`message`** (string, OPTIONAL) A phrase describing the response. + #### Example ``` @@ -181,20 +226,26 @@ See [create](#example) for an example. [create]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#create [delete]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#delete [exit_group.2]: http://man7.org/linux/man-pages/man2/exit_group.2.html +[ioctl.3]: http://pubs.opengroup.org/onlinepubs/9699919799/ [kill]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#kill [kill.2]: http://man7.org/linux/man-pages/man2/kill.2.html [process]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#process [posix-encoding]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02 [posix-lang]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02 [posix-locale-encoding]: http://www.unicode.org/reports/tr35/#Bundle_vs_Item_Lookup +[posix_openpt.3]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/posix_openpt.html [posix-signals]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/signal.h.html#tag_13_42_03 [prctl.2]: http://man7.org/linux/man-pages/man2/prctl.2.html [ptrace.2]: http://man7.org/linux/man-pages/man2/ptrace.2.html [semver]: http://semver.org/spec/v2.0.0.html +[socket-queue]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_11 +[socket-types]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_06 +[socket.h]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_socket.h.html [standard-streams]: https://github.com/opencontainers/specs/blob/v0.1.1/runtime-linux.md#file-descriptors [start]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#start [state]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#state [state-request]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#query-state [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html [runtime-spec-version]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#specification-version +[unix-socket]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_17 [UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From 74ca4036e7b8df85b88e34b8c194963aba36b974 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 20:39:18 -0800 Subject: [PATCH 52/63] runtime: Mention TIOCSTI privilege escalation As requested by Aleksa [1]. [1]: https://github.com/opencontainers/runtime-spec/pull/513#issuecomment-254230937 Signed-off-by: W. Trevor King --- runtime.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/runtime.md b/runtime.md index cb60b8299..533f7d446 100644 --- a/runtime.md +++ b/runtime.md @@ -51,6 +51,9 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * If [`process.terminal`][process] is not true: * *stdin:* The runtime MUST pass its stdin file descriptor through to the container process without manipulation or modification. "Without manipulation or modification" means that the runtime MUST not seek on the file descriptor, or close it, or read or write to it, or [`ioctl`][ioctl.3] it, or perform any other action on it besides passing it through to the container process. + + When using a container to drop privileges, note that providing a privileged terminal's file descriptor may allow the container to [execute privileged operations via `TIOCSTI`][TIOCSTI-security] or other [TTY ioctls][tty_ioctl.4]. + On Linux, [`TIOCSTI` requires `CAP_SYS_ADMIN`][capabilities.7] unless the target terminal is the caller's [controlling terminal][controlling-terminal]. * *stdout:* The runtime MUST pass its stdout file descriptor through to the container process without manipulation or modification. * *stderr:* When `create` exists with a zero code, the runtime MUST pass its stderr file descriptor through to the container process without manipulation or modification. When `create` exits with a non-zero code, the runtime MAY print diagnostic messages to stderr, and the format for those lines is not specified in this document. @@ -223,6 +226,8 @@ $ echo $? See [create](#example) for an example. [bundle]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/bundle.md +[capabilities.7]: http://man7.org/linux/man-pages/man7/capabilities.7.html +[controlling-terminal]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html#tag_11_01_03 [create]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#create [delete]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#delete [exit_group.2]: http://man7.org/linux/man-pages/man2/exit_group.2.html @@ -247,5 +252,7 @@ See [create](#example) for an example. [state-request]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#query-state [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html [runtime-spec-version]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#specification-version +[TIOCSTI-security]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=628843 +[tty_ioctl.4]: http://man7.org/linux/man-pages/man4/tty_ioctl.4.html [unix-socket]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_10_17 [UTF-8]: http://www.unicode.org/versions/Unicode8.0.0/ch03.pdf From a26d73b2f6c36520ee8205b8da42a39b1c186480 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 20:42:43 -0800 Subject: [PATCH 53/63] runtime: Replace '--console-socket PATH' with '$LISTEN_FDS + 3' As Michael requested [1]. The upsides of this approach are: * No need to drop the console onto the filesystem. You can use socketpair(2) in the runtime caller and be done with it. * No need for the runtime to connect (since the connection is already open when 'create' is invoked). This allows you to use SOCK_SEQPACKET or SOCK_STREAM as you see fit. The downsides are: * Overloading $LISTEN_FDS. Using a positional-argument approach to pass the FD number would feel less magical and make me much happier. [1]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-10-19-21.00.log.html#l-30 Signed-off-by: W. Trevor King --- runtime.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/runtime.md b/runtime.md index 533f7d446..e31284689 100644 --- a/runtime.md +++ b/runtime.md @@ -42,7 +42,6 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). * *`--pid-file `* The runtime MUST write the container PID to this path. - * *`--console-socket `* The runtime MUST pass the [pseudoterminal master][posix_openpt.3] through the socket at ``; the protocol is [described below](#console-socket). * *Standard streams:* * If [`process.terminal`][process] is true: * *stdin:* The runtime MUST NOT attempt to read from its stdin. @@ -60,14 +59,17 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the standard streams) to support [socket activation][systemd-listen-fds]. +* *Additional file descriptors* + * If [`process.terminal`][process] is true, the caller MUST provide an open [`AF_UNIX` socket][unix-socket] on file descriptor `$LISTEN_FDS + 3`. + The runtime MUST pass the [pseudoterminal master][posix_openpt.3] through the socket; the protocol is [described below](#console-socket). * *Exit code:* Zero if the container was successfully created and non-zero on errors. Callers MAY block on this command's successful exit to trigger post-create activity. #### Console socket -The [`AF_UNIX`][unix-socket] used by [`--console-socket`](#create) handles request and response messages between a runtime and server. -The socket type MUST be [`SOCK_SEQPACKET`][socket-types]. +The [`AF_UNIX`][unix-socket] used by the [`$LISTEN_FDS + 3` socket](#create) handles request and response messages between a runtime and server. +The socket type MUST be [`SOCK_SEQPACKET`][socket-types] or [`SOCK_STREAM`][socket-types]. The server MUST send a single response for each runtime request. The [normal data][socket-queue] ([`msghdr.msg_iov*`][socket.h]) of all messages MUST be [UTF-8][] [JSON](glossary.md#json). From 26440a5153a074276d1094764292c6c784333812 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 8 Feb 2017 20:50:09 -0800 Subject: [PATCH 54/63] runtime: Replace '$LISTEN_FDS + 3' with '--console-socket FD' This bridges the gap between my original '--console-socket PATH' proposal from 15350b31 (runtime: Add --console-socket for terminal handling, 2017-02-08) and Michael's approach from baa24120 (runtime: Replace '--console-socket PATH' with '$LISTEN_FDS + 3', 2017-02-08). All of the benefits of not needing an on-disk socket, and none of the magic of overloading $LISTEN_FDS. I don't know why Michael didn't like this approach as much [1], but it feels a lot better to me. [1]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2016/opencontainers.2016-10-19-21.00.log.html#l-31 Signed-off-by: W. Trevor King --- runtime.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/runtime.md b/runtime.md index e31284689..e637c58bd 100644 --- a/runtime.md +++ b/runtime.md @@ -41,6 +41,7 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *``* Set the container ID to create. * *Options* * *`--bundle `* Override the path to the [bundle directory][bundle] (defaults to the current working directory). + * *`--console-socket `* The runtime MUST pass the [pseudoterminal master][posix_openpt.3] through the open socket at file descriptor ``; the protocol is [described below](#console-socket). * *`--pid-file `* The runtime MUST write the container PID to this path. * *Standard streams:* * If [`process.terminal`][process] is true: @@ -59,16 +60,13 @@ For example, POSIX systems define [`LANG` and related environment variables][pos * *Environment variables* * *`LISTEN_FDS`:* The number of file descriptors passed. For example, `LISTEN_FDS=2` would mean that the runtime MUST pass file descriptors 3 and 4 to the container process (in addition to the standard streams) to support [socket activation][systemd-listen-fds]. -* *Additional file descriptors* - * If [`process.terminal`][process] is true, the caller MUST provide an open [`AF_UNIX` socket][unix-socket] on file descriptor `$LISTEN_FDS + 3`. - The runtime MUST pass the [pseudoterminal master][posix_openpt.3] through the socket; the protocol is [described below](#console-socket). * *Exit code:* Zero if the container was successfully created and non-zero on errors. Callers MAY block on this command's successful exit to trigger post-create activity. #### Console socket -The [`AF_UNIX`][unix-socket] used by the [`$LISTEN_FDS + 3` socket](#create) handles request and response messages between a runtime and server. +The [`AF_UNIX`][unix-socket] used by [`--console-socket`](#create) handles request and response messages between a runtime and server. The socket type MUST be [`SOCK_SEQPACKET`][socket-types] or [`SOCK_STREAM`][socket-types]. The server MUST send a single response for each runtime request. The [normal data][socket-queue] ([`msghdr.msg_iov*`][socket.h]) of all messages MUST be [UTF-8][] [JSON](glossary.md#json). From 32244e6bbc53442131c98d0ade15d0a2e33a81bc Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 9 Feb 2017 16:11:44 -0800 Subject: [PATCH 55/63] runtime: Document RFC 2119 wording Using the RFC's recommended wording [1]. [1]: https://tools.ietf.org/html/rfc2119 Signed-off-by: W. Trevor King --- runtime.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/runtime.md b/runtime.md index e637c58bd..1921baaf1 100644 --- a/runtime.md +++ b/runtime.md @@ -1,6 +1,10 @@ # OCI Runtime Command Line Interface -This section defines the OCI Runtime Command Line Interface version 1.0.0. +This file defines the OCI Runtime Command Line Interface version 1.0.0. + +## Notation + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119][rfc2119]. ## Versioning @@ -251,6 +255,7 @@ See [create](#example) for an example. [state]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#state [state-request]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#query-state [systemd-listen-fds]: http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html +[rfc2119]: https://tools.ietf.org/html/rfc2119 [runtime-spec-version]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/config.md#specification-version [TIOCSTI-security]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=628843 [tty_ioctl.4]: http://man7.org/linux/man-pages/man4/tty_ioctl.4.html From 7941be469ddde105d834606b302b714db247acc6 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 9 Feb 2017 16:15:09 -0800 Subject: [PATCH 56/63] runtime: Define "unspecified", "undefined", and "implementation-defined" I had been unaware of formal distinctions between these terms until Stephen Walli called it out [1] in the context of his suggestion to use "implementation defined" for uploading application/vnd.oci.image.layer.nondistributable.tar+gzip [2]. I couldn't find anything as compact as RFC 2119 around this idea, but linking to a section of the C99 rationale seems reasonable enough. The PDF I'm linking is "Rationale for International Standard - Programming Languages - C Revision 5.10 April-2003" and the referenced content appears in section 3: The terms *unspecified behavior*, *undefined behavior*, and *implementation-defined behavior* are used to categorize the result of writing programs whose properties the Standard does not, or cannot, completely describe. The goal of adopting this categorization is to allow a certain variety among implementations which permits *quality of implementation* to be an active force in the marketplace as well as to allow certain popular extensions, without removing the cachet of *conformance to the Standard*. Informative Annex J of the Standard catalogs those behaviors which fall into one of these three categories. *Unspecified behavior* gives the implementor some latitude in translating programs. This latitude does not extend as far as failing to translate the program, however, because all possible behaviors are "correct" in the sense that they don't cause undefined behavior in *any* implementation. *Undefined behavior* gives the implementor license not to catch certain program errors that are difficult to diagnose. It also identifies areas of possible conforming language extension: the implementor may augment the language by providing a definition of the officially undefined behavior. *Implementation-defined behavior* gives an implementor the freedom to choose the appropriate approach, but requires that this choice be explained to the user. Behaviors designated as implementation-defined are generally those in which a user could make meaningful coding decisions based on the implementation's definition. Implementors should bear in mind this criterion when deciding how extensive an implementation definition ought to be. As with unspecified behavior, simply failing to translate the source containing the implementation-defined behavior is not an adequate response. The "rationale for the C99 standard" link text seems pretty informal, but that's what WG14 uses to refer to the document [3]. And I've got the full title, revision, date, and referenced text in here in case the link dies and there is any ambiguity about the particular revision intended ;). [1]: https://github.com/opencontainers/image-spec/pull/233#issuecomment-245416276 [2]: https://github.com/opencontainers/image-spec/pull/233#issuecomment-245354663 [3]: http://www.open-std.org/jtc1/sc22/wg14/ Signed-off-by: W. Trevor King --- runtime.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/runtime.md b/runtime.md index 1921baaf1..c8f2a5cc4 100644 --- a/runtime.md +++ b/runtime.md @@ -6,6 +6,8 @@ This file defines the OCI Runtime Command Line Interface version 1.0.0. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119][rfc2119]. +The key words "unspecified", "undefined", and "implementation-defined" are to be interpreted as described in the [rationale for the C99 standard][c99-unspecified]. + ## Versioning The command line interface is versioned with [SemVer v2.0.0][semver]. @@ -230,6 +232,7 @@ $ echo $? See [create](#example) for an example. [bundle]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/bundle.md +[c99-unspecified]: http://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf#page=18 [capabilities.7]: http://man7.org/linux/man-pages/man7/capabilities.7.html [controlling-terminal]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html#tag_11_01_03 [create]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/runtime.md#create From be7de4f8a73a866e80cb796bdf58477152c46de6 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 15 Feb 2017 10:54:25 -0800 Subject: [PATCH 57/63] runtime: Add compliance language This language is based on existing language in RFC 2616 [1], RFC 7230 [2], and the runtime spec [3]. By defining both "compliant" and "not compliant" we leave the door open for a third case where an implementation (which can be a runtime or a runtime caller) does not violate any MUST-level requirements but does not allow some MUST-level requirements to be tested. For example, a runtime whose 'create' always exited with a non-zero exit code would not allow for testing the output of 'state' calls, because the test suite would have no valid container ID to use in the 'state' calls. Such a runtime would be neither compliant nor non-compliant. [1]: https://tools.ietf.org/html/rfc2616#section-1.2 [2]: https://tools.ietf.org/html/rfc7230#section-2.5 [3]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/spec.md#notational-conventions Signed-off-by: W. Trevor King --- runtime.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/runtime.md b/runtime.md index c8f2a5cc4..4e6d246fb 100644 --- a/runtime.md +++ b/runtime.md @@ -8,6 +8,13 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S The key words "unspecified", "undefined", and "implementation-defined" are to be interpreted as described in the [rationale for the C99 standard][c99-unspecified]. +## Compliance + +This specification targets compliance criteria according to the role of a participant in runtime invocation. +Requirements are placed on both runtimes and runtime callers, depending on what behavior is being constrained by the requirement. +An implementation is compliant if it satisfies all the applicable MUST, REQUIRED, and SHALL requirements. +An implementation is not compliant if it fails to satisfy one or more of the applicable MUST, REQUIRED, and SHALL requirements. + ## Versioning The command line interface is versioned with [SemVer v2.0.0][semver]. From d3b41c14b4a5e18eb507c3f4023a9fd1f73ec5cf Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 6 Jul 2016 11:44:47 -0700 Subject: [PATCH 58/63] command-line-interface: Rename from runtime.md to avoid collisions runtime-spec already has a runtime.md talking about operations at an interface-agnostic level. Rename this file so that it's clearly limited to the CLI. runtime-tools doesn't have a runtime.md, but the more-specific command-line-interface.md will make rebasing a merge into runtime-tools easier if runtime-tools happens to grow a runtime.md in the meantime. Signed-off-by: W. Trevor King --- runtime.md => command-line-interface.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename runtime.md => command-line-interface.md (100%) diff --git a/runtime.md b/command-line-interface.md similarity index 100% rename from runtime.md rename to command-line-interface.md From 215d47e092c7c9ee986e779b99510301b216e23e Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 6 Jul 2016 11:42:45 -0700 Subject: [PATCH 59/63] Remove files that duplicate runtime-spec content (LICENSE, etc.) These were useful when oci-command-line-api was an independent project, but are not longer needed now that we're folding it into runtime-tools. Signed-off-by: W. Trevor King --- CONTRIBUTING.md | 59 --------------- LICENSE | 191 ------------------------------------------------ README.md | 14 ---- 3 files changed, 264 deletions(-) delete mode 100644 CONTRIBUTING.md delete mode 100644 LICENSE delete mode 100644 README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 9cf17a5c3..000000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,59 +0,0 @@ -# Contributing - -The specification and code is licensed under the Apache 2.0 license found in -the `LICENSE` file of this repository. - -### Sign your work - -The sign-off is a simple line at the end of the explanation for the -patch, which certifies that you wrote it or otherwise have the right to -pass it on as an open-source patch. The rules are pretty simple: if you -can certify the below (from -[developercertificate.org](http://developercertificate.org/)): - -``` -Developer Certificate of Origin -Version 1.1 - -Copyright (C) 2004, 2006 The Linux Foundation and its contributors. -660 York Street, Suite 102, -San Francisco, CA 94110 USA - -Everyone is permitted to copy and distribute verbatim copies of this -license document, but changing it is not allowed. - - -Developer's Certificate of Origin 1.1 - -By making a contribution to this project, I certify that: - -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or - -(b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or - -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. -``` - -then you just add a line to every git commit message: - - Signed-off-by: Joe Smith - -using your real name (sorry, no pseudonyms or anonymous contributions.) - -You can add the sign off when creating the git commit via `git commit -s`. diff --git a/LICENSE b/LICENSE deleted file mode 100644 index bdc403653..000000000 --- a/LICENSE +++ /dev/null @@ -1,191 +0,0 @@ - - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - Copyright 2015 The Linux Foundation. - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/README.md b/README.md deleted file mode 100644 index e513349ea..000000000 --- a/README.md +++ /dev/null @@ -1,14 +0,0 @@ -# OCI Runtime Command Line Interface - -The [OCI Runtime Specification][runtime-spec] currently focuses on the bundle-author ↔ runtime interface, but there is renewed interest in specifying a command-line API for the runtime-caller ↔ runtime interface. -A common command-line API would make it easier to build higher-level tools that are runtime-agnostic (e.g. compliance testers like [runtime-tools][runtime-tools-compliance]). -This repository contains initial work on that API, with more detailed discussion in [this thread][thread]. -The usual [development rules][rules] apply, and the legal stuff is spelled out [here](CONTRIBUTING.md). -The target for the inital design will be to match [the specified lifecycle][lifecycle], keeping as much similarity with the existing [runC][] command-line as possible. - -[runtime-spec]: https://github.com/opencontainers/runtime-spec -[runtime-tools-compliance]: https://github.com/opencontainers/runtime-tools#testing-oci-runtimes -[thread]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/BIxya5eSNLo -[rules]: https://github.com/opencontainers/runtime-spec#contributing -[lifecycle]: https://github.com/opencontainers/runtime-spec/blob/master/runtime.md#lifecycle -[runC]: https://github.com/opencontainers/runc From 6551b2d240e25799b96767a6f87683a6b5b80b34 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 9 Feb 2017 16:58:09 -0800 Subject: [PATCH 60/63] docs/command-line-interface: Move command line API into position With the position requested by Mrunal [1,2]. [1]: http://ircbot.wl.linuxfoundation.org/eavesdrop/%23opencontainers/%23opencontainers.2017-02-09.log.html#t2017-02-09T22:21:59 [2]: http://ircbot.wl.linuxfoundation.org/eavesdrop/%23opencontainers/%23opencontainers.2017-02-10.log.html#t2017-02-10T00:03:25 Signed-off-by: W. Trevor King --- command-line-interface.md => docs/command-line-interface.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename command-line-interface.md => docs/command-line-interface.md (100%) diff --git a/command-line-interface.md b/docs/command-line-interface.md similarity index 100% rename from command-line-interface.md rename to docs/command-line-interface.md From 2ce4a5dfb9e03fa9f0f1e3668464a6e2910892e8 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Wed, 15 Feb 2017 07:40:50 -0800 Subject: [PATCH 61/63] docs/runtime-compliance-testing: Document list of supported APIs Currently just the command line API, but the test suite might add support for more APIs in the future. The command line API itself is independent of whether the test suite supports additional APIs, so I put the list of supported APIs in a new file. Signed-off-by: W. Trevor King --- docs/command-line-interface.md | 1 + docs/runtime-compliance-testing.md | 9 +++++++++ 2 files changed, 10 insertions(+) create mode 100644 docs/runtime-compliance-testing.md diff --git a/docs/command-line-interface.md b/docs/command-line-interface.md index 4e6d246fb..6d091946d 100644 --- a/docs/command-line-interface.md +++ b/docs/command-line-interface.md @@ -1,6 +1,7 @@ # OCI Runtime Command Line Interface This file defines the OCI Runtime Command Line Interface version 1.0.0. +It is one of potentially several [runtime APIs supported by the runtime compliance test suite](runtime-compliance-testing.md#supported-apis). ## Notation diff --git a/docs/runtime-compliance-testing.md b/docs/runtime-compliance-testing.md new file mode 100644 index 000000000..9ff409973 --- /dev/null +++ b/docs/runtime-compliance-testing.md @@ -0,0 +1,9 @@ +# Runtime compliance testing + +## Supported APIs + +In order to be tested for [compliance][], runtimes MUST support at least one of the following APIs: + +* Version 1.0.0 of the [OCI Runtime Command Line Interface](command-line-interface.md). + +[compliance]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc4/spec.md#notational-conventions From ebae480158cc1d17b75b0cb77fb93138e662b7ec Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Thu, 9 Feb 2017 17:01:58 -0800 Subject: [PATCH 62/63] api/socket: Add JSON Schema and Go types for the command-line API socket messages Even though there aren't many new types, these are fairly different from the rest of the runtime-tools code, and a separate package lets folks pull in only the code they need (assuming they are sophisticated enough to grab only a subset of the Git repository). Mrunal approved the JSON Schema landing under schema/ [1]. [1]: http://ircbot.wl.linuxfoundation.org/meetings/opencontainers/2017/opencontainers.2017-03-15-21.02.log.html#l-122 Signed-off-by: W. Trevor King --- api/socket/socket.go | 23 +++++++++++++++++++++++ docs/command-line-interface.md | 2 +- schema/README.md | 14 ++++++++++++++ schema/socket-response.json | 24 ++++++++++++++++++++++++ schema/socket-terminal-request.json | 24 ++++++++++++++++++++++++ 5 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 api/socket/socket.go create mode 100644 schema/README.md create mode 100644 schema/socket-response.json create mode 100644 schema/socket-terminal-request.json diff --git a/api/socket/socket.go b/api/socket/socket.go new file mode 100644 index 000000000..7bb23dc94 --- /dev/null +++ b/api/socket/socket.go @@ -0,0 +1,23 @@ +package socket + +// Message is the normal data for messages passed on the console socket. +type Message struct { + // Type of message being passed + Type string `json:"type"` +} + +// TerminalRequest is the normal data for messages passing a pseudoterminal master. +type TerminalRequest struct { + Message + + // Container ID for the container whose pseudoterminal master is being set. + Container string `json:"container"` +} + +// Response is the normal data for response messages. +type Response struct { + Message + + // Message is a phrase describing the response. + Message string `json:"message,omitempty"` +} diff --git a/docs/command-line-interface.md b/docs/command-line-interface.md index 6d091946d..8fbd496de 100644 --- a/docs/command-line-interface.md +++ b/docs/command-line-interface.md @@ -85,7 +85,7 @@ The socket type MUST be [`SOCK_SEQPACKET`][socket-types] or [`SOCK_STREAM`][sock The server MUST send a single response for each runtime request. The [normal data][socket-queue] ([`msghdr.msg_iov*`][socket.h]) of all messages MUST be [UTF-8][] [JSON](glossary.md#json). -There are [JSON Schemas](schema/README.md) and [Go bindings](specs-go/socket/socket.go) for the messages specified in this section. +There are [JSON Schemas](../schema/README.md#oci-runtime-command-line-interface) and [Go bindings](../api/socket/socket.go) for the messages specified in this section. ##### Requests diff --git a/schema/README.md b/schema/README.md new file mode 100644 index 000000000..ba852eeb1 --- /dev/null +++ b/schema/README.md @@ -0,0 +1,14 @@ +# JSON schema + +This directory contains [JSON Schema][json-schema] for validating JSON covered by local specifications. +The runtime specification includes [a generic command line tool][validate] which may be used to validate JSON with these schemas. + +## OCI Runtime Command Line Interface + +The [Runtime Command Line Interface](../docs/command-line-interface.md) defines: + +* [Terminal requests](../docs/command-line-interface.md#requests), which may be validated against [`socket-terminal-request.json`](socket-terminal-request.json). +* [Responses](../docs/command-line-interface.md#reqponses), which may be validated against [`socket-response.json`](socket-response.json). + +[json-schema]: http://json-schema.org/ +[validate]: https://github.com/opencontainers/runtime-spec/tree/v1.0.0-rc5/schema#utility diff --git a/schema/socket-response.json b/schema/socket-response.json new file mode 100644 index 000000000..e1c50b157 --- /dev/null +++ b/schema/socket-response.json @@ -0,0 +1,24 @@ +{ + "description": "Open Container Runtime Socket Response Schema", + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "https://opencontainers.org/schema/runtime/socket/response", + "type": "object", + "properties": { + "type": { + "id": "https://opencontainers.org/schema/runtime/socket/response/type", + "type": "string", + "enum": [ + "success", + "error" + ] + }, + "message": { + "id": "https://opencontainers.org/schema/runtime/socket/response/message", + "description": "A phrase describing the response.", + "type": "string" + } + }, + "required": [ + "type" + ] +} diff --git a/schema/socket-terminal-request.json b/schema/socket-terminal-request.json new file mode 100644 index 000000000..2fff2b844 --- /dev/null +++ b/schema/socket-terminal-request.json @@ -0,0 +1,24 @@ +{ + "description": "Open Container Runtime Socket Terminal Request Schema", + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "https://opencontainers.org/schema/runtime/socket/terminal-request", + "type": "object", + "properties": { + "type": { + "id": "https://opencontainers.org/schema/runtime/socket/terminal/request/type", + "type": "string", + "enum": [ + "terminal" + ] + }, + "container": { + "id": "https://opencontainers.org/schema/runtime/state/id", + "description": "the container's ID", + "type": "string" + } + }, + "required": [ + "type", + "container" + ] +} From a2725b42d7b312681ed5315cf7e663a13c487445 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Tue, 11 Apr 2017 09:41:34 -0700 Subject: [PATCH 63/63] docs/command-line-interface: Add "NOT RECOMMENDED" to RFC 2119 keywords Catching up with erratum 499 [1]. [1]: https://www.rfc-editor.org/errata_search.php?rfc=2119&eid=499 Signed-off-by: W. Trevor King --- docs/command-line-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/command-line-interface.md b/docs/command-line-interface.md index 8fbd496de..507890594 100644 --- a/docs/command-line-interface.md +++ b/docs/command-line-interface.md @@ -5,7 +5,7 @@ It is one of potentially several [runtime APIs supported by the runtime complian ## Notation -The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119][rfc2119]. +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119][rfc2119]. The key words "unspecified", "undefined", and "implementation-defined" are to be interpreted as described in the [rationale for the C99 standard][c99-unspecified].