reform to suppport mulitple runtimes

[Mashimiao]

Signed-off-by: Ma Shimiao mashimiao.fnst@cn.fujitsu.com

The spec does not specify any command-line APIs, so different runtime may have different APIs.
I create ContainerOperation interface and let Runtime to be parent structure, then we can implement them for different runtimes

[wking]

On Tue, Sep 12, 2017 at 07:16:55AM +0000, Ma Shimiao wrote: The spec does not specify any command-line APIs…

There's a command-line API in flight with #321.

I create ContainerOperation interface and let Runtime to be parent structure, then we can implement them for different runtimes

There's certainly no reason that we can't consolidate around a Go interface API instead of a command-line API (or in addition to a command-line API), but I think #321 is already very solid, and command-line wrappers around runtimes that don't natively support that API are easy shims to write. In this case, runC already supports the command-line API in #321 (or something very close to it. I'm not aware of any differences but haven't done a thorough audit). Another benefit of a command-line API is that swapping the runtime under test is not a build-time decision. Swapping an external runtime is really easy [1], but I'm not clear on how you'd swap in an external runtime if you wanted to test some other ContainerOperation interface (say, if I wrote a Go wrapper around ccon [2]). I'd rather land #321, since that's already had *lots* of review ;), and then revisit a Go API if we find a runtime that cannot be easily wrapped to support that command-line API. [1]: https://github.com/opencontainers/runtime-tools/blob/v0.1.0/validation/validation_test.go#L20-L29 [2]: https://github.com/wking/ccon

[Mashimiao]

I have to admit command line API is better for Swapping an external runtime, we have to write them into New() first if we choose to Go interface API.
But I think they have different advantages and disadvantages.
For Go interface API, with it, we can just care about whether the runtime passed the validation, nothing else.
For command line API, we not only validate runtime based on runtime-spec, but also have to check whether the harness follows the command line spec.
From this side, Go interface API may be a little mandatory but clearer for runtime validation.

Generally, I don't have strong objection to command line interface, but think Go interface API is also a good way.
Hope to hear opinions from other @opencontainers/runtime-tools-maintainers

[wking]

For Go interface API, with it, we can just care about whether the runtime passed the validation, nothing else.
For command line API, we not only validate runtime based on runtime-spec, but also have to check whether the harness follows the command line spec.

If you drive the runtime through the $X API, you'll be testing whether the runtime-plus-any-$X-shim satisfies the runtime spec and the $X API. That doesn't go away if $X is Go. The "who's fault are failures" bit just gets murky if one party maintains the shim and another party maintains the runtime core. Ideally, runtimes interested in compliance testing will implement the $X API themselves, so no runtime-tools or third-party shim is needed. Runc already implements the command-line API natively, although they could alway implement other APIs instead (or in addition) if they felt like it.

[wking]

Possibly obsolete with #321 landing a commitment to at least supporting the CLI? I have no problem with runtime-tools supporting additional APIs in parallel, but am not sure we have the bandwidth to develop them now.

[Mashimiao]

As #321 landed, just close this, will reopen if needed in the future