What flavor tests do we want?

[wking]

#257 shows we could really use a test suite. I'm flexible, but it feels like our command line API is more stable than our current Go APIs. How would folks feel about a test harness exercising the command line API? Do we have a sharness/bats preference (I have more experience with sharness, but runC uses bats)? Should this wait until after opencontainers/tob#18 (is there an estimated timeframe for that decision)?

[glestaris]

I agree that the CLI should be tested instead of any Go API. I would prefer if we could use a Go testing framework, even though it's a CLI. Ginkgo [1] is being used in Cloud Foundry for all types of testing (API/CLI/unit).

Also, I guess this issue is mainly targeting a CLI test suite, but, would it make sense to consider having unit tests on some of the most complex operations?

[1] http://onsi.github.io/ginkgo/

[wking]

On Wed, Sep 07, 2016 at 06:24:42AM -0700, George Lestaris wrote:

I would prefer if we could use a Go testing framework, even though
it's a CLI. Ginkgo [1] is being used in Cloud Foundry for all types
of testing (API/CLI/unit).

Ginko looks nice, but I prefer shell-oriented test harnesses for the
command line because there's less boilerplate around invocation.
However, any framework around the command line is better than none, so
if maintainers want tests in Ginko (or whatever), I'll add them in
that framework.

Also, I guess this issue is mainly targeting a CLI test suite, but,
would it make sense to consider having unit tests on some of the
most complex operations?

As things currently stand, I think ‘oci-image-tool unpack SRC DEST’ is
more likely to be around in two months than manifest.unpack(walker,
dest), and testing I'd rather test the more stable interface. Testing
the command line API is also going to cover
cmd/oci-image-tool/unpack.go, and I'd rather have coverage first. We
can fill in unit tests later to make it easier to isolate errors and
avoid combinatoric issues ;). But again, if maintainers want Go tests
in image/*_test.go first, I'll go that way.

[philips]

This issue was moved to opencontainers/image-tools#18