Skip to content

Make conformance test failures more actionable #763

@bobcatfish

Description

@bobcatfish

/area test-and-release
/kind bug
/assign @bobcatfish

Expected Behavior

When a conformance test fails, it should be as obvious as possible why the test failed so that folks submitting PRs which result in failing tests can get some idea of what might be going wrong.

Actual Behavior

Failures often look like this:

STEP: The Revision will be updated when it is ready to serve traffic

• Failure [134.472 seconds]
Route
/usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:156
  Deploying an app with a pre-built container
  /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:193
    Creates a route, serves traffic to it, and serves traffic to subsequent revisions [It]
    /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/route_test.go:194

    Expected error:
        <*errors.errorString | 0xc42008b6b0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    not to have occurred

    /usr/local/google/home/christiewilson/Code/go/src/github.com/elafros/elafros/test/conformance/crd_polling.go:78
------------------------------

This means to debug them, folks have to run the tests and alongside run commands such as watch kubectl get revisions -o yaml --namespace=pizzaplanet to see what's going wrong.

Steps to Reproduce the Problem

  1. make a test fail, e.g. by changing the conditions to not match the actual spec, then run the tests

Additional Info

n/a

Metadata

Metadata

Assignees

Labels

area/test-and-releaseIt flags unit/e2e/conformance/perf test issues for product featureskind/bugCategorizes issue or PR as related to a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions