suite: Provision for Skipping a test

[nanjekyejoannah-zz]

We have many reasons why we need to skip particular tests. I needed to provide for skipping a test given some condition depending on the environment the tests are running in. Apparently this is not available.

[bits01]

Sounds like you need to call testing.T.Skip() based on your conditions (flags, env vars, ...):
https://godoc.org/testing#T.Skip

[nanjekyejoannah-zz]

You see am using testify/suite for the tests am writing and my tests look like this. I decided to use suite in order to manage my teardowns and setup in a saner way.

func (suite *S3Suite) TestObjectCreateBadContentlengthNone() {

// some test code
}

If am to use testing.T.Skip. This will become

func TestObjectCreateBadContentlengthNone() {

	if sommecondition {

		testing.t.Skip("message for skip")
	}

	// more testcode
}

As you can see I have to remove the test I want to skip from the suite. I tried to use testing.T.Skip while keeping the test in the suite and it did not work. This also means that I have to provide separate teardown methods for this test that is outside the suite which did not play nice with me. Of course for now I am using testing.T.Skip for all tests I may want to skip for lack of a good option.

Therefore if testify/suite had a provision of skipping, I would have some kind of consistency. I will still have my test in question part of the suite i created and the test will still use the teardown/setup methods like the other tests in the suite. Plus Teardown/setup without using testify/suite is not so clean IMO.

[bits01]

Have you looked at using standard lib subtests instead?
https://blog.golang.org/subtests

[nanjekyejoannah-zz]

Well the idea of sub tests is a good one however it does not solve my problem.

There is no need for anyone running the test suite to decide what tests to execute. I want the tests themselves to skip some tests depending on the environment in which they are running. The person running the tests should just do go test -v . For example in my case I want some tests to skip when there is no SSL connection.In my test project, the tests that depend on this condition are so many and spread in different files. The subtest way may not be a very good solution.

[bits01]

Subtests can skip based on their own conditions. They are just normal tests, you just group them together so you can do the pre setup and post teardown, not any different from suites.

[nanjekyejoannah-zz]

This person had the same thing in mind while reporting this issue. check out that thread.

[Mistobaan]

was looking for the same thing. So suite.T().Skip() will skip only the current method-test right?

[smoyer64]

I found this issue as I was about to suggest that testify support assume in addition to assert and require. Coming from the Java world, the two major assertion libraries I'm familiar with have somewhat parallel assumptions.

The idea behind assumptions is that in some cases, you want to conditionally skip a test (instead of outright failing it) as described in this issue. Since the require and forward methods are already generated from the assertions, it seems like it would be practical to add generated assumptions to this library.

[charleswhchan]

@Mistobaan: yes. suite.T().Skip() will skip only the current test method. (workaround until the feature is implemented)

[mgsweet]

How to skip the whole suite? I am finding a way to skip some test suites in the CI env.

[puzzlermike]

You can skip the entire test suite by calling suite.T().Skip() inside the SetupSuite() method. For example:

type IntegrationTestSuite struct {
	suite.Suite
}

func (suite *IntegrationTestSuite) SetupSuite() {
	if os.Getenv("TEST_PROFILE") != "integration" {
		suite.T().Skip()
	}
        ...
}

And single tests could be skipped by calling the Skip() method inside the particular test, for example:

func (suite *IntegrationTestSuite) TestX() {
	if os.Getenv("TEST_PROFILE") != "integration" {
		suite.T().Skip()
	}
        ...
}

[brackendawson]

As noted above, you can call testing.T.Skip in a "TestMethod" to skip that method and you can call it in the SetupSuite method to skip an entire suite.

In teardown methods you can check if the test or suite has been skipped by calling testing.T.Skipped.