Skip to content

suite.Run: validate Test* methods signature#1512

Open
Barugoo wants to merge 3 commits into
stretchr:masterfrom
Barugoo:suite-signature-check
Open

suite.Run: validate Test* methods signature#1512
Barugoo wants to merge 3 commits into
stretchr:masterfrom
Barugoo:suite-signature-check

Conversation

@Barugoo
Copy link
Copy Markdown

@Barugoo Barugoo commented Nov 30, 2023

Summary

Added panic in suite.Run if Test* method has args or returning values

P.S. This is my first ever opensource contribution, so I'm very open to any code logic/styling criticism!

Changes

  • Now, before calling Suite method there's an additional check for signature correctness: it should have not more than 1 arg (receiver) and 0 returning values
  • If Test* method signature fails the above check test panics with the message: testify: suite method <MethodName> shouldn't have any arguments or returning values
  • Added test cases for incorrect signature variants

Motivation

As @wvell mentioned in issue #1508 currently, suite.Run returns an ambiguous error when executing with instance of Suite that has incorrect Test* methods signatures.

Related issues

Closes #1508

@Barugoo Barugoo changed the title Added panic in suite.Run if Test* method has args or returning values suite.Run checks if Test* method has args or returning values Nov 30, 2023
Comment thread suite/suite.go

if method.Type.NumIn() > 1 || method.Type.NumOut() > 0 {
msg := fmt.Sprintf("testify: suite method '%s' shouldn't have any arguments or returning values\n", method.Name)
panic(msg)
Copy link
Copy Markdown
Author

@Barugoo Barugoo Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about using panic here tho, seems appropriate to me, but i will appreciate if you take a look @dolmen

Copy link
Copy Markdown

@wvell wvell Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to use t.Error(msg)? Then you can directly see the TestMethod violating the contract. I find panic always quite hard to read.

Copy link
Copy Markdown
Author

@Barugoo Barugoo Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wvell the motivation to add panic here instead of t.Error is the source of this condition. I.e. It is not the tested code returning unexpected result, but rather the programmer who messed up the signature of the Test* method itself. In that case panic is more fitting IMHO

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah fair point, I just hate reading panics.

Thanks for putting in the work.

Copy link
Copy Markdown
Collaborator

@dolmen dolmen Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm a strong advocate of panic to catch programmer's mistake, I think a panic is not appropriate in this case as the stack trace will not be useful to the user.

Instead Run is being called with *testing.T, so let's just use t.Fatal (or t.Fatalf) to report the issue.

However, the main problem with this implementation is that this check must be done much much earlier, line 151, just after the successful call to methodFilter.

@brackendawson brackendawson added this to the v1.8.6 milestone Feb 24, 2024
@brackendawson brackendawson modified the milestones: v1.9.1, v1.10 Mar 5, 2024
@dolmen dolmen added the pkg-suite Change related to package testify/suite label Mar 6, 2024
Comment thread suite/suite.go

if method.Type.NumIn() > 1 || method.Type.NumOut() > 0 {
msg := fmt.Sprintf("testify: suite method '%s' shouldn't have any arguments or returning values\n", method.Name)
panic(msg)
Copy link
Copy Markdown
Collaborator

@dolmen dolmen Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm a strong advocate of panic to catch programmer's mistake, I think a panic is not appropriate in this case as the stack trace will not be useful to the user.

Instead Run is being called with *testing.T, so let's just use t.Fatal (or t.Fatalf) to report the issue.

However, the main problem with this implementation is that this check must be done much much earlier, line 151, just after the successful call to methodFilter.

@dolmen dolmen added the bug label Mar 6, 2024
@dolmen dolmen changed the title suite.Run checks if Test* method has args or returning values suite.Run: validate Test* methods signature Mar 6, 2024
@brackendawson brackendawson modified the milestones: v1.10, v1.10.1 Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug pkg-suite Change related to package testify/suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

suite: missing signature check before calling Test* method

4 participants