New: Improve debugging for RuleTester

[CoryDanielson]

Summary

Add a before hook to the valid/invalid code configuration, so that developers can inject a debugger into RuleTester before code is passed into the rule being tested.

Related Issues

Open issue in eslint/eslint: eslint/eslint#13625

[nzakas]

Thanks for writing this up. I feel like this is really a proposal to figure out a way to test just one snippet of code easily and less about using the debugger statement.

For instance, if we added a only: true option to each code block you care about and skipped the rest, similar to Mocha’s it.only, would that solve the problem?

[nzakas]

This seems very unexpected. Can you explain why it would be run twice?

[CoryDanielson]

I was thinking that the before hook might be called before RuleTester passes code into the create and fix methods for an ESLint rule. I might be mistaken about how that works, because I haven't written a rule that implements fix yet.

[nzakas]

I’m a bit worried at how focused this is on the debugger case. Are there any other potential uses?

[CoryDanielson]

I couldn't immediately think of another use case that wasn't already solved by another API. I thought before and after would allow for performance timing, but ESLint has documentation about performance analysis on rules.

[CoryDanielson]

Thanks for writing this up. I feel like this is really a proposal to figure out a way to test just one snippet of code easily and less about using the debugger statement.

For instance, if we added a only: true option to each code block you care about and skipped the rest, similar to Mocha’s it.only, would that solve the problem?

My concern with this was that by testing a single code snippit, you don't know if you've broken other tests. I bumped into this problem exactly while writing a rule. I was developing against a single code snippit, got it passing, then uncommented the others and saw that my changes had broken a handful of others.

it.only would be an improvement over the current scenario but using it would still be a less-than-ideal workflow when debugging

[nzakas]

I guess I’m used to the it.only process of getting that test working and then checking to see if any other tests broke, so that doesn’t bother me too much.

I’m not sold on this proposal as of this moment, but let’s see what others think.

[btmills]

I've done a similar workflow in the past of faking .only by commenting out all the other cases. Having an API to debug one case while still running all the others without debugging would be ideal in a vacuum, though implementing that might be pretty hairy because of the way RuleTester works. If a before hook is indeed too complex, I'd be open to considering a .only equivalent since it'd still be an improvement over the status quo.

[btmills]

RuleTester doesn't call rules directly. It calls linter.verify(...). If RuleTester were to call before immediately before the verify() call, it would still take developers on a tour of ESLint internals before they get to their rule code.

[CoryDanielson]

That might make this a non-starter. A way to avoid stepping through the internals would be if the rule was conditionally edited and a debugger was inserted into it by RuleTester

[CoryDanielson]

Rejected in favor of #73