Move towards pytest style tests

[jdufresne]

Is the project interested in moving to use pytest conventions? This would mean something of the lines:

• Use assert instead of unittest assert methods
• Removing unittest classes
• Using fixtures for stdout/stderr capturing, temporary files, and more (and perhaps some internal ones too.)
• Using parameterized tests?

If so, I can help out here and there. As there are quite a lot of tests, it might make sense to this on a file by file basis or in pieces.

Thoughts?

[mfonism]

I'm interested in helping out with this.

[hugovk]

Well, no one has objected yet!

I had originally wanted to keep things fairly agnostic with regards third-party tools, to stick with the stdlib unittest for the tests.

But that was when there was nose, nose2 and pytest, in addition to unittest. Now:

• Nose is in maintenance mode with no updates for 4 years (but still 3m downloads per month)
• Nose2 (165k/m) is still going but mainly maintained for those "stuck" using it and they encourage people new to testing to try pytest
• Pytest (12m/m) is very popular and well-maintained, and arguably becoming a standard for unit tests

So I'd be up for giving it a go.

I'd suggest to try and keep the main usage of it fairly simple, so there's a low barrier for new contributors looking at tests. Although parameterised tests could certainly be used in a few cases, it's a good way to remove duplication. And if any other pytest things are really helpful, then let's have a look at that too.

If so, I can help out here and there. As there are quite a lot of tests, it might make sense to this on a file by file basis or in pieces.

I agree, it definitely makes sense to do it in parts rather than one big bang.

Would you be able to do a smallish PR so we can see how it looks?

Thank you!

[jdufresne]

A first pass as the easier tests to port: #4369