Check test coverage

[mgeier]

A coverage report should be part of the tests anyway, probably we could use http://coveralls.io/ for automated coverage reports?

[mgeier]

To check coverage locally I installed pytest-cov from https://pypi.python.org/pypi/pytest-cov and then ran:

py.test --cov pysoundfile.py

This prints a small summary on the terminal. To get a much nicer and very detailed HTML coverage report I did this:

py.test --cov pysoundfile.py --cov-report html

It can also check the coverage of the test code itself:

py.test --cov tests/ --cov-report html

[mgeier]

An alternative way to check the coverage is coverage.py.

coverage run --source pysoundfile.py -m py.test
coverage html

An advantage over the pytest-cov plugin is that it can check branch coverage:

coverage run --branch --source pysoundfile.py -m py.test
coverage html

[mgeier]

BTW, I changed my opinion regarding automated coverage reports:
We should not advertise the percentage of coverage because it is a very un-reliable metric for the quality of the code.
Therefore, automated coverage reports (e.g. after each commit) are not necessary.

I still think coverage reports are a useful tool for improving the tests, but it is sufficient to run coverage analysis locally.
We should probably add the aforementioned commands in the README file (#26).

[bastibe]

All of this sounds very reasonable. I agree that we don't need to advertise coverage in the README.

I think that the README should be primarily focused on getting users up to speed with the library. Coverage is probably not something a user is very concerned with (except maybe as a weak measure of code quality). I therefore think that it is unnecessary to even put the commands in the README.

Plus, we don't ship any special accomodations for coverage reports and you can use any coverage tool you like to generate coverage reports. This is another reason why we probably should not put instructions for one particular tool into the README.

This is purely my intuition talking though. We could look at a few other projects and see how they are doing it.

[mgeier]

OK, the README is probably the wrong place for that.

But I think it could be nice to have a section somewhere in the Sphinx documentation for potential contributors where we could mention the tools we use on the development side.
This would also serve as a reminder for me, because I tend to forget the exact command line arguments.

[bastibe]

Great idea!

[mgeier]

As we agreed that automated coverage reports are not necessary, I'm closing this issue.

Documentation of coverage tools will be done in #66.