Document internal pytest warnings

[nicoddemus]

As discussued in #2466, we should document in a single place all internal warnings that can be raised by pytest

[RonnyPfannschmidt]

@nicoddemus i wonder if additionally we should ensure that if pytest warnings raise an error we turn them into always