[Python] Reduce warnings during tests

[asfimport]

Numerous warnings are displayed at the end of a test run, we should strive them to reduce them:
https://github.com/apache/arrow/actions/runs/3533792571/jobs/5929880345#step:6:5489

Reporter: Antoine Pitrou / @pitrou
Assignee: Miles Granger / @milesgranger

Related issues:

• [Python] ResourceWarnings raised by s3 related tests (is a parent of)
• [Python] Handle pytest 8 deprecations about pytest.warns(None) (is a parent of)

PRs and other links:

• GitHub Pull Request #14729

_{Note: This issue was originally created as ARROW-18399. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
cc @milesgranger

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
We have ARROW-17651 and ARROW-18125 already for specific subsets of those warnings (in case you might tackle them as separate PRs, you can use those / make them child tasks of this one, or otherwise close them as duplicate)

[asfimport]

Miles Granger / @milesgranger:
Didn't see your comment until now, I just put them all in the one PR. Thanks for the references though! :)

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Issue resolved by pull request 14729
#14729