CI: pyarrow 10 broke our ci

[phofl]

Some pyarrow tests were failing. Additionally this caused some weird Matplotlib behavior that pulled a really old version (3.3.2)

https://github.com/pandas-dev/pandas/actions/runs/3613825618/jobs/6089843744

[rhshadrach]

Not sure how helpful this is, but when I create a new dev environment via mamba on ubuntu, I get very old versions of matplotlib and pyarrow (I think the same version mentioned above and pyarrow 1.0.0). I have to manually upgrade these after the creation of the new environment.

[mroeschke]

I get very old versions of matplotlib and pyarrow

This happens to me too. I think this is due to mamba's solver. If upgrading the libraries locally is okay without conflict (which I think doesn't downgrade other packages also), for some reason an unpinned library is not opting to install the most recent version