[WIP] Allow Black to use a different Python version than Vim

[sbivol]

Description

This branch should fix #2547

It is sometimes the case that Vim was built with a different Python version than the one used by Black.
One such scenario is when the black virtualenv was created by an older version of Vim and then the system was upgraded in place.
Another scenario is when g:black_use_virtualenv is disabled and the manually created environment in g:black_virtualenv uses a different Python version. Right now Ubuntu 24.04 builds Vim with Python 3.12 but the system uses Python 3.11 as default, making this scenario more likely to happen:

$ python3 --version
Python 3.11.8

$ vim --version | grep lpython
Linking: gcc -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -lselinux -lsodium -lacl -lattr -lgpm -L/usr/lib/python3.12/config-3.12-x86_64-linux-gnu -lpython3.12 -ldl -lm

Currently, :BlackVersion reports Vim's Python version instead of the one found in the virtualenv. This PR fixes that inconsistency as well.

Finally, I added a few TODO comments for issues that I encountered while testing various scenarios but didn't have the time to fix in this PR.

Checklist - did you ...

• Add an entry in CHANGES.md if necessary?
• Add / update tests if necessary?
• Add new / update outdated documentation?

[JelleZijlstra]

Thank you!

[JelleZijlstra]

It's about Black primarily; Black itself requires 3.8+ (since Python 3.7 is no longer supported). By default I would want the same version requirement for the Vim plugin, but if there's some reason to keep supporting older versions in the plugin code, I'm open to it.

[JelleZijlstra]

I don't understand this comment; if Black is uninstalled, surely the import would fail?

[sbivol]

Unfortunately, not always. This can be triggered by Vim built with Py3.12 importing black from env/lib/python3.11/site-packages, which this PR makes possible.
If black is uninstalled after it was used once in such a setup, then pip uninstall leaves site-packages/black/__pycache__/ behind and that causes import black to succeed:

$ env/bin/python3 -c "import black; print(dir(black)); print(black.__file__)"
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
None

It's a corner case of a corner case, but it's a pain to troubleshoot when it bites.

[sbivol]

An alternative approach is to detect the Python version in the virtualenv, and if it doesn't match what Vim was built with, then go through these steps:

• inform the user that Vim won't use that virtualenv
• reset g:black_virtualenv to its default value
• create the default virtualenv as usual

This approach wouldn't enable mixing different Python versions and would use a bit of extra space for a new ~/.vim/black virtualenv, but should still leave the user with a working Black and no chance of confusion.

[sbivol]

After giving this some more thought, and further discussion in pypa/pip#11835, I'm now convinced that my approach isn't ideal. Mixing Python versions can have undesired side effects.
I am going to rework this PR.

[sbivol]

I used a different approach in #4675 which should hopefully make the changes easier to review and to test.
Closing this PR.