Cannot install multiple python versions since PR #404

[dcrowe]

Hi. Great project.

I have a self-hosted renovate-bot running on Docker to support multiple projects that are built using a variety of python versions. Unfortunately, due to the way pipenv works, that means I need to have both python3.8 and python3.9 installed in the one Docker image.

Before PR #404 it was possible (with a little effort) to operate multiple python versions side-by-side. Since this PR, it is not possible. The culprit appears to be the addition of the PYTHONHOME environment variable which causes python to look in the wrong location for libraries.

It looks like I can probably use a combination of reset_tool_env and manually set paths to make it work. Is that the correct approach?

This is an example of a previously working build

FROM containerbase/buildpack:4.0.0
USER root

RUN install-tool python 3.10.5
RUN install-tool python 3.9.9
ENV PATH=${PATH};/usr/local/python/3.10.5/bin/;/usr/local/python/3.10.5/bin/

RUN python3.10 -m pip install pipenv

This is an example of a failing Dockerfile after the PR:

FROM containerbase/buildpack:4.1.0
USER root

RUN install-tool python 3.10.5
RUN install-tool python 3.9.9
ENV PATH=${PATH};/opt/buildpack/tools/python/3.10.5/bin/;/opt/buildpack/tools/python/3.10.5/bin/

RUN python3.10 -m pip install pipenv

Here is the build failure:

 > [4/4] RUN python3.10 -m pip install pipenv:
#7 0.364 Python path configuration:
#7 0.364   PYTHONHOME = '/opt/buildpack/tools/python/3.9.9'
#7 0.364   PYTHONPATH = (not set)
#7 0.364   program name = '/opt/buildpack/tools/python/3.10.5/bin/python3.10'
#7 0.364   isolated = 0
#7 0.364   environment = 1
#7 0.364   user site = 1
#7 0.364   import site = 1
#7 0.364   sys._base_executable = '/opt/buildpack/tools/python/3.10.5/bin/python3.10'
#7 0.364   sys.base_prefix = '/opt/buildpack/tools/python/3.9.9'
#7 0.364   sys.base_exec_prefix = '/opt/buildpack/tools/python/3.9.9'
#7 0.364   sys.platlibdir = 'lib'
#7 0.364   sys.executable = '/opt/buildpack/tools/python/3.10.5/bin/python3.10'
#7 0.364   sys.prefix = '/opt/buildpack/tools/python/3.9.9'
#7 0.364   sys.exec_prefix = '/opt/buildpack/tools/python/3.9.9'
#7 0.364   sys.path = [
#7 0.364     '/opt/buildpack/tools/python/3.9.9/lib/python310.zip',
#7 0.364     '/opt/buildpack/tools/python/3.9.9/lib/python3.10',
#7 0.364     '/opt/buildpack/tools/python/3.9.9/lib/python3.10/lib-dynload',
#7 0.364   ]
#7 0.364 Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
#7 0.364 Python runtime state: core initialized
#7 0.364 ModuleNotFoundError: No module named 'encodings'
#7 0.364 
#7 0.364 Current thread 0x00007f6f1837b280 (most recent call first):
#7 0.364   <no Python frame>

[viceice]

Will try to find another workaround for that, completly removing PYTHONHOME isn't an option, as it causes a lot other issues.

Running python3.10 -m pip was never planned to work, but i classify it as bug. Hopefully it's easily fixable.

[dcrowe]

Thanks for taking a look.

python3.10 -m pip was just an example. Given that the PYTHONHOME var is forcing the use of the 3.9 libs when running the 3.10 interpreter, I would expect to be able to find many similar incompatibilities.

I've found a work-around that works for me:

RUN rm /opt/buildpack/env.d/python.sh
ENV PATH="${PATH}:/opt/buildpack/tools/python/3.10.5/bin"
ENV PATH="${PATH}:/opt/buildpack/tools/python/3.9.9/bin"

Most things seems to work after just removing the env file, but the path additions are needed when installing packages that provide CLI tools.

[viceice]

Our preferred way to install packages is this:

FROM containerbase/buildpack:4.1.0
USER root

RUN install-tool python 3.10.5
RUN install-tool python 3.9.9
RUN install-pip pipenv 2022.7.24

the alternate command pip install --upgrade pipenv should work too

[viceice]

if you like to have pipenv to be installed via python 3.10 use this order

FROM containerbase/buildpack:4.1.0
USER root

RUN install-tool python 3.9.9
RUN install-tool python 3.10.5
RUN install-pip pipenv 2022.7.24

So the python 3.10 bin is on path instead of the 3.9