Move pip into its own layer

[edmorley]

Currently pip is installed into the same layer as Python, since it is installed into the system site-packages directory.

This is primarily because the user site-packages directory is used for the app dependencies, leaving us few other options as to where to install pip, given that:

• we don't want pip in the layer with the app dependencies (otherwise it can't be cached, given the app dependencies layer for pip can't be cached due to pip's non-determinism since it doesn't sync environments)
• we can't use PYTHONPATH since any directories specified via PYTHONPATH are given a higher precedence in Python's sys.path than the Python stdlib (unlike system and user site-packages, which are added to sys.path after the Python stdlib) - which can then cause hard to debug issues if apps use outdated backport libraries (which can often happen unintentionally via broken/suboptimal packages in their transitive dependency tree).

pip being in the same layer as Python means that:

• we can't exclude pip from the run image (ie: make it a build time only layer), either optionally or by default (see Exclude pip from the app image #255)
• a change in pip version means the Python layer has to be unnecessarily unvalidated (albeit this only occurs a few times a year)
• the Python layer will vary based on the choice of package manager (since we wouldn't need to install pip when using Poetry or uv), which will reduce layer re-use between apps (if the images are stored in an environment where layers can be shared across apps).

However, once we move the app dependencies into a virtual environment in #253 this will free up the user site-packages, meaning we perform a user install of pip, into its own layer.

GUS-W-16616956.