Import dependencies from pyproject.toml: [project] and [project.optional-dependencies]

[paugier]

Problem description

Python packages dependencies are now described in the pyproject.toml file, sections [project] and [project.optional-dependencies].

I understand that pixi [dependencies] are something else because they are about conda packages and usually not PyPI dependencies. However, when initializing pixi for a package with a clean pyproject.toml, it would be nice to be able to fill the pixi.toml file with values from the pyproject.toml file.

Note that it is not convenient to just copy/paste the text because the format is not the same:

[project]
name = 'fluidsim'
[...]
dependencies = [
    "importlib_metadata",
    "fluidsim-core>=0.7.4",
    "h5py",
    "h5netcdf",
    "transonic",
    "xarray",
    "rich",
    "matplotlib>=3.3",
    "scipy",
]

[project.optional-dependencies]

fft = [
    "pyfftw >= 0.10.4",
    "fluidfft >= 0.2.7",
]
mpi = [
    "mpi4py",
]

[ruben-arts]

We've had internal talks about a pixi import pyproject.toml or pixi init --from pyproject.toml command to do the conversion automatically. If anyone would like to take a stab at it that would be awesome!

[vnling]

Hey! Can I pick this up?

[wolfv]

Sure @vnling :)

[wolfv]

Note we have a partial pyproject.toml parser in rip I believe (https://github.com/prefix-dev/rip) cc @tdejager could we use this here?

[tdejager]

I think so yes, we could also add real pyproject.toml parsing to rip if that would be interesting.

[whlavina]

Since pyproject.toml is the (new) Python standard for project dependencies, and I don't see that changing to pixi.toml within that community - at least not anytime soon - I'd suggest that Pixi support these 3 capabilities:

• An explicit import process, for anyone wishing to migrate entirely to pixi and drop pyproject.toml.
• Hybrid support, where pixi.toml can be used as primary configuration, but also honoring a sibling pyproject.toml for Python dependencies - with no need for any copying of content. Perhaps the pixi.toml could specify whether such hybrid mode is enabled (i.e. a config entry in pixi.toml could specify whether to honor or ignore the content of a pyproject.toml file).
• Compatibility mode, where only pyproject.toml exists, and ideally, pixi will use it as-is, perhaps with defaults for any Pixi settings that cannot be represented.

[olivier-lacroix]

In the next pixi version, it will become possible to run pixi init in a folder containing a pyproject.toml.

• after this init, pixi will be able to use that file instead of a pixi.toml
• once feat: Create features and environments from extras #1077 is merged, optional dependencies will be managed as well

@tdejager I believe this issue can be closed?

[tdejager]

@olivier-lacroix yes let's close and accept individual features as issues :)