feat: Add the project itself as an editable dependency

[olivier-lacroix]

Fixes #1083

[pavelzw]

Would this also work for projects that are compiled with maturin for example?

[tdejager]

Would this also work for projects that are compiled with maturin for example?

If you would specify that correctly in the pyproject.toml it should. However, I'm kinda unsure about the UV BuildIsolation so we would need to check that it uses the rust and cargo from the conda environment.

[pavelzw]

Would be nice if pixi read the build-system.requires in some way

[olivier-lacroix]

Note this does not seem to work currently

running cargo run shell --manifest-path examples/flask-hello-world-pyproject/pyproject.toml

then python

then import flask_hello_world_pyproject

yields a ModuleNotFoundError

[pavelzw]

I just tested maturin with the regular editable install feature from main.
There, it doesn't re-compile after changing something in the rust code...
Not sure if this is intentional.

# somewhere in rust code
println!("Hello World");


# ...
pixi run test
Hello World

# change it to 
println!("Hello World 2");

# ... 
pixi run test
Hello World

Seems like pip install -e . only builds once and doesn't rebuild on every code change with maturin. Should pixi have different behavior? We could just advise not to use the editable install mode with maturin projects.

[pavelzw]

Maybe having an option to turn off automatically adding . in editable mode could fix this for projects that require maturin etc.?

[olivier-lacroix]

@pavelzw , does automatic recompilation work with pip install -e ? looking at the maturin doc, it seems to say it works only for edits of the python code.

[pavelzw]

yeah it looks like maturin doesn't work that way. maybe adding somewhere in the docs that you should disable this feature (using tool.pixi.no-editable-install = true in pyproject.toml) for projects using maturin etc.

[tdejager]

We can also add a flag or maybe a boolean to always rebuild, although it does kind of have a bit of feature overlap with the pixi update we've been disucssing.

[tdejager]

@olivier-lacroix seems main wasn't merged in this branch yet, so there were no editable installs at all 😄 I've merged main and added some small changes. Hope you dont't mind me pushing on this.

I believe it should now work.

Also maybe we should use a proper more python directory structure for this example, even though this works with setuptools, hatch will fail for example: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

Even setuptools recommends having a seperate source folder: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

[tdejager]

Also, do we need to expand the CLI to include installing the local editable projects with extra's?

[olivier-lacroix]

@olivier-lacroix seems main wasn't merged in this branch yet, so there were no editable installs at all 😄

🤦

Indeed, it works :-)

[olivier-lacroix]

Also maybe we should use a proper more python directory structure for this example, even though this works with setuptools, hatch will fail for example: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

Yeah. I have reorganised the folder into a proper python package

[olivier-lacroix]

Maybe having an option to turn off automatically adding . in editable mode could fix this for projects that require maturin etc.?

@pavelzw , I do not see the harm in having it installed as editable, even without the rebuild trigger?

[olivier-lacroix]

Also, do we need to expand the CLI to include installing the local editable projects with extra's?

Yes maybe. Also, in #1077, I propose to create pixi features & environments based on the extras automatically, which is another way to provide that feature

[olivier-lacroix]

Quick question @tdejager , I see the source dependency includes a sha256 in the lock file: when is this value re-rendered?

In case that lockfile is used in a dockerfile, it may be useful to be able to opt-out of recording that hash: otherwise, you cannot cache the layer containing all the dependencies if the hash has changed.

what do you think?

[tdejager]

Quick question @tdejager , I see the source dependency includes a sha256 in the lock file: when is this value re-rendered?

In case that lockfile is used in a dockerfile, it may be useful to be able to opt-out of recording that hash: otherwise, you cannot cache the layer containing all the dependencies if the hash has changed.

what do you think?

This is regenerated when the lock file is invalidated, I think there is the '--frozen' flag or something that bypasses this.

[pavelzw]

I do not see the harm in having it installed as editable, even without the rebuild trigger?

I can think of some situations (for example in docker containers) where you would want to install the package in non-editable mode. So having an on/off switch might still help in these cases

[olivier-lacroix]

Alright. message added @tdejager . I kept it short :-).

[tdejager]

Sorry. meant to select Request changes.

[olivier-lacroix]

I have added more details in the doc about the build-system section. Let me know if you have any comment @tdejager

[tdejager]

I think this looks great! WDYT @ruben-arts?

[tdejager]

Pushed a fix where it was using a wrong path in the test. Let's see if that solves things.

[olivier-lacroix]

@tdejager ,added platforms. now the downstream tests pass, but not the cargo tests on windows! any idea where that error could come from?

[olivier-lacroix]

It was passing before: is this just a temporary failure?

[ruben-arts]

He I've only followed this feature from the side line. What is the conclusion? Do we auto add the project itself or do we initialize with the flask-hello-world-pyproject = { path = ".", editable = true } Because while testing it somehow seems to do both for me.

[olivier-lacroix]

@ruben-arts we initialise with the flask-hello-world-pyproject = { path = ".", editable = true } which will then install it upon a call to pixi install or such

[olivier-lacroix]

@tdejager , happy with this PR? all tests are 🆗