--find-links equivalent

[severinsimmler]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Using:

• Python 3.7.3
• Poetry 1.0.0b1

Feature Request

Because the CPU version of PyTorch is not on PyPI, they recommend installing it this way:

pip install torch==1.2.0+cpu --find-links https://download.pytorch.org/whl/torch_stable.html

However, there is no way to do this with Poetry?

Pointing directly to the wheel:

poetry add https://download.pytorch.org/whl/cpu/torch-1.2.0%2Bcpu-cp36-cp36m-manylinux1_x86_64.whl

results in

Updating dependencies
Resolving dependencies... (2.0s)


Package operations: 1 install, 0 updates, 0 removals

  - Installing torch (1.2.0+cpu https://download.pytorch.org/whl/cpu/torch-1.2.0%2Bcpu-cp36-cp36m-manylinux1_x86_64.whl)

[EnvCommandError]
Command ['/home/simmler/.cache/pypoetry/virtualenvs/epss-WeFenOJ_-py3.7/bin/python', '-m', 'pip', 'install', '--no-deps', 'torch==1.2.0+cpu'] errored with the following return code 1, and output: 
Collecting torch==1.2.0+cpu
  Could not find a version that satisfies the requirement torch==1.2.0+cpu (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 0.4.1, 0.4.1.post2, 1.0.0, 1.0.1, 1.0.1.post2, 1.1.0, 1.2.0)
No matching distribution found for torch==1.2.0+cpu

And adding https://download.pytorch.org/whl/torch_stable.html as a repository to the pyproject.toml does not work, because it (the PyTorch wheel index page) is not PEP conform.

[cc-a]

Hi. I was thinking of having a go at this for Hacktoberfest.

The approach I was thinking of:

• Implement a variation of the LegacyRepository class
• Extend the expected fields for the source toml section to add a flag that specifies this type of simple html repository
• Implement the relevant logic in poetry.py to determine the correct repository class to use

Sound about right?

[severinsimmler]

Sounds like a plan 👍

[woodlandhunter]

@cc-a a --find-links that is equivalent to pip's would also accept file paths (a directory). See pip docs here. We could really use this. It's also mentioned in these issues:

#558
#1289

Your comment doesn't mention this scenario, would --find-links=DIR be covered by what you are working on?

[cc-a]

I'll have a crack at it, probably in a second PR. In principle the same logic should be required just working with file paths instead of urls.

[golyalpha]

Right now the workaround is defining remote file dependencies (in pytorch's case, platform specific).
This works for me on the current release:

poetry add https://download.pytorch.org/whl/cu101/torch-1.3.1-cp37-cp37m-win_amd64.whl --platform win32

Unfortunately, this will limit you to only one platform, as you can't have the same dependency for multiple platforms with different file links.

[germn]

After installing torch with

poetry add https://download.pytorch.org/whl/cu101/torch-1.3.1-cp37-cp37m-win_amd64.whl

each time I add any package poetry does following:

- Updating torch (1.3.1 -> 1.3.1 https://download.pytorch.org/whl/cu101/torch-1.3.1-cp37-cp37m-win_amd64.whl)

It doesn't break anything, but takes some time.

Anyone knows a way to avoid "updating"?

[golyalpha]

Not really, it's a workaround for a current limitation in Poetry, but until that limitation is fixed, you could probably limit the impact by using a local cache (like devpi) @germn

[allen-chin]

@germn Something like below should work fine for your case.
pyspark = {file = "pyspark-3.0.0.dev0.tar.gz"}

[golyalpha]

@allen-chin The sideeffect of that is continual reinstallations of the package. Just because the package is local, doesn't mean Poetry won't reinstall it.

[cpbotha]

I created a valid simple index that can be used with index-url and as a tool.poetry.source for all of the pytorch.org official packages. You can find the new index here: https://vxlabs.com/pypi/

I massaged the official PyTorch find-urls file to look more like a PEP 503 compatible simple index, but it's still pointing to all of the official packages at pytorch.org.

It's currently working like a charm in my poetry setup.

[aviramha]

@cpbotha Plan on adding 1.5.0 soon?
BTW Any news on this issue?