poetry 1.1.4 locks all packages to be pulled from secondary repo

[maksbotan]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.14

• Poetry version: 1.1.4

Issue

Hi,

I'm experiencing this behavior on 1.1.4. In pyproject.toml I have configured a secondary repo:

[[tool.poetry.source]]
name = "xxx"
url = "https://PRIVATE"
secondary = true

As such, I expect that poetry will fetch most packages from the main PyPI. And 1.1.3 indeed does that. However, 1.1.4 adds this configuration to every package (non-private as well) in poetry.lock:

[[package]]
name = "alabaster"
version = "0.7.12"
description = "A configurable sidebar-enabled Sphinx theme"
category = "main"
optional = true
python-versions = "*"

[package.source]
type = "legacy"
url = "https://PRIVATE"
reference = "xxx"

IMO this is unexpected behavior. I did git bisect and found out that this bug was introduced in 74fc8eb, in an attempt to fix other pypi-related issues.

[jaklan]

        # Always put PyPI last to prefer private repositories
        # but only if we have no other default source
        if not poetry.pool.has_default():
            has_sources = bool(sources)
            poetry.pool.add_repository(PyPiRepository(), not has_sources, has_sources)
        else:
            if io.is_debug():
                io.write_line("Deactivating the PyPI repository")

It seems there's missing logic about secondary repos here, sth like if not poetry.pool.has_primary() would be needed (just a concept, there's no such a function as for now).

[aviramha]

Also affected by this.

[yebinama]

Also experiencing the same issue. I have fixed it on my own and was going to open a pull request but it seems it was already done :)

A quick workaround is to redefine the Pypi repository but it's kind of ugly as it add an entry "package.source" in the lock file for every package coming from Pypi.

[[tool.poetry.source]]
name = "private"
url = "https://private_url"
secondary = true
default = false

[[tool.poetry.source]]
name = "official_pypi"
url = "https://pypi.org/simple"
default = true

[sinoroc]

@maksbotan @jaklan @aviramha @yebinama
@tomzx contributed a possible fix here: #3406
Would you mind testing it if you get a chance?

[mxab]

I tried it with 1.1.5 but my lock ran for more than 5 minutes. I guess this is not yet resolved

[dwyatte]

@sinoroc I can confirm that this is still affecting poetry 1.1.5. Here's a simple pyproject.toml that uses https://mirrors.aliyun.com/pypi/simple/ as a secondary index:

[tool.poetry]
name = "poetry-3306"
version = "0.0.1"
description = ""
authors = []

[tool.poetry.dependencies]
alabaster = "0.7.12"

[[tool.poetry.source]]
name = "xxx"
url = "https://mirrors.aliyun.com/pypi/simple/"
secondary = true

And the resulting poetry.lock:

[[package]]
name = "alabaster"
version = "0.7.12"
description = "A configurable sidebar-enabled Sphinx theme"
category = "main"
optional = false
python-versions = "*"

[package.source]
type = "legacy"
url = "https://mirrors.aliyun.com/pypi/simple"
reference = "xxx"

[metadata]
lock-version = "1.1"
python-versions = "*"
content-hash = "db905a1e716f615a6a6b2ec78b39cfafb347e0fd2d32e1a09c25486f5e292751"

[metadata.files]
alabaster = [
    {file = "alabaster-0.7.12-py2.py3-none-any.whl", hash = "sha256:446438bdcca0e05bd45ea2de1668c1d9b032e1a9154c2c259092d77031ddd359"},
    {file = "alabaster-0.7.12.tar.gz", hash = "sha256:a661d72d58e6ea8a57f7a86e37d86716863ee5e92788398526d58b26a4e4dc02"},
]

alabaster should be pulled from pypi.org since I've marked https://mirrors.aliyun.com/pypi/simple/ as secondary.