`poetry export` uses branch name for VCS dependencies rather than resolved commit hash

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian 10
• Poetry version: 1.1.7 (also reproducible with 1.2.0a1)
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/0xallie/072e72cbf1f002f48944d0f93585e4ae

Issue

Given the above pyproject.toml file, poetry export --without-hashes -o requirements.txt will generate the following output:

backports.functools-lru-cache==1.6.4; python_version >= "2.6" and python_version < "3"
beautifulsoup4==4.9.3
cssutils==1.0.2
enum34==1.1.10; python_version < "3.4"
future==0.18.2; python_version >= "2.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0"
lxml==4.4.0
pycaption @ git+https://github.com/pbs/pycaption@master
six==1.16.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0"
soupsieve==1.9.6

Relevant section of poetry.lock:

[[package]]                                                                                                                                                       
name = "pycaption"                                                                                                                                                
version = "1.0.6"                                                                                                                                                 
description = ""                                                                                                                                                  
category = "main"                                                                                                                                                 
optional = false                                                                                                                                                  python-versions = "*"                                                            
develop = false                                                                                                                                                   
                                                                                                                                                                  
[package.dependencies]                                                                                                                                            
beautifulsoup4 = ">=4.8.1,<4.10"                                                                                                                                  
cssutils = ">=0.9.10"                                                                                                                                             enum34 = {version = "*", markers = "python_version < \"3.4\""}
future = "*"                                                                                                                                                      
lxml = ">=3.2.3"                                                                                                                                                  
six = ">=1.9.0"                                                                                                                                                   
                                                                                                                                                                  
[package.source]                                                                                                                                                  
type = "git"
url = "https://github.com/pbs/pycaption" 
reference = "master"
resolved_reference = "3f377a32eb04a0ef79548d8f595092e249499bb4"             

Based on this, I would expect requirements.txt to contain the following:

pycaption @ git+https://github.com/pbs/pycaption@3f377a32eb04a0ef79548d8f595092e249499bb4

[koffie]

I just created python-poetry/poetry-core#603 in poetry-core to make it easier to resolve this issue.

[koffie]

@radoering Thanks for helping getting python-poetry/poetry-core#603 merged. I would now also like to update the poetry-plugin-export so that this new functionality is also available to the end user.

However I would like to know what the best way to go is before implementing this. So what is better do you think?

1. Update the required version of poetry-core in pyproject to the release that will include the functionality and assume that the new flag is available in the call to to_pep_508.
2. Only expose the new functionality if the version of python core is recent enough.

Also is this issue considered a bug fix or the addition of a new feature?

[radoering]

Also is this issue considered a bug fix or the addition of a new feature?

I would consider it a bugfix. Since poetry export exports the locked packages I assume the goal is to export unambiguous requirements. Using the branch name instead of the locked commit hash contradicts this goal.

However I would like to know what the best way to go is before implementing this. So what is better do you think?

Both are valid options. The first one is less work, but requires a poetry-core (and poetry) release first. The second one allows to do it now but requires some cleanup later (when the required poetry-core version will be updated eventually). Do as you like.

If we choose the second approach, I'd probably add a try-except and a comment that this can be removed after updating the required poetry-core version to a version that contains python-poetry/poetry-core#603. Further, we have to make a distinction in the test depending on the poetry-core version, which can be removed later.

[koffie]

Ok, I made a pull request that already contains the necessary fix. I went for the approach by putting the required version in pyproject.toml by cheating a bit and using vcs dependencies for poetry and poetry-core. It passes all tests except for those on 3.7 which is not so strange since the most recent version of poetry dropped support for 3.7.

One can try out this new functionality already by doing:

poetry add git+https://github.com/koffie/poetry-plugin-export.git@main
poetry run poetry export

Although it is probably better to wait for an official release instead of using these kind of hacks.

[koffie]

@radoering I updated the pr since poetry 1.6.0 has been released and it is now ready for review.

[jjmonsalveg]

@koffie I see that direct dependencies added in the toml file now are being translated from branch name to commit hash. However, I can not see the dependencies of dependencies do the same.

Imagine this example:

• lib A package is created with branch test and version 1.0.1 current commit hash pointed by test abc
• lib B needs package A and the dependencies have git+https://github.com/<org>/libA.git@test <=== notice that use branch test
• MyAwesome project needs lib A and also lib B (both using git dependencies and branch names rather commit hashes), so both appear in the toml file, when exported and installed by pip, it receives this error:

The conflict is caused by:
    The user requested libA 1.0.1 (from git+https://github.com/<org>/libA.git@abc)
    libB 2.1.2 depends on libA 1.0.1 (from git+https://github.com/camino-financial/libA.git@test)
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

I'm not able to share the repositories since they are private however I did my best to explain the problem. Is this a bug or do I need to do something?

Python: 3.12.4
poetry: 1.8.3

[radoering]

However, I can not see the dependencies of dependencies do the same.

Did you check the content of the resulting requirements.txt? That is where you see if poetry exports branch names or hashes - not in the output of a pip command.

Is this a bug or do I need to do something?

It looks like you posted the output of pip failing to resolve dependencies. However, since you have already resolved dependencies via poetry and exported all resolved dependencies (direct and transitive dependencies), it makes no sense to let pip resolve again. You should use --no-deps when installing the requirements.txt via pip.

[jjmonsalveg]

Did you check the content of the resulting requirements.txt? That is where you see if poetry exports branch names or hashes - not in the output of a pip command.

I did, but I guess your second answer clarify the point

It looks like you posted the output of pip failing to resolve dependencies. However, since you have already resolved dependencies via poetry and exported all resolved dependencies (direct and transitive dependencies), it makes no sense to let pip resolve again. You should use --no-deps when installing the requirements.txt via pip.

Ok will use this --no-deps approach however I read that may be dangerous but I'll keep an eye on the behavior of the system. Thank you this was helpful

[sastorsl]

There is part of the landscape that still has issues with this.
As of https://github.com/python-poetry/poetry-plugin-export/releases/tag/1.6.0 poetry-plugin-export creates a requirements.txt with the resolved commit hash instead of the branch name.

But during pip install we still get an error message. Why pip, to create a complete deliverable.

In pyproject.toml

external_dependency = { git = "https://gitrepo.tld/my_project/my_repo.git", branch = "feature/topicbranch"}

cd ~/tmp &&
rm -rf my_repo &&
git clone ssh://git@gitrepo.tld/my_project/my_repo.git &&
cd my_repo &&
git checkout feature/topicbranch  &&
pyenv shell 3.11 &&
eval $(poetry env activate) &&
poetry export -f requirements.txt --without-hashes -o ./package-requirements.txt &&
mkdir bin &&
poetry run pip install . -r package-requirements.txt -t bin

Results in

INFO: pip is looking at multiple versions of my_repo to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install external_dependency 0.0.0 (from git+https://gitrepo.tld/my_project/external_dependency.git@cbefdb2299bd820b0b4eaa181a0c062d7c18bba7) and my_repo==1.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested external_dependency 0.0.0 (from git+https://gitrepo.tld/my_project/external_dependency.git@cbefdb2299bd820b0b4eaa181a0c062d7c18bba7)
    my_repo 1.0.0 depends on external_dependency 0.0.0 (from git+https://gitrepo.tld/my_project/external_dependency.git@feature/topicbranch)

I assume this is because pip install sees the branch named in pyproject.toml somewhere even though the package-requirements.txt file have no reference to it.