Multiple packages in the same project

[tlecarrour]

I'm migrating a project from setuptools to Poetry. The project contains 2 packages, and my setuptools setup was packaging them together. This doesn't seem to work with my Poetry setup!? But maybe it's because the only place I mentioned my second package is in the [package] include section.

Is it possible to do it with Poetry or is it considered as a bad practice ?!

[sdispater]

I am not sure I understand. What do you mean? Also what does your pyproject.toml look like?

[tlecarrour]

My apologies, I should have included both my setup.py and my pyproject.toml!

The setup.py I was using before migrating to Poetry looks like this:

from setuptools import setup, find_packages

setup(
    name='multiple_packages',
    version='0.1.0',
    description='',
    author='TLC',
    author_email='tlc@ll.fr',
    packages=find_packages()
)

My project folder layout is:

─ multiple_packages
   ├── setup.py
   ├── multiple_packages
   │   └── __init__.py
   └── second_package
       └── __init__.py

Using python setup.py sdist, I build a tar file that contains:

multiple_packages-0.1.0/
multiple_packages-0.1.0/second_package/
multiple_packages-0.1.0/second_package/__init__.py
multiple_packages-0.1.0/multiple_packages/
multiple_packages-0.1.0/multiple_packages/__init__.py

When I realized that the features I was missing was packages=find_packages(), I searched again it the issues and found #56 and #92. So I tried the "src layout", but I couldn't get the second_package to be included in the tar, even with the following pyproject.toml:

[tool.poetry]
name = "multiple_packages"
version = "0.1.0"
description = ""
authors = ["TLC <tlc@ll.fr>"]

[package]
include = ["src/second_package/*.py"]

And my layout is now:

 ─multiple_packages
   ├── pyproject.toml
   └── src
     ├── multiple_packages
     │   └── __init__.py
     └── second_package
          └── __init__.py

The tar file generated by poetry build (I use Poetry 0.9.1) contains:

multiple_packages-0.1.0/pyproject.toml
multiple_packages-0.1.0/src/multiple_packages/__init__.py
multiple_packages-0.1.0/setup.py

I must be doing something wrong. Either with my pyproject.toml or trying to distribute both packages at the same time. Anyway, thanks for your time!

[sdispater]

Unless I am missing something, shouldn't your layout be:

─ setup.py
─ multiple_packages
   ├── __init__.py
   ├── multiple_packages
   │   └── __init__.py
   └── second_package
       └── __init__.py

and with poetry:

─ pyproject.toml
─ multiple_packages
   ├── __init__.py
   ├── multiple_packages
   │   └── __init__.py
   └── second_package
       └── __init__.py

[merwok]

I think the OP really means two independent packages, not one package with sub-packages.

Distutils always supported to create a distribution with 0 or more modules, 0 or more packages, 0 or more scripts, or even only data files.

[tlecarrour]

As @merwok said, I meant 2 independent packages, not 2 subpackages in a multiple_packages package.

I don't know if it's something that should be doable and if this issue is still relevant. In the mean time I moved the second_package to a different project, that's solved the problem for me. But now I want to publish it on a private repo and I'm getting a error on poetry update after adding [[tool.poetry.source]] in my pyproject.toml. … but that's a different problem and I'll first investigate by myself (and read the doc ^_^') before submitting a new issue.

Feel free to close this issue and thanks again for your time!

[sdispater]

It is now supported with the packages property

[tlecarrour]

Thanks !!! :-)

[aantn]

Just to be clear, this refers to import packages not distribution packages, right? (Using the terminology from here: https://sinoroc.gitlab.io/kb/python/packaging.html)

[merwok]

From the first messages, yes. A Python project can contain multiple modules, packages, scripts and data files.

[RoelantStegmann]

Just to be clear, this refers to import packages not distribution packages, right? (Using the terminology from here: https://sinoroc.gitlab.io/kb/python/packaging.html)

Wondering this too. Otherwise how could I separate the dependencies of the individual distribution packages?

For now I make a different poetry project for each, but there is quite some boiler plate double (ci/cd pipelines, etc)