Skip to content

support pep517 builds - poetry install on Mac fails #133

@nigelmegitt

Description

@nigelmegitt

On MacOS 14.3, python 3.11.7, in a project that needs python-snappy as a dependency of another library (in this case, localstack). After installing snappy with brew, the attempt to build python-snappy fails with:

 • Installing python-snappy (0.6.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  /private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py:265: UserWarning: Unknown distribution option: 'cffi_modules'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-14-x86_64-cpython-311
  creating build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/snappy.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/snappy_cffi.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/__init__.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/hadoop_snappy.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/snappy_formats.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/__main__.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  copying src/snappy/snappy_cffi_builder.py -> build/lib.macosx-14-x86_64-cpython-311/snappy
  running build_ext
  building 'snappy._snappy' extension
  creating build/temp.macosx-14-x86_64-cpython-311
  creating build/temp.macosx-14-x86_64-cpython-311/src
  creating build/temp.macosx-14-x86_64-cpython-311/src/snappy
  clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/include -I/usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/snappy/crc32c.c -o build/temp.macosx-14-x86_64-cpython-311/src/snappy/crc32c.o
  clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -I/private/var/folders/s1/ls1zsdv57yj_hxx0ldh9h6jh0000gn/T/tmp2shjyr97/.venv/include -I/usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c src/snappy/snappymodule.cc -o build/temp.macosx-14-x86_64-cpython-311/src/snappy/snappymodule.o
  src/snappy/snappymodule.cc:33:10: fatal error: 'snappy-c.h' file not found
  #include <snappy-c.h>
           ^~~~~~~~~~~~
  1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  

  at /usr/local/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with python-snappy (0.6.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "python-snappy (==0.6.1)"'.

I can manually cause python-snappy to be built with pip, outside of the pep517 / poetry infrastructure, with a workaround command such as the one in the README, e.g. for me:

CPPFLAGS="-I$(brew --prefix)/include -L$(brew --prefix)/lib" poetry run pip install python-snappy

This works but is really frustrating - I see there are quite a lot of MacOS-related build issues already, and am trying to avoid raising a duplicate.

Obviously I'd prefer that the build script just worked entirely, but even if I had to manually set CPPFLAGS it would be a big improvement if python-snappy were to support PEP 517 builds directly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions