[Python] Failed to install on aarch64

[asfimport]

My team is attempting to migrate some workloads from x86-64 to ARM64, a blocker for this is PyArrow failing to install. pip install pyarrow fails to build the wheel as -march isn't correctly resolved:

 -- System processor: aarch64
 -- Performing Test CXX_SUPPORTS_ARMV8_ARCH
 -- Performing Test CXX_SUPPORTS_ARMV8_ARCH - Failed
 -- Arrow build warning level: PRODUCTION
 CMake Error at cmake_modules/SetupCxxFlags.cmake:338 (message):
 Unsupported arch flag: -march=.

It's possible to get the build to work after editing cmake_modules/SetupCxxFlags.cmake to force ARROW_ARMV8_ARCH_FLAG to end up as an architecture such as 'armv8-a' - although some more elaborate logic is really needed to pick up the correct extensions.

I can see that there  have been a number of items discussed in the past both on Jira and in GitHub issues ranging from simple fixes to the cmake script to more elaborate fixes cross-product for arch detection - but I wasn't able to discern how the project wishes to proceed.

With AWS pushing their ARM-based instances heavily at this point I would advocate for picking a direction before an influx of new issues.

 

Environment: AWS m6g (ARM64 'Graviton2' CPU):
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part : 0xd0c
CPU revision : 1

OS: Linux version 5.3.0-1032-aws (buildd@bos02-arm64-053) (gcc version 7.5.0 (Ubuntu/Linaro 7.5.0-3ubuntu118.04)) #3418.04.2-Ubuntu SMP Fri Jul 24 10:03:03 UTC 2020
Reporter: Matthew Meen
Assignee: Kouhei Sutou / @kou

Original Issue Attachments:

• cmake-info.txt
• pyarrow_017.txt

PRs and other links:

• GitHub Pull Request #7982

_{Note: This issue was originally created as ARROW-9744. Please see the migration documentation for further details.}

[asfimport]

Kouhei Sutou / @kou:
Could you show your CMAKE_SYSTEM_PROCESSOR?
It will be able show by adding message("CMAKE_SYSTEM_PROCESSOR: ${CMAKE_SYSTEM_PROCESSOR}") into cmake_modules/SetupCxxFlags.cmake.

[asfimport]

Matthew Meen:
Sure, it's set to:

CMAKE_SYSTEM_PROCESSOR "aarch64"

I've attached the output of cmake --system-information too.

cmake-info.txt

[asfimport]

Kouhei Sutou / @kou:
Thanks and sorry. The information is already provided in the description...

Could you also show full log when you run pip install pyarrow?

And could you try 1.0.0?

[asfimport]

Matthew Meen:
There's isn't a 1.0.0 .tar.gz on the https://pypi.org/simple/pyarrow/, so this fails to find the package:

(test_env) ubuntu@ip-10-143-19-162:/usr/local/test_env$ sudo pip install pyarrow==1.0.0
ERROR: Could not find a version that satisfies the requirement pyarrow==1.0.0 (from versions: 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.12.1, 0.13.0, 0.14.0, 0.15.1, 0.16.0, 0.17.0, 0.17.1)
ERROR: No matching distribution found for pyarrow==1.0.0

This is the full output for pip install pyarrow, which finds the latest as 0.17.1: pyarrow_017.txt

I'll try cloning and building 1.0.0 directly shortly.

[asfimport]

Matthew Meen:
1.0.0 behaves the same as 0.17.1 where the value to cmake's -march argument ends up as a blank value.

[asfimport]

Kouhei Sutou / @kou:
@kszucs It seems that we forgot to release source package to PyPI. Could you upload it? (If you prefer, I can do it.)

[asfimport]

Krisztian Szucs / @kszucs:
Ouch, I'm uploading it.

[asfimport]

Kouhei Sutou / @kou:
Thanks!

[asfimport]

Krisztian Szucs / @kszucs:
It is available now on PyPI https://pypi.org/project/pyarrow/#files

[asfimport]

Eamonn Nugent:
Hiya. Just wanted to tune in about pip install pyarrow with 1.0 on an ARMv8 instance (AWS m6g.medium). It seems to have the same error:

CMake Error at cmake_modules/SetupCxxFlags.cmake:368 (message):
 Unsupported arch flag: -march=.
Call Stack (most recent call first):
 CMakeLists.txt:100 (include)

 

Is there a good workaround for this? Or should I wait for the next release

[asfimport]

Kouhei Sutou / @kou:
Ah, I got it.
pyarrow uses only SetupCxxFlags.cmake. It doesn't use DefineOptions.cmake.

I'll create a pull request to fix this.

Workaround: PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow

[asfimport]

Eamonn Nugent:
Just tuning back in. Tried out the workaround, and received this:

 

-- Looking for python3.8
 -- Found Python lib /usr/local/lib/libpython3.8.so
 -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29")
 -- Could NOT find Arrow (missing: Arrow_DIR)
 -- Checking for module 'arrow'
 -- No package 'arrow' found
 CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
 Could NOT find Arrow (missing: ARROW_INCLUDE_DIR ARROW_LIB_DIR
 ARROW_FULL_SO_VERSION ARROW_SO_VERSION)
 Call Stack (most recent call first):
 /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
 cmake_modules/FindArrow.cmake:412 (find_package_handle_standard_args)
 cmake_modules/FindArrowPython.cmake:46 (find_package)
 CMakeLists.txt:210 (find_package)

 -- Configuring incomplete, errors occurred!
 See also "/tmp/pip-install-av0q_7o5/pyarrow/build/temp.linux-aarch64-3.8/CMakeFiles/CMakeOutput.log".
 error: command 'cmake' failed with exit status 1
 ----------------------------------------
 ERROR: Failed building wheel for pyarrow
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly
The command '/bin/sh -c PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow' returned a non-zero code: 1

 

Failing Dockerfile on an ARMv8 system:

FROM python:3.8-buster
RUN apt update
RUN apt -y install gcc g++ cmake
RUN PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow

 

 

If there's anything I can do to help debug, please, feel free to let me know.

 

[asfimport]

Kouhei Sutou / @kou:
You need to install Apache Arrow C++ separately or build it while building pyarrow.

You can do the latter by PYARROW_BUNDLE_ARROW_CPP=1 PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow.

[asfimport]

Matthew Meen:
Just to confirm, the workaround works as long as the C++ side is already built. The build it while building pyarrow option doesn't work for me with the same error that Eamon received.

[~space55] the instructions on https://arrow.apache.org/docs/developers/python.html#building-on-linux-and-macos to build the C++ are straightforward enough. The built .whl can be installed either directly using pip, or if you have a private PyPi then drop it there to avoid copying it manually.

@kou - thanks for the quick turnaround on this, hopefully it makes it into a release build soon.

[asfimport]

Kouhei Sutou / @kou:
Let's handle PYARROW_BUNDLE_ARROW_CPP=1 case in ARROW-9791.