ARROW-10658: [Python][Packaging] Wheel builds for Apple Silicon #10659
Conversation
| cmake \ | ||
| -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 \ |
There was a problem hiding this comment.
We don't need to use the CMAKE_SYSTEM_* variables.
| -DARROW_BUILD_STATIC=OFF \ | ||
| -DARROW_BUILD_TESTS=OFF \ | ||
| -DARROW_DATASET=${ARROW_DATASET} \ | ||
| -DARROW_DEPENDENCY_SOURCE="VCPKG" \ | ||
| -DARROW_DEPENDENCY_USE_SHARED=OFF \ | ||
| -DARROW_FLIGHT==${ARROW_FLIGHT} \ | ||
| -DARROW_FLIGHT=${ARROW_FLIGHT} \ |
There was a problem hiding this comment.
gRPC is not supported on apple silicon at the moment
kszucs
changed the title
cpp/thirdparty/versions.txt
Outdated
| @@ -42,7 +42,7 @@ ARROW_JEMALLOC_BUILD_VERSION=5.2.1 | |||
| ARROW_LZ4_BUILD_VERSION=v1.9.3 | |||
| # mimalloc 1.6.7 didn't build on Visual Studio 2015 | |||
| # https://github.com/microsoft/mimalloc/issues/353 | |||
| ARROW_MIMALLOC_BUILD_VERSION=v1.6.4 | |||
| ARROW_MIMALLOC_BUILD_VERSION=v1.7.2 | |||
There was a problem hiding this comment.
Need to check that it won't cause any performance regressions.
There was a problem hiding this comment.
As far as I can see that should only apply to 2.0.0 and 1.7.x should be safe (the commenter in microsoft/mimalloc#363 claims 1.7.0 doesn't affect them at least)
There was a problem hiding this comment.
And the problematic test is still there - so assuming tests pass we should be OK
kszucs
changed the title
|
@github-actions crossbow submit wheel-macos-* |
|
Revision: 0f094730b1e9f3e49077a8d746668251e2bb8ddf Submitted crossbow builds: ursacomputing/crossbow @ actions-585
|
|
@github-actions crossbow submit wheel-macos-* |
|
Revision: e1ab42b861b169217a02102c969c939f47695808 Submitted crossbow builds: ursacomputing/crossbow @ actions-586
|
ci/scripts/install_python.sh
Outdated
| versions=([3.6]=3.6.8 | ||
| [3.7]=3.7.9 | ||
| [3.8]=3.8.9 | ||
| [3.9]=3.9.6) |
There was a problem hiding this comment.
Seems like there is no universal2 installers for python < 3.9
|
@kou could you please give it a preliminary review? |
| ARROW_JEMALLOC: OFF | ||
| ARROW_SIMD_LEVEL: NONE | ||
| CC: "clang" | ||
| CMAKE_CXX_COMPILER_LAUNCHER: "ccache" |
There was a problem hiding this comment.
Do we need this?
I think that our CMake configuration detects ccache automatically.
There was a problem hiding this comment.
We need this to use ccache when compiling the vcpkg binaries.
| @@ -19,28 +19,29 @@ | |||
| {{ macros.github_header() }} | |||
|
|
|||
| env: | |||
| CC: "clang" | |||
| CXX: "clang++" | |||
There was a problem hiding this comment.
Could you keep this list in alphabetical order?
|
@github-actions crossbow submit -g wheel |
|
Revision: af0f71743fce3a9b0e4ce648b1c861a4cbc3d079 Submitted crossbow builds: ursacomputing/crossbow @ actions-587
|
kszucs
changed the title
kszucs
changed the title
|
@github-actions crossbow submit -g wheel |
|
Revision: 736739d28dcacc2256671660be0a6bbe8ed0008d Submitted crossbow builds: ursacomputing/crossbow @ actions-588
|
|
@github-actions crossbow submit -g nightly |
|
Revision: d29f104 Submitted crossbow builds: ursacomputing/crossbow @ actions-612
|
|
@github-actions crossbow submit wheel-macos-* |
|
Revision: bb3b170 Submitted crossbow builds: ursacomputing/crossbow @ actions-614
|
kszucs
left a comment
kszucs
left a comment
There was a problem hiding this comment.
+1, all the relevant builds are passing
|
It's great to see wheel builds for Apple Silicon. Any chance you may back port them to older versions? The snowflake python connector still relies on versions of pyarrow>=3.0.0,<3.1.0, and I've gone through quite a few backflips but haven't been able to get pyarrow installed from source. |
|
I'm not sure since:
Either way, could you please create a JIRA ticket so we can track this? |
|
Cool. I crated a JIRA: https://issues.apache.org/jira/browse/ARROW-13583 Snowflake is pretty popular, so I'm surprised nobody has asked yet. They're saying they can't/won't upgrade their pyarrow until the September timeframe. |
5 participants
The following compiles the 3.9 wheel on apple silicon though I'm getting a segfault on import despite that all binaries have the right arch and linkage/bundling seems correct. I wasn't able to produce a coredump yet.