Skip to content

ARROW-5082: [Python] Stop exporting copies of shared libraries in wheel #4577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
fsaintjacques wants to merge 1 commit into from
Closed

ARROW-5082: [Python] Stop exporting copies of shared libraries in wheel #4577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions python/manylinux1/build_arrow.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ PATH="$PATH:${CPYTHON_PATH}/bin" $PYTHON_INTERPRETER setup.py build_ext \
--bundle-boost \
--boost-namespace=arrow_boost
PATH="$PATH:${CPYTHON_PATH}/bin" $PYTHON_INTERPRETER setup.py bdist_wheel
# Source distribution is used for debian pyarrow packages.
Copy link
Member

@wesm wesm Jun 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any documentation about this?

Copy link
Member

@kou kou Jun 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pyarrow Debian package doesn't exist.
Source distribution is needed to use pyarrow with libarrow Debian package.

Copy link
Contributor Author

@fsaintjacques fsaintjacques Jun 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you care to explain, I'm missing some pieces. My comment was regarding uwe comment on zulip.

Copy link
Member

@pitrou pitrou Jun 17, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, I'm not sure why this needs to be here. Anyone can build a source distribution from scratch (as the name suggests, it just packages source code together).

Copy link
Member

@xhochy xhochy Jun 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gives us a reproducible and controlled environment to build a source tarball. You still need the right versions of setuptools amd friends to actually support Markdown in the description field of the package information.

PATH="$PATH:${CPYTHON_PATH}/bin" $PYTHON_INTERPRETER setup.py sdist

if [ -n "$UBUNTU_WHEELS" ]; then
Expand Down
5 changes: 0 additions & 5 deletions python/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -499,11 +499,6 @@ def _move_shared_libs_unix(build_prefix, build_lib, lib_name):
lib_filename = os.path.basename(libs[0])
shutil.move(pjoin(build_prefix, lib_filename),
pjoin(build_lib, 'pyarrow', lib_filename))
for lib in libs[1:]:
filename = os.path.basename(lib)
link_name = pjoin(build_lib, 'pyarrow', filename)
if not os.path.exists(link_name):
os.symlink(lib_filename, link_name)
Copy link
Member

@wesm wesm Jun 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user wants to link a C++ library to the libarrow that is bundled with wheels, will it still work if there is only libarrow.so.14 there?

Copy link
Member

@xhochy xhochy Jun 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tensorflow is doing the same. Downstream users might need to add something to their linker commands though.

Copy link
Contributor Author

@fsaintjacques fsaintjacques Jun 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note, when I extracted the tensorflow packages, I noted they used the versionless path.



# If the event of not running from a git clone (e.g. from a git archive
Expand Down