ARROW-2461: [Python] Build manylinux2010 wheels #4675

pitrou · 2019-06-24T13:15:52Z

No description provided.

pitrou · 2019-06-24T13:16:25Z

@ursabot package build -g wheel

ursabot · 2019-06-24T13:16:26Z

No such command "package".

pitrou · 2019-06-24T13:16:47Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-24T13:17:04Z

AMD64 Conda Crossbow (#24577) builder has been succeeded.

Revision: b69df8192d6eb647ecd42db9a8d906749b7aa4b1

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-54

Task	Status
wheel-manylinux2010-cp37m
wheel-manylinux1-cp37m
wheel-manylinux2010-cp27mu
wheel-manylinux2010-cp36m
wheel-win-cp36m
wheel-manylinux2010-cp27m
wheel-manylinux2010-cp35m
wheel-win-cp35m
wheel-manylinux1-cp27mu
wheel-manylinux1-cp36m
wheel-osx-cp36m
wheel-osx-cp27m
wheel-manylinux1-cp35m
wheel-osx-cp37m
wheel-manylinux1-cp27m
wheel-win-cp37m
wheel-osx-cp35m

pitrou · 2019-06-24T14:19:56Z

@ursabot crossbow package -g wheel

pitrou · 2019-06-24T14:21:21Z

I pushed the image to quay.io.

ursabot · 2019-06-24T14:24:21Z

AMD64 Conda Crossbow (#24623) builder has been succeeded.

Revision: edd39a2a8ddc1b2aef2397fff0af817e639bb30f

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-55

Task	Status
wheel-win-cp35m
wheel-manylinux1-cp37m
wheel-manylinux2010-cp27mu
wheel-manylinux2010-cp35m
wheel-manylinux2010-cp27m
wheel-osx-cp36m
wheel-manylinux1-cp27m
wheel-osx-cp35m
wheel-manylinux1-cp27mu
wheel-win-cp36m
wheel-osx-cp37m
wheel-manylinux2010-cp36m
wheel-manylinux1-cp35m
wheel-osx-cp27m
wheel-manylinux2010-cp37m
wheel-manylinux1-cp36m
wheel-win-cp37m

dev/tasks/python-wheels/travis.manylinux2010.yml

codecov-io · 2019-06-24T16:02:35Z

Codecov Report

Merging #4675 into master will decrease coverage by 23.63%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #4675       +/-   ##
===========================================
- Coverage   88.39%   64.76%   -23.64%     
===========================================
  Files         905      482      -423     
  Lines      112429    62655    -49774     
  Branches     1420        0     -1420     
===========================================
- Hits        99385    40576    -58809     
- Misses      12682    22079     +9397     
+ Partials      362        0      -362

Impacted Files	Coverage Δ
cpp/src/arrow/util/memory.h	`0% <0%> (-100%)`	⬇️
cpp/src/gandiva/date_utils.h	`0% <0%> (-100%)`	⬇️
cpp/src/arrow/util/memory.cc	`0% <0%> (-100%)`	⬇️
cpp/src/arrow/filesystem/util-internal.cc	`0% <0%> (-100%)`	⬇️
cpp/src/arrow/util/sse-util.h	`0% <0%> (-100%)`	⬇️
cpp/src/gandiva/decimal_type_util.h	`0% <0%> (-100%)`	⬇️
cpp/src/arrow/compute/logical_type.h	`0% <0%> (-100%)`	⬇️
cpp/src/parquet/hasher.h	`0% <0%> (-100%)`	⬇️
cpp/src/gandiva/basic_decimal_scalar.h	`0% <0%> (-100%)`	⬇️
cpp/src/arrow/compute/kernels/boolean.cc	`0% <0%> (-100%)`	⬇️
... and 658 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 596faf3...ab5f833. Read the comment docs.

pitrou · 2019-06-24T16:30:29Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-24T16:30:42Z

AMD64 Conda Crossbow (#24719) builder failed.

Revision: bebc6abcdda09ccab9ff37707cf710d071b09949

Crossbow: 'python crossbow.py ...' step's stderr:

Traceback (most recent call last):
  File "crossbow.py", line 937, in <module>
    crossbow(obj={}, auto_envvar_prefix='CROSSBOW')
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "crossbow.py", line 770, in submit
    queue.put(job, prefix=job_prefix)
  File "crossbow.py", line 434, in put
    files = task.render_files(job=job, arrow=job.target)
  File "crossbow.py", line 563, in render_files
    template = Template(path.read_text(), undefined=StrictUndefined)
  File "/opt/conda/lib/python3.7/pathlib.py", line 1199, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/opt/conda/lib/python3.7/pathlib.py", line 1186, in open
    opener=self._opener)
  File "/opt/conda/lib/python3.7/pathlib.py", line 1039, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/buildbot/AMD64_Conda_Crossbow/arrow/dev/tasks/python-wheels/travis.manylinux2010.yml'

pitrou · 2019-06-24T16:33:35Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-24T16:40:33Z

AMD64 Conda Crossbow (#24737) builder has been succeeded.

Revision: b07ac217afb9f0b39d076861e47b0f68d8fe5df9

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-56

Task	Status
wheel-manylinux1-cp37m
wheel-manylinux2010-cp27m
wheel-manylinux2010-cp27mu
wheel-manylinux2010-cp35m
wheel-win-cp36m
wheel-osx-cp27m
wheel-osx-cp36m
wheel-manylinux1-cp27mu
wheel-osx-cp37m
wheel-manylinux1-cp27m
wheel-manylinux1-cp35m
wheel-osx-cp35m
wheel-manylinux1-cp36m
wheel-manylinux2010-cp37m
wheel-win-cp35m
wheel-win-cp37m
wheel-manylinux2010-cp36m

pitrou · 2019-06-24T16:59:22Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-24T17:03:28Z

AMD64 Conda Crossbow (#24779) builder has been succeeded.

Revision: ab5f8334561c404743ca4d56c9805782ec6ad18d

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-57

Task	Status
wheel-win-cp36m
wheel-manylinux2010-cp27m
wheel-win-cp35m
wheel-manylinux2010-cp36m
wheel-win-cp37m
wheel-osx-cp27m
wheel-manylinux2010-cp35m
wheel-manylinux2010-cp27mu
wheel-osx-cp36m
wheel-osx-cp35m
wheel-manylinux1-cp37m
wheel-manylinux1-cp35m
wheel-manylinux1-cp36m
wheel-manylinux1-cp27m
wheel-manylinux2010-cp37m
wheel-manylinux1-cp27mu
wheel-osx-cp37m

pitrou · 2019-06-24T17:35:00Z

Crossbow builds look basically ok, though obviously we'll have to wait for all of them to finish.

pitrou · 2019-06-25T10:06:11Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-25T10:10:44Z

AMD64 Conda Crossbow (#25374) builder has been succeeded.

Revision: cb5f285

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-64

Task	Status
wheel-manylinux1-cp27m
wheel-manylinux1-cp27mu
wheel-manylinux1-cp35m
wheel-manylinux1-cp36m
wheel-manylinux1-cp37m
wheel-manylinux2010-cp27m
wheel-manylinux2010-cp27mu
wheel-manylinux2010-cp35m
wheel-manylinux2010-cp36m
wheel-manylinux2010-cp37m
wheel-osx-cp27m
wheel-osx-cp35m
wheel-osx-cp36m
wheel-osx-cp37m
wheel-win-cp35m
wheel-win-cp36m
wheel-win-cp37m

python/manylinux2010/README.md

python/manylinux2010/build_arrow.sh

kszucs

Thanks for doing this @pitrou! Just a few nits, otherwise it looks good to me. The build scripts look almost identical, so we should have a minor priority issue about reusing them.

pitrou · 2019-06-25T11:06:22Z

@ursabot crossbow package -g wheel

ursabot · 2019-06-25T11:06:42Z

AMD64 Conda Crossbow (#25435) builder has been succeeded.

Revision: 1186b52

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-66

Task	Status
wheel-manylinux1-cp27m
wheel-manylinux1-cp27mu
wheel-manylinux1-cp35m
wheel-manylinux1-cp36m
wheel-manylinux1-cp37m
wheel-manylinux2010-cp27m
wheel-manylinux2010-cp27mu
wheel-manylinux2010-cp35m
wheel-manylinux2010-cp36m
wheel-manylinux2010-cp37m
wheel-osx-cp27m
wheel-osx-cp35m
wheel-osx-cp36m
wheel-osx-cp37m
wheel-win-cp35m
wheel-win-cp36m
wheel-win-cp37m

kszucs · 2019-06-25T12:01:42Z

@ursabot crossbow package wheel-osx-cp27m wheel-osx-cp35m wheel-osx-cp36m wheel-osx-cp37m

ursabot · 2019-06-25T12:04:09Z

AMD64 Conda Crossbow (#25489) builder has been succeeded.

Revision: 2f218d5

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-68

Task	Status
wheel-osx-cp27m
wheel-osx-cp35m
wheel-osx-cp36m
wheel-osx-cp37m

kszucs · 2019-06-25T12:09:22Z

~~I don't why does the badge report error while it is still running~~ (because they were restarted)

kszucs

LGTM, just waiting for the green wall.

kszucs · 2019-06-25T14:06:24Z

Green wall has arrived, artifacts are correctly uploaded too, merging once travis gets green. Thanks @pitrou!

kszucs · 2019-06-25T14:44:37Z

Travis error is unrelated, merging.

pitrou force-pushed the ARROW-2461-manylinux2010 branch from b69df81 to 2253887 Compare June 24, 2019 14:14

pitrou mentioned this pull request Jun 24, 2019

WIP: ARROW-2461: [Python] Build manylinux2010 wheels #4391

Closed

pitrou force-pushed the ARROW-2461-manylinux2010 branch 2 times, most recently from ddb8aee to edd39a2 Compare June 24, 2019 14:19

kszucs reviewed Jun 24, 2019

View reviewed changes

dev/tasks/python-wheels/travis.manylinux2010.yml Outdated Show resolved Hide resolved

pitrou force-pushed the ARROW-2461-manylinux2010 branch from bebc6ab to b07ac21 Compare June 24, 2019 16:31

pitrou force-pushed the ARROW-2461-manylinux2010 branch from b07ac21 to ab5f833 Compare June 24, 2019 16:59

pitrou added 2 commits June 25, 2019 12:05

ARROW-2461: [Python] Build manylinux2010 wheels

bcd0761

Attempt to factor out package scripts

cb5f285

pitrou force-pushed the ARROW-2461-manylinux2010 branch from ab5f833 to cb5f285 Compare June 25, 2019 10:05

kszucs reviewed Jun 25, 2019

View reviewed changes

python/manylinux2010/README.md Outdated Show resolved Hide resolved

kszucs reviewed Jun 25, 2019

View reviewed changes

python/manylinux2010/README.md Show resolved Hide resolved

kszucs reviewed Jun 25, 2019

View reviewed changes

python/manylinux2010/README.md Outdated Show resolved Hide resolved

kszucs reviewed Jun 25, 2019

View reviewed changes

python/manylinux2010/build_arrow.sh Show resolved Hide resolved

kszucs requested changes Jun 25, 2019

View reviewed changes

pitrou and others added 2 commits June 25, 2019 12:47

Nits

17a4c7f

use FETCH_HEAD in the CI templates

1186b52

pitrou requested a review from kszucs June 25, 2019 11:54

remove multibuild's clean_code function call

2f218d5

kszucs approved these changes Jun 25, 2019

View reviewed changes

kszucs mentioned this pull request Jun 25, 2019

ARROW-5690: [Packaging][Python] Fix macOS wheel building #4685

Closed

kszucs closed this in 0568544 Jun 25, 2019

pitrou deleted the ARROW-2461-manylinux2010 branch June 25, 2019 14:51

asfimport mentioned this pull request Jun 25, 2019

[Python] Build wheels for manylinux2010 tag #18630

Closed

ARROW-2461: [Python] Build manylinux2010 wheels #4675

ARROW-2461: [Python] Build manylinux2010 wheels #4675

Uh oh!

Conversation

pitrou commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

Uh oh!

codecov-io commented Jun 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

ursabot commented Jun 24, 2019

Uh oh!

pitrou commented Jun 24, 2019

Uh oh!

pitrou commented Jun 25, 2019

Uh oh!

ursabot commented Jun 25, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kszucs left a comment

Choose a reason for hiding this comment

Uh oh!

pitrou commented Jun 25, 2019

Uh oh!

ursabot commented Jun 25, 2019

Uh oh!

kszucs commented Jun 25, 2019

Uh oh!

ursabot commented Jun 25, 2019

Uh oh!

kszucs commented Jun 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kszucs left a comment

Choose a reason for hiding this comment

Uh oh!

kszucs commented Jun 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kszucs commented Jun 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov-io commented Jun 24, 2019 •

edited

Loading

kszucs commented Jun 25, 2019 •

edited

Loading

kszucs commented Jun 25, 2019 •

edited

Loading