GH-47234: [C++][Python] Add test for fill_null regression on Windows #47249

rok · 2025-07-31T14:25:44Z

Rationale for this change

Add regression test for an issue that surfaced on specific MSVC versions, presumably due to a compiler bug.

Possible upstream tickets are https://developercommunity.visualstudio.com/t/Code-optimization-bug-SIMD-std::transf/10912292 and https://developercommunity.visualstudio.com/t/SIMD-Code-optimization-bug/10945478.

What changes are included in this PR?

Additional tests to catch any further occurrence of this specific problem.

Are these changes tested?

Obviously yes.

Are there any user-facing changes?

No.

GitHub Issue: [C++][Python] pa.compute.fill_null produces unexpected result in arrow 21 on windows #47234

rok · 2025-07-31T14:26:00Z

@github-actions crossbow submit wheel-windows-*

rok · 2025-07-31T16:13:57Z

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

@amoeba following your repro I'm trying the same in cpp.

rok · 2025-08-01T10:07:56Z

@github-actions crossbow submit wheel-windows-*

github-actions · 2025-08-01T10:10:19Z

Revision: 2a5b093fe13beebfe5f690ba9221a4333e22b00d

Submitted crossbow builds: ursacomputing/crossbow @ actions-6614a425dd

Task	Status
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp39-cp39-amd64

python/pyarrow/tests/test_compute.py

pitrou · 2025-08-18T08:40:45Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

github-actions · 2025-08-18T08:42:55Z

Revision: 85a2071aae01eeda65449b35599c13ee5c3606f7

Submitted crossbow builds: ursacomputing/crossbow @ actions-53ecca2d5e

Task	Status
wheel-windows-cp313-cp313-amd64

pitrou · 2025-08-18T12:57:44Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

pitrou · 2025-08-18T13:48:07Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

github-actions · 2025-08-18T13:50:27Z

Revision: 61913132ee21b589a1f1baddbc6df9f05488ef52

Submitted crossbow builds: ursacomputing/crossbow @ actions-28a873a6f2

Task	Status
wheel-windows-cp313-cp313-amd64

pitrou · 2025-08-28T12:07:38Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

pitrou · 2025-08-28T12:07:59Z

(retrying a new build, who knows, perhaps something was fixed?)

github-actions · 2025-08-28T12:09:52Z

Revision: 61913132ee21b589a1f1baddbc6df9f05488ef52

Submitted crossbow builds: ursacomputing/crossbow @ actions-184d8e9469

Task	Status
wheel-windows-cp313-cp313-amd64

pitrou · 2025-09-08T14:13:34Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

pitrou · 2025-09-08T14:25:48Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

github-actions · 2025-09-08T14:28:00Z

Revision: a24190d14e4fe0966216efd33ed5e52ce1cc155d

Submitted crossbow builds: ursacomputing/crossbow @ actions-ffeb60e918

Task	Status
wheel-windows-cp313-cp313-amd64

WillAyd · 2025-09-08T14:32:38Z

Perhaps this requires #47455 for the cpp regression test? Currently, no cpp tests are running on Windows CI

pitrou · 2025-09-08T15:06:49Z

No, we're really running the Python tests here.

pitrou · 2025-09-15T13:24:38Z

@github-actions crossbow submit wheel-windows-cp313-cp313-amd64

pitrou · 2025-09-15T13:25:59Z

@github-actions crossbow submit -g cpp

github-actions · 2025-09-15T13:27:04Z

Revision: 289eea8

Submitted crossbow builds: ursacomputing/crossbow @ actions-2547cd8dd3

Task	Status
wheel-windows-cp313-cp313-amd64

github-actions · 2025-09-15T13:28:39Z

Revision: 289eea8

Submitted crossbow builds: ursacomputing/crossbow @ actions-f189892110

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

pitrou · 2025-09-15T15:16:15Z

@github-actions crossbow submit -g wheel

github-actions · 2025-09-15T15:18:40Z

⚠️ GitHub issue #47234 has been automatically assigned in GitHub to PR creator.

github-actions · 2025-09-15T15:19:20Z

Revision: 24e90dd

Submitted crossbow builds: ursacomputing/crossbow @ actions-2d68d9e6cc

Task	Status
python-sdist
wheel-macos-monterey-cp310-cp310-amd64
wheel-macos-monterey-cp310-cp310-arm64
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-macos-monterey-cp312-cp312-amd64
wheel-macos-monterey-cp312-cp312-arm64
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-macos-monterey-cp313-cp313t-amd64
wheel-macos-monterey-cp313-cp313t-arm64
wheel-manylinux-2-28-cp310-cp310-amd64
wheel-manylinux-2-28-cp310-cp310-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-manylinux-2-28-cp312-cp312-amd64
wheel-manylinux-2-28-cp312-cp312-arm64
wheel-manylinux-2-28-cp313-cp313-amd64
wheel-manylinux-2-28-cp313-cp313-arm64
wheel-manylinux-2-28-cp313-cp313t-amd64
wheel-manylinux-2-28-cp313-cp313t-arm64
wheel-musllinux-1-2-cp310-cp310-amd64
wheel-musllinux-1-2-cp310-cp310-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-musllinux-1-2-cp312-cp312-amd64
wheel-musllinux-1-2-cp312-cp312-arm64
wheel-musllinux-1-2-cp313-cp313-amd64
wheel-musllinux-1-2-cp313-cp313-arm64
wheel-musllinux-1-2-cp313-cp313t-amd64
wheel-musllinux-1-2-cp313-cp313t-arm64
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64

pitrou · 2025-09-15T15:19:33Z

Let's check that all wheel builds are ok now. If ok, then this PR will be mergeable.

pitrou · 2025-09-15T16:48:43Z

The failing wheel builds also failed on the nightlies. Will merge.

conbench-apache-arrow · 2025-09-15T23:48:40Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 4630313.

There weren't enough matching historic benchmark results to make a call on whether there were regressions.

The full Conbench report has more details.

…ndows (apache#47249) ### Rationale for this change Add regression test for an issue that surfaced on specific MSVC versions, presumably due to a compiler bug. Possible upstream tickets are https://developercommunity.visualstudio.com/t/Code-optimization-bug-SIMD-std::transf/10912292 and https://developercommunity.visualstudio.com/t/SIMD-Code-optimization-bug/10945478. ### What changes are included in this PR? Additional tests to catch any further occurrence of this specific problem. ### Are these changes tested? Obviously yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#47234 Lead-authored-by: Antoine Pitrou <antoine@python.org> Co-authored-by: Rok Mihevc <rok@mihevc.org> Co-authored-by: Antoine Pitrou <pitrou@free.fr> Signed-off-by: Antoine Pitrou <antoine@python.org>

github-actions bot added Component: Python awaiting committer review Awaiting committer review labels Jul 31, 2025

rok force-pushed the fill_null_repro branch from e588184 to bdd53a9 Compare July 31, 2025 14:27

github-actions bot added the Component: C++ label Jul 31, 2025

rok commented Jul 31, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Jul 31, 2025

pitrou reviewed Aug 18, 2025

View reviewed changes

python/pyarrow/tests/test_compute.py Outdated Show resolved Hide resolved

pitrou reviewed Aug 18, 2025

View reviewed changes

python/pyarrow/tests/test_compute.py Outdated Show resolved Hide resolved

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 18, 2025

This comment was marked as outdated.

Sign in to view

pitrou force-pushed the fill_null_repro branch from ca3a37b to 6191313 Compare August 18, 2025 13:47

pitrou mentioned this pull request Aug 18, 2025

[C++][Python] pa.compute.fill_null produces unexpected result in arrow 21 on windows #47234

Closed

pitrou force-pushed the fill_null_repro branch from 6191313 to 483d497 Compare September 8, 2025 14:13

This comment was marked as outdated.

Sign in to view

pitrou force-pushed the fill_null_repro branch from 483d497 to a24190d Compare September 8, 2025 14:25

pitrou force-pushed the fill_null_repro branch from a24190d to 289eea8 Compare September 15, 2025 13:24

This comment was marked as outdated.

Sign in to view

rok and others added 5 commits September 15, 2025 17:07

add regression test

432a41e

add cpp regression test

8f0d2e0

More insightful test vectors

aa5fbf9

Add low-level logs

b4d8b6f

Remove temporary changes

24e90dd

pitrou force-pushed the fill_null_repro branch from 289eea8 to 24e90dd Compare September 15, 2025 15:15

pitrou marked this pull request as ready for review September 15, 2025 15:19

pitrou requested review from AlenkaF and raulcd as code owners September 15, 2025 15:19

pitrou changed the title ~~GH-47234: [C++][Python] fill_null regression on windows~~ GH-47234: [C++][Python] Add test for fill_null regression on Windows Sep 15, 2025

pitrou merged commit 4630313 into apache:main Sep 15, 2025
42 of 44 checks passed

pitrou removed the awaiting change review Awaiting change review label Sep 15, 2025

GH-47234: [C++][Python] Add test for fill_null regression on Windows #47249

GH-47234: [C++][Python] Add test for fill_null regression on Windows #47249

Uh oh!

Conversation

rok commented Jul 31, 2025 • edited by pitrou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

rok commented Jul 31, 2025

Uh oh!

rok Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

rok commented Aug 1, 2025

Uh oh!

github-actions bot commented Aug 1, 2025

Uh oh!

Uh oh!

Uh oh!

pitrou commented Aug 18, 2025

Uh oh!

github-actions bot commented Aug 18, 2025

Uh oh!

pitrou commented Aug 18, 2025

Uh oh!

This comment was marked as outdated.

pitrou commented Aug 18, 2025

Uh oh!

github-actions bot commented Aug 18, 2025

Uh oh!

pitrou commented Aug 28, 2025

Uh oh!

pitrou commented Aug 28, 2025

Uh oh!

github-actions bot commented Aug 28, 2025

Uh oh!

pitrou commented Sep 8, 2025

Uh oh!

This comment was marked as outdated.

pitrou commented Sep 8, 2025

Uh oh!

github-actions bot commented Sep 8, 2025

Uh oh!

WillAyd commented Sep 8, 2025

Uh oh!

pitrou commented Sep 8, 2025

Uh oh!

pitrou commented Sep 15, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

pitrou commented Sep 15, 2025

Uh oh!

github-actions bot commented Sep 15, 2025

Uh oh!

github-actions bot commented Sep 15, 2025

Uh oh!

pitrou commented Sep 15, 2025

Uh oh!

github-actions bot commented Sep 15, 2025

Uh oh!

github-actions bot commented Sep 15, 2025

Uh oh!

pitrou commented Sep 15, 2025

Uh oh!

pitrou commented Sep 15, 2025

Uh oh!

Uh oh!

conbench-apache-arrow bot commented Sep 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

rok commented Jul 31, 2025 •

edited by pitrou

Loading