ARROW-15101: [Python] Fix build failure on CSV writer #11950

pitrou · 2021-12-14T16:50:35Z

On some combination of configuration options, I would get the following errors:

/home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:16716:70: error: no member named 'WriteCSV' in namespace 'arrow::csv'
          __pyx_t_4 = __pyx_f_7pyarrow_3lib_check_status(arrow::csv::WriteCSV((*__pyx_v_batch), __pyx_v_c_write_options, __pyx_v_stream.get())); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(0, 1040, __pyx_L5_error)
                                                         ~~~~~~~~~~~~^
/home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:16797:70: error: no member named 'WriteCSV' in namespace 'arrow::csv'
          __pyx_t_4 = __pyx_f_7pyarrow_3lib_check_status(arrow::csv::WriteCSV((*__pyx_v_table), __pyx_v_c_write_options, __pyx_v_stream.get())); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(0, 1044, __pyx_L8_error)
                                                         ~~~~~~~~~~~~^
/home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:17089:109: error: no member named 'MakeCSVWriter' in namespace 'arrow::csv'
        __pyx_t_2 = arrow::py::GetResultValue<std::shared_ptr< arrow::ipc::RecordBatchWriter> >(arrow::csv::MakeCSVWriter(__pyx_v_c_stream, __pyx_v_c_schema, __pyx_v_c_write_options)); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(0, 1076, __pyx_L4_error)
                                                                                                ~~~~~~~~~~~~^
3 errors generated.

On some combination of configuration options, I would get the following errors: ``` /home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:16716:70: error: no member named 'WriteCSV' in namespace 'arrow::csv' __pyx_t_4 = __pyx_f_7pyarrow_3lib_check_status(arrow::csv::WriteCSV((*__pyx_v_batch), __pyx_v_c_write_options, __pyx_v_stream.get())); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(0, 1040, __pyx_L5_error) ~~~~~~~~~~~~^ /home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:16797:70: error: no member named 'WriteCSV' in namespace 'arrow::csv' __pyx_t_4 = __pyx_f_7pyarrow_3lib_check_status(arrow::csv::WriteCSV((*__pyx_v_table), __pyx_v_c_write_options, __pyx_v_stream.get())); if (unlikely(__pyx_t_4 == ((int)-1))) __PYX_ERR(0, 1044, __pyx_L8_error) ~~~~~~~~~~~~^ /home/antoine/arrow/dev/python/build/temp.linux-x86_64-3.9/_csv.cpp:17089:109: error: no member named 'MakeCSVWriter' in namespace 'arrow::csv' __pyx_t_2 = arrow::py::GetResultValue<std::shared_ptr< arrow::ipc::RecordBatchWriter> >(arrow::csv::MakeCSVWriter(__pyx_v_c_stream, __pyx_v_c_schema, __pyx_v_c_write_options)); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(0, 1076, __pyx_L4_error) ~~~~~~~~~~~~^ 3 errors generated. ```

github-actions · 2021-12-14T17:02:13Z

https://issues.apache.org/jira/browse/ARROW-15101

github-actions · 2021-12-14T17:02:14Z

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

westonpace

LGTM. Scanning through references to the macros in config.h it seems we might also be missing (from an IWYU perspective) a #include "arrow/util/config.h" in arrow/util/basic_decimal.cc (it needs ARROW_USE_NATIVE_INT128 and gets it indirectly via arrow/util/int128_internal.h)

pitrou · 2021-12-14T18:08:47Z

Nice catch, I will add it there too.

ursabot · 2021-12-14T19:00:56Z

Benchmark runs are scheduled for baseline = 0f41ed4 and contender = 994074d. 994074d is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.45% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.13% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

bkmgit · 2021-12-15T05:55:59Z

@github-actions crossbow submit -g conda-win-vs2017-py36-r40

bkmgit · 2021-12-15T05:56:23Z

@github-actions crossbow submit -g test-r-arrow-backwards-compatibility

github-actions · 2021-12-15T05:56:54Z

Invalid group(s) {'conda-win-vs2017-py36-r40'}. Must be one of {'packaging', 'conda', 'linux-amd64', 'cpp', 'verify-rc-jars', 'verify-rc-wheels', 'fuzz', 'nightly', 'verify-rc-source-macos', 'test', 'verify-rc', 'verify-rc-binaries', 'homebrew', 'python', 'wheel', 'r', 'ruby', 'linux-arm64', 'c-glib', 'example', 'verify-rc-source', 'integration', 'vcpkg', 'example-cpp', 'linux', 'verify-rc-source-linux'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/1581372147```

bkmgit · 2021-12-15T05:56:56Z

@github-actions crossbow submit -g test-r-linux-as-cran

github-actions · 2021-12-15T05:57:18Z

Invalid group(s) {'test-r-arrow-backwards-compatibility'}. Must be one of {'linux-amd64', 'packaging', 'nightly', 'vcpkg', 'wheel', 'integration', 'verify-rc-source-macos', 'verify-rc-binaries', 'verify-rc', 'test', 'verify-rc-jars', 'linux-arm64', 'cpp', 'homebrew', 'ruby', 'r', 'verify-rc-wheels', 'verify-rc-source', 'linux', 'conda', 'python', 'verify-rc-source-linux', 'fuzz', 'example-cpp', 'example', 'c-glib'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/1581373165```

github-actions · 2021-12-15T05:57:46Z

Invalid group(s) {'test-r-linux-as-cran'}. Must be one of {'linux', 'packaging', 'homebrew', 'wheel', 'python', 'linux-arm64', 'verify-rc-binaries', 'verify-rc', 'vcpkg', 'verify-rc-source-macos', 'verify-rc-wheels', 'verify-rc-source', 'c-glib', 'linux-amd64', 'cpp', 'conda', 'integration', 'fuzz', 'r', 'verify-rc-jars', 'verify-rc-source-linux', 'example-cpp', 'test', 'ruby', 'nightly', 'example'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/1581374182```

bkmgit · 2021-12-15T05:58:20Z

@github-actions crossbow submit test-r-linux-as-cran

kou · 2021-12-15T05:58:20Z

@bkmgit -g is needless.

github-actions · 2021-12-15T05:59:32Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1301

Task	Status
test-r-linux-as-cran

bkmgit · 2021-12-15T05:59:35Z

@github-actions crossbow submit test-r-arrow-backwards-compatibility

bkmgit · 2021-12-15T06:00:07Z

@github-actions crossbow submit conda-win-vs2017-py36-r40

github-actions · 2021-12-15T06:00:41Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1302

Task	Status
test-r-arrow-backwards-compatibility

github-actions · 2021-12-15T06:00:58Z

Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-1302-azure-conda-win-vs2017-py36-r40', 'refs/tags/actions-1302-azure-conda-win-vs2017-py36-r40', 'refs/heads/actions-1302']
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/1581381068```

bkmgit · 2021-12-15T06:01:11Z

@github-actions crossbow submit test-r-versions

bkmgit · 2021-12-15T06:02:03Z

@github-actions crossbow submit test-ubuntu-18.04-cpp-release

github-actions · 2021-12-15T06:02:21Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1303

Task	Status
test-r-versions

github-actions · 2021-12-15T06:03:21Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1304

Task	Status
test-ubuntu-18.04-cpp-release

bkmgit · 2021-12-15T06:26:25Z

@github-actions crossbow submit test-fedora-33-cpp

github-actions · 2021-12-15T06:27:27Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1305

Task	Status
test-fedora-33-cpp

bkmgit · 2021-12-15T06:27:31Z

@github-actions crossbow submit conda-linux-gcc-py36-cuda

github-actions · 2021-12-15T06:28:29Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1306

Task	Status
conda-linux-gcc-py36-cuda

bkmgit · 2021-12-15T06:30:09Z

@github-actions crossbow submit conda-win-vs2017-py36-r40

github-actions · 2021-12-15T06:31:13Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1307

Task	Status
conda-win-vs2017-py36-r40

bkmgit · 2021-12-15T06:39:27Z

@github-actions crossbow submit test-conda-cpp-valgrind

github-actions · 2021-12-15T06:40:29Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1308

Task	Status
test-conda-cpp-valgrind

bkmgit · 2021-12-15T07:34:09Z

@github-actions crossbow submit conda-linux-gcc-py36-cpu-r40

github-actions · 2021-12-15T07:35:15Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1309

Task	Status
conda-linux-gcc-py36-cpu-r40

bkmgit · 2021-12-15T07:49:51Z

@github-actions crossbow submit test-build-vcpkg-win

github-actions · 2021-12-15T07:51:06Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1310

Task	Status
test-build-vcpkg-win

bkmgit · 2021-12-15T08:21:37Z

@github-actions crossbow submit test-ubuntu-18.04-r-sanitizer

github-actions · 2021-12-15T08:23:09Z

Revision: 97a9a4f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1311

Task	Status
test-ubuntu-18.04-r-sanitizer

pitrou requested a review from westonpace December 14, 2021 16:50

github-actions bot added Component: C++ Component: Parquet labels Dec 14, 2021

westonpace approved these changes Dec 14, 2021

View reviewed changes

Make sure arrow/util/config.h is included where needed.

97a9a4f

pitrou closed this in 994074d Dec 14, 2021

pitrou deleted the ARROW-15101-csv-writer branch December 14, 2021 18:59

bkmgit mentioned this pull request Dec 15, 2021

ARROW-14229: [C++] Bump versions of bundled dependencies #11494

Closed

ARROW-15101: [Python] Fix build failure on CSV writer #11950

ARROW-15101: [Python] Fix build failure on CSV writer #11950

Uh oh!

Conversation

pitrou commented Dec 14, 2021

Uh oh!

github-actions bot commented Dec 14, 2021

Uh oh!

github-actions bot commented Dec 14, 2021

Uh oh!

westonpace left a comment

Choose a reason for hiding this comment

Uh oh!

pitrou commented Dec 14, 2021

Uh oh!

ursabot commented Dec 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

kou commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

bkmgit commented Dec 15, 2021

Uh oh!

github-actions bot commented Dec 15, 2021

Uh oh!

ursabot commented Dec 14, 2021 •

edited

Loading