[C++][Python] Remove deprecated functions in 20.0

[pitrou]

Describe the enhancement requested

We should review deprecated APIs and remove them if their deadline has been reached.

Component(s)

C++, Python

[pitrou]

cc @AlenkaF

[AlenkaF]

I have checked the C++ and Python parts of the codebase an found these deprecated APIs. I will start with the Python part. Would be great, in the meantime, to get some feedback on the C++ and Flight part in case I am off somewhere or I should split the work in multiple PRs for easier review.

C++

• GetNextSegment and related in grouper.cc from GH-44052 (Deprecated in 18.0.0)
• GetCapacity in Gandiva cache.cc from GH-41329(Deprecated in 17.0.0)
• BufferReader constructors in io/memory.h from GH-37212 (Deprecated in 14.0.0)
• ObjectType, FileStatistics in io/hdfs.h from ARROW-8224 (Deprecated in 0.17.0)
• OpenAppendStream in filesystem/gcsfs.h from ARROW-14222 (Deprecated in 6.0.0)
• Leftover from ARROW-15732 (Deprecated in 11.0.0)
  • owned_thread_pool_

C++ Deprecated in 19.0.0, need to wait

• GetRecordBatchReader in reader.h from GH-44808
• NewRowGroup with chunk_size parquet/arrow/writer.h from GH-45048

Python C++

• PARQUET_2_0 see [Python][C++] Fix deprecations when building pyarrow #45129 (comment)
• serialize/deserialize [Python] Remove no longer used serialize/deserialize PyArrow C++ code #43587
  • see GH-44063: [Python] Deprecate the no longer used serialize/deserialize Pyarrow C++ functions #44064

Python

• Python use_legacy_format in ipc writer

  arrow/python/pyarrow/ipc.py

  Lines 62 to 64 in c124bb5

  	use_legacy_format : bool, default None
  	Deprecated in favor of setting options. Cannot be provided with
  	options.

• Python use_legacy_dataset in ParquetDataset
  • Opened issue: [Python] Remove use_legacy_dataset from code base #44790
  • test
• PyExtensionType from [Python] Disable PyExtensionType autoload #38607
  • test

Flight connected

• Authenticate (Deprecated in 13.0.0)
• Flight UCX (Deprecated in 19.0.0)
• StartCall (Deprecated in 13.0.0)

[pitrou]

cc @zanmato1984 for GetNextSegment. Other C++ removals look fine to me.

[pitrou]

cc @lidavidm @zeroshade for Flight deprecations.

[zanmato1984]

The removal of GetNextSegment looks good to me. Thank you @AlenkaF for bringing this up!

[lidavidm]

Those removals look good to me, thanks.