ARROW-16721: [C++] Drop support for bundled Thrift < 0.13 #13292

kou · 2022-06-02T08:26:00Z

Our bundled Thrift is 0.16.0. Users can use other version but will not
use 0.13 or earlier.

This change also cleans FindThrift.cmake up.

This change also fixes build errors with -DARROW_HIVESERVER2=ON. We
may be able to remove cpp/src/arrow/dbi/hiveserver2/. It seems that
nobody uses it.

github-actions · 2022-06-02T08:26:23Z

https://issues.apache.org/jira/browse/ARROW-16721

github-actions · 2022-06-02T08:26:25Z

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

pitrou · 2022-06-02T08:28:54Z

cpp/src/arrow/dbi/hiveserver2/CMakeLists.txt

I think our hiveserver adapter doesn't even build currently, do we want to bother fixing it?

pitrou · 2022-06-02T08:32:25Z

There's some compatibility code lying around (look for PARQUET_THRIFT_USE_BOOST and FORCE_BOOST_SMART_PTR), we should probably remove it as well.

pitrou · 2022-06-02T08:32:46Z

We
may be able to remove cpp/src/arrow/dbi/hiveserver2/. It seems that
nobody uses it.

+1

kou · 2022-06-02T23:34:18Z

I've started a discussion for removing cpp/src/arrow/dbi/hiveserver2/:
[C++] Can we remove cpp/src/arrow/dbi/hiveserver2? https://lists.apache.org/thread/70qv1q9krx7ztk35tzxq8jp11vq5b5zt

kou · 2022-06-02T23:55:09Z

There's some compatibility code lying around (look for PARQUET_THRIFT_USE_BOOST and FORCE_BOOST_SMART_PTR), we should probably remove it as well.

Good catch!
I've removed them because they are for Thrift < 0.11 that isn't supported.

kou · 2022-06-02T23:56:34Z

I've started a discussion for removing cpp/src/arrow/dbi/hiveserver2/:
[C++] Can we remove cpp/src/arrow/dbi/hiveserver2? https://lists.apache.org/thread/70qv1q9krx7ztk35tzxq8jp11vq5b5zt

I'll remove cpp/src/arrow/dbi/hiveserver2/ in a follow-up pull request once we get a consensus of it.

pitrou · 2022-06-03T09:23:33Z

The R CI failures look related.

kou · 2022-06-06T05:03:30Z

I've fixed R failures but new R failures are happen. The new failures are also happen on master: https://github.com/apache/arrow/runs/6747296145?check_suite_focus=true

It seems that duckdb is updated to 0.3.4 from 0.3.2.
Cc: @nealrichardson

nealrichardson · 2022-06-06T11:54:01Z

I've fixed R failures but new R failures are happen. The new failures are also happen on master: https://github.com/apache/arrow/runs/6747296145?check_suite_focus=true

It seems that duckdb is updated to 0.3.4 from 0.3.2. Cc: @nealrichardson

Yeah I noticed that too. We'll figure out something today.

nealrichardson · 2022-06-06T20:41:41Z

#13323 is merged; rebase should make this go away

Our bundled Thrift is 0.16.0. Users can use other version but will not use 0.13 or earlier. This change also cleans FindThrift.cmake up. This change also fixes build errors with -DARROW_HIVESERVER2=ON. We may be able to remove cpp/src/arrow/dbi/hiveserver2/. It seems that nobody uses it.

kou · 2022-06-07T01:23:10Z

Thanks!

This is ready to merge.

pitrou · 2022-06-07T06:54:50Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

  set(ARROW_BOOST_REQUIRE_LIBRARY TRUE)
 elseif(ARROW_GANDIVA
-       OR (ARROW_WITH_THRIFT AND THRIFT_REQUIRES_BOOST)
+       OR ARROW_WITH_THRIFT


So boost is always required by Thrift nevertheless?

Yes. The condition was wrong. We needs Boost to use Thrift.

We need Thrift only when we use Parquet or arrow/dbi/hiveserver2/.
Parquet uses thrift/transport/TBufferTransports.h: https://github.com/apache/arrow/blob/master/cpp/src/parquet/thrift_internal.h#L43
thrift/transport/TBufferTransports.h uses Boost: https://github.com/apache/thrift/blob/master/lib/cpp/src/thrift/transport/TBufferTransports.h#L26

This is a follow-up of ARROW-16721/apache#13292.

This is a follow-up of ARROW-16721/#13292. Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>

…it.properties (#13578) Rename file that contains metadata about commit: From: git.properties To: gitarrowinfo.properties Main reason: git.properties is a generic name that could be use for third party libraries for a different purpose than inspect arrow metadata information (i.e: [SpringBoot](https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html#application-properties.core.spring.info.git.location) git.properties configuration). Current Arrow Java artifacts are using [git-commit-id-maven-plugin](https://github.com/git-commit-id/git-commit-id-maven-plugin) to generate a file `./git.properties` with commit metadata such as: ``` #Generated by Git-Commit-Id-Plugin #Wed Jun 08 11:43:58 UTC 2022 git.build.user.email= git.build.host=Mac-1654686060479.local git.dirty=true git.remote.origin.url=https\://github.com/apache/arrow git.closest.tag.name=apache-arrow-8.0.0 git.commit.id.describe-short=apache-arrow-3.0.0-3327-dirty git.commit.user.email=kou@clear-code.com git.commit.time=08.06.2022 @ 07\:51\:03 UTC git.commit.message.full=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\#13337)\n\nThis is a follow-up of ARROW-16721/\#13292.\n\nAuthored-by\: Sutou Kouhei <kou@clear-code.com>\nSigned-off-by\: KrisztiSz<szucs.krisztian@gmail.com> git.build.version=9.0.0.dev191 git.commit.message.short=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\#13337) git.commit.id.abbrev=5fe3a60 git.branch=5fe3a606f16b183cf706ae8d9a4adaff593c18a1 git.build.user.name= git.closest.tag.commit.count=11694 git.commit.id.describe=apache-arrow-3.0.0-3327-g5fe3a60-dirty git.commit.id=5fe3a606f16b183cf706ae8d9a4adaff593c18a1 git.tags= git.build.time=08.06.2022 @ 11\:43\:58 UTC git.commit.user.name=Sutou Kouhei ``` Artifacts affected: ``` ./plasma/target/arrow-plasma-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./tools/target/arrow-tools-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./tools/target/arrow-tools-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-unsafe/target/arrow-memory-unsafe-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-netty/target/arrow-memory-netty-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-core/target/arrow-memory-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./dataset/target/arrow-dataset-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-sql/target/flight-sql-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-grpc/target/flight-grpc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded-ext.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/jdbc/target/arrow-jdbc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/avro/target/arrow-avro-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/orc/target/arrow-orc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./algorithm/target/arrow-algorithm-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./compression/target/arrow-compression-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./format/target/arrow-format-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./gandiva/target/arrow-gandiva-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./performance/target/benchmarks.jar,gitarrowinfo.properties ./performance/target/arrow-performance-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./vector/target/arrow-vector-9.0.0-SNAPSHOT-shade-format-flatbuffers.jar,gitarrowinfo.properties ./vector/target/arrow-vector-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./c/target/arrow-c-data-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ``` Authored-by: david dali susanibar arce <davi.sarces@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>

…it.properties (apache#13578) Rename file that contains metadata about commit: From: git.properties To: gitarrowinfo.properties Main reason: git.properties is a generic name that could be use for third party libraries for a different purpose than inspect arrow metadata information (i.e: [SpringBoot](https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html#application-properties.core.spring.info.git.location) git.properties configuration). Current Arrow Java artifacts are using [git-commit-id-maven-plugin](https://github.com/git-commit-id/git-commit-id-maven-plugin) to generate a file `./git.properties` with commit metadata such as: ``` #Generated by Git-Commit-Id-Plugin #Wed Jun 08 11:43:58 UTC 2022 git.build.user.email= git.build.host=Mac-1654686060479.local git.dirty=true git.remote.origin.url=https\://github.com/apache/arrow git.closest.tag.name=apache-arrow-8.0.0 git.commit.id.describe-short=apache-arrow-3.0.0-3327-dirty git.commit.user.email=kou@clear-code.com git.commit.time=08.06.2022 @ 07\:51\:03 UTC git.commit.message.full=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\apache#13337)\n\nThis is a follow-up of ARROW-16721/\apache#13292.\n\nAuthored-by\: Sutou Kouhei <kou@clear-code.com>\nSigned-off-by\: KrisztiSz<szucs.krisztian@gmail.com> git.build.version=9.0.0.dev191 git.commit.message.short=ARROW-16785\: [Packaging][Linux] Add FindThrift.cmake (\apache#13337) git.commit.id.abbrev=5fe3a60 git.branch=5fe3a606f16b183cf706ae8d9a4adaff593c18a1 git.build.user.name= git.closest.tag.commit.count=11694 git.commit.id.describe=apache-arrow-3.0.0-3327-g5fe3a60-dirty git.commit.id=5fe3a606f16b183cf706ae8d9a4adaff593c18a1 git.tags= git.build.time=08.06.2022 @ 11\:43\:58 UTC git.commit.user.name=Sutou Kouhei ``` Artifacts affected: ``` ./plasma/target/arrow-plasma-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./tools/target/arrow-tools-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./tools/target/arrow-tools-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-unsafe/target/arrow-memory-unsafe-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-netty/target/arrow-memory-netty-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./memory/memory-core/target/arrow-memory-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./dataset/target/arrow-dataset-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-sql/target/flight-sql-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./flight/flight-integration-tests/target/flight-integration-tests-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-grpc/target/flight-grpc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-jar-with-dependencies.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT-shaded-ext.jar,gitarrowinfo.properties ./flight/flight-core/target/flight-core-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/jdbc/target/arrow-jdbc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/avro/target/arrow-avro-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./adapter/orc/target/arrow-orc-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./algorithm/target/arrow-algorithm-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./compression/target/arrow-compression-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./format/target/arrow-format-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./gandiva/target/arrow-gandiva-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./performance/target/benchmarks.jar,gitarrowinfo.properties ./performance/target/arrow-performance-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./vector/target/arrow-vector-9.0.0-SNAPSHOT-shade-format-flatbuffers.jar,gitarrowinfo.properties ./vector/target/arrow-vector-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ./c/target/arrow-c-data-9.0.0-SNAPSHOT.jar,gitarrowinfo.properties ``` Authored-by: david dali susanibar arce <davi.sarces@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>

github-actions bot added Component: C++ Component: Parquet labels Jun 2, 2022

kou force-pushed the ci-conan-parquet branch from c796815 to 05aa644 Compare June 2, 2022 08:27

pitrou reviewed Jun 2, 2022

View reviewed changes

kou added 4 commits June 7, 2022 08:44

Remove codes for Thrift < 0.11

07f047a

Add missing ARROW_USE_BOOST check

2677f6c

Thrift always requires Boost

974de3b

kou force-pushed the ci-conan-parquet branch from 7d01d61 to 974de3b Compare June 6, 2022 23:44

Fix a typo

9d1c083

pitrou reviewed Jun 7, 2022

View reviewed changes

pitrou approved these changes Jun 7, 2022

View reviewed changes

pitrou merged commit 5a2f198 into apache:master Jun 7, 2022

kou deleted the ci-conan-parquet branch June 7, 2022 07:30

kou added a commit to kou/arrow that referenced this pull request Jun 8, 2022

ARROW-16785: [Packaging][Linux] Add FindThrift.cmake

394b1a3

This is a follow-up of ARROW-16721/apache#13292.

kou mentioned this pull request Jun 8, 2022

ARROW-16785: [Packaging][Linux] Add FindThrift.cmake #13337

Merged

kszucs pushed a commit that referenced this pull request Jun 8, 2022

ARROW-16785: [Packaging][Linux] Add FindThrift.cmake (#13337)

5fe3a60

This is a follow-up of ARROW-16721/#13292. Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>

asfimport mentioned this pull request Jun 7, 2022

[C++] Drop support for bundled Thrift < 0.13 #20271

Closed

kou mentioned this pull request Feb 17, 2025

Add support for the arrow 19.0.1 package conan-io/conan-center-index#26623

Merged

3 tasks

ARROW-16721: [C++] Drop support for bundled Thrift < 0.13 #13292

ARROW-16721: [C++] Drop support for bundled Thrift < 0.13 #13292

Uh oh!

Conversation

kou commented Jun 2, 2022

Uh oh!

github-actions bot commented Jun 2, 2022

Uh oh!

github-actions bot commented Jun 2, 2022

Uh oh!

pitrou Jun 2, 2022

Choose a reason for hiding this comment

Uh oh!

pitrou commented Jun 2, 2022

Uh oh!

pitrou commented Jun 2, 2022

Uh oh!

kou commented Jun 2, 2022

Uh oh!

kou commented Jun 2, 2022

Uh oh!

kou commented Jun 2, 2022

Uh oh!

pitrou commented Jun 3, 2022

Uh oh!

kou commented Jun 6, 2022

Uh oh!

nealrichardson commented Jun 6, 2022

Uh oh!

nealrichardson commented Jun 6, 2022

Uh oh!

kou commented Jun 7, 2022

Uh oh!

pitrou Jun 7, 2022

Choose a reason for hiding this comment

Uh oh!

kou Jun 7, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants