Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions ci/conda_env_cpp_linux.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# The Conda packages for `google-cloud-cpp` lack Windows support:
# https://github.com/conda-forge/google-cloud-cpp-feedstock/issues/76
google-cloud-cpp>=1.34.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is google-cloud-cpp only supported on linux?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

google-cloud-cpp is supported on Linux, Windows and macOS. The Conda packages only work on Linux and macOS:

conda-forge/google-cloud-cpp-feedstock#76

If needed, I am happy to fix the Conda package first and then resubmit this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would certainly simplify this PR. Until that we can use the bundled google-cloud-cpp.


# Required by google-cloud-cpp, the Conda package is missing the dependency:
# https://github.com/conda-forge/google-cloud-cpp-feedstock/issues/28
nlohmann_json
11 changes: 8 additions & 3 deletions ci/docker/conda-cpp.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,26 +22,30 @@ FROM ${repo}:${arch}-conda
COPY ci/scripts/install_minio.sh /arrow/ci/scripts
RUN /arrow/ci/scripts/install_minio.sh latest /opt/conda

COPY ci/scripts/install_gcs_testbench.sh /arrow/ci/scripts
RUN /arrow/ci/scripts/install_gcs_testbench.sh default

# install the required conda packages into the test environment
COPY ci/conda_env_cpp.txt \
ci/conda_env_cpp_linux.txt \
ci/conda_env_gandiva.txt \
/arrow/ci/
RUN mamba install \
--file arrow/ci/conda_env_cpp.txt \
--file arrow/ci/conda_env_cpp_linux.txt \
--file arrow/ci/conda_env_gandiva.txt \
compilers \
doxygen \
valgrind && \
mamba clean --all

# We want to install the GCS testbench using the same Python binary that the Conda code will use.
COPY ci/scripts/install_gcs_testbench.sh /arrow/ci/scripts
RUN /arrow/ci/scripts/install_gcs_testbench.sh default

ENV ARROW_BUILD_TESTS=ON \
ARROW_DATASET=ON \
ARROW_DEPENDENCY_SOURCE=CONDA \
ARROW_FLIGHT=ON \
ARROW_GANDIVA=ON \
ARROW_GCS=ON \
ARROW_HOME=$CONDA_PREFIX \
ARROW_ORC=ON \
ARROW_PARQUET=ON \
Expand All @@ -54,6 +58,7 @@ ENV ARROW_BUILD_TESTS=ON \
ARROW_WITH_SNAPPY=ON \
ARROW_WITH_ZLIB=ON \
ARROW_WITH_ZSTD=ON \
CMAKE_CXX_STANDARD=17 \
GTest_SOURCE=BUNDLED \
PARQUET_BUILD_EXAMPLES=ON \
PARQUET_BUILD_EXECUTABLES=ON \
Expand Down
1 change: 1 addition & 0 deletions ci/scripts/cpp_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ cmake \
-DCMAKE_BUILD_TYPE=${ARROW_BUILD_TYPE:-debug} \
-DCMAKE_C_FLAGS="${CFLAGS:-}" \
-DCMAKE_CXX_FLAGS="${CXXFLAGS:-}" \
-DCMAKE_CXX_STANDARD="${CMAKE_CXX_STANDARD:-11}" \
-DCMAKE_INSTALL_LIBDIR=${CMAKE_INSTALL_LIBDIR:-lib} \
-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX:-${ARROW_HOME}} \
-DCMAKE_UNITY_BUILD=${CMAKE_UNITY_BUILD:-OFF} \
Expand Down
1 change: 1 addition & 0 deletions dev/tasks/conda-recipes/arrow-cpp/build-arrow.sh
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ cmake \
-DARROW_DEPENDENCY_SOURCE=SYSTEM \
-DARROW_FLIGHT=ON \
-DARROW_FLIGHT_REQUIRE_TLSCREDENTIALSOPTIONS=ON \
-DARROW_GCS=ON \
-DARROW_HDFS=ON \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The meta.yml must also contain the google-cloud-cpp host dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I suspect this may break things (see below).

-DARROW_JEMALLOC=ON \
-DARROW_MIMALLOC=ON \
Expand Down
1 change: 1 addition & 0 deletions dev/tasks/conda-recipes/arrow-cpp/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ outputs:
- c-ares
- gflags
- glog
- google-cloud-cpp >=1.34.0
- grpc-cpp
- libprotobuf
- clangdev 10 # [not (osx and arm64)]
Expand Down