Skip to content

GH-45747: [C++] Remove deprecated ObjectType and FileStatistics, refactor hdfs code #45998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AlenkaF wants to merge 27 commits into
base: main
Choose a base branch
from
Open

GH-45747: [C++] Remove deprecated ObjectType and FileStatistics, refactor hdfs code #45998

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c2b3735
First remove only ObjectType
AlenkaF Mar 18, 2025
e59f0d2
Remove FileStatistics
AlenkaF Apr 1, 2025
938322a
Run linter
AlenkaF Apr 1, 2025
55321ad
Update tests and filesystem/hdfs.cc
AlenkaF Apr 1, 2025
3279a54
Move definitions from libarrow.pxd to libarrow_fs.pxd
AlenkaF Apr 1, 2025
7b0afca
Move hdfs code from io to filesystem
AlenkaF Apr 8, 2025
07f2aae
Fix a small mess after merge conflict resolve
AlenkaF Apr 8, 2025
65dac4a
Fix linter errors and revert changes in the test namespace
AlenkaF Apr 9, 2025
61c4063
Add docstrings to python have_libhdfs method
AlenkaF Apr 9, 2025
8dc8745
More linter fixes
AlenkaF Apr 9, 2025
a51a059
And Python linter
AlenkaF Apr 9, 2025
ed1b17f
Remove target_link_libraries for ARROW_IO
AlenkaF May 5, 2025
6009c5a
Rename hdfs_io_test
AlenkaF May 5, 2025
0433bc1
Remove hdfs_io include in filesystem/api.h
AlenkaF May 5, 2025
6621fc9
Merge internal hdfs classes
AlenkaF May 12, 2025
dc17708
Remove commented code
AlenkaF May 19, 2025
722c670
Fix linter error in CMakeLists
AlenkaF May 19, 2025
866903b
Add missing linter fix
AlenkaF May 19, 2025
acb844c
Make MSVC happy
AlenkaF May 19, 2025
f4d5118
Change friend Status Make* methods to static Result Make
AlenkaF May 19, 2025
10146e3
Try to remove the correct std::move
AlenkaF May 19, 2025
e3259bc
Define the static method on the outer class
AlenkaF May 20, 2025
48d0c76
Try to safeguard client_ from nullptr and dangling ptr
AlenkaF May 20, 2025
21b67cf
Update hdfs Impl
AlenkaF May 20, 2025
0e8ff01
Change io::internal to fs::internal
AlenkaF Jun 10, 2025
267a1e5
Run C++ linter
AlenkaF Jun 10, 2025
f09d6d9
Remove HdfsDriver from PyArrow Cython file
AlenkaF Jun 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions cpp/src/arrow/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,19 +408,11 @@ arrow_add_object_library(ARROW_IO
io/caching.cc
io/compressed.cc
io/file.cc
io/hdfs.cc
io/hdfs_internal.cc
io/interfaces.cc
io/memory.cc
io/slow.cc
io/stdio.cc
io/transform.cc)
foreach(ARROW_IO_TARGET ${ARROW_IO_TARGETS})
target_link_libraries(${ARROW_IO_TARGET} PRIVATE arrow::hadoop)
if(NOT MSVC)
target_link_libraries(${ARROW_IO_TARGET} PRIVATE ${CMAKE_DL_LIBS})
endif()
endforeach()

set(ARROW_MEMORY_POOL_SRCS memory_pool.cc)
if(ARROW_JEMALLOC)
Expand Down Expand Up @@ -906,7 +898,7 @@ if(ARROW_FILESYSTEM)
PROPERTIES SKIP_UNITY_BUILD_INCLUSION ON)
endif()
if(ARROW_HDFS)
list(APPEND ARROW_FILESYSTEM_SRCS filesystem/hdfs.cc)
list(APPEND ARROW_FILESYSTEM_SRCS filesystem/hdfs.cc filesystem/hdfs_internal.cc)
endif()
if(ARROW_S3)
list(APPEND ARROW_FILESYSTEM_SRCS filesystem/s3fs.cc)
Expand Down
8 changes: 8 additions & 0 deletions cpp/src/arrow/filesystem/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,12 @@ endif()

if(ARROW_HDFS)
add_arrow_test(hdfs_test EXTRA_LABELS filesystem)
add_arrow_test(hdfs_internal_test
NO_VALGRIND
EXTRA_LABELS
filesystem
EXTRA_LINK_LIBS
arrow::hadoop
Boost::filesystem
Boost::system)
endif()
Loading
Loading