ARROW-8283: [Python] Limit FileSystemDataset constructor from fragments/paths, no filesystem interaction #7468
Conversation
fsaintjacques
left a comment
fsaintjacques
left a comment
There was a problem hiding this comment.
One minor comments with the interface with regards to schema. Otherwise this is a good cleanup and will remove confusion with factory parameters.
python/pyarrow/_dataset.pyx
Outdated
There was a problem hiding this comment.
Schema shouldn't be optional, add a test if the user doesn't pass a schema to see the result.
There was a problem hiding this comment.
Yes, it's not optional in practice: a bit below I check that the value is a Schema (and not None).
I did this here to give proper error messages when omitting one of those (there is some discussion about this in the original PR that added this: #6913 (comment)), because cython is otherwise generating very confusing error messages. I still need to create a minimal example to report to cython ..
But added a test for from_paths that covers this to ensure it is indeed checked.
jorisvandenbossche
force-pushed
the
ARROW-8283
branch
from
a7b3672 to
e16b33e
Compare
2 participants
No description provided.