[Python][Dataset] Write a custom field to _metadata caching file size

[asfimport]

#7547 (comment)

This will save unecessary HEAD requests when using S3

Reporter: Ben Kietzman / @bkietz

_{Note: This issue was originally created as ARROW-9227. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
The parquet FileMetadata don't directly include the file size, I think (https://parquet.apache.org/documentation/latest/#metadata, only the serialized size of the metadata itself, and each RowGroup has a total_byte_suze). So this would be custom metadata? And needs to be stored for each file of a partitioned dataset?

[asfimport]

Antoine Pitrou / @pitrou:
It's not obvious which kind of metadata field would store the info, and it would probably be non-standard.
Suggest closing.

[thisisnic]

This issue hasn't had activity in a long time. If it's still being worked on, please leave a comment. Otherwise, it will be closed on 23rd June.

Labelled Status: Stale-Warning for tracking.