feat: Support hierarchical(slash separated) date partitioning for Hudi tables #17787
+194
−37
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the issue this Pull Request addresses
This PR adds support for hierarchical date partitioning, allowing users to store date partition values in yyyy/MM/dd directory structure while specifying them as yyyy-mm-dd format. This provides a more intuitive folder hierarchy for date-based partitions.
Summary and Changelog
Users can now enable hierarchical date partitioning to automatically convert date values from
yyyy-mm-ddformat toyyyy/MM/dddirectory structure.Changes:
HIERARCHICAL_DATE_PARTITIONINGconfig property to enable date path transformationSimpleAvroKeyGenerator,BuiltinKeyGenerator) to support hierarchical date formattingTestSlashSeparatedPartitionValue.scalaImpact
Public API Changes:
hoodie.datasource.write.hierarchial.date.partitionpath(default:false)HoodieTableConfig.HIERARCHICAL_DATE_PARTITIONINGUser-facing Changes:
2026/01/05instead of2026-01-05)Risk Level
Low - Feature is opt-in via configuration flag. Existing tables are unaffected unless explicitly enabled. Added validation to prevent misconfiguration with hive-style partitioning.
Documentation Update
Config documentation needs to be updated to include:
hoodie.datasource.write.hierarchial.date.partitionpathconfiguration propertyContributor's checklist