Skip to content

Add MetadataDynamicCoder to support encode-decode for new fields in Metatdata #21341

@damccorm

Description

@damccorm

This Issue is to solve similar problems to wha I had in BEAM-12883.

Currently there are 2 Coders for Metadata: default one: org.apache.beam.sdk.io.fs.MetadataCoder and enhanced one org.apache.beam.sdk.io.fs.MetadataCoderV2, the last can also decode-encode lastModifiedMillis and it is done in a new coder in order to support backward compatibility.

This will be hard to maintain, we will need to create a new coder for any new field that will be added to Metadata.

So, as suggested in this comment: #15510, we need to have some new generic coder : MetadataDynamicCoder.

MetadataDynamicCoder can decode/encode any new fields added to Metadata by sending getter, setter and coder.

Imported from Jira BEAM-13640. Original Jira may contain additional context.
Reported by: brachi_packter.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions