Skip to content

Problematic separation between IndexerMetadataStorageCoordinator and MetadataSegmentManager abstractions #8187

@leventov

Description

@leventov

Several methods from IndexerMetadataStorageCoordinator, namely

  • getUsedSegmentsForInterval
  • getAllUsedSegments
  • getUsedSegmentsAndCreatedDates
  • getUsedSegmentsForIntervals
  • getveUnusedSegmentsForInterval

Duplicate the MetadataSegmentManager's functionality and look like much better suited for the latter class.

The conceptual difference is that MetadataSegmentManager caches the segments' data in memory, while IndexerMetadataStorageCoordinator always accesses the metadata store. This difference should be retained, but there should be comments to those methods, as well as parallel methods returning cached metadata with warnings and guidance regarding which method to use. For example, MaterializedViewSupervisor uses the methods from IndexerMetadataStorageCoordinator which seems unnecessary to me (could use the methods from MetadataSegmentManager instead).

If we decide that "cached metadata - direct access to metadata store" is the separation principle between IndexerMetadataStorageCoordinator and MetadataSegmentManager, then this fact should be reflected in these interfaces' names, Javadocs, and retrieveAllDataSourceNames() method should be moved from MetadataSegmentManager to IndexerMetadataStorageCoordinator.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions