Skip to content

Conversation

@SteNicholas
Copy link
Contributor

@SteNicholas SteNicholas commented Jan 28, 2026

Purpose

Linked issue: close #88

Support scan metrics of FileStoreScan to align with ScanMetrics including:

  • lastScanDuration
  • lastScannedSnapshotId
  • lastScannedManifests
  • lastScanSkippedTableFiles
  • lastScanResultedTableFiles

Tests

KeyValueFileStoreScanTest, TestMaxSequenceNumber.

API and Format

No.

Documentation

No.

Copilot AI review requested due to automatic review settings January 28, 2026 06:32
@SteNicholas
Copy link
Contributor Author

Ping @lucasfang, @lxy-9602.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds scan metrics support to FileStoreScan to align with Java Paimon's ScanMetrics API. The implementation tracks key scanning operations metrics including scan duration, scanned snapshot ID, number of manifests scanned, and counts of skipped and resulted table files.

Changes:

  • Added ScanMetrics class defining metric name constants for scan operations
  • Integrated metrics collection into FileStoreScan with a new metrics member and getter method
  • Modified ReadManifests to track both all manifests and filtered manifests for metrics calculation
  • Added test coverage for the scan metrics functionality
  • Changed commit duration metric from nanoseconds to milliseconds for consistency

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/paimon/core/operation/metrics/scan_metrics.h Defines constants for scan metric names (duration, snapshot ID, manifests count, file counts)
src/paimon/core/operation/file_store_scan.h Adds metrics member variable, getter method, and updates ReadManifests signature
src/paimon/core/operation/file_store_scan.cpp Implements metrics collection logic during scan plan creation and updates ReadManifests implementation
src/paimon/core/operation/key_value_file_store_scan_test.cpp Adds basic test validation for scan metrics (snapshot ID metric)
src/paimon/core/operation/file_store_commit_impl.cpp Changes commit duration metric unit from nanoseconds to milliseconds for consistency

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@SteNicholas SteNicholas force-pushed the PAIMON-88 branch 2 times, most recently from d8582c6 to bab1188 Compare January 28, 2026 12:49
Copy link
Collaborator

@lxy-9602 lxy-9602 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lszskye lszskye merged commit 321ba0c into alibaba:main Jan 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support scan metrics

3 participants