-
Notifications
You must be signed in to change notification settings - Fork 25
feat: support scan metrics of FileStoreScan to align with ScanMetrics #89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Ping @lucasfang, @lxy-9602. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request adds scan metrics support to FileStoreScan to align with Java Paimon's ScanMetrics API. The implementation tracks key scanning operations metrics including scan duration, scanned snapshot ID, number of manifests scanned, and counts of skipped and resulted table files.
Changes:
- Added
ScanMetricsclass defining metric name constants for scan operations - Integrated metrics collection into
FileStoreScanwith a new metrics member and getter method - Modified
ReadManifeststo track both all manifests and filtered manifests for metrics calculation - Added test coverage for the scan metrics functionality
- Changed commit duration metric from nanoseconds to milliseconds for consistency
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/paimon/core/operation/metrics/scan_metrics.h | Defines constants for scan metric names (duration, snapshot ID, manifests count, file counts) |
| src/paimon/core/operation/file_store_scan.h | Adds metrics member variable, getter method, and updates ReadManifests signature |
| src/paimon/core/operation/file_store_scan.cpp | Implements metrics collection logic during scan plan creation and updates ReadManifests implementation |
| src/paimon/core/operation/key_value_file_store_scan_test.cpp | Adds basic test validation for scan metrics (snapshot ID metric) |
| src/paimon/core/operation/file_store_commit_impl.cpp | Changes commit duration metric unit from nanoseconds to milliseconds for consistency |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d66c6c7 to
5dc22e7
Compare
31706d3 to
32ec934
Compare
af6b879 to
20c5102
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
6f7f83a to
7b9e035
Compare
7b9e035 to
0ce48ff
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d8582c6 to
bab1188
Compare
bab1188 to
f00d8fd
Compare
lxy-9602
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Purpose
Linked issue: close #88
Support scan metrics of
FileStoreScanto align withScanMetricsincluding:lastScanDurationlastScannedSnapshotIdlastScannedManifestslastScanSkippedTableFileslastScanResultedTableFilesTests
KeyValueFileStoreScanTest, TestMaxSequenceNumber.API and Format
No.
Documentation
No.