HDDS-8270. Measure checkAccess latency for Ozone objects #4467
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aswinshakil
reviewed
Mar 28, 2023
Member
aswinshakil
left a comment
aswinshakil
left a comment
There was a problem hiding this comment.
Thanks for the patch @tanvipenumudy. LGTM!
neils-dev
previously requested changes
Apr 3, 2023
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMPerformanceMetrics.java
Outdated
Show resolved
Hide resolved
adoroszlai
reviewed
Apr 17, 2023
Contributor
adoroszlai
left a comment
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @tanvipenumudy for the patch, LGTM.
@neils-dev please take another look, I think the latest update addresses your concern.
adoroszlai
approved these changes
Apr 18, 2023
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Apr 20, 2023
* master: (440 commits) HDDS-8445. Move PlacementPolicy back to SCM (apache#4588) HDDS-8335. ReplicationManager: EC Mis and Under replication handlers should handle overloaded exceptions (apache#4593) HDDS-8355. Intermittent failure in TestOMRatisSnapshots#testInstallSnapshot (apache#4592) HDDS-8444. Increase timeout of CI build (apache#4586) HDDS-8446. Selective checks: handle change in ci.yaml (apache#4587) HDDS-8440. Ozone Manager crashed with ClassCastException when deleting FSO bucket. (apache#4582) HDDS-7309. Enable by default GRPC between S3G and OM (apache#3820) HDDS-8458. Mark TestBlockDeletion#testBlockDeletion as flaky HDDS-8385. Ozone can't process snapshot when service UID > 2097151 (apache#4580) HDDS-8424: Preserve legacy bucket getKeyInfo behavior (apache#4576) HDDS-8453. Mark TestDirectoryDeletingServiceWithFSO#testDirDeletedTableCleanUpForSnapshot as flaky HDDS-8137. [Snapshot] SnapDiff to use tombstone entries in SST files (apache#4376) HDDS-8270. Measure checkAccess latency for Ozone objects (apache#4467) HDDS-8109. Seperate Ratis and EC MisReplication Handling (apache#4577) HDDS-8429. Checkpoint is not closed properly in OMDBCheckpointServlet (apache#4575) HDDS-8253. Set ozone.metadata.dirs to temporary dir if not defined in S3 Gateway (apache#4455) HDDS-8400. Expose rocksdb last sequence number through metrics (apache#4557) HDDS-8333. ReplicationManager: Allow partial EC reconstruction if insufficient nodes available (apache#4579) HDDS-8147. Introduce latency metrics for S3 Gateway operations (apache#4383) HDDS-7908. Support OM Metadata operation Generator in `Ozone freon` (apache#4251) ...
Contributor
|
Thanks @tanvipenumudy for the changes. Thanks @adoroszlai for seeing the comments addressed and for merging this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Introducing a new OM performance metric for capturing the latency of the
checkAccessmethod in theOmMetadataReaderclass. This metric shall be crucial in measuring the time it takes for the security providers that extend support for Ozone ACLs to verify the access permissions granted for a given Ozone object.This metric could potentially serve as a valuable tool in identifying any performance bottlenecks and improving efficiency.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8270
How was this patch tested?
The patch has been tested over a cluster that has the Ozone services running to see what the metric looks like. A sample screenshot of the Prometheus UI has been attached (for reference) capturing
om_performance_metrics_check_access_latency_ns_avg_time(in nanoseconds):