This repository was archived by the owner on Jan 24, 2024. It is now read-only.
Maintain a StatsLogger map to avoid creating new object every time #1021
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BewareMyPower
added
type/enhancement
BewareMyPower
changed the title
BewareMyPower
changed the title
Demogorgon314
approved these changes
Jan 19, 2022
BewareMyPower
added a commit
that referenced
this pull request
Jan 23, 2022
…1021) ### Motivation Currently when the metrics related to Kafka requests are updated, a `PrometheusStatsLogger` will be created each time a request arrived. ```java requestStats.getStatsLogger() .scopeLabel(KopServerStats.REQUEST_SCOPE, apiName) ``` The `StatsLogger#scopeLabel` method calls `scope` method: ```java public StatsLogger scope(String name) { return new PrometheusStatsLogger(provider, completeName(name), labels); } ``` The newly created `PrometheusStatsLogger` object is nearly the same with the caller itself except the name. The shared `provider` field is responsible to maintain the cache of Prometheus stats. Therefore, there is no need to create another object if the name keeps the same. This PR is to avoid the redundant small object creations of `PrometheusStatsLogger`. ### Modifications Maintain a `ConcurrentHashMap` for each channel. The key is the `ApiVersions` object that represents a Kafka request's type, the value is the associated `StatsLogger` object. In addition, since a `RequestStats` instance is associated with only one `StatsLogger` instance, this PR creates the `RequestStats` instance at the beginning and avoid creating the instance for each connection.
BewareMyPower
added a commit
that referenced
this pull request
Jan 23, 2022
…1021) ### Motivation Currently when the metrics related to Kafka requests are updated, a `PrometheusStatsLogger` will be created each time a request arrived. ```java requestStats.getStatsLogger() .scopeLabel(KopServerStats.REQUEST_SCOPE, apiName) ``` The `StatsLogger#scopeLabel` method calls `scope` method: ```java public StatsLogger scope(String name) { return new PrometheusStatsLogger(provider, completeName(name), labels); } ``` The newly created `PrometheusStatsLogger` object is nearly the same with the caller itself except the name. The shared `provider` field is responsible to maintain the cache of Prometheus stats. Therefore, there is no need to create another object if the name keeps the same. This PR is to avoid the redundant small object creations of `PrometheusStatsLogger`. ### Modifications Maintain a `ConcurrentHashMap` for each channel. The key is the `ApiVersions` object that represents a Kafka request's type, the value is the associated `StatsLogger` object. In addition, since a `RequestStats` instance is associated with only one `StatsLogger` instance, this PR creates the `RequestStats` instance at the beginning and avoid creating the instance for each connection.
BewareMyPower
added a commit
that referenced
this pull request
Feb 9, 2022
…1021) ### Motivation Currently when the metrics related to Kafka requests are updated, a `PrometheusStatsLogger` will be created each time a request arrived. ```java requestStats.getStatsLogger() .scopeLabel(KopServerStats.REQUEST_SCOPE, apiName) ``` The `StatsLogger#scopeLabel` method calls `scope` method: ```java public StatsLogger scope(String name) { return new PrometheusStatsLogger(provider, completeName(name), labels); } ``` The newly created `PrometheusStatsLogger` object is nearly the same with the caller itself except the name. The shared `provider` field is responsible to maintain the cache of Prometheus stats. Therefore, there is no need to create another object if the name keeps the same. This PR is to avoid the redundant small object creations of `PrometheusStatsLogger`. ### Modifications Maintain a `ConcurrentHashMap` for each channel. The key is the `ApiVersions` object that represents a Kafka request's type, the value is the associated `StatsLogger` object. In addition, since a `RequestStats` instance is associated with only one `StatsLogger` instance, this PR creates the `RequestStats` instance at the beginning and avoid creating the instance for each connection. (cherry picked from commit 0a47184)
eolivelli
pushed a commit
to eolivelli/kop
that referenced
this pull request
Feb 24, 2022
…treamnative#1021) Currently when the metrics related to Kafka requests are updated, a `PrometheusStatsLogger` will be created each time a request arrived. ```java requestStats.getStatsLogger() .scopeLabel(KopServerStats.REQUEST_SCOPE, apiName) ``` The `StatsLogger#scopeLabel` method calls `scope` method: ```java public StatsLogger scope(String name) { return new PrometheusStatsLogger(provider, completeName(name), labels); } ``` The newly created `PrometheusStatsLogger` object is nearly the same with the caller itself except the name. The shared `provider` field is responsible to maintain the cache of Prometheus stats. Therefore, there is no need to create another object if the name keeps the same. This PR is to avoid the redundant small object creations of `PrometheusStatsLogger`. Maintain a `ConcurrentHashMap` for each channel. The key is the `ApiVersions` object that represents a Kafka request's type, the value is the associated `StatsLogger` object. In addition, since a `RequestStats` instance is associated with only one `StatsLogger` instance, this PR creates the `RequestStats` instance at the beginning and avoid creating the instance for each connection. (cherry picked from commit 0a47184)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Currently when the metrics related to Kafka requests are updated, a
PrometheusStatsLoggerwill be created each time a request arrived.The
StatsLogger#scopeLabelmethod callsscopemethod:The newly created
PrometheusStatsLoggerobject is nearly the same with the caller itself except the name. The sharedproviderfield is responsible to maintain the cache of Prometheus stats.Therefore, there is no need to create another object if the name keeps the same. This PR is to avoid the redundant small object creations of
PrometheusStatsLogger.Modifications
Maintain a
ConcurrentHashMapfor each channel. The key is theApiVersionsobject that represents a Kafka request's type, the value is the associatedStatsLoggerobject.In addition, since a
RequestStatsinstance is associated with only oneStatsLoggerinstance, this PR creates theRequestStatsinstance at the beginning and avoid creating the instance for each connection.