Add CPU and memory information to web console#18613
Merged
FrankChen021 merged 20 commits intoapache:masterfrom Dec 8, 2025
Merged
Add CPU and memory information to web console#18613FrankChen021 merged 20 commits intoapache:masterfrom
FrankChen021 merged 20 commits intoapache:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds CPU and memory information to the web console by exposing these metrics through the sys.servers table. The new fields help understand resource availability for each server, which is particularly useful for Kubernetes deployments where each server runs in an isolated pod.
Key changes:
- Added
available_processorsandtotal_memoryfields to the sys.servers table and web console - Implemented container-aware resource detection using Java 17's OperatingSystemMXBean
- Updated web console to display CPU processors and total memory with proper formatting
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| processing/src/main/java/org/apache/druid/utils/JvmUtils.java | Added getTotalMemory() method using OperatingSystemMXBean for container-aware memory detection |
| server/src/main/java/org/apache/druid/discovery/DiscoveryDruidNode.java | Added availableProcessors and totalMemory fields with proper serialization and backwards compatibility |
| sql/src/main/java/org/apache/druid/sql/calcite/schema/SystemSchema.java | Extended sys.servers table schema to include new CPU and memory columns |
| web-console/src/views/services-view/services-view.tsx | Added CPU processors and total memory columns to services view with aggregation support |
| web-console/src/utils/general.tsx | Enhanced formatBytes function to support binary byte formatting |
| sql/src/test/java/org/apache/druid/sql/calcite/schema/SystemSchemaTest.java | Updated tests to include new fields in expected results |
| server/src/test/java/org/apache/druid/discovery/DiscoveryDruidNodeTest.java | Updated serialization tests to include new fields |
| docs/querying/sql-metadata-tables.md | Added documentation for new available_processors and total_memory columns |
| website/.spelling | Added new field names to spelling whitelist |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Member
FrankChen021
left a comment
There was a problem hiding this comment.
generally looks good, left some minor comments
FrankChen021
approved these changes
Oct 16, 2025
| public class DiscoveryDruidNode | ||
| { | ||
| private static final Logger LOG = new Logger(DiscoveryDruidNode.class); | ||
| private static final Integer UNKNOWN_VALUE = -1; |
Member
There was a problem hiding this comment.
Suggested change
| private static final Integer UNKNOWN_VALUE = -1; | |
| private static final int UNKNOWN_VALUE = -1; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds 2 new fields to the sys.servers table which is displayed on the web console. These will be stored on
DiscoveryDruidNodeBoth fields represent the available resource on the machine. If the service is hosted in a container, it will represent the resource available to the container instead. (Container Aware)
The reason for adding this feature is so that we are able to know the cost of running each of the servers. This is useful when deploying on Kubernetes as we have each server running on an isolated pod.
Interestingly, it seems that from Java 17, the
OPERATING_SYSTEM_MX_BEANis container aware which helps to simplify the implementation. It might be possible to simplify some legacy code which can be found in thecgroupsfolder i.e.Cpu.javaandMemory.java. I have not looked into detail but I feel that it is something worth looking into.https://developers.redhat.com/articles/2022/04/19/java-17-whats-new-openjdks-container-awareness#recent_changes_in_openjdk_s_container_awareness_code