Get Node Pool Information for Remote Models

[ryanweiler92]

Description

Adds a reactor that fetches data about the deployed models in the node pools from the Kubernetes model scaler.

Changes Made

• Adding GetNodePoolsInfoReactor
• Adding a method to the KubernetesModelScaler class to fetch the data

How to Test

GetNodePoolsInfo()

Note: Requires KMS ingress env variable and admin mode.

Notes

Example return:

{"pools": [{"machine_type": "c3-standard-8", "models": [{"node": "gke-cfgai-standard-n-cpu-sapphire-poo-cb209854-33pn", "name": "dinov2-large", "namespace": "huggingface-models", "resources": {"memory_requests_gi": 8.01, "gpu_requests": 0, "cpu_requests": 2.01}, "url": "http://dinov2-large-predictor.huggingface-models.svc.cluster.local", "status": "True"}], "memory_total_gi": 32, "gpu_count": 0, "gpu_type": {}, "name": "cpu-sapphire-pool", "memory_available_gi": 18.84, "cpu_total": 8, "cpu_available": 5.47, "gpu_available": 0}, {"machine_type": "n1-standard-16", "models": [{"node": "gke-cfgai-standard-new-t4-gpu-pool-c1b8a6fd-03n2", "name": "florence-2-large", "namespace": "huggingface-models", "resources": {"memory_requests_gi": 10.01, "gpu_requests": 1, "cpu_requests": 1.01}, "url": "http://florence-2-large-predictor.huggingface-models.svc.cluster.local", "status": "True"}, {"node": "gke-cfgai-standard-new-t4-gpu-pool-c1b8a6fd-kkz6", "name": "stable-diffusion-v1-5", "namespace": "huggingface-models", "resources": {"memory_requests_gi": 10, "gpu_requests": 1, "cpu_requests": 1}, "url": "http://stable-diffusion-v1-5-predictor.huggingface-models.svc.cluster.local", "status": "True"}], "memory_total_gi": 240, "gpu_count": 1, "gpu_type": "nvidia.com/gpu", "name": "t4-gpu-pool", "memory_available_gi": 190.52, "cpu_total": 64, "cpu_available": 59.72, "gpu_available": 2}, {"machine_type": "g2-standard-12", "models": [], "memory_total_gi": 48, "gpu_count": 1, "gpu_type": "nvidia.com/gpu", "name": "l4-gpu-pool", "memory_available_gi": 41.58, "cpu_total": 12, "cpu_available": 11.46, "gpu_available": 1}], "message": "Successfully retrieved node pool information"}

[github-actions]

@CodiumAI-Agent /describe

[QodoAI-Agent]

Title

Get Node Pool Information for Remote Models

---

User description

Description

Adds a reactor that fetches data about the deployed models in the node pools from the Kubernetes model scaler.

Changes Made

• Adding GetNodePoolsInfoReactor
• Adding a method to the KubernetesModelScaler class to fetch the data

How to Test

GetNodePoolsInfo()

Note: Requires KMS ingress env variable and admin mode.

Notes

---

PR Type

Enhancement, Bug fix

---

Description

• Enhance KubernetesModelScaler to fetch node pools info

• Add jsonToMap helper for JSON-to-Map conversion

• Introduce GetNodePoolsInfoReactor with admin check

• Correct logger class in CanItRunReactor

---

Changes walkthrough 📝

	Relevant files
Enhancement	KubernetesModelScaler.java Support fetching node pools info                                                  src/prerna/engine/impl/model/KubernetesModelScaler.java Added getNodePoolsInfo HTTP GET method. Parsed JSON response into Map structure. Implemented jsonToMap recursive conversion. Added error handling and logging. +90/-0    GetNodePoolsInfoReactor.java Add node pools info reactor                                                            src/prerna/reactor/model/GetNodePoolsInfoReactor.java Created reactor invoking getNodePoolsInfo. Added admin permission check. Returned node pools info as NounMetadata. Logged errors on connection failure. +36/-0
Bug fix	CanItRunReactor.java Fix logger class reference                                                              src/prerna/reactor/model/CanItRunReactor.java Updated logger to use CanItRunReactor.class. Corrected class name in LogManager instantiation. +1/-1

---

Need help?

Type /help how to ... in the comments thread for any questions about PR-Agent usage.

Check out the documentation for more information.

[github-actions]

@CodiumAI-Agent /review

[github-actions]

@CodiumAI-Agent /improve

[QodoAI-Agent]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Null Response Entity The code does not check if the HTTP response entity is null before calling EntityUtils.toString, which can cause a NullPointerException when the server returns no body. HttpEntity responseEntity = response.getEntity(); String responseString = EntityUtils.toString(responseEntity); Error Logging In the catch block, the error log omits exception details; include the exception or stack trace in the logging call to facilitate debugging. } catch (Exception e) { classLogger.error("Error connecting to the Kubernetes Model Scaler endpoint for Nodepool information.."); throw new RuntimeException("Failed to connect to Kubernetes Model Scaler endpoint: " + e.getMessage()); Unauthorized Exception Throwing IllegalArgumentException for permission failures may confuse error handling; consider using a more specific exception or HTTP status for unauthorized access. if (!SecurityAdminUtils.userIsAdmin(this.insight.getUser())) { throw new IllegalArgumentException("User does not have permission to query this endpoint."); }

[github-actions]

@CodiumAI-Agent /update_changelog

[QodoAI-Agent]

PR Code Suggestions ✨

Category	Suggestion	Impact
General	Map JSON null to Java null Handle JSONObject.NULL values explicitly so they map to Java null instead of the JSON null placeholder. This ensures downstream code receives actual null values. src/prerna/engine/impl/model/KubernetesModelScaler.java [147-160] } else if (value instanceof JSONArray) { JSONArray array = (JSONArray) value; List<Object> list = new ArrayList<>(); for (int i = 0; i < array.length(); i++) { Object item = array.get(i); - if (item instanceof JSONObject) { + if (item == JSONObject.NULL) { + list.add(null); + } else if (item instanceof JSONObject) { list.add(jsonToMap((JSONObject) item)); } else { list.add(item); } } map.put(key, list); Suggestion importance[1-10]: 7 __ Why: Explicitly handling JSONObject.NULL ensures JSON nulls map to actual Java nulls, preventing downstream logic issues and improving data integrity.	Medium
	Log exception with stack trace Include the caught exception in the log call to preserve the stack trace for debugging. Use the logger’s overloaded method that accepts a throwable. src/prerna/reactor/model/GetNodePoolsInfoReactor.java [31-34] } catch (Exception e) { - classLogger.error("Error connecting to the Kubernetes Model Scaler endpoint for Nodepool information.."); - throw new RuntimeException("Failed to connect to Kubernetes Model Scaler endpoint: " + e.getMessage()); + classLogger.error("Error connecting to the Kubernetes Model Scaler endpoint for Nodepool information.", e); + throw new RuntimeException("Failed to connect to Kubernetes Model Scaler endpoint: " + e.getMessage(), e); } Suggestion importance[1-10]: 7 __ Why: Including the caught exception in the classLogger.error call preserves the stack trace for debugging without altering control flow.	Medium
	Add connection request timeout Add a connection request timeout to prevent hangs when the connection pool is exhausted. This will ensure the client fails fast under high load. src/prerna/engine/impl/model/KubernetesModelScaler.java [86-89] RequestConfig requestConfig = RequestConfig.custom() .setConnectTimeout(5000) + .setConnectionRequestTimeout(5000) .setSocketTimeout(5000) .build(); Suggestion importance[1-10]: 5 __ Why: Adding ConnectionRequestTimeout prevents hangs when the connection pool is exhausted, but it's a minor improvement for high-load scenarios.	Low
Possible issue	Check for null HTTP response entity Add a null check for the response entity before calling EntityUtils.toString to avoid potential NullPointerExceptions when the response has no body. If the entity is null, throw or handle an appropriate exception. src/prerna/engine/impl/model/KubernetesModelScaler.java [100-101] HttpEntity responseEntity = response.getEntity(); +if (responseEntity == null) { + throw new RuntimeException("Empty response received from node pools endpoint"); +} String responseString = EntityUtils.toString(responseEntity); Suggestion importance[1-10]: 6 __ Why: The suggestion correctly adds a null check before calling EntityUtils.toString() to avoid a potential NPE and improve robustness.	Low