From f421b434282b54df3a2b50b9d4dba8c6ae4fecca Mon Sep 17 00:00:00 2001 From: hui lai Date: Mon, 20 Jan 2025 15:18:56 +0800 Subject: [PATCH] [fix](cloud)(bulk load) fix memory leak in FE observer node (#47074) ![image](https://github.com/user-attachments/assets/ad83cf90-e148-414c-9278-2cad8f2cd9ed) The Memory of fe observer node increase over time and cannot significantly reduce after tigger full GC. ![image](https://github.com/user-attachments/assets/ac607b0c-73d1-44bc-a0ef-1a0e91c910fe) In follower node, when bulk load job replay, A key-value(jobId,-job) will be added to` CallBackFactory` to record the job but will not `removeCallback`, which cause memory leak in FE observer node. --- .../src/main/java/org/apache/doris/load/loadv2/LoadManager.java | 1 + 1 file changed, 1 insertion(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java index 07c459d61cf056..f661513cd66558 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java +++ b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java @@ -336,6 +336,7 @@ public void replayEndLoadJob(LoadJobFinalOperation operation) { LOG.info(new LogBuilder(LogKey.LOAD_JOB, operation.getId()).add("operation", operation) .add("msg", "replay end load job").build()); + Env.getCurrentGlobalTransactionMgr().getCallbackFactory().removeCallback(operation.getId()); // When idToLoadJob size increase 10000 roughly, we run removeOldLoadJob to reduce mem used if ((idToLoadJob.size() > 0) && (idToLoadJob.size() % 10000 == 0)) { removeOldLoadJob();