HDDS-12607. Parallelize recon tasks to speed up OM rocksdb reading tasks #9243

ArafatKhan2198 · 2025-11-04T05:15:35Z

Summary

Implemented parallel table iteration in Recon snapshot reprocessing to significantly improve performance. Instead of reading tables sequentially with a single iterator, this change uses multiple iterator threads to read different RocksDB SST file segments concurrently, with worker threads processing batches in parallel.

What Changed

ContainerKeyMapperHelper - Parallel iteration for FSO and OBS tables
FileSizeCountTaskHelper - Parallel iteration for FSO and OBS tables
OmTableInsightTask - Parallel counting for simple tables, sequential with keyIterator() for tables with non-String keys (dTokenTable, s3SecretTable)

Key Implementation Details

2-Tier Thread Pool: 5 iterator threads read from different SST file segments based on LiveFileMetaData boundaries, creating batches. 20 worker threads (or 2 for simple counting) process these batches.
Lockless Design: Each worker gets its own map via ConcurrentHashMap.computeIfAbsent(Thread.currentThread().getId()) - no read/write locks needed for map updates
Flush Coordination: Workers flush independently when their map reaches threshold (200k / 20 = 10k per worker). Only synchronize on DB write operation.
Cross-Task Synchronization: ContainerKeyMapperTask uses shared ConcurrentHashMap<Long, AtomicLong> because FSO and OBS tasks write to the same container IDs. FileSizeCountTask doesn't need shared state (FSO and OBS write different bucket keys).
Configuration: Added 4 tunable config keys with defaults optimized for common workloads:
- ozone.recon.task.reprocess.max.iterators = 5
- ozone.recon.task.reprocess.max.workers = 20
- ozone.recon.task.reprocess.max.keys.in.memory = 2000
- ozone.recon.filesizecount.flush.db.max.threshold = 200000

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12607

How was this patch tested?

Performance Results

Tested on OM RocksDB with 104M keys and 5.4M directories:

Task	Keys Processed	Duration	Throughput
ContainerKeyMapperTaskFSO	104,408,884	1,400 s	≈ 74,578 keys/sec
FileSizeCountTaskFSO	104,408,884	205.4 s	≈ 508,300 keys/sec
OmTableInsightTask	All the tables	69.3 s	N/A
FileSizeCountTaskOBS	5,437,734	14.7 s	≈ 369,900 keys/sec
ContainerKeyMapperTaskOBS	5,437,734	59.8 s	≈ 90,900 keys/sec

…sks.

sumitagrawl · 2025-11-04T08:42:33Z

share some performance data as seen improved due to parallel reprocess

sumitagrawl

@ArafatKhan2198 Due to recent features and old issue, data is not updated with synchronized way. Might be we can change model,
Producer-- (table iterator with decode) ==> consumer (single thread that will update NSSummary) instead of having multiple times lock take / release and avoid starvation.

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OmTableInsightTask.java

...one/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/NSSummaryTaskDbEventHandler.java

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

devmadhuu

Thanks @ArafatKhan2198 for the patch. Given some comments, Also the PR description and objective mentioned is confusing.

We are not parallelizing the recon tasks, rather we are improving all recon tasks's full snapshot reprocessing where we are processing all sst files in the respective rocksDB table needed for that task in concurrent fashion rather iterating the respective table's DB keys in sequential fashion. Pls correct the PR description to avoid any confusion. Recon tasks as such in itself are already being running in parallel before this PR change.

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

...on/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java

devmadhuu

@ArafatKhan2198 , Pls check , few more comments.

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

devmadhuu · 2025-11-07T12:08:08Z

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

+    this.maxIteratorTasks = 2 * iteratorCount;
+    this.maxWorkerTasks = workerCount * 2;
+    this.iteratorExecutor = new ThreadPoolExecutor(iteratorCount, iteratorCount, 1, TimeUnit.MINUTES,
+                    new ArrayBlockingQueue<>(iteratorCount * 2));


This has a potential issue where if queue fills up, there is no RejectionPolicy defined and RejectedExecutionException lead Recon to crash

Implemented with CallerRunsPolicy. Added rejection policy to both thread pools:

devmadhuu · 2025-11-07T12:37:10Z

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

+
+    try (ParallelTableIteratorOperation<String, OmKeyInfo> keyIter =
+             new ParallelTableIteratorOperation<>(omMetadataManager, omKeyInfoTable,
+                 StringCodec.get(), maxIterators, maxWorkers, maxKeysInMemory, 100000)) {


Where the flush logic gone ? Why removed ?

Had removed it by mistake, will add it back

devmadhuu · 2025-11-07T12:48:50Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

-              reconContainerMetadataManager)) {
-            LOG.error("Failed to flush container key data for {}", taskName);
-            return false;
+      ReentrantReadWriteLock lock = new ReentrantReadWriteLock();


Use fair locking, else thread starvation issue can happen.

ReentrantReadWriteLock is non-fair by default

With 20 worker threads continuously acquiring read locks, write lock requests can be starved indefinitely

When containerKeyMap reaches threshold, threads need write lock to flush, but may wait forever

Scenario:
Thread 1: [Read Lock] -> processing key 1
Thread 2: [Read Lock] -> processing key 2
Thread 3: [Read Lock] -> processing key 3
...
Thread 20: [Read Lock] -> processing key 20
Thread 21: [Waiting Write Lock] <- BLOCKED! (containerKeyMap full, needs to flush)
Thread 1: [Read Lock] -> processing key 21 (lock released and reacquired)
Thread 2: [Read Lock] -> processing key 22
... Thread 21 STILL WAITING

Evaluated but decided against due to performance impact. I initially implemented fair locking (new ReentrantReadWriteLock(true)), but observed 60%+ performance degradation in testing (throughput dropped from ~60K keys/sec to ~7K keys/sec).

Alternative mitigation implemented:

AtomicBoolean isFlushingInProgress = new AtomicBoolean(false); if (map.size() >= threshold && isFlushingInProgress.compareAndSet(false, true)) { // Only ONE thread attempts flush }

This flag-based coordination prevents the mass queueing scenario while maintaining performance. In our 8.7M key workload, flushing happens ~58 times total (every 150K keys), making starvation statistically unlikely. Open to revisiting if starvation is observed in production.

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

… for fileSizeCountTask, and made ReentrantReadWriteLock fair

ArafatKhan2198 · 2025-11-10T15:32:53Z

Thank you for the thorough review! @devmadhuu
Please do a re-review and let me know if any more changes.

ArafatKhan2198 · 2025-11-10T15:38:49Z

I tested on an OM DB with 8.7 million keys and 5.6 million directories, totaling about 1.2 TB of data.

Task	Bucket/Layout	Items	Duration (s)	Throughput (/s)	Notes
ContainerKeyMapper	FSO	8,749,130 keys	215.22	40,651
FileSizeCount	FSO	8,749,130 keys	40.72	214,887
ContainerKeyMapper	OBS	680,167 keys	20.81	32,685
FileSizeCount	OBS	680,167 keys	4.26	159,814
OmTableInsightTask (handlers)	—	22 tables	164.68	—	3 parallelized, 19 sequential

I have also attached the recon logs for verification - recon.log

devmadhuu

@ArafatKhan2198 thanks for improving the patch. Pls see some more comments.

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

devmadhuu · 2025-11-11T07:09:24Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

+            try {
+              lock.writeLock().lock();
+              try {
+                if (!checkAndCallFlushToDB(containerKeyMap, containerKeyFlushToDBMaxThreshold,


This is triple locking here which increases contention and not needed. checkAndCallFlushToDB method also using static synchronization.

write lock → static monitor → RocksDB locks

Increases contention

Increases latency

Increases likelihood of deadlock

Thanks, I have made the change.

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

devmadhuu · 2025-11-11T10:06:11Z

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

+            // Double-check after acquiring write lock
+            if (fileSizeCountMap.size() >= FLUSH_THRESHOLD) {
+              LOG.debug("Flushing {} accumulated counts to RocksDB for {}", fileSizeCountMap.size(), taskName);
+              writeCountsToDB(fileSizeCountMap, reconFileMetadataManager);


Now this is being called by multiple threads in hot path and in this method, for every key, we are doing read DB operation, this could be performance issue,

Long existingCount = reconFileMetadataManager.getFileSizeCount(key);

It is O(N) DB reads instead of O(1) batch operations

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

…tHashMap<Long,AtomicLong> for container counts

devmadhuu

Thanks @ArafatKhan2198 for improving the patch, Need to properly check the error handling behavior and few more comments. Please check carefully.

Please add some test coverage:

Concurrent FSO + OBS task execution
Thread pool exhaustion

Below can be done in separate PR, you can mention the JIRA for this comment here.
Add metrics for parallel processing
- Worker thread utilization
- Lock wait times
- Queue depths

...on/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

sumitagrawl

@swamirishi there are few comments over parallel iterator correctness when filter is present, please check
cc: @ArafatKhan2198

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

sumitagrawl · 2025-11-18T11:35:46Z

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java

+            }
+            if (!keyValues.isEmpty()) {
+              waitForQueueSize(workerFutures, maxWorkerTasks - 10);
+              workerFutures.add(valueExecutors.submit(() -> {


workerFutures will wait for the batch being queued. May have some delay in lesser number of splits. Otherway is,
Blocking-workerQueue --
Producer (multi-threaded, ie iterator-executor): kv is added as soon identified
consumer: Set of thread waiting on queue to retrive and process
End point: set flag when iterator is completed

This might be simpler implementation to avoid intermediate wait.

sumitagrawl · 2025-11-21T12:20:50Z

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/ReconConstants.java


  public static final AtomicBoolean CONTAINER_KEY_TABLES_TRUNCATED = new AtomicBoolean(false);

+  public static final AtomicBoolean CONTAINER_KEY_COUNT_MAP_INITIALIZED = new AtomicBoolean(false);


no need this Atomic boolean.

sumitagrawl · 2025-11-21T12:29:58Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

+   * cross-task data corruption where FSO and OBS tasks overwrite each other's counts.
+   */
+  private static void initializeSharedContainerCountMapIfNeeded(String taskName) {
+    synchronized (SHARED_MAP_LOCK) {


init of map and clear can be done using truncate_lock, rename this lock.

sumitagrawl · 2025-11-21T12:43:46Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

    try {
      // No deleted container list needed since "reprocess" only has put operations
-      writeToTheDB(containerKeyMap, containerKeyCountMap, Collections.emptyList(), reconContainerMetadataManager);
+      writeToTheDB(localContainerKeyMap, sharedContainerKeyCountMap, Collections.emptyList(), reconContainerMetadataManager);


only last task getting executed need flush and cleanup map, else data can got wrong.

sumitagrawl · 2025-11-21T13:24:13Z

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

-    writeCountsToDB(fileSizeCountMap, reconFileMetadataManager);
+    long writeStartTime = Time.monotonicNow();
+    // Acquire GLOBAL lock (cross-task) before writing to DB
+    FILE_COUNT_WRITE_LOCK.writeLock().lock();


FILE_COUNT_LOCK needs to be local lock

sumitagrawl · 2025-12-02T06:51:27Z

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OmTableInsightTask.java

+   * Process table in parallel using multiple iterators and workers.
+   * Only for tables with String keys.
+   */
+  private void processTableInParallel(String tableName, Table<?, ?> table, 


no need pass table here, get table with StringCodec as done in try-catch and use same if required other place

Implemented. Removed table parameter and get table inside method with StringCodec as suggested. Method now takes only tableName and omMetadataManager.

sumitagrawl · 2025-12-02T06:52:01Z

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OmTableInsightTask.java

+   * Calculate logging threshold based on table size.
+   * Logs progress every 1% of total keys, minimum 1.
+   */
+  private long calculateLoggingThreshold(Table<?, ?> table) {


can pass estimatedCount directly.

Implemented. Changed method to take long estimatedCount instead of Table.

sumitagrawl · 2025-12-02T06:54:04Z

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OmTableInsightTask.java

+   * Process table sequentially using raw iterator (no type assumptions).
+   * Used for tables with non-String keys or as fallback.
+   */
+  private void processTableSequentially(String tableName, Table<?, ?> table) throws IOException {


we can get table inside only with keyIterator(), no need value iterator.

Implemented. Changed to use table.keyIterator() instead of full iterator.

sumitagrawl · 2025-12-02T06:57:49Z

hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OmTableInsightTask.java

-          replicatedSizeMap.put(getReplicatedSizeKeyFromTable(tableName),
-              details.getRight());
+          Table<String, ?> stringTable = (Table<String, ?>) table;
+          try (TableIterator<String, ? extends Table.KeyValue<String, ?>> iterator = stringTable.iterator()) {


we can remove table object and getTableSizeAndCount can get table object and return count

Implemented. Modified OmTableHandler interface and both implementations (OpenKeysInsightHandler, DeletedKeysInsightHandler) to take tableName and omMetadataManager instead of iterator. Handlers now get table and create iterators themselves.

sumitagrawl · 2025-12-02T07:01:45Z

...p-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTaskHelper.java

+        if (myMap.size() >= PER_WORKER_THRESHOLD) {
+            synchronized (flushLock) {
+                writeCountsToDB(myMap, reconFileMetadataManager);
+                myMap.clear();


rename myMap variable as fileSizeCountMap

Implemented. Renamed myMap to workerFileSizeCountMap for clarity

sumitagrawl · 2025-12-02T07:04:34Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

+          SHARED_CONTAINER_KEY_COUNT_MAP.clear();
+
+          // Step 3: Initialize reference counter (2 tasks: FSO + OBS)
+          ACTIVE_TASK_COUNT.set(2);


need increment in synchronized_lock by one every task, and decrement in same lock when finish

Implemented

sumitagrawl · 2025-12-02T07:06:44Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

+      Function<Table.KeyValue<String, OmKeyInfo>, Void> kvOperation = kv -> {
+        try {
+          // Get or create this worker's private local map using thread ID
+          Map<ContainerKeyPrefix, Integer> myLocalMap = allLocalMaps.computeIfAbsent(


rename myLocalMap to containerKeyPrefixMap

Renamed myLocalMap to containerKeyPrefixMap for better clarity.

sumitagrawl · 2025-12-02T07:08:38Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

-          if (!checkAndCallFlushToDB(containerKeyMap, containerKeyFlushToDBMaxThreshold,
+          // Flush this worker's map when it reaches threshold
+          if (myLocalMap.size() >= PER_WORKER_THRESHOLD) {
+            synchronized (flushLock) {


we do not need lock as containerKeyPrefix is unique, added to table.

Thanks I have now Removed flushLock from periodic flush as each worker flushes unique ContainerKeyPrefix values (containerId + keyPrefix + keyVersion combinations).

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

sumitagrawl · 2025-12-02T07:13:20Z

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

+      long totalContainers = SHARED_CONTAINER_KEY_COUNT_MAP.size();
+
+      // Final flush: Shared container count map
+      if (!flushAndCommitContainerKeyInfoToDB(Collections.emptyMap(), SHARED_CONTAINER_KEY_COUNT_MAP, reconContainerMetadataManager)) {


flush to be done at last, else can have twice flush.

Implemented. Moved shared container count map flush inside the if (remainingTasks == 0) block. Now only the last completing task flushes the shared map once, preventing duplicate flushes.

sumitagrawl

Given few comment

...ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/DeletedKeysInsightHandler.java

...op-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/OpenKeysInsightHandler.java

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

…NT leak on initialization exception.

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java

...ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/DeletedKeysInsightHandler.java

sumitagrawl

LGTM

ArafatKhan2198 · 2025-12-10T07:04:22Z

Got a green run for CI -
https://github.com/ArafatKhan2198/ozone/actions/runs/20088033032

HDDS-12607. Parallelize recon tasks to speed up OM rocksdb reading ta…

848d52b

…sks.

ArafatKhan2198 requested review from devmadhuu and swamirishi November 4, 2025 05:15

sumitagrawl reviewed Nov 4, 2025

View reviewed changes

ArafatKhan2198 added 3 commits November 5, 2025 13:58

Reverted the changes for NSSummaryTask

b9100a6

Removed unnecessary changes

1b39a9a

Refactored and changed the code a bit

266138a

ArafatKhan2198 requested a review from sumitagrawl November 7, 2025 06:17

devmadhuu reviewed Nov 7, 2025

View reviewed changes

...n/src/main/java/org/apache/hadoop/ozone/recon/tasks/util/ParallelTableIteratorOperation.java Show resolved Hide resolved

...on/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java Outdated Show resolved Hide resolved

devmadhuu reviewed Nov 7, 2025

View reviewed changes

ArafatKhan2198 added 4 commits November 10, 2025 12:11

Removed the commented code

7cfc000

Made code review changes- Added RejectionPolicy, Re-added flush logic…

e67b310

… for fileSizeCountTask, and made ReentrantReadWriteLock fair

Added a flag isFlushingInProgress to avoid race condition while flushing

a9855b0

Made final code changes for testing

56a2e40

ArafatKhan2198 requested a review from devmadhuu November 10, 2025 14:18

devmadhuu reviewed Nov 11, 2025

View reviewed changes

ArafatKhan2198 added 4 commits November 12, 2025 13:44

Prevent FSO/OBS container count overwrite by using a shared Concurren…

8424170

…tHashMap<Long,AtomicLong> for container counts

Added write lock in the writeCountsToDB method

b93511e

Removed extra locking and improved exception log messages

e37bb11

Defined proper caller based locking for writeConuntsToDB

c02718e

ArafatKhan2198 requested a review from devmadhuu November 14, 2025 07:00

devmadhuu reviewed Nov 14, 2025

View reviewed changes

Some more improvements

648e94c

sumitagrawl reviewed Nov 18, 2025

View reviewed changes

sumitagrawl reviewed Nov 21, 2025

View reviewed changes

ArafatKhan2198 added 2 commits November 24, 2025 11:00

Added changes to the parallel Iterator class

0c0c09a

Consolidated all the locks in a single lock for simplicity

f7e0cf3

sumitagrawl reviewed Dec 2, 2025

View reviewed changes

ArafatKhan2198 added 6 commits December 4, 2025 12:14

Fixed final review comments

eccd370

Removed unnecessary changes

940b271

Removed some extra changes

702e92b

Reverted back to the old change

58bcccd

Fixed some more code

f55fe25

Removed testing code

47eaf82

sumitagrawl reviewed Dec 4, 2025

View reviewed changes

Recon: Improve type safety in OmTableHandlers and fix ACTIVE_TASK_COU…

980135d

…NT leak on initialization exception.

ArafatKhan2198 requested a review from sumitagrawl December 5, 2025 05:32

sumitagrawl reviewed Dec 8, 2025

View reviewed changes

...-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ContainerKeyMapperHelper.java Show resolved Hide resolved

...ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/DeletedKeysInsightHandler.java Outdated Show resolved Hide resolved

Removed the supress warning

cff7278

sumitagrawl approved these changes Dec 8, 2025

View reviewed changes

ArafatKhan2198 and others added 12 commits December 9, 2025 11:29

Fixed checkstyle

04df207

Fixed findbugs

bed2151

Removed testing code

1a26915

Merge branch 'master' into HDDS-12607-updated

73434a1

Fixed failing tests

0c9cd92

Fixed TestConfigurationFieldsBase

60a197a

Fixed container endpoint

e670c93

Fixed checkstyle issues

a4d4f35

Fixed failing tests

d60d4fe

Fixed TestContainerEndpoint

6addd6d

Fixed checkstyle

1a1e838

Fixed TestReconAndAdminContainerCLI

ca379ff

ArafatKhan2198 marked this pull request as ready for review December 10, 2025 07:04

ArafatKhan2198 added performance recon labels Dec 10, 2025

ArafatKhan2198 merged commit ad891ec into apache:master Dec 10, 2025
56 checks passed


		public static final AtomicBoolean CONTAINER_KEY_TABLES_TRUNCATED = new AtomicBoolean(false);

		public static final AtomicBoolean CONTAINER_KEY_COUNT_MAP_INITIALIZED = new AtomicBoolean(false);

HDDS-12607. Parallelize recon tasks to speed up OM rocksdb reading tasks #9243

HDDS-12607. Parallelize recon tasks to speed up OM rocksdb reading tasks #9243

Uh oh!

Conversation

ArafatKhan2198 commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What is the link to the Apache JIRA

How was this patch tested?

Performance Results

Uh oh!

sumitagrawl commented Nov 4, 2025

Uh oh!

sumitagrawl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devmadhuu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

devmadhuu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArafatKhan2198 Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ArafatKhan2198 commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArafatKhan2198 commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I have also attached the recon logs for verification - recon.log

Uh oh!

devmadhuu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

devmadhuu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sumitagrawl left a comment

Choose a reason for hiding this comment

ArafatKhan2198 commented Nov 4, 2025 •

edited

Loading

ArafatKhan2198 Nov 10, 2025 •

edited

Loading

ArafatKhan2198 commented Nov 10, 2025 •

edited

Loading

ArafatKhan2198 commented Nov 10, 2025 •

edited

Loading

ArafatKhan2198 Dec 4, 2025 •

edited

Loading