MLE-28401: Add missing guard for logging retry message by yunzvanessa · Pull Request #591 · marklogic/marklogic-contentpump

yunzvanessa · 2026-04-14T20:27:15Z

The "Retrying committing batch" debug message was logged on every batch, even the first attempt, producing misleading output that suggested retries were happening when they weren't. Added a commitRetry > 0 guard to match the existing pattern in the parent class TransformWriter.

This issue surfaced when investigating and mlcp regression failure: https://progresssoftware.atlassian.net/browse/MLE-28401

…rnal Entity Polaris medium severity issues (#577) * MLE-27554 MLE-27556: Fix XXE Injection - XML External Entity related to XMLInputFactory * MLE-27553 MLE-27555: Fix XXE Injection - XML External Entity related to TransformerFactory * MLE-27553 MLE-27554 MLE-27555 MLE-27556: Split XXE-prevention try/catch blocks and improve warning messages

…ontentReader.java

MLE-27557: Fix weak crypto - replace insecure TLSv1 with TLSv1.3 in ContentReader.java

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ss spurious wakeup vulnerability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

MLE-26865, MLE-26867: Fixes for issues identified by Polaris

…om-develop PDP-1182: Remove per-repo pr-workflow.yaml

Copilot

Pull request overview

This PR addresses misleading “Retrying committing batch” debug logging by adding a commitRetry > 0 guard, and it also includes a broad set of additional robustness/security/concurrency updates across mlcp and the MapReduce connector.

Changes:

Guard retry-specific debug logging so it only appears on actual retry attempts.
Add defensive checks (null/empty config values, filesystem edge cases, safer iteration over server responses) and harden XML processing against XXE.
Refactor local multithreaded execution coordination (wait/notify → CountDownLatch) and bump project/dependency versions (plus CI/workflow/config updates).

Reviewed changes

Copilot reviewed 33 out of 34 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/main/java/com/marklogic/tree/ExpandedTree.java	Avoid repeated string computation in trace logging path (minor refactor).
src/main/java/com/marklogic/mapreduce/utilities/InternalUtilities.java	More defensive handling of `INPUT_HOST` configuration.
src/main/java/com/marklogic/mapreduce/test/FCheck.java	Guard against `File.listFiles()` returning null.
src/main/java/com/marklogic/mapreduce/examples/HelloWorld.java	Null-safe handling of document element when extracting text.
src/main/java/com/marklogic/mapreduce/examples/ContentReader.java	Update SSL example defaults (protocols/ciphers) and validate output path.
src/main/java/com/marklogic/mapreduce/JSONDocument.java	Configure `TransformerFactory` to restrict external DTD/stylesheet access (XXE hardening).
src/main/java/com/marklogic/mapreduce/ForestReader.java	Null-check parent directory path before building related file paths.
src/main/java/com/marklogic/mapreduce/DOMDocument.java	Configure `TransformerFactory` to restrict external DTD/stylesheet access (XXE hardening) and add null guards.
src/main/java/com/marklogic/mapreduce/ContentOutputFormat.java	Validate output hosts and add safer result iteration (`hasNext()` checks).
src/main/java/com/marklogic/dom/NodeImpl.java	Prevent NPE when parent node is null; make `isEqualNode` local-name compare null-safe.
src/main/java/com/marklogic/dom/DocumentImpl.java	Null-safe namespace lookup/prefix lookup when document element is missing.
src/main/java/com/marklogic/dom/CommentImpl.java	Null-safe namespace lookup delegation when parent is missing.
src/main/java/com/marklogic/dom/CharacterDataImpl.java	Null-safe length/substring handling for character data.
src/main/java/com/marklogic/dom/AttrImpl.java	Null-safe namespace lookup delegation when owner element is missing.
src/main/java/com/marklogic/contentpump/utilities/PermissionUtil.java	Validate `OUTPUT_HOST` presence before creating output ContentSource.
src/main/java/com/marklogic/contentpump/utilities/JSONDocBuilder.java	Guard against null exception messages when filtering “missing value”.
src/main/java/com/marklogic/contentpump/TransformWriter.java	Null-safe handling of exception message when detecting “Module Not Found”.
src/main/java/com/marklogic/contentpump/TransformOutputFormat.java	Validate `OUTPUT_HOST` presence before querying mimetypes.
src/main/java/com/marklogic/contentpump/ThreadManager.java	Replace wait/notify coordination with latch; validate output hosts; minor control-flow guard.
src/main/java/com/marklogic/contentpump/SingleDocumentWriter.java	Null-check parent directory before `mkdirs()`.
src/main/java/com/marklogic/contentpump/RDFReader.java	Safer permissions initialization; validate hosts; add result iteration guards.
src/main/java/com/marklogic/contentpump/OutputArchive.java	Null-check parent directory before `mkdirs()`.
src/main/java/com/marklogic/contentpump/MultithreadedMapper.java	Introduce latch-based signaling when runners are created.
src/main/java/com/marklogic/contentpump/LocalJobRunner.java	Wire latch into `MultithreadedMapper` tasks; ensure latch is released on failure.
src/main/java/com/marklogic/contentpump/InputType.java	Validate output hosts; add `ResultSequence` `hasNext()` guards for role map query.
src/main/java/com/marklogic/contentpump/DatabaseTransformWriter.java	Add `commitRetry > 0` guard to retry log message.
src/main/java/com/marklogic/contentpump/DatabaseContentReader.java	Add `ResultSequence` `hasNext()` guards; null-check W3C permission element conversion.
src/main/java/com/marklogic/contentpump/ContentPump.java	Guard against `listFiles()` returning null when building lib jar list.
src/main/java/com/marklogic/contentpump/CompressedAggXMLReader.java	Disable external entities/DTDs on `XMLInputFactory` (XXE hardening).
src/main/java/com/marklogic/contentpump/AggregateXMLReader.java	Disable external entities/DTDs on `XMLInputFactory` (XXE hardening).
pom.xml	Bump project version and Avro dependency version.
.github/workflows/pr-workflow.yaml	Remove JIRA ID validator workflow.
.copyrightconfig	Add copyright validation config and exclusions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-14T20:31:45Z

+    	            String atomStr = atomString(textReps[index]);
    	            LOG.trace("atom " + textReps[index] + " [" + 
-    	                atomString(textReps[index]) + "] length " + 
-    	                atomString(textReps[index]).length());
+    	                atomStr + "] length " + 
+    	                (atomStr != null ? atomStr.length() : "0"));
    	        }
    		buf.append(atomString(textReps[index++]));
    	    }


atomString(textReps[index]) is now computed into atomStr for logging, but buf.append(atomString(...)) still recomputes it. Reuse atomStr for the append too to avoid duplicate work and to ensure the trace output matches the actual appended value if atomString is non-trivial/non-deterministic.

Copilot · 2026-04-14T20:31:46Z

+        String[] hosts = conf.getStrings(INPUT_HOST);
+        if (hosts == null || hosts.length == 0) {
            throw new IllegalArgumentException(INPUT_HOST + 
                    " is not specified.");
        }
-
-        return getInputContentSource(conf, host);
+        return getInputContentSource(conf, hosts[0]);
    }


This change now checks for hosts == null || hosts.length == 0, but it no longer validates that hosts[0] is non-empty/blank. If INPUT_HOST is set to an empty string, getInputContentSource(conf, hosts[0]) will still be called and will likely fail later with a less clear error; consider restoring an explicit empty/blank check for hosts[0] here.

Copilot · 2026-04-14T20:31:46Z

        }
        String data = getData();
+        if (data == null) {
+            throw new DOMException(DOMException.DOMSTRING_SIZE_ERR, null);


substringData now throws DOMSTRING_SIZE_ERR when getData() returns null, but getLength() treats null data as length 0. This makes behavior inconsistent and introduces a new exception path for callers; consider treating null data as an empty string here (or throwing a more appropriate DOMException like INVALID_STATE_ERR), consistent with getLength().

Suggested change

throw new DOMException(DOMException.DOMSTRING_SIZE_ERR, null);

data = "";

Copilot · 2026-04-14T20:31:47Z

+                        if (LOG.isDebugEnabled()) {
+                            LOG.debug(getFormattedBatchId() +
+                                "Retrying committing batch, attempts: " +
+                                commitRetry + "/" + MAX_RETRIES);


The retry log message prints commitRetry + "/" + MAX_RETRIES, but the loop condition is commitRetry < commitRetryLimit. If commitRetryLimit is configurable and diverges from MAX_RETRIES, this debug message becomes misleading; consider logging against commitRetryLimit here to match the actual retry limit being enforced.

Suggested change

commitRetry + "/" + MAX_RETRIES);

commitRetry + "/" + commitRetryLimit);

Copilot · 2026-04-14T20:31:47Z

@@ -17,7 +17,7 @@
  </licenses>

  <properties>
-    <versionString>12.0</versionString>
+    <versionString>12.1</versionString>


PR description/title focuses on adding a logging guard for retry messages, but this PR also bumps the project version (and includes many other changes across the codebase). If these additional changes are intentional, please reflect them in the PR description or consider splitting into separate PRs to keep scope focused and simplify review/release notes.

builder and others added 30 commits August 27, 2025 00:08

Checking in the nightly xcc - 20250827

9186380

Checking in the nightly xcc - 20250828

eab6f4a

Checking in the nightly xcc - 20250829

91e6de4

Checking in the nightly xcc - 20250830

9bbf196

Checking in the nightly xcc - 20250831

7c9ee81

Checking in the nightly xcc - 20250901

a6aa75a

Checking in the nightly xcc - 20250902

5217fb3

Checking in the nightly xcc - 20250903

3f6c7ad

Checking in the nightly xcc - 20250904

2009df9

PDP-536: Adding copyright check (#547)

350d271

Checking in the nightly xcc - 20250905

e500864

Checking in the nightly xcc - 20250906

3ef062c

Checking in the nightly xcc - 20250907

098bc6c

Checking in the nightly xcc - 20250908

8f7171a

Checking in the nightly xcc - 20250909

df8d719

Checking in the nightly xcc - 20250910

2bc2ff5

Checking in the nightly xcc - 20250911

68dd002

Checking in the nightly xcc - 20250912

7afeb87

Checking in the nightly xcc - 20250913

b7b5e37

Checking in the nightly xcc - 20250914

0f7cd30

Checking in the nightly xcc - 20250915

0f55d06

Checking in the nightly xcc - 20250916

d06dc69

Checking in the nightly xcc - 20250917

744e3ba

Checking in the nightly xcc - 20250918

7ce2294

Checking in the nightly xcc - 20250919

b9c2214

Checking in the nightly xcc - 20250920

23f15c1

Checking in the nightly xcc - 20250921

5a21d5e

Checking in the nightly xcc - 20250922

b4f106a

Checking in the nightly xcc - 20250923

91884b8

Checking in the nightly xcc - 20250924

e6d0a26

builder and others added 26 commits March 29, 2026 00:08

Checking in the nightly xcc - 20260329

d91721e

MLE-27557: Fix weak crypto - replace insecure TLSv1 with TLSv1.3 in C…

599e569

…ontentReader.java

Checking in the nightly xcc - 20260330

a973220

Merge pull request #579 from vshaniga/MLE-27557-weak-crypto-fix

f1694e1

MLE-27557: Fix weak crypto - replace insecure TLSv1 with TLSv1.3 in ContentReader.java

Checking in the nightly xcc - 20260331

e5a5e18

MLE-26867: Fix potential divide by zero issue identified by Polaris

68463da

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

MLE-26865: replaced pool.wait()/notify() with CountDownLatch to addre…

b7155a4

…ss spurious wakeup vulnerability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Checking in the nightly xcc - 20260401

141431c

Checking in the nightly xcc - 20260402

49371d1

Merge pull request #581 from SiddharthMaheshB/develop

9b3c171

MLE-26865, MLE-26867: Fixes for issues identified by Polaris

Checking in the nightly xcc - 20260403

6e8a31f

Checking in the nightly xcc - 20260404

d59da1e

Checking in the nightly xcc - 20260405

c80f918

Checking in the nightly xcc - 20260406

14fb0c8

Checking in the nightly xcc - 20260407

ebcec77

Checking in the nightly xcc - 20260408

5f7118a

PDP-1182: Remove per-repo pr-workflow.yaml

89ed2d7

Checking in the nightly xcc - 20260409

4f67f19

Merge pull request #585 from marklogic/PDP-1182-remove-pr-workflow-fr…

4fc3ea5

…om-develop PDP-1182: Remove per-repo pr-workflow.yaml

Checking in the nightly xcc - 20260410

cfaaa85

Checking in the nightly xcc - 20260411

bcac6ab

Checking in the nightly xcc - 20260412

035a7f2

Checking in the nightly xcc - 20260413

4cafd31

Checking in the nightly xcc - 20260414

d02d827

MLE-28401: Add missing guard for logging retry message

6d8ce7c

Copilot AI review requested due to automatic review settings April 14, 2026 20:27

Copilot started reviewing on behalf of yunzvanessa April 14, 2026 20:27 View session

yunzvanessa closed this Apr 14, 2026

Copilot AI reviewed Apr 14, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MLE-28401: Add missing guard for logging retry message#591

MLE-28401: Add missing guard for logging retry message#591
yunzvanessa wants to merge 294 commits intomainfrom
MLE-28401-fix-retry-logging

yunzvanessa commented Apr 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 14, 2026

Uh oh!

Copilot AI Apr 14, 2026

Uh oh!

Copilot AI Apr 14, 2026

Uh oh!

Copilot AI Apr 14, 2026

Uh oh!

Copilot AI Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

	throw new DOMException(DOMException.DOMSTRING_SIZE_ERR, null);
	data = "";

	commitRetry + "/" + MAX_RETRIES);
	commitRetry + "/" + commitRetryLimit);

Conversation

yunzvanessa commented Apr 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants