733 vectordatabasequeryreactor output divider value for word doc chunks always 1

[Varaham]

Description :

The VectorDatabaseQueryReactor enables users to perform similarity searches between a user input and a vector database.
The "output" JSON value in the "pixelReturn" Response JSON contains the "Divider" as the attribute which is meant to indicate the page number(s) the text chunk is on.

Current issue:

In the current output ,for .docx files, the "Divider" is equal to 1 for all text chunks irrespective of the page number at which the text exists / spanned in the document.

Workaround implementation :

Below are the steps in workaround implementation -

The docx contents are divided into individual paragraphs as separate chunks using XWPF API POI class.
When the page break is encountered, the page count is incremented and the Divider will have the page number at which the text chunk (paragraph) exists.

Exceptions :

The page break will be ignored in the below scenarios:

Whenever any image has been spanned in between the pages.
Whenever any tabular fields are spanned in between the pages
Any JSON values are spanned in between the pages.
In the above scenario, the DocReader will take the entire value as a single text chunk (paragraph) and will take the last page till the objects are spanned. The Divider will take all the page numbers with comma separated values and will be displayed in the JSON output.

Steps :

Click on 'Open Vector' and create a new Vector Catalog database with all the required fields.
In the Vector Catalog window, go to the 'Files' tab to upload a new docx file. (pic 1)
In the Files tab, click on 'Embed new Document' button (pic 2)
Select any docx file (pic 3)
Click on 'Embed' and ensure that it is embedded (pic 4)
Once the document is uploaded , listed (pic 5)
Go to the 'Q & A' tab. Type a question and click on ' Generate Answer'. See in the response JSON in the 'Networks tab'.

Response JSON.txt

[github-actions]

@CodiumAI-Agent /describe

[QodoAI-Agent]

Title

733 vectordatabasequeryreactor output divider value for word doc chunks always 1

---

User description

Description

Changes Made

How to Test

1. Steps to reproduce/test the behavior
2. Expected outcomes

Notes

---

PR Type

Bug fix

---

Description

• Fix incorrect divider value for Word doc chunks

• Detect page breaks via CTBr and CTEmpty runs

• Track last paragraph for accurate page logging

• Remove deprecated paragraph.isPageBreak logic

---

Changes walkthrough 📝

	Relevant files
Bug fix	DocProcessor.java Fix page break detection logic                                                      src/prerna/reactor/frame/gaas/processors/DocProcessor.java Added XWPFRun, CTBr, CTEmpty imports Implemented run-based page break detection Tracked last paragraph for logging Removed paragraph.isPageBreak handling +34/-7

---

Need help?

Type /help how to ... in the comments thread for any questions about PR-Agent usage.

Check out the documentation for more information.

[github-actions]

@CodiumAI-Agent /review

[ppatel9703]

Can we add in a proper description and testing steps.

[QodoAI-Agent]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Unused variable The variable pageCount is retrieved but never utilized anywhere in the method. Consider removing it to avoid confusion and keep the code clean. int pageCount = document.getProperties().getExtendedProperties().getUnderlyingProperties().getPages(); Debug print instead of logging There is a System.out.println call for debugging. Replace it with a proper logger call (e.g., classLogger.debug) and remove any temporary print statements. System.out.println(" pageCount -> " + pageCount);

[github-actions]

@CodiumAI-Agent /improve

[ppatel9703]

Pls Remove the sysout

[Varaham]

code committed after replacing sysout with loggers

[QodoAI-Agent]

PR Code Suggestions ✨

Latest suggestions up to d195d84

Category	Suggestion	Impact
General	Null-check paragraph runs Add a null check for paragraph.getRuns() to prevent NullPointerException when a paragraph has no runs. src/prerna/reactor/frame/gaas/processors/DocProcessor.java [124] -for (XWPFRun run : paragraph.getRuns()) { +List<XWPFRun> runs = paragraph.getRuns(); +if (runs != null) { + for (XWPFRun run : runs) { + // existing break detection logic + } +} Suggestion importance[1-10]: 2 __ Why: Adding a null check for getRuns() is unnecessary since Apache POI always returns a list, so this defensive change has minimal value.	Low

---

Previous suggestions

Suggestions up to commit 138af49

Category	Suggestion	Impact
Possible issue	Always count both break types Split the page‐break detection into two independent loops instead of an if/else, so both explicit and rendered page breaks are always counted. This avoids missing page breaks when both lists have entries. src/prerna/reactor/frame/gaas/processors/DocProcessor.java [125-139] -if (brList != null && !brList.isEmpty()) { +List<CTBr> brList = run.getCTR().getBrList(); +if (brList != null) { for (CTBr br : brList) { if (br.getType() == STBrType.PAGE) { pageNo++; } } -} else { - List<CTEmpty> lastRenderedPageBreakList = run.getCTR().getLastRenderedPageBreakList(); - if (lastRenderedPageBreakList != null) { - for (CTEmpty lastRenderedPageBreak : lastRenderedPageBreakList) { - pageNo++; - } - } +} +List<CTEmpty> lastRenderedPageBreakList = run.getCTR().getLastRenderedPageBreakList(); +if (lastRenderedPageBreakList != null) { + pageNo += lastRenderedPageBreakList.size(); } Suggestion importance[1-10]: 8 __ Why: The current if/else logic skips counting one break type when the other is present; separating the loops ensures both CTBr and CTEmpty page breaks are always accounted for.	Medium
	Fix undefined logger reference Use the existing logger instance (e.g., LOGGER) instead of the undefined classLogger and parameterize the message. This fixes the compile error and ensures proper logging. src/prerna/reactor/frame/gaas/processors/DocProcessor.java [145] -classLogger.info("Last paragraph on Page " + pageNo); +LOGGER.info("Last paragraph on page {}", pageNo); Suggestion importance[1-10]: 8 __ Why: Replacing the undefined classLogger with the existing LOGGER corrects the compile error and ensures the log message is emitted properly.	Medium
	Guard run iteration against null Add a null check around paragraph.getRuns() to prevent a potential NullPointerException when there are no runs. This guards the loop against invalid state. src/prerna/reactor/frame/gaas/processors/DocProcessor.java [124] -for (XWPFRun run : paragraph.getRuns()) { +List<XWPFRun> runs = paragraph.getRuns(); +if (runs != null) { + for (XWPFRun run : runs) { Suggestion importance[1-10]: 3 __ Why: Apache POI’s paragraph.getRuns() typically returns an empty list rather than null, so this null check is largely redundant and adds unnecessary complexity.	Low
General	Use logger instead of println Replace the stdout print with a logger call and remove or validate the unused pageCount variable. This ensures consistent logging and prevents stray System.out usage. src/prerna/reactor/frame/gaas/processors/DocProcessor.java [113] -System.out.println(" pageCount -> " + pageCount); +LOGGER.debug("Page count: {}", pageCount); Suggestion importance[1-10]: 6 __ Why: Replacing System.out.println with LOGGER.debug aligns with the existing logging approach and removes stray console output, improving consistency.	Low

[Varaham]

Console Output replaced with Loggers as per review comments

[github-actions]

@CodiumAI-Agent /update_changelog

[QodoAI-Agent]

Changelog updates: 🔄

2025-05-14 *

Fixed

• Correct page-based divider values in vector database query output for Word documents

Changed

• Replace console logging with structured loggers in document processing

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'