Skip to content

Vector-DB based RAG Implementation#1

Merged
aks8m merged 18 commits intomainfrom
tool-rag
Mar 23, 2026
Merged

Vector-DB based RAG Implementation#1
aks8m merged 18 commits intomainfrom
tool-rag

Conversation

@aks8m
Copy link
Copy Markdown
Collaborator

@aks8m aks8m commented Mar 23, 2026

I diverted away from using Neo4j because the Text-to-Cypher was unreliable, especially working on an Ollama based LLM.

aks8m added 18 commits March 13, 2026 16:08
Streamlines database configuration by unifying MapDB and vector store management under a single root directory. This change removes Neo4j-based graph traversal and writing logic from the charting and RAG components as part of a transition toward a tool-based RAG approach.
…ion entities

Relocates scope selection and search functionality from the global Observatory configuration to individual Constellations. This shift allows for more granular control over data boundaries during constellation creation. The change updates entities, services, and UI fragments to handle scopes at the constellation level and introduces a portal prompt field for charting jobs.
Completes the transition of scope management from Observatory to Constellation by updating HTMX endpoints and UI fragment paths. Enhances the constellation form with an EasyMDE Markdown editor for portal prompts and streamlines Maven configuration by removing the build cache extension and adopting the Spring Data BOM.
Improves the constellation creation UI by optimizing the grid layout and using CSS `:has()` to dynamically toggle list placeholders. This update also ensures that form reset logic only triggers on primary form submissions and adds activity indicators to scope management actions to improve feedback.
… modal

Updates the constellation management UI to display scopes directly in the table and moves the portal prompt into a scrollable modal to handle long text. This change also transitions record component access to standard property syntax in Thymeleaf templates and adds the @StringToFacade annotation to ensure proper mapping of scope sets.
Updates the constellation creation workflow to ensure a clean UI state after successful submission. The changes clear scope selections and search results, refresh the scope navigation tree from the root, and automatically hide the creation panel.
Replaces granular progress tracking of concepts, semantics, and patterns with a single unified progress metric. This change refactors the multi-step charting logic into a simplified processor interface, removes the specific Step enumeration, and consolidates charting operations into specialized embedding and deletion processors.
Provides a full implementation of EmbeddingChartProcessor to transform domain concepts into vector embeddings. The processor generates descriptive text segments including concept definitions, categories, and semantic features, then stores them with associated metadata in a vector store to support retrieval-augmented generation (RAG).
…-based memory

Transitions the portal's chat functionality to a declarative AI service model using LangChain4j. This change introduces session-aware chat memory, enables the use of constellation-specific system prompts during chat, and reorganizes the AI configuration into dedicated agent and vector database components.
Updates the embedding processor to conditionally include concept attributes, preventing empty fields in the generated text segments. It also refactors semantic feature generation into a structured set and corrects the ancestor category selection logic to improve the quality of data stored in the vector database.
…n selection, and Markdown rendering

Updates the portal UI to utilize unique session identifiers for chat persistence and replaces the integration toggle with a specific constellation selection dropdown. It also integrates Markdown-to-HTML rendering for AI responses to improve the presentation of chat results.
Updates the portal UI to require a constellation selection before submitting messages, providing visual feedback for invalid states. It also adds server-side validation for the constellation ID and removes an unused Markdown parser from the portal service.
Enables the portal to process user-uploaded files by extracting their text content and injecting it into the AI agent's system prompt. This update includes logic to truncate file data to 50,000 characters to respect context limits and ensures the model uses the provided data to answer questions accurately.
Introduces a Retrieval-Augmented Generation (RAG) workflow that searches the vector store for relevant clinical concepts based on user prompts and constellation filters. This update adds dedicated services for context generation and vector search, refactors chat orchestration into a modular structure, and updates the AI agent prompt to utilize retrieved clinical definitions.
…and hierarchy data

Refactors the context generator to provide more detailed clinical concept descriptions for the AI agent. This update introduces a logical definition parser to extract EL++ axioms (necessary and sufficient conditions) from Tinkar graphs and implements taxonomic hierarchy resolution, replacing the previous template-based placeholder logic.
… prompt logging

Refactors the context generator to recursively extract and format deep semantic metadata from Tinkar entities, allowing the AI agent to access a more comprehensive view of clinical concepts. This update also introduces detailed logging of the complete prompt context for improved observability and removes the minimum search score threshold to increase the recall of retrieved clinical knowledge.
Introduces a query optimization step that uses an AI service to extract core medical concepts from user input before performing vector searches. This improves the relevance of retrieved clinical context by stripping away conversational filler and non-clinical terminology from the search terms.
Introduces a dependencyManagement block for LangChain4j and its community extensions. This change separates core and community version properties, updates the Lucene community extension version, and removes inline versioning from the main dependency list to ensure consistency across the project.
@aks8m aks8m self-assigned this Mar 23, 2026
@aks8m aks8m merged commit 6ea53fb into main Mar 23, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant