Refactor/introduce evaluator pattern and refactor base llm#10
Merged
oshorefueled merged 3 commits intoTRocket-Labs:mainfrom Nov 14, 2025
Conversation
- Add Evaluator interface defining contract for all evaluators - Create EvaluatorRegistry with factory pattern for evaluator creation - Implement BaseLLMEvaluator wrapping existing evaluation logic - Add 'evaluator' field to prompt metadata schema (optional, defaults to 'base-llm') - Update main orchestrator to route prompts through evaluator registry - Maintain full backward compatibility with existing prompts This establishes the foundation for multiple evaluation strategies without modifying core orchestration logic. Existing prompts continue to work unchanged using BaseLLMEvaluator.
- Remove verbose JSDoc blocks from evaluator files - Keep only inline comments for complex logic - Maintain clean, readable code without over-documentation - Params and descriptions belong in spec docs, not source code
oshorefueled
approved these changes
Nov 14, 2025
oshorefueled
added a commit
that referenced
this pull request
Feb 11, 2026
- Resolved conflict in directive-loader.ts by prioritizing simple_zeroconfig - Added fix field instruction from main as item #10
oshorefueled
added a commit
that referenced
this pull request
Mar 2, 2026
…r-pattern-and-refactor-base-llm Refactor/introduce evaluator pattern and refactor base llm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces the evaluator pattern to VectorLint, establishing a flexible architecture for supporting multiple evaluation strategies. This refactor extracts the existing LLM-based evaluation logic into a clean, extensible pattern while maintaining full backward compatibility.