feat: Implement caching for rule evaluations

[dkargatzis]

Overview

Currently, the rule evaluation system evaluates all rules from scratch on every event, which can be inefficient and cause performance issues. We need to implement an intelligent caching system that only re-validates rules when relevant data has changed.

Current Problem

• Every PR event triggers full re-evaluation of all applicable rules
• No caching means repeated API calls and LLM evaluations
• Performance degrades with frequent events (labels, comments, etc.)
• Users experience delays and potential rate limiting

Requirements

1. Cache Implementation

• Create a cache system that stores rule evaluation results
• Cache key should include: rule_id + event_type + event_data_hash
• TTL: 5 minutes (rules can change, so don't cache too long)
• Max cache size: 1000 entries

2. Change Detection System

• Detect which fields in the event data have changed between evaluations
• Key fields to monitor:
  • pr_title, pr_body, pr_state
  • pr_labels, pr_assignees, pr_requested_reviewers
  • pr_reviews, pr_reviews_count
  • pr_files, pr_changed_files
  • commits, current_time
  • deployment_environment, deployment_ref

3. Rule Dependency Mapping

• Define which fields each rule type depends on:
  • required_labels → depends on pr_labels
  • min_approvals → depends on pr_reviews, pr_reviews_count
  • title_pattern → depends on pr_title
  • min_description_length → depends on pr_body
  • file-size-limit → depends on pr_files
  • no-weekend-merges → depends on current_time
  • etc.

4. Sophisticated Re-validation Logic

• Only re-validate rules when their dependent fields have changed
• If no relevant fields changed, use cached result
• If relevant fields changed, re-evaluate and update cache
• Handle cache misses gracefully (evaluate and cache)

5. Cache Management

• Implement cache invalidation on rule configuration changes
• Provide cache statistics and monitoring
• Handle cache eviction when full
• Support manual cache clearing

Technical Implementation

Files to Modify/Create:

1. src/agents/engine_agent/cache.py - Cache implementation
2. src/agents/engine_agent/nodes.py - Integrate cache into evaluation logic
3. src/agents/engine_agent/agent.py - Update agent to use cache

Key Functions to Implement:

class RuleEvaluationCacheManager:
    def get_cached_result(self, rule_id: str, event_type: str, event_data: Dict[str, Any]) -> Optional[Dict[str, Any]]
    def cache_result(self, rule_id: str, event_type: str, event_data: Dict[str, Any], result: Dict[str, Any]) -> None
    def get_rules_to_revalidate(self, rule_ids: List[str], changed_fields: List[str]) -> List[str]
    def _get_changed_fields(self, old_event_data: Dict[str, Any], new_event_data: Dict[str, Any]) -> List[str]
    def _get_rule_dependencies(self, rule_id: str) -> List[str]

Integration Points:

1. Event Processors: Pass previous event data to detect changes
2. Rule Engine: Check cache before evaluation, store results after
3. Logging: Track cache hits/misses for performance monitoring

Success Criteria

• Cache reduces API calls by 70%+ for repeated events
• Performance improvement of 50%+ for label/comment events
• No stale violation detections (cache invalidation works correctly)
• Cache statistics are logged and monitored
• Graceful handling of cache misses and errors
• No breaking changes to existing functionality

Testing

• Test with PR events that only change labels (should use cache)
• Test with PR events that change title (should re-validate title-related rules)
• Test with deployment events (different event types)
• Test cache invalidation when rules change
• Test performance under load

Notes

• Start with simple caching, then add intelligent re-validation
• Ensure cache doesn't interfere with real-time rule updates
• Consider using in-memory caching only, Redis or other persistent solutions are out of the scope of this project
• Monitor memory usage and cache hit rates