Enhance tool registry with new categories and distillation improvements

[fajarhide]

PR Auto Describe

Summary

This PR overhauls Omni's filter system, adds dedicated output distillers for database, security, and VCS tools, refines CLI UX for omni doctor/omni learn, and expands tool profile support for modern development workflows.

---

Key Changes

1. Filter Tooling: Rewrote filter loading logic with auto-repair, warning tracking, and updated omni doctor output
2. New Distillers: Added support for database, security scanning, and GitHub/VCS CLI tools
3. CLI Refinements: Improved omni learn test reporting, queue auto-clearing, and omni stats help text
4. Pipeline Updates: Expanded tool profile resolution and added chain-aware command profiling

---

Detailed Breakdown

CLI Commands

• omni doctor: Refactored filter source tracking, replaced simple .bak renaming of broken user filters with auto-repair, fixed project filter trust validation, and added capped aggregated warning display for filter loading issues.
• omni learn: Split test results into built-in vs user filter categories, redesigned candidate table formatting, auto-clears the learn queue after successful pattern application, added a troubleshooting tip for learned filter failures, and improved pattern preview truncation.
• omni stats: Updated help text to mention the --all-commands flag for full command listings.

Filter System Rework

• Updated toml_filter module: Made match_command optional in TOML config, added LoadReport struct to bundle loaded filters + loading warnings, added try_repair_file to auto-fix common broken TOML issues (missing schema version, trailing whitespace, dangerous catch-all patterns).
• Rewrote all filter loading functions (load_from_file, load_from_dir, load_embedded_filters) to return LoadReport instead of raw filter vectors.
• Updated get_filters_by_source to return LoadReport tuples for built-in, user, and local filters.

New Distillers & Tooling

• Added 3 new distillers:
  • DatabaseDistiller: Parses PostgreSQL/MySQL/Redis CLI output, including errors and query results
  • SecurityDistiller: Summarizes vulnerability findings from semgrep/trivy/snyk/bandit
  • VcsDistiller: Truncates long output for GitHub/GitLab CLI tools
• Expanded distill_with_command to support new tool categories: database, security, VCS, Java/Flutter, monorepos, cloud infra, and CI/CD tools.
• Updated existing distillers (jsts, cloud, test) to use case-insensitive matching and fix regex edge cases.

Pipeline & Registry

• Rewrote resolve_profile to add cargo subcommand awareness (clippy, test, bench), and added profiles for database, Java, Flutter, cloud, and security tools.
• Added resolve_profile_for_chain to select the most specific tool profile for multi-command shell pipelines.
• Updated post-tool hook to use the new chain-aware profile lookup.

Minor Fixes

• Fixed cloud distillers (Docker/Terraform) to use case-insensitive matching for step/plan/apply lines
• Fixed test distiller to catch more failure patterns
• Fixed session learn module to safely generate TOML with escaped regex and handle numeric placeholders correctly
• Updated pytest snapshot test output

---

Notes

• Added new test fixtures and snapshot tests for the new database and security distillers
• Added 8 new media assets for documentation/UI updates

---

Breaking Changes

1. Filter Loading API: All toml_filter loading functions now return LoadReport instead of Vec<TomlFilter>, and get_filters_by_source now returns tuples of LoadReport instead of raw filter vectors.
2. TOML Config: match_command is now optional in filter TOML files; filters without valid non-empty match_command will be skipped with warnings instead of failing to load.
3. Profile Resolution: Post-tool hooks now use chain-aware profile lookup via resolve_profile_for_chain instead of direct resolve_profile for single commands, changing how multi-command pipelines are profiled.
4. Cargo commands now have specific tool profiles based on their subcommand, replacing the previous generic build tool profile.

Last updated: 2026-04-14 09:41:58