Implement Synix v0.9: multi-provider, concurrent, hybrid search

[marklubin]

Summary

• Module restructure: New layout (core/, build/, adapters/, cli/, search/) with backward-compat shims for old import paths
• Multi-provider LLM (S03): LLMClient wrapping openai.OpenAI(base_url=...) — works with any OpenAI-compatible API (OpenAI, Anthropic, Ollama, vLLM, DeepSeek)
• Concurrent execution (S04): ConcurrentExecutor with ThreadPool + Semaphore + exponential backoff. --concurrency/-j CLI flag
• Embeddings & hybrid search (S05): EmbeddingProvider with content-hash caching, HybridRetriever with keyword/semantic/hybrid modes, RRF fusion (k=60)
• Logging & observability (S01): SynixLogger with per-run JSONL file logs, RunLog/StepLog tracking LLM calls/tokens/cache hits, -v/-vv verbosity
• synix plan (S06): Dry-run showing what would build, with per-layer LLM call/token/cost estimates
• Shadow index swap (S07): Build to search_shadow.db, atomic os.replace() on success
• Artifact diffing (S08): diff_builds(), diff_artifact() with unified diff output
• synix verify (S09): 8 integrity checks including merge cross-customer contamination detection
• Text adapter (S10): YAML frontmatter, date inference, turn detection, adapter registry
• Merge transform: Jaccard similarity clustering with union-find, natural language constraints, cache key
• Search CLI extensions: --step (layer filter), --trace (provenance tree), --customer (metadata filter)
• Test infrastructure: Mock LLM server, 3 demo corpora (30/50/100 conversations), 477 tests passing

Test plan

• uv run pytest tests/ -v — 477 passed, 0 skipped, 0 failed (~20s)
• Unit tests cover all new modules (executor, LLM client, embeddings, retriever, diff, plan, verify, merge transform, text adapter, shadow index, logging, search CLI)
• Integration tests: full pipeline runs, incremental rebuilds, config changes
• E2E Demo 1 (personal): fresh build, search with --step/--trace, config swap, full cache hit (12 tests)
• E2E Demo 2 (startup): financial profiles, parallel pipeline paths, independent search indexes, cross-path diffing (18 tests)
• E2E Demo 3 (incident): merge contamination detection, provenance tracing, verify checks, incremental fix, collateral damage check (12 tests)

[marklubin]

Review Response

Accepted all P0 and P1 items. Working on fixes now.

P0 (adapter correctness):

1. ChatGPT linearization: switching to current_node path with first-child fallback
2. Role filtering: user/assistant only by default
3. Claude sender normalization: human → user
4. Claude timestamp: proper ISO parsing with fallback

P1 (perf + reliability):
5. Merge transform: pre-tokenize to eliminate O(n²) re-tokenization
6. Semantic search: in-memory embedding cache (load once per query session, not per query)
7. Single root-layer constraint: relaxed to allow multi-source pipelines
8. Atomic cache writes: temp file + fsync + rename pattern
9. Verify output: added fix_hint field with actionable suggestions
10. Thread safety: deep-copy config before passing to concurrent transform workers

Note on anthropic dep: reviewer was incorrect — anthropic IS used in llm_client.py for the native Anthropic provider path. Keeping as required dep for now; will consider making optional in a future PR.

Pushing fix commit shortly.