docs: add CLAUDE.md, architecture docs, and /sesh-mode skill by g-talbot · Pull Request #6247 · quickwit-oss/quickwit

g-talbot · 2026-03-30T18:43:18Z

Summary

Adds CLAUDE.md with development guide, coding standards, known pitfalls, and repo layout
Adds docs/internals architecture documentation: ADRs (4 decisions + 10 gap analyses), TLA+ formal specs (ParquetDataModel, SortSchema, TimeWindowedCompaction), verification guides, style references, and compaction architecture
Splits CLAUDE.md into universal repo context (auto-loaded) + opt-in /sesh-mode skill for verification-first workflow (TLA+, DST, Stateright)

Combines pomsky PRs #468 and #474.

Stacks on gtt/phase-31-execute (PR #6246).

The split logic (from PR #474)

Stays in CLAUDE.md	Moves to `/sesh-mode`
What CI already enforces	What requires human buy-in
Factual repo structure	Workflow sequencing
"Don't do X" (pitfalls)	"You must do X before Y" (process)
Coding conventions	Formal verification requirements
Defensive Rust patterns	TLA+, Stateright, DST

Test plan

Verify CLAUDE.md references correct paths for this repo
Verify docs/internals files are present and readable
/sesh-mode skill contains the full verification workflow
No code changes — documentation only

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…metadata

…ter-wiring

Pre-existing splits were serialized before the parquet_file field was added, so their JSON doesn't contain it. Adding #[serde(default)] makes deserialization fall back to empty string for old splits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ter-wiring

When the commit timeout fires and the accumulator contains only zero-column batches, union_fields is empty and concat_batches fails with "must either specify a row count or at least one column". Now flush_internal treats empty union_fields the same as empty pending_batches — resets state and returns None. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Resolve Cargo.lock/Cargo.toml merge conflicts - P1 (sort column lookup): Already addressed by sort fields tag_ prefix fix — sort field names now match Parquet column names - P2 (window_start at epoch 0): Remove time_range.start_secs > 0 guard so window_start is computed for all batches when window_duration > 0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Resolve writer.rs conflict: keep META-07 self-describing roundtrip test - P1 (create_timestamp serde): Add #[serde(default)] to StoredMetricsSplit.create_timestamp for backward-compatible reads of pre-existing file-backed index JSON - P1 (compaction window overlap): No change needed — Bound::Included vs Bound::Excluded already handles half-open interval semantics correctly, and the edge case (zero duration) is impossible - fields.rs: No change — Matt noted it resolves with wide schema rebase Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Resolve postgres.rs conflict: keep check_invariant! macros, add window_duration_secs consistency check - Group setup_dogstatsd_exporter + setup_invariant_recorder into single setup_metrics() function (fulmicoton-dd review) - Rename `id` to `invariant_id` in invariant_recorder (fulmicoton-dd review) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Reverts c8bf8d7, cafcac5, a088f53 — these are code changes (delete_metrics_splits error handling, doc comment tweaks) that don't belong in a docs-only PR. They will land in a separate PR. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This ADR contains company-specific information and should live in the private fork, not in the upstream quickwit-oss repo. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Rewrite CLAUDE.md as generic Quickwit AI development guide - Replace Quickhouse-Pomsky -> Quickwit branding across all docs - Replace "Datadog" observability references with generic "production observability" language - Remove "Husky (Datadog)" qualifier from gap docs (keep Husky citations — the blog post is public) - Generalize internal knowledge (query rate numbers, product-specific lateness guarantees) - Remove PomChi reference, private Google Doc link - Add docs/internals/UPSTREAM-CANDIDATES.md for pomsky tracking Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Remove all ClickHouse/ClickStack references from gap docs and ADRs (keep Prometheus, Mimir, InfluxDB, Husky as prior art) - Restore gap-005 Option C (compaction-time dedup) without ClickHouse citation - Mark /sesh-mode reference in CLAUDE.md as aspirational - Add aspirational items section to UPSTREAM-CANDIDATES.md tracking items described in docs but not yet implemented (TLA+ specs, DST, Kani, Bloodhound, performance baselines, benchmark binaries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mattmkim added 2 commits March 30, 2026 14:21

feat: replace fixed MetricDataPoint fields with dynamic tag HashMap

d892f23

feat: replace ParquetField enum with constants and dynamic validation

4e928fe

g-talbot requested review from fulmicoton-dd and mattmkim March 30, 2026 18:43

mattmkim added 7 commits March 30, 2026 14:44

feat: derive sort order and bloom filters from batch schema

d8d71ed

feat: union schema accumulation and schema-agnostic ingest validation

ad115bf

feat: dynamic column lookup in split writer

d0a995e

feat: remove ParquetSchema dependency from indexing actors

35c3942

refactor: deduplicate test batch helpers

33c4070

lint

7a5979f

Merge branch 'main' into matthew.kim/metrics-wide-schema

440631b

g-talbot force-pushed the gtt/docs-claude-md branch from 0ee18f5 to b077bf3 Compare March 31, 2026 20:41

g-talbot force-pushed the gtt/phase-31-execute branch from ae409ac to 3eb3316 Compare March 31, 2026 20:41

g-talbot force-pushed the gtt/docs-claude-md branch from 70a48be to d225cc7 Compare March 31, 2026 20:55

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from c4418dd to aa5ff45 Compare March 31, 2026 21:03

g-talbot force-pushed the gtt/docs-claude-md branch 2 times, most recently from 8d701de to bcdda05 Compare March 31, 2026 21:08

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from 142ddd6 to 972bd20 Compare March 31, 2026 21:26

g-talbot force-pushed the gtt/docs-claude-md branch from bcdda05 to 0fd7087 Compare March 31, 2026 21:26

Merge branch 'main' into matthew.kim/metrics-wide-schema

5eae799

g-talbot force-pushed the gtt/phase-31-execute branch 2 times, most recently from 73c90c4 to 44bd6bb Compare March 31, 2026 21:31

g-talbot force-pushed the gtt/docs-claude-md branch from 468ac45 to 1e5722d Compare March 31, 2026 21:31

g-talbot force-pushed the gtt/phase-31-execute branch from 44bd6bb to add1902 Compare March 31, 2026 21:33

g-talbot force-pushed the gtt/docs-claude-md branch from 1e5722d to 9b6c961 Compare March 31, 2026 21:33

g-talbot force-pushed the gtt/phase-31-execute branch from add1902 to d88f67f Compare March 31, 2026 21:40

g-talbot force-pushed the gtt/docs-claude-md branch 2 times, most recently from 06a6296 to 8f1cbde Compare March 31, 2026 21:50

g-talbot and others added 23 commits April 8, 2026 07:38

style: rustfmt long match arm in default_sort_fields

4481bef

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Merge branch 'gtt/phase-31-sort-schema' into gtt/phase-31-compaction-…

64c5d5f

…metadata

Merge branch 'gtt/phase-31-compaction-metadata' into gtt/phase-31-wri…

a8bf948

…ter-wiring

Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore

caa9c3e

Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute

42ab5c2

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

35dd926

Merge branch 'gtt/phase-31-compaction-metadata' into gtt/phase-31-wri…

b968085

…ter-wiring

Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore

1c99ddd

Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute

bbfef12

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

ca192e0

Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore

e8c6be4

Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute

71586e4

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

19d67a8

Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute

7dbd1ef

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

6f1bf5b

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

4f118b3

style: rustfmt check_invariant macro argument

501f014

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md

cf37871

Base automatically changed from gtt/phase-31-execute to main April 8, 2026 19:02

Merge branch 'main' into gtt/docs-claude-md

03cf20f

g-talbot mentioned this pull request Apr 8, 2026

fix: improve delete_metrics_splits error handling and doc comments #6280

Merged

2 tasks

g-talbot and others added 4 commits April 8, 2026 16:22

remove ADR-004 from upstream PR — moving to DataDog/pomsky

86e0881

This ADR contains company-specific information and should live in the private fork, not in the upstream quickwit-oss repo. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add CLAUDE.md, architecture docs, and /sesh-mode skill#6247

docs: add CLAUDE.md, architecture docs, and /sesh-mode skill#6247
g-talbot wants to merge 73 commits intomainfrom
gtt/docs-claude-md

g-talbot commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

g-talbot commented Mar 30, 2026

Summary

The split logic (from PR #474)

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants