feat: streaming output, orjson, and memory-efficient search rendering

[tomaz-lc]

Details

Adds streaming output support across all search command paths, replacing the previous approach that buffered all results in memory before rendering. Output formats that support it (JSONL, JSON, expand, table) now stream results with constant or near-constant memory usage, making large searches (100K+ events) feasible on memory-constrained VMs.

Also adds orjson as an optional-but-preferred JSON backend (~3-10x faster than stdlib json), with automatic fallback to stdlib if orjson is unavailable.

Output behavior by path and format

Path	JSONL	JSON	expand	table	CSV/YAML
search run (no checkpoint)	Stream, O(1)	Stream, O(1)	Stream, O(1)	Stream (sampled widths), O(sample)	Buffer, O(N)
search run --checkpoint	Stream from file, O(1)	Stream from file, O(1)	Stream from file, O(1)	2-pass stream from file, O(cols)	Buffer from file, O(N)
search run --resume	Stream from file, O(1)	Stream from file, O(1)	Stream from file, O(1)	2-pass stream from file, O(cols)	Buffer from file, O(N)
search checkpoint-show	Stream from file, O(1)	Stream from file, O(1)	Stream from file, O(1)	2-pass stream from file, O(cols)	Buffer from file, O(N)
search saved-run	Stream, O(1)	Stream, O(1)	Stream, O(1)	Stream (sampled widths), O(sample)	Buffer, O(N)

Peak memory by format

Format	Without checkpoint	With checkpoint
JSONL	O(1) - one result at a time	O(1) - one line at a time
JSON	O(1) - streaming [, items, ]	O(1) - streaming from file
expand	O(1) - one event block at a time	O(1) - one event block at a time
table	O(sample_pages * cols) - buffers ~3 pages for column width sampling, then streams	O(cols) - two-pass file scan: pass 1 computes exact column widths, pass 2 streams rows
CSV	O(N) - inherent to format	O(N) - loaded from file
YAML	O(N) - inherent to format	O(N) - loaded from file

Memory profile - live validation

Measured during a real 30-day --resume --checkpoint search over a production org. The checkpoint file grew to 632 MB (484 result pages) while RSS stayed flat at ~138 MB, confirming O(1) memory for the checkpoint write path.

Elapsed	RSS (MB)	Checkpoint File (MB)	Result Pages
0:00	85	113	80
1:30	87	161	128
3:00	100	208	164
4:30	116	270	200
6:00	119	321	252
7:30	142	413	304
9:00	141	498	368
10:30	141	587	444
14:46	138	632	484

Key observations:

• RSS rose from 85 MB to ~140 MB in the first ~7 minutes (Python allocator warm-up, HTTP connection pools, orjson parser buffers), then stabilized completely for the remaining 7+ minutes.
• Checkpoint file grew linearly from 113 MB to 632 MB (519 MB of new data written) with zero corresponding RSS growth.
• At 0.4% of 32 GB system memory, this can comfortably run on 512 MB VMs.

Warnings

Three independent warnings based on search time range:

1. Billing cost notice (>30 days): Notifies that data older than 30 days may incur additional charges. Shows the exact search estimate command to check costs.
2. Memory warning (>7 days, CSV/YAML only): Warns that all results are buffered in memory. Suggests streaming formats or --checkpoint.
3. Resumability warning (>14 days, all formats): Recommends --checkpoint so interrupted searches can be resumed.

Cost notice (>30 day search)

$ limacharlie search run --query "* | * | *" --start 1771065591 --end 1773657591 --output jsonl
Notice: this search spans 30 days. Searches over data older than 30 days may incur additional costs.
To estimate the cost before running:

  limacharlie search estimate \
    --query '* | * | *' \
    --start 1771065591 \
    --end 1773657591

Validate/estimate exit codes and output

search validate and search estimate now exit with code 1 when the server returns an error (e.g. invalid query syntax). Previously they always exited 0.

For table output, stats and estimatedPrice fields are flattened into individual columns (stats.bytesScanned, price.value, etc.) so full values are visible without truncation.

Validate: invalid query (exit code 1)

$ limacharlie search validate --query "* | * | * x" --output table
Field                     Value
------------------------  -------------------------------------------------------------------
query                     * | * | * x
startTime                 1773667570
endTime                   1773753970
error                     failed to transcode query: 1:56 (55): no match found, expected: ...
stats.bytesScanned        0
stats.eventsScanned       0
stats.eventsMatched       0
stats.eventsProcessed     0
stats.rulesEvaluated      0
stats.walltime            0
price.value               0
price.currency            USD cents

$ echo $?
1

Validate: valid query (exit code 0)

$ limacharlie search validate --query "* | * | *" --output table
Field                     Value
------------------------  -------------------------------------------------------------------
query                     * | * | *
startTime                 1773667570
endTime                   1773753970
stats.bytesScanned        1234567
stats.eventsScanned       50000
stats.eventsMatched       0
stats.eventsProcessed     0
stats.rulesEvaluated      0
stats.walltime            0
price.value               42
price.currency            USD cents

$ echo $?
0

Estimate: JSON output (preserves nested structure)

$ limacharlie search estimate --query "* | * | *" --start 1700000000 --end 1700086400 --output json
{
  "query": "* | * | *",
  "stats": {
    "bytesScanned": 1234567,
    "eventsScanned": 50000,
    "eventsMatched": 0,
    "walltime": 0
  },
  "estimatedPrice": {
    "value": 42,
    "currency": "USD cents"
  }
}

-h/--help flag fix

The -h short flag for help was not wired up at the CLI root level. This affected all commands and subcommands. Added context_settings={"help_option_names": ["-h", "--help"]} to the root group, which propagates to all subcommands.

Missing help/description strings for search subcommands

Added missing Click docstrings for: validate, estimate, saved-get, saved-create, saved-delete, saved-run.

Checkpoints list improvements

• Added size column showing human-readable file size on disk
• Moved created column to first position
• Sorted by created timestamp descending (most recent first)

Changes

• Streaming output functions: _stream_search_output() handles JSONL, JSON, expand, and table formats without buffering. Returns False for CSV/YAML/raw so the caller can fall back to list().
• Two-pass table renderer: _stream_table_from_file() for checkpoint paths - pass 1 scans the JSONL file to compute exact column widths (O(cols) memory), pass 2 streams rows with the computed layout.
• Sample-based table streaming: _stream_table_events() for live search paths - buffers first ~3 pages to determine column widths, then streams remaining rows.
• orjson integration: limacharlie/json_compat.py provides dumps, loads, dumps_pretty using orjson when available, stdlib json fallback.
• Billing cost notice: _warn_cost_if_over_30_days() warns when search spans >30 days and shows the search estimate command. Threshold matches server-side billing logic (replay: 31d, insight-go: 30d with >).
• Validate/estimate fixes: Non-zero exit code on error response. Table output flattens stats/estimatedPrice into individual columns via _output_validate_or_estimate().
• Two-tier warnings: Memory buffering warning (>7d, CSV/YAML only) and checkpoint resumability recommendation (>14d, all formats).
• CLI help fixes: Added -h as help short flag (affects all commands/subcommands); added missing docstrings for search subcommands.
• Checkpoints list: Added file size column, reordered columns (created first), sorted by created descending.
• --ai-help updated: Added output format documentation to search run explain text covering streaming vs buffered behavior, --expand, --raw, and memory guidance.
• Tests: 370 tests passing across related test files. New test_search_helpers.py with 144 unit tests covering all helper functions, streaming, warnings, cost notice, validate/estimate exit codes, and output formatting.

Blast radius / isolation

• Affected: All search output paths (search run, search run --checkpoint, search run --resume, search checkpoint-show, search saved-run), search validate, search estimate, search help text, checkpoints list display, CLI -h flag.
• NOT affected: Non-search CLI commands (except -h fix which benefits all), SDK classes, authentication, config.
• Backward compatible: Output content is identical, only the internal buffering strategy changed. Users see the same results. Validate/estimate exit code change is a bug fix (was always 0, now 1 on error).

Performance characteristics

• JSONL/JSON/expand output: memory drops from O(N) to O(1) for all search paths.
• Table output (checkpoint): memory drops from O(N) to O(cols). Two file passes add ~50% wall time vs single pass, but avoids OOM on large result sets.
• Table output (live search): memory is O(sample_pages * cols) instead of O(N). Column widths may be slightly off for late pages (sampled, not exact).
• orjson gives ~3-10x faster JSON serialization/deserialization across all paths.
• Live validation: 30-day checkpoint resume search - RSS stabilized at 138 MB while checkpoint file grew to 632 MB (484 pages). See memory profile table above.

Notable contracts / APIs

• No wire format or API contract changes.
• json_compat module is internal, not part of the public SDK API.
• orjson is added as a dependency in pyproject.toml but the fallback ensures backward compatibility if it cannot be installed.
• Validate/estimate exit code change: was 0, now 1 when response contains error field. This is a correctness fix.

Test plan

• Unit tests for billing cost notice (12 unit + 6 CLI tests in TestWarnCostIfOver30Days, TestCostWarningCli)
• Unit tests for validate/estimate exit codes (9 tests in TestValidateEstimateExitCode)
• Unit tests for validate/estimate table output (6 tests in TestOutputValidateOrEstimate)
• Unit tests for memory buffering warning (8 tests in TestLargeTimeRangeWarning)
• Unit tests for checkpoint resumability recommendation (5 tests in TestCheckpointRecommendWarning)
• Unit tests for streaming output correctness (JSONL, JSON, expand, table)
• Unit tests for search helper functions (144 tests in test_search_helpers.py)
• Manual test: search run --resume --checkpoint with 30-day range - RSS stable at 138 MB while file grew to 632 MB
• Manual test: search validate with invalid query returns exit code 1
• Manual test: search estimate with >30 day range shows cost notice

🤖 Generated with Claude Code

[tomaz-lc]

Keep in mind this only applies for source builds - orjson is a well maintained library and they ship pre-built wheels for all most common platforms (Linux, OS X, Windows) and architectures (x86, arm).