Skip to content

fix(tail): reduce ring buffer and byte-mode buffer caps from 64/32 MiB to 5 MiB#148

Merged
thieman merged 3 commits intomainfrom
thieman/reduce-tail-memory-limits
Mar 26, 2026
Merged

fix(tail): reduce ring buffer and byte-mode buffer caps from 64/32 MiB to 5 MiB#148
thieman merged 3 commits intomainfrom
thieman/reduce-tail-memory-limits

Conversation

@thieman
Copy link
Copy Markdown
Collaborator

@thieman thieman commented Mar 25, 2026

Problem

tail had two memory limits well above 10 MiB per invocation:

M-1: Ring buffer cap was 64 MiB

MaxRingBytes = 64 << 20 — a single tail -n <large> on a file with long lines could hold 64 MiB of live data in the ring buffer simultaneously.

M-2: Byte-mode circular buffer cap was 32 MiB

tail -c <large> allocated a circular buffer capped at 32 MiB.

Both limits are far above the project's 10 MiB per-invocation guideline.

Fix

Both caps reduced to 5 MiB (5 << 20). MaxTotalReadBytes (the read cap for infinite streams like pipes) is unchanged — that controls how much total data is consumed from an infinite source before bailing, which is a different concern from how much live data is held in memory.

Test plan

  • tail -n 100 file still works correctly
  • tail -c 1000 file still works correctly
  • tail -n 1000000 file correctly returns the last lines within the 5 MiB cap
  • Existing tail scenario tests pass

🤖 Generated with Claude Code

thieman and others added 2 commits March 25, 2026 16:12
MaxRingBytes: 64 MiB → 5 MiB
Byte-mode circular buffer: 32 MiB → 5 MiB

A single tail invocation could previously hold up to 64 MiB of live
data in the ring buffer, and up to 32 MiB in byte mode. These limits
are reduced to 5 MiB to keep single-command memory usage bounded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@thieman
Copy link
Copy Markdown
Collaborator Author

thieman commented Mar 25, 2026

@codex review this PR

Copy link
Copy Markdown
Collaborator Author

@thieman thieman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-Review Summary — PR #148: Reduce tail memory limits to 5 MiB

Overall assessment: safe to merge.

This PR reduces two memory caps in the tail builtin:

  • MaxRingBytes (line-mode ring buffer): 64 MiB → 5 MiB
  • MaxBytesBuffer (byte-mode circular buffer): 32 MiB → 5 MiB
  • Bonus cleanup: replaces a 3-line if/else with max() builtin (semantically identical)

All constants, package-level doc comments, and inline function comments are updated consistently. No security concerns — this change is strictly a memory limit reduction. Sandbox integrity is unaffected.

Findings summary

# Priority File Finding
1 P3 Badge builtins/tail/builtin_tail_pentest_test.go:447 Stale comment says "32 MiB" after MaxBytesBuffer was reduced to 5 MiB
2 P3 Badge builtins/tail/tail_test.go No test covers MaxRingBytes (5 MiB ring-byte cap) being exceeded
3 P3 Badge builtins/tail/tail_test.go No test covers MaxBytesBuffer overflow error path (byte-mode buffer limit)

Coverage summary

Code path Scenario test Go test Status
MaxRingBytes constant reduction No direct test
MaxBytesBuffer constant reduction No direct test
max() refactor (suppressPos) tests/scenarios/cmd/tail/headers/ tail_test.go Covered via header behavior tests
Ring line-count overflow error tail_test.go:TestTailRingOverflowError Covered
Byte-mode overflow error Missing

Positive observations

  • Constants, doc comments, and inline comments are all updated consistently — no stale references remain in tail.go itself.
  • The max() refactor is a clean simplification with identical semantics.
  • Memory reduction is in the right direction for the project's safety goals.

Comment thread builtins/tail/tail.go
// MaxBytesBuffer is the maximum size of the circular byte buffer used in
// last-N-bytes mode.
const MaxBytesBuffer = 32 << 20 // 32 MiB
const MaxBytesBuffer = 5 << 20 // 5 MiB
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge MaxBytesBuffer comment could mention the overflow scenario

The constant comment describes the buffer's role correctly, but it doesn't mention that when count > MaxBytesBuffer and input exceeds MaxBytesBuffer, readLastBytes returns an error. Consider adding a note matching the style of MaxRingBytes above:

Suggested change
const MaxBytesBuffer = 5 << 20 // 5 MiB
// MaxBytesBuffer is the maximum size of the circular byte buffer used in
// last-N-bytes mode. If the input exceeds this size and the requested count
// also exceeds it, tail returns an error rather than silently truncating.
const MaxBytesBuffer = 5 << 20 // 5 MiB

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@thieman thieman marked this pull request as ready for review March 26, 2026 18:23
@thieman thieman added this pull request to the merge queue Mar 26, 2026
Merged via the queue into main with commit 52a48bf Mar 26, 2026
34 checks passed
@thieman thieman deleted the thieman/reduce-tail-memory-limits branch March 26, 2026 18:55
thieman added a commit that referenced this pull request Mar 27, 2026
…B to 5 MiB (#148)

* fix(tail): reduce ring buffer and byte-mode buffer caps to 5 MiB

MaxRingBytes: 64 MiB → 5 MiB
Byte-mode circular buffer: 32 MiB → 5 MiB

A single tail invocation could previously hold up to 64 MiB of live
data in the ring buffer, and up to 32 MiB in byte mode. These limits
are reduced to 5 MiB to keep single-command memory usage bounded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style(tail): use builtin max() per modernize linter

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants