Skip to content

fix(tail): reduce ring buffer and byte-mode buffer caps from 64/32 MiB to 5 MiB #148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thieman merged 3 commits into from
Mar 26, 2026
Merged

fix(tail): reduce ring buffer and byte-mode buffer caps from 64/32 MiB to 5 MiB #148

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8b7987a
fix(tail): reduce ring buffer and byte-mode buffer caps to 5 MiB
thieman Mar 25, 2026
9ad6082
style(tail): use builtin max() per modernize linter
thieman Mar 25, 2026
2c345a8
Merge branch 'main' into thieman/reduce-tail-memory-limits
thieman Mar 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 7 additions & 10 deletions builtins/tail/tail.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
// Line mode uses a ring buffer of size min(N, MaxRingLines) slots. Each slot
// holds one line; lines exceeding MaxLineBytes cause a scanner error. The
// ring buffer's total memory footprint is additionally capped at MaxRingBytes
// (64 MiB). If the input has more lines than the ring can hold and N exceeds
// (5 MiB). If the input has more lines than the ring can hold and N exceeds
// MaxRingLines, tail returns an error rather than silently truncating output.
//
// Byte mode uses a circular buffer of size min(N, MaxBytesBuffer). If the
Expand Down Expand Up @@ -104,12 +104,12 @@ const MaxRingLines = 100_000
// MaxRingBytes is the maximum total bytes the ring buffer may hold at any
// one time. Without this cap, MaxRingLines (100 000) × MaxLineBytes (1 MiB)
// yields a worst-case memory envelope of ~97.6 GiB. This constant reduces
// the bound to 64 MiB.
const MaxRingBytes = 64 << 20 // 64 MiB
// the bound to 5 MiB.
const MaxRingBytes = 5 << 20 // 5 MiB

// MaxBytesBuffer is the maximum size of the circular byte buffer used in
// last-N-bytes mode.
const MaxBytesBuffer = 32 << 20 // 32 MiB
const MaxBytesBuffer = 5 << 20 // 5 MiB
Copy link
Copy Markdown
Collaborator Author

@thieman thieman Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge MaxBytesBuffer comment could mention the overflow scenario

The constant comment describes the buffer's role correctly, but it doesn't mention that when count > MaxBytesBuffer and input exceeds MaxBytesBuffer, readLastBytes returns an error. Consider adding a note matching the style of MaxRingBytes above:

Suggested change
const MaxBytesBuffer = 5 << 20 // 5 MiB
// MaxBytesBuffer is the maximum size of the circular byte buffer used in
// last-N-bytes mode. If the input exceeds this size and the requested count
// also exceeds it, tail returns an error rather than silently truncating.
const MaxBytesBuffer = 5 << 20 // 5 MiB


// MaxTotalReadBytes is the maximum total bytes tail will consume from a
// single input source. Both last-N-lines and last-N-bytes modes must read
Expand Down Expand Up @@ -201,10 +201,7 @@ func registerFlags(fs *builtins.FlagSet) builtins.HandlerFunc {

// Determine header printing using last-flag-wins: the highest pos among
// quiet/silent (suppress) vs verbose (force) controls the outcome.
suppressPos := quietFlag.pos
if silentFlag.pos > suppressPos {
suppressPos = silentFlag.pos
}
suppressPos := max(quietFlag.pos, silentFlag.pos)
printHeaders := len(files) > 1
if verboseFlag.pos > suppressPos {
printHeaders = true
Expand Down Expand Up @@ -432,11 +429,11 @@ func readLastBytes(ctx context.Context, callCtx *builtins.CallContext, r io.Read
}

// Allocate the circular buffer eagerly. bufSize is capped at MaxBytesBuffer
// (32 MiB), so this allocation is bounded regardless of the user-supplied
// (5 MiB), so this allocation is bounded regardless of the user-supplied
// count value.
bufSize := MaxBytesBuffer
if count < MaxBytesBuffer {
bufSize = int(count) // count < MaxBytesBuffer (32 MiB) fits safely in int
bufSize = int(count) // count < MaxBytesBuffer (5 MiB) fits safely in int
}
circ := make([]byte, bufSize)
var totalWritten int64
Expand Down
Loading