Skip to content

fix(streaming): add buffer size limit to StreamProcessor #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
echobt wants to merge 1 commit into from
Closed

fix(streaming): add buffer size limit to StreamProcessor #84

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion src/cortex-engine/src/streaming.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ use futures::Stream;
use serde::{Deserialize, Serialize};
use tokio::sync::mpsc;

/// Maximum number of events to buffer before dropping old ones.
/// Prevents unbounded memory growth if drain_events() is not called regularly.
const MAX_BUFFER_SIZE: usize = 10_000;

/// Token usage for streaming.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct StreamTokenUsage {
Expand Down Expand Up @@ -213,7 +217,7 @@ impl StreamProcessor {
Self {
state: StreamState::Idle,
content: StreamContent::new(),
buffer: VecDeque::new(),
buffer: VecDeque::with_capacity(1024), // Pre-allocate reasonable capacity
start_time: None,
first_token_time: None,
last_event_time: None,
Expand Down Expand Up @@ -284,6 +288,10 @@ impl StreamProcessor {
}
}

// Enforce buffer size limit to prevent unbounded memory growth
if self.buffer.len() >= MAX_BUFFER_SIZE {
self.buffer.pop_front();
}
self.buffer.push_back(event);
Comment on lines +291 to 295
Copy link

@greptile-apps greptile-apps bot Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider logging when dropping events - silent data loss could be hard to debug

Suggested change
// Enforce buffer size limit to prevent unbounded memory growth
if self.buffer.len() >= MAX_BUFFER_SIZE {
self.buffer.pop_front();
}
self.buffer.push_back(event);
// Enforce buffer size limit to prevent unbounded memory growth
if self.buffer.len() >= MAX_BUFFER_SIZE {
self.buffer.pop_front();
// Consider: log::warn!("StreamProcessor buffer full, dropping oldest event");
}

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI 
This is a comment left during a code review.
Path: src/cortex-engine/src/streaming.rs
Line: 291:295

Comment:
Consider logging when dropping events - silent data loss could be hard to debug

```suggestion
        // Enforce buffer size limit to prevent unbounded memory growth
        if self.buffer.len() >= MAX_BUFFER_SIZE {
            self.buffer.pop_front();
            // Consider: log::warn!("StreamProcessor buffer full, dropping oldest event");
        }
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

}

Expand Down