fix: OOM prevention with dynamic memory guard#65
Merged
Conversation
- Always apply LIMIT to SELECT/SHOW/DESCRIBE/EXPLAIN (was conditional) - Hard cap at 50,000 rows regardless of user settings - Fix rows_truncated detection to compare against capped_limit - Lower frontend virtualization threshold from 500 to 100 rows - Add truncation warning banner when results are capped Closes #25 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove hardcoded 50k row cap - Add sysinfo crate for process memory monitoring - MemoryGuard checks RSS every 1000 rows during fetch - Triggers when process hits 75% of system RAM - Gracefully stops fetch and returns accumulated rows with warning - Add OutOfMemory error variant to CoreError - Update frontend to show backend warnings - Truncation banner no longer mentions arbitrary 50k number
- Collapse nested if into single condition (clippy::collapsible_if) - Use is_multiple_of(1000) instead of % 1000 == 0 (clippy::manual_is_multiple_of) - Run cargo fmt for consistent formatting
- Changed MemoryGuard to monitor available system memory, not total - Trigger threshold: 95% of available memory consumed (5% buffer) - Previous logic used 75% of total memory which could allow crashes when other processes consumed RAM - New logic prevents runaway queries when system memory is critically low - Removed per-process RSS tracking; now monitors system-wide availability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EVWorth
force-pushed
the
fix/oom-large-tables
branch
from
07b1f1f to
834ae0e
Compare
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces hardcoded 50k row limit with a dynamic memory guard that monitors system-wide available memory.
Changes
MemoryGuard (executor.rs)
sysinfo::Systemas a struct field; callsrefresh_memory()on each check instead of creating a newSystemper invocationcheck(&mut self)setsself.triggered = trueinternally — removed separateset_triggered()method and eliminated the redundant two-liner at the call siteDeduplication (executor.rs)
build_select_result()helper shared by the normalEither::Leftpath and the OOM mid-stream recovery blockFrontend (ResultsGrid.tsx)
shouldVirtualizethreshold from 100 → 500 rows (change had no justification)warningsarray;rows_truncated: truealready shows the truncation banner — two banners for the same event was confusingTesting