Skip to content

fix: improve error handling for environment variable parsing#5560

Merged
yanghua merged 6 commits intolance-format:mainfrom
XuQianJin-Stars:fix/env-var-parsing-error-handling
Jan 16, 2026
Merged

fix: improve error handling for environment variable parsing#5560
yanghua merged 6 commits intolance-format:mainfrom
XuQianJin-Stars:fix/env-var-parsing-error-handling

Conversation

@XuQianJin-Stars
Copy link
Copy Markdown
Contributor

Replace unwrap() calls with proper error handling in environment variable parsing functions to prevent panics when invalid values are provided.

Changes:

  • get_default_batch_size(): Handle invalid LANCE_DEFAULT_BATCH_SIZE values
  • DEFAULT_FRAGMENT_READAHEAD: Handle invalid LANCE_DEFAULT_FRAGMENT_READAHEAD values
  • DEFAULT_XTR_OVERFETCH: Handle invalid LANCE_XTR_OVERFETCH values
  • DEFAULT_IO_BUFFER_SIZE: Handle invalid LANCE_DEFAULT_IO_BUFFER_SIZE values

When parsing fails, the functions now:

  1. Log a warning message with the invalid value and error
  2. Fall back to default values gracefully
  3. Prevent application crashes due to configuration errors

Added test_env_var_parsing() to verify the new error handling behavior.

@github-actions github-actions Bot added the bug Something isn't working label Dec 22, 2025
@XuQianJin-Stars XuQianJin-Stars force-pushed the fix/env-var-parsing-error-handling branch from aeb086d to 9f91118 Compare December 24, 2025 02:12
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@XuQianJin-Stars XuQianJin-Stars force-pushed the fix/env-var-parsing-error-handling branch 2 times, most recently from 24657a1 to 66eab2e Compare December 25, 2025 10:39
Copy link
Copy Markdown
Collaborator

@yanghua yanghua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments.

Comment thread rust/lance/src/dataset/scanner.rs Outdated
Ok(value) => Some(value),
Err(e) => {
log::warn!(
"Failed to parse {}='{}': {}. {}",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed to parse the environment variable {}='{}': {}, the default value is: {}.

Comment thread rust/lance/src/dataset/scanner.rs Outdated
pub(crate) const BATCH_SIZE_FALLBACK: usize = 8192;

/// Parse an environment variable as a specific type, logging a warning on parse failure.
fn parse_env_var<T: std::str::FromStr>(env_var_name: &str, default_hint: &str) -> Option<T>
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default_hint -> default_val?

Comment thread rust/lance/src/dataset/scanner.rs Outdated
std::env::var("LANCE_DEFAULT_BATCH_SIZE")
.map(|val| Some(val.parse().unwrap()))
.unwrap_or(None)
parse_env_var("LANCE_DEFAULT_BATCH_SIZE", "Using default.")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, here the second parameter we can just pass the string representing the value. WDYT?

Comment thread rust/lance/src/dataset/scanner.rs Outdated
.unwrap_or(None)
});
pub static DEFAULT_FRAGMENT_READAHEAD: LazyLock<Option<usize>> =
LazyLock::new(|| parse_env_var("LANCE_DEFAULT_FRAGMENT_READAHEAD", "Using default."));
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@XuQianJin-Stars XuQianJin-Stars force-pushed the fix/env-var-parsing-error-handling branch from 9794d17 to e2640bd Compare January 4, 2026 07:32
Comment thread rust/lance/src/dataset/scanner.rs Outdated
std::env::var("LANCE_DEFAULT_BATCH_SIZE")
.map(|val| Some(val.parse().unwrap()))
.unwrap_or(None)
parse_env_var("LANCE_DEFAULT_BATCH_SIZE", "8192")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8192 -> &BATCH_SIZE_FALLBACK.to_string()

Can we try it?

Copy link
Copy Markdown
Collaborator

@yanghua yanghua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for your patience!

@yanghua yanghua merged commit 3692fbe into lance-format:main Jan 16, 2026
26 of 28 checks passed
jackye1995 pushed a commit to jackye1995/lance that referenced this pull request Jan 21, 2026
…nce-format#5560)

Replace unwrap() calls with proper error handling in environment
variable parsing functions to prevent panics when invalid values are
provided.

Changes:
- get_default_batch_size(): Handle invalid LANCE_DEFAULT_BATCH_SIZE
values
- DEFAULT_FRAGMENT_READAHEAD: Handle invalid
LANCE_DEFAULT_FRAGMENT_READAHEAD values
- DEFAULT_XTR_OVERFETCH: Handle invalid LANCE_XTR_OVERFETCH values
- DEFAULT_IO_BUFFER_SIZE: Handle invalid LANCE_DEFAULT_IO_BUFFER_SIZE
values

When parsing fails, the functions now:
1. Log a warning message with the invalid value and error
2. Fall back to default values gracefully
3. Prevent application crashes due to configuration errors

Added test_env_var_parsing() to verify the new error handling behavior.
vivek-bharathan pushed a commit to vivek-bharathan/lance that referenced this pull request Feb 2, 2026
…nce-format#5560)

Replace unwrap() calls with proper error handling in environment
variable parsing functions to prevent panics when invalid values are
provided.

Changes:
- get_default_batch_size(): Handle invalid LANCE_DEFAULT_BATCH_SIZE
values
- DEFAULT_FRAGMENT_READAHEAD: Handle invalid
LANCE_DEFAULT_FRAGMENT_READAHEAD values
- DEFAULT_XTR_OVERFETCH: Handle invalid LANCE_XTR_OVERFETCH values
- DEFAULT_IO_BUFFER_SIZE: Handle invalid LANCE_DEFAULT_IO_BUFFER_SIZE
values

When parsing fails, the functions now:
1. Log a warning message with the invalid value and error
2. Fall back to default values gracefully
3. Prevent application crashes due to configuration errors

Added test_env_var_parsing() to verify the new error handling behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants