⚡️ Speed up function `_sanitize_query_string` by 1,011% in PR #11934 (`temp-branch`) by codeflash-ai[bot] · Pull Request #11935 · langflow-ai/langflow

codeflash-ai · 2026-02-27T09:22:49Z

⚡️ This pull request contains optimizations for PR #11934

If you approve this dependent PR, these changes will be merged into the original PR branch temp-branch.

This PR will be automatically closed if the original PR is merged.

📄 1,011% (10.11x) speedup for `_sanitize_query_string` in `src/backend/base/langflow/api/v1/traces.py`

⏱️ Runtime : 26.0 milliseconds → 2.34 milliseconds (best of 156 runs)

📝 Explanation and details

Brief: The optimized version replaces the Python-level per-character filter ("".join(ch for ch in value if " " <= ch <= "~")) with a single, precompiled regular-expression substitution that removes characters outside the printable ASCII range. This moves the heavy work into C (the regex engine) and avoids Python bytecode overhead per character, producing the ~10x runtime improvement observed (26.0 ms → 2.34 ms).

What changed

Replaced generator + join + per-character Python comparisons with a precompiled regex: _re_non_printable = re.compile(r"[^ -~]") and used _re_non_printable.sub("", value).
The regex object is compiled once at module import (module-global), so repeated calls reuse the compiled pattern.

Key reasons this is faster

Avoids per-character Python interpretation: the original generator expression executes Python bytecode and creates Python-level objects/operations for every character in the string (the line profiler shows that line dominated the runtime). The regex substitution runs in optimized C loops and reduces per-character interpreter overhead drastically.
Single C-level pass: re.sub performs the filtering in C and is memory/cache-friendly compared to many small Python operations.
One-time compile cost: precompiling the pattern removes the cost of parsing/compiling the regex on every call.

Behavior & correctness considerations

Semantics are preserved: both implementations keep characters with codepoints in the inclusive range [32 (space), 126 (~)] and then call .strip() and slice. Edge cases (space-only -> becomes empty after strip, non-str inputs raise TypeError) remain the same.
Unicode behavior matches the original character-comparison approach because both operate on Python str codepoints; non-ASCII characters outside the [32..126] range are removed.
The precompiled regex adds a tiny import-time cost but pays off for repeated calls.

Profiling evidence

Line profiler shows the generator/join was the dominant cost in the original implementation; after the change the regex substitution line is the main cost but its absolute time is much smaller. Overall runtime dropped from 26.0 ms to 2.34 ms (≈10x speedup).

When this matters

Hot paths / repeated calls: huge wins when sanitizing long strings or when calling this function many times (the annotated tests include loops with 1000 calls and large inputs — these are exactly the cases that benefit most).
Large inputs: the larger the string, the more Python per-character overhead the original code paid; regex scales much better.

Tradeoffs / risks

Adds one import (re) and a module-level compiled pattern (negligible).
Regex is simple and safe (no backtracking/complex patterns), so no maintainability or performance fragility introduced.

Summary
Using a precompiled regex moves the character filtering into C, eliminates Python-level looping/allocation per character, and yields the observed ~10x speedup while preserving behavior across the tested edge cases.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	✅ 6 Passed
🌀 Generated Regression Tests	✅ 3168 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

⚙️ Click to see Existing Unit Tests

🌀 Click to see Generated Regression Tests

import pytest  # used for our unit tests
from langflow.api.v1.traces import _sanitize_query_string


def test_none_input_returns_none():
    # Passing None should return None (explicit early-return case).
    codeflash_output = _sanitize_query_string(None)


def test_basic_ascii_keeps_printable_characters():
    # A typical printable ASCII string should be preserved (no leading/trailing whitespace).
    s = "Hello, World!"
    codeflash_output = _sanitize_query_string(s)


def test_strips_leading_and_trailing_whitespace_and_handles_empty_result():
    # Leading/trailing spaces are stripped. If everything is whitespace the result becomes empty and function returns None.
    codeflash_output = _sanitize_query_string("   abc  ")  # trimmed
    # Only spaces -> cleaned is "   " -> trimmed to "" -> should return None.
    codeflash_output = _sanitize_query_string("   ")


def test_default_truncation_to_50_characters():
    # Strings longer than default max_len (50) are truncated to 50 characters.
    long_input = "A" * 60  # 60 characters long
    codeflash_output = _sanitize_query_string(long_input); result = codeflash_output  # default max_len is 50


def test_custom_max_len_truncation_after_strip():
    # Custom max_len should apply after trimming. Here the trimmed string is used and then truncated.
    s = "  abcdefghij  "
    codeflash_output = _sanitize_query_string(s, max_len=3)  # trimmed "abcdefghij" then truncated to "abc"


def test_filters_out_non_printable_control_characters():
    # Control characters (e.g., newline, tab, \x01) should be removed.
    mixed = "\n\t\x01X\x7F"  # \x7F (DEL) has ord 127 and should be removed as it's > '~' (126).
    # The only printable ASCII in the allowed range is 'X', so expect 'X'.
    codeflash_output = _sanitize_query_string(mixed)
    # If there are only non-printable characters, result should be None
    only_controls = "\n\r\t\x00"
    codeflash_output = _sanitize_query_string(only_controls)


def test_non_ascii_unicode_characters_are_filtered_out():
    # Characters with codepoints outside the range [32, 126] are removed.
    s = "é😊abc"  # 'é' and '😊' are outside the allowed ASCII printable range
    codeflash_output = _sanitize_query_string(s)
    # Leading/trailing spaces combined with non-ascii should still trim spaces and remove non-ascii
    s2 = "  éabc  "
    codeflash_output = _sanitize_query_string(s2)


def test_single_space_becomes_none_because_of_strip():
    # A single space is within the allowed character range but gets removed by strip() -> empty -> returns None.
    codeflash_output = _sanitize_query_string(" ")


def test_boundary_ascii_characters_behavior():
    # Test behavior at boundary ASCII values.
    # ord 32 (space) is allowed by filter but removed by strip: expect None for sole space.
    codeflash_output = _sanitize_query_string(chr(32))
    # ord 126 ('~') is allowed and not stripped: expect '~'
    codeflash_output = _sanitize_query_string(chr(126))
    # ord 31 is below allowed range -> filtered out -> None
    codeflash_output = _sanitize_query_string(chr(31))
    # ord 127 (DEL) is above allowed -> filtered out -> None
    codeflash_output = _sanitize_query_string(chr(127))


def test_empty_string_returns_none():
    # Empty input string should return None (cleaned becomes empty).
    codeflash_output = _sanitize_query_string("")


def test_non_string_types_raise_type_error():
    # Passing non-string, non-None values should raise a TypeError because the function iterates characters.
    with pytest.raises(TypeError):
        _sanitize_query_string(123)  # integer is not iterable in the expected manner
    with pytest.raises(TypeError):
        _sanitize_query_string(b"bytes")  # bytes iterate integers; comparison to strings inside will raise


def test_ascii_range_single_character_behavior_loop():
    # For every ASCII codepoint in 0..127, check the function's deterministic behavior on a single-character input.
    # Allowed printable range is ord 32..126; however ord 32 (space) becomes None due to strip.
    for i in range(128):
        ch = chr(i)
        codeflash_output = _sanitize_query_string(ch); result = codeflash_output
        if 33 <= i <= 126:  # printable visible characters (space excluded)
            pass
        else:
            pass


def test_large_scale_string_truncation_and_repeated_calls_performance():
    # Build a very large allowed-character string (2000 chars) to test truncation and performance.
    big = ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789~!@#$%^&*()-_=+[]{};:',.<>/?\\| ") * 20
    # The result should be trimmed (no leading/trailing spaces) then truncated to 50 characters.
    codeflash_output = _sanitize_query_string(big); res = codeflash_output

    # Call the function 1000 times in a loop with deterministic inputs to ensure consistent results and no state leakage.
    # Use a variety of strings (prefixes of the big string) to verify behavior across many calls.
    for i in range(1000):
        inp = big[: (i % 200) + 1]  # lengths cycle through 1..200 deterministically
        expected = inp.strip()[:50] if inp.strip() else None
        codeflash_output = _sanitize_query_string(inp)


def test_many_distinct_strings_list_of_1000_elements():
    # Create 1000 distinct strings consisting of only allowed printable characters.
    # Verify the function processes each element deterministically and truncates accordingly.
    base = "abcdefghijklmnopqrstuvwxyz0123456789~"
    inputs = [base * ((i % 5) + 1) + (" " * (i % 3)) for i in range(1000)]  # 1000 elements; some have trailing spaces
    # Check each resulting output matches cleaned.strip()[:50] or None if empty after strip
    for original in inputs:
        cleaned = "".join(ch for ch in original if " " <= ch <= "~")
        expected = cleaned.strip()[:50] if cleaned.strip() else None
        codeflash_output = _sanitize_query_string(original)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
#------------------------------------------------
import pytest  # used for our unit tests
from langflow.api.v1.traces import _sanitize_query_string


def test_basic_ascii_and_trimming():
    # Leading and trailing spaces should be stripped; ASCII printable preserved
    s = "  hello world  "  # has leading/trailing spaces
    codeflash_output = _sanitize_query_string(s)  # expect trimmed content

    # A simple ASCII string shorter than max_len returns unchanged content
    codeflash_output = _sanitize_query_string("simple-test")

    # Tilde (~) is within the allowed printable range and should be preserved
    codeflash_output = _sanitize_query_string("edge~case")


def test_removal_of_non_printable_and_non_ascii_characters():
    # Null byte \x00 should be removed between letters
    s = "a\x00b\x01c"  # contains control characters \x00 and \x01
    codeflash_output = _sanitize_query_string(s)  # control chars removed

    # Unicode symbols outside ASCII printable range should be removed (e.g., emoji, snowman)
    s2 = "foo\u2603bar😊baz"  # includes snowman and emoji
    codeflash_output = _sanitize_query_string(s2)  # non-ASCII removed

    # Characters just below space (0x1F) are removed; space and tilde are preserved; DEL (0x7F) removed
    s3 = "\x1f ~\x7f"
    # cleaned -> " ~" then .strip() -> "~"
    codeflash_output = _sanitize_query_string(s3)


def test_none_and_only_control_or_space_inputs():
    # None input must return None
    codeflash_output = _sanitize_query_string(None)

    # Input consisting solely of control characters results in cleaned == '' -> function returns None
    codeflash_output = _sanitize_query_string("\x00\x01\x02")

    # Input consisting solely of spaces: cleaned is non-empty (spaces), but strip() -> ''
    # Since the function checks cleaned (before strip), it will return the empty string.
    codeflash_output = _sanitize_query_string("   ")  # empty string result (not None)


def test_max_len_truncation_and_zero_negative_values():
    # max_len truncates the result to the specified length after stripping
    s = "   abcdefghijklmnop   "
    # After strip -> "abcdefghijklmnop", truncated to 5 -> "abcde"
    codeflash_output = _sanitize_query_string(s, max_len=5)

    # max_len == 0 should return an empty string (cleaned is truthy before strip)
    codeflash_output = _sanitize_query_string("nonempty", max_len=0)

    # Negative max_len uses Python slicing semantics; e.g., -1 drops the last character
    codeflash_output = _sanitize_query_string("abcdef", max_len=-1)
    codeflash_output = _sanitize_query_string(" a b c ", max_len=-2)  # strip -> "a b c", slice :-2 -> "a b"


def test_boundary_characters_space_and_tilde():
    # Space (0x20) should be kept (but later removed by strip if at ends)
    codeflash_output = _sanitize_query_string(" a ")  # spaces trimmed by strip

    # Tilde (0x7E) should be kept as it's the upper bound of allowed characters
    codeflash_output = _sanitize_query_string("~")


def test_empty_result_when_cleaned_but_strip_makes_empty_and_max_len():
    # If the cleaned string contains only whitespace, cleaned is truthy and .strip() becomes ''
    # Check that when max_len is > 0 we still get '' (empty string)
    codeflash_output = _sanitize_query_string("    ", max_len=10)
    # And when max_len == 0 also empty string
    codeflash_output = _sanitize_query_string("    ", max_len=0)


def test_large_scale_batch_processing_deterministic_results():
    # Create 1000 inputs with predictable removals (a trailing null char to be removed)
    inputs = [f"item-{i}\x00" for i in range(1000)]
    # Expected outputs are the items without the null char and no extra spaces
    expected = [f"item-{i}" for i in range(1000)]

    # Process all inputs and collect outputs
    outputs = [_sanitize_query_string(s) for s in inputs]

    # Also verify performance-ish loop by varying max_len; ensure no unexpected exceptions and correct truncation
    outputs2 = []
    expected2 = []
    for i in range(1000):
        # Construct string of repeating 'x' of varying length with a non-printable char in between
        part = "x" * (i % 120)  # length cycles 0..119
        s = f"{part}\x00{part}"  # null byte removed; resulting cleaned is part+part
        # compute expected result using the same logic: remove control char, strip (no spaces here), then slice to max_len
        max_len = (i % 60)  # vary max_len 0..59
        expected_cleaned = (part + part)[:max_len]
        outputs2.append(_sanitize_query_string(s, max_len=max_len))
        expected2.append(expected_cleaned)


def test_various_edge_cases_sorted_by_difficulty():
    # Empty string input: cleaned is empty -> function returns None
    codeflash_output = _sanitize_query_string("")

    # String that contains only characters at the exact allowed edge: space and tilde
    codeflash_output = _sanitize_query_string(" ~ ")

    # Very long ASCII string should be truncated to default max_len (50)
    long_input = "a" * 200
    codeflash_output = _sanitize_query_string(long_input)

    # Mixed content: control chars interspersed with printable yielding exact boundary length
    mixed = ("\x01" + "A" * 49 + "\x02")  # control chars removed, leaves 49 A's
    codeflash_output = _sanitize_query_string(mixed)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr11934-2026-02-27T09.22.43 and push.

Brief: The optimized version replaces the Python-level per-character filter ("".join(ch for ch in value if " " <= ch <= "~")) with a single, precompiled regular-expression substitution that removes characters outside the printable ASCII range. This moves the heavy work into C (the regex engine) and avoids Python bytecode overhead per character, producing the ~10x runtime improvement observed (26.0 ms → 2.34 ms). What changed - Replaced generator + join + per-character Python comparisons with a precompiled regex: _re_non_printable = re.compile(r"[^ -~]") and used _re_non_printable.sub("", value). - The regex object is compiled once at module import (module-global), so repeated calls reuse the compiled pattern. Key reasons this is faster - Avoids per-character Python interpretation: the original generator expression executes Python bytecode and creates Python-level objects/operations for every character in the string (the line profiler shows that line dominated the runtime). The regex substitution runs in optimized C loops and reduces per-character interpreter overhead drastically. - Single C-level pass: re.sub performs the filtering in C and is memory/cache-friendly compared to many small Python operations. - One-time compile cost: precompiling the pattern removes the cost of parsing/compiling the regex on every call. Behavior & correctness considerations - Semantics are preserved: both implementations keep characters with codepoints in the inclusive range [32 (space), 126 (~)] and then call .strip() and slice. Edge cases (space-only -> becomes empty after strip, non-str inputs raise TypeError) remain the same. - Unicode behavior matches the original character-comparison approach because both operate on Python str codepoints; non-ASCII characters outside the [32..126] range are removed. - The precompiled regex adds a tiny import-time cost but pays off for repeated calls. Profiling evidence - Line profiler shows the generator/join was the dominant cost in the original implementation; after the change the regex substitution line is the main cost but its absolute time is much smaller. Overall runtime dropped from 26.0 ms to 2.34 ms (≈10x speedup). When this matters - Hot paths / repeated calls: huge wins when sanitizing long strings or when calling this function many times (the annotated tests include loops with 1000 calls and large inputs — these are exactly the cases that benefit most). - Large inputs: the larger the string, the more Python per-character overhead the original code paid; regex scales much better. Tradeoffs / risks - Adds one import (re) and a module-level compiled pattern (negligible). - Regex is simple and safe (no backtracking/complex patterns), so no maintainability or performance fragility introduced. Summary Using a precompiled regex moves the character filtering into C, eliminates Python-level looping/allocation per character, and yields the observed ~10x speedup while preserving behavior across the tested edge cases.

codecov · 2026-02-27T09:26:22Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 35.72%. Comparing base (0596d77) to head (e758492).

❌ Your project check has failed because the head coverage (42.40%) is below the target coverage (60.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@               Coverage Diff               @@
##           temp-branch   #11935      +/-   ##
===============================================
- Coverage        35.74%   35.72%   -0.02%     
===============================================
  Files             1532     1532              
  Lines            74562    74564       +2     
  Branches         11146    11146              
===============================================
- Hits             26651    26640      -11     
- Misses           46475    46488      +13     
  Partials          1436     1436

Flag	Coverage Δ
backend	`55.74% <100.00%> (-0.06%)`	⬇️
lfx	`42.40% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/backend/base/langflow/api/v1/traces.py	`42.85% <100.00%> (+0.56%)`	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codeflash-ai · 2026-03-02T20:39:54Z

This PR has been automatically closed because the original PR #11934 by Adam-Aghili was closed.

codeflash-ai Bot added the ⚡️ codeflash Optimization PR opened by Codeflash AI label Feb 27, 2026

github-actions Bot added the community Pull Request from an external contributor label Feb 27, 2026

codeflash-ai Bot closed this Mar 2, 2026

codeflash-ai Bot deleted the codeflash/optimize-pr11934-2026-02-27T09.22.43 branch March 2, 2026 20:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚡️ Speed up function `_sanitize_query_string` by 1,011% in PR #11934 (`temp-branch`)#11935

⚡️ Speed up function `_sanitize_query_string` by 1,011% in PR #11934 (`temp-branch`)#11935
codeflash-ai[bot] wants to merge 1 commit into
temp-branchfrom
codeflash/optimize-pr11934-2026-02-27T09.22.43

codeflash-ai Bot commented Feb 27, 2026

Uh oh!

codecov Bot commented Feb 27, 2026 •

edited

Loading

Uh oh!

codeflash-ai Bot commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

Conversation

codeflash-ai Bot commented Feb 27, 2026

⚡️ This pull request contains optimizations for PR #11934

📄 1,011% (10.11x) speedup for _sanitize_query_string in src/backend/base/langflow/api/v1/traces.py

📝 Explanation and details

Uh oh!

codecov Bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codeflash-ai Bot commented Mar 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

📄 1,011% (10.11x) speedup for `_sanitize_query_string` in `src/backend/base/langflow/api/v1/traces.py`

codecov Bot commented Feb 27, 2026 •

edited

Loading