fix: add disable_bare_second flag (#22) + restore TN abbreviation matching

[Alex-Wengg]

Summary

• Closes Should not have conversion: second -> 2nd in "Give me a second to check" #22: adds opt-in disable_bare_second flag on NormalizeOptions so the bare word second (case-insensitive, single-token) is left alone instead of being normalized to 2nd. Compound ordinals (twenty second → 22nd) and date contexts are unaffected. Default is false so existing behavior is preserved.
• Also addresses Devin AI review on PR #25: the new pretokenizer split trailing punctuation off of words so ITN could match twenty one, as twenty one, but the shared sentence_loop re-joined pretokens with a literal space. That made the TN whitelist see Dr . instead of Dr., so abbreviation entries like e.g., Prof., Inc., etc., vs. stopped matching, and both-form entries like Dr/Dr. left an orphaned period (I see Dr. Smith → I see doctor. Smith).

Issue #22 fix

disable_bare_second is plumbed through:

• NormalizeOptions builder (with_disable_bare_second)
• parse_span and normalize_sentence_inner / normalize_inner
• FFI: nemo_normalize_with_options, nemo_normalize_sentence_with_options (extra uint32_t disable_bare_second arg)
• WASM bindings (normalizeWithOptions, normalizeSentenceWithOptions)
• Swift wrapper (disableBareSecond parameter)
• C headers in both `swift/include` and `swift-test/Sources/CNemoTextProcessing/include`

let opts = NormalizeOptions::new().with_disable_bare_second(true);
normalize_sentence_with_options(\"Give me a second to check.\", opts);
// -> \"Give me a second to check.\"

normalize_sentence_with_options(\"He finished in the twenty second place\", opts);
// -> \"He finished in the 22nd place\"

PR #25 review fix

sentence_loop now reconstructs each candidate span using the per-pretoken sep, preserving original adjacency. The longest-span-first iteration still tries shorter spans, so trailing-punctuation cases like twenty one, → 21, continue to work.

for end in (i + 1..=max_end).rev() {
    let mut span = String::new();
    for (idx, p) in pretokens[i..end].iter().enumerate() {
        if idx > 0 { span.push_str(p.sep); }
        span.push_str(&p.text);
    }
    // ...
}

After the fix:

• I see Dr. Smith today. → I see doctor Smith today.
• e.g. she is here → for example she is here
• Inc. and Co. → incorporated and company
• Prof. Jones → professor Jones
• vs. them → versus them

Test plan

• cargo fmt
• cargo build --features ffi
• cargo test --features ffi — 1070 tests pass, 0 failures
• tests/en_tests.rs::test_issue_22_default_behavior_unchanged
• tests/en_tests.rs::test_issue_22_sentence_disable_bare_second
• tests/en_tests.rs::test_issue_22_single_expression_disable_bare_second
• tests/en_tn_tests.rs::test_pr25_tn_abbreviation_regression (Dr., vs., e.g., Inc., Co., Prof.)
• src/ffi.rs::test_ffi_normalize_sentence_with_options_disable_bare_second

---

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.