perf: fast rotation for RQ quantization

[BubbleCal]

this introduces a new rotation algorithm which can do rotation in O(dlogd) time (O(d^2) before), and reduces the memory footprint of rotation matrix from d^2 floats to d bits

this doesn't reduce recall, and significantly improves performance:

query performance is 93% faster, and 307% faster than IVF_PQ
indexing is 5.4x faster, and 11.9x faster than IVF_PQ

[github-actions]

PR Review: Fast Rotation for RQ Quantization

This PR introduces a new FHT-Kac rotation algorithm for RaBitQ, improving time complexity from O(d²) to O(d log d) and reducing memory from d² floats to d bits. The implementation looks solid overall.

P1 Issues

1. Potential panic in builder.rs:444-445

   let input = column
       .as_slice()
       .expect("RabitQ input vectors should be contiguous");

   Using .expect() here can cause a runtime panic if data isn't contiguous. This should return a proper Result error instead.

2. Minor: Python documentation typo (python/dataset.py:2866)
   Documentation says "Rabit Quantization" but should be "RaBitQ" for consistency with the rest of the codebase.

Observations (not blocking)

• The default_rotation_type_compat() returning Matrix for backwards compatibility with older metadata is good.
• SIMD AVX2 optimization with scalar fallback is well implemented.
• Test coverage for rotation type preservation through optimize operations is appreciated.

The performance improvements are impressive. LGTM pending the panic-to-error change.

[codecov]

Codecov Report

❌ Patch coverage is 82.99180% with 83 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-index/src/vector/bq/builder.rs	81.21%	35 Missing and 2 partials ⚠️
rust/lance-index/src/vector/bq/storage.rs	72.00%	19 Missing and 2 partials ⚠️
rust/lance/src/index/vector.rs	0.00%	14 Missing ⚠️
rust/lance-index/src/vector/bq/rotation.rs	94.73%	5 Missing and 2 partials ⚠️
rust/lance-index/src/vector/bq.rs	87.87%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Xuanwo]

It looks like if dim is not byte-aligned, we could hit panic here, for example, if dim is 129 (is this possible?)

I ask codex to have a repro:

use std::sync::Arc;

use arrow::datatypes::Float32Type;
use arrow_array::{ArrayRef, FixedSizeListArray, Float32Array};
use arrow_schema::{DataType, Field};
use lance_index::vector::bq::builder::RabitQuantizer;
use lance_index::vector::bq::RQRotationType;
use lance_index::vector::quantizer::Quantization;

fn main() {
    let dim = 129; // non-byte-aligned dimension
    let values = Float32Array::from(vec![1.0f32; dim]);
    let field = Arc::new(Field::new("item", DataType::Float32, true));
    let vectors =
        FixedSizeListArray::try_new(field, dim as i32, Arc::new(values) as ArrayRef, None).unwrap();

    // Panic is nondeterministic (depends on random rotation signs), loop to trigger reliably.
    for i in 0..5000 {
        let q =
            RabitQuantizer::new_with_rotation::<Float32Type>(1, dim as i32, RQRotationType::Fast);
        let _ = q.quantize(&vectors).unwrap();
        if i % 1000 == 0 {
            println!("iter={i}");
        }
    }

    println!("done");
}

This logic will panic at index out of bounds

[BubbleCal]

good catch
will add requirement of dimension % 8 == 0, i think it's fine because real world vectors are almost always dividable by 8