add lexicographically partition points and ranges

[jimexist]

Which issue does this PR close?

Closes #428

Rationale for this change

in order to support order by and partition by within window functions, we'll need to find out partition points on already sorted columns (lexicographically), and with binary search the time complexity can be O(log(n)). Here we'll utilize the lexicographical comparator extracted from #423 and add two functions.

What changes are included in this PR?

                        time:   [20.391 us 20.433 us 20.490 us]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe

lexicographical_partition_ranges(u8) 2^12
                        time:   [28.129 us 28.181 us 28.254 us]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe

Benchmarking lexicographical_partition_ranges(u8) 2^10 with nulls: Collecting 100 samples in estimated 5.0664 s (268k iterat                                                                                                                            lexicographical_partition_ranges(u8) 2^10 with nulls
                        time:   [18.841 us 18.889 us 18.945 us]
Found 13 outliers among 100 measurements (13.00%)
  2 (2.00%) low mild
  5 (5.00%) high mild
  6 (6.00%) high severe

Benchmarking lexicographical_partition_ranges(u8) 2^12 with nulls: Collecting 100 samples in estimated 5.0278 s (172k iterat                                                                                                                            lexicographical_partition_ranges(u8) 2^12 with nulls
                        time:   [29.202 us 29.293 us 29.402 us]
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe

lexicographical_partition_ranges(f64) 2^10
                        time:   [67.934 us 68.701 us 69.617 us]
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

Benchmarking lexicographical_partition_ranges(low cardinality) 1024: Collecting 100 samples in estimated 5.0047 s (4.1M iter                                                                                                                            lexicographical_partition_ranges(low cardinality) 1024
                        time:   [1.2215 us 1.2272 us 1.2336 us]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) low mild
  1 (1.00%) high severe

Are there any user-facing changes?

[jorgecarleitao]

Looks great. I left an optional comment; it can be moved to a separate PR 👍

[jorgecarleitao]

It should be possible to write the below as an iterator, thereby avoiding the allocation of the vector.

The general idea: create a struct with a method try_new. That method initializes the struct's content and errors if something is wrong (e.g. empty columns). It also initializes lexicographical_comparator.

Next, implement Iterator<Item=usize> for that struct where Iterator::next yields previous_partition_point and increments it as written in this function.

[jimexist]

tracked in #437

[alamb]

I filed #428 to track this. Thanks @jimexist

[alamb]

BTW I hope to get this into Arrow 4.3 -- I plan to build a release candidate for that on Thursday or Friday this week and release early next week. Once it gets released it will then be available in DataFusion