Execute sort in parallel when a limit is used after sort

[Dandandan]

Which issue does this PR close?

Closes #3526

Rationale for this change

Improves performance on ORDER BY <expr> LIMIT N queries.

❯ CREATE EXTERNAL TABLE t
STORED AS PARQUET
LOCATION  './arrow-datafusion/benchmarks/parquet/lineitem';
0 rows in set. Query took 0.153 seconds.

❯ select l_orderkey from t order by l_orderkey limit 10;
+------------+
| l_orderkey |
+------------+
| 1          |
| 1          |
| 1          |
| 1          |
| 1          |
| 2          |
| 3          |
| 3          |
| 3          |
| 3          |
+------------+
10 rows in set. Query took 0.772 seconds.

vs master:

❯ select l_orderkey from t order by l_orderkey limit 10;
+------------+
| l_orderkey |
+------------+
| 1          |
| 1          |
| 1          |
| 1          |
| 1          |
| 2          |
| 3          |
| 3          |
| 3          |
| 3          |
+------------+
10 rows in set. Query took 3.804 seconds.

What changes are included in this PR?

Are there any user-facing changes?

[codecov-commenter]

Codecov Report

Merging #3527 (830f0f3) into master (25c029c) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3527      +/-   ##
==========================================
+ Coverage   85.75%   85.79%   +0.03%     
==========================================
  Files         299      300       +1     
  Lines       55311    55356      +45     
==========================================
+ Hits        47432    47492      +60     
+ Misses       7879     7864      -15     

Impacted Files	Coverage Δ
datafusion/core/tests/sql/explain_analyze.rs	83.87% <ø> (ø)
datafusion/core/src/execution/context.rs	79.33% <100.00%> (+0.02%)	⬆️
...usion/core/src/physical_optimizer/parallel_sort.rs	100.00% <100.00%> (ø)
datafusion/core/src/physical_plan/planner.rs	77.35% <100.00%> (ø)
datafusion/core/src/physical_plan/sorts/sort.rs	94.37% <100.00%> (+1.27%)	⬆️
datafusion/expr/src/logical_plan/plan.rs	77.35% <0.00%> (-0.50%)	⬇️
datafusion/common/src/lib.rs	0.00% <0.00%> (ø)
datafusion/physical-expr/src/aggregate/count.rs	98.05% <0.00%> (ø)
datafusion/physical-expr/src/aggregate/average.rs	94.96% <0.00%> (ø)
...tafusion/core/src/physical_plan/file_format/mod.rs	96.95% <0.00%> (ø)
... and 11 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[alamb]

LGTM -- nice work @Dandandan

[alamb]

👍 FYI @tustvold -- more use of this operator 💯

[alamb]

👍

[alamb]

I wonder if https://docs.rs/datafusion/12.0.0/datafusion/physical_plan/trait.ExecutionPlan.html#tymethod.output_partitioning could be used here instead? But maybe it doesn't matter as we already know it is exactly a SortExec

[alamb]

👍

[ursabot]

Benchmark runs are scheduled for baseline = 9b22100 and contender = 3a9e0d0. 3a9e0d0 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java