chore: Speed up "PR Builds" CI workflows

[andygrove]

Which issue does this PR close?

Closes #1800
Closes #1783

Rationale for this change

This PR speeds up the "PR Builds" (the Comet scalatest suites) by splitting them up into 4 groups that can run concurrently.

Previously it took ~90 minutes to run these tests in a single job but now there are 4 jobs taking 25-35 minutes each and they run concurrently.

What changes are included in this PR?

• Split the Comet test suites into four separate jobs per configuration, to remove the 90-minute wait
• Move away from a matrix approach and specify a set of profiles to test with, since it is not feasible to test every possible combination of spark, java, scala, os/arch and native scan implementation (there are 216 combinations).
• Stop testing on Intel macOS
• Only run the Rust tests once per OS per JDK version (no longer running for each Spark and Scala version because that never made sense)

How are these changes tested?

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.45%. Comparing base (f09f8af) to head (d46c8a5).
Report is 222 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1807      +/-   ##
============================================
+ Coverage     56.12%   59.45%   +3.32%     
- Complexity      976     1139     +163     
============================================
  Files           119      129      +10     
  Lines         11743    12590     +847     
  Branches       2251     2357     +106     
============================================
+ Hits           6591     7485     +894     
+ Misses         4012     3905     -107     
- Partials       1140     1200      +60     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[comphead]

Thats great! Thanks @andygrove

[andygrove]

Thanks for the reviews @parthchandra and @comphead