[18.03] Cherry picking scheduler and task reaper fixes, plus allocator logging.

[nishanttotla]

This PR is a cherry-pick of commits from the following PRs (in order, on top of the bump_v18.03 branch):

• Scheduler/TaskReaper: handle unassigned tasks marked for shutdown #2574 Scheduler/TaskReaper: handle unassigned tasks marked for shutdown
• Task reaper should delete tasks with removed slots that were not yet assigned #2557 Task reaper should delete tasks with removed slots that were not yet assigned
• [manager/allocator] Logging for allocation workflows #2614 [manager/allocator] Logging for allocation workflows

In order to fix some test flakiness, I've also cherry-picked

• Fix some flakiness with the manager tests #2575 Fix some flakiness with the manager tests

git cherry-pick -s -x e3fcf6d9c553e9c69f83df6c49364eb270e62445
git cherry-pick -s -x 88064b5e2a8ca3431285333b37176a13db357648
git cherry-pick -s -x 7f638251835ccc3d791f128e3dbad76ccb9aaebd
git cherry-pick -s -x bef2bd052936d3aefe36cc7e8f6fefeb593e2531
git cherry-pick -s -x 649826059c3b65f2cc54c88de9e448174dfb15ff

All cherry-picks were clean.

cc @anshulpundir @chungers @rchourasia @andrewhsu

[codecov]

Codecov Report

Merging #2616 into bump_v18.03 will increase coverage by 0.32%.
The diff coverage is 59.37%.

@@               Coverage Diff               @@
##           bump_v18.03    #2616      +/-   ##
===============================================
+ Coverage        61.42%   61.75%   +0.32%     
===============================================
  Files              133      133              
  Lines            21726    21746      +20     
===============================================
+ Hits             13345    13429      +84     
+ Misses            6943     6879      -64     
  Partials          1438     1438