Skip to content

Fix ScheduleTimeRecord.AccumulatedTime hang during solution close#13375

Merged
YuliiaKovalova merged 3 commits intomainfrom
dev/fix-scheduletimerecord-hang
Mar 13, 2026
Merged

Fix ScheduleTimeRecord.AccumulatedTime hang during solution close#13375
YuliiaKovalova merged 3 commits intomainfrom
dev/fix-scheduletimerecord-hang

Conversation

@YuliiaKovalova
Copy link
Copy Markdown
Member

@YuliiaKovalova YuliiaKovalova commented Mar 12, 2026

Bug

ScheduleTimeRecord.get_AccumulatedTime() throws InternalErrorException with the message "Can't get the accumulated time while the timer is still running" during Scheduler.WriteDetailedSummary().

This causes the build to fail with an internal error when it should have succeeded. The build result is lost and replaced with the InternalErrorException. In VS, this can manifest as a build that appears to fail or hang since the unexpected exception in MSBuild isn't propagated to VS.

Stack Trace

at ErrorUtilities.ThrowInternalError(String message, Object[] args)
at ScheduleTimeRecord.get_AccumulatedTime()
at Scheduler.WriteRecursiveSummary(...)
at Scheduler.WriteDetailedSummary(Int32 submissionId)
at BuildManager.PerformSchedulingActions(IEnumerable responses)
at BuildManager.HandleResult(Int32 node, BuildResult result)
at BuildManager.ProcessPacket(Int32 node, INodePacket packet)
at BuildManager.ProcessWorkQueue(Action action)

Fix

Changed ScheduleTimeRecord.AccumulatedTime to return the best-effort elapsed time when the timer is still running (accumulated time + current elapsed since timer started), instead of throwing. This is diagnostic summary data used only for logging -- throwing serves no correctness purpose but causes builds to fail with an unrelated internal error.

ADO bug:

https://devdiv.visualstudio.com/DevDiv/_workitems/edit/2901459

ScheduleTimeRecord.AccumulatedTime throws InternalErrorException with
'Can't get the accumulated time while the timer is still running' during
Scheduler.WriteDetailedSummary(). This exception kills the BuildManager
work queue, preventing any further build results from being processed.
EndBuild() hangs indefinitely, causing VS to freeze for hours.

The fix returns the best-effort elapsed time (accumulated + current
elapsed) when the timer is still running, instead of throwing.
This is diagnostic summary data — throwing has no correctness benefit
but causes a catastrophic hang.

11 hits in 30 days confirmed via telemetry (StackHash: 2C721D65...).
All occurrences during solution close with running timers.
Copilot AI review requested due to automatic review settings March 12, 2026 17:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a scheduler diagnostic timing bug that can hang VS during solution close by making ScheduleTimeRecord.AccumulatedTime tolerant of a still-running timer, avoiding an InternalErrorException that can kill the BuildManager work queue thread.

Changes:

  • Updated ScheduleTimeRecord.AccumulatedTime to return best-effort elapsed time when the timer is still running (instead of throwing).
  • Added explanatory XML documentation describing the rationale and impact of the behavior change.

Comment thread src/Build/BackEnd/Components/Scheduler/ScheduleTimeRecord.cs
Comment thread src/Build/BackEnd/Components/Scheduler/ScheduleTimeRecord.cs Outdated
- Remove placeholder issues/XXXXX URL from XML doc comment
- Add ScheduleTimeRecord_AccumulatedTime_DoesNotThrowWhenTimerIsRunning test
- Add ScheduleTimeRecord_AccumulatedTime_IncludesPreviousAccumulation test
Comment thread src/Build.UnitTests/BackEnd/Scheduler_Tests.cs Outdated
Comment thread src/Build.UnitTests/BackEnd/Scheduler_Tests.cs Outdated
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
@YuliiaKovalova YuliiaKovalova merged commit 5215c7e into main Mar 13, 2026
10 checks passed
@YuliiaKovalova YuliiaKovalova deleted the dev/fix-scheduletimerecord-hang branch March 13, 2026 10:11
AR-May pushed a commit to AR-May/msbuild that referenced this pull request Mar 19, 2026
…tnet#13375)

## Bug

`ScheduleTimeRecord.get_AccumulatedTime()` throws
`InternalErrorException` with the message *"Can't get the accumulated
time while the timer is still running"* during
`Scheduler.WriteDetailedSummary()`.

This causes the build to **fail with an internal error** when it should
have succeeded. The build result is lost and replaced with the
`InternalErrorException`. In VS, this can manifest as a build that
appears to fail or hang depending on how the VS solution build manager
handles the unexpected exception from `EndBuild`.

### Stack Trace

```
at ErrorUtilities.ThrowInternalError(String message, Object[] args)
at ScheduleTimeRecord.get_AccumulatedTime()
at Scheduler.WriteRecursiveSummary(...)
at Scheduler.WriteDetailedSummary(Int32 submissionId)
at BuildManager.PerformSchedulingActions(IEnumerable responses)
at BuildManager.HandleResult(Int32 node, BuildResult result)
at BuildManager.ProcessPacket(Int32 node, INodePacket packet)
at BuildManager.ProcessWorkQueue(Action action)
```

## Fix

Changed `ScheduleTimeRecord.AccumulatedTime` to return the best-effort
elapsed time when the timer is still running (accumulated time + current
elapsed since timer started), instead of throwing. This is diagnostic
summary data used only for logging -- throwing serves no correctness
purpose but causes builds to fail with an unrelated internal error.

---------

Co-authored-by: Rainer Sigwald <raines@microsoft.com>
@MichalPavlik
Copy link
Copy Markdown
Member

/backport to vs18.5

@github-actions
Copy link
Copy Markdown
Contributor

Started backporting to vs18.5 (link to workflow run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants