Bundle_extraction_is_reused failed in CI

[ericstj]

See #33563 (comment)

Build failure https://dev.azure.com/dnceng/public/_build/results?buildId=561783&view=ms.vss-test-web.build-test-results-tab&runId=17679122&resultId=100000&paneView=debug

Platform: linux-x64

Error:

System.ComponentModel.Win32Exception : Text file busy


Stack trace
   at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.DotNet.Cli.Build.Framework.Command.Start() in /_/src/installer/test/TestUtils/Command.cs:line 200
   at Microsoft.DotNet.Cli.Build.Framework.Command.Execute(Boolean fExpectedToFail) in /_/src/installer/test/TestUtils/Command.cs:line 240
   at Microsoft.DotNet.Cli.Build.Framework.Command.Execute() in /_/src/installer/test/TestUtils/Command.cs:line 172
   at AppHost.Bundle.Tests.BundleExtractToSpecificPath.Bundle_extraction_is_reused() in /_/src/installer/test/Microsoft.NET.HostModel.Tests/AppHost.Bundle.Tests/BundleExtractToSpecificPath.cs:line 87

/cc @swaroop-sridhar

[swaroop-sridhar]

Seems to be a one-off failure, where a copy/execute have collided. I'm not sure whether this is within the product, or a test-fixture copy step that raced with the test execution. I was unable to reproduce the failure in master or maryamariyan:ref-src branches locally.

[ericstj]

I believe this is like a file-in-use situation on Linux. Does the test or product code need to account for such a file-in-use? Often times these are one-off but common enough to require mitigations (virus scanners, search indexers).

[swaroop-sridhar]

@ericstj I completely agree with your assessment; I'm just trying to get a repro for this failure to see where the race could have happened.

The AppHost creation involves a few edits of the executable, and has had a few runins with anti-virus scanners. But I don't know of any AVs running on the linux test machines. The proposed fix for the AppHost issue is: #3829 and #3828.

But at least on the surface, this doesn't look like the same problem:

• Every failure in the apphost writer (or when executing the host) I've seen so far has a stack trace that shows a failure when accessing something within the product.
• In this case the failure was in System.Diagnostics.Process.ForkAndExecProcess(). That is, the SDK/bundler (GenerateBundle) steps have completed (which would rule out the above mentioned issues).
• The AppHost spews an error trace when a failure occurs while processing the bundle contents. There's actually no indication that the host got to run at all.

This stack trace may indicate a race in the test-infrastructure -- ex: between writing the single-file app and running it. But we need more repros to be actionable.