Skip to content

Enable handshake timeout on Mono#5318

Merged
ladipro merged 3 commits intodotnet:masterfrom
ladipro:enable-handshake-timeout-on-mono
Apr 27, 2020
Merged

Enable handshake timeout on Mono#5318
ladipro merged 3 commits intodotnet:masterfrom
ladipro:enable-handshake-timeout-on-mono

Conversation

@ladipro
Copy link
Copy Markdown
Member

@ladipro ladipro commented Apr 27, 2020

I am seeing consistent test hangs on macOS Mono. A node is getting into a bad state and is failing to respond to handshakes. While I do not yet understand the root cause, it is clear that having a timeout on the handshake operation mitigates the issue. The test is now failing but does not hang, saving developer time as well as test pipeline resources.

@ladipro ladipro changed the title Enable handshake timeout on mono Enable handshake timeout on Mono Apr 27, 2020

#if NETCOREAPP2_1
#if NETCOREAPP2_1 || MONO
if (!NativeMethodsShared.IsWindows)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity:

  1. Why was this enabled only for Core 2.1 and not for all non-Framework builds?
  2. Why don't we use a/the timeout in the else branch below? It looks like stream.ReadByte() may also hang forever if the node is broken.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. We're pretty inconsistent across the codebase with our ifdefs. This was (checks history to make sure it was me) probably just sloppiness and forgetting about Mono. Unfortunately, since Mono presents as Framework, we can't just do #if !NETFRAMEWORK
  2. I think that was mostly "don't change legacy code without good reason" + "no async or timeout-enabled PipeStream.ReadByte(). But I don't really remember :(

@ladipro
Copy link
Copy Markdown
Member Author

ladipro commented Apr 27, 2020

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).


#if NETCOREAPP2_1
#if NETCOREAPP2_1 || MONO
if (!NativeMethodsShared.IsWindows)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. We're pretty inconsistent across the codebase with our ifdefs. This was (checks history to make sure it was me) probably just sloppiness and forgetting about Mono. Unfortunately, since Mono presents as Framework, we can't just do #if !NETFRAMEWORK
  2. I think that was mostly "don't change legacy code without good reason" + "no async or timeout-enabled PipeStream.ReadByte(). But I don't really remember :(

@ladipro ladipro merged commit 70c00b4 into dotnet:master Apr 27, 2020
@ladipro ladipro deleted the enable-handshake-timeout-on-mono branch April 27, 2020 12:19
Forgind added a commit that referenced this pull request May 4, 2020
* LOC CHECKIN | microsoft/msbuild vs16.6 | 20200420 (#5299)

* Final branding_16.6 (#5273)

* merge

* Enable handshake timeout on Mono (#5318)

I am seeing consistent test hangs on macOS Mono. A node is getting into a bad state and is failing to respond to handshakes. While I do not yet understand the root cause, it is clear that having a timeout on the handshake operation mitigates the issue. The test is now failing but does not hang, saving developer time as well as test pipeline resources.

* Revert "Reverted and rebuilt for localizable strings." (#5246)

This adds back support for logging an error when a task returns false
without logging an error. This was originally added in #4940 but was
reverted because of multiple difficulties.

* Changed error code

* Add escape hatch

* Fix typo

* Filtering for duplicate content files with referencing projects (#4931)

* Updating content filtering based on content copying changes

* Add a flag that is enabled by default on Core; otherwise disabled by default.

* Adding Shutdown Message to Worker Nodes (#5262)

* Changed where Trace is being called and removed old functionality.

* Updating .NET Core branding to .NET (#5286)

* Override default Arcade Xunit configuration (#5302)

* Update Directory.Build.targets

* prevent arcade from injecting its own xunit file

* Don't log @(ReferencePath) at the end of ImplicitlyExpandDesignTimeFacades (#5317)

The actual ItemGroups inside the target already do a good job of logging exactly what items were added and/or removed and in what order and with what metadata.

Emitting an extra low-pri message which is unstructured here just adds noise, slows the builds down, wastes binlog space and is otherwise redundant.

* Compute hashes in parallel in GetFileHash (#5303)

* Compute hashes in parallel. This scales better for larger number of files.

* Use a dedicated write lock

* Allow disabling logging of task parameters and item metadata (#5268)

This enables fine-grained control over whether:

 * to log log each parameter (whether input or output)
 * or whether to log item metadata for each ITaskItem[] parameter.

When LogTaskInputs is set the default behavior is still to log all parameters and all item metadata for ITaskItem[] parameters. Since this is very verbose and hurts performance without adding any useful information it is valuable to be able to turn this logging off in certain situations.

This approach allows controlling logging via setting simple properties or environment variables.

I've identified the specific tasks and parameters that we want to restrict logging for that would give us the most gains without losing any significant useful info:

https://github.com/KirillOsenkov/MSBuildStructuredLog/wiki/Task-Parameter-Logging

* Update dependencies from https://github.com/dotnet/arcade build 20200430.5 (#5325)

- Microsoft.DotNet.Arcade.Sdk: 1.0.0-beta.20221.2 -> 1.0.0-beta.20230.5

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Use environment variable for handshake Resolves #4961 (#5196)

* Use environment variable for handshake Resolves #4961

* Combine means of hashing

* Support transitive project references in static graph (#5326)

Transitive project references are a thing now. Added support to static graph, so that buildxl and qb can avoid adding the transitive refs.
This PR is independent of #5222. Ideally, review this one first, as QB has a stronger dependency on this PR than on #5222.

Design

- transitive references are opt-in, per project evaluation
- once a project opts-in, transitivity is applied for all ProjectReference items
- a project opt-ins by setting the property AddTransitiveProjectReferencesInStaticGraph to true. The sdk does this automatically in Microsoft.Managed.After.Targets.
- interaction with crosstargeting: transitive refs are added only to inner builds, not the outer builds. This mimics vanilla msbuild.

Co-authored-by: Rainer Sigwald <raines@microsoft.com>

Co-authored-by: Martin Chromecek (Moravia IT) <v-chmart@microsoft.com>
Co-authored-by: Ladi Prosek <laprosek@microsoft.com>
Co-authored-by: Sarah Oslund <sfoslund@microsoft.com>
Co-authored-by: Ben Villalobos <villalobosb93@gmail.com>
Co-authored-by: Mihai Codoban <codobanmihai@gmail.com>
Co-authored-by: Kirill Osenkov <KirillOsenkov@users.noreply.github.com>
Co-authored-by: Pranav K <prkrishn@hotmail.com>
Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants