Could not resolve SDK location on mac

[yamachu]

Description

ref?: #79237
ref?: dotnet/sdk#30546

When I try to run dotnet tools, like dotnet-format, it fails to resolve the sdk.

Reproduction Steps

1. LANG=en_US.UTF-8 dotnet new console -o sample
2. cd sample/
3. LANG=en_US.UTF-8 dotnet format

# ~/Projects/github.com/yamachu $
 LANG=en_US.UTF-8 dotnet new console -o sample
The template "Console App" was created successfully.

Processing post-creation actions...
Restoring /Users/yamachu/Projects/github.com/yamachu/sample/sample.csproj:
  Determining projects to restore...
  Restored /Users/yamachu/Projects/github.com/yamachu/sample/sample.csproj (in 32 ms).
Restore succeeded.


# ~/Projects/github.com/yamachu $
 cd sample/
# ~/Projects/github.com/yamachu/sample $
 LANG=en_US.UTF-8 dotnet format
No .NET SDKs were found.

Download a .NET SDK:
https://aka.ms/dotnet/download

Learn about SDK resolution:
https://aka.ms/dotnet/sdk-not-found
Unable to locate MSBuild. Ensure the .NET SDK was installed with the official installer.

Expected behavior

dotnet-format works

Actual behavior

dotnet-format does not work due to fail to resolve sdks

Regression?

No response

Known Workarounds

No response

Configuration

• dotnet 7.0.202
  • sdks
    • 6.0.201 [/usr/local/share/dotnet/sdk]
    • 7.0.101 [/usr/local/share/dotnet/sdk]
    • 7.0.200 [/usr/local/share/dotnet/sdk]
    • 7.0.202 [/usr/local/share/dotnet/sdk]
• macOS 13.2.1
  • arm64
• $DOTNET_ROOT is empty

$ cat /etc/dotnet/install_location /etc/dotnet/install_location_arm64 /etc/dotnet/install_location_x64
/usr/local/share/dotnet/x64
/usr/local/share/dotnet
/usr/local/share/dotnet/x64

Other information

The following gist shows the results of running the following command.

$ COREHOST_TRACE=1 dotnet format

https://gist.github.com/yamachu/18773fb0bf9443c289664cb9b28aa737

The code searching the sdk seems to be as follows.

runtime/src/native/corehost/fxr/sdk_resolver.cpp

Line 69 in 8ab7a2f

get_framework_and_sdk_locations(dotnet_root, /*disable_multilevel_lookup*/ true, &locations);

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

ref?: #79237
ref?: dotnet/sdk#30546

When I try to run dotnet tools, like dotnet-format, it fails to resolve the sdk.

Reproduction Steps

1. LANG=en_US.UTF-8 dotnet new console -o sample
2. cd sample/
3. LANG=en_US.UTF-8 dotnet format

# ~/Projects/github.com/yamachu $
 LANG=en_US.UTF-8 dotnet new console -o sample
The template "Console App" was created successfully.

Processing post-creation actions...
Restoring /Users/yamachu/Projects/github.com/yamachu/sample/sample.csproj:
  Determining projects to restore...
  Restored /Users/yamachu/Projects/github.com/yamachu/sample/sample.csproj (in 32 ms).
Restore succeeded.


# ~/Projects/github.com/yamachu $
 cd sample/
# ~/Projects/github.com/yamachu/sample $
 LANG=en_US.UTF-8 dotnet format
No .NET SDKs were found.

Download a .NET SDK:
https://aka.ms/dotnet/download

Learn about SDK resolution:
https://aka.ms/dotnet/sdk-not-found
Unable to locate MSBuild. Ensure the .NET SDK was installed with the official installer.

Expected behavior

dotnet-format works

Actual behavior

dotnet-format does not work due to fail to resolve sdks

Regression?

No response

Known Workarounds

No response

Configuration

• dotnet 7.0.202
  • sdks
    • 6.0.201 [/usr/local/share/dotnet/sdk]
    • 7.0.101 [/usr/local/share/dotnet/sdk]
    • 7.0.200 [/usr/local/share/dotnet/sdk]
    • 7.0.202 [/usr/local/share/dotnet/sdk]
• macOS 13.2.1
  • arm64
• $DOTNET_ROOT is empty

$ cat /etc/dotnet/install_location /etc/dotnet/install_location_arm64 /etc/dotnet/install_location_x64
/usr/local/share/dotnet/x64
/usr/local/share/dotnet
/usr/local/share/dotnet/x64

Other information

The following gist shows the results of running the following command.

$ COREHOST_TRACE=1 dotnet format

https://gist.github.com/yamachu/18773fb0bf9443c289664cb9b28aa737

The code searching the sdk seems to be as follows.

runtime/src/native/corehost/fxr/sdk_resolver.cpp

Line 69 in 8ab7a2f

get_framework_and_sdk_locations(dotnet_root, /*disable_multilevel_lookup*/ true, &locations);

Author:	yamachu
Assignees:	-
Labels:	area-Host, untriaged
Milestone:	-

[elinor-fung]

Interesting portions of the host trace:

Launch host: /usr/local/share/dotnet/dotnet, app: /usr/local/share/dotnet/sdk/7.0.202/DotnetTools/dotnet-format/dotnet-format.dll, argc: 0, args:
[...]
--- Invoked hostfxr_resolve_sdk2 [commit hash: 0a396acafe9a7d46bce11f4338dbb3dd0d99b1b4]
[...]
Resolving SDKs with version = 'latest', rollForward = 'latestMajor', allowPrerelease = true
Searching for SDK versions in [/usr/local/bin/sdk]
No .NET SDKs were found.

The dotnet-format app is run and it ends up invoking hostfxr_resolve_sdk2 - apparently with /usr/local/bin as the exe_dir. The host just appends sdk to the directory it is given in hostfxr_resolve_sdk2 when looking for SDKs.

Not sure where the /usr/local/bin value is coming from yet.
cc @jmarolf @JoeRobich for dotnet-format

@yamachu do you actually have .NET under /usr/local/bin (does /usr/local/bin/dotnet exist)?

[elinor-fung]

This is the managed stack that calls into hostfxr_resolve_sdk2.

  Microsoft.Build.Locator!Microsoft.Build.Locator.DotNetSdkLocationHelper+<GetInstances>d__4.MoveNext()
  Microsoft.Build.Locator!Microsoft.Build.Locator.MSBuildLocator+<GetInstances>d__20.MoveNext()
  System.Linq!System.Linq.Enumerable+WhereEnumerableIterator`1[System.__Canon].MoveNext()
  System.Linq!System.Linq.Enumerable.TryGetFirst(class System.Collections.Generic.IEnumerable`1<!!0>,bool&)
  System.Linq!System.Linq.Enumerable.FirstOrDefault(class System.Collections.Generic.IEnumerable`1<!!0>)
  dotnet-format!Microsoft.CodeAnalysis.Tools.FormatCommandCommon.TryLoadMSBuild(class System.String&)
  dotnet-format!Microsoft.CodeAnalysis.Tools.FormatCommandCommon+<FormatAsync>d__21.MoveNext()

@rainersigwald @Forgind for MSBuildLocator - do you have suggestions for what to look at here?

It looks like DotNetSdkLocationHelper does its own search for dotnet and passes the directory to hostfxr_resolve_sdk2.
https://github.com/microsoft/MSBuildLocator/blob/d563efd188ea5786f85bf272470cf7cd4dac9500/src/MSBuildLocator/DotNetSdkLocationHelper.cs#L91-L116
It is searching through PATH to find the dotnet directory. (Even if it is running in the dotnet process itself?)

[Forgind]

Is /usr/local/bin on the PATH, and is there a /usr/local/bin/dotnet there? If so, MSBuildLocator might well be using that one even if we're running in the dotnet process. MSBuildLocator doesn't normally expect to be in a dotnet process; client apps can and do call our APIs fairly often. That said, it seems reasonable to me to check if we're running in dotnet and use that if so. We could also consider exposing a new API to specify which dotnet someone wants to use in searching for SDKs—that would make us maximally flexible, and then dotnet could pass in its own directory directly.

I'd be happy to make either of those changes if we decide that's the route we want to take 🙂

[yamachu]

 ls -al /usr/local/bin/ |grep dotnet
lrwxr-xr-x root wheel  34 B Wed Nov 10 20:00:27 2021 dotnet ⇒ /usr/local/share/dotnet/x64/dotnet
# ~/Projects/github.com/yamachu/sample $
 ls /usr/local/share/dotnet/
 dotnet   LICENSE.txt   packs   sdk-manifests   template-packs   ThirdPartyNotices.txt
 host     metadata      sdk     shared          templates

I have found that there is an invalid link to dotnet in the specified directory.
Then, after renaming the target link, I successfully ran dotnet-format.

# ~/Projects/github.com/yamachu/sample $
 sudo mv /usr/local/bin/dotnet /usr/local/bin/_dotnet
# ~/Projects/github.com/yamachu/sample $
 ls /usr/local/bin/|grep dotnet
_dotnet
# ~/Projects/github.com/yamachu/sample $
 ls
 bin   obj   Program.cs   sample.csproj
# ~/Projects/github.com/yamachu/sample $
 dotnet format

Thanks a lot!

[elinor-fung]

Thanks for checking that @yamachu.

@Forgind I'm going to move this over to the MSBuildLocator repo for consideration of handling the invalid link scenario (maybe doing a file existence check after the realpath when applicable).

[Forgind]

Thanks for checking that @yamachu.

@Forgind I'm going to move this over to the MSBuildLocator repo for consideration of handling the invalid link scenario (maybe doing a file existence check after the realpath when applicable).

That sounds good; thanks @elinor-fung!

I'm thinking we might have to move things around a little to do the File.Exists check before we stop parsing the PATH, but that should be doable.

[elinor-fung]

Forgot we can't transfer issues between orgs - filed microsoft/MSBuildLocator#201.