Add new hostfxr API to report installed runtimes and SDKs - equivalent of --info

[vitek-karas]

Goal

Currently the hostfxr has the ability to enumerate all accessible runtimes and SDKs given a certain environment. This functionality is only exposed via the dotnet --info command which provides humanly readable output. This is useful for humans, but it's hard to consume for machines - if there's a need to get this information from code, parsing this text is problematic at best.

Design

Add a new API hostfxr_get_dotnet_environment_info which would provide both runtimes, SDKs and other information in one call.

struct hostfxr_dotnet_environment_sdk_info
{
    size_t size;
    const pal::char_t *version;
    const pal::char_t *path;
}

struct hostfxr_dotnet_environment_framework_info
{
    size_t size;
    const pal::char_t *name;
    const pal::char_t *version;
    const pal::char_t *path;
}

struct hostfxr_dotnet_environment_info
{
    size_t size;

    const pal::char_t *hostfxr_version;
    const pal::char_t *hostfxr_commit_hash;

    size_t sdk_count;
    const hostfxr_dotnet_environment_sdk_info *sdks;

    size_t framework_count;
    const hostfxr_dotnet_environment_framework_info *frameworks;
}

typedef void (HOSTFXR_CALLTYPE *hostfxr_get_dotnet_environment_info_result_fn)(
    const hostfxr_dotnet_environment_info *info,
    void *result_context);

SHARED_API int32_t HOSTFXR_CALLTYPE hostfxr_get_dotnet_environment_info(
    const pal::char_t *dotnet_root,
    void *reserved,
    hostfxr_get_dotnet_environment_info_result_fn result,
    void *result_context);

This follows the existing pattern in hostfxr APIs where the caller provides a function which is called with the results of the operation. This is to avoid memory ownership/allocation issues. The result function is passed buffers allocated by the hostfxr which are only valid for that one call. This allows hostfxr to use internal buffers or stack allocated memory and makes it easy to correctly cleanup.

The result function must make copies of everything it needs to keep before returning.

The size fields are added for versioning and extensibility. The field is initialized with the sizeof(struct), the consumer should check that the value in size is equal or greater to the size it expects. This allows us to add more fields to the structures going forward without versioning the API itself.

For similar reasons the reserved parameter is added, right now it should always be null. If we need to pass additional parameter in the future, the parameter would pass a pointer to a structure with those parameters.

The dotnet_root parameter should point to the directory with dotnet.exe if the caller needs to specify one. It can be null in which case the default global location will be used instead. This is to make the API equivalent in capability to dotnet --info. Running dotnet --info passes a "hidden" path to the tool - the path to the dotnet.exe actually used.

The result_context parameter is to be used by the caller to pass additional context to the result callback. The function will not work with its value in any way, it will just pass it as the result_context parameter to the result callback.

Locating and loading hostfxr

It's the responsibility of the caller to locate and load hostfxr. The nethost library can be used to do this.

The location of hostfxr used to make this API call should not affect the result in any way. That is, it doesn't matter which hostfxr is used (ignoring differences in functionality across future versions), given the same input, all of them should produce the same output.

Behavior

The intended behavior of this API is to mirror the existing behavior of dotnet --info. Which means it enumerates SDKs and frameworks from the dotnet root directory (either explicitly specified or using global install location per design). If DOTNET_MULTILEVEL_LOOKUP is enabled (Windows-only), and the dotnet root is specified and it's not the global install location, then it will also enumerate SDKs and frameworks from the global install location.

Ordering of SDKs should be the same as for hostfxr_get_available_sdks - that is order by version ascending and put multi-level lookup locations before private locations - items later in the list have priority over items earlier in the list

Ordering frameworks should be done by name ascending followed by version ascending. It should put multi-level lookup locations before private locations.

Note: dotnet --info already implements almost all of the order rules. The only missing bit is multi-level lookup ordering of frameworks. Currently if both locations have the same name/version framework, the order seems to be undefined.

Performance considerations

While having one API which gathers all of the information is technically slower, since if only runtimes are needed, it would still do all the work for SDKs as well, scenarios using this API are very unlikely to be that performance sensitive and there's really no use case to call this API repeatedly.

Open questions

Result order

The existing hostfxr_get_available_sdks implements a very specific ordering algorithm to make it easy to search for latest SDK version - this includes special handling of multi-level lookup locations. See the code comment for more details.

Options:

1. (currently proposed above) Use the existing ordering for SDK as per hostfxr_get_available_sdks and implement something similar for frameworks.
2. Declare the ordering as "undefined" and leave it up to the consumer - this would cause potential problems around multi-level lookup handling as mentioned above.

Alternatives

dotnet --info --json

#3049 Add support for JSON output in dotnet --info

Implement just dotnet --info --json (or similar) which would produce JSON to stdout. This would require out-of-proc usage and JSON parsing in all use cases. This is not always desirable. Having an in-proc API still allows us to implement dotnet --info --json on top of it relatively easily.

Use exiting hostfxr_get_available_sdks and add new API for runtimes

hostfxr already exposes hostfxr_get_available_sdks which basically implements the SDKs part of the dotnet --info, but there's no runtime counterpart. Alternatively something like hostfxr_get_available_frameworks could be added. This approach has couple of issues:

• hostfxr_get_available_sdks only reports the paths to the SDKs, so if the consumer wants to get version numbers it would have to parse the paths. We would need to introduce something like hostfxr_get_available_sdks2 to fix this problem as the current API is not extensible.

• There would be no natural API to return "global" information about the host. If there's a need for such information, another API would have to be addedd.

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Goal

Currently the hostfxr has the ability to enumerate all accessible runtimes and SDKs given a certain environment. This functionality is only exposed via the dotnet --info command which provides humanly readable output. This is useful for humans, but it's hard to consume for machines - if there's a need to get this information from code, parsing this text is problematic at best.

Design

Add a new API hostfxr_get_dotnet_environment_info which would provide both runtimes, SDKs and other information in one call.

struct hostfxr_dotnet_environment_sdk_info
{
    size_t size;
    const pal::char_t *version;
    const pal::char_t *path;
}

struct hostfxr_dotnet_environment_framework_info
{
    size_t size;
    const pal::char_t *name;
    const pal::char_t *version;
    const pal::char_t *path;
}

struct hostfxr_dotnet_environment_info
{
    size_t size;

    const pal::char_t *hostfxr_version;
    const pal::char_t *hostfxr_commit_hash;

    int32_t sdk_count;
    const hostfxr_dotnet_environment_sdk_info *sdks;

    int32_t framework_count;
    const hostfxr_dotnet_environment_framework_info *frameworks;
}

typedef void (HOSTFXR_CALLTYPE *hostfxr_get_dotnet_environment_info_result_fn)(
    const hostfxr_dotnet_environment_info *info);

SHARED_API int32_t HOSTFXR_CALLTYPE hostfxr_get_dotnet_environment_info(
    const pal::char_t *dotnet_root,
    void *reserved,
    hostfxr_get_dotnet_environment_info_result_fn result);

This follows the existing pattern in hostfxr APIs where the caller provides a function which is called with the results of the operation. This is to avoid memory ownership/allocation issues. The result function is passed buffers allocated by the hostfxr which are only valid for that one call. This allows hostfxr to use internal buffers or stack allocated memory and makes it easy to correctly cleanup.

The result function must make copies of everything it needs to keep before returning.

The size fields are added for versioning and extensibility. The field is initialized with the sizeof(struct), the consumer should check that the value in size is equal or greater to the size it expects. This allows us to add more fields to the structures going forward without versioning the API itself.

For similar reasons the reserved parameter is added, right now it should always be null. If we need to pass additional parameter in the future, the parameter would pass a pointer to a structure with those parameters.

The dotnet_root parameter should point to the directory with dotnet.exe if the caller needs to specify one. It can be null in which case the default global location will be used instead. This is to make the API equivalent in capability to dotnet --info. Running dotnet --info passes a "hidden" path to the tool - the path to the dotnet.exe actually used.

Locating and loading hostfxr

It's the responsibility of the caller to locate and load hostfxr. The nethost library can be used to do this.

The location of hostfxr used to make this API call should not affect the result in any way. That is, it doesn't matter which hostfxr is used (ignoring differences in functionality across future versions), given the same input, all of them should produce the same output.

Performance considerations

While having one API which gathers all of the information is technically slower, since if only runtimes are needed, it would still do all the work for SDKs as well, scenarios using this API are very unlikely to be that performance sensitive and there's really no use case to call this API repeatedly.

Alternatives

dotnet --info --json

#3049 Add support for JSON output in dotnet --info

Implement just dotnet --info --json (or similar) which would produce JSON to stdout. This would require out-of-proc usage and JSON parsing in all use cases. This is not always desirable. Having an in-proc API still allows us to implement dotnet --info --json on top of it relatively easily.

Use exiting hostfxr_get_available_sdks and add new API for runtimes

hostfxr already exposes hostfxr_get_available_sdks which basically implements the SDKs part of the dotnet --info, but there's no runtime counterpart. Alternatively something like hostfxr_get_available_frameworks could be added. This approach has couple of issues:

• hostfxr_get_available_sdks only reports the paths to the SDKs, so if the consumer wants to get version numbers it would have to parse the paths. We would need to introduce something like hostfxr_get_available_sdks2 to fix this problem as the current API is not extensible.

• There would be no natural API to return "global" information about the host. If there's a need for such information, another API would have to be addedd.

Author:	vitek-karas
Assignees:	-
Labels:	area-Host
Milestone:	-

[rseanhall]

I haven't looked at the API itself yet, but here are my thoughts at the high level. For our use case of detecting which runtimes are installed (wixtoolset/issues#6257), we have to run out-of-proc anyway to be able to detect runtimes for architectures other than our own. Having said that, this API would be much nicer than having to call into dotnet because then we don't have to know how to locate dotnet.

I brought up in #35244 that it would be nice to be able to statically link hostfxr. This is the perfect scenario to do this so our utility executable can be a single file.

[rseanhall]

The API looks good. How much of an implementation detail is fx_ver? It would be useful if the API could also provide a data structure with the version pre-parsed, especially whether the version is considered a prerelease version.

[vitek-karas]

Well - for one we would need a C version of the struct. But it doesn't sound too complicated.

[vitek-karas]

I updated the issue with:

• Describe behavior with regard to searching for SDKs and frameworks (identical to dotnet --info existing behavior)
• Describe ordering of results (very similar to existing dotnet --info existing behavior)

[rseanhall]

From the spec in the linked issue:

As the current implementation of the dotnet --info command determines installed bundles based only on directories being present, it is possible for it to wrongly report that a bundle exists. We will mitigate this issue by adding some further sanity checking to ensure that a given bundle is present, for example, by ensuring that the bundle's directory is not empty.

It would be nice if this API contains this mitigation logic.

[vitek-karas]

@rseanhall We've been discussing this - and I don't think there's an agreement. It's potentially never ending thing - what check is good enough to validate that a given SDK/framework is actually present on disk? Is it a non-empty directory? What if there are some files, but for example coreclr.dll is missing - is that "existing" or not? And so on...

The reason this is problematic is not just the unclear definition of the behavior but also performance implications. These checks would have to happen during framework version resolution on each application start (for framework dependent apps). And these checks typically consist of File IO - which is not exactly cheap.

There's also the question of "What to do about it" - if the host completely ignores "corrupted" frameworks, then there might be issues with installers. For example I run dotnet --info and see that I don't have 5.0 installed, so I go and run the 5.0 installer and that fails because 5.0 is actually at least partially on disk. This is solvable, but it makes things a bit weird. In this regard the installers (or installer related code) has more knowledge about things and can make better decisions - so for the host to skip over frameworks feels a bit dangerous.

[rseanhall]

The reason this is problematic is not just the unclear definition of the behavior but also performance implications. These checks would have to happen during framework version resolution on each application start (for framework dependent apps). And these checks typically consist of File IO - which is not exactly cheap.

I don't understand the direction you started going here. Why does the host or dotnet --info need to start using the logic in this API?

[vitek-karas]

I would find it really weird if the API, dotnet --info and app start had different behaviors. For example if the new dotnet CLI told me I don't have 5.0, but running dotnet --info shows it?