IL2104 when trimming application using System.CommandLine

[perlun]

Hi,

When starting to prepare my application for .NET 6 (perlang-org/perlang#223), I am seeing errors like this:

/home/per/.nuget/packages/system.commandline/2.0.0-beta1.21216.1/lib/netstandard2.0/System.CommandLine.dll : error IL2104: Assembly 'System.CommandLine' produced trim warnings. For more information see https://aka.ms/dotnet-illink/libraries [/home/per/git/perlang/src/Perlang.ConsoleApp/Perlang.ConsoleApp.csproj]

I wonder if there would be any way to fix this, apart from disabling the trimming warnings altogether? 🤔 .NET 6 does introduce a bunch of trimming-related changes (https://devblogs.microsoft.com/dotnet/announcing-net-6/#il-trimming), so I guess this is a consequence of the more advanced analysis it does before performing the trimming.

[jonsequitur]

Without looking into this more closely, my guess is this is mostly around the name-based command handler APIs. We're looking at layering those into a separate library which might help here. The ones that don't use name-based matching are much more performant anyway.

[perlun]

Sounds good @jonsequitur. I ended up disabling the linker warnings altogether in my project 🙈 for a few reasons:

• The attribute-based approach was not completely straightforward for some of the warnings (was getting them for some serialization assembly or something)
• I'm not sure how to silence it for 3rd party dependencies like System.CommandLine
• It feels like it does clutter my codebase in a somewhat unpleasant way.

All of these might be resolvable, but for now, putting the head in the sand seemed like the easiest way to move forward. 😁

---

Having that said, feel free to keep this issue since it might be relevant to others running into similar problems.

[omajid]

I am seeing a functional impact here.

From my deps.json:

  "targets": {
    ".NETCoreApp,Version=v6.0": {},
    ".NETCoreApp,Version=v6.0/linux-x64": {
      "Turkey/9-3664abe": {
        "dependencies": {
          "Microsoft.CodeQuality.Analyzers": "2.6.0",
          "Microsoft.NetCore.Analyzers": "2.6.0",
          "Newtonsoft.Json": "12.0.1",
          "System.CommandLine": "2.0.0-beta1.21308.1",
          "Text.Analyzers": "2.6.0",
          "runtimepack.Microsoft.NETCore.App.Runtime.linux-x64": "6.0.0"
...

I am publishing it using: dotnet publish -c Release -r linux-x64 --self-contained -p:PublishSingleFile=true -p:PublishTrimmed=true

Running this application shows:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.CSharp, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'Microsoft.CSharp, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at System.CommandLine.IO.ConsoleExtensions.ResetTerminalForegroundColor(IConsole console)
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<UseExceptionHandler>g__Default|1(Exception exception, InvocationContext context)
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<<UseExceptionHandler>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<CancelOnProcessTermination>b__5_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(IConsole console)
   at System.CommandLine.CommandExtensions.InvokeAsync(Command command, String[] args, IConsole console)
   at Turkey.Program.Main(String[] args)
   at Turkey.Program.<Main>(String[] args)

I copied some code from ResetTerminalForegroundColor into sharplab.io here. It does look like there's an actual dependency from System.CommandLine.IO.ConsoleExtensions to Microsoft.CSharp.RuntimeBinder.

[jonsequitur]

We plan to relayer to remove the dependency on Microsoft.CSharp and move the reflection-dependent CommandHandler.Create methods out to a separate assembly.

[jonsequitur]

Related: #1472

[omajid]

Thanks! Do you have any suggestions on how to make the current releases work? Is there a particular DynamicDependencyAttribute or TrimmerRootDescriptor that's known to work? I have been playing around with them based on advice from this blog post but haven't had any luck. Or should I give up on PublishTrimmed with .NET 6 and current versions of System.CommandLine?

[Caleb9]

Hi.

I just resolved the same issue in another library I'm using, then my build failed because System.CommandLine also suffers from it, and I had warnings as errors enabled. The solution required targeting net6.0 and adding <IsTrimmable>true</IsTrimmable> to .csproj file (see here - this tag is only valid in .NET 6+). I could make a similar PR to this repo but I'm not sure if you're OK with targeting net6.0 aside from netstandard2.0? Seeing that this manifests itself only in .NET 6 projects, which requires adding that <IsTrimmable> tag I'm not sure if there's a way around it though (assuming it would also solve the problem here).

[MichalStrehovsky]

If a library generates trimming warnings, adding IsTrimmable to it will not fix them. The library is still not trimmable, but you're allowing trimming anyway. There are two outcomes of that:

1. People who use the library with trimming and happen to only use parts that are fine for trimming (no unpredictable reflection) will see warnings disappear because the problematic code got trimmed away.
2. People who use the library with trimming and happen to use parts that are not okay with trimming will now see dozens of warnings for each individual part of the library instead of a single "Assembly 'System.CommandLine' produced trim warnings". IsTrimmable enables detailed warning reporting instead of the aggregate "There were some problems in this library".

Adding IsTrimmable without changing anything else in the library is never the right fix for "Assembly 'X' produced trim warnings".

The right workflow for libraries is documented at https://docs.microsoft.com/en-us/dotnet/core/deploying/trimming/prepare-libraries-for-trimming. You first deal with the warnings, and then mark the library trimmable.

---

You can do an equivalent of adding IsTrimmable to library X by using the TrimmableAssembly item in your app's csproj:

https://docs.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options#trim-additional-assemblies

E.g. for System.CommandLine you would do:

<ItemGroup>
  <TrimmableAssembly Include="System.CommandLine" />
</ItemGroup>

If you're in category 1 above, the warnings will disappear. If you're in category 2, you'll see more warnings.

[Caleb9]

@MichalStrehovsky good to know, thanks, I'll look into it when I get the time.

However I observed different behavior in the mentioned library that I "fixed", i.e. I started by just targeting net6.0 and NOT adding <IsTrimmable> and the warning was still generated from dotnet publish in a project that referenced the library. According to your description, adding the tag should generate even more warnings, if I understand correctly? But it actually removed the warning altogether 🤷‍♂️. If you happen to know what's going on, then please let me know.

Thanks.

[MichalStrehovsky]

It will remove the warnings if the consuming app is in category 1 above - trimming the library got rid of code that couldn't be reasoned about and there's nothing left that would be problematic. But a different user of the library might have a different experience (unless the problematic code in question is always unreachable, in which case it would be better to just delete the unreachable code).

The main thing with PublishTrimmed that people often don't realize is that by default, only assemblies that are marked IsTrimmable will participate in trimming. The other assemblies will be analyzed for trimming problems, and preserved in their entirety by the trimming process for compat reasons. Marking a library as IsTrimmable enables trimming the library. As of right now, very few libraries outside the framework are marked trimmable.

[Caleb9]

@MichalStrehovsky Ok I see and understand a bit better now. Thanks.

So to sum up:

• Without <IsTrimmable> you'll get one aggregate warning when there are potential trimming problems within the library. Even when the code using the library does not use the problematic parts. But the library will not actually get trimmed so risk of creating code that will break because of trimming is removed. If the trimming analyzer does not find problems within the library, you don't get any warnings.
• With <IsTrimmable> you'll get detailed warnings when code is using problematic parts of the library, but you won't get any if either the library does not have any problems, or if the code using it is not using the problematic parts (the latter must've been my case and that's why adding the tag removed the warning).

Is that correct?

[MichalStrehovsky]

Yup.

Without IsTrimmable, the trimming process considers everything in the assembly implicitly used, won't trim it, and will raise warnings if there's any problems in the code.

[Caleb9]

I better go back and tell them to revert my "fix" then :D. Thanks for taking time to explain this to me.

[Caleb9]

One remark is that setting

<ItemGroup>
  <TrimmableAssembly Include="System.CommandLine" />
</ItemGroup>

I still get the aggregate warning (and not dozens of specific warnings, or nothing). Maybe because the library is targeting netstandard2.0 and not net6.0? It would make sense if it really is equivalent to setting <IsTrimmable> on that assembly but from the consuming app level, and <IsTrimmable> is not available in netstandard2.0.