Performance issue calling Directory.GetFileSystemEntries

[aurnor]

Description

On .NET 10, when calling Directory.GetFileSystemEntries to find a folder on a remote share, the performance is very slow as compared to the exact same call using .NET Framework.
All tests done on Windows 11.

When the share contains 34k folders:

• .NET Framework call duration: 1ms
• .NET 10 call duration: ~1,000ms

When the share contains 450k folders:

• .NET Framework call duration: ~35ms
• .NET 10 call duration: ~30,000ms

Repro steps:

• On Windows create a folder containing thousands of subfolders. This can be done easily with PowerShell:

    $parentPath = "C:\temp\DotNetTest"
    
    # Create the parent directory if it doesn't exist
    if (-not (Test-Path -Path $parentPath)) {
        New-Item -ItemType Directory -Path $parentPath | Out-Null
    }
    
    # Loop to create 34,000 folders
    for ($i = 1; $i -le 34000; $i++) {
        $folderPath = Join-Path -Path $parentPath -ChildPath $i
        New-Item -ItemType Directory -Path $folderPath | Out-Null
    }
  

  Now C:\temp\DotNetTest contains 34k folders named 1, 2, 3, ... 35000

• Share the DotNetTest folder.

• From the same machine (or a remote machine, both scenarios show the same performance issue), run the following code to search any folder in the share (for example folder named "25002"):

  using System;
  using System.IO;
  using System.Runtime.InteropServices;
  
  namespace TestConsoleApp
  {
      internal class Program
      {
          static void Main(string[] args)
          {
              Console.WriteLine(RuntimeInformation.FrameworkDescription);
              var stopwatch = System.Diagnostics.Stopwatch.StartNew();
  
  #if NET8_0_OR_GREATER
              //var enumOptions = new EnumerationOptions()
              //{
              //    RecurseSubdirectories = false,
              //    IgnoreInaccessible = false,
              //    AttributesToSkip = 0,
              //    MatchType = MatchType.Win32,
              //    ReturnSpecialDirectories = false,
              //    MaxRecursionDepth = 0,
              //    MatchCasing = MatchCasing.PlatformDefault,
              //    BufferSize = 1024,
              //};
  
              var entries = Directory.GetFileSystemEntries(@"\\<machineName>\DotNetTest\", "25002"); //, enumOptions);
  #elif NET472_OR_GREATER
              var entries = Directory.GetFileSystemEntries(@"\\<machineName>\DotNetTest\", "25002");
  #endif
              stopwatch.Stop();
              Console.WriteLine($"Entries found: {string.Join(", ", entries)}");
              Console.WriteLine($"Execution time: {stopwatch.ElapsedMilliseconds} ms");
          }
      }
  }
  

With net10.0, the output is (with 35k folders):
.NET 10.0.0
Entries found: \\machineName\DotNetTest\25002
Execution time: 1218 ms

With net481, the output is (with 35k folders):
.NET Framework 4.8.9323.0
Entries found: \\machineName\DotNetTest\25002
Execution time: 1 ms

With .NET 10, a System.IO.EnumerationOptions object can be passed as a parameter to GetFileSystemEntries. But none of the proposed options helps reduce the time taken.

Configuration

.NET 10
Windows 11 25H2 x64

Regression?

Yes, fast on .NET Framework, slow on .NET 8+ (not tested on previous versions of .NET / .NET Core)

Data

Running a procmon trace explains why the operation takes so much more time:
With .NET Framework:

15:19:36 | TestConsoleApp.exe | 40832 | QueryDirectory | \\machineName\DotNetTest\25002 | SUCCESS | FileInformationClass: FileBothDirectoryInformation, Filter: 25002, 2: 25002
15:19:36 | TestConsoleApp.exe | 40832 | QueryDirectory | \\machineName\DotNetTest\ | NO MORE FILES | FileInformationClass: FileBothDirectoryInformation

We can see that only 1 QueryDirectory call is made with FileInformationClass: FileBothDirectoryInformation

With .NET 10:
15:21:06 | TestConsoleApp.exe | 40156 | QueryDirectory | \\machineName\DotNetTest\ | SUCCESS | FileInformationClass: FileFullDirectoryInformation, 1: ., 2: .., 3: 1, 4: 10, 5: 100, 6: 1000
15:21:06 | TestConsoleApp.exe | 40156 | QueryDirectory | \\machineName\DotNetTest\ | SUCCESS | FileInformationClass: FileFullDirectoryInformation, 1: 10041, 2: 10042, 3: 10043, 4: 10044, 5: 10045, 6: 10046
15:21:06 | TestConsoleApp.exe | 40156 | QueryDirectory | \\machineName\DotNetTest\ | SUCCESS | FileInformationClass: FileFullDirectoryInformation, 1: 10088, 2: 10089, 3: 1009, 4: 10090, 5: 10091, 6: 10092
(...)
15:21:10 | TestConsoleApp.exe | 40156 | QueryDirectory | \\machineName\DotNetTest\ | SUCCESS | FileInformationClass: FileFullDirectoryInformation, 1: 9964, 2: 9965, 3: 9966, 4: 9967, 5: 9968, 6: 9969

We can see that many QueryDirectory calls (almost 700 calls) are made with FileInformationClass: FileFullDirectoryInformation

As the number of folders grows, the number of calls to QueryDirectory grows as well, resulting in performance degradation.
Increasing the System.IO.EnumerationOptions BufferSize helps reduce the number of calls, but has no effect on the total execution time.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

[MihuBot]

I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

• System.IO.Directory.GetFiles with searchFilter is about 1000 times slower compared to .net framework #56464

[jozkee]

As the number of folders grows, the number of calls to QueryDirectory grows as well, resulting in performance degradation.

IIRC, is not only the number of calls, filling the buffer also has a significant cost, you could fetch all results in one call using a large enough buffer and still it would take significantly longer because of the amount of bytes being copied. This is consequence of filtering results in managed code as opposed to .NET Framework where the filter was passed to the Windows API.

I gave a similar response in #110754 (comment).

[stephentoub]

This should be addressed by #122947 and #123695.