RAR cracks open system files in empty incremental build

[ladipro]

Issue Description

The RAR on-disk cache appears to be missed for some subset of system dependencies. Even if the solution is built incrementally with no changes since the last successful build, I'm seeing a large number of calls to AssemblyName.GetAssemblyName for files like:

C:\Users\laprosek.EUROPE\.nuget\packages\microsoft.identitymodel.protocols.openidconnect\5.6.0\lib\netstandard2.0\Microsoft.IdentityModel.Protocols.OpenIdConnect.dll
C:\Users\laprosek.EUROPE\.nuget\packages\microsoft.identitymodel.tokens\5.6.0\lib\netstandard2.0\Microsoft.IdentityModel.Tokens.dll
C:\Users\laprosek.EUROPE\.nuget\packages\system.identitymodel.tokens.jwt\5.6.0\lib\netstandard2.0\System.IdentityModel.Tokens.Jwt.dll
C:\Users\laprosek.EUROPE\.nuget\packages\cachemanager.core\2.0.0-beta-1629\lib\netstandard2.0\CacheManager.Core.dll
C:\Users\laprosek.EUROPE\.nuget\packages\cachemanager.microsoft.extensions.configuration\2.0.0-beta-
C:\Users\laprosek.EUROPE\.nuget\packages\cachemanager.microsoft.extensions.logging\2.0.0-beta-1629\lib\netstandard2.0\CacheManager.Microsoft.Extensions.Logging.dll
C:\Users\laprosek.EUROPE\.nuget\packages\system.configuration.configurationmanager\4.5.0\ref\netstandard2.0\System.Configuration.ConfigurationManager.dll
...

Steps to Reproduce

Clone the Ocelot repo and build it. Notice that this code is executed for many assemblies, suggesting that the cache does not work well:

msbuild/src/Tasks/SystemState.cs

Line 449 in ea1d6d9

fileState.Assembly = getAssemblyName(path);

Data

~340 cache misses when building Ocelot, costing us ~2.5% of incremental build time.

Analysis

Nothing obvious. The cache entry is updated with the assembly name and the cache is marked dirty so it should be written to the cache file and read back in subsequent builds.

Versions & Configurations

Microsoft (R) Build Engine version 17.0.0-dev-21464-01+c82d55e9b for .NET Framework

Regression?

N/A

Attach a binlog

N/A

[rokonec]

It might be related to way RAR optimized amount of data stored in RAR on-disk-cache.
The current implementation of this

msbuild/src/Tasks/SystemState.cs

Lines 366 to 379 in c8d4b38

	// If the process-wide cache contains an up-to-date FileState, always use it
	if (isProcessFileStateUpToDate)
	{
	// If a FileState already exists in this instance cache due to deserialization, remove it;
	// another instance has taken responsibility for serialization, and keeping this would
	// result in multiple instances serializing the same data to disk
	if (isCachedInInstance)
	{
	instanceLocalFileStateCache.Remove(path);
	isDirty = true;
	}
	return cachedProcessFileState;
	}

is, IMHO, wrong (indeterministic, sub-optimal).
Maybe with recent optimization of RAR on-disk-cache size and serialization perf, we can now afford to store all project dependencies, and allow cross-project redundancy.

[Forgind]

That was actually a problem I'd mentioned in my RAR precaching document but never followed up on. Wish I had!