Add layout logic to replace duplicate files with links (Linux/MacOS only)#52044
Merged
MichaelSimons merged 14 commits intodotnet:mainfrom Feb 4, 2026
Merged
Add layout logic to replace duplicate files with links (Linux/MacOS only)#52044MichaelSimons merged 14 commits intodotnet:mainfrom
MichaelSimons merged 14 commits intodotnet:mainfrom
Conversation
MichaelSimons
force-pushed
the
file-deduplication
branch
from
435653b to
f0f84b6
Compare
MichaelSimons
force-pushed
the
file-deduplication
branch
6 times, most recently
from
e9c827c to
c244f53
Compare
This was referenced Jan 12, 2026
MichaelSimons
force-pushed
the
file-deduplication
branch
2 times, most recently
from
8dde6ac to
7a5f9bc
Compare
MichaelSimons
force-pushed
the
file-deduplication
branch
from
af91260 to
6391fa5
Compare
3 tasks
MichaelSimons
force-pushed
the
file-deduplication
branch
4 times, most recently
from
707bf9f to
8cdddbf
Compare
MichaelSimons
changed the title
MichaelSimons
force-pushed
the
file-deduplication
branch
4 times, most recently
from
6ca1b07 to
5f75bc1
Compare
MichaelSimons
force-pushed
the
file-deduplication
branch
from
62a67d2 to
5f75bc1
Compare
MichaelSimons
force-pushed
the
file-deduplication
branch
from
9ecdf93 to
7e233d1
Compare
MichaelSimons
commented
Jan 29, 2026
| /// <summary> | ||
| /// Shared helpers for verifying symbolic links and extracting archives in tests. | ||
| /// </summary> | ||
| internal static class SymbolicLinkHelpers |
Member
Author
There was a problem hiding this comment.
This logic was factored into a shared helper so that it can be shared with the linux/max installer tests which are coming in a follow-up PR.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds automated deduplication of assemblies in the SDK layout for Linux and macOS builds by replacing duplicate .dll and .exe files with symbolic links. The implementation uses content hashing (XxHash64) to identify duplicates and selects a deterministic "master" file (closest to root, alphabetically first) to which all duplicates are linked using relative symbolic links.
Changes:
- New MSBuild task
DeduplicateAssembliesWithLinksthat scans a layout directory, groups assemblies by content hash, and replaces duplicates with links - End-to-end tests verifying that SDK archives contain the expected number of symbolic links with relative paths
- Test infrastructure updates to locate and verify SDK acquisition artifacts
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Tasks/sdk-tasks/DeduplicateAssembliesWithLinks.cs | New MSBuild task implementing assembly deduplication via hard/symbolic links |
| src/Tasks/sdk-tasks/sdk-tasks.csproj | Added System.IO.Hashing package dependency for content hashing |
| src/Tasks/sdk-tasks/sdk-tasks.InTree.targets | Registered the new DeduplicateAssembliesWithLinks task for Core runtime |
| src/Tasks/sdk-tasks/ReplaceFilesWithSymbolicLinks.cs | Fixed comment to correctly state "symbolic links" instead of "hard links" |
| src/Layout/redist/targets/GenerateInstallerLayout.targets | Integrated deduplication task into build pipeline for non-Windows platforms |
| test/sdk-tasks.Tests/DeduplicateAssembliesWithLinksTests.cs | Comprehensive unit tests for the deduplication task |
| test/EndToEnd.Tests/GivenSdkArchives.cs | End-to-end test verifying deduplication in SDK archives |
| test/EndToEnd.Tests/Utilities/SymbolicLinkHelpers.cs | Shared utilities for extracting archives and verifying symbolic links |
| test/Microsoft.NET.TestFramework/TestContext.cs | Added ShippingPackagesDirectory property and helper method to locate SDK artifacts |
MiYanni
approved these changes
Jan 29, 2026
MichaelSimons
changed the title
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
baronfel
reviewed
Jan 30, 2026
Co-authored-by: Chet Husk <baronfel@users.noreply.github.com>
dsplaisted
approved these changes
Feb 3, 2026
Update terminology to use 'primary' instead of 'master' for the file that duplicates are linked to.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR reduces SDK size by deduplicating assemblies in the SDK layout using symbolic links on Linux and macOS. Duplicate .dll and .exe files are identified by content hash and replaced with symbolic links pointing to a single "master" copy.
This PR only affects the linux/mac tarballs. Installers will be handled in a separate PR.
Key Changes
Why Symbolic Links Instead of Hard Links?
Hard links would be preferred, but are blocked by dotnet/arcade#16453 which affects RPM packaging. Symbolic links work correctly with current packaging infrastructure until unblocked.
Why No Windows Support?
Windows deduplication requires signing infrastructure changes and is blocked by dotnet/arcade#16484. This will be addressed in #52182.
Related Issues