From 7e8a883ca25e52a6d7a78f6d726b2befdc49bdf9 Mon Sep 17 00:00:00 2001 From: Muhammad Danish Date: Sat, 26 Aug 2023 12:16:47 +0500 Subject: [PATCH 1/3] Replace argument for removing a previous manifest --- doc/update.md | 18 ++-- src/WingetCreateCLI/Commands/BaseCommand.cs | 6 +- src/WingetCreateCLI/Commands/UpdateCommand.cs | 40 ++++++++- .../Properties/Resources.Designer.cs | 27 ++++++ src/WingetCreateCLI/Properties/Resources.resx | 13 +++ src/WingetCreateCore/Common/GitHub.cs | 85 +++++++++++++------ 6 files changed, 153 insertions(+), 36 deletions(-) diff --git a/doc/update.md b/doc/update.md index e5947093..2670a332 100644 --- a/doc/update.md +++ b/doc/update.md @@ -1,24 +1,25 @@ # update command (Winget-Create) -The **update** command of the [Winget-Create](../README.md) tool is designed to update an existing manifest. The **update** command is non-interactive so that it can be seamlessly integrated into your build pipeline to assist with the publishing of your installer. The **update** command will update the manifest with the new URL, hash and version and can automatically submit the pull request (PR) to the [Windows Package Manager repo](https://docs.microsoft.com/windows/package-manager/). +The **update** command of the [Winget-Create](../README.md) tool is designed to update an existing manifest. The **update** command is non-interactive so that it can be seamlessly integrated into your build pipeline to assist with the publishing of your installer. The **update** command will update the manifest with the new URL, hash and version and can automatically submit the pull request (PR) to the [Windows Package Manager repo](https://docs.microsoft.com/windows/package-manager/). ## Usage -`wingetcreate.exe update [-u ] [-v ] [-s] [-t ] [-o ]` +`wingetcreate.exe update [-u ] [-v ] [-s] [-t ] [-o ] [-p ] [-r] []` -The **update** command can be called with the installer URL(s) that you wish to update the manifest with. **Please make sure that the number of installer URL(s) included matches the number of existing installer nodes in the manifest you are updating. Otherwise, the command will fail.** This is to ensure that we can deterministically update each installer node with the correct matching installer url provided. +The **update** command can be called with the installer URL(s) that you wish to update the manifest with. **Please make sure that the number of installer URL(s) included matches the number of existing installer nodes in the manifest you are updating. Otherwise, the command will fail.** This is to ensure that we can deterministically update each installer node with the correct matching installer url provided. > **Note**\ > The [show](show.md) command can be used to quickly view an existing manifest from the packages repository. ### *How does Winget-Create know which installer(s) to match when executing an update?* -[Winget-Create](../README.md) will attempt to match installers based on the installer architecture and installer type. The installer type will always be derived from downloading and analyzing the installer package. +[Winget-Create](../README.md) will attempt to match installers based on the installer architecture and installer type. The installer type will always be derived from downloading and analyzing the installer package. There are cases where the intended architecture specified in the existing manifest can sometimes differ from the actual architecture of the installer package. To mitigate this discrepancy, the installer architecture will first be determined by performing a regex string match to identify the possible architecture in the installer url. If no match is found, [Winget-Create](../README.md) will resort to obtaining the architecture from the downloaded installer. ## Usage Examples + Search for an existing manifest and update the version: `wingetcreate.exe update --version ` @@ -53,14 +54,17 @@ The following arguments are available: | **-v, --version** | Version to be used when updating the package version field. | **-o, --out** | The output directory where the newly created manifests will be saved locally | **-s, --submit** | Boolean value for submitting to the Windows Package Manager repo. If true, updated manifest will be submitted directly using the provided GitHub Token +| **-r, --replace** | Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false. | **-p, --prtitle** | The title of the pull request submitted to GitHub. | **-t, --token** | GitHub personal access token used for direct submission to the Windows Package Manager repo. If no token is provided, tool will prompt for GitHub login credentials. | **-?, --help** | Gets additional help on this command. | -## Submit +## Submit -If you have provided your [GitHub token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) on the command line along with the **--submit** flag, **Winget-Create** will automatically submit your PR to [Windows Package Manager repo](https://docs.microsoft.com/windows/package-manager/). +If you have provided your [GitHub token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) on the command line along with the **--submit** flag, **Winget-Create** will automatically submit your PR to [Windows Package Manager repo](https://docs.microsoft.com/windows/package-manager/). Instructions on setting up GitHub Token for Winget-Create can be found [here](../README.md#github-personal-access-token-classic-permissions). -## Output + +## Output + If you would like to write the file to disk rather than submit to the repository, you can pass in the **--output** command along with the file name to write to. diff --git a/src/WingetCreateCLI/Commands/BaseCommand.cs b/src/WingetCreateCLI/Commands/BaseCommand.cs index 6b98db50..e1ca5ff8 100644 --- a/src/WingetCreateCLI/Commands/BaseCommand.cs +++ b/src/WingetCreateCLI/Commands/BaseCommand.cs @@ -611,8 +611,10 @@ protected async Task CheckGitHubTokenAndSetClient() /// /// Wrapper object for manifest object models to be submitted. /// Optional parameter specifying the title for the pull request. + /// Optional parameter specifying whether the new submission should replace an existing manifest. + /// Optional parameter specifying the version of the manifest to be replaced. /// A representing the success of the asynchronous operation. - protected async Task GitHubSubmitManifests(Manifests manifests, string prTitle = null) + protected async Task GitHubSubmitManifests(Manifests manifests, string prTitle = null, bool shouldReplace = false, string replaceVersion = null) { if (string.IsNullOrEmpty(this.GitHubToken)) { @@ -625,7 +627,7 @@ protected async Task GitHubSubmitManifests(Manifests manifests, string prT try { - PullRequest pullRequest = await this.GitHubClient.SubmitPullRequestAsync(manifests, this.SubmitPRToFork, prTitle); + PullRequest pullRequest = await this.GitHubClient.SubmitPullRequestAsync(manifests, this.SubmitPRToFork, prTitle, shouldReplace, replaceVersion); this.PullRequestNumber = pullRequest.Number; PullRequestEvent pullRequestEvent = new PullRequestEvent { IsSuccessful = true, PullRequestNumber = pullRequest.Number }; TelemetryManager.Log.WriteEvent(pullRequestEvent); diff --git a/src/WingetCreateCLI/Commands/UpdateCommand.cs b/src/WingetCreateCLI/Commands/UpdateCommand.cs index 2d06f340..d1aac9ad 100644 --- a/src/WingetCreateCLI/Commands/UpdateCommand.cs +++ b/src/WingetCreateCLI/Commands/UpdateCommand.cs @@ -54,6 +54,12 @@ public static IEnumerable Examples [Value(0, MetaName = "PackageIdentifier", Required = true, HelpText = "PackageIdentifier_HelpText", ResourceType = typeof(Resources))] public string Id { get; set; } + /// + /// Gets or sets the previous version to remove from the Windows Package Manager repository. + /// + [Value(1, MetaName = "ReplaceVersion", Required = false, HelpText = "ReplaceVersion_HelpText", ResourceType = typeof(Resources))] + public string ReplaceVersion { get; set; } + /// /// Gets or sets the new value used to update the manifest version element. /// @@ -78,6 +84,12 @@ public static IEnumerable Examples [Option('s', "submit", Required = false, HelpText = "SubmitToWinget_HelpText", ResourceType = typeof(Resources))] public bool SubmitToGitHub { get; set; } + /// + /// Gets or sets a value indicating whether or not to remove a previous version of the manifest with the update. + /// + [Option('r', "replace", Required = false, HelpText = "ReplacePrevious_HelpText", ResourceType = typeof(Resources))] + public bool Replace { get; set; } + /// /// Gets or sets a value indicating whether to launch an interactive mode for users to manually select which installers to update. /// @@ -97,9 +109,9 @@ public static IEnumerable Examples public IEnumerable InstallerUrls { get; set; } = new List(); /// - /// Gets or sets the unbound arguments that exist after the first positional parameter. + /// Gets or sets the unbound arguments that exist after the positional parameters. /// - [Value(1, Hidden = true)] + [Value(2, Hidden = true)] public IList UnboundArgs { get; set; } = new List(); /// @@ -144,6 +156,26 @@ public override async Task Execute() this.Id = exactId; } + if (!string.IsNullOrEmpty(this.ReplaceVersion)) + { + // If update version is same as replace version, it's a regular update. + if (this.Version == this.ReplaceVersion) + { + this.Replace = false; + } + + // Check if the replace version exists in the repository. + try + { + await this.GitHubClient.GetManifestContentAsync(this.Id, this.ReplaceVersion); + } + catch (Octokit.NotFoundException) + { + Logger.ErrorLocalized(nameof(Resources.VersionDoesNotExist_Error), this.Version, this.Id); + return false; + } + } + List latestManifestContent; try @@ -209,7 +241,9 @@ await this.UpdateManifestsInteractively(initialManifests) : return await this.LoadGitHubClient(true) ? (commandEvent.IsSuccessful = await this.GitHubSubmitManifests( updatedManifests, - this.GetPRTitle(updatedManifests, originalManifests))) + this.GetPRTitle(updatedManifests, originalManifests), + this.Replace, + this.ReplaceVersion)) : false; } diff --git a/src/WingetCreateCLI/Properties/Resources.Designer.cs b/src/WingetCreateCLI/Properties/Resources.Designer.cs index 45088def..0cc840d0 100644 --- a/src/WingetCreateCLI/Properties/Resources.Designer.cs +++ b/src/WingetCreateCLI/Properties/Resources.Designer.cs @@ -2157,6 +2157,24 @@ public static string RemoveLastItem_MenuItem { } } + /// + /// Looks up a localized string similar to Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false.. + /// + public static string ReplacePrevious_HelpText { + get { + return ResourceManager.GetString("ReplacePrevious_HelpText", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Optional. Package version used in conjunction with the replace argument to remove an older version of the manifest from the Windows Package Manager repo.. + /// + public static string ReplaceVersion_HelpText { + get { + return ResourceManager.GetString("ReplaceVersion_HelpText", resourceCulture); + } + } + /// /// Looks up a localized string similar to Repository "{0}/{1}" not found. Please verify the Windows Package Manager repository owner and name in your settings file.. /// @@ -2706,6 +2724,15 @@ public static string Version_HelpText { } } + /// + /// Looks up a localized string similar to Version {0} does not exist for {1} in the Windows Package Manager repository.. + /// + public static string VersionDoesNotExist_Error { + get { + return ResourceManager.GetString("VersionDoesNotExist_Error", resourceCulture); + } + } + /// /// Looks up a localized string similar to Display the version manifest of the package.. /// diff --git a/src/WingetCreateCLI/Properties/Resources.resx b/src/WingetCreateCLI/Properties/Resources.resx index 702f2c58..81b14dd1 100644 --- a/src/WingetCreateCLI/Properties/Resources.resx +++ b/src/WingetCreateCLI/Properties/Resources.resx @@ -1099,4 +1099,17 @@ Version Manifest: + + Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false. + + + Version {0} does not exist for {1} in the Windows Package Manager repository. + {0} - will be replaced with the package version + +{1} - will be replaced with the package ID + + + + Optional. Package version used in conjunction with the replace argument to remove an older version of the manifest from the Windows Package Manager repo. + \ No newline at end of file diff --git a/src/WingetCreateCore/Common/GitHub.cs b/src/WingetCreateCore/Common/GitHub.cs index 87d68d3e..88246766 100644 --- a/src/WingetCreateCore/Common/GitHub.cs +++ b/src/WingetCreateCore/Common/GitHub.cs @@ -97,32 +97,14 @@ public async Task> GetAppVersions() /// Manifest as a string. public async Task> GetManifestContentAsync(string packageId, string version = null) { - string appPath = Utils.GetAppManifestDirPath(packageId, string.Empty, '/'); - var contents = await this.github.Repository.Content.GetAllContents(this.wingetRepoOwner, this.wingetRepo, appPath); - string versionDirectory; + string versionDirectoryPath = await this.GetVersionDirectoryPath(packageId, version); - if (string.IsNullOrEmpty(version)) - { - versionDirectory = contents - .Where(c => c.Type == ContentType.Dir) - .OrderByDescending(c => c.Name, new VersionComparer()) - .Select(c => c.Path) - .FirstOrDefault(); - } - else - { - versionDirectory = contents - .Where(c => c.Type == ContentType.Dir && c.Name.EqualsIC(version)) - .Select(c => c.Path) - .FirstOrDefault(); - } - - if (string.IsNullOrEmpty(versionDirectory)) + if (string.IsNullOrEmpty(versionDirectoryPath)) { throw new NotFoundException(nameof(version), System.Net.HttpStatusCode.NotFound); } - var packageContents = (await this.github.Repository.Content.GetAllContents(this.wingetRepoOwner, this.wingetRepo, versionDirectory)) + var packageContents = (await this.github.Repository.Content.GetAllContents(this.wingetRepoOwner, this.wingetRepo, versionDirectoryPath)) .Where(c => c.Type != ContentType.Dir && Path.GetExtension(c.Name).EqualsIC(".yaml")); // If all contents of version directory are directories themselves, user must've provided an invalid packageId. @@ -148,8 +130,10 @@ public async Task> GetManifestContentAsync(string packageId, string /// Wrapper object for manifest object models to be submitted in the PR. /// Bool indicating whether or not to submit the PR via a fork. /// Optional parameter specifying the title for the pull request. + /// Optional parameter specifying whether the new submission should replace an existing manifest. + /// Optional parameter specifying the version of the manifest to be replaced. /// Pull request object. - public Task SubmitPullRequestAsync(Manifests manifests, bool submitToFork, string prTitle = null) + public Task SubmitPullRequestAsync(Manifests manifests, bool submitToFork, string prTitle = null, bool shouldReplace = false, string replaceVersion = null) { Dictionary contents = new Dictionary(); string id; @@ -173,7 +157,7 @@ public Task SubmitPullRequestAsync(Manifests manifests, bool submit contents.Add($"{id}.locale.{manifests.DefaultLocaleManifest.PackageLocale}", manifests.DefaultLocaleManifest.ToYaml()); } - return this.SubmitPRAsync(id, version, contents, submitToFork, prTitle); + return this.SubmitPRAsync(id, version, contents, submitToFork, prTitle, shouldReplace, replaceVersion); } /// @@ -293,7 +277,7 @@ private async Task FindPackageIdRecursive(string[] packageId, string pat return null; } - private async Task SubmitPRAsync(string packageId, string version, Dictionary contents, bool submitToFork, string prTitle = null) + private async Task SubmitPRAsync(string packageId, string version, Dictionary contents, bool submitToFork, string prTitle = null, bool shouldReplace = false, string replaceVersion = null) { bool createdRepo = false; Repository repo; @@ -374,6 +358,12 @@ await retryPolicy.ExecuteAsync(async () => await this.github.Git.Reference.Update(repo.Id, newBranchNameHeads, new ReferenceUpdate(commit.Sha)); + // Remove a previous manifest + if (shouldReplace) + { + await this.DeletePackageManifest(repo.Id, packageId, replaceVersion, newBranchName); + } + // Get latest description template from repo string description = await this.GetFileContentsAsync(PRDescriptionRepoPath); @@ -406,6 +396,53 @@ await retryPolicy.ExecuteAsync(async () => } } + private async Task GetVersionDirectoryPath(string packageId, string version = null) + { + string appPath = Utils.GetAppManifestDirPath(packageId, string.Empty, '/'); + var contents = await this.github.Repository.Content.GetAllContents(this.wingetRepoOwner, this.wingetRepo, appPath); + string directory; + + if (string.IsNullOrEmpty(version)) + { + // Get the latest version directory + directory = contents + .Where(c => c.Type == ContentType.Dir) + .OrderByDescending(c => c.Name, new VersionComparer()) + .Select(c => c.Path) + .FirstOrDefault(); + } + else + { + // Get the specified version directory + directory = contents + .Where(c => c.Type == ContentType.Dir && c.Name.EqualsIC(version)) + .Select(c => c.Path) + .FirstOrDefault(); + } + + return directory; + } + + private async Task DeletePackageManifest(long forkRepoId, string packageId, string version, string branchName) + { + string versionDirectoryPath = await this.GetVersionDirectoryPath(packageId, version); + + if (string.IsNullOrEmpty(versionDirectoryPath)) + { + throw new NotFoundException(nameof(version), System.Net.HttpStatusCode.NotFound); + } + + // Get all files in the version directory + var versionDirectoryContents = await this.github.Repository.Content.GetAllContents(forkRepoId, versionDirectoryPath); + + // Delete files from the new branch in the forked repository + foreach (var file in versionDirectoryContents) + { + var fileContent = await this.github.Repository.Content.GetAllContentsByRef(forkRepoId, file.Path, branchName); + await this.github.Repository.Content.DeleteFile(forkRepoId, file.Path, new DeleteFileRequest($"Delete {file.Path}", fileContent[0].Sha, branchName)); + } + } + /// /// Checks if the provided forked repository is behind on upstream commits and updates the default branch with the fetched commits. Update can only be a fast-forward update. /// From 457bc54c83eea887ed7310bc633e0374d1bcb7e3 Mon Sep 17 00:00:00 2001 From: Muhammad Danish Date: Tue, 12 Sep 2023 20:05:38 +0500 Subject: [PATCH 2/3] address PR feedback --- doc/update.md | 2 +- src/WingetCreateCLI/Commands/UpdateCommand.cs | 7 ++++--- .../Properties/Resources.Designer.cs | 13 +++++++++++-- src/WingetCreateCLI/Properties/Resources.resx | 7 +++++-- 4 files changed, 21 insertions(+), 8 deletions(-) diff --git a/doc/update.md b/doc/update.md index 2670a332..3c4443a4 100644 --- a/doc/update.md +++ b/doc/update.md @@ -54,7 +54,7 @@ The following arguments are available: | **-v, --version** | Version to be used when updating the package version field. | **-o, --out** | The output directory where the newly created manifests will be saved locally | **-s, --submit** | Boolean value for submitting to the Windows Package Manager repo. If true, updated manifest will be submitted directly using the provided GitHub Token -| **-r, --replace** | Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false. +| **-r, --replace** | Boolean value for replacing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be replaced. Default is false. | **-p, --prtitle** | The title of the pull request submitted to GitHub. | **-t, --token** | GitHub personal access token used for direct submission to the Windows Package Manager repo. If no token is provided, tool will prompt for GitHub login credentials. | **-?, --help** | Gets additional help on this command. | diff --git a/src/WingetCreateCLI/Commands/UpdateCommand.cs b/src/WingetCreateCLI/Commands/UpdateCommand.cs index d1aac9ad..55854bfd 100644 --- a/src/WingetCreateCLI/Commands/UpdateCommand.cs +++ b/src/WingetCreateCLI/Commands/UpdateCommand.cs @@ -55,7 +55,7 @@ public static IEnumerable Examples public string Id { get; set; } /// - /// Gets or sets the previous version to remove from the Windows Package Manager repository. + /// Gets or sets the previous version to replace from the Windows Package Manager repository. /// [Value(1, MetaName = "ReplaceVersion", Required = false, HelpText = "ReplaceVersion_HelpText", ResourceType = typeof(Resources))] public string ReplaceVersion { get; set; } @@ -85,7 +85,7 @@ public static IEnumerable Examples public bool SubmitToGitHub { get; set; } /// - /// Gets or sets a value indicating whether or not to remove a previous version of the manifest with the update. + /// Gets or sets a value indicating whether or not to replace a previous version of the manifest with the update. /// [Option('r', "replace", Required = false, HelpText = "ReplacePrevious_HelpText", ResourceType = typeof(Resources))] public bool Replace { get; set; } @@ -161,7 +161,8 @@ public override async Task Execute() // If update version is same as replace version, it's a regular update. if (this.Version == this.ReplaceVersion) { - this.Replace = false; + Logger.ErrorLocalized(nameof(Resources.ReplaceVersionEqualsUpdateVersion_ErrorMessage)); + return false; } // Check if the replace version exists in the repository. diff --git a/src/WingetCreateCLI/Properties/Resources.Designer.cs b/src/WingetCreateCLI/Properties/Resources.Designer.cs index 0cc840d0..3830e9ca 100644 --- a/src/WingetCreateCLI/Properties/Resources.Designer.cs +++ b/src/WingetCreateCLI/Properties/Resources.Designer.cs @@ -2158,7 +2158,7 @@ public static string RemoveLastItem_MenuItem { } /// - /// Looks up a localized string similar to Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false.. + /// Looks up a localized string similar to Boolean value for replacing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be replaced. Default is false.. /// public static string ReplacePrevious_HelpText { get { @@ -2167,7 +2167,7 @@ public static string ReplacePrevious_HelpText { } /// - /// Looks up a localized string similar to Optional. Package version used in conjunction with the replace argument to remove an older version of the manifest from the Windows Package Manager repo.. + /// Looks up a localized string similar to Optional. Package version used in conjunction with the replace argument to replace an older version of the manifest from the Windows Package Manager repo.. /// public static string ReplaceVersion_HelpText { get { @@ -2175,6 +2175,15 @@ public static string ReplaceVersion_HelpText { } } + /// + /// Looks up a localized string similar to The replace version cannot be equal to the update version.. + /// + public static string ReplaceVersionEqualsUpdateVersion_ErrorMessage { + get { + return ResourceManager.GetString("ReplaceVersionEqualsUpdateVersion_ErrorMessage", resourceCulture); + } + } + /// /// Looks up a localized string similar to Repository "{0}/{1}" not found. Please verify the Windows Package Manager repository owner and name in your settings file.. /// diff --git a/src/WingetCreateCLI/Properties/Resources.resx b/src/WingetCreateCLI/Properties/Resources.resx index 81b14dd1..b6ff8f63 100644 --- a/src/WingetCreateCLI/Properties/Resources.resx +++ b/src/WingetCreateCLI/Properties/Resources.resx @@ -1100,7 +1100,7 @@ Version Manifest: - Boolean value for removing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be removed. Default is false. + Boolean value for replacing an existing manifest from the Windows Package Manager repo. Optionally provide a version or else the latest version will be replaced. Default is false. Version {0} does not exist for {1} in the Windows Package Manager repository. @@ -1110,6 +1110,9 @@ - Optional. Package version used in conjunction with the replace argument to remove an older version of the manifest from the Windows Package Manager repo. + Optional. Package version used in conjunction with the replace argument to replace an older version of the manifest from the Windows Package Manager repo. + + + The replace version cannot be equal to the update version. \ No newline at end of file From 8c7cbe4652c9a2393d3a929b349f6635a51a3b00 Mon Sep 17 00:00:00 2001 From: Muhammad Danish Date: Tue, 12 Sep 2023 20:56:25 +0500 Subject: [PATCH 3/3] fix github manual merge --- src/WingetCreateCLI/Properties/Resources.resx | 1 + 1 file changed, 1 insertion(+) diff --git a/src/WingetCreateCLI/Properties/Resources.resx b/src/WingetCreateCLI/Properties/Resources.resx index e193a723..b923f1fc 100644 --- a/src/WingetCreateCLI/Properties/Resources.resx +++ b/src/WingetCreateCLI/Properties/Resources.resx @@ -1114,6 +1114,7 @@ The replace version cannot be equal to the update version. + Try using the architecture and/or scope overrides to uniquely match new URLs to existing installer nodes in the manifest.