diff --git a/README.md b/README.md index 6dd712c8..7fda5743 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,26 @@ # CCVTAC -CCVTAC (CodeConscious Video-to-Audio Converter) is a small .NET-powered CLI tool that acts as a wrapper around [yt-dlp](https://github.com/yt-dlp/yt-dlp) to enable easier download and extractions of audio from YouTube videos, playlists, and channels, plus do some automatic post-processing (tagging, renaming, and moving). +CCVTAC (CodeConscious Video-to-Audio Converter) is a small .NET-powered CLI tool written in F# that acts as a wrapper around [yt-dlp](https://github.com/yt-dlp/yt-dlp) to enable easier download and extractions of audio from YouTube videos, playlists, and channels, plus do some automatic post-processing (tagging, renaming, and moving). -While I maintain it for my own use, feel free to use it yourself! However, please note it's geared to my own personal use cases and that no warranties or guarantees are provided. +Feel free to use it yourself, but please note that it's geared to my personal use case and that no warranties or guarantees are provided. [![Build and test](https://github.com/codeconscious/ccvtac/actions/workflows/build-test.yml/badge.svg)](https://github.com/codeconscious/ccvtac/actions/workflows/build-test.yml) ## Features - Converts YouTube videos, playlists, and channels to local audio files (via [yt-dlp](https://github.com/yt-dlp/yt-dlp)) -- Writes ID3 tags to files where possible using available or regex-detected metadata -- Adds video metadata (channel name and URL, video URL, etc.) to files' Comment tags -- Auto-renames files via custom regex patterns (to remove media IDs, etc.) +- Writes ID3 tags to files where possible using available metadata via regex-based detection +- Logs video metadata (channel name and URL, video URL, etc.) to files' Comments tags +- Auto-renames files via custom regex patterns (to remove video IDs, etc.) - Optionally writes video thumbnails to files as artwork (if [mogrify](https://imagemagick.org/script/mogrify.php) is installed) -- Customized behavior via a user settings file — e.g., chapter splitting, image embedding, directories +- Customizable behavior via a settings file - Saves entered URLs to a local history file ## Prerequisites - [.NET 10 runtime](https://dotnet.microsoft.com/en-us/download/dotnet/10.0) - [yt-dlp](https://github.com/yt-dlp/yt-dlp) -- [ffmpeg](https://ffmpeg.org/) (for yt-dlp artwork extraction) +- [ffmpeg](https://ffmpeg.org/) (for yt-dlp artwork extraction and conversion) - Optional: [mogrify](https://imagemagick.org/script/mogrify.php) (for auto-trimming album art) ## Screenshots @@ -37,18 +37,21 @@ While I maintain it for my own use, feel free to use it yourself! However, pleas ### Settings -A valid settings file is mandatory to use this application. +A valid JSON settings file is mandatory to use this application. By default, the application will look for a file named `settings.json` in its directory. However, you can manually specify an existing file path using the `-s` option, such as `dotnet run -- -s `. -If your `settings.json` file does not exist, one will be created in the application directory with default settings. At minimum, you will need to enter (1) an existing directory for temporary working files, (2) an existing directory to which the final audio files should be moved, and (3) a path to your history file. The other settings have sensible defaults. Some settings require familiarity with regular expressions (regex). +> [!TIP] +> The `--` is necessary to indicate that the command and arguments are for this program and not for `dotnet`. -#### Starter file with comments - -You can copy and paste the sample settings file below to a JSON file named `settings.json` to get started. You will, in particular, need to update the three directories at the top. You can leave the commented lines as-is, as they will be ignored. +If your `settings.json` file does not exist, a default one will be created. At minimum, you will need to enter (1) an existing directory for temporary working files, (2) an existing directory to which the final audio files should be moved, and (3) a path to your history file. The other settings have sensible defaults. Some settings require familiarity with regular expressions (regex).
- Click here to expand! + Click to see a sample settings file + +The sample below contains explanations and some example values as well. + +**Important:** When entering regular expressions, you must double-up backslashes. For example, to match a whitespace character, use `\\s` instead of `\s`. ```js { @@ -65,13 +68,10 @@ You can copy and paste the sample settings file below to a JSON file named `sett // Count of entries to show for `history` command. "historyDisplayCount": 20, - // The directory to which the log file should be saved. - "logDirectory": "/Users/me/Downloads", - // The audio formats (codec) audio should be extracted to. // Options: best, aac, alac, flac, m4a, mp3, opus, vorbis, wav. // Not all options are available for all videos. - "audioFormats": ["best"], + "audioFormats": ["m4a", "best"], // The audio quality to use, with 10 being the lowest and 0 being the highest. "audioQuality": 0, @@ -105,7 +105,14 @@ You can copy and paste the sample settings file below to a JSON file named `sett // The full command you use to update your local yt-dlp installation. // This is a sample entry. - "downloaderUpdateCommand": "pip install --upgrade yt-dlp", + "downloaderUpdateCommand": "pip install --upgrade yt-dlp", + + // Arbitrary yt-dlp options to be included in all yt-dlp commands. + // Use with caution, as some options could disrupt operation of this program. + // Intended to be used only when necessary to resolve download issues. + // For example, see https://github.com/yt-dlp/yt-dlp/wiki/EJS, + // upon which this sample is based. + "downloaderAdditionalOptions": "--remote-components ejs:github", // Channel names for which the video thumbnail should // never be embedded in the audio file. @@ -125,7 +132,7 @@ You can copy and paste the sample settings file below to a JSON file named `sett // These require familiarity with regular expressions (regex). "tagDetectionPatterns": { - // Currently supports 5 tags: this one (Title) and its siblings listed below. + // Currently supports 5 tags: this one (title) and its siblings listed below. "title": [ { // A regex pattern for searching in the video metadata field specified below. @@ -139,7 +146,7 @@ You can copy and paste the sample settings file below to a JSON file named `sett // Which video metadata field should be searched, `title` or `description`? "searchField": "description", - // An arbitrary summary to the rule. If quiet mode is off, this name will appear + // An arbitrary summary of the rule. If quiet mode is off, this name will appear // in the output when this pattern is matched. "summary": "Topic style" } @@ -184,7 +191,7 @@ You can copy and paste the sample settings file below to a JSON file named `sett ### Using the application -Once your settings file is ready, run the application with `dotnet run` within the `CCVTAC.Console` directory. Alternatively, pass `-h` or `--help` for instructions (e.g., `dotnet run -- --help`). +Once your settings file is ready, run the application with `dotnet run` within the `CCVTAC.Console` directory, optionally passing the path to your settings file using `-s`. Alternatively, pass `-h` or `--help` for instructions (e.g., `dotnet run -- --help`). When the application is running, enter at least one YouTube media URL (video, playlist, or channel) or command at the prompt and press Enter. No spaces between items are necessary. @@ -192,14 +199,26 @@ List of commands: - `\help` to see this list of commands - `\quit` or `\q` to quit - `\history` to see the URLs you most recently entered -- `\update-downloader` or `\update-dl` to update yt-dlp using the command in your settings (If you start experiencing constant download errors, try this command) +- `\update-downloader` or `\update-dl` to update yt-dlp using the command in your settings (Note: If you start experiencing constant download errors, try this command to ensure you have the latest version) - Modify the current session only (without updating the settings file): - `\split` toggles chapter splitting - `\images` toggles image embedding - `\quiet` toggles quiet mode - - `\format-` followed by a supported audio format (e.g., `\format-m4a`) changes the format + - `\format-` followed by a supported audio format (e.g., `\format-m4a`) changes the audio format - `\quality-` followed by a supported audio quality (e.g., `\quality-0`) changes the audio quality +Enter `\commands` in the application to see this summary. + ## Reporting issues -If you run into any issues, feel free to create an issue on GitHub. Please provide as much information as possible (i.e., entered URLs, system information, yt-dlp version, etc.). +If you run into any issues, feel free to create an issue on GitHub. Please provide as much information as possible (i.e., entered URLs or comments, system information, yt-dlp version, etc.) and I'll try to take a look. + +However, do keep in mind that this is ultimately a hobby project for myself, so I cannot guarantee every issue will be fixed. + +## History + +The first incarnation of this application was written in C#. However, after picking up [F#](https://fsharp.org/) out of curiosity about it and functional programming (FP) in 2024 and successfully using it to create other tools (mainly [Audio Tag Tools](https://github.com/codeconscious/audio-tag-tools/)) in an FP style, I become curious about F#'s OOP capabilities as well. + +As an experiment, I rewrote this application in OOP-style F#, using LLMs solely for the rough initial conversion (which greatly reduced the overall time and labor necessary at the cost of requiring a *lot* of manual cleanup). Ultimately, I was surprised how much I preferred the F# code over the C#, so I decided to keep this tool in F#. + +Due to this background, the code is not particularly idiomatic F#, but it is perfectly viable in its current blended-style form. That said, I'll probably tweak it over time to gradually to introduce more FP, mainly for practice. diff --git a/src/CCVTAC.Console.Tests/CCVTAC.Console.Tests.csproj b/src/CCVTAC.Console.Tests/CCVTAC.Console.Tests.csproj deleted file mode 100644 index 3ef4d8b6..00000000 --- a/src/CCVTAC.Console.Tests/CCVTAC.Console.Tests.csproj +++ /dev/null @@ -1,31 +0,0 @@ - - - - net10.0 - disable - enable - false - true - true - - - - - - - runtime; build; native; contentfiles; analyzers; buildtransitive - all - - - runtime; build; native; contentfiles; analyzers; buildtransitive - all - - - - - - - - - - diff --git a/src/CCVTAC.Console.Tests/ExtensionMethodTests.cs b/src/CCVTAC.Console.Tests/ExtensionMethodTests.cs deleted file mode 100644 index ef955dbf..00000000 --- a/src/CCVTAC.Console.Tests/ExtensionMethodTests.cs +++ /dev/null @@ -1,214 +0,0 @@ -using System; -using System.Collections.Generic; -using System.IO; - -namespace CCVTAC.Console.Tests; - -public sealed class ExtensionMethodTests -{ - public sealed class ReplaceInvalidPathCharsTests - { - private const string ValidBaseFileName = "filename123あいうえお漢字!@#$%^()_+ "; - private const char DefaultReplaceWithChar = '_'; - - private static readonly char[] PathInvalidChars = [ - Path.PathSeparator, - Path.DirectorySeparatorChar, - Path.AltDirectorySeparatorChar, - Path.VolumeSeparatorChar - ]; - - [Fact] - public void ReplaceInvalidPathChars_StringContainsInvalidPathChars_Fixes() - { - string badFileName = ValidBaseFileName + new string(PathInvalidChars); - string fixedPathName = badFileName.ReplaceInvalidPathChars(DefaultReplaceWithChar); - string expected = ValidBaseFileName + new string(DefaultReplaceWithChar, PathInvalidChars.Length); - Assert.Equal(expected, fixedPathName); - } - - [Fact] - public void ReplaceInvalidPathChars_StringContainsNoInvalidPathChars_DoesNotChange() - { - string result = ValidBaseFileName.ReplaceInvalidPathChars(DefaultReplaceWithChar); - Assert.Equal(ValidBaseFileName, result); - } - - [Fact] - public void ReplaceInvalidPathCharsIncludingCustom_StringContainsInvalidPathChars_Fixes() - { - char[] customInvalidChars = ['&', '＆']; - var badFileName = ValidBaseFileName + new string(customInvalidChars); - var fixedPathName = badFileName.ReplaceInvalidPathChars(DefaultReplaceWithChar, customInvalidChars); - var expectedName = ValidBaseFileName + new string(DefaultReplaceWithChar, customInvalidChars.Length); - Assert.Equal(expectedName, fixedPathName); - } - - [Fact] - public void ReplaceInvalidPathCharsIncludingCustom_StringContainsNoInvalidPathChars_DoesNotChange() - { - char[] customInvalidChars = ['&', '＆']; - const string goodFileName = ValidBaseFileName + "++"; - string result = goodFileName.ReplaceInvalidPathChars(DefaultReplaceWithChar, customInvalidChars); - Assert.Equal(goodFileName, result); - } - - [Fact] - public void ReplaceInvalidPathChars_InvalidReplaceChar_ThrowsException() - { - const char knownInvalidChar = '/'; - Assert.Throws(() => ValidBaseFileName.ReplaceInvalidPathChars(knownInvalidChar)); - } - } - - public sealed class NoneTests - { - [Fact] - public void None_WithEmptyCollection_ReturnsTrue() - { - List numbers = []; - Assert.True(numbers.None()); - } - - [Fact] - public void None_WithPopulatedCollectionAndMatchingPredicate_ReturnsFalse() - { - List numbers = [2, 4, 6]; - static bool IsEven(byte s) => s % 2 == 0; - Assert.False(numbers.None(IsEven)); - } - - [Fact] - public void None_WithPopulatedCollectionAndNonMatchingPredicate_ReturnsTrue() - { - List numbers = [1, 3, 5]; - static bool IsEven(byte s) => s % 2 == 0; - Assert.True(numbers.None(IsEven)); - } - - [Fact] - public void None_WithEmptyCollectionAndPredicate_ReturnsTrue() - { - List numbers = []; - static bool IsEven(byte s) => s % 2 == 0; - Assert.True(numbers.None(IsEven)); - } - } - - public sealed class HasTextTests - { - [Fact] - public void HasText_Null_ReturnsFalse() - { - string? noText = null; - Assert.False(noText.HasText(false)); - Assert.False(noText.HasText(true)); - } - - [Fact] - public void HasText_EmptyString_ReturnsFalse() - { - var emptyText = string.Empty; - Assert.False(emptyText.HasText(false)); - Assert.False(emptyText.HasText(true)); - } - - [Fact] - public void HasText_SingleByteWhiteSpaceOnlyWhenDisallowed_ReturnsFalse() - { - const string whiteSpace = " "; - Assert.False(whiteSpace.HasText(false)); - } - - [Fact] - public void HasText_SingleByteWhiteSpaceOnlyWhenAllowed_ReturnsTrue() - { - const string whiteSpace = " "; - Assert.True(whiteSpace.HasText(true)); - } - - [Fact] - public void HasText_DoubleByteWhiteSpaceOnlyWhenDisallowed_ReturnsFalse() - { - const string whiteSpace = "　　　"; - Assert.False(whiteSpace.HasText(false)); - } - - [Fact] - public void HasText_DoubleByteWhiteSpaceOnlyWhenAllowed_ReturnsTrue() - { - const string whiteSpace = "　　　"; - Assert.True(whiteSpace.HasText(true)); - } - - [Fact] - public void HasText_WithText_ReturnsTrue() - { - const string text = "こんにちは！"; - Assert.True(text.HasText(false)); - Assert.True(text.HasText(true)); - } - } - - public sealed class CaseInsensitiveContainsTests - { - private static readonly List CelestialBodies = - ["Moon", "Mercury", "Mars", "Jupiter", "Venus"]; - - [Fact] - public void CaseInsensitiveContains_EmptyCollection_ReturnsFalse() - { - List collection = []; - var actual = collection.CaseInsensitiveContains("text"); - Assert.False(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchAllCapsInPopulatedCollection_ReturnsTrue() - { - List collection = CelestialBodies; - var actual = collection.CaseInsensitiveContains("MOON"); - Assert.True(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchAllLowercaseInPopulatedCollection_ReturnsTrue() - { - List collection = CelestialBodies; - var actual = collection.CaseInsensitiveContains("moon"); - Assert.True(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchExactInPopulatedCollection_ReturnsTrue() - { - List collection = CelestialBodies; - var actual = collection.CaseInsensitiveContains("Moon"); - Assert.True(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchPartialInPopulatedCollection_ReturnsFalse() - { - List collection = CelestialBodies; - var actual = collection.CaseInsensitiveContains("Mo"); - Assert.False(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchExactButDoubleWidthInPopulatedCollection_ReturnsFalse() - { - List collection = CelestialBodies; - var actual = collection.CaseInsensitiveContains("Ｍｏｏｎ"); - Assert.False(actual); - } - - [Fact] - public void CaseInsensitiveContains_SearchTextInEmptyCollection_ReturnsFalse() - { - List collection = []; - var actual = collection.CaseInsensitiveContains("text"); - Assert.False(actual); - } - } -} diff --git a/src/CCVTAC.Console.Tests/InputHelperTests.cs b/src/CCVTAC.Console.Tests/InputHelperTests.cs deleted file mode 100644 index 3906faf3..00000000 --- a/src/CCVTAC.Console.Tests/InputHelperTests.cs +++ /dev/null @@ -1,77 +0,0 @@ -using System.Collections.Generic; - -namespace CCVTAC.Console.Tests; - -public sealed class InputHelperTests -{ - [Fact] - public void MultipleUrlsEntered_CorrectlyParsed() - { - const string combinedInput = "https://youtu.be/5OpuZHsPBhQhttps://youtu.be/NT22EGxTuNw"; - List expected = ["https://youtu.be/5OpuZHsPBhQ", "https://youtu.be/NT22EGxTuNw"]; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Equal(expected.Count, actual.Length); - Assert.Equal(expected[0], actual[0]); - Assert.Equal(expected[1], actual[1]); - } - - [Fact] - public void MultipleUrlsEnteredWithSpaces_CorrectlyParsed() - { - const string combinedInput = " https://youtu.be/5OpuZHsPBhQ https://youtu.be/NT22EGxTuNw "; - List expected = ["https://youtu.be/5OpuZHsPBhQ", "https://youtu.be/NT22EGxTuNw"]; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Equal(expected.Count, actual.Length); - Assert.Equal(expected[0], actual[0]); - Assert.Equal(expected[1], actual[1]); - } - - [Fact] - public void MultipleDuplicateUrlsEntered_CorrectlyParsed() - { - const string combinedInput = "https://youtu.be/5OpuZHsPBhQhttps://youtu.be/NT22EGxTuNwhttps://youtu.be/5OpuZHsPBhQ"; - List expected = ["https://youtu.be/5OpuZHsPBhQ", "https://youtu.be/NT22EGxTuNw"]; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Equal(expected.Count, actual.Length); - Assert.Equal(expected[0], actual[0]); - Assert.Equal(expected[1], actual[1]); - } - - [Fact] - public void SingleCommandEntered_CorrectlyParsed() - { - const string combinedInput = "\\images"; - List expected = ["\\images"]; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Equal(expected.Count, actual.Length); - Assert.Equal(expected[0], actual[0]); - } - - [Fact] - public void MultipleDuplicateCommandsAndUrlsEntered_CorrectlyParsed() - { - const string combinedInput = @"\imageshttps://youtu.be/5OpuZHsPBhQ https://youtu.be/NT22EGxTuNw\images https://youtu.be/5OpuZHsPBhQ"; - List expected = ["\\images", "https://youtu.be/5OpuZHsPBhQ", "https://youtu.be/NT22EGxTuNw"]; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Equal(expected.Count, actual.Length); - Assert.Equal(expected[0], actual[0]); - Assert.Equal(expected[1], actual[1]); - Assert.Equal(expected[2], actual[2]); - } - - [Fact] - public void EmptyInput_CorrectlyParsed() - { - var combinedInput = string.Empty; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Empty(actual); - } - - [Fact] - public void InvalidInput_CorrectlyParsed() - { - const string combinedInput = "invalid"; - var actual = InputHelper.SplitInput(combinedInput); - Assert.Empty(actual); - } -} diff --git a/src/CCVTAC.Console.Tests/Usings.cs b/src/CCVTAC.Console.Tests/Usings.cs deleted file mode 100644 index 9df1d421..00000000 --- a/src/CCVTAC.Console.Tests/Usings.cs +++ /dev/null @@ -1 +0,0 @@ -global using Xunit; diff --git a/src/CCVTAC.Console/CCVTAC.Console.csproj b/src/CCVTAC.Console/CCVTAC.Console.csproj deleted file mode 100644 index 691ccc88..00000000 --- a/src/CCVTAC.Console/CCVTAC.Console.csproj +++ /dev/null @@ -1,18 +0,0 @@ - - - Exe - net10.0 - disable - enable - true - - - - - - - - - - - diff --git a/src/CCVTAC.Console/Commands.cs b/src/CCVTAC.Console/Commands.cs deleted file mode 100644 index 2df524c8..00000000 --- a/src/CCVTAC.Console/Commands.cs +++ /dev/null @@ -1,63 +0,0 @@ -namespace CCVTAC.Console; - -internal static class Commands -{ - internal const char Prefix = '\\'; - - internal static string[] QuitCommands { get; } = - [MakeCommand("quit"), MakeCommand("q"), MakeCommand("exit")]; - - internal static string HelpCommand { get; } = MakeCommand("help"); - - internal static string[] SettingsSummary { get; } = [MakeCommand("settings")]; - - internal static string[] History { get; } = [MakeCommand("history")]; - internal static string[] UpdateDownloader { get; } = [MakeCommand("update-downloader"), MakeCommand("update-dl")]; - - internal static string[] SplitChapterToggles { get; } = - [MakeCommand("split"), MakeCommand("toggle-split")]; - - internal static string[] EmbedImagesToggles { get; } = - [MakeCommand("images"), MakeCommand("toggle-images")]; - - internal static string[] QuietModeToggles { get; } = - [MakeCommand("quiet"), MakeCommand("toggle-quiet")]; - - internal static string UpdateAudioFormatPrefix { get; } = MakeCommand("format-"); - - internal static string UpdateAudioQualityPrefix { get; } = MakeCommand("quality-"); - - internal static Dictionary Summary { get; } = - new() - { - { string.Join(" or ", History), "See the most recently entered URLs" }, - { string.Join(" or ", SplitChapterToggles), "Toggles chapter splitting for the current session only" }, - { string.Join(" or ", EmbedImagesToggles), "Toggles image embedding for the current session only" }, - { string.Join(" or ", QuietModeToggles), "Toggles quiet mode for the current session only" }, - { string.Join(" or ", UpdateDownloader), "Updates the downloader using the command specified in the settings" }, - { - UpdateAudioFormatPrefix, - $"Followed by a supported audio format (e.g., {UpdateAudioFormatPrefix}m4a), changes the audio format for the current session only" - }, - { - UpdateAudioQualityPrefix, - $"Followed by a supported audio quality (e.g., {UpdateAudioQualityPrefix}0), changes the audio quality for the current session only" - }, - { string.Join(" or ", QuitCommands), "Quit the application" }, - { string.Join(" or ", HelpCommand), "See this help message" }, - }; - - private static string MakeCommand(string text) - { - if (string.IsNullOrWhiteSpace(text)) - throw new ArgumentException("The text cannot be null or white space.", nameof(text)); - - if (text.Contains(' ')) - throw new ArgumentException( - "The text should not contain any white space.", - nameof(text) - ); - - return $"{Prefix}{text}"; - } -} diff --git a/src/CCVTAC.Console/Comparers.cs b/src/CCVTAC.Console/Comparers.cs deleted file mode 100644 index 2960209d..00000000 --- a/src/CCVTAC.Console/Comparers.cs +++ /dev/null @@ -1,22 +0,0 @@ -namespace CCVTAC.Console; - -internal class Comparers -{ - public sealed class CaseInsensitiveStringComparer : IEqualityComparer - { - public bool Equals(string? x, string? y) - { - return (x, y) switch - { - (null, null) => true, - (null, _) or (_, null) => false, - _ => string.Equals(x.Trim(), y.Trim(), StringComparison.OrdinalIgnoreCase), - }; - } - - public int GetHashCode(string obj) - { - return obj.ToLower().GetHashCode(); - } - } -} diff --git a/src/CCVTAC.Console/Downloading/Downloader.cs b/src/CCVTAC.Console/Downloading/Downloader.cs deleted file mode 100644 index 2972e284..00000000 --- a/src/CCVTAC.Console/Downloading/Downloader.cs +++ /dev/null @@ -1,193 +0,0 @@ -using CCVTAC.Console.ExternalTools; -using MediaTypeWithUrls = CCVTAC.FSharp.Downloading.MediaType; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.Downloading; - -internal static class Downloader -{ - private static readonly string ProgramName = "yt-dlp"; - - private record Urls(string Primary, string? Supplementary); - - internal static Result WrapUrlInMediaType(string url) - { - var result = FSharp.Downloading.MediaTypeWithIds(url); - - return result.IsOk ? Result.Ok(result.ResultValue) : Result.Fail(result.ErrorValue); - } - - /// - /// Completes the actual download process. - /// - /// A `Result` that, if successful, contains the name of the successfully downloaded format. - internal static Result Run( - MediaTypeWithUrls mediaType, - UserSettings settings, - Printer printer - ) - { - if (mediaType is { IsVideo: false, IsPlaylistVideo: false }) - { - printer.Info("Please wait for multiple videos to be downloaded..."); - } - - var rawUrls = FSharp.Downloading.ExtractDownloadUrls(mediaType); - var urls = new Urls(rawUrls[0], rawUrls.Length == 2 ? rawUrls[1] : null); - - var downloadResult = new Result<(int, string)>(); - string? successfulFormat = null; - - foreach (string format in settings.AudioFormats) - { - string args = GenerateDownloadArgs(format, settings, mediaType, urls.Primary); - var commandWithArgs = $"{ProgramName} {args}"; - var downloadSettings = new ToolSettings(commandWithArgs, settings.WorkingDirectory!); - - downloadResult = Runner.Run(downloadSettings, otherSuccessExitCodes: [1], printer); - - if (downloadResult.IsSuccess) - { - successfulFormat = format; - - var (exitCode, warnings) = downloadResult.Value; - if (exitCode != 0) - { - printer.Warning("Downloading completed with minor issues."); - if (warnings.HasText()) - { - printer.Warning(warnings); - } - } - - break; - } - - printer.Debug($"Failure downloading \"{format}\" format."); - } - - var errors = downloadResult.Errors.Select(e => e.Message).ToList(); - - int audioFileCount = IoUtilities.Directories.AudioFileCount(settings.WorkingDirectory); - if (audioFileCount == 0) - { - return Result.Fail( - string.Join(Environment.NewLine, errors.Prepend("No audio files were downloaded.")) - ); - } - - if (errors.Count != 0) - { - downloadResult.Errors.ToList().ForEach(e => printer.Error(e.Message)); - printer.Info("Post-processing will still be attempted."); // For any partial downloads - } - else if (urls.Supplementary is not null) - { - string supplementaryArgs = GenerateDownloadArgs( - null, - settings, - null, - urls.Supplementary - ); - - var commandWithArgs = $"{ProgramName} {supplementaryArgs}"; - - var supplementaryDownloadSettings = new ToolSettings(commandWithArgs, settings.WorkingDirectory!); - - var supplementaryDownloadResult = Runner.Run( - supplementaryDownloadSettings, - otherSuccessExitCodes: [1], - printer - ); - - if (supplementaryDownloadResult.IsSuccess) - { - printer.Info("Supplementary download completed OK."); - } - else - { - printer.Error("Supplementary download failed."); - errors.AddRange(supplementaryDownloadResult.Errors.Select(e => e.Message)); - } - } - - return errors.Count > 0 - ? Result.Fail(string.Join(" / ", errors)) - : Result.Ok(successfulFormat); - } - - /// - /// Generate the entire argument string for the download tool. - /// - /// One of the supported audio format codes. - /// - /// A `MediaType` or null (which indicates a metadata-only supplementary download). - /// - /// A string of arguments that can be passed directly to the download tool. - private static string GenerateDownloadArgs( - string? audioFormat, - UserSettings settings, - MediaTypeWithUrls? mediaType, - params string[]? additionalArgs - ) - { - const string writeJson = "--write-info-json"; - const string trimFileNames = "--trim-filenames 250"; - - // yt-dlp warning: "-f best" selects the best pre-merged format which is often not the best option. - // To let yt-dlp download and merge the best available formats, simply do not pass any format selection." - var formatArg = - !audioFormat.HasText() || audioFormat == "best" ? string.Empty : $"-f {audioFormat}"; - - HashSet args = mediaType switch - { - // For metadata-only downloads - null => [$"--flat-playlist {writeJson} {trimFileNames}"], - - // For video(s) with their respective metadata files (JSON and artwork). - _ => - [ - "--extract-audio", - formatArg, - $"--audio-quality {settings.AudioQuality}", - "--write-thumbnail --convert-thumbnails jpg", // For album art - writeJson, // Contains metadata - trimFileNames, - "--retries 2", // Default is 10, which seems like overkill - ], - }; - - // yt-dlp has a `--verbose` option too, but that's too much data. - // It might be worth incorporating it in the future as a third option. - args.Add(settings.QuietMode ? "--quiet --no-warnings" : string.Empty); - - if (mediaType is not null) - { - if (settings.SplitChapters) - { - args.Add("--split-chapters"); - } - - if (mediaType is { IsVideo: false, IsPlaylistVideo: false }) - { - args.Add($"--sleep-interval {settings.SleepSecondsBetweenDownloads}"); - } - - // The numbering of regular playlists should be reversed because the newest items are - // always placed at the top of the list at position #1. Instead, the oldest items - // (at the end of the list) should begin at #1. - if (mediaType.IsStandardPlaylist) - { - // The digits followed by `B` induce trimming to the specified number of bytes. - // Use `s` instead of `B` to trim to a specified number of characters. - // Reference: https://github.com/yt-dlp/yt-dlp/issues/1136#issuecomment-1114252397 - // Also, it's possible this trimming should be applied to `ReleasePlaylist`s too. - args.Add( - """-o "%(uploader).80B - %(playlist).80B - %(playlist_autonumber)s - %(title).150B [%(id)s].%(ext)s" --playlist-reverse""" - ); - } - } - - return string.Join(" ", args.Concat(additionalArgs ?? [])); - } -} diff --git a/src/CCVTAC.Console/Downloading/Updater.cs b/src/CCVTAC.Console/Downloading/Updater.cs deleted file mode 100644 index e66049af..00000000 --- a/src/CCVTAC.Console/Downloading/Updater.cs +++ /dev/null @@ -1,56 +0,0 @@ -using CCVTAC.Console.ExternalTools; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.Downloading; - -internal static class Updater -{ - private record Urls(string Primary, string? Supplementary); - - /// - /// Completes the actual download process. - /// - /// A `Result` that, if successful, contains the name of the successfully downloaded format. - internal static Result Run(UserSettings settings, Printer printer) - { - if (string.IsNullOrWhiteSpace(settings.DownloaderUpdateCommand)) - { - printer.Info("No downloader update command provided, so will skip."); - return Result.Ok(); - } - - var args = new ToolSettings(settings.DownloaderUpdateCommand, settings.WorkingDirectory!); - - var result = Runner.Run(args, otherSuccessExitCodes: [], printer); - - if (result.IsSuccess) - { - var (exitCode, warnings) = result.Value; - - if (exitCode != 0) - { - printer.Warning("Update completed with minor issues."); - - if (warnings.HasText()) - { - printer.Warning(warnings); - } - } - - return Result.Ok(); - } - - printer.Error($"Failure updating..."); - - var errors = result.Errors.Select(e => e.Message).ToList(); - - if (errors.Count != 0) - { - result.Errors.ToList().ForEach(e => printer.Error(e.Message)); - } - - return errors.Count > 0 - ? Result.Fail(string.Join(" / ", errors)) - : Result.Ok(); - } -} diff --git a/src/CCVTAC.Console/ExtensionMethods.cs b/src/CCVTAC.Console/ExtensionMethods.cs deleted file mode 100644 index 1e44e5f4..00000000 --- a/src/CCVTAC.Console/ExtensionMethods.cs +++ /dev/null @@ -1,80 +0,0 @@ -using System.IO; -using System.Text; - -namespace CCVTAC.Console; - -public static class ExtensionMethods -{ - /// - /// Determines whether a string contains any text. - /// - /// - /// Specifies whether whitespace characters should be considered as text. - /// Returns true if the string contains text; otherwise, false. - public static bool HasText(this string? maybeText, bool allowWhiteSpace = false) - { - return allowWhiteSpace - ? !string.IsNullOrEmpty(maybeText) - : !string.IsNullOrWhiteSpace(maybeText); - } - - extension(IEnumerable collection) - { - /// - /// Determines whether a collection is empty. - /// - public bool None() => !collection.Any(); - - /// - /// Determines whether no elements of a sequence satisfy a given condition. - /// - public bool None(Func predicate) => - !collection.Any(predicate); - } - - public static bool CaseInsensitiveContains(this IEnumerable collection, string text) => - collection.Contains(text, new Comparers.CaseInsensitiveStringComparer()); - - /// - extension(string sourceText) - { - /// - /// Returns a new string in which all invalid path characters for the current OS - /// have been replaced by specified replacement character. - /// Throws if the replacement character is an invalid path character. - /// - /// - /// Optional additional characters to consider invalid. - public string ReplaceInvalidPathChars(char replaceWith = '_', - char[]? customInvalidChars = null - ) - { - var invalidChars = Path.GetInvalidFileNameChars() - .Concat(Path.GetInvalidPathChars()) - .Concat( - [ - Path.PathSeparator, - Path.DirectorySeparatorChar, - Path.AltDirectorySeparatorChar, - Path.VolumeSeparatorChar, - ] - ) - .Concat(customInvalidChars ?? Enumerable.Empty()) - .ToFrozenSet(); - - if (invalidChars.Contains(replaceWith)) - throw new ArgumentException( - $"The replacement char ('{replaceWith}') must be a valid path character." - ); - - return invalidChars.Aggregate( - new StringBuilder(sourceText), - (workingText, ch) => workingText.Replace(ch, replaceWith), - workingText => workingText.ToString() - ); - } - - public string TrimTerminalLineBreak() => - sourceText.HasText() ? sourceText.TrimEnd(Environment.NewLine.ToCharArray()) : sourceText; - } -} diff --git a/src/CCVTAC.Console/ExternalTools/ExternalTool.cs b/src/CCVTAC.Console/ExternalTools/ExternalTool.cs deleted file mode 100644 index 8596a3f3..00000000 --- a/src/CCVTAC.Console/ExternalTools/ExternalTool.cs +++ /dev/null @@ -1,73 +0,0 @@ -using System.Diagnostics; - -namespace CCVTAC.Console.ExternalTools; - -internal record ExternalTool -{ - /// - /// The name of the program. This should be the exact text used to call it - /// on the command line, excluding any arguments. - /// - internal string Name { get; } - - /// - /// The URL of the program, from which users should install it if needed. - /// - internal string Url { get; } - - /// - /// A brief summary of the purpose of the program within the context of this program. - /// Should be phrased as a noun (e.g., "image processing" or "audio normalization"). - /// - internal string Purpose { get; } - - /// - /// - /// - /// The name of the program. This should be the exact text used to call it - /// on the command line, excluding any arguments. - /// The URL of the program, from which users should install it if needed. - /// A brief summary of the purpose of the program within the context of this program. - /// Should be phrased as a noun (e.g., "image processing" or "audio normalization"). - internal ExternalTool(string name, string url, string purpose) - { - Name = name.Trim(); - Url = url.Trim(); - Purpose = purpose.Trim(); - } - - /// - /// Attempts a dry run of the program to determine if it is installed and available on this system. - /// - /// A Result indicating whether the program is available or not. - internal Result ProgramExists() - { - ProcessStartInfo processStartInfo = new() - { - FileName = Name, - UseShellExecute = false, - RedirectStandardOutput = true, - RedirectStandardError = true, - CreateNoWindow = true, - }; - - try - { - using var process = Process.Start(processStartInfo); - - if (process is null) - { - return Result.Fail( - $"The program \"{Name}\" was not found. (The process was null.)" - ); - } - - process.WaitForExit(); - return Result.Ok(); - } - catch (Exception) - { - return Result.Fail($"The program \"{Name}\" was not found."); - } - } -} diff --git a/src/CCVTAC.Console/ExternalTools/Runner.cs b/src/CCVTAC.Console/ExternalTools/Runner.cs deleted file mode 100644 index 88e90acb..00000000 --- a/src/CCVTAC.Console/ExternalTools/Runner.cs +++ /dev/null @@ -1,60 +0,0 @@ -using System.Diagnostics; - -namespace CCVTAC.Console.ExternalTools; - -internal static class Runner -{ - private const int AuthenticSuccessExitCode = 0; - - private static bool IsSuccessExitCode(HashSet otherSuccessExitCodes, int exitCode) => - otherSuccessExitCodes.Append(AuthenticSuccessExitCode).Contains(exitCode); - - /// - /// Calls an external application. - /// - /// - /// Additional exit codes, other than 0, that can be treated as non-failures. - /// - /// A `Result` containing the exit code, if successful, or else an error message. - internal static Result<(int SuccessExitCode, string Warnings)> Run( - ToolSettings settings, - HashSet otherSuccessExitCodes, - Printer printer - ) - { - Watch watch = new(); - - printer.Info($"Running {settings.CommandWithArgs}..."); - - var splitCommandWithArgs = settings.CommandWithArgs.Split([' '], 2); - - ProcessStartInfo processStartInfo = new() - { - FileName = splitCommandWithArgs[0], - Arguments = splitCommandWithArgs.Length > 1 ? splitCommandWithArgs[1] : string.Empty, - UseShellExecute = false, - RedirectStandardOutput = false, - RedirectStandardError = true, - CreateNoWindow = true, - WorkingDirectory = settings.WorkingDirectory, - }; - - using Process? process = Process.Start(processStartInfo); - - if (process is null) - { - return Result.Fail($"Could not locate {splitCommandWithArgs[0]}."); - } - - string errors = process.StandardError.ReadToEnd(); // Must precede `WaitForExit()` - process.WaitForExit(); - printer.Info($"{splitCommandWithArgs[0]} finished in {watch.ElapsedFriendly}."); - - var trimmedErrors = errors.TrimTerminalLineBreak(); - return IsSuccessExitCode(otherSuccessExitCodes, process.ExitCode) - ? Result.Ok((process.ExitCode, trimmedErrors)) // Errors will be considered warnings. - : Result.Fail( - $"{splitCommandWithArgs[0]} exited with code {process.ExitCode}: {trimmedErrors}." - ); - } -} diff --git a/src/CCVTAC.Console/ExternalTools/ToolSettings.cs b/src/CCVTAC.Console/ExternalTools/ToolSettings.cs deleted file mode 100644 index f02244ca..00000000 --- a/src/CCVTAC.Console/ExternalTools/ToolSettings.cs +++ /dev/null @@ -1,6 +0,0 @@ -namespace CCVTAC.Console.ExternalTools; - -/// -/// Settings to govern the behavior of an external program. -/// -internal sealed record ToolSettings(string CommandWithArgs, string WorkingDirectory); diff --git a/src/CCVTAC.Console/Help.cs b/src/CCVTAC.Console/Help.cs deleted file mode 100644 index dd7fd327..00000000 --- a/src/CCVTAC.Console/Help.cs +++ /dev/null @@ -1,92 +0,0 @@ -namespace CCVTAC.Console; - -public static class Help -{ - internal static void Print(Printer printer) - { - const string helpText = """ - CCVTAC (CodeConscious Video-to-Audio Converter) is a small .NET-powered CLI - tool that acts as a wrapper around yt-dlp (https://github.com/yt-dlp/yt-dlp) - to enable easier downloads of audio from YouTube videos, playlists, and - channels, plus do some automatic post-processing (tagging, renaming, and - moving) too. - - While I maintain it primarily for my own use, feel free to use it yourself. - No warranties or guarantees are provided. - - FEATURES - - - Converts YouTube videos, playlists, and channels to local audio files (via yt-dlp) - - Writes ID3 tags to files where possible using available or regex-detected metadata - - Adds video metadata (channel name and URL, video URL, etc.) to files' Comment tags - - Auto-renames files via custom regex patterns (to remove media IDs, etc.) - - Optionally writes video thumbnails to files as artwork (if mogrify is installed) - - Customized behavior via a user settings file -- e.g., chapter splitting, image embedding, directories - - Saves entered URLs to a local history file - - PREREQUISITES - - • .NET 10 runtime (https://dotnet.microsoft.com/en-us/download/dotnet/10.0) - • yt-dlp (https://github.com/yt-dlp/yt-dlp) - • [ffmpeg](https://ffmpeg.org/) (for yt-dlp artwork extraction) - • Optional: mogrify https://imagemagick.org/script/mogrify.php - (for auto-trimming album art) - - RUNNING IT - - Settings: - - A valid settings file is mandatory to use this application. - - The application will look for a file named `settings.json` in its directory. - However, you can manually specify an existing file path using the `-s` - option, such as `dotnet run -- -s `. - - If your `settings.json` file does not exist, a default file will be created in the - application directory with default settings. At minimum, you will need to - enter (1) an existing directory for temporary working files, (2) an existing - directory to which the final audio files should be moved, and (3) a path to - your history file. The other settings have sensible defaults. - - I added the `sleepSecondsBetweenDownloads` and `sleepSecondsBetweenURLs` - settings to help reduce concentrated loads on YouTube servers. Please avoid - lowering these values too much and slamming their servers with enormous, - long-running downloads (even if you feel their servers can take it). Such behavior - might get you rate-limited by YouTube. - - See the README file on the GitHub repo for more about settings. - - Using the application: - - Once your settings are ready, run the application with `dotnet run`. - Alternatively, pass `-h` or `--help` for instructions (e.g., - `dotnet run -- --help`). - - When the application is running, enter at least one YouTube media URL (video, - playlist, or channel) at the prompt and press Enter. No spaces between - items are necessary. - - You can also enter the following commands: - - "\help" to see this list of commands - - "\quit" or "\q" to quit - - "\history" to see the last few URLs you entered - - "\update-downloader" or "\update-dl" to update yt-dlp using the command in your settings - (If you start experiencing constant download errors, try this command) - - Modify the current session only (without updating the settings file): - - `\split` toggles chapter splitting - - `\images` toggles image embedding - - `\quiet` toggles quiet mode - - `\format-` followed by a supported audio format (e.g., `\format-m4a`) changes the format - - `\quality-` followed by a supported audio quality (e.g., `\quality-0`) changes the audio quality - - Enter `\commands` to see this summary in the application. - - Reporting issues: - - If you run into any issues, feel free to create an issue on GitHub. Please provide as much - information as possible (e.g., entered URLs, system information, yt-dlp version). - """; - - printer.Info(helpText, processMarkup: false); - } -} diff --git a/src/CCVTAC.Console/History.cs b/src/CCVTAC.Console/History.cs deleted file mode 100644 index 8c5fac8c..00000000 --- a/src/CCVTAC.Console/History.cs +++ /dev/null @@ -1,73 +0,0 @@ -using System.IO; -using System.Text.Json; -using Spectre.Console; - -namespace CCVTAC.Console; - -/// -/// Handles storing, retrieving, and (eventually) analyzing data relating -/// to URLs that the user has entered. -/// -public class History -{ - private const char Separator = ';'; - private string FilePath { get; } - private byte DisplayCount { get; } - - public History(string filePath, byte displayCount) - { - FilePath = filePath; - DisplayCount = displayCount; - } - - /// - /// Add a URL and related data to the history file. - /// - public void Append(string url, DateTime entryTime, Printer printer) - { - try - { - string serializedEntryTime = JsonSerializer.Serialize(entryTime).Replace("\"", ""); - File.AppendAllText( - FilePath, - serializedEntryTime + Separator + url + Environment.NewLine - ); - - printer.Debug($"Added \"{url}\" to the history log."); - } - catch (Exception ex) - { - printer.Error("Could not append URL(s) to history log: " + ex.Message); - } - } - - public void ShowRecent(Printer printer) - { - try - { - IEnumerable> historyData = File.ReadAllLines(FilePath) - .TakeLast(DisplayCount) - .Select(line => line.Split(Separator)) - .Where(lineItems => lineItems.Length == 2) // Only lines with date-times - .GroupBy(line => DateTime.Parse(line[0]), line => line[1]); - - Table table = new(); - table.Border(TableBorder.None); - table.AddColumns("Time", "URL"); - table.Columns[0].PadRight(3); - - foreach (IGrouping thisDate in historyData) - { - var formattedTime = $"{thisDate.Key:yyyy-MM-dd HH:mm:ss}"; - var urls = string.Join(Environment.NewLine, thisDate); - table.AddRow(formattedTime, urls); - } - - Printer.PrintTable(table); - } - catch (Exception ex) - { - printer.Error($"Could not display recent history: {ex.Message}"); - } - } -} diff --git a/src/CCVTAC.Console/InputHelper.cs b/src/CCVTAC.Console/InputHelper.cs deleted file mode 100644 index b1adb630..00000000 --- a/src/CCVTAC.Console/InputHelper.cs +++ /dev/null @@ -1,92 +0,0 @@ -using System.Text.RegularExpressions; - -namespace CCVTAC.Console; - -public static partial class InputHelper -{ - internal static readonly string Prompt = - $"Enter one or more YouTube media URLs or commands (or \"{Commands.HelpCommand}\"):\n▶︎"; - - /// - /// A regular expression that detects where commands and URLs begin in input strings. - /// - [GeneratedRegex("""(?:https:|\\)""")] - private static partial Regex UserInputRegex(); - - private record IndexPair(int Start, int End); - - /// - /// Takes a user input string and splits it into a collection of inputs based - /// upon substrings detected by the class's regular expression pattern. - /// - public static ImmutableArray SplitInput(string input) - { - var matches = UserInputRegex().Matches(input).OfType().ToImmutableArray(); - - if (matches.Length == 0) - { - return []; - } - - if (matches.Length == 1) - { - return [input]; - } - - var startIndices = matches.Select(m => m.Index).ToImmutableArray(); - - var indexPairs = startIndices.Select( - (startIndex, iterIndex) => - { - int endIndex = - iterIndex == startIndices.Length - 1 - ? input.Length - : startIndices[iterIndex + 1]; - - return new IndexPair(startIndex, endIndex); - } - ); - - var splitInputs = indexPairs.Select(p => input[p.Start..p.End].Trim()).Distinct(); - - return [.. splitInputs]; - } - - internal enum InputCategory - { - Url, - Command, - } - - internal record CategorizedInput(string Text, InputCategory Category); - - internal static ImmutableArray CategorizeInputs(ICollection inputs) - { - return - [ - .. inputs.Select(input => new CategorizedInput( - input, - input.StartsWith(Commands.Prefix) ? InputCategory.Command : InputCategory.Url - )), - ]; - } - - internal class CategoryCounts - { - private readonly Dictionary _counts; - - internal CategoryCounts(Dictionary counts) - { - _counts = counts; - } - - public int this[InputCategory category] => _counts.GetValueOrDefault(category, 0); - } - - internal static CategoryCounts CountCategories(ICollection inputs) - { - var counts = inputs.GroupBy(i => i.Category).ToDictionary(gr => gr.Key, gr => gr.Count()); - - return new CategoryCounts(counts); - } -} diff --git a/src/CCVTAC.Console/IoUtilities/Directories.cs b/src/CCVTAC.Console/IoUtilities/Directories.cs deleted file mode 100644 index 2703a8c6..00000000 --- a/src/CCVTAC.Console/IoUtilities/Directories.cs +++ /dev/null @@ -1,118 +0,0 @@ -using System.IO; -using System.Text; -using CCVTAC.Console.PostProcessing; - -namespace CCVTAC.Console.IoUtilities; - -internal static class Directories -{ - private const string AllFilesSearchPattern = "*"; - private static readonly EnumerationOptions EnumerationOptions = new(); - - internal static int AudioFileCount(string directory) - { - return new DirectoryInfo(directory) - .EnumerateFiles() - .Count(f => PostProcessor.AudioExtensions.CaseInsensitiveContains(f.Extension)); - } - - internal static Result WarnIfAnyFiles(string directory, int showMax) - { - var fileNames = GetDirectoryFileNames(directory); - var fileCount = fileNames.Length; - - if (fileNames.IsEmpty) - { - return Result.Ok(); - } - - var fileLabel = fileCount == 1 ? "file" : "files"; - var report = new StringBuilder( - $"Unexpectedly found {fileCount} {fileLabel} in working directory \"{directory}\":{Environment.NewLine}" - ); - - foreach (string fileName in fileNames.Take(showMax)) - { - report.AppendLine($"• {fileName}"); - } - - if (fileCount > showMax) - { - report.AppendLine($"... plus {fileCount - showMax} more."); - } - - report.AppendLine("This generally occurs due to the same video appearing twice in playlists."); - - return Result.Fail(report.ToString()); - } - - internal static Result DeleteAllFiles(string workingDirectory, int showMaxErrors) - { - var fileNames = GetDirectoryFileNames(workingDirectory); - - int successCount = 0; - var errors = new List(); - - foreach (var fileName in fileNames) - { - try - { - File.Delete(fileName); - successCount++; - } - catch (Exception ex) - { - errors.Add(ex.Message); - } - } - - if (errors.Count == 0) - { - return Result.Ok(successCount); - } - - var output = new StringBuilder( - $"While {successCount} files were deleted successfully, some files could not be deleted:" - ); - foreach (string error in errors.Take(showMaxErrors)) - { - output.AppendLine($"• {error}"); - } - - if (errors.Count > showMaxErrors) - { - output.AppendLine($"... plus {errors.Count - showMaxErrors} more."); - } - - return Result.Fail(output.ToString()); - } - - internal static Result AskToDeleteAllFiles(string workingDirectory, Printer printer) - { - bool doDelete = printer.AskToBool("Delete all temporary files?", "Yes", "No"); - - return doDelete - ? DeleteAllFiles(workingDirectory, 10) - : Result.Fail("Will not delete the files."); - } - - /// - /// Returns the filenames in a given directory, optionally ignoring specific filenames. - /// - /// - /// An optional list of files to be excluded. - private static ImmutableArray GetDirectoryFileNames( - string directoryName, - IEnumerable? customIgnoreFiles = null - ) - { - var ignoreFiles = customIgnoreFiles?.Distinct() ?? []; - - return - [ - .. Directory - .GetFiles(directoryName, AllFilesSearchPattern, EnumerationOptions) - .Where(filePath => ignoreFiles.None(filePath.EndsWith)), - ]; - } -} diff --git a/src/CCVTAC.Console/Orchestrator.cs b/src/CCVTAC.Console/Orchestrator.cs deleted file mode 100644 index 21d8cf73..00000000 --- a/src/CCVTAC.Console/Orchestrator.cs +++ /dev/null @@ -1,418 +0,0 @@ -using System.Threading; -using CCVTAC.Console.Downloading; -using CCVTAC.Console.IoUtilities; -using CCVTAC.Console.PostProcessing; -using CCVTAC.Console.Settings; -using Spectre.Console; -using static CCVTAC.Console.InputHelper; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console; - -/// -/// Drives the primary input gathering and processing tasks. -/// -internal class Orchestrator -{ - /// - /// Ensures the download environment is ready, then initiates the UI input and download process. - /// - internal static void Start(UserSettings settings, Printer printer) - { - // The working directory should start empty. Give the user a chance to empty it. - var emptyDirResult = Directories.WarnIfAnyFiles(settings.WorkingDirectory, 10); - if (emptyDirResult.IsFailed) - { - printer.FirstError(emptyDirResult); - - var deleteResult = Directories.AskToDeleteAllFiles(settings.WorkingDirectory, printer); - if (deleteResult.IsSuccess) - { - printer.Info($"{deleteResult.Value} file(s) deleted."); - } - else - { - printer.FirstError(deleteResult); - printer.Info("Aborting..."); - return; - } - } - - var results = new ResultTracker(printer); - var history = new History(settings.HistoryFile, settings.HistoryDisplayCount); - var nextAction = NextAction.Continue; - - while (nextAction is NextAction.Continue) - { - var input = printer.GetInput(InputHelper.Prompt); - var splitInputs = InputHelper.SplitInput(input); - - if (splitInputs.IsEmpty) - { - printer.Error( - $"Invalid input. Enter only URLs or commands beginning with \"{Commands.Prefix}\"." - ); - continue; - } - - var categorizedInputs = InputHelper.CategorizeInputs(splitInputs); - var categoryCounts = InputHelper.CountCategories(categorizedInputs); - - SummarizeInput(categorizedInputs, categoryCounts, printer); - - nextAction = ProcessBatch( - categorizedInputs, - categoryCounts, - ref settings, - results, - history, - printer - ); - } - - results.PrintSessionSummary(); - } - - /// - /// Processes a single user request, from input to downloading and file post-processing. - /// - /// Returns the next action the application should take (e.g., continue or quit). - private static NextAction ProcessBatch( - ImmutableArray categorizedInputs, - CategoryCounts categoryCounts, - ref UserSettings settings, - ResultTracker resultTracker, - History history, - Printer printer - ) - { - var inputTime = DateTime.Now; - var nextAction = NextAction.Continue; - Watch watch = new(); - - var batchResults = new ResultTracker(printer); - int inputIndex = 0; - - foreach (var input in categorizedInputs) - { - var result = - input.Category is InputCategory.Command - ? ProcessCommand(input.Text, ref settings, history, printer) - : ProcessUrl( - input.Text, - settings, - resultTracker, - history, - inputTime, - categoryCounts[InputCategory.Url], - ++inputIndex, - printer - ); - - batchResults.RegisterResult(input.Text, result); - - if (result.IsFailed) - { - printer.Error(result.Errors.First().Message); - continue; - } - - nextAction = result.Value; - - if (nextAction is not NextAction.Continue) - { - break; - } - } - - if (categoryCounts[InputCategory.Url] > 1) - { - printer.Info( - $"{Environment.NewLine}Finished with batch of {categoryCounts[InputCategory.Url]} URLs in {watch.ElapsedFriendly}." - ); - batchResults.PrintBatchFailures(); - } - - return nextAction; - } - - private static Result ProcessUrl( - string url, - UserSettings settings, - ResultTracker resultTracker, - History history, - DateTime urlInputTime, - int batchSize, - int urlIndex, - Printer printer - ) - { - var emptyDirResult = Directories.WarnIfAnyFiles(settings.WorkingDirectory, 10); - if (emptyDirResult.IsFailed) - { - printer.FirstError(emptyDirResult); - return NextAction.QuitDueToErrors; // TODO: Perhaps determine a better way. - } - - if (urlIndex > 1) // Don't sleep for the very first URL. - { - Sleep(settings.SleepSecondsBetweenURLs); - printer.Info( - $"Slept for {settings.SleepSecondsBetweenURLs} second(s).", - appendLines: 1 - ); - } - - if (batchSize > 1) - { - printer.Info($"Processing group {urlIndex} of {batchSize}..."); - } - - Watch jobWatch = new(); - - var mediaTypeResult = Downloader.WrapUrlInMediaType(url); - if (mediaTypeResult.IsFailed) - { - var errorMsg = $"URL parse error: {mediaTypeResult.Errors.First().Message}"; - printer.Error(errorMsg); - return Result.Fail(errorMsg); - } - var mediaType = mediaTypeResult.Value; - - printer.Info($"{mediaType.GetType().Name} URL '{url}' detected."); - history.Append(url, urlInputTime, printer); - - var downloadResult = Downloader.Run(mediaType, settings, printer); - resultTracker.RegisterResult(url, downloadResult); - - if (downloadResult.IsFailed) - { - var errorMsg = $"Download error: {downloadResult.Errors.First().Message}"; - printer.Error(errorMsg); - return Result.Fail(errorMsg); - } - - printer.Debug($"Successfully downloaded \"{downloadResult.Value}\" format."); - - PostProcessor.Run(settings, mediaType, printer); - - string groupClause = batchSize > 1 ? $" (group {urlIndex} of {batchSize})" : string.Empty; - - printer.Info($"Processed '{url}'{groupClause} in {jobWatch.ElapsedFriendly}."); - return NextAction.Continue; - } - - private static Result ProcessCommand( - string command, - ref UserSettings settings, - History history, - Printer printer - ) - { - if (Commands.HelpCommand.Equals(command, StringComparison.InvariantCultureIgnoreCase)) - { - foreach (var (cmd, description) in Commands.Summary) - { - printer.Info(cmd); - printer.Info($" {description}"); - } - - return Result.Ok(NextAction.Continue); - } - - if (Commands.QuitCommands.CaseInsensitiveContains(command)) - { - return Result.Ok(NextAction.QuitAtUserRequest); - } - - if (Commands.History.CaseInsensitiveContains(command)) - { - history.ShowRecent(printer); - return Result.Ok(NextAction.Continue); - } - - if (Commands.UpdateDownloader.CaseInsensitiveContains(command)) - { - Updater.Run(settings, printer); - return Result.Ok(NextAction.Continue); - } - - if (Commands.SettingsSummary.CaseInsensitiveContains(command)) - { - SettingsAdapter.PrintSummary(settings, printer); - return Result.Ok(NextAction.Continue); - } - - static string SummarizeToggle(string settingName, bool setting) => - $"{settingName} was toggled to {(setting ? "ON" : "OFF")} for this session."; - - static string SummarizeUpdate(string settingName, string setting) => - $"{settingName} was updated to \"{setting}\" for this session."; - - if (Commands.SplitChapterToggles.CaseInsensitiveContains(command)) - { - settings = SettingsAdapter.ToggleSplitChapters(settings); - printer.Info(SummarizeToggle("Split Chapters", settings.SplitChapters)); - return Result.Ok(NextAction.Continue); - } - - if (Commands.EmbedImagesToggles.CaseInsensitiveContains(command)) - { - settings = SettingsAdapter.ToggleEmbedImages(settings); - printer.Info(SummarizeToggle("Embed Images", settings.EmbedImages)); - return Result.Ok(NextAction.Continue); - } - - if (Commands.QuietModeToggles.CaseInsensitiveContains(command)) - { - settings = SettingsAdapter.ToggleQuietMode(settings); - printer.Info(SummarizeToggle("Quiet Mode", settings.QuietMode)); - printer.ShowDebug(!settings.QuietMode); - return Result.Ok(NextAction.Continue); - } - - if ( - command.StartsWith( - Commands.UpdateAudioFormatPrefix, - StringComparison.InvariantCultureIgnoreCase - ) - ) - { - var format = command - .Replace(Commands.UpdateAudioFormatPrefix, string.Empty) - .ToLowerInvariant(); - - if (format == string.Empty) - { - return Result.Fail( - $"You must append one or more supported audio format separated by commas (e.g., \"m4a,opus,best\")." - ); - } - - var updateResult = SettingsAdapter.UpdateAudioFormat(settings, format); - if (updateResult.IsError) - { - return Result.Fail(updateResult.ErrorValue); - } - - settings = updateResult.ResultValue; - printer.Info( - SummarizeUpdate("Audio Formats", string.Join(", ", settings.AudioFormats)) - ); - return Result.Ok(NextAction.Continue); - } - - if ( - command.StartsWith( - Commands.UpdateAudioQualityPrefix, - StringComparison.InvariantCultureIgnoreCase - ) - ) - { - var inputQuality = command.Replace(Commands.UpdateAudioQualityPrefix, string.Empty); - - if (inputQuality == string.Empty) - { - return Result.Fail($"You must enter a number representing an audio quality."); - } - - if (!byte.TryParse(inputQuality, out var quality)) - { - return Result.Fail($"\"{inputQuality}\" is an invalid quality value."); - } - - var updateResult = SettingsAdapter.UpdateAudioQuality(settings, quality); - if (updateResult.IsError) - { - return Result.Fail(updateResult.ErrorValue); // For out-of-range values - } - - settings = updateResult.ResultValue; - printer.Info(SummarizeUpdate("Audio Quality", settings.AudioQuality.ToString())); - return Result.Ok(NextAction.Continue); - } - - return Result.Fail( - $"\"{command}\" is not a valid command. Enter \"\\commands\" to see a list of commands." - ); - } - - private static void SummarizeInput( - ImmutableArray categorizedInputs, - CategoryCounts counts, - Printer printer - ) - { - if (categorizedInputs.Length > 1) - { - var urlSummary = counts[InputCategory.Url] switch - { - 1 => "1 URL", - > 1 => $"{counts[InputCategory.Url]} URLs", - _ => string.Empty, - }; - - var commandSummary = counts[InputCategory.Command] switch - { - 1 => "1 command", - > 1 => $"{counts[InputCategory.Command]} commands", - _ => string.Empty, - }; - - var connector = - urlSummary.HasText() && commandSummary.HasText() ? " and " : string.Empty; - - printer.Info($"Batch of {urlSummary}{connector}{commandSummary} entered."); - - foreach (CategorizedInput input in categorizedInputs) - { - printer.Info($" • {input.Text}"); - } - Printer.EmptyLines(1); - } - } - - private static void Sleep(ushort sleepSeconds) - { - ushort remainingSeconds = sleepSeconds; - - AnsiConsole - .Status() - .Start( - $"Sleeping for {sleepSeconds} seconds...", - ctx => - { - ctx.Spinner(Spinner.Known.Star); - ctx.SpinnerStyle(Style.Parse("blue")); - - while (remainingSeconds > 0) - { - ctx.Status($"Sleeping for {remainingSeconds} seconds..."); - remainingSeconds--; - Thread.Sleep(1000); - } - } - ); - } - - /// - /// Actions that inform the program what it should do after the current step is done. - /// - private enum NextAction : byte - { - /// - /// Program execution should continue and not end. - /// - Continue, - - /// - /// Program execution should end at the user's request. - /// - QuitAtUserRequest, - - /// - /// Program execution should end due to an inability to continue. - /// - QuitDueToErrors, - } -} diff --git a/src/CCVTAC.Console/PostProcessing/CollectionMetadata.cs b/src/CCVTAC.Console/PostProcessing/CollectionMetadata.cs deleted file mode 100644 index b79008ef..00000000 --- a/src/CCVTAC.Console/PostProcessing/CollectionMetadata.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CCVTAC.Console.PostProcessing; - -/// -/// Represents the JSON file of a YouTube playlist or channel. (Both contain -/// the fields that I use, so this is a combined object for now. This might -/// be changed in the future, though.) -/// -public readonly record struct CollectionMetadata( - [property: JsonPropertyName("id")] string Id, - [property: JsonPropertyName("title")] string Title, - [property: JsonPropertyName("availability")] string Availability, - [property: JsonPropertyName("description")] string Description, - [property: JsonPropertyName("tags")] IReadOnlyList Tags, - [property: JsonPropertyName("modified_date")] string ModifiedDate, - [property: JsonPropertyName("view_count")] int? ViewCount, - [property: JsonPropertyName("playlist_count")] int? PlaylistCount, - [property: JsonPropertyName("channel")] string Channel, - [property: JsonPropertyName("channel_id")] string ChannelId, - [property: JsonPropertyName("uploader_id")] string UploaderId, - [property: JsonPropertyName("uploader")] string Uploader, - [property: JsonPropertyName("channel_url")] string ChannelUrl, - [property: JsonPropertyName("uploader_url")] string UploaderUrl, - [property: JsonPropertyName("_type")] string Type, - [property: JsonPropertyName("webpage_url")] string WebpageUrl, - [property: JsonPropertyName("webpage_url_basename")] string WebpageUrlBasename, - [property: JsonPropertyName("webpage_url_domain")] string WebpageUrlDomain, - [property: JsonPropertyName("epoch")] int? Epoch -); diff --git a/src/CCVTAC.Console/PostProcessing/Deleter.cs b/src/CCVTAC.Console/PostProcessing/Deleter.cs deleted file mode 100644 index f0d7a300..00000000 --- a/src/CCVTAC.Console/PostProcessing/Deleter.cs +++ /dev/null @@ -1,79 +0,0 @@ -using System.IO; - -namespace CCVTAC.Console.PostProcessing; - -internal static class Deleter -{ - internal static void Run( - IReadOnlyCollection taggingSetFileNames, - CollectionMetadata? collectionMetadata, - string workingDirectory, - Printer printer - ) - { - ImmutableList collectionFileNames; - var getFileResult = GetCollectionFiles(collectionMetadata, workingDirectory); - if (getFileResult.IsSuccess) - { - var files = getFileResult.Value; - collectionFileNames = files; - - printer.Debug($"Found {files.Count} collection files."); - } - else - { - collectionFileNames = []; - printer.Warning(getFileResult.Errors.First().Message); - } - - var allFileNames = taggingSetFileNames.Concat(collectionFileNames).ToList(); - - if (allFileNames.Count == 0) - { - printer.Warning("No files to delete were found."); - return; - } - - printer.Debug($"Deleting {allFileNames.Count} temporary files..."); - - DeleteAll(allFileNames, printer); - - printer.Info("Deleted temporary files."); - } - - private static Result> GetCollectionFiles( - CollectionMetadata? collectionMetadata, - string workingDirectory - ) - { - if (collectionMetadata is null) - return Result.Ok(ImmutableList.Empty); - - try - { - var id = collectionMetadata.Value.Id; - return Directory.GetFiles(workingDirectory, $"*{id}*").ToImmutableList(); - } - catch (Exception ex) - { - return Result.Fail($"Error collecting filenames: {ex.Message}"); - } - } - - private static void DeleteAll(IEnumerable fileNames, Printer printer) - { - foreach (var fileName in fileNames) - { - try - { - File.Delete(fileName); - - printer.Debug($"• Deleted \"{fileName}\""); - } - catch (Exception ex) - { - printer.Error($"• Deletion error: {ex.Message}"); - } - } - } -} diff --git a/src/CCVTAC.Console/PostProcessing/ImageProcessor.cs b/src/CCVTAC.Console/PostProcessing/ImageProcessor.cs deleted file mode 100644 index c690d57e..00000000 --- a/src/CCVTAC.Console/PostProcessing/ImageProcessor.cs +++ /dev/null @@ -1,15 +0,0 @@ -using CCVTAC.Console.ExternalTools; - -namespace CCVTAC.Console.PostProcessing; - -internal static class ImageProcessor -{ - internal static readonly string ProgramName = "mogrify"; - - internal static void Run(string workingDirectory, Printer printer) - { - ToolSettings imageEditToolSettings = new($"{ProgramName} -trim -fuzz 10% *.jpg", workingDirectory); - - Runner.Run(imageEditToolSettings, [], printer); - } -} diff --git a/src/CCVTAC.Console/PostProcessing/Mover.cs b/src/CCVTAC.Console/PostProcessing/Mover.cs deleted file mode 100644 index 63ff2109..00000000 --- a/src/CCVTAC.Console/PostProcessing/Mover.cs +++ /dev/null @@ -1,249 +0,0 @@ -using System.IO; -using System.Text.Json; -using System.Text.RegularExpressions; -using CCVTAC.Console.PostProcessing.Tagging; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.PostProcessing; - -internal static class Mover -{ - private static readonly Regex PlaylistImageRegex = new(@"\[[OP]L[\w\d_-]{12,}\]"); - private const string ImageFileWildcard = "*.jp*"; - - internal static void Run( - IEnumerable taggingSets, - CollectionMetadata? maybeCollectionData, - UserSettings settings, - bool overwrite, - Printer printer - ) - { - Watch watch = new(); - - var workingDirInfo = new DirectoryInfo(settings.WorkingDirectory); - - string subFolderName = GetSafeSubDirectoryName(maybeCollectionData, taggingSets.First()); - string collectionName = maybeCollectionData?.Title ?? string.Empty; - string fullMoveToDir = Path.Combine( - settings.MoveToDirectory, - subFolderName, - collectionName - ); - - var dirResult = EnsureDirectoryExists(fullMoveToDir, printer); - if (dirResult.IsFailed) - { - return; // The error message is printed within the above method. - } - - var audioFileNames = workingDirInfo - .EnumerateFiles() - .Where(f => PostProcessor.AudioExtensions.CaseInsensitiveContains(f.Extension)) - .ToImmutableList(); - - if (audioFileNames.IsEmpty) - { - printer.Error("No audio filenames to move found."); - return; - } - - printer.Debug($"Moving {audioFileNames.Count} audio file(s) to \"{fullMoveToDir}\"..."); - - var (successCount, failureCount) = MoveAudioFiles( - audioFileNames, - fullMoveToDir, - overwrite, - printer - ); - - MoveImageFile( - collectionName, - subFolderName, - workingDirInfo, - fullMoveToDir, - audioFileNames.Count, - overwrite, - printer - ); - - var fileLabel = successCount == 1 ? "file" : "files"; - printer.Info($"Moved {successCount} audio {fileLabel} in {watch.ElapsedFriendly}."); - - if (failureCount > 0) - { - fileLabel = failureCount == 1 ? "file" : "files"; - printer.Warning($"However, {failureCount} audio {fileLabel} could not be moved."); - } - } - - private static bool IsPlaylistImage(string fileName) - { - return PlaylistImageRegex.IsMatch(fileName); - } - - private static FileInfo? GetCoverImage(DirectoryInfo workingDirInfo, int audioFileCount) - { - var images = workingDirInfo.EnumerateFiles(ImageFileWildcard).ToImmutableArray(); - if (images.IsEmpty) - { - return null; - } - - var playlistImages = images.Where(i => IsPlaylistImage(i.FullName)); - if (playlistImages.Any()) - { - return playlistImages.First(); - } - - return audioFileCount > 1 && images.Length == 1 ? images.First() : null; - } - - private static Result EnsureDirectoryExists(string moveToDir, Printer printer) - { - try - { - if (Path.Exists(moveToDir)) - { - printer.Debug($"Found move-to directory \"{moveToDir}\"."); - - return Result.Ok(); - } - - printer.Debug( - $"Creating move-to directory \"{moveToDir}\" (based on playlist metadata)... ", - appendLineBreak: false - ); - - Directory.CreateDirectory(moveToDir); - printer.Debug("OK."); - return Result.Ok(); - } - catch (Exception ex) - { - printer.Error($"Error creating move-to directory \"{moveToDir}\": {ex.Message}"); - return Result.Fail(string.Empty); - } - } - - private static (uint successCount, uint failureCount) MoveAudioFiles( - ICollection audioFiles, - string moveToDir, - bool overwrite, - Printer printer - ) - { - uint successCount = 0; - uint failureCount = 0; - - foreach (FileInfo file in audioFiles) - { - try - { - File.Move(file.FullName, Path.Combine(moveToDir, file.Name), overwrite); - - successCount++; - - printer.Debug($"• Moved \"{file.Name}\""); - } - catch (Exception ex) - { - failureCount++; - printer.Error($"• Error moving file \"{file.Name}\": {ex.Message}"); - } - } - - return (successCount, failureCount); - } - - private static void MoveImageFile( - string maybeCollectionName, - string subFolderName, - DirectoryInfo workingDirInfo, - string moveToDir, - int audioFileCount, - bool overwrite, - Printer printer - ) - { - try - { - var baseFileName = string.IsNullOrWhiteSpace(maybeCollectionName) - ? subFolderName - : $"{subFolderName} - {maybeCollectionName.ReplaceInvalidPathChars()}"; - - if (GetCoverImage(workingDirInfo, audioFileCount) is not FileInfo image) - { - return; - } - - image.MoveTo( - Path.Combine(moveToDir, $"{baseFileName.Trim()}.jpg"), - overwrite: overwrite - ); - - printer.Info("Moved image file."); - } - catch (Exception ex) - { - printer.Warning($"Error copying the image file: {ex.Message}"); - } - } - - private static string GetSafeSubDirectoryName( - CollectionMetadata? collectionData, - TaggingSet taggingSet - ) - { - string workingName; - - if ( - collectionData is CollectionMetadata metadata - && metadata.Uploader.HasText() - && metadata.Title.HasText() - ) - { - workingName = metadata.Uploader; - } - else - { - var jsonResult = GetParsedVideoJson(taggingSet); - - workingName = jsonResult.IsSuccess ? jsonResult.Value.Uploader : string.Empty; - } - - var safeName = workingName.ReplaceInvalidPathChars().Trim(); - - const string topicSuffix = " - Topic"; // Official channels append this to uploader names. - return safeName.EndsWith(topicSuffix) - ? safeName.Replace(topicSuffix, string.Empty) - : safeName; - } - - private static Result GetParsedVideoJson(TaggingSet taggingSet) - { - string json; - try - { - json = File.ReadAllText(taggingSet.JsonFilePath); - } - catch (Exception ex) - { - return Result.Fail( - $"Error reading JSON file \"{taggingSet.JsonFilePath}\": {ex.Message}." - ); - } - - try - { - var videoData = JsonSerializer.Deserialize(json); - return Result.Ok(videoData); - } - catch (JsonException ex) - { - return Result.Fail( - $"Error deserializing JSON from file \"{taggingSet.JsonFilePath}\": {ex.Message}" - ); - } - } -} diff --git a/src/CCVTAC.Console/PostProcessing/PostProcessing.cs b/src/CCVTAC.Console/PostProcessing/PostProcessing.cs deleted file mode 100644 index f4ddcb80..00000000 --- a/src/CCVTAC.Console/PostProcessing/PostProcessing.cs +++ /dev/null @@ -1,159 +0,0 @@ -using System.IO; -using System.Text.Json; -using System.Text.RegularExpressions; -using CCVTAC.Console.IoUtilities; -using CCVTAC.Console.PostProcessing.Tagging; -using static CCVTAC.FSharp.Downloading; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.PostProcessing; - -internal static partial class PostProcessor -{ - internal static readonly string[] AudioExtensions = - [ - ".aac", - ".alac", - ".flac", - ".m4a", - ".mp3", - ".ogg", - ".vorbis", - ".opus", - ".wav", - ]; - - internal static void Run(UserSettings settings, MediaType mediaType, Printer printer) - { - Watch watch = new(); - string workingDirectory = settings.WorkingDirectory; - - printer.Info("Starting post-processing..."); - - var taggingSetsResult = GenerateTaggingSets(workingDirectory); - if (taggingSetsResult.IsFailed) - { - printer.Error("No tagging sets were generated, so tagging cannot be done."); - return; - } - var taggingSets = taggingSetsResult.Value; - - var collectionJsonResult = GetCollectionJson(workingDirectory); - CollectionMetadata? collectionJson; - if (collectionJsonResult.IsFailed) - { - printer.Debug( - $"No playlist or channel metadata found: {collectionJsonResult.Errors.First().Message}" - ); - collectionJson = null; - } - else - { - printer.Debug("Found playlist/channel metadata."); - collectionJson = collectionJsonResult.Value; - } - - if (settings.EmbedImages) - { - ImageProcessor.Run(workingDirectory, printer); - } - - var tagResult = Tagger.Run(settings, taggingSets, collectionJson, mediaType, printer); - if (tagResult.IsSuccess) - { - printer.Info(tagResult.Value); - - Renamer.Run(settings, workingDirectory, printer); - - Mover.Run(taggingSets, collectionJson, settings, true, printer); - - var taggingSetFileNames = taggingSets.SelectMany(set => set.AllFiles).ToList(); - Deleter.Run(taggingSetFileNames, collectionJson, workingDirectory, printer); - - var leftoverFilesResult = Directories.WarnIfAnyFiles(workingDirectory, 20); - if (leftoverFilesResult.IsFailed) - { - printer.FirstError(leftoverFilesResult); - - printer.Info("Will delete the remaining files..."); - var deleteResult = Directories.DeleteAllFiles(workingDirectory, 20); - if (deleteResult.IsSuccess) - { - printer.Info($"{deleteResult.Value} file(s) deleted."); - } - else - { - printer.FirstError(deleteResult); - } - } - } - else - { - printer.Errors("Tagging error(s) preventing further post-processing: ", tagResult); - } - - printer.Info($"Post-processing done in {watch.ElapsedFriendly}."); - } - - private static Result GetCollectionJson(string workingDirectory) - { - try - { - var fileNames = Directory - .GetFiles(workingDirectory) - .Where(f => CollectionMetadataRegex().IsMatch(f)) - .ToImmutableHashSet(); - - if (fileNames.Count == 0) - { - return Result.Fail("No relevant files found."); - } - - if (fileNames.Count > 1) - { - return Result.Fail( - "Unexpectedly found more than one relevant file, so none will be processed." - ); - } - - string fileName = fileNames.Single(); - string json = File.ReadAllText(fileName); - var collectionData = JsonSerializer.Deserialize(json); - - return Result.Ok(collectionData); - } - catch (Exception ex) - { - return Result.Fail($"{ex.Message}"); - } - } - - private static Result> GenerateTaggingSets(string directory) - { - try - { - string[] files = Directory.GetFiles(directory); - var taggingSets = TaggingSet.CreateSets(files); - - return taggingSets.Any() - ? Result.Ok(taggingSets) - : Result.Fail( - $"No tagging sets were created using working directory \"{directory}\"." - ); - } - catch (DirectoryNotFoundException) - { - return Result.Fail($"Directory \"{directory}\" does not exist."); - } - catch (Exception ex) - { - return Result.Fail($"Error reading working directory files: {ex.Message}"); - } - } - - /// - /// A regular expression that detects metadata files for collections. - /// - [GeneratedRegex("""(?<=\[)[\w\-]{17,}(?=\]\.info.json)""")] - private static partial Regex CollectionMetadataRegex(); -} diff --git a/src/CCVTAC.Console/PostProcessing/Renamer.cs b/src/CCVTAC.Console/PostProcessing/Renamer.cs deleted file mode 100644 index 7386c98a..00000000 --- a/src/CCVTAC.Console/PostProcessing/Renamer.cs +++ /dev/null @@ -1,130 +0,0 @@ -using System.IO; -using System.Text; -using System.Text.RegularExpressions; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.PostProcessing; - -internal static class Renamer -{ - public static void Run(UserSettings settings, string workingDirectory, Printer printer) - { - Watch watch = new(); - - var workingDirInfo = new DirectoryInfo(workingDirectory); - var audioFiles = workingDirInfo - .EnumerateFiles() - .Where(f => PostProcessor.AudioExtensions.CaseInsensitiveContains(f.Extension)) - .ToImmutableList(); - - if (audioFiles.None()) - { - printer.Warning("No audio files to rename were found."); - return; - } - - printer.Debug($"Renaming {audioFiles.Count} audio file(s)..."); - - Regex regex; - List matches; - string matchedPatternSummary; - - foreach (FileInfo file in audioFiles) - { - var newFileName = settings.RenamePatterns.Aggregate( - new StringBuilder(file.Name), - (newNameSb, renamePattern) => - { - regex = new Regex(renamePattern.RegexPattern); - - // A match is generated for each instance of the matched substring. - matches = regex - .Matches(newNameSb.ToString()) - .Where(m => m.Success) - .Reverse() // Avoids index errors. - .ToList(); - - if (matches.Count == 0) - { - return newNameSb; - } - - if (!settings.QuietMode) - { - matchedPatternSummary = renamePattern.Summary is null - ? $"`{renamePattern.RegexPattern}` (no description)" - : $"\"{renamePattern.Summary}\""; - - printer.Debug( - $"Rename pattern {matchedPatternSummary} matched × {matches.Count}." - ); - } - - foreach (Match match in matches) - { - // Delete the matched substring from the filename by index. - newNameSb.Remove(match.Index, match.Length); - - // Generate replacement text to be inserted at the same starting index - // using the matches and the replacement patterns from the settings. - var replacementText = match - .Groups.OfType() - // `Select()` indexing begins at 0, but usable regex matches begin at 1, - // so add 1 to both the match group and replacement placeholder indices. - .Select( - (_, i) => - ( - SearchFor: $"%<{i + 1}>s", // Start with placeholder #1 because... - ReplaceWith: match.Groups[i + 1].Value.Trim() // ...we start with regex group #1. - ) - ) - // Starting with the placeholder text from the settings, replace each - // individual placeholder with the correlated match text, then - // return the final string. - .Aggregate( - new StringBuilder(renamePattern.ReplaceWithPattern), - (workingText, replacementParts) => - workingText.Replace( - replacementParts.SearchFor, - replacementParts.ReplaceWith - ), - workingText => workingText.ToString() - ); - - newNameSb.Insert(match.Index, replacementText); - } - - return newNameSb; - }, - newFileNameSb => newFileNameSb.ToString() - ); - - try - { - File.Move( - file.FullName, - Path.Combine(workingDirectory, newFileName) - .Normalize(GetNormalizationForm(settings.NormalizationForm)) - ); - - printer.Debug($"• From: \"{file.Name}\""); - printer.Debug($" To: \"{newFileName}\""); - } - catch (Exception ex) - { - printer.Error($"• Error renaming \"{file.Name}\": {ex.Message}"); - } - } - - printer.Info($"Renaming done in {watch.ElapsedFriendly}."); - } - - private static NormalizationForm GetNormalizationForm(string form) => - form.Trim().ToUpperInvariant() switch - { - "D" => NormalizationForm.FormD, - "KD" => NormalizationForm.FormKD, - "KC" => NormalizationForm.FormKC, - _ => NormalizationForm.FormC, - }; -} diff --git a/src/CCVTAC.Console/PostProcessing/Tagging/Detectors.cs b/src/CCVTAC.Console/PostProcessing/Tagging/Detectors.cs deleted file mode 100644 index 0ed5e786..00000000 --- a/src/CCVTAC.Console/PostProcessing/Tagging/Detectors.cs +++ /dev/null @@ -1,118 +0,0 @@ -using System.Text.RegularExpressions; -using static CCVTAC.FSharp.Settings; - -namespace CCVTAC.Console.PostProcessing.Tagging; - -/// -/// Detection of specific text within video metadata files. -/// -internal static class Detectors -{ - /// - /// Finds and returns the first instance of text matching a given detection scheme pattern, - /// parsing into T if necessary. - /// - /// A match of type T if there was a match; otherwise, the default value provided. - internal static T? DetectSingle( - VideoMetadata videoMetadata, - IEnumerable patterns, - T? defaultValue - ) - { - foreach (TagDetectionPattern pattern in patterns) - { - string fieldText = ExtractMetadataText(videoMetadata, pattern.SearchField); - - // TODO: Instantiate regexes during settings deserialization. - var match = new Regex(pattern.RegexPattern).Match(fieldText); - - if (!match.Success) - { - continue; - } - - string matchedText = match.Groups[pattern.MatchGroup].Value.Trim(); - return Cast(matchedText, defaultValue); - } - - return defaultValue; // No matches were found. - } - - /// - /// Finds and returns all instances of text matching a given detection scheme pattern, - /// concatenating them into a single string (using a custom separator), then casting - /// to type T if necessary. - /// - /// A match of type T if there were any matches; otherwise, the default value provided. - internal static T? DetectMultiple( - VideoMetadata data, - IEnumerable patterns, - T? defaultValue, - string separator - ) - { - HashSet matchedValues = []; - - foreach (TagDetectionPattern pattern in patterns) - { - string fieldText = ExtractMetadataText(data, pattern.SearchField); - - // TODO: Instantiate regexes during settings deserialization. - var matches = new Regex(pattern.RegexPattern).Matches(fieldText); - - foreach (Match match in matches.Where(m => m.Success)) - { - matchedValues.Add(match.Groups[pattern.MatchGroup].Value.Trim()); - } - } - - if (matchedValues.Count == 0) - { - return defaultValue; - } - - string joinedMatchedText = string.Join(separator, matchedValues); - return Cast(joinedMatchedText, defaultValue); - } - - /// - /// Attempts casting the input text to type T and returning it. - /// If casting fails, the default value is returned instead. - /// - private static T? Cast(string? text, T? defaultValue) - { - if (text is T) - { - return (T)(object)text; - } - - try - { - return (T?)Convert.ChangeType(text, typeof(T)); - } - catch (InvalidCastException) - { - return defaultValue; - } - } - - /// - /// Extracts the value of the specified tag field from the given data. - /// - /// - /// The name of the field within the video metadata to read. - /// The text content of the requested field of the video metadata. - private static string ExtractMetadataText(VideoMetadata metadata, string fieldName) - { - return fieldName switch - { - "title" => metadata.Title, - "description" => metadata.Description, - - // TODO: It would be best to check for invalid entries upon settings deserialization. - _ => throw new ArgumentException( - $"\"{fieldName}\" is an invalid video metadata field name." - ), - }; - } -} diff --git a/src/CCVTAC.Console/PostProcessing/Tagging/TagDetector.cs b/src/CCVTAC.Console/PostProcessing/Tagging/TagDetector.cs deleted file mode 100644 index 4e7c9957..00000000 --- a/src/CCVTAC.Console/PostProcessing/Tagging/TagDetector.cs +++ /dev/null @@ -1,45 +0,0 @@ -using static CCVTAC.FSharp.Settings; - -namespace CCVTAC.Console.PostProcessing.Tagging; - -/// -/// Provides methods to search for specific tag field data (artist, album, etc.) within video metadata. -/// -internal sealed class TagDetector -{ - private TagDetectionPatterns Patterns { get; } - - internal TagDetector(TagDetectionPatterns tagDetectionPatterns) - { - Patterns = tagDetectionPatterns; - } - - internal string? DetectTitle(VideoMetadata videoData, string? defaultTitle = null) - { - return Detectors.DetectSingle(videoData, Patterns.Title, null) ?? defaultTitle; - } - - internal string? DetectArtist(VideoMetadata videoData, string? defaultArtist = null) - { - return Detectors.DetectSingle(videoData, Patterns.Artist, null) ?? defaultArtist; - } - - internal string? DetectAlbum(VideoMetadata videoData, string? defaultAlbum = null) - { - return Detectors.DetectSingle(videoData, Patterns.Album, null) ?? defaultAlbum; - } - - internal string? DetectComposers(VideoMetadata videoData) - { - return Detectors.DetectMultiple(videoData, Patterns.Composer, null, "; "); - } - - internal ushort? DetectReleaseYear(VideoMetadata videoData, ushort? defaultYear) - { - ushort detectedYear = Detectors.DetectSingle(videoData, Patterns.Year, default); - - return detectedYear is default(ushort) // The default ushort value indicates no match was found. - ? defaultYear - : detectedYear; - } -} diff --git a/src/CCVTAC.Console/PostProcessing/Tagging/Tagger.cs b/src/CCVTAC.Console/PostProcessing/Tagging/Tagger.cs deleted file mode 100644 index 288028af..00000000 --- a/src/CCVTAC.Console/PostProcessing/Tagging/Tagger.cs +++ /dev/null @@ -1,302 +0,0 @@ -using System.IO; -using System.Text.Json; -using static CCVTAC.FSharp.Downloading; -using TaggedFile = TagLib.File; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.PostProcessing.Tagging; - -internal static class Tagger -{ - internal static Result Run( - UserSettings settings, - IEnumerable taggingSets, - CollectionMetadata? collectionJson, - MediaType mediaType, - Printer printer - ) - { - printer.Debug("Adding file tags..."); - - Watch watch = new(); - - bool embedImages = settings.EmbedImages && mediaType.IsVideo || mediaType.IsPlaylistVideo; - - foreach (var taggingSet in taggingSets) - { - ProcessSingleTaggingSet(settings, taggingSet, collectionJson, embedImages, printer); - } - - return Result.Ok($"Tagging done in {watch.ElapsedFriendly}."); - } - - private static void ProcessSingleTaggingSet( - UserSettings settings, - TaggingSet taggingSet, - CollectionMetadata? collectionJson, - bool embedImages, - Printer printer - ) - { - printer.Debug( - $"{taggingSet.AudioFilePaths.Count} audio file(s) with resource ID \"{taggingSet.ResourceId}\"" - ); - - var parsedJsonResult = ParseVideoJson(taggingSet); - if (parsedJsonResult.IsFailed) - { - printer.Errors( - $"Error deserializing video metadata from \"{taggingSet.JsonFilePath}\":", - parsedJsonResult - ); - return; - } - - TaggingSet finalTaggingSet = DeleteSourceFile(taggingSet, printer); - - // If a single video was split, the tagging set will have multiple audio file paths. - // In this case, we will NOT embed the image file (with the assumption that - // the standalone image file will be available in the move-to directory). - string? maybeImagePath = - embedImages && finalTaggingSet.AudioFilePaths.Count == 1 - ? finalTaggingSet.ImageFilePath - : null; - - foreach (string audioPath in finalTaggingSet.AudioFilePaths) - { - try - { - TagSingleFile( - settings, - parsedJsonResult.Value, - audioPath, - maybeImagePath, - collectionJson, - printer - ); - } - catch (Exception ex) - { - printer.Error($"Error tagging file: {ex.Message}"); - } - } - } - - private static void TagSingleFile( - UserSettings settings, - VideoMetadata videoData, - string audioFilePath, - string? imageFilePath, - CollectionMetadata? collectionData, - Printer printer - ) - { - var audioFileName = Path.GetFileName(audioFilePath); - - printer.Debug($"Current audio file: \"{audioFileName}\""); - - using var taggedFile = TaggedFile.Create(audioFilePath); - TagDetector tagDetector = new(settings.TagDetectionPatterns); - - if (videoData.Track is { } metadataTitle) - { - printer.Debug($"• Using metadata title \"{metadataTitle}\""); - taggedFile.Tag.Title = metadataTitle; - } - else - { - var title = tagDetector.DetectTitle(videoData, videoData.Title); - printer.Debug($"• Found title \"{title}\""); - taggedFile.Tag.Title = title; - } - - if (videoData.Artist is { } metadataArtists) - { - var firstArtist = metadataArtists.Split(", ").First(); - var diffSummary = - firstArtist == metadataArtists - ? string.Empty - : $" (extracted from \"{metadataArtists}\")"; - taggedFile.Tag.Performers = [firstArtist]; - - printer.Debug($"• Using metadata artist \"{firstArtist}\"{diffSummary}"); - } - else if (tagDetector.DetectArtist(videoData) is { } artist) - { - printer.Debug($"• Found artist \"{artist}\""); - taggedFile.Tag.Performers = [artist]; - } - - if (videoData.Album is { } metadataAlbum) - { - printer.Debug($"• Using metadata album \"{metadataAlbum}\""); - taggedFile.Tag.Album = metadataAlbum; - } - else if (tagDetector.DetectAlbum(videoData, collectionData?.Title) is { } album) - { - printer.Debug($"• Found album \"{album}\""); - taggedFile.Tag.Album = album; - } - - if (tagDetector.DetectComposers(videoData) is { } composers) - { - printer.Debug($"• Found composer(s) \"{composers}\""); - taggedFile.Tag.Composers = [composers]; - } - - if (videoData.PlaylistIndex is { } trackNo) - { - printer.Debug($"• Using playlist index of {trackNo} for track number"); - taggedFile.Tag.Track = trackNo; - } - - if (videoData.ReleaseYear is { } releaseYear) - { - printer.Debug($"• Using metadata release year \"{releaseYear}\""); - taggedFile.Tag.Year = releaseYear; - } - else - { - ushort? maybeDefaultYear = GetAppropriateReleaseDateIfAny(settings, videoData); - - if (tagDetector.DetectReleaseYear(videoData, maybeDefaultYear) is { } year) - { - printer.Debug($"• Found year \"{year}\""); - taggedFile.Tag.Year = year; - } - } - - taggedFile.Tag.Comment = videoData.GenerateComment(collectionData); - - if ( - settings.EmbedImages - && !settings.DoNotEmbedImageUploaders.Contains(videoData.Uploader) - && imageFilePath is not null - ) - { - printer.Info("Embedding artwork."); - WriteImage(taggedFile, imageFilePath, printer); - } - else - { - printer.Debug("Skipping artwork embedding."); - } - - taggedFile.Save(); - printer.Debug($"Wrote tags to \"{audioFileName}\"."); - return; - - // If the supplied video uploader is specified in the settings, returns the video's upload year. - // Otherwise, returns null. - static ushort? GetAppropriateReleaseDateIfAny( - UserSettings settings, - VideoMetadata videoData - ) - { - if ( - settings.IgnoreUploadYearUploaders?.Contains( - videoData.Uploader, - StringComparer.OrdinalIgnoreCase - ) == true - ) - { - return null; - } - - return ushort.TryParse(videoData.UploadDate[..4], out var parsedYear) - ? parsedYear - : null; - } - } - - private static Result ParseVideoJson(TaggingSet taggingSet) - { - string json; - - try - { - json = File.ReadAllText(taggingSet.JsonFilePath); - } - catch (Exception ex) - { - return Result.Fail( - $"Error reading JSON file \"{taggingSet.JsonFilePath}\": {ex.Message}." - ); - } - - try - { - var videoData = JsonSerializer.Deserialize(json); - return Result.Ok(videoData); - } - catch (JsonException ex) - { - return Result.Fail($"{ex.Message}{Environment.NewLine}{ex.StackTrace}"); - } - } - - /// - /// Deletes the pre-split source audio for split videos (if any), each of which will have the same resource ID. - /// Returns a modified TaggingSet with the deleted files removed. - /// - /// - /// - private static TaggingSet DeleteSourceFile(TaggingSet taggingSet, Printer printer) - { - // If there is only one file, then there are no child files, so no action is necessary. - if (taggingSet.AudioFilePaths.Count <= 1) - { - return taggingSet; - } - - // The largest audio file must be the source file. - var largestFileInfo = taggingSet - .AudioFilePaths.Select(fileName => new FileInfo(fileName)) - .OrderByDescending(fi => fi.Length) - .First(); - - try - { - File.Delete(largestFileInfo.FullName); - printer.Debug($"Deleted pre-split source file \"{largestFileInfo.Name}\""); - - return taggingSet with - { - AudioFilePaths = taggingSet.AudioFilePaths.Remove(largestFileInfo.FullName), - }; - } - catch (Exception ex) - { - printer.Error( - $"Error deleting pre-split source file \"{largestFileInfo.Name}\": {ex.Message}" - ); - return taggingSet; - } - } - - /// - /// Write the video thumbnail to the file tags. - /// - /// Heavily inspired by https://stackoverflow.com/a/61264720/11767771. - private static void WriteImage(TaggedFile taggedFile, string imageFilePath, Printer printer) - { - if (string.IsNullOrWhiteSpace(imageFilePath)) - { - printer.Error("No image file path was provided, so cannot add an image to the file."); - return; - } - - try - { - var pics = new TagLib.IPicture[1]; - pics[0] = new TagLib.Picture(imageFilePath); - taggedFile.Tag.Pictures = pics; - - printer.Debug("Image written to file tags OK."); - } - catch (Exception ex) - { - printer.Error($"Error writing image to the audio file: {ex.Message}"); - } - } -} diff --git a/src/CCVTAC.Console/PostProcessing/Tagging/TaggingSet.cs b/src/CCVTAC.Console/PostProcessing/Tagging/TaggingSet.cs deleted file mode 100644 index e1b36730..00000000 --- a/src/CCVTAC.Console/PostProcessing/Tagging/TaggingSet.cs +++ /dev/null @@ -1,129 +0,0 @@ -using System.IO; -using System.Text.RegularExpressions; - -namespace CCVTAC.Console.PostProcessing.Tagging; - -/// -/// Contains all the data necessary for tagging a related set of files. -/// -/// -/// Files are "related" if they share the same resource ID. Generally, only a single downloaded video -/// has a certain video ID, but if the split-chapter option is used, all the child videos that were -/// split out will also have the same resource ID. -/// -internal readonly record struct TaggingSet -{ - /// - /// The ID of a single video and perhaps its child videos (if "split chapters" was used). - /// Used to locate all the related files (whose filenames will contain the same ID). - /// - internal string ResourceId { get; } - - /// - /// All audio files for the associated resource ID. Several files with identical IDs indicates - /// that the original video was split into several audio files. - /// - internal ImmutableHashSet AudioFilePaths { get; init; } - - /// - /// The path to the JSON file containing metadata related to the source video. - /// - internal string JsonFilePath { get; } - - /// - /// The path to the image file containing the thumbnail related to the source video. - /// - internal string ImageFilePath { get; } - - internal IReadOnlyList AllFiles => [.. AudioFilePaths, JsonFilePath, ImageFilePath]; - - /// - /// A regex that finds all files whose filename includes a video ID. - /// Group 1 contains the video ID itself. - /// - private static readonly Regex FileNamesWithVideoIdsRegex = new( - @".+\[([\w_\-]{11})\](?:.*)?\.(\w+)" - ); - - private TaggingSet( - string resourceId, - ICollection audioFilePaths, - string jsonFilePath, - string imageFilePath - ) - { - if (string.IsNullOrWhiteSpace(resourceId)) - throw new ArgumentException("The resource ID must be provided."); - if (string.IsNullOrWhiteSpace(jsonFilePath)) - throw new ArgumentException("The JSON file path must be provided."); - if (string.IsNullOrWhiteSpace(imageFilePath)) - throw new ArgumentException("The image file path must be provided."); - if (audioFilePaths.Count == 0) - throw new ArgumentException( - "At least one audio file path must be provided.", - nameof(audioFilePaths) - ); - - ResourceId = resourceId.Trim(); - AudioFilePaths = audioFilePaths.ToImmutableHashSet(); - JsonFilePath = jsonFilePath.Trim(); - ImageFilePath = imageFilePath.Trim(); - } - - /// - /// Create a collection of TaggingSets from a collection of file paths - /// related to several video IDs. Files that don't match the requirements - /// will be ignored. - /// - /// - /// A collection of file paths. Expected to contain all related audio files (>=1), - /// 1 JSON file, and 1 image file for each distinct video ID. - /// - /// Does not include collection (playlist or channel) metadata files. - internal static ImmutableList CreateSets(ICollection filePaths) - { - if (filePaths.None()) - { - return Enumerable.Empty().ToImmutableList(); - } - - const string jsonFileExt = ".json"; - const string imageFileExt = ".jpg"; - - return filePaths - // First, get regex matches of all files whose filenames contain a video ID regex. - .Select(f => FileNamesWithVideoIdsRegex.Match(f)) - .Where(m => m.Success) - .Select(m => m.Captures.OfType().First()) - // Then, group those files as key-value pairs using the video ID as the key. - .GroupBy( - m => m.Groups[1].Value, // Video ID - m => m.Groups[0].Value // Full filenames (1 or more for each video ID) - ) - // Next, ensure the correct count of image and JSON files, ignoring those that don't match. - // (For thought: It might be an option to track and report the invalid ones as well.) - .Where(gr => - gr.Any(f => - PostProcessor.AudioExtensions.CaseInsensitiveContains(Path.GetExtension(f)) - ) - && gr.Count(f => f.EndsWith(jsonFileExt, StringComparison.OrdinalIgnoreCase)) == 1 - && gr.Count(f => f.EndsWith(imageFileExt, StringComparison.OrdinalIgnoreCase)) == 1 - ) - // Lastly, group everything into new TaggingSets. - .Select(gr => - { - return new TaggingSet( - gr.Key, // Video ID - gr.Where(f => - PostProcessor.AudioExtensions.CaseInsensitiveContains( - Path.GetExtension(f) - ) - ) - .ToList(), - gr.Single(f => f.EndsWith(jsonFileExt)), - gr.Single(f => f.EndsWith(imageFileExt)) - ); - }) - .ToImmutableList(); - } -} diff --git a/src/CCVTAC.Console/PostProcessing/VideoMetadata.cs b/src/CCVTAC.Console/PostProcessing/VideoMetadata.cs deleted file mode 100644 index 90ef5860..00000000 --- a/src/CCVTAC.Console/PostProcessing/VideoMetadata.cs +++ /dev/null @@ -1,81 +0,0 @@ -using System.Text.Json.Serialization; - -namespace CCVTAC.Console.PostProcessing; - -/// -/// Represents the data containing the JSON file downloaded alongside the video. -/// -public readonly record struct VideoMetadata( - [property: JsonPropertyName("id")] string Id, - [property: JsonPropertyName("title")] string Title, - [property: JsonPropertyName("thumbnail")] string Thumbnail, - [property: JsonPropertyName("description")] string Description, - [property: JsonPropertyName("channel_id")] string ChannelId, - [property: JsonPropertyName("channel_url")] string ChannelUrl, - [property: JsonPropertyName("duration")] int? Duration, - [property: JsonPropertyName("view_count")] int? ViewCount, - [property: JsonPropertyName("age_limit")] int? AgeLimit, - [property: JsonPropertyName("webpage_url")] string WebpageUrl, - [property: JsonPropertyName("categories")] IReadOnlyList Categories, - [property: JsonPropertyName("tags")] IReadOnlyList Tags, - [property: JsonPropertyName("playable_in_embed")] bool? PlayableInEmbed, - [property: JsonPropertyName("live_status")] string LiveStatus, - [property: JsonPropertyName("release_timestamp")] int? ReleaseTimestamp, - [property: JsonPropertyName("_format_sort_fields")] IReadOnlyList FormatSortFields, - [property: JsonPropertyName("album")] string Album, - [property: JsonPropertyName("artist")] string Artist, - [property: JsonPropertyName("track")] string Track, - [property: JsonPropertyName("comment_count")] int? CommentCount, - [property: JsonPropertyName("like_count")] int? LikeCount, - [property: JsonPropertyName("channel")] string Channel, - [property: JsonPropertyName("channel_follower_count")] int? ChannelFollowerCount, - [property: JsonPropertyName("channel_is_verified")] bool? ChannelIsVerified, - [property: JsonPropertyName("uploader")] string Uploader, - [property: JsonPropertyName("uploader_id")] string UploaderId, - [property: JsonPropertyName("uploader_url")] string UploaderUrl, - [property: JsonPropertyName("upload_date")] string UploadDate, - [property: JsonPropertyName("creator")] string Creator, - [property: JsonPropertyName("alt_title")] string AltTitle, - [property: JsonPropertyName("availability")] string Availability, - [property: JsonPropertyName("webpage_url_basename")] string WebpageUrlBasename, - [property: JsonPropertyName("webpage_url_domain")] string WebpageUrlDomain, - [property: JsonPropertyName("extractor")] string Extractor, - [property: JsonPropertyName("extractor_key")] string ExtractorKey, - [property: JsonPropertyName("playlist_count")] int? PlaylistCount, - [property: JsonPropertyName("playlist")] string Playlist, - [property: JsonPropertyName("playlist_id")] string PlaylistId, - [property: JsonPropertyName("playlist_title")] string PlaylistTitle, - [property: JsonPropertyName("n_entries")] int? NEntries, - [property: JsonPropertyName("playlist_index")] uint? PlaylistIndex, - [property: JsonPropertyName("display_id")] string DisplayId, - [property: JsonPropertyName("fulltitle")] string Fulltitle, - [property: JsonPropertyName("duration_string")] string DurationString, - [property: JsonPropertyName("release_date")] string ReleaseDate, - [property: JsonPropertyName("release_year")] uint? ReleaseYear, - [property: JsonPropertyName("is_live")] bool? IsLive, - [property: JsonPropertyName("was_live")] bool? WasLive, - [property: JsonPropertyName("epoch")] int? Epoch, - [property: JsonPropertyName("asr")] int? Asr, - [property: JsonPropertyName("filesize")] int? Filesize, - [property: JsonPropertyName("format_id")] string FormatId, - [property: JsonPropertyName("format_note")] string FormatNote, - [property: JsonPropertyName("source_preference")] int? SourcePreference, - [property: JsonPropertyName("audio_channels")] int? AudioChannels, - [property: JsonPropertyName("quality")] double? Quality, - [property: JsonPropertyName("has_drm")] bool? HasDrm, - [property: JsonPropertyName("tbr")] double? Tbr, - [property: JsonPropertyName("url")] string Url, - [property: JsonPropertyName("language_preference")] int? LanguagePreference, - [property: JsonPropertyName("ext")] string Ext, - [property: JsonPropertyName("vcodec")] string Vcodec, - [property: JsonPropertyName("acodec")] string Acodec, - [property: JsonPropertyName("container")] string Container, - [property: JsonPropertyName("protocol")] string Protocol, - [property: JsonPropertyName("resolution")] string Resolution, - [property: JsonPropertyName("audio_ext")] string AudioExt, - [property: JsonPropertyName("video_ext")] string VideoExt, - [property: JsonPropertyName("vbr")] int? Vbr, - [property: JsonPropertyName("abr")] double? Abr, - [property: JsonPropertyName("format")] string Format, - [property: JsonPropertyName("_type")] string Type -); diff --git a/src/CCVTAC.Console/PostProcessing/YouTubeMetadataExtensionMethods.cs b/src/CCVTAC.Console/PostProcessing/YouTubeMetadataExtensionMethods.cs deleted file mode 100644 index 2b915aff..00000000 --- a/src/CCVTAC.Console/PostProcessing/YouTubeMetadataExtensionMethods.cs +++ /dev/null @@ -1,84 +0,0 @@ -namespace CCVTAC.Console.PostProcessing; - -public static class YouTubeMetadataExtensionMethods -{ - extension(VideoMetadata videoData) - { - /// - /// Returns a string summarizing video uploader information. - /// - private string UploaderSummary() - { - string uploaderLinkOrIdOrEmpty = - videoData.UploaderUrl.HasText() ? videoData.UploaderUrl - : videoData.UploaderId.HasText() ? videoData.UploaderId - : string.Empty; - - return videoData.Uploader - + (uploaderLinkOrIdOrEmpty.HasText() ? $" ({uploaderLinkOrIdOrEmpty})" : string.Empty); - } - - /// - /// Returns a formatted MM/DD/YYYY version of the upload date (e.g., "08/27/2023") from the - /// plain YYYYMMDD version (e.g., "20230827") within the parsed JSON file data. - /// - private string FormattedUploadDate() - { - return $"{videoData.UploadDate[4..6]}/{videoData.UploadDate[6..8]}/{videoData.UploadDate[..4]}"; - } - - /// - /// Returns a formatted comment using data parsed from the JSON file. - /// - public string GenerateComment(CollectionMetadata? maybeCollectionData - ) - { - System.Text.StringBuilder sb = new(); - - sb.AppendLine("CCVTAC SOURCE DATA:"); - sb.AppendLine($"■ Downloaded: {DateTime.Now}"); - // sb.AppendLine($"■ Service: {videoData.ExtractorKey}"); // "Youtube" - sb.AppendLine($"■ URL: {videoData.WebpageUrl}"); - sb.AppendLine($"■ Title: {videoData.Fulltitle}"); - sb.AppendLine($"■ Uploader: {videoData.UploaderSummary()}"); - if (videoData.Creator != videoData.Uploader && videoData.Creator.HasText()) - { - sb.AppendLine($"■ Creator: {videoData.Creator}"); - } - if (videoData.Artist.HasText()) - { - sb.AppendLine($"■ Artist: {videoData.Artist}"); - } - if (videoData.Album.HasText()) - { - sb.AppendLine($"■ Album: {videoData.Album}"); - } - if (videoData.Title.HasText() && videoData.Title != videoData.Fulltitle) - { - sb.AppendLine($"■ Track Title: {videoData.Title}"); - } - sb.AppendLine($"■ Uploaded: {videoData.FormattedUploadDate()}"); - var description = string.IsNullOrWhiteSpace(videoData.Description) - ? "None." - : videoData.Description; - sb.AppendLine($"■ Video description: {description}"); - - if (maybeCollectionData is { } collectionData) - { - sb.AppendLine(); - sb.AppendLine($"■ Playlist name: {collectionData.Title}"); - sb.AppendLine($"■ Playlist URL: {collectionData.WebpageUrl}"); - if (videoData.PlaylistIndex is { } index) - { - sb.AppendLine($"■ Playlist index: {index}"); - } - if (collectionData.Description.HasText()) - { - sb.AppendLine($"■ Playlist description: {collectionData.Description}"); - } - } - - return sb.ToString(); - } - } -} diff --git a/src/CCVTAC.Console/Printer.cs b/src/CCVTAC.Console/Printer.cs deleted file mode 100644 index 085e9dd2..00000000 --- a/src/CCVTAC.Console/Printer.cs +++ /dev/null @@ -1,227 +0,0 @@ -using Spectre.Console; - -namespace CCVTAC.Console; - -public sealed class Printer -{ - private enum Level - { - Critical, - Error, - Warning, - Info, - Debug, - } - - private record ColorFormat(string? Foreground, string? Background, bool Bold = false); - - /// - /// Color reference: https://spectreconsole.net/appendix/colors - /// - private static readonly Dictionary Colors = new() - { - { Level.Critical, new("white", "red3", true) }, - { Level.Error, new("red", null) }, - { Level.Warning, new("yellow", null) }, - { Level.Info, new(null, null) }, - { Level.Debug, new("grey70", null) }, - }; - - private Level MinimumLogLevel { get; set; } - - public Printer(bool showDebug) - { - MinimumLogLevel = showDebug ? Level.Debug : Level.Info; - } - - public void ShowDebug(bool show) - { - MinimumLogLevel = show ? Level.Debug : Level.Info; - } - - /// - /// - /// - private static string EscapeText(string text) => - text.Replace("{", "{{").Replace("}", "}}").Replace("[", "[[").Replace("]", "]]"); - - private static string AddMarkup(string message, ColorFormat colors) - { - if (colors.Foreground is null && colors.Background is null && !colors.Bold) - { - return message; - } - - var bold = colors.Bold ? "bold " : string.Empty; - var fg = colors.Foreground ?? "default"; - var bg = colors.Background is null ? string.Empty : $" on {colors.Background}"; - var markUp = $"{bold}{fg}{bg}"; - - return $"[{markUp}]{message}[/]"; - } - - private void Print( - Level logLevel, - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - if (logLevel > MinimumLogLevel) - { - return; - } - - if (string.IsNullOrWhiteSpace(message)) - { - throw new ArgumentNullException(nameof(message), "Message cannot be empty."); - } - - EmptyLines(prependLines); - - var escapedMessage = EscapeText(message); - if (processMarkup) - { - var markedUpMessage = AddMarkup(escapedMessage, Colors[logLevel]); - AnsiConsole.Markup(markedUpMessage); - } - else - { - // `AnsiConsole.Write()` calls an internal function that uses format strings, - // so we must duplicate any curly brackets to safely escape the message text. - // See https://github.com/spectreconsole/spectre.console/issues/1495. - AnsiConsole.Write(escapedMessage); - } - - if (appendLineBreak) - { - AnsiConsole.WriteLine(); - } - - EmptyLines(appendLines); - } - - public static void PrintTable(Table table) - { - AnsiConsole.Write(table); - } - - public void Critical( - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - Print(Level.Critical, message, appendLineBreak, prependLines, appendLines, processMarkup); - } - - public void Error( - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - Print(Level.Error, message, appendLineBreak, prependLines, appendLines, processMarkup); - } - - public void Errors(ICollection errors, byte appendLines = 0) - { - if (errors.Count == 0) - throw new ArgumentException("No errors were provided!", nameof(errors)); - - foreach (var error in errors.Where(e => e.HasText())) - { - Error(error); - } - - EmptyLines(appendLines); - } - - private void Errors(string headerMessage, IEnumerable errors) - { - Errors([headerMessage, .. errors]); - } - - public void Errors(Result failResult, byte appendLines = 0) - { - Errors(failResult.Errors.Select(e => e.Message).ToList(), appendLines); - } - - public void Errors(string headerMessage, Result failingResult) - { - Errors(headerMessage, failingResult.Errors.Select(e => e.Message)); - } - - public void FirstError(IResultBase failResult, string? prepend = null) - { - string pre = prepend is null ? string.Empty : $"{prepend} "; - string message = failResult.Errors?.FirstOrDefault()?.Message ?? string.Empty; - - Error($"{pre}{message}"); - } - - public void Warning( - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - Print(Level.Warning, message, appendLineBreak, prependLines, appendLines, processMarkup); - } - - public void Info( - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - Print(Level.Info, message, appendLineBreak, prependLines, appendLines, processMarkup); - } - - public void Debug( - string message, - bool appendLineBreak = true, - byte prependLines = 0, - byte appendLines = 0, - bool processMarkup = true - ) - { - Print(Level.Debug, message, appendLineBreak, prependLines, appendLines, processMarkup); - } - - /// - /// Prints the requested number of blank lines. - /// - /// - public static void EmptyLines(byte count) - { - if (count == 0) - return; - - AnsiConsole.WriteLine(string.Concat(Enumerable.Repeat(Environment.NewLine, count - 1))); - } - - public string GetInput(string prompt) - { - EmptyLines(1); - return AnsiConsole.Ask($"[skyblue1]{prompt}[/]"); - } - - private static string Ask(string title, string[] options) - { - return AnsiConsole.Prompt(new SelectionPrompt().Title(title).AddChoices(options)); - } - - public bool AskToBool(string title, string trueAnswer, string falseAnswer) => - Ask(title, [trueAnswer, falseAnswer]) == trueAnswer; -} diff --git a/src/CCVTAC.Console/Program.cs b/src/CCVTAC.Console/Program.cs deleted file mode 100644 index d7bfb1e1..00000000 --- a/src/CCVTAC.Console/Program.cs +++ /dev/null @@ -1,83 +0,0 @@ -using CCVTAC.Console.IoUtilities; -using CCVTAC.Console.Settings; -using Spectre.Console; - -namespace CCVTAC.Console; - -internal static class Program -{ - private static readonly string[] HelpFlags = ["-h", "--help"]; - private static readonly string[] SettingsFileFlags = ["-s", "--settings"]; - private const string DefaultSettingsFileName = "settings.json"; - - private static void Main(string[] args) - { - Printer printer = new(showDebug: true); - - if (args.Length > 0 && HelpFlags.CaseInsensitiveContains(args[0])) - { - Help.Print(printer); - return; - } - - string maybeSettingsPath = - args.Length >= 2 && SettingsFileFlags.CaseInsensitiveContains(args[0]) - ? args[1] // Expected to be a settings file path. - : DefaultSettingsFileName; - - var settingsResult = SettingsAdapter.ProcessSettings(maybeSettingsPath, printer); - if (settingsResult.IsFailed) - { - printer.Errors(settingsResult.Errors.Select(e => e.Message).ToList()); - return; - } - if (settingsResult.Value is null) // If a new settings file was created. - { - return; - } - - var settings = settingsResult.Value; - SettingsAdapter.PrintSummary(settings, printer, header: "Settings loaded OK."); - - printer.ShowDebug(!settings.QuietMode); - - // Catch Ctrl-C (SIGINT). - System.Console.CancelKeyPress += delegate - { - printer.Warning("\nQuitting at user's request."); - - var warnResult = Directories.WarnIfAnyFiles(settings.WorkingDirectory, 10); - - if (warnResult.IsSuccess) - { - return; - } - - printer.FirstError(warnResult); - - var deleteResult = Directories.AskToDeleteAllFiles(settings.WorkingDirectory, printer); - if (deleteResult.IsSuccess) - { - printer.Info($"{deleteResult.Value} file(s) deleted."); - } - else - { - printer.FirstError(deleteResult); - } - }; - - // Top-level `try` block to catch and pretty-print unexpected exceptions. - try - { - Orchestrator.Start(settings, printer); - } - catch (Exception topException) - { - printer.Critical($"Fatal error: {topException.Message}"); - AnsiConsole.WriteException(topException); - printer.Info( - "Please help improve this tool by reporting this error and any relevant URLs at https://github.com/codeconscious/ccvtac/issues." - ); - } - } -} diff --git a/src/CCVTAC.Console/ResultTracker.cs b/src/CCVTAC.Console/ResultTracker.cs deleted file mode 100644 index 758083dc..00000000 --- a/src/CCVTAC.Console/ResultTracker.cs +++ /dev/null @@ -1,79 +0,0 @@ -namespace CCVTAC.Console; - -/// -/// Tracks the successes and failures of various operations. -/// Successes are only counted; failure error messages are tracked. -/// -internal sealed class ResultTracker -{ - private nuint _successCount; - private readonly Dictionary _failures = []; - private readonly Printer _printer; - - private static string CombineErrors(Result result) => - string.Join(" / ", result.Errors.Select(e => e.Message)); - - internal ResultTracker(Printer printer) - { - ArgumentNullException.ThrowIfNull(printer); - _printer = printer; - } - - /// - /// Logs the result for a specific corresponding input. - /// - internal void RegisterResult(string input, Result result) - { - if (result.IsSuccess) - { - _successCount++; - return; - } - - string errors = CombineErrors(result); - if (!_failures.TryAdd(input, errors)) - { - // Keep only the latest error for each input. - _failures[input] = errors; - } - } - - /// - /// Prints any failures for the current batch. - /// - internal void PrintBatchFailures() - { - if (_failures.Count == 0) - { - _printer.Debug("No failures in batch."); - return; - } - - var failureLabel = _failures.Count == 1 ? "failure" : "failures"; - _printer.Info($"{_failures.Count} {failureLabel} in this batch:"); - - foreach (var (url, error) in _failures) - { - _printer.Warning($"- {url}: {error}"); - } - } - - /// - /// Prints the output for the current application session. - /// Expected to be used upon quitting. - /// - internal void PrintSessionSummary() - { - var successLabel = _successCount == 1 ? "success" : "successes"; - var failureLabel = _failures.Count == 1 ? "failure" : "failures"; - - _printer.Info( - $"Quitting with {_successCount} {successLabel} and {_failures.Count} {failureLabel}." - ); - - foreach (var (url, error) in _failures) - { - _printer.Warning($"- {url}: {error}"); - } - } -} diff --git a/src/CCVTAC.Console/Settings/Id3Version.cs b/src/CCVTAC.Console/Settings/Id3Version.cs deleted file mode 100644 index bf038728..00000000 --- a/src/CCVTAC.Console/Settings/Id3Version.cs +++ /dev/null @@ -1,28 +0,0 @@ -namespace CCVTAC.Console.Settings; - -public sealed class TagFormat -{ - /// - /// Point versions of ID3 version 2 (such as 2.3 or 2.4). - /// - public enum Id3V2Version : byte - { - TwoPoint2 = 2, - TwoPoint3 = 3, - TwoPoint4 = 4, - } - - /// - /// Locks the ID3v2.x version to a valid one and optionally forces that version. - /// - /// The ID3 version 2 subversion to use. - /// - /// When true, forces the specified version when writing the file. - /// When false, will defer to the version within the file, if any. - /// - public static void SetId3V2Version(Id3V2Version version, bool forceAsDefault) - { - TagLib.Id3v2.Tag.DefaultVersion = (byte)version; - TagLib.Id3v2.Tag.ForceDefaultVersion = forceAsDefault; - } -} diff --git a/src/CCVTAC.Console/Settings/SettingsAdapter.cs b/src/CCVTAC.Console/Settings/SettingsAdapter.cs deleted file mode 100644 index 1fd95aa0..00000000 --- a/src/CCVTAC.Console/Settings/SettingsAdapter.cs +++ /dev/null @@ -1,138 +0,0 @@ -using Microsoft.FSharp.Core; -using Spectre.Console; -using UserSettings = CCVTAC.FSharp.Settings.UserSettings; - -namespace CCVTAC.Console.Settings; - -/// -/// Settings are managed by the corresponding F# library. This class acts as -/// the bridge between this project and the F# library. -/// -/// -/// It would be easier to do everything here in C#, but I incorporated F# in -/// PR #38 as functional programming practice. -/// -public static class SettingsAdapter -{ - private const string DefaultFileName = "settings.json"; - - /// - /// Reads settings or creates a new default settings file. - /// - /// - /// - /// - /// A Result indicating three possible conditions: - /// 1. `Ok` with successfully parsed user settings from the disk. - /// 2. `OK` with no value, indicating that a new default file was created. - /// 3. `Fail`, indicating a failure in the read or write process or in settings validation. - /// - /// This is intended to be a temporary solution until more code is moved to F#. - internal static Result ProcessSettings(string? maybeSettingsPath, Printer printer) - { - var path = FSharp.Settings.FilePath.NewFilePath(maybeSettingsPath); - - if (FSharp.Settings.IO.FileExists(path) is { IsOk: true }) - { - try - { - var result = FSharp.Settings.IO.Read(path); - - return result is { IsError: true } - ? Result.Fail($"Settings validation error: {result.ErrorValue}") - : Result.Ok(result.ResultValue); - } - catch (Exception ex) - { - return Result.Fail($"Error reading settings: {ex.Message}"); - } - } - - try - { - var result = FSharp.Settings.IO.WriteDefaultFile(path, DefaultFileName); - if (result is { IsError: true }) - { - return Result.Fail( - $"Unexpected error writing the default settings: {result.ErrorValue}" - ); - } - - printer.Info(result.ResultValue); // The new-file message. - return Result.Ok(); - } - catch (Exception ex) - { - return Result.Fail($"Error writing default settings: {ex.Message}"); - } - } - - /// - /// Returns a new Settings instance with the Split Chapters value toggled. - /// This only affects the current session; the settings file is not updated. - /// - internal static UserSettings ToggleSplitChapters(UserSettings settings) => - FSharp.Settings.LiveUpdating.ToggleSplitChapters(settings); - - /// - /// Returns a new Settings instance with the Embed Images value toggled. - /// This only affects the current session; the settings file is not updated. - /// - internal static UserSettings ToggleEmbedImages(UserSettings settings) => - FSharp.Settings.LiveUpdating.ToggleEmbedImages(settings); - - /// - /// Returns a new Settings instance with the Quiet Mode value toggled. - /// This only affects the current session; the settings file is not updated. - /// - internal static UserSettings ToggleQuietMode(UserSettings settings) => - FSharp.Settings.LiveUpdating.ToggleQuietMode(settings); - - /// - /// Returns a Result containing a new Settings instance with the Audio Format value updated, - /// or else an Error. This only affects the current session; the settings file is not updated. - /// - internal static FSharpResult UpdateAudioFormat( - UserSettings settings, - string newFormat - ) => FSharp.Settings.LiveUpdating.UpdateAudioFormat(settings, newFormat.Split(',')); - - /// - /// Returns a Result containing a new Settings instance with the Audio Quality value updated, - /// or else an Error. This only affects the current session; the settings file is not updated. - /// - internal static FSharpResult UpdateAudioQuality( - UserSettings settings, - byte newQuality - ) => FSharp.Settings.LiveUpdating.UpdateAudioQuality(settings, newQuality); - - /// - /// Prints a summary of the given settings. - /// - /// - /// - /// An optional line of text to appear above the settings. - internal static void PrintSummary(UserSettings settings, Printer printer, string? header = null) - { - if (header.HasText()) - { - printer.Info(header!); - } - - var table = new Table(); - table.Expand(); - table.Border(TableBorder.HeavyEdge); - table.BorderColor(Color.Grey27); - table.AddColumns("Name", "Value"); - table.HideHeaders(); - table.Columns[1].Width = 100; // Ensure maximum width. - - var settingPairs = FSharp.Settings.Summarize(settings); - foreach (var pair in settingPairs) - { - table.AddRow(pair.Item1, pair.Item2); - } - - Printer.PrintTable(table); - } -} diff --git a/src/CCVTAC.Console/Usings.cs b/src/CCVTAC.Console/Usings.cs deleted file mode 100644 index 3de73b43..00000000 --- a/src/CCVTAC.Console/Usings.cs +++ /dev/null @@ -1,7 +0,0 @@ -global using System; -global using System.Collections.Frozen; -global using System.Collections.Generic; -global using System.Collections.Immutable; -global using System.Linq; -global using FluentResults; -global using Startwatch.Library; diff --git a/src/CCVTAC.Console/settings.default.json b/src/CCVTAC.Console/settings.default.json deleted file mode 100644 index d28e1749..00000000 --- a/src/CCVTAC.Console/settings.default.json +++ /dev/null @@ -1,25 +0,0 @@ -{ - "workingDirectory": "", - "moveToDirectory": "", - "historyFile": "", - "historyDisplayCount": 25, - "logDirectory": "", - "audioFormats": ["m4a", "best"], - "audioQuality": 0, - "splitChapters": true, - "embedImages": true, - "quietMode": false, - "sleepSecondsBetweenDownloads": 7, - "sleepSecondsBetweenURLs": 15, - "normalizationForm": "C", - "doNotEmbedImageUploaders": [], - "ignoreUploadYearUploaders": [], - "tagDetectionPatterns": { - "title": [], - "artist": [], - "album": [], - "composer": [], - "year": [] - }, - "renamePatterns": [] -} diff --git a/src/CCVTAC.FSharp/CCVTAC.FSharp.fsproj b/src/CCVTAC.FSharp/CCVTAC.FSharp.fsproj deleted file mode 100644 index 210dd2b4..00000000 --- a/src/CCVTAC.FSharp/CCVTAC.FSharp.fsproj +++ /dev/null @@ -1,11 +0,0 @@ - - - net10.0 - true - true - - - - - - diff --git a/src/CCVTAC.Main/CCVTAC.Main.fsproj b/src/CCVTAC.Main/CCVTAC.Main.fsproj new file mode 100644 index 00000000..5f7431df --- /dev/null +++ b/src/CCVTAC.Main/CCVTAC.Main.fsproj @@ -0,0 +1,50 @@ + + + net10.0 + true + true + enable + Exe + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/CCVTAC.Main/Commands.fs b/src/CCVTAC.Main/Commands.fs new file mode 100644 index 00000000..e55342a9 --- /dev/null +++ b/src/CCVTAC.Main/Commands.fs @@ -0,0 +1,40 @@ +namespace CCVTAC.Main + +open System + +module Commands = + + let prefix = '\\' + + let private toCommand text : string = + if String.hasNoText text then raise (ArgumentException("Commands cannot be null or white space.", "text")) + if text.Contains ' ' then raise (ArgumentException("Commands cannot contain white space.", "text")) + $"%c{prefix}%s{text}" + + let quitCommands = [ toCommand "quit"; toCommand "q"; toCommand "exit" ] + let helpCommand = toCommand "help" + let settingsSummary = [ toCommand "settings" ] + let history = [ toCommand "history" ] + let updateDownloader = [ toCommand "update-downloader"; toCommand "update-dl" ] + let splitChapterToggles = [ toCommand "split"; toCommand "toggle-split" ] + let embedImagesToggles = [ toCommand "images"; toCommand "toggle-images" ] + let quietModeToggles = [ toCommand "quiet"; toCommand "toggle-quiet" ] + let updateAudioFormatPrefix = toCommand "format-" + let updateAudioQualityPrefix = toCommand "quality-" + + let summary: Map = + [ + String.Join(" or ", history), "See the most recently entered URLs" + String.Join(" or ", splitChapterToggles), "Toggles chapter splitting for the current session only" + String.Join(" or ", embedImagesToggles), "Toggles image embedding for the current session only" + String.Join(" or ", quietModeToggles), "Toggles quiet mode for the current session only" + String.Join(" or ", updateDownloader), "Updates the downloader using the command specified in the settings" + (updateAudioFormatPrefix, + $"Followed by a supported audio format (e.g., %s{updateAudioFormatPrefix}m4a), changes the audio format for the current session only") + (updateAudioQualityPrefix, + $"Followed by a supported audio quality (e.g., %s{updateAudioQualityPrefix}0), changes the audio quality for the current session only") + String.Join(" or ", settingsSummary), "See the current settings" + String.Join(" or ", quitCommands), "Quit this application" + helpCommand, "See this help message" + ] + |> Map.ofList diff --git a/src/CCVTAC.Main/Downloading/Downloader.fs b/src/CCVTAC.Main/Downloading/Downloader.fs new file mode 100644 index 00000000..d1c44d31 --- /dev/null +++ b/src/CCVTAC.Main/Downloading/Downloader.fs @@ -0,0 +1,132 @@ +namespace CCVTAC.Main.Downloading + +open CCVTAC.Main +open CCVTAC.Main.ExternalTools.Runner +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.IoUtilities.Directories +open CCVTAC.Main.Downloading.Downloading +open CCVTAC.Main.ExternalTools +open CCVTAC.Main.Settings.Settings +open FsToolkit.ErrorHandling +open System + +module Downloader = + + [] + let private programName = "yt-dlp" + + let generateDownloadArgs audioFormat userSettings (mediaType: MediaType option) additionalArgs : string = + let writeJsonArg = "--write-info-json" + let trimFileNamesArg = "--trim-filenames 250" + + let formatArg = + match audioFormat with + | None -> String.Empty + | Some format when format = "best" -> String.Empty + | Some format -> $"-f {format}" + + let mutable args = + match mediaType with + | None -> + [ $"--flat-playlist {writeJsonArg} {trimFileNamesArg}" ] + | Some _ -> + [ $"--extract-audio {formatArg}" + $"--audio-quality {userSettings.AudioQuality}" + "--write-thumbnail --convert-thumbnails jpg" + writeJsonArg + trimFileNamesArg + "--retries 2" ] + |> Set.ofList + + if userSettings.QuietMode then + args <- args.Add "--quiet --no-warnings" + + match userSettings.DownloaderAdditionalOptions with + | Some x -> args <- args.Add x + | None -> () + + // No MediaType indicates that this is a supplemental metadata-only download. + // TODO: Perhaps add a union type to more clearly indicate this difference. + match mediaType with + | Some mt -> + if userSettings.SplitChapters then + args <- args.Add "--split-chapters" + + if not mt.IsVideo && not mt.IsPlaylistVideo then + args <- args.Add $"--sleep-interval {userSettings.SleepSecondsBetweenDownloads}" + + if mt.IsStandardPlaylist then + args <- args.Add + """-o "%(uploader).80B - %(playlist).80B - %(playlist_autonumber)s - %(title).150B [%(id)s].%(ext)s" --playlist-reverse""" + | None -> () + + let extraArgs = defaultArg additionalArgs [] |> Set.ofList + String.Join(" ", Set.union args extraArgs) + + let wrapUrlInMediaType url : Result = + mediaTypeWithIds url + + let downloadMedia (printer: Printer) (mediaType: MediaType) userSettings (PrimaryUrl url) + : Result = + + if not mediaType.IsVideo && not mediaType.IsPlaylistVideo then + printer.Info("Please wait for multiple videos to be downloaded...") + + let rec loop errors audioFormats = + match audioFormats with + | [] -> + Error errors + | format :: formats -> + let args = generateDownloadArgs (Some format) userSettings (Some mediaType) (Some [url]) + let commandWithArgs = $"{programName} {args}" + let downloadSettings = ToolSettings.create commandWithArgs userSettings.WorkingDirectory + + let downloadResult = runTool downloadSettings [1] printer + let filesDownloaded = audioFileCount userSettings.WorkingDirectory Files.audioFileExtensions > 0 + + match downloadResult, filesDownloaded with + | Ok result, true -> + Ok <| + $"Successfully downloaded the \"{format}\" format." + :: match result.Error with + | Some err -> [$"However, a minor issue was reported: {err}"] + | None -> [] + | Ok result, false -> + Error <| + $"The \"{format}\" format download was reported as successful, but no audio files were downloaded!" + :: match result.Error with + | Some err -> [$"Error: {err}"] + | None -> [] + | Error err, true -> + Error <| + [$"The downloader reported failure for \"{format}\", yet audio files were unexpectedly downloaded!" + $"Error: {err}"] + | Error err, false -> + let newErr = $"A download error was reported for the \"{format}\" format, and no audio files were downloaded. {err}" + loop (List.append errors [newErr]) formats + + loop [] userSettings.AudioFormats + + let downloadMetadata (printer: Printer) userSettings (SupplementaryUrl url) : Result = + match url with + | None -> Ok "No supplementary metadata link found." + | Some url' -> + let args = generateDownloadArgs None userSettings None (Some [url']) + let commandWithArgs = $"{programName} {args}" + let downloadSettings = ToolSettings.create commandWithArgs userSettings.WorkingDirectory + let metadataDownloadResult = runTool downloadSettings [1] printer + + match metadataDownloadResult with + | Ok _ -> Ok "Supplementary metadata download completed OK." + | Error err -> Error [$"Supplementary metadata download failed: {err}"] + + let run (mediaType: MediaType) userSettings (printer: Printer) : Result = + result { + let rawUrls = generateDownloadUrl mediaType + let urls = + { Primary = PrimaryUrl rawUrls[0] + Metadata = SupplementaryUrl <| if rawUrls.Length = 2 then Some rawUrls[1] else None } + let! _ = downloadMedia printer mediaType userSettings urls.Primary + let! metadataDownloadResult = downloadMetadata printer userSettings urls.Metadata + return! Ok metadataDownloadResult + } diff --git a/src/CCVTAC.FSharp/Downloading.fs b/src/CCVTAC.Main/Downloading/Downloading.fs similarity index 58% rename from src/CCVTAC.FSharp/Downloading.fs rename to src/CCVTAC.Main/Downloading/Downloading.fs index 90fd242c..1d8673b4 100644 --- a/src/CCVTAC.FSharp/Downloading.fs +++ b/src/CCVTAC.Main/Downloading/Downloading.fs @@ -1,47 +1,52 @@ -namespace CCVTAC.FSharp - -module public Downloading = - open System.Text.RegularExpressions - - type MediaType = - | Video of Id: string - | PlaylistVideo of VideoId: string * PlaylistId: string - | StandardPlaylist of Id: string - | ReleasePlaylist of Id: string - | Channel of Id: string - - let private (|Regex|_|) pattern input = - match Regex.Match(input, pattern) with - | m when m.Success -> Some (List.tail [for g in m.Groups -> g.Value]) - | _ -> None - - [] - let mediaTypeWithIds url = - match url with - | Regex @"(?<=v=|v\=)([\w-]{11})(?:&list=([\w_-]+))" [videoId; playlistId] -> - Ok (PlaylistVideo (videoId, playlistId)) - | Regex @"^([\w-]{11})$" [id] - | Regex @"(?<=v=|v\\=)([\w-]{11})" [id] - | Regex @"(?<=youtu\.be/)(.{11})" [id] -> - Ok (Video id) - | Regex @"(?<=list=)(P[\w\-]+)" [id] -> - Ok (StandardPlaylist id) - | Regex @"(?<=list=)(O[\w\-]+)" [id] -> - Ok (ReleasePlaylist id) - | Regex @"((?:www\.)?youtube\.com\/(?:channel\/|c\/|user\/|@)(?:[A-Za-z0-9\-@%\/]+))" [ id ] -> - Ok (Channel id) - | _ -> - Error $"Unable to determine media type of URL \"{url}\". (Might it contain invalid characters?)" - - [] - let extractDownloadUrls mediaType = - let fullUrl urlBase id = urlBase + id - let videoUrl = fullUrl "https://www.youtube.com/watch?v=" - let playlistUrl = fullUrl "https://www.youtube.com/playlist?list=" - let channelUrl = fullUrl "https://" // For channels, the domain portion is also matched. - - match mediaType with - | Video id -> [videoUrl id] - | PlaylistVideo (vId, pId) -> [videoUrl vId; playlistUrl pId] - | StandardPlaylist id | ReleasePlaylist id -> [playlistUrl id] - | Channel id -> [channelUrl id] +namespace CCVTAC.Main.Downloading + +open System.Text.RegularExpressions + +module Downloading = + + type MediaType = + | Video of Id: string + | PlaylistVideo of VideoId: string * PlaylistId: string + | StandardPlaylist of Id: string + | ReleasePlaylist of Id: string + | Channel of Id: string + + type PrimaryUrl = PrimaryUrl of string + type SupplementaryUrl = SupplementaryUrl of string option + + type Urls = { Primary: PrimaryUrl + Metadata: SupplementaryUrl } + + let private (|RegexMatch|_|) pattern input = + match Regex.Match(input, pattern) with + | m when m.Success -> Some (List.tail [for g in m.Groups -> g.Value]) + | _ -> None + + let mediaTypeWithIds url : Result = + match url with + | RegexMatch @"(?<=v=|v\=)([\w-]{11})(?:&list=([\w_-]+))" [videoId; playlistId] -> + Ok (PlaylistVideo (videoId, playlistId)) + | RegexMatch @"^([\w-]{11})$" [id] + | RegexMatch @"(?<=v=|v\\=)([\w-]{11})" [id] + | RegexMatch @"(?<=youtu\.be/)(.{11})" [id] -> + Ok (Video id) + | RegexMatch @"(?<=list=)(P[\w\-]+)" [id] -> + Ok (StandardPlaylist id) + | RegexMatch @"(?<=list=)(O[\w\-]+)" [id] -> + Ok (ReleasePlaylist id) + | RegexMatch @"((?:www\.)?youtube\.com\/(?:channel\/|c\/|user\/|@)(?:[A-Za-z0-9\-@%\/]+))" [id] -> + Ok (Channel id) + | _ -> + Error $"Unable to determine media type of URL \"{url}\". (Might it contain invalid characters?)" + + let generateDownloadUrl mediaType = + let fullUrl urlBase id = urlBase + id + let videoUrl = fullUrl "https://www.youtube.com/watch?v=" + let playlistUrl = fullUrl "https://www.youtube.com/playlist?list=" + let channelUrl = fullUrl "https://" // For channels, the domain portion is also matched. + + match mediaType with + | Video id -> [videoUrl id] + | PlaylistVideo (vId, pId) -> [videoUrl vId; playlistUrl pId] + | StandardPlaylist id | ReleasePlaylist id -> [playlistUrl id] + | Channel id -> [channelUrl id] diff --git a/src/CCVTAC.Main/Downloading/Updater.fs b/src/CCVTAC.Main/Downloading/Updater.fs new file mode 100644 index 00000000..596b1792 --- /dev/null +++ b/src/CCVTAC.Main/Downloading/Updater.fs @@ -0,0 +1,29 @@ +namespace CCVTAC.Main.Downloading + +open CCVTAC.Main.ExternalTools +open CCVTAC.Main +open CCVTAC.Main.Settings.Settings + +module Updater = + + let run userSettings (printer: Printer) : Result = + if String.hasNoText userSettings.DownloaderUpdateCommand then + printer.Info("No downloader update command provided, so will skip.") + Ok() + else + let toolSettings = ToolSettings.create userSettings.DownloaderUpdateCommand userSettings.WorkingDirectory + + match Runner.runTool toolSettings [] printer with + | Ok result -> + if result.ExitCode <> 0 then + printer.Warning("Tool updated with minor issues.") + + match result.Error with + | Some w -> printer.Warning w + | None -> () + + Ok() + + | Error err -> + printer.Error $"Failure updating: {err}" + Error err diff --git a/src/CCVTAC.Main/Extensions.fs b/src/CCVTAC.Main/Extensions.fs new file mode 100644 index 00000000..77128327 --- /dev/null +++ b/src/CCVTAC.Main/Extensions.fs @@ -0,0 +1,148 @@ +namespace CCVTAC.Main + +open System +open System.Globalization +open System.IO +open System.Text + +type SB = StringBuilder + +module Numerics = + + let inline isZero (n: ^a) = + n = LanguagePrimitives.GenericZero<'a> + + let inline isOne (n: ^a) = + n = LanguagePrimitives.GenericOne<'a> + + /// Formats a number of any type to a comma-formatted string. + let inline formatNumber (i: ^T) : string + when ^T : (member ToString : string * IFormatProvider -> string) = + (^T : (member ToString : string * IFormatProvider -> string) (i, "#,##0", CultureInfo.InvariantCulture)) + +module String = + + let newLine = Environment.NewLine + + let hasNoText text = + String.IsNullOrWhiteSpace text + + let hasText text = + not (hasNoText text) + + let allHaveText xs = + xs |> List.forall hasText + + let textOrFallback fallback text = + if hasText text then text else fallback + + let textOrEmpty text = + textOrFallback text String.Empty + + let equalIgnoringCase x y = + String.Equals(x, y, StringComparison.OrdinalIgnoreCase) + + let startsWithIgnoreCase startText (text: string) = + text.StartsWith(startText, StringComparison.InvariantCultureIgnoreCase) + + let endsWithIgnoreCase endText (text: string) = + text.EndsWith(endText, StringComparison.InvariantCultureIgnoreCase) + + /// Pluralize text using a specified count. + let inline pluralize ifOne ifNotOne count = + if Numerics.isOne count then ifOne else ifNotOne + + /// Pluralize text including its count, such as "1 file", "30 URLs". + let inline pluralizeWithCount ifOne ifNotOne count = + sprintf "%d %s" count (pluralize ifOne ifNotOne count) + + let inline private fileLabeller descriptor (count: int) = + match descriptor with + | None -> $"""%s{Numerics.formatNumber count} %s{pluralize "file" "files" count}""" + | Some d -> $"""%s{Numerics.formatNumber count} %s{d} {pluralize "file" "files" count}""" + + /// Returns a file-count string, such as "0 files" or 1 file" or "140 files". + let fileLabel count = + fileLabeller None count + + /// Returns a file-count string with a descriptor, such as "0 audio files" or "140 deleted files". + let fileLabelWithDescriptor (descriptor: string) count = + fileLabeller (Some (descriptor.Trim())) count + + /// Returns a new string in which all invalid path characters for the current OS + /// have been replaced by the specified replacement character. + /// Throws if the replacement character is an invalid path character. + let replaceInvalidPathChars + (replaceWith: char option) + (customInvalidChars: char list option) + (text: string) + : string = + + let replaceWith = defaultArg replaceWith '_' + let custom = defaultArg customInvalidChars [] + + let invalidChars = + seq { + yield! Path.GetInvalidFileNameChars() + yield! Path.GetInvalidPathChars() + yield Path.PathSeparator + yield Path.DirectorySeparatorChar + yield Path.AltDirectorySeparatorChar + yield Path.VolumeSeparatorChar + yield! custom + } + |> Set.ofSeq + + if invalidChars |> Set.contains replaceWith then + invalidArg "replaceWith" $"The replacement char ('%c{replaceWith}') must be a valid path character." + + Set.fold + (fun (sb: SB) ch -> sb.Replace(ch, replaceWith)) + (SB text) + invalidChars + |> _.ToString() + + let trimTerminalLineBreak (text: string) = + text.TrimEnd(newLine.ToCharArray()) + +[] +module Seq = + + let isNotEmpty seq = not (Seq.isEmpty seq) + + let doesNotContain x seq = not <| Seq.contains x seq + + let hasOne seq = seq |> Seq.length |> Numerics.isOne + + let hasMultiple seq = seq |> Seq.length |> (<) 1 + + let caseInsensitiveContains text (xs: string seq) : bool = + xs |> Seq.exists (fun x -> String.Equals(x, text, StringComparison.OrdinalIgnoreCase)) + +[] +module List = + + let isNotEmpty lst = not (List.isEmpty lst) + + let doesNotContain x lst = not <| List.contains x lst + + let hasOne lst = lst |> List.length |> Numerics.isOne + + let hasMultiple lst = lst |> List.length |> (<) 1 + + let caseInsensitiveContains text (lst: string list) : bool = + lst |> List.exists (fun x -> String.Equals(x, text, StringComparison.OrdinalIgnoreCase)) + +[] +module Array = + + let isNotEmpty arr = not <| Array.isEmpty arr + + let doesNotContain x arr = not <| Array.contains x arr + + let hasOne arr = arr |> Array.length |> Numerics.isOne + + let hasMultiple arr = arr |> Array.length |> (<) 1 + + let caseInsensitiveContains text (arr: string array) : bool = + arr |> Array.exists (fun x -> String.Equals(x, text, StringComparison.OrdinalIgnoreCase)) diff --git a/src/CCVTAC.Main/ExternalTools/ExternalTool.fs b/src/CCVTAC.Main/ExternalTools/ExternalTool.fs new file mode 100644 index 00000000..aa4a2353 --- /dev/null +++ b/src/CCVTAC.Main/ExternalTools/ExternalTool.fs @@ -0,0 +1,40 @@ +namespace CCVTAC.Main.ExternalTools + +open System.Diagnostics + +type ExternalTool = private { + Name: string + Url: string + Purpose: string +} + +module ExternalTool = + /// Creates a new ExternalTool instance + /// The name of the program. This should be the exact text used to call it + /// on the command line, excluding any arguments. + /// The URL of the program, from which users should install it if needed. + /// A brief summary of the purpose of the program within the context of this application. + /// Should be phrased as a noun (e.g., "image processing" or "audio normalization"). + let create (name: string) (url: string) (purpose: string) : ExternalTool = + { Name = name.Trim() + Url = url.Trim() + Purpose = purpose.Trim() } + + /// Attempts a dry run of the program to determine if it is installed and available on this system. + let programExists name : Result = + let processStartInfo = ProcessStartInfo( + FileName = name, + UseShellExecute = false, + RedirectStandardOutput = true, + RedirectStandardError = true, + CreateNoWindow = true) + + try + match Process.Start processStartInfo with + | Null -> + Error $"The program \"{name}\" was not found (i.e., the process was null)." + | NonNull process' -> + process'.WaitForExit() + Ok() + with + | exn -> Error $"The program \"{name}\" was not found or could not be run: {exn.Message}." diff --git a/src/CCVTAC.Main/ExternalTools/Runner.fs b/src/CCVTAC.Main/ExternalTools/Runner.fs new file mode 100644 index 00000000..5fd2a1a4 --- /dev/null +++ b/src/CCVTAC.Main/ExternalTools/Runner.fs @@ -0,0 +1,53 @@ +namespace CCVTAC.Main.ExternalTools + +open CCVTAC.Main +open Startwatch.Library +open System +open System.Diagnostics + +module Runner = + + type ToolResult = { ExitCode: int; Error: string option } + + [] + let private authenticSuccessExitCode = 0 + + let private isSuccessExitCode (otherSuccessExitCodes: int list) (exitCode: int) = + List.contains exitCode (authenticSuccessExitCode :: otherSuccessExitCodes) + + /// Calls an external application. + /// Tool settings for execution + /// Additional exit codes, other than 0, that can be treated as non-failures + /// + let runTool toolSettings otherSuccessExitCodes (printer: Printer) : Result = + let watch = Watch() + printer.Info $"Running {toolSettings.CommandWithArgs}..." + + let command = + toolSettings.CommandWithArgs.Split([|' '|], 2) + |> fun arr -> {| Tool = arr[0] + Args = if Array.hasMultiple arr then arr[1] else String.Empty |} + + let processStartInfo = ProcessStartInfo command.Tool + processStartInfo.Arguments <- command.Args + processStartInfo.RedirectStandardOutput <- false + processStartInfo.RedirectStandardError <- true + processStartInfo.CreateNoWindow <- true + processStartInfo.WorkingDirectory <- toolSettings.WorkingDirectory + + match Process.Start processStartInfo with + | Null -> + Error $"Could not locate or start {command.Tool}." + | NonNull process' -> + let error = process'.StandardError.ReadToEnd() + + process'.WaitForExit() + printer.Info $"{command.Tool} finished in {watch.ElapsedFriendly}." + + let trimmedError = if String.hasText error + then Some (String.trimTerminalLineBreak error) + else None + + if isSuccessExitCode otherSuccessExitCodes process'.ExitCode + then Ok { ExitCode = process'.ExitCode; Error = trimmedError } + else Error $"{command.Tool} exited with code {process'.ExitCode}: {trimmedError}." diff --git a/src/CCVTAC.Main/ExternalTools/ToolSettings.fs b/src/CCVTAC.Main/ExternalTools/ToolSettings.fs new file mode 100644 index 00000000..208c5f3c --- /dev/null +++ b/src/CCVTAC.Main/ExternalTools/ToolSettings.fs @@ -0,0 +1,13 @@ +namespace CCVTAC.Main.ExternalTools + +/// Settings to govern the behavior of an external program. +type ToolSettings = private { + CommandWithArgs: string + WorkingDirectory: string +} + +module ToolSettings = + let create commandWithArgs workingDirectory = + { CommandWithArgs = commandWithArgs + WorkingDirectory = workingDirectory } + diff --git a/src/CCVTAC.Main/Help.fs b/src/CCVTAC.Main/Help.fs new file mode 100644 index 00000000..6e99b617 --- /dev/null +++ b/src/CCVTAC.Main/Help.fs @@ -0,0 +1,67 @@ +namespace CCVTAC.Main + +module Help = + + let helpText = """ + CCVTAC (CodeConscious Video-to-Audio Converter) is a small .NET-powered CLI + tool that acts as a wrapper around yt-dlp (https://github.com/yt-dlp/yt-dlp) + to enable easier downloads of audio from YouTube videos, playlists, and + channels, plus do some automatic post-processing (tagging, renaming, and + moving) too. + + Feel free to use it yourself, but please note that it's geared to my personal + use case and that no warranties or guarantees are provided. + + PREREQUISITES + + • .NET 10 runtime (https://dotnet.microsoft.com/en-us/download/dotnet/10.0) + • yt-dlp (https://github.com/yt-dlp/yt-dlp) + • ffmpeg (https://ffmpeg.org/) for yt-dlp artwork extraction + • Optional: mogrify (https://imagemagick.org/script/mogrify.php) + for auto-trimming album art + + RUNNING IT + + Settings: + + A valid settings file is mandatory to use this application. + + The application will look for a file named `settings.json` in its directory. + However, you can manually specify an existing file path using the `-s` + option, such as `dotnet run -- -s `. + + Note: The `--` is necessary to indicate that the command and arguments are + for this program and not for `dotnet`. + + If your `settings.json` file does not exist, a default file will be created. + At minimum, you must enter (1) an existing directory for temporary working files, + (2) an existing directory to which the final audio files should be moved, and + (3) a path to your history file. The other settings have sensible defaults. + + See the README file on the GitHub repo for more about settings. + + Using the application: + + Once your settings are ready, run the application with `dotnet run`. + Alternatively, pass `-h` or `--help` for instructions (e.g., + `dotnet run -- --help`). + + When the application is running, enter at least one YouTube media URL (video, + playlist, or channel) at the prompt and press Enter. No spaces between + items are necessary. + + You can also enter the following commands: + - "\help" to see this list of commands + - "\quit" or "\q" to quit + - "\history" to see the last few URLs you entered + - "\update-downloader" or "\update-dl" to update yt-dlp using the command in your settings + (If you start experiencing constant download errors, try this command!) + - Modify the current session only (without updating the settings file): + - `\split` toggles chapter splitting + - `\images` toggles image embedding + - `\quiet` toggles quiet mode + - `\format-` followed by a supported audio format (e.g., `\format-m4a`) changes the format + - `\quality-` followed by a supported audio quality (e.g., `\quality-0`) changes the audio quality + + Enter `\commands` in the application to see this summary. + """ diff --git a/src/CCVTAC.Main/History.fs b/src/CCVTAC.Main/History.fs new file mode 100644 index 00000000..80b084d1 --- /dev/null +++ b/src/CCVTAC.Main/History.fs @@ -0,0 +1,62 @@ +namespace CCVTAC.Main + +open CCVTAC.Main.IoUtilities.Files +open System +open System.IO +open System.Text.Json +open Spectre.Console + +type History(filePath: string, displayCount: int) = + + let separator = ';' + + member private _.FilePath = filePath + member private _.DisplayCount = displayCount + + /// Write a URL and its related data to the history file. + member this.Append(url: string, entryTime: DateTime, printer: Printer) : unit = + try + let serializedTime = JsonSerializer.Serialize(entryTime).Replace("\"", "") + let text = serializedTime + string separator + url + String.newLine + + match appendToFile this.FilePath text with + | Ok _ -> printer.Debug $"Added \"%s{url}\" to the history log." + | Error err -> printer.Error $"Failed to write \"%s{url}\" to the history log at \"{this.FilePath}\": {err}" + with exn -> + printer.Error $"Could not append URL(s) to history log: {exn.Message}" + + member this.ShowRecent(printer: Printer) : unit = + try + let lines = + File.ReadAllLines this.FilePath + |> Seq.rev + |> Seq.truncate this.DisplayCount + |> Seq.rev + |> Seq.toList + + let historyData = + lines + |> Seq.choose (fun line -> + match line.Split separator with + | [| dateText; url |] -> + match DateTime.TryParse dateText with + | true, date -> Some (date, url) + | _ -> None + | _ -> None) + |> Seq.groupBy fst + |> Seq.map (fun (dt, pairs) -> dt, pairs |> Seq.map snd |> Seq.toList) + + // TODO: These presentation matters shouldn't be here. + let table = Table() + table.Border <- TableBorder.None + table.AddColumns("Time", "URL") |> ignore + table.Columns[0].PadRight(3) |> ignore + + for dateTime, urls in historyData do + let formattedTime = sprintf "%s" (dateTime.ToString("yyyy-MM-dd HH:mm:ss")) + let joinedUrls = String.Join(String.newLine, urls) + table.AddRow(formattedTime, joinedUrls) |> ignore + + Printer.PrintTable table + with exn -> + printer.Error $"Could not display history: %s{exn.Message}" diff --git a/src/CCVTAC.Main/InputHelper.fs b/src/CCVTAC.Main/InputHelper.fs new file mode 100644 index 00000000..85497375 --- /dev/null +++ b/src/CCVTAC.Main/InputHelper.fs @@ -0,0 +1,64 @@ +namespace CCVTAC.Main + +open System.Text.RegularExpressions + +module InputHelper = + + let prompt = $"Enter one or more YouTube media URLs or commands (or \"%s{Commands.helpCommand}\"):\n▶︎" + + /// A regular expression that detects the beginnings of URLs and application commands in input strings. + let private userInputRegex = Regex(@"(?:https:|\\)", RegexOptions.Compiled) + + type private IndexPair = { Start: int; End: int } + + type InputCategory = Url | Command + + type CategorizedInput = { Text: string; Category: InputCategory } + + type CategoryCounts (counts: Map) = + member _.Item + with get (category: InputCategory) = + match counts.TryGetValue category with + | true, v -> v + | _ -> 0 + + /// Takes a user input string and splits it into a collection of inputs. + let splitInputText input : string list = + let matches = userInputRegex.Matches input |> Seq.cast |> Seq.toList + + match matches with + | [] -> [] + | [_] -> [input] + | _ -> + let startIndices = matches |> List.map _.Index + + let indexPairs : IndexPair list = + startIndices + |> List.mapi (fun idx startIndex -> + let endIndex = + if idx = startIndices.Length - 1 + then input.Length + else startIndices[idx + 1] + { Start = startIndex + End = endIndex }) + + indexPairs + |> List.map (fun p -> input[p.Start..(p.End - 1)].Trim()) + |> List.distinct + + let categorizeInputs inputs : CategorizedInput list = + inputs + |> List.map (fun input -> + { Text = input + Category = if input.StartsWith(string Commands.prefix) + then InputCategory.Command + else InputCategory.Url }) + + let countCategories (inputs: CategorizedInput list) : CategoryCounts = + let counts = + inputs + |> List.groupBy _.Category + |> List.map (fun (k, grp) -> k, grp |> Seq.length) + |> Map.ofSeq + + CategoryCounts counts diff --git a/src/CCVTAC.Main/IoUtilities/Directories.fs b/src/CCVTAC.Main/IoUtilities/Directories.fs new file mode 100644 index 00000000..4b342438 --- /dev/null +++ b/src/CCVTAC.Main/IoUtilities/Directories.fs @@ -0,0 +1,97 @@ +namespace CCVTAC.Main.IoUtilities + +open CCVTAC.Main +open System.IO + +module Directories = + + [] + let private allFilesSearchPattern = "*" + + /// Counts the number of audio files in a directory. + let audioFileCount (directory: string) (includedExtensions: string list) = + DirectoryInfo(directory).EnumerateFiles() + |> Seq.filter (fun f -> List.caseInsensitiveContains f.Extension includedExtensions) + |> Seq.length + + /// Returns the filenames in a given directory, optionally ignoring specific filenames. + let private getDirectoryFileNames + (directoryName: string) + (customIgnoreFiles: string seq option) + : Result = + + let ignoreFiles = + customIgnoreFiles + |> Option.defaultValue Seq.empty + |> Seq.distinct + |> Seq.toArray + + ofTry (fun _ -> + Directory.GetFiles(directoryName, allFilesSearchPattern, EnumerationOptions()) + |> Array.filter (fun filePath -> not (ignoreFiles |> Array.exists filePath.EndsWith))) + + let deleteAllFiles workingDirectory + : Result = + + let delete fileNames = + let successes, failures = ResizeArray(), ResizeArray() + + for fileName in fileNames do + try + File.Delete fileName + successes.Add $"• Deleted \"%s{fileName}\"" + with exn -> + failures.Add $"• Error deleting \"%s{fileName}\": %s{exn.Message}" + + { Successes = successes |> Seq.toList |> List.rev + Failures = failures |> Seq.toList |> List.rev } + + match getDirectoryFileNames workingDirectory None with + | Error errMsg -> Error errMsg + | Ok fileNames -> Ok (delete fileNames) + + /// Ask the user to confirm the deletion of files in the specified directory. + let askToDeleteAllFiles dirName (printer: Printer) = + if printer.AskToBool("Delete all temporary files?", "Yes", "No") + then deleteAllFiles dirName + else Error "Will not delete the files." + + let printDeletionResults (printer: Printer) (results: ResultMessageCollection) : unit = + printer.Info $"Deleted %s{String.fileLabel results.Successes.Length}." + results.Successes |> List.iter printer.Debug + + if List.isNotEmpty results.Failures then + printer.Warning $"However, %s{String.fileLabel results.Failures.Length} could not be deleted:" + results.Failures |> List.iter printer.Error + + let warnIfAnyFiles showMax dirName = + match getDirectoryFileNames dirName None with + | Error errMsg -> Error errMsg + | Ok fileNames -> + if Array.isEmpty fileNames then + Ok () + else + SB($"Unexpectedly found {String.fileLabel fileNames.Length} in working directory \"{dirName}\":{String.newLine}") + .AppendLine + (fileNames + |> Array.truncate showMax + |> Array.map (sprintf "• %s") + |> String.concat String.newLine) + |> fun sb -> + if fileNames.Length > showMax + then sb.AppendLine $"... plus {fileNames.Length - showMax} more." + else sb + |> _.AppendLine("This sometimes occurs due to the same video appearing twice in playlists.") + |> _.ToString() + |> Error + + /// Ensures the specified directory exists, including creation of it if necessary. + let ensureDirectoryExists dirName : Result = + try + dirName + |> Path.GetFullPath + |> Directory.CreateDirectory + |> Ok + with exn -> + Error $"Error accessing or creating directory \"%s{dirName}\": %s{exn.Message}" + diff --git a/src/CCVTAC.Main/IoUtilities/Files.fs b/src/CCVTAC.Main/IoUtilities/Files.fs new file mode 100644 index 00000000..d438d207 --- /dev/null +++ b/src/CCVTAC.Main/IoUtilities/Files.fs @@ -0,0 +1,15 @@ +namespace CCVTAC.Main.IoUtilities + +open System.IO +open CCVTAC.Main + +module Files = + + let audioFileExtensions = + [ ".aac"; ".alac"; ".flac"; ".m4a"; ".mp3"; ".ogg"; ".vorbis"; ".opus"; ".wav" ] + + let readAllText (filePath: string) : Result = + ofTry (fun _ -> File.ReadAllText filePath) + + let appendToFile (filePath: string) (text: string) : Result = + ofTry (fun _ -> File.AppendAllText(filePath, text)) diff --git a/src/CCVTAC.Main/Orchestrator.fs b/src/CCVTAC.Main/Orchestrator.fs new file mode 100644 index 00000000..23d10304 --- /dev/null +++ b/src/CCVTAC.Main/Orchestrator.fs @@ -0,0 +1,297 @@ +namespace CCVTAC.Main + +open CCVTAC.Main.Downloading +open CCVTAC.Main.InputHelper +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.PostProcessing +open CCVTAC.Main.Settings +open CCVTAC.Main.Settings.Settings +open CCVTAC.Main.Settings.Settings.LiveUpdating +open Startwatch.Library +open System + +module Orchestrator = + + type NextAction = + | Continue + | QuitAtUserRequest + | QuitDueToErrors + + type BatchResults = + { NextAction: NextAction + UpdatedSettings: UserSettings option } + + let summarizeInput + (categorizedInputs: CategorizedInput list) + (counts: CategoryCounts) + (printer: Printer) + : unit = + + if List.hasMultiple categorizedInputs then + let urlSummary = String.pluralizeWithCount "URL" "URLs" counts[InputCategory.Url] + let cmdSummary = String.pluralizeWithCount "command" "commands" counts[InputCategory.Command] + + printer.Info <| + match counts[InputCategory.Url], counts[InputCategory.Command] with + | u, c when u > 0 && c > 0 -> $"Batch of %s{urlSummary} and %s{cmdSummary} entered:" + | u, _ when u > 0 -> $"Batch of %s{urlSummary} entered:" + | _, c when c > 0 -> $"Batch of %s{cmdSummary} entered:" + | _, _ -> "No URLs or commands were entered!" + + for input in categorizedInputs do + printer.Info $" • %s{input.Text}" + + Printer.EmptyLines 1uy + + let processUrl + (url: string) + (settings: UserSettings) + (resultTracker: ResultTracker) + (history: History) + (urlInputTime: DateTime) + (batchSize: int) + (urlIndex: int) + (printer: Printer) + : Result = + + match Directories.warnIfAnyFiles 10 settings.WorkingDirectory with + | Error err -> + printer.Error err + Ok { NextAction = NextAction.QuitDueToErrors; UpdatedSettings = None } + | Ok () -> + if urlIndex > 1 then // Don't sleep for the first URL. + settings.SleepSecondsBetweenURLs + |> String.pluralize "second" "seconds" + |> fun secondsLabel -> + sleep + (fun seconds -> $"Sleeping for {seconds} {secondsLabel}...") + (fun seconds -> $"Slept for {seconds} {secondsLabel}.") + settings.SleepSecondsBetweenURLs + |> fun msg -> printer.Info($"{String.newLine}{msg}", appendLines = 1uy) + + if batchSize > 1 then + printer.Info $"Processing item %d{urlIndex} of %d{batchSize}..." + + let jobWatch = Watch() + + match Downloading.mediaTypeWithIds url with + | Error e -> + let errorMsg = $"URL parse error: %s{e}" + printer.Error errorMsg + Error errorMsg + | Ok mediaType -> + printer.Info $"%s{mediaType.GetType().Name} URL '%s{url}' detected." + history.Append(url, urlInputTime, printer) + + let downloadResult = Downloader.run mediaType settings printer + resultTracker.RegisterResult(url, downloadResult) + + match downloadResult with + | Error errs -> + errs + |> List.map (sprintf "Media download error: %s") + |> String.concat String.newLine + |> Error + | Ok message -> + printer.Debug "Media download(s) successful!" + if String.hasText message then printer.Info message + PostProcessor.run settings mediaType printer + + let groupClause = + if batchSize > 1 + then $" (item %d{urlIndex} of %d{batchSize})" + else String.Empty + + printer.Info $"Processed '%s{url}'%s{groupClause} in %s{jobWatch.ElapsedFriendly}." + + Ok { NextAction = NextAction.Continue + UpdatedSettings = None } + + let summarizeToggle settingName setting = + sprintf "%s was toggled to %s for this session." settingName (if setting then "ON" else "OFF") + + let summarizeUpdate settingName setting = + sprintf "%s was updated to \"%s\" for this session." settingName setting + + let processCommand + (command: string) + (settings: UserSettings) + (history: History) + (printer: Printer) + : Result = + + let checkCommand = List.caseInsensitiveContains command + + // Help + if String.equalIgnoringCase Commands.helpCommand command then + for kvp in Commands.summary do + printer.Info(kvp.Key) + printer.Info $" %s{kvp.Value}" + Ok { NextAction = NextAction.Continue; UpdatedSettings = None } + + // Quit + elif checkCommand Commands.quitCommands then + Ok { NextAction = NextAction.QuitAtUserRequest; UpdatedSettings = None } + + // History + elif checkCommand Commands.history then + history.ShowRecent printer + Ok { NextAction = NextAction.Continue; UpdatedSettings = None } + + // Update downloader + elif checkCommand Commands.updateDownloader then + Updater.run settings printer |> ignore + Ok { NextAction = NextAction.Continue; UpdatedSettings = None } + + // Settings summary + elif checkCommand Commands.settingsSummary then + Settings.printSummary settings printer None + Ok { NextAction = NextAction.Continue; UpdatedSettings = None } + + // Toggle split chapters + elif checkCommand Commands.splitChapterToggles then + let newSettings = toggleSplitChapters settings + printer.Info(summarizeToggle "Split Chapters" newSettings.SplitChapters) + Ok { NextAction = NextAction.Continue; UpdatedSettings = Some newSettings } + + // Toggle embed images + elif checkCommand Commands.embedImagesToggles then + let newSettings = toggleEmbedImages settings + printer.Info(summarizeToggle "Embed Images" newSettings.EmbedImages) + Ok { NextAction = NextAction.Continue; UpdatedSettings = Some newSettings } + + // Toggle quiet mode + elif checkCommand Commands.quietModeToggles then + let newSettings = toggleQuietMode settings + printer.Info(summarizeToggle "Quiet Mode" newSettings.QuietMode) + printer.ShowDebug(not newSettings.QuietMode) + Ok { NextAction = NextAction.Continue; UpdatedSettings = Some newSettings } + + // Update audio formats + elif command |> String.startsWithIgnoreCase Commands.updateAudioFormatPrefix then + let format = command.Replace(Commands.updateAudioFormatPrefix, String.Empty).ToLowerInvariant() + if String.hasNoText format then + Error "You must append one or more supported audio formats separated by commas (e.g., \"m4a,opus,best\")." + else + let updateResult = updateAudioFormat settings format + match updateResult with + | Error err -> Error err + | Ok newSettings -> + printer.Info(summarizeUpdate "Audio Formats" (String.Join(", ", newSettings.AudioFormats))) + Ok { NextAction = NextAction.Continue; UpdatedSettings = Some newSettings } + + // Update audio quality + elif command |> String.startsWithIgnoreCase Commands.updateAudioQualityPrefix then + let inputQuality = command.Replace(Commands.updateAudioQualityPrefix, String.Empty) + if String.hasNoText inputQuality then + Error "You must enter a number representing an audio quality between 10 (lowest) and 0 (highest)." + else + match Byte.TryParse inputQuality with + | true, quality -> + let updateResult = updateAudioQuality settings quality + match updateResult with + | Error err -> + Error err + | Ok updatedSettings -> + printer.Info(summarizeUpdate "Audio Quality" (updatedSettings.AudioQuality.ToString())) + Ok { NextAction = NextAction.Continue; UpdatedSettings = Some updatedSettings } + | _ -> + Error $"\"%s{inputQuality}\" is an invalid quality value." + + // Unknown command + else + Error <| + sprintf "\"%s\" is not a valid command. Enter \"%shelp\" to see a list of commands." + command + (string Commands.prefix) + + + /// Processes a single user request, from input to downloading and file post-processing. + /// Returns the next action the application should take (e.g., continue or quit). + let processBatch + (categorizedInputs: CategorizedInput list) + (categoryCounts: CategoryCounts) + (settings: UserSettings) + (resultTracker: ResultTracker) + (history: History) + (printer: Printer) + : BatchResults = + + let inputTime = DateTime.Now + let watch = Watch() + let batchResults = ResultTracker printer + let mutable nextAction = NextAction.Continue + let mutable inputIndex = 0 + let mutable currentSettings = settings + + for input in categorizedInputs do + let mutable stop = false + inputIndex <- inputIndex + 1 + + let result = + match input.Category with + | InputCategory.Command -> + processCommand input.Text currentSettings history printer + | InputCategory.Url -> + processUrl input.Text currentSettings resultTracker history inputTime + categoryCounts[InputCategory.Url] inputIndex printer + + batchResults.RegisterResult(input.Text, result) + + match result with + | Error e -> + printer.Error e + | Ok r -> + nextAction <- r.NextAction + match r.UpdatedSettings with + | None -> () + | Some us -> currentSettings <- us + if nextAction <> NextAction.Continue then + stop <- true + + if categoryCounts[InputCategory.Url] > 1 then + printer.Info(sprintf "%sFinished with batch of %d URLs in %s." + String.newLine + categoryCounts[InputCategory.Url] + watch.ElapsedFriendly) + batchResults.PrintBatchFailures() + + { NextAction = nextAction + UpdatedSettings = Some currentSettings } + + /// Ensures the download environment is ready, then initiates the input and download process. + let start (settings: UserSettings) (printer: Printer) : unit = + // The working directory should start empty. Give the user a chance to empty it. + match Directories.warnIfAnyFiles 10 settings.WorkingDirectory with + | Ok () -> () + | Error filesFoundErr -> + printer.Error filesFoundErr + Directories.askToDeleteAllFiles settings.WorkingDirectory printer |> function + | Ok results -> Directories.printDeletionResults printer results + | Error deletionError -> + printer.Error deletionError + printer.Info "Aborting..." + + let results = ResultTracker printer + let history = History(settings.HistoryFile, settings.HistoryDisplayCount) + let mutable nextAction = NextAction.Continue + let mutable currentSettings = settings + + while nextAction = NextAction.Continue do + let input = printer.GetInput prompt + let splitInputs = splitInputText input + + if List.isEmpty splitInputs then + printer.Error $"Invalid input. Enter only URLs or commands beginning with \"%c{Commands.prefix}\"." + else + let categorizedInputs = categorizeInputs splitInputs + let categoryCounts = countCategories categorizedInputs + summarizeInput categorizedInputs categoryCounts printer + let batchResult = processBatch categorizedInputs categoryCounts currentSettings results history printer + nextAction <- batchResult.NextAction + match batchResult.UpdatedSettings with + | Some s -> currentSettings <- s + | None -> () + + results.PrintSessionSummary() + diff --git a/src/CCVTAC.Main/PostProcessing/CollectionMetadata.fs b/src/CCVTAC.Main/PostProcessing/CollectionMetadata.fs new file mode 100644 index 00000000..5303e4f1 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/CollectionMetadata.fs @@ -0,0 +1,24 @@ +namespace CCVTAC.Main.PostProcessing + +open System.Text.Json.Serialization + +type CollectionMetadata = + { [] Id: string + [] Title: string + [] Availability: string + [] Description: string + [] Tags: string list + [] ModifiedDate: string + [] ViewCount: int option + [] PlaylistCount: int option + [] Channel: string + [] ChannelId: string + [] UploaderId: string + [] Uploader: string + [] ChannelUrl: string + [] UploaderUrl: string + [] Type: string + [] WebpageUrl: string + [] WebpageUrlBasename: string + [] WebpageUrlDomain: string + [] Epoch: int option } diff --git a/src/CCVTAC.Main/PostProcessing/Deleter.fs b/src/CCVTAC.Main/PostProcessing/Deleter.fs new file mode 100644 index 00000000..b850d6c7 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Deleter.fs @@ -0,0 +1,51 @@ +namespace CCVTAC.Main.PostProcessing + +open CCVTAC.Main +open System.IO + +module Deleter = + let private getCollectionFiles + (collectionMetadata: CollectionMetadata option) + (workingDirectory: string) + : Result = + + match collectionMetadata with + | None -> Ok [||] + | Some metadata -> + try Ok (Directory.GetFiles(workingDirectory, $"*{metadata.Id}*")) + with exn -> Error $"Error collecting filenames: {exn.Message}" + + let private deleteAll (fileNames: string array) (printer: Printer) : unit = + fileNames + |> Array.iter (fun fileName -> + try + File.Delete fileName + printer.Debug $"• Deleted \"{fileName}\"" + with + | ex -> printer.Error $"• Deletion error: {ex.Message}" + ) + + let run + (taggingSetFileNames: string seq) + (collectionMetadata: CollectionMetadata option) + (workingDirectory: string) + (printer: Printer) + : unit = + + let collectionFileNames = + match getCollectionFiles collectionMetadata workingDirectory with + | Ok files -> + printer.Debug $"""Found {String.fileLabelWithDescriptor "collection" files.Length}.""" + files + | Error err -> + printer.Warning err + [||] + + let allFileNames = Seq.concat [taggingSetFileNames; collectionFileNames] |> Seq.toArray + + if Array.isEmpty allFileNames then + printer.Warning "No files to delete were found." + else + printer.Debug $"""Deleting {String.fileLabelWithDescriptor "temporary" allFileNames.Length}...""" + deleteAll allFileNames printer + printer.Info "Deleted temporary files." diff --git a/src/CCVTAC.Main/PostProcessing/ImageProcessor.fs b/src/CCVTAC.Main/PostProcessing/ImageProcessor.fs new file mode 100644 index 00000000..ee62388c --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/ImageProcessor.fs @@ -0,0 +1,11 @@ +namespace CCVTAC.Main.PostProcessing + +open CCVTAC.Main.ExternalTools + +module ImageProcessor = + + let private programName = "mogrify" + + let run workingDirectory printer : unit = + let toolSettings = workingDirectory |> ToolSettings.create $"{programName} -trim -fuzz 10%% *.jpg" + Runner.runTool toolSettings [] printer |> ignore diff --git a/src/CCVTAC.Main/PostProcessing/MetadataUtilities.fs b/src/CCVTAC.Main/PostProcessing/MetadataUtilities.fs new file mode 100644 index 00000000..e107389d --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/MetadataUtilities.fs @@ -0,0 +1,59 @@ +namespace CCVTAC.Main.PostProcessing + +open System +open System.Text +open CCVTAC.Main + +module MetadataUtilities = + + let private uploaderSummary (v: VideoMetadata) : string = + let suffix = + match List.tryFind String.hasText [v.UploaderUrl; v.UploaderId] with + | Some x -> $" (%s{x})" + | None -> String.Empty + v.Uploader + suffix + + let private formattedUploadDate (dateText: string) : string = + let y = dateText[0..3] + let m = dateText[4..5] + let d = dateText[6..7] + sprintf "%s/%s/%s" m d y + + let generateComment (v: VideoMetadata) (c: CollectionMetadata option) : string = + let sb = SB() + sb.AppendLine("CCVTAC SOURCE DATA:") |> ignore + sb.AppendLine $"■ Downloaded: {DateTime.Now}" |> ignore + sb.AppendLine $"■ URL: %s{v.WebpageUrl}" |> ignore + sb.AppendLine $"■ Title: %s{v.Fulltitle}" |> ignore + sb.AppendLine $"■ Uploader: %s{uploaderSummary v}" |> ignore + + if String.hasText v.Creator && v.Creator <> v.Uploader then + sb.AppendLine $"■ Creator: %s{v.Creator}" |> ignore + + if String.hasText v.Artist then + sb.AppendLine $"■ Artist: %s{v.Artist}" |> ignore + + if String.hasText v.Album then + sb.AppendLine $"■ Album: %s{v.Album}" |> ignore + + if String.hasText v.Title && v.Title <> v.Fulltitle then + sb.AppendLine $"■ Track Title: %s{v.Title}" |> ignore + + if v.UploadDate.Length = 8 then + sb.AppendLine $"■ Uploaded: %s{formattedUploadDate v.UploadDate}" |> ignore + + let description = String.textOrFallback "None." v.Description + sb.AppendLine $"■ Video description: %s{description}" |> ignore + + match c with + | Some c' -> + sb.AppendLine() |> ignore + sb.AppendLine $"■ Playlist name: %s{c'.Title}" |> ignore + sb.AppendLine $"■ Playlist URL: %s{c'.WebpageUrl}" |> ignore + match v.PlaylistIndex with + | Some index -> if index > 0u then sb.AppendLine $"■ Playlist index: %d{index}" |> ignore + | None -> () + sb.AppendLine($"■ Playlist description: %s{String.textOrEmpty c'.Description}") |> ignore + | None -> () + + sb.ToString() diff --git a/src/CCVTAC.Main/PostProcessing/Mover.fs b/src/CCVTAC.Main/PostProcessing/Mover.fs new file mode 100644 index 00000000..c82e32c3 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Mover.fs @@ -0,0 +1,156 @@ +namespace CCVTAC.Main.PostProcessing + +open CCVTAC.Main +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.PostProcessing +open CCVTAC.Main.PostProcessing.Tagging +open CCVTAC.Main.Settings.Settings +open TaggingSets +open System +open System.IO +open System.Text.Json +open System.Text.RegularExpressions +open Startwatch.Library + +module Mover = + + let private playlistImageRegex = Regex(@"\[[OP]L[\w\d_-]{12,}\]", RegexOptions.Compiled) + + let private imageFileWildcard = "*.jp*" + + let private isPlaylistImage (fileName: string) = + playlistImageRegex.IsMatch fileName + + let private getCoverImage (workingDirInfo: DirectoryInfo) audioFileCount : FileInfo option = + let images = workingDirInfo.EnumerateFiles imageFileWildcard |> Seq.toList + if List.isEmpty images then + None + else + let playlistImages = images |> List.filter (fun i -> isPlaylistImage i.FullName) + if not (List.isEmpty playlistImages) + then Some playlistImages[0] + elif audioFileCount > 1 && images.Length = 1 + then Some images[0] + else None + + let private moveAudioFiles + (audioFiles: FileInfo list) + (moveToDir: string) + (overwrite: bool) + : {| Successes: string list + Failures: string list |} = + + let successes, failures = ResizeArray(), ResizeArray() + + for file in audioFiles do + try + File.Move(file.FullName, Path.Combine(moveToDir, file.Name), overwrite) + successes.Add $"• Moved \"%s{file.Name}\"" + with exn -> + failures.Add $"• Error moving \"%s{file.Name}\": %s{exn.Message}" + + {| Successes = successes |> Seq.toList |> List.rev + Failures = failures |> Seq.toList |> List.rev |} + + let private moveImageFile + (maybeCollectionName: string) + (subFolderName: string) + (workingDirInfo: DirectoryInfo) + (moveToDir: string) + (audioFileCount: int) + (overwrite: bool) + : Result = + + try + match getCoverImage workingDirInfo audioFileCount with + | None -> + Ok "No image to move was found." + | Some fileInfo -> + let baseFileName = + if String.hasNoText maybeCollectionName + then subFolderName + else $"%s{subFolderName} - %s{String.replaceInvalidPathChars None None maybeCollectionName}" + let dest = Path.Combine(moveToDir, $"%s{baseFileName.Trim()}.jpg") + fileInfo.MoveTo(dest, overwrite = overwrite) + Ok $"Image file \"{fileInfo.Name}\" was moved." + with exn -> + Error $"Error copying the image file: %s{exn.Message}" + + let private getParsedVideoJson taggingSet : Result = + try + let json = File.ReadAllText taggingSet.JsonFilePath + match JsonSerializer.Deserialize json with + | Null -> Error $"Deserialized JSON was null for \"%s{taggingSet.JsonFilePath}\"" + | NonNull v -> Ok v + with + | :? JsonException as exn -> + Error $"Error deserializing JSON from file \"%s{taggingSet.JsonFilePath}\": %s{exn.Message}" + | exn -> + Error $"Error reading JSON file \"%s{taggingSet.JsonFilePath}\": %s{exn.Message}." + + let private getSafeSubDirectoryName (collectionData: CollectionMetadata option) taggingSet : string = + let workingName = + match collectionData with + | Some metadata when String.hasText metadata.Uploader && + String.hasText metadata.Title -> metadata.Uploader + | _ -> + match getParsedVideoJson taggingSet with + | Ok v -> v.Uploader + | Error _ -> "COLLECTION_DATA_NOT_FOUND" + + let safeName = workingName |> String.replaceInvalidPathChars None None |> _.Trim() + let topicSuffix = " - Topic" + safeName.Replace(topicSuffix, String.Empty) + + let run + (taggingSets: TaggingSet seq) + (maybeCollectionData: CollectionMetadata option) + (settings: UserSettings) + (overwrite: bool) + (printer: Printer) + : unit = + + printer.Debug "Starting move..." + let watch = Watch() + + let workingDirInfo = DirectoryInfo settings.WorkingDirectory + + match taggingSets |> Seq.tryHead with + | None -> printer.Error "No tagging sets provided" + | Some firstTaggingSet -> + let subFolderName = getSafeSubDirectoryName maybeCollectionData firstTaggingSet + let collectionName = maybeCollectionData |> Option.map _.Title |> Option.defaultValue String.Empty + let fullMoveToDir = Path.Combine(settings.MoveToDirectory, subFolderName, collectionName) + + match Directories.ensureDirectoryExists fullMoveToDir with + | Error err -> + printer.Error err + | Ok dirInfo -> + printer.Debug $"Move-to directory \"%s{dirInfo.Name}\" exists." + + let audioFileNames = + workingDirInfo.EnumerateFiles() + |> Seq.filter (fun f -> List.caseInsensitiveContains f.Extension Files.audioFileExtensions) + |> List.ofSeq + + if audioFileNames.IsEmpty then + printer.Error "No audio filenames to move were found." + else + let fileCountMsg = String.fileLabelWithDescriptor "audio" + + printer.Debug $"Moving %s{fileCountMsg audioFileNames.Length} to \"%s{fullMoveToDir}\"..." + + let results = moveAudioFiles audioFileNames fullMoveToDir overwrite + + printer.Info $"Moved %s{fileCountMsg results.Successes.Length} in %s{watch.ElapsedFriendly}." + results.Successes |> List.iter printer.Debug + + if List.isNotEmpty results.Failures then + printer.Warning $"However, %s{fileCountMsg results.Failures.Length} could not be moved:" + results.Failures |> List.iter printer.Error + + moveImageFile collectionName subFolderName workingDirInfo fullMoveToDir + audioFileNames.Length overwrite + |> function + | Ok msg -> printer.Info msg + | Error err -> printer.Error $"Error moving the image file: %s{err}." diff --git a/src/CCVTAC.Main/PostProcessing/PostProcessing.fs b/src/CCVTAC.Main/PostProcessing/PostProcessing.fs new file mode 100644 index 00000000..78998c9e --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/PostProcessing.fs @@ -0,0 +1,93 @@ +namespace CCVTAC.Main.PostProcessing + +open CCVTAC.Main +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.PostProcessing.Tagging +open CCVTAC.Main.Settings.Settings +open System.IO +open System.Linq +open System.Text.Json +open System.Text.RegularExpressions +open Startwatch.Library +open TaggingSets + +module PostProcessor = + + let private collectionMetadataRegex = + Regex(@"(?<=\[)[\w\-]{17,}(?=\]\.info.json)", RegexOptions.Compiled) + + let private isCollectionMetadataMatch (path: string) : bool = + collectionMetadataRegex.IsMatch path + + let private getCollectionJson workingDirectoryName : Result = + try + let fileNames = + Directory.GetFiles workingDirectoryName + |> Seq.filter isCollectionMetadataMatch + |> Set.ofSeq + + if Seq.isEmpty fileNames then + Error "No relevant files found." + elif Seq.hasMultiple fileNames then + Error "Unexpectedly found more than one relevant file, so none will be processed." + else + let fileName = fileNames.Single() + let json = File.ReadAllText fileName + match JsonSerializer.Deserialize json with + | Null -> Error $"Deserialized collection metadata for \"%s{fileName}\" was null." + | NonNull parsedData -> Ok parsedData + with + | ex -> Error ex.Message + + let private generateTaggingSets dir : Result = + try + let taggingSets = createSets <| Directory.GetFiles dir + if List.isEmpty taggingSets + then Error $"No tagging sets were created using files in working directory \"%s{dir}\". Are all file extensions correct?" + else Ok taggingSets + with exn -> + Error $"Error reading working files in \"{dir}\": %s{exn.Message}" + + let run settings mediaType (printer: Printer) : unit = + let watch = Watch() + let workingDirectory = settings.WorkingDirectory + + printer.Info "Starting post-processing..." + + match generateTaggingSets workingDirectory with + | Error _ -> + printer.Error $"No tagging sets were generated for directory {workingDirectory}, so tagging cannot be done." + | Ok taggingSets -> + let collectionJson = + match getCollectionJson workingDirectory with + | Error e -> + printer.Debug $"No playlist or channel metadata found: %s{e}" + None + | Ok cm -> + printer.Debug "Found playlist/channel metadata." + Some cm + + if settings.EmbedImages then + ImageProcessor.run workingDirectory printer + + match Tagger.run settings taggingSets collectionJson mediaType printer with + | Ok msg -> + printer.Info msg + Renamer.run settings workingDirectory printer + Mover.run taggingSets collectionJson settings true printer + + let allTaggingSetFiles = taggingSets |> List.collect allFiles + Deleter.run allTaggingSetFiles collectionJson workingDirectory printer + + match Directories.warnIfAnyFiles 20 workingDirectory with + | Ok _ -> () + | Error err -> + printer.Error err + printer.Info "Will delete the remaining files..." + match Directories.deleteAllFiles workingDirectory with + | Ok results -> Directories.printDeletionResults printer results + | Error e -> printer.Error e + | Error e -> + printer.Error($"Tagging error(s) preventing further post-processing: {e}") + + printer.Info $"Post-processing done in %s{watch.ElapsedFriendly}." diff --git a/src/CCVTAC.Main/PostProcessing/Renamer.fs b/src/CCVTAC.Main/PostProcessing/Renamer.fs new file mode 100644 index 00000000..48dab0b4 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Renamer.fs @@ -0,0 +1,99 @@ +namespace CCVTAC.Main.PostProcessing + +open CCVTAC.Main +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.Settings.Settings +open System +open System.IO +open System.Text +open System.Text.RegularExpressions +open Startwatch.Library + +module Renamer = + + let private toNormalizationForm (form: string) = + match form.Trim().ToUpperInvariant() with + | "D" -> NormalizationForm.FormD + | "KD" -> NormalizationForm.FormKD + | "KC" -> NormalizationForm.FormKC + | _ -> NormalizationForm.FormC + + let updateTextViaPatterns isQuietMode (printer: Printer) (sb: SB) (renamePattern: RenamePattern) = + let regex = Regex renamePattern.RegexPattern + + let matches = + regex.Matches(sb.ToString()) + |> Seq.cast + |> Seq.filter _.Success + |> Seq.rev + |> Seq.toList + + if List.isEmpty matches + then sb + else + if not isQuietMode then + let patternSummary = + if String.hasText renamePattern.Summary then + $"\"%s{renamePattern.Summary}\"" + else + $"`%s{renamePattern.RegexPattern}` (no description)" + + printer.Debug $"Rename pattern %s{patternSummary} matched × %d{matches.Length}." + + for m in matches do + sb.Remove(m.Index, m.Length) |> ignore + + // Build replacement text by replacing % placeholders with group captures. + let replacementText = + m.Groups + |> Seq.cast + |> Seq.mapi (fun i _ -> + let searchFor = sprintf "%%<%d>s" (i + 1) + let replaceWith = + // Group 0 is the entire match, so we only want groups starting at 1. + if i + 1 < m.Groups.Count + then m.Groups[i + 1].Value.Trim() + else String.Empty + (searchFor, replaceWith)) + |> Seq.fold (fun (sb': SB) -> sb'.Replace) (SB renamePattern.ReplaceWithPattern) + |> _.ToString() + + sb.Insert(m.Index, replacementText) |> ignore + + sb + + let run userSettings workingDirectory (printer: Printer) : unit = + let watch = Watch() + let workingDirInfo = DirectoryInfo workingDirectory + + let audioFiles = + workingDirInfo.EnumerateFiles() + |> Seq.filter (fun f -> List.caseInsensitiveContains f.Extension Files.audioFileExtensions) + |> List.ofSeq + + if List.isEmpty audioFiles then + printer.Warning "No audio files to rename were found." + else + printer.Debug $"""Renaming %s{String.fileLabelWithDescriptor "audio" audioFiles.Length}...""" + + for audioFile in audioFiles do + let newFileName = + userSettings.RenamePatterns + |> List.fold + (fun (sb: SB) -> updateTextViaPatterns userSettings.QuietMode printer sb) + (SB audioFile.Name) + |> _.ToString() + + try + let destinationPath = + Path.Combine(workingDirectory, newFileName) + |> _.Normalize(toNormalizationForm userSettings.NormalizationForm) + + File.Move(audioFile.FullName, destinationPath) + + printer.Debug $"• From: \"%s{audioFile.Name}\"" + printer.Debug $" To: \"%s{newFileName}\"" + with exn -> + printer.Error $"• Error renaming \"%s{audioFile.Name}\": %s{exn.Message}" + + printer.Info $"Renaming done in %s{watch.ElapsedFriendly}." diff --git a/src/CCVTAC.Main/PostProcessing/Tagging/Detectors.fs b/src/CCVTAC.Main/PostProcessing/Tagging/Detectors.fs new file mode 100644 index 00000000..ab81e01e --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Tagging/Detectors.fs @@ -0,0 +1,81 @@ +namespace CCVTAC.Main.PostProcessing.Tagging + +open CCVTAC.Main.Settings.Settings +open CCVTAC.Main.PostProcessing +open System +open System.Text.RegularExpressions + +module Detectors = + /// Attempts casting the input text to type 'a and returning it. + /// If casting fails, the default value is returned instead. + let tryCast<'a> (text: string) : 'a option = + try + // If 'a is string, return the text directly. + if typeof<'a> = typeof then + Some (box text :?> 'a) + else + Some (Convert.ChangeType(text, typeof<'a>) :?> 'a) + with + | _ -> None + + /// Extracts the value of the specified tag field from the given data. + /// Video metadata + /// The name of the field within the video metadata to read + /// The text content of the requested field of the video metadata + let private extractMetadataText (metadata: VideoMetadata) (fieldName: string) : string = + match fieldName with + | "title" -> metadata.Title + | "description" -> metadata.Description + | _ -> + // TODO: It would be best to check for invalid entries upon settings deserialization. + raise (ArgumentException($"\"{fieldName}\" is an invalid video metadata field name.")) + + /// Finds and returns the first instance of text matching a given detection scheme pattern, + /// parsing to type 'a if necessary. + /// A match of type 'a if there was a match; otherwise, the default value provided, if any. + let detectSingle<'a> + (videoMetadata: VideoMetadata) + (patterns: TagDetectionPattern seq) + (defaultValue: 'a option) + : 'a option = + + patterns + |> Seq.tryPick (fun pattern -> + let fieldText = extractMetadataText videoMetadata pattern.SearchField + let match' = Regex(pattern.RegexPattern).Match(fieldText) + + if match'.Success then + let matchedText = match'.Groups[pattern.MatchGroup].Value.Trim() + tryCast<'a> matchedText + else + None) + |> Option.orElse defaultValue + + /// Finds and returns all instances of text matching a given detection scheme pattern, + /// concatenating them into a single string (using a custom separator), then casting + /// to type 'a if necessary. + /// A match of type 'a if there were any matches; otherwise, the default value provided, if any. + let detectMultiple<'a> + (data: VideoMetadata) + (patterns: TagDetectionPattern seq) + (defaultValue: 'a option) + (separator: string) + : 'a option = + + let matchedValues = + patterns + |> Seq.collect (fun pattern -> + let fieldText = extractMetadataText data pattern.SearchField + Regex(pattern.RegexPattern).Matches(fieldText) + |> Seq.filter _.Success + |> Seq.map _.Groups[pattern.MatchGroup].Value.Trim()) + |> Seq.distinct + |> Seq.toArray + + if Array.isEmpty matchedValues then + defaultValue + else + String.Join(separator, matchedValues) + |> tryCast<'a> + |> Option.orElse defaultValue + diff --git a/src/CCVTAC.Main/PostProcessing/Tagging/TagDetector.fs b/src/CCVTAC.Main/PostProcessing/Tagging/TagDetector.fs new file mode 100644 index 00000000..c23ee4e7 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Tagging/TagDetector.fs @@ -0,0 +1,45 @@ +namespace CCVTAC.Main.PostProcessing.Tagging + +open CCVTAC.Main.Settings.Settings +open CCVTAC.Main.PostProcessing.Tagging + +module TagDetection = + + let detectTitle videoData fallback (tagDetectionPatterns: TagDetectionPatterns) : string option = + let detectedTitle = + Detectors.detectSingle videoData tagDetectionPatterns.Title None + + match detectedTitle, fallback with + | Some title, _ -> Some title + | None, Some title -> Some title + | None, None -> None + + let detectArtist videoData fallback (tagDetectionPatterns: TagDetectionPatterns) : string option = + let detectedArtist = + Detectors.detectSingle videoData tagDetectionPatterns.Artist None + + match detectedArtist, fallback with + | Some artist, _ -> Some artist + | None, Some artist -> Some artist + | None, None -> None + + let detectAlbum videoData fallback (tagDetectionPatterns: TagDetectionPatterns) : string option = + let detectedAlbum = + Detectors.detectSingle videoData tagDetectionPatterns.Album None + + match detectedAlbum, fallback with + | Some album, _ -> Some album + | None, Some album -> Some album + | None, None -> None + + let detectComposers videoData (tagDetectionPatterns: TagDetectionPatterns) : string option = + Detectors.detectMultiple videoData tagDetectionPatterns.Composer None "; " + + let detectReleaseYear videoData fallback (tagDetectionPatterns: TagDetectionPatterns) : uint32 option = + let detectedYear = + Detectors.detectSingle videoData tagDetectionPatterns.Year None + + match detectedYear, fallback with + | Some year, _ -> Some year + | None, Some year -> Some year + | None, None -> None diff --git a/src/CCVTAC.Main/PostProcessing/Tagging/Tagger.fs b/src/CCVTAC.Main/PostProcessing/Tagging/Tagger.fs new file mode 100644 index 00000000..15c396e5 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Tagging/Tagger.fs @@ -0,0 +1,220 @@ +namespace CCVTAC.Main.PostProcessing.Tagging + +open CCVTAC.Main +open CCVTAC.Main.Settings.Settings +open CCVTAC.Main.PostProcessing +open CCVTAC.Main.PostProcessing.Tagging +open CCVTAC.Main.Downloading.Downloading +open Startwatch.Library +open TaggingSets +open MetadataUtilities +open System +open System.IO +open System.Text.Json + +type TaggedFile = TagLib.File + +module Tagger = + + let private parseVideoJson taggingSet : Result = + try + let json = File.ReadAllText taggingSet.JsonFilePath + match JsonSerializer.Deserialize json with + | Null -> Error $"Deserialized JSON was null for \"%s{taggingSet.JsonFilePath}\"." + | NonNull v -> Ok v + with + | :? JsonException as exn -> Error $"%s{exn.Message}%s{String.newLine}%s{exn.StackTrace}" + | exn -> Error $"Error reading JSON file \"%s{taggingSet.JsonFilePath}\": %s{exn.Message}." + + /// If a video was split into sub-videos, then the original video is unneeded and should be deleted. + let private deleteSourceFile taggingSet (printer: Printer) : TaggingSet = + if not (List.hasMultiple taggingSet.AudioFilePaths) then + taggingSet + else + let largestFileInfo = + taggingSet.AudioFilePaths + |> Seq.map FileInfo + |> Seq.sortByDescending _.Length + |> Seq.head + + try + File.Delete largestFileInfo.FullName + printer.Debug $"Deleted pre-split source file \"%s{largestFileInfo.Name}\"" + { taggingSet with + AudioFilePaths = taggingSet.AudioFilePaths + |> List.except [largestFileInfo.FullName] } + with exn -> + printer.Error $"Error deleting pre-split source file \"%s{largestFileInfo.Name}\": %s{exn.Message}" + taggingSet + + let private writeImageToFile (taggedFile: TaggedFile) imageFilePath (printer: Printer) = + if String.hasNoText imageFilePath then + printer.Error "No image file path was provided, so cannot add an image to the file." + else + try + let pics = Array.zeroCreate 1 + pics[0] <- TagLib.Picture imageFilePath + taggedFile.Tag.Pictures <- pics + printer.Debug "Image written to file tags OK." + with exn -> + printer.Error $"Error writing image to the audio file: %s{exn.Message}" + + let private releaseYear userSettings videoMetadata : uint32 option = + if userSettings.IgnoreUploadYearUploaders |> List.caseInsensitiveContains videoMetadata.Uploader then + None + elif videoMetadata.UploadDate.Length <> 4 then + None + else + match UInt32.TryParse(videoMetadata.UploadDate.Substring(0, 4)) with + | true, parsed -> Some parsed + | _ -> None + + let private tagSingleFile + (settings: UserSettings) + (videoData: VideoMetadata) + (audioFilePath: string) + (imageFilePath: string option) + (collectionData: CollectionMetadata option) + (printer: Printer) + : unit = + + let audioFileName = Path.GetFileName audioFilePath + printer.Debug $"Current audio file: \"%s{audioFileName}\"" + + use taggedFile = TaggedFile.Create audioFilePath + let patterns = settings.TagDetectionPatterns + + // Title + if String.hasText videoData.Track then + printer.Debug $"• Using metadata title \"%s{videoData.Track}\"" + taggedFile.Tag.Title <- videoData.Track + else + match TagDetection.detectTitle videoData (Some videoData.Title) patterns with + | Some title -> + printer.Debug $"• Detected title \"%s{title}\"" + taggedFile.Tag.Title <- title + | None -> printer.Debug "No title was found." + + // Artists + if String.hasText videoData.Artist then + let metadataArtists = videoData.Artist + let firstArtist = metadataArtists.Split([|", "|], StringSplitOptions.None)[0] + let diffSummary = + if firstArtist = metadataArtists + then String.Empty + else $" (extracted from \"%s{metadataArtists}\")" + taggedFile.Tag.Performers <- [| firstArtist |] + printer.Debug $"• Using metadata artist \"%s{firstArtist}\"%s{diffSummary}" + else + match TagDetection.detectArtist videoData None patterns with + | None -> () + | Some artist -> + printer.Debug $"• Detected artist \"%s{artist}\"" + taggedFile.Tag.Performers <- [| artist |] + + // Album + if String.hasText videoData.Album then + printer.Debug $"• Using metadata album \"%s{videoData.Album}\"" + taggedFile.Tag.Album <- videoData.Album + else + let collectionTitle = collectionData |> Option.map _.Title + match TagDetection.detectAlbum videoData collectionTitle patterns with + | None -> () + | Some album -> + printer.Debug $"• Detected album \"%s{album}\"" + taggedFile.Tag.Album <- album + + // Composers + match TagDetection.detectComposers videoData patterns with + | None -> () + | Some composers -> + printer.Debug $"• Detected composer(s) \"%s{composers}\"" + taggedFile.Tag.Composers <- [| composers |] + + // Track number + match videoData.PlaylistIndex with + | None -> () + | Some (trackNo: uint32) -> + printer.Debug $"• Using playlist index of %d{trackNo} for track number" + taggedFile.Tag.Track <- uint32 trackNo + + // Year + match videoData.ReleaseYear with + | Some (year: uint32) -> + printer.Debug $"• Using metadata release year \"%d{year}\"" + taggedFile.Tag.Year <- year + | None -> + let defaultYear = releaseYear settings videoData + match TagDetection.detectReleaseYear videoData defaultYear patterns with + | None -> () + | Some year -> + printer.Debug $"• Detected year \"%d{year}\"" + taggedFile.Tag.Year <- year + + // Comment + taggedFile.Tag.Comment <- generateComment videoData collectionData + + // Artwork embedding + match imageFilePath with + | Some path -> + if settings.EmbedImages + && settings.DoNotEmbedImageUploaders |> List.doesNotContain videoData.Uploader + then + printer.Info "Embedding artwork..." + writeImageToFile taggedFile path printer + else + printer.Debug "Skipping artwork embedding." + | None -> + printer.Debug "Skipping artwork embedding as no artwork was found." + + try + taggedFile.Save() + printer.Debug $"Wrote tags to \"%s{audioFileName}\"." + with exn -> + printer.Error $"Failed to save tags: ${exn.Message}" + + let private processTaggingSet + (settings: UserSettings) + (taggingSet: TaggingSet) + (collectionJson: CollectionMetadata option) + (embedImages: bool) + (printer: Printer) + : unit = + + printer.Debug $"""Found %s{String.fileLabelWithDescriptor "audio" taggingSet.AudioFilePaths.Length} with resource ID %s{taggingSet.ResourceId}.""" + + match parseVideoJson taggingSet with + | Ok videoData -> + let finalTaggingSet = deleteSourceFile taggingSet printer + + let imagePath = + if embedImages && List.isNotEmpty finalTaggingSet.AudioFilePaths then + Some finalTaggingSet.ImageFilePath + else + None + + for audioPath in finalTaggingSet.AudioFilePaths do + try + tagSingleFile settings videoData audioPath imagePath collectionJson printer + with exn -> + printer.Error $"Error tagging file: %s{exn.Message}" + | Error err -> + printer.Error $"Error deserializing video metadata from \"%s{taggingSet.JsonFilePath}\": {err}" + + let run + (settings: UserSettings) + (taggingSets: TaggingSet seq) + (collectionJson: CollectionMetadata option) + (mediaType: MediaType) + (printer: Printer) + : Result = + + printer.Debug "Adding file tags..." + let watch = Watch() + + let embedImages = settings.EmbedImages && (mediaType.IsVideo || mediaType.IsPlaylistVideo) + + for taggingSet in taggingSets do + processTaggingSet settings taggingSet collectionJson embedImages printer + + Ok $"Tagging done in %s{watch.ElapsedFriendly}." diff --git a/src/CCVTAC.Main/PostProcessing/Tagging/TaggingSet.fs b/src/CCVTAC.Main/PostProcessing/Tagging/TaggingSet.fs new file mode 100644 index 00000000..ddee46d0 --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/Tagging/TaggingSet.fs @@ -0,0 +1,57 @@ +namespace CCVTAC.Main.PostProcessing.Tagging + +open CCVTAC.Main +open CCVTAC.Main.IoUtilities +open System.IO +open System.Text.RegularExpressions + +/// Contains all the data necessary for tagging a related set of files. +module TaggingSets = + + type TaggingSet = + { ResourceId: string + AudioFilePaths: string list + JsonFilePath: string + ImageFilePath: string } + + let allFiles taggingSet = + List.concat [taggingSet.AudioFilePaths; [taggingSet.JsonFilePath; taggingSet.ImageFilePath]] + + /// Create a collection of TaggingSets from a collection of file paths related to several video IDs. + /// Files that don't match the requirements will be ignored. + let createSets filePaths : TaggingSet list = + if Seq.isEmpty filePaths then + [] + else + let jsonFileExt = ".json" + let imageFileExt = ".jpg" + + // Regex group 0 is the full filename, and group 1 contains the video ID. + let fileNamesWithVideoIdsRegex = + Regex(@".+\[([\w_\-]{11})\](?:.*)?\.(\w+)", RegexOptions.Compiled) + + let fileHasSupportedExtension (file: string) = + match Path.GetExtension file with + | Null -> false + | NonNull (ext: string) -> Seq.caseInsensitiveContains ext Files.audioFileExtensions + + filePaths + |> Seq.map fileNamesWithVideoIdsRegex.Match + |> Seq.filter _.Success + |> Seq.map (fun m -> m.Captures |> Seq.cast |> Seq.head) + |> Seq.groupBy _.Groups[1].Value + |> Seq.map (fun (videoId, matches) -> videoId, matches |> Seq.map _.Groups[0].Value) + |> Seq.filter (fun (_, files) -> + let isSupportedExt = files |> Seq.exists fileHasSupportedExtension + let hasOneJson = files |> Seq.filter (String.endsWithIgnoreCase jsonFileExt) |> Seq.hasOne + let hasOneImage = files |> Seq.filter (String.endsWithIgnoreCase imageFileExt) |> Seq.hasOne + isSupportedExt && hasOneJson && hasOneImage) + |> Seq.map (fun (videoId, files) -> + let audioFiles = files |> Seq.filter fileHasSupportedExtension + let jsonFile = files |> Seq.find (String.endsWithIgnoreCase jsonFileExt) + let imageFile = files |> Seq.find (String.endsWithIgnoreCase imageFileExt) + { ResourceId = videoId + AudioFilePaths = audioFiles |> Seq.toList + JsonFilePath = jsonFile + ImageFilePath = imageFile }) + |> List.ofSeq diff --git a/src/CCVTAC.Main/PostProcessing/VideoMetadata.fs b/src/CCVTAC.Main/PostProcessing/VideoMetadata.fs new file mode 100644 index 00000000..0ca0deae --- /dev/null +++ b/src/CCVTAC.Main/PostProcessing/VideoMetadata.fs @@ -0,0 +1,77 @@ +namespace CCVTAC.Main.PostProcessing + +open System.Text.Json.Serialization + +type VideoMetadata = { + [] Id: string + [] Title: string + [] Thumbnail: string + [] Description: string + [] ChannelId: string + [] ChannelUrl: string + [] Duration: int option + [] ViewCount: int option + [] AgeLimit: int option + [] WebpageUrl: string + [] Categories: string list + [] Tags: string list + [] PlayableInEmbed: bool option + [] LiveStatus: string + [] ReleaseTimestamp: int option + [] FormatSortFields: string list + [] Album: string + [] Artist: string + [] Track: string + [] CommentCount: int option + [] LikeCount: int option + [] Channel: string + [] ChannelFollowerCount: int option + [] ChannelIsVerified: bool option + [] Uploader: string + [] UploaderId: string + [] UploaderUrl: string + [] UploadDate: string + [] Creator: string + [] AltTitle: string + [] Availability: string + [] WebpageUrlBasename: string + [] WebpageUrlDomain: string + [] Extractor: string + [] ExtractorKey: string + [] PlaylistCount: int option + [] Playlist: string + [] PlaylistId: string + [] PlaylistTitle: string + [] NEntries: int option + [] PlaylistIndex: uint32 option + [] DisplayId: string + [] Fulltitle: string + [] DurationString: string + [] ReleaseDate: string + [] ReleaseYear: uint32 option + [] IsLive: bool option + [] WasLive: bool option + [] Epoch: int option + [] Asr: int option + [] Filesize: int option + [] FormatId: string + [] FormatNote: string + [] SourcePreference: int option + [] AudioChannels: int option + [] Quality: double option + [] HasDrm: bool option + [] Tbr: double option + [] Url: string + [] LanguagePreference: int option + [] Ext: string + [] Vcodec: string + [] Acodec: string + [] Container: string + [] Protocol: string + [] Resolution: string + [] AudioExt: string + [] VideoExt: string + [] Vbr: int option + [] Abr: double option + [] Format: string + [] Type: string } diff --git a/src/CCVTAC.Main/Printer.fs b/src/CCVTAC.Main/Printer.fs new file mode 100644 index 00000000..d73f12b2 --- /dev/null +++ b/src/CCVTAC.Main/Printer.fs @@ -0,0 +1,155 @@ +namespace CCVTAC.Main + +open System +open System.Collections.Generic +open System.Linq +open Spectre.Console + +type private Level = + | Critical = 0 + | Error = 1 + | Warning = 2 + | Info = 3 + | Debug = 4 + +type private ColorFormat = + { Foreground: string option + Background: string option + Bold: bool } + +type Printer(showDebug: bool) = + + static let colors : Dictionary = + Dictionary() + |> fun d -> + d.Add(Level.Critical, { Foreground = Some "white"; Background = Some "red3"; Bold = true }) + d.Add(Level.Error, { Foreground = Some "red"; Background = None; Bold = false }) + d.Add(Level.Warning, { Foreground = Some "yellow"; Background = None; Bold = false }) + d.Add(Level.Info, { Foreground = None; Background = None; Bold = false }) + d.Add(Level.Debug, { Foreground = Some "grey70"; Background = None; Bold = false }) + d + + let mutable minimumLogLevel = + if showDebug then Level.Debug else Level.Info + + let extractedErrors (result: Result<'a,'b list>) : 'b list = + match result with + | Ok _ -> [] + | Error errors -> errors + + /// Toggle showing or hiding debug messages. + member this.ShowDebug(show: bool) = + minimumLogLevel <- (if show then Level.Debug else Level.Info) + + /// Escape text so Spectre markup and format strings are safe. + static member private EscapeText(text: string) : string = + text.Replace("{", "{{") + .Replace("}", "}}") + .Replace("[", "[[") + .Replace("]", "]]") + + static member private AddMarkup(message: string, colors: ColorFormat) : string = + match colors.Foreground, colors.Background, colors.Bold with + | None, None, false -> message + | fg, bg, bold -> + let boldPart = if bold then "bold " else String.Empty + let fgPart = defaultArg fg "default" + let bgPart = match bg with Some b -> $" on {b}" | None -> String.Empty + let markUp = $"{boldPart}{fgPart}{bgPart}" + $"[{markUp}]{message}[/]" + + member private this.Print + ( + logLevel: Level, + message: string, + ?appendLineBreak: bool, + ?prependLines: byte, + ?appendLines: byte, + ?processMarkup: bool + ) : unit = + + let appendLineBreak = defaultArg appendLineBreak true + let prependLines = defaultArg prependLines 0uy + let appendLines = defaultArg appendLines 0uy + let processMarkup = defaultArg processMarkup true + + if int logLevel > int minimumLogLevel then + () + else + if String.hasNoText message then + raise (ArgumentNullException("message", "Message cannot be empty.")) + + Printer.EmptyLines prependLines + + let escapedMessage = Printer.EscapeText message + + if processMarkup then + let markedUp = Printer.AddMarkup(escapedMessage, colors[logLevel]) + AnsiConsole.Markup markedUp + else + // AnsiConsole.Write uses format strings internally; escapedMessage already duplicates braces + AnsiConsole.Write escapedMessage + + if appendLineBreak then AnsiConsole.WriteLine() + + Printer.EmptyLines appendLines + + static member PrintTable(table: Table) = + AnsiConsole.Write table + + member this.Critical(message: string, ?appendLineBreak: bool, ?prependLines: byte, ?appendLines: byte, ?processMarkup: bool) = + this.Print(Level.Critical, message, ?appendLineBreak = appendLineBreak, ?prependLines = prependLines, + ?appendLines = appendLines, ?processMarkup = processMarkup) + + member this.Error(message: string, ?appendLineBreak: bool, ?prependLines: byte, ?appendLines: byte, ?processMarkup: bool) = + this.Print(Level.Error, message, ?appendLineBreak = appendLineBreak, ?prependLines = prependLines, + ?appendLines = appendLines, ?processMarkup = processMarkup) + + member this.Errors(errors: string seq, ?appendLines: byte) = + if Seq.isEmpty errors then raise (ArgumentException("No errors were provided!", "errors")) + for err in errors |> Seq.filter String.hasText do + this.Error err + Printer.EmptyLines(defaultArg appendLines 0uy) + + member private this.Errors(headerMessage: string, errors: string seq) = + // Create an array with headerMessage followed by the items in errors. + let items = seq { yield headerMessage; yield! errors } |> Seq.toArray + this.Errors(items, 0uy) + + member this.Errors<'a>(failResult: Result<'a, string list>, ?appendLines: byte) = + this.Errors(extractedErrors failResult, ?appendLines = appendLines) + + member this.Errors<'a>(headerMessage: string, failingResult: Result<'a, string list>) = + this.Errors(headerMessage, extractedErrors failingResult) + + member this.Warning(message: string, ?appendLineBreak: bool, ?prependLines: byte, ?appendLines: byte, ?processMarkup: bool) = + this.Print(Level.Warning, message, ?appendLineBreak = appendLineBreak, ?prependLines = prependLines, + ?appendLines = appendLines, ?processMarkup = processMarkup) + + member this.Info(message: string, ?appendLineBreak: bool, ?prependLines: byte, ?appendLines: byte, ?processMarkup: bool) = + this.Print(Level.Info, message, ?appendLineBreak = appendLineBreak, ?prependLines = prependLines, + ?appendLines = appendLines, ?processMarkup = processMarkup) + + member this.Debug(message: string, ?appendLineBreak: bool, ?prependLines: byte, ?appendLines: byte, ?processMarkup: bool) = + this.Print(Level.Debug, message, ?appendLineBreak = appendLineBreak, ?prependLines = prependLines, + ?appendLines = appendLines, ?processMarkup = processMarkup) + + /// Prints the requested number of blank lines. + static member EmptyLines(count: byte) = + if Numerics.isZero count + then () + else + let repeats = int count - 1 + if repeats <= 0 + then AnsiConsole.WriteLine() + else Enumerable.Repeat(String.newLine, repeats) |> String.Concat |> AnsiConsole.WriteLine + + member this.GetInput(prompt: string) : string = + Printer.EmptyLines 1uy + AnsiConsole.Ask($"[skyblue1]{prompt}[/]") + + static member private Ask(title: string, options: string list) : string = + AnsiConsole.Prompt(SelectionPrompt().Title(title).AddChoices(options)) + + member this.AskToBool(title: string, trueAnswer: string, falseAnswer: string) : bool = + Printer.Ask(title, [ trueAnswer; falseAnswer ]) = trueAnswer diff --git a/src/CCVTAC.Main/Program.fs b/src/CCVTAC.Main/Program.fs new file mode 100644 index 00000000..dd1634b3 --- /dev/null +++ b/src/CCVTAC.Main/Program.fs @@ -0,0 +1,73 @@ +namespace CCVTAC.Main + +open CCVTAC.Main +open CCVTAC.Main.IoUtilities +open CCVTAC.Main.Settings +open CCVTAC.Main.Settings.Settings +open Settings.IO +open System +open System.IO +open Spectre.Console + +module Program = + + let private helpFlags = [| "-h"; "--help" |] + let private settingsFileFlags = [| "-s"; "--settings" |] + let private defaultSettingsFileName = "settings.json" + + type ExitCodes = + | Success = 0 + | ArgError = 1 + | OperationError = 2 + + [] + let main args : int = + let printer = Printer(showDebug = true) + + if Array.isNotEmpty args && Array.caseInsensitiveContains args[0] helpFlags then + printer.Info Help.helpText + int ExitCodes.Success + else + let settingsPath = + FileInfo <| + if Array.hasMultiple args && Array.caseInsensitiveContains args[0] settingsFileFlags then + args[1] // Expected to be a settings file path. + else + defaultSettingsFileName + + if not settingsPath.Exists then + match writeDefaultFile settingsPath with + | Ok msg -> + printer.Info msg + int ExitCodes.Success + | Error err -> + printer.Error err + int ExitCodes.OperationError + else + match read settingsPath with + | Error err -> + printer.Error err + int ExitCodes.ArgError + | Ok settings -> + printSummary settings printer (Some "Settings loaded OK.") + printer.ShowDebug(not settings.QuietMode) + + // Catch Ctrl-C (SIGINT) + Console.CancelKeyPress.Add(fun _ -> + printer.Warning($"{String.newLine}Quitting at user's request.") + + match Directories.warnIfAnyFiles 10 settings.WorkingDirectory with + | Ok () -> () + | Error warnResult -> + printer.Error warnResult + match Directories.askToDeleteAllFiles settings.WorkingDirectory printer with + | Error err -> printer.Error err + | Ok results -> Directories.printDeletionResults printer results) + try + Orchestrator.start settings printer + int ExitCodes.Success + with exn -> + printer.Critical $"Fatal error: %s{exn.Message}" + AnsiConsole.WriteException exn + printer.Info "Please help improve this tool by reporting this error and any relevant URLs at https://github.com/codeconscious/ccvtac/issues." + int ExitCodes.OperationError diff --git a/src/CCVTAC.Main/ResultTracker.fs b/src/CCVTAC.Main/ResultTracker.fs new file mode 100644 index 00000000..1a60397b --- /dev/null +++ b/src/CCVTAC.Main/ResultTracker.fs @@ -0,0 +1,58 @@ +namespace CCVTAC.Main + +open System +open System.Collections.Generic + +type ResultTracker<'a>(printer: Printer) = + + let mutable successCount : uint64 = 0UL + + let failures = Dictionary() + + static let combineErrors (errors: string list) = + String.Join(" / ", errors) + + member _.AddSuccess() : unit = + successCount <- successCount + 1UL + + member _.AddFailure(url: string, error: string) : unit = + failures.Add(url, error) + + /// Logs the result for a specific corresponding input. + member _.RegisterResult(input: string, result: Result<'a, string>) : unit = + match result with + | Ok _ -> + successCount <- successCount + 1UL + | Error e -> + if not (failures.TryAdd(input, e)) then + failures[input] <- e + + /// Logs the result for a specific corresponding input. + member _.RegisterResult(input: string, result: Result<'a, string list>) : unit = + match result with + | Ok _ -> + successCount <- successCount + 1UL + | Error e -> + let msg = if e.Length > 0 then List.head e else String.Empty + if not (failures.TryAdd(input, msg)) then + failures[input] <- msg + + /// Prints any failures for the current batch. + member _.PrintBatchFailures() : unit = + if Numerics.isZero failures.Count then + printer.Debug "No failures in batch." + else + let failureLabel = String.pluralize "failure" "failures" failures.Count + printer.Info $"%d{failures.Count} %s{failureLabel} in this batch:" + for pair in failures do + printer.Warning $"- %s{pair.Key}: %s{pair.Value}" + + /// Prints the output for the current application session. + member _.PrintSessionSummary() : unit = + let successLabel = String.pluralize "success" "successes" successCount + let failureLabel = String.pluralize "failure" "failures" failures.Count + + printer.Info $"Quitting with %d{successCount} %s{successLabel} and %d{failures.Count} %s{failureLabel}." + + for pair in failures do + printer.Warning $"- %s{pair.Key}: %s{pair.Value}" diff --git a/src/CCVTAC.Main/Settings/Id3Version.fs b/src/CCVTAC.Main/Settings/Id3Version.fs new file mode 100644 index 00000000..0b1a6e3c --- /dev/null +++ b/src/CCVTAC.Main/Settings/Id3Version.fs @@ -0,0 +1,14 @@ +namespace CCVTAC.Main.Settings + +module TagFormat = + + /// Point versions of ID3 version 2 (such as 2.3 or 2.4). + type Id3V2Version = + | TwoPoint2 = 2 + | TwoPoint3 = 3 + | TwoPoint4 = 4 + + /// Locks the ID3v2.x version to a valid one and optionally forces that version. + let setId3V2Version (version: Id3V2Version) (forceAsDefault: bool) : unit = + TagLib.Id3v2.Tag.DefaultVersion <- byte version + TagLib.Id3v2.Tag.ForceDefaultVersion <- forceAsDefault diff --git a/src/CCVTAC.FSharp/Settings.fs b/src/CCVTAC.Main/Settings/Settings.fs similarity index 52% rename from src/CCVTAC.FSharp/Settings.fs rename to src/CCVTAC.Main/Settings/Settings.fs index 71c5a35f..23c8f4cd 100644 --- a/src/CCVTAC.FSharp/Settings.fs +++ b/src/CCVTAC.Main/Settings/Settings.fs @@ -1,12 +1,11 @@ -namespace CCVTAC.FSharp +namespace CCVTAC.Main.Settings -module Settings = - open System - open System.Text.Json.Serialization - - let newLine = Environment.NewLine +open System +open System.Text.Json.Serialization +open CCVTAC.Main +open Spectre.Console - type FilePath = FilePath of string +module Settings = type RenamePattern = { [] RegexPattern : string @@ -16,49 +15,76 @@ module Settings = type TagDetectionPattern = { [] RegexPattern : string - [] MatchGroup : byte + [] MatchGroup : int [] SearchField : string [] Summary : string option } type TagDetectionPatterns = { - [] Title : TagDetectionPattern array - [] Artist : TagDetectionPattern array - [] Album : TagDetectionPattern array - [] Composer : TagDetectionPattern array - [] Year : TagDetectionPattern array + [] Title : TagDetectionPattern list + [] Artist : TagDetectionPattern list + [] Album : TagDetectionPattern list + [] Composer : TagDetectionPattern list + [] Year : TagDetectionPattern list } type UserSettings = { [] WorkingDirectory: string [] MoveToDirectory: string [] HistoryFile: string - [] HistoryDisplayCount: byte - [] AudioFormats: string array + [] HistoryDisplayCount: int + [] AudioFormats: string list [] AudioQuality: byte [] SplitChapters: bool [] SleepSecondsBetweenDownloads: uint16 [] SleepSecondsBetweenURLs: uint16 [] QuietMode: bool [] EmbedImages: bool - [] DoNotEmbedImageUploaders: string array - [] IgnoreUploadYearUploaders: string array + [] DoNotEmbedImageUploaders: string list + [] IgnoreUploadYearUploaders: string list [] TagDetectionPatterns: TagDetectionPatterns - [] RenamePatterns: RenamePattern array + [] RenamePatterns: RenamePattern list [] NormalizationForm : string [] DownloaderUpdateCommand : string + [] DownloaderAdditionalOptions : string option } - [] - let summarize settings = + let private defaultSettings = + { + WorkingDirectory = String.Empty + MoveToDirectory = String.Empty + HistoryFile = String.Empty + HistoryDisplayCount = 25 + SplitChapters = true + SleepSecondsBetweenDownloads = 10us + SleepSecondsBetweenURLs = 15us + AudioFormats = [] + AudioQuality = 0uy + QuietMode = false + EmbedImages = true + DoNotEmbedImageUploaders = [] + IgnoreUploadYearUploaders = [] + TagDetectionPatterns = { + Title = [] + Artist = [] + Album = [] + Composer = [] + Year = [] + } + RenamePatterns = [] + NormalizationForm = "C" // Recommended for compatibility between Linux and macOS. + DownloaderUpdateCommand = String.Empty + DownloaderAdditionalOptions = None + } + + let summarize settings : (string * string) list = let onOrOff = function | true -> "ON" | false -> "OFF" - let pluralize (label: string) count = - if count = 1 - then $"{count} {label}" - else $"{count} {label}s" // Intentionally naive implementation + let simplePluralize label count = + String.pluralize label $"{label}s" count + |> sprintf "%d %s" count let tagDetectionPatternCount (patterns: TagDetectionPatterns) = patterns.Title.Length + @@ -75,48 +101,63 @@ module Settings = ("Embed images", onOrOff settings.EmbedImages) ("Quiet mode", onOrOff settings.QuietMode) ("Audio formats", String.Join(", ", settings.AudioFormats)) - ("Audio quality (10 up to 0)", settings.AudioQuality |> sprintf "%B") - ("Sleep between URLs", settings.SleepSecondsBetweenURLs |> int |> pluralize "second") - ("Sleep between downloads", settings.SleepSecondsBetweenDownloads |> int |> pluralize "second") - ("Ignore-upload-year channels", settings.IgnoreUploadYearUploaders.Length |> pluralize "channel") - ("Do-not-embed-image channels", settings.DoNotEmbedImageUploaders.Length |> pluralize "channel") - ("Tag-detection patterns", tagDetectionPatternCount settings.TagDetectionPatterns |> pluralize "pattern") - ("Rename patterns", settings.RenamePatterns.Length |> pluralize "pattern") + ("Audio quality (10 up to 0)", settings.AudioQuality |> sprintf "%d") + ("Sleep between URLs", settings.SleepSecondsBetweenURLs |> int |> simplePluralize "second") + ("Sleep between downloads", settings.SleepSecondsBetweenDownloads |> int |> simplePluralize "second") + ("Ignore-upload-year channels", settings.IgnoreUploadYearUploaders.Length |> simplePluralize "channel") + ("Do-not-embed-image channels", settings.DoNotEmbedImageUploaders.Length |> simplePluralize "channel") + ("Tag-detection patterns", tagDetectionPatternCount settings.TagDetectionPatterns |> simplePluralize "pattern") + ("Rename patterns", settings.RenamePatterns.Length |> simplePluralize "pattern") ] + let printSummary settings (printer: Printer) headerOpt : unit = + match headerOpt with + | Some h when String.hasText h -> printer.Info h + | _ -> () + + let table = Table() + table.Expand() |> ignore + table.Border <- TableBorder.HeavyEdge + table.BorderColor(Color.Grey27) |> ignore + table.AddColumns("Name", "Value") |> ignore + table.HideHeaders() |> ignore + table.Columns[1].Width <- 100 // Ensure maximum width. + + for description, value in summarize settings do + table.AddRow(description, value) |> ignore + + Printer.PrintTable table + module Validation = open System.IO let validate settings = - let isEmpty str = str |> String.IsNullOrWhiteSpace let dirMissing str = not (Directory.Exists str) // Source: https://github.com/yt-dlp/yt-dlp/?tab=readme-ov-file#post-processing-options - let supportedAudioFormats = [| "best"; "aac"; "alac"; "flac"; "m4a"; "mp3"; "opus"; "vorbis"; "wav" |] - let supportedNormalizationForms = [| "C"; "D"; "KC"; "KD" |] + let supportedAudioFormats = [ "best"; "aac"; "alac"; "flac"; "m4a"; "mp3"; "opus"; "vorbis"; "wav" ] + let supportedNormalizationForms = [ "C"; "D"; "KC"; "KD" ] - let validAudioFormat fmt = - supportedAudioFormats |> Array.contains fmt + let validAudioFormat fmt = supportedAudioFormats |> List.contains fmt match settings with - | { WorkingDirectory = d } when d |> isEmpty -> - Error "No working directory was specified." - | { WorkingDirectory = d } when d |> dirMissing -> - Error $"Working directory \"{d}\" is missing." - | { MoveToDirectory = d } when d |> isEmpty -> - Error "No move-to directory was specified." - | { MoveToDirectory = d } when d |> dirMissing -> - Error $"Move-to directory \"{d}\" is missing." - | { AudioQuality = q } when q > 10uy -> + | { WorkingDirectory = dir } when String.hasNoText dir -> + Error "No working directory was specified in the settings." + | { WorkingDirectory = dir } when dirMissing dir -> + Error $"Working directory \"{dir}\" is missing." + | { MoveToDirectory = dir } when String.hasNoText dir -> + Error "No move-to directory was specified in the settings." + | { MoveToDirectory = dir } when dirMissing dir -> + Error $"Move-to directory \"{dir}\" is missing." + | { AudioQuality = q } when q < 0uy || q > 10uy -> Error "Audio quality must be in the range 10 (lowest) and 0 (highest)." - | { NormalizationForm = nf } when not(supportedNormalizationForms |> Array.contains (nf.ToUpperInvariant())) -> + | { NormalizationForm = nf } when not(supportedNormalizationForms |> List.contains (nf.ToUpperInvariant())) -> let okFormats = String.Join(", ", supportedNormalizationForms) Error $"\"{nf}\" is an invalid normalization form. Use one of the following: {okFormats}." - | { AudioFormats = fmt } when not (fmt |> Array.forall (fun f -> f |> validAudioFormat)) -> + | { AudioFormats = fmt } when not (fmt |> List.forall validAudioFormat) -> let formats = String.Join(", ", fmt) let approved = supportedAudioFormats |> String.concat ", " - let nl = Environment.NewLine - Error $"Audio formats (\"%s{formats}\") include an unsupported audio format.{nl}Only the following supported formats: {approved}." + Error $"Audio formats (\"%s{formats}\") include an unsupported audio format.{String.newLine}Only the following supported formats: {approved}." | _ -> Ok settings @@ -127,105 +168,60 @@ module Settings = open System.Text.Encodings.Web open Validation - let deserialize<'a> (json: string) = + let deserialize<'a> (json: string) : Result<'a, string> = let options = JsonSerializerOptions() options.AllowTrailingCommas <- true options.ReadCommentHandling <- JsonCommentHandling.Skip - JsonSerializer.Deserialize<'a>(json, options) - - [] - let fileExists (FilePath path) = - match path |> File.Exists with - | true -> Ok() - | false -> Error $"File \"{path}\" does not exist." + try + match JsonSerializer.Deserialize<'a>(json, options) with + | null -> Error "Could not deserialize the settings JSON" + | s -> Ok s + with e -> Error e.Message - [] - let read (FilePath path) = + let read (fileInfo: FileInfo) : Result = try - path + fileInfo.FullName |> File.ReadAllText |> deserialize - |> validate + |> Result.bind validate with - | :? FileNotFoundException -> Error $"File \"{path}\" was not found." - | :? JsonException as e -> Error $"Parse error in \"{path}\": {e.Message}" - | e -> Error $"Unexpected error reading from \"{path}\": {e.Message}" + | :? FileNotFoundException -> Error $"File \"{fileInfo.FullName}\" was not found." + | :? JsonException as e -> Error $"Parse error in \"{fileInfo.FullName}\": {e.Message}" + | e -> Error $"Unexpected error reading from \"{fileInfo.FullName}\": {e.Message}" - [] - let private writeFile (FilePath file) settings = + let private writeFile (filePath: FileInfo) settings : Result = let unicodeEncoder = JavaScriptEncoder.Create UnicodeRanges.All let writeIndented = true let options = JsonSerializerOptions(WriteIndented = writeIndented, Encoder = unicodeEncoder) try let json = JsonSerializer.Serialize(settings, options) - (file, json) |> File.WriteAllText - Ok $"A new settings file was saved to \"{file}\". Please populate it with your desired settings." + File.WriteAllText(filePath.FullName, json) + Ok $"A new settings file was saved to \"{filePath.FullName}\". Please populate it with your desired settings." with - | :? FileNotFoundException -> Error $"File \"{file}\" was not found." + | :? FileNotFoundException -> Error $"File \"{filePath.FullName}\" was not found." | :? JsonException -> Error "Failure parsing user settings to JSON." - | e -> Error $"Failure writing \"{file}\": {e.Message}" - - [] - let writeDefaultFile (filePath: FilePath option) defaultFileName = - let confirmedPath = - match filePath with - | Some p -> p - | None -> FilePath (Path.Combine(AppContext.BaseDirectory, defaultFileName)) - - let defaultSettings = - { - WorkingDirectory = String.Empty - MoveToDirectory = String.Empty - HistoryFile = String.Empty - HistoryDisplayCount = 25uy // byte - SplitChapters = true - SleepSecondsBetweenDownloads = 10us - SleepSecondsBetweenURLs = 15us - AudioFormats = [||] - AudioQuality = 0uy - QuietMode = false - EmbedImages = true - DoNotEmbedImageUploaders = [||] - IgnoreUploadYearUploaders = [||] - TagDetectionPatterns = { - Title = [||] - Artist = [||] - Album = [||] - Composer = [||] - Year = [||] - } - RenamePatterns = [||] - NormalizationForm = "C" // Recommended for compatibility between Linux and macOS. - DownloaderUpdateCommand = String.Empty - } - - writeFile confirmedPath defaultSettings + | e -> Error $"Unexpected error writing \"{filePath.FullName}\": {e.Message}" + + let writeDefaultFile (fileInfo: FileInfo) : Result = + writeFile fileInfo defaultSettings module LiveUpdating = open Validation - [] let toggleSplitChapters settings = - let toggledSetting = not settings.SplitChapters - { settings with SplitChapters = toggledSetting } + { settings with SplitChapters = not settings.SplitChapters } - [] let toggleEmbedImages settings = - let toggledSetting = not settings.EmbedImages - { settings with EmbedImages = toggledSetting } + { settings with EmbedImages = not settings.EmbedImages } - [] let toggleQuietMode settings = - let toggledSetting = not settings.QuietMode - { settings with QuietMode = toggledSetting } + { settings with QuietMode = not settings.QuietMode } - [] - let updateAudioFormat settings newFormat = - let updatedSettings = { settings with AudioFormats = newFormat} + let updateAudioFormat settings (newFormat: string) = + let updatedSettings = { settings with AudioFormats = newFormat.Split ',' |> List.ofArray } validate updatedSettings - [] let updateAudioQuality settings newQuality = - let updatedSettings = { settings with AudioQuality = newQuality} + let updatedSettings = { settings with AudioQuality = newQuality } validate updatedSettings diff --git a/src/CCVTAC.Main/Shared.fs b/src/CCVTAC.Main/Shared.fs new file mode 100644 index 00000000..d607d8a0 --- /dev/null +++ b/src/CCVTAC.Main/Shared.fs @@ -0,0 +1,32 @@ +namespace CCVTAC.Main + +open System.Threading +open Spectre.Console + +[] +module Shared = + + type ResultMessageCollection = { Successes: string list; Failures: string list } + + /// Safely runs a function that might raise an exception. + /// If an exception is thrown, only returns its message. + let ofTry (f: unit -> 'a) : Result<'a, string> = + try Ok (f()) + with exn -> Error exn.Message + + let sleep workingMsgFn doneMsgFn seconds : string = + let rec loop remaining (ctx: StatusContext) = + if remaining > 0us then + ctx.Status (workingMsgFn remaining) |> ignore + Thread.Sleep 1000 + loop (remaining - 1us) ctx + + AnsiConsole + .Status() + .Start((workingMsgFn seconds), fun ctx -> + ctx.Spinner(Spinner.Known.Star) + .SpinnerStyle(Style.Parse "blue") + |> loop seconds) + + doneMsgFn seconds + diff --git a/src/CCVTAC.FSharp.Tests/CCVTAC.FSharp.Tests.fsproj b/src/CCVTAC.Tests/CCVTAC.Tests.fsproj similarity index 84% rename from src/CCVTAC.FSharp.Tests/CCVTAC.FSharp.Tests.fsproj rename to src/CCVTAC.Tests/CCVTAC.Tests.fsproj index bb09d563..3ad3e9f7 100644 --- a/src/CCVTAC.FSharp.Tests/CCVTAC.FSharp.Tests.fsproj +++ b/src/CCVTAC.Tests/CCVTAC.Tests.fsproj @@ -8,6 +8,9 @@ + + + @@ -23,6 +26,6 @@ - + diff --git a/src/CCVTAC.FSharp.Tests/DownloadEntityTests.fs b/src/CCVTAC.Tests/DownloadEntityTests.fs similarity index 90% rename from src/CCVTAC.FSharp.Tests/DownloadEntityTests.fs rename to src/CCVTAC.Tests/DownloadEntityTests.fs index 9a7541ef..8b5eb598 100644 --- a/src/CCVTAC.FSharp.Tests/DownloadEntityTests.fs +++ b/src/CCVTAC.Tests/DownloadEntityTests.fs @@ -1,184 +1,184 @@ -module DownloadEntityTests - -open Xunit -open CCVTAC.FSharp.Downloading - -module MediaTypeWithIdsTests = - let incorrectMediaType = "Incorrect media type" - let unexpectedError e = $"Unexpected error: {e}" - let unexpectedOk = "Unexpectedly parsed a MediaType" - - [] - let ``Detects video URL with its ID`` () = - let url = "https://www.youtube.com/watch?v=12312312312" - let expectedId = "12312312312" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | Video actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects playlist video URL with its ID`` () = - let url = "https://www.youtube.com/watch?v=12312312312&list=OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg&index=1" - let expectedVideoId = "12312312312" - let expectedPlaylistId = "OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | PlaylistVideo (actualVideoId, actualPlaylistId) -> - Assert.Equal(expectedVideoId, actualVideoId) - Assert.Equal(expectedPlaylistId, actualPlaylistId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects standard playlist URL with its ID`` () = - let url = "https://www.youtube.com/playlist?list=PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L&index=1" - let expectedId = "PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | StandardPlaylist actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects release playlist URL with its ID`` () = - let url = "https://www.youtube.com/playlist?list=OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L&index=1" - let expectedId = "OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | ReleasePlaylist actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects channel URL type 1 with its ID`` () = - let url = "https://www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" - let expectedId = "www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | Channel actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects channel URL type 2 with its ID`` () = - let url = "https://www.youtube.com/@NicknameBasedYouTubeChannelName" - let expectedId = "www.youtube.com/@NicknameBasedYouTubeChannelName" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | Channel actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects channel URL type 2 with encoded Japanese characters`` () = - let url = "https://www.youtube.com/@%E3%81%8A%E3%81%91%E3%83%91%E3%83%A9H" - let expectedId = "www.youtube.com/@%E3%81%8A%E3%81%91%E3%83%91%E3%83%A9H" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | Channel actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects channel videos URL with encoded Japanese characters`` () = - let url = "https://www.youtube.com/@%E3%81%8A%91%E3%83%A9H/videos" - let expectedId = "www.youtube.com/@%E3%81%8A%91%E3%83%A9H/videos" - let result = mediaTypeWithIds url - - match result with - | Ok mediaType -> match mediaType with - | Channel actualId -> Assert.Equal(expectedId, actualId) - | _ -> failwith incorrectMediaType - | Error e -> failwith (unexpectedError e) - - [] - let ``Detects unsupported channel URL type 2 with unencoded Japanese characters`` () = - let url = "https://www.youtube.com/@日本語" - let result = mediaTypeWithIds url - - match result with - | Error _ -> Assert.True true - | Ok _ -> Assert.True (false, unexpectedOk) - - [] - let ``Detects unsupported channel videos URL with unencoded Japanese characters`` () = - let url = "https://www.youtube.com/@日本語/videos" - let result = mediaTypeWithIds url - - match result with - | Error _ -> Assert.True true - | Ok _ -> Assert.True (false, unexpectedOk) - - [] - let ``Error result when an invalid URL is passed`` () = - let url = "INVALID URL" - let result = mediaTypeWithIds url - - match result with - | Error _ -> Assert.True true - | Ok _ -> Assert.True (false, unexpectedOk) - -module DownloadUrlsTests = - [] - let ``Generates expected URL for video`` () = - let video = Video "12312312312" - let expectedUrl = ["https://www.youtube.com/watch?v=12312312312"] - let result = extractDownloadUrls video - Assert.Equal(result.Length, 1) - Assert.Equal(expectedUrl.Length, result.Length) - Assert.Equal(expectedUrl.Head, result.Head) - - [] - let ``Generates expected URL pair for playlist video`` () = - let playlistVideo = PlaylistVideo ("12312312312", "OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg") - let videoUrl = "https://www.youtube.com/watch?v=12312312312" - let playlistUrl = "https://www.youtube.com/playlist?list=OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg" - let expectedUrls = [videoUrl; playlistUrl] - let result = extractDownloadUrls playlistVideo - Assert.Equal(result.Length, 2) - Assert.Equal(expectedUrls.Length, result.Length) - Assert.Equal(expectedUrls.Head, result.Head) - Assert.Equal(expectedUrls[1], result[1]) - - [] - let ``Generates expected URL for standard playlist`` () = - let sPlaylist = StandardPlaylist "PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" - let expectedUrls = ["https://www.youtube.com/playlist?list=PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L"] - let result = extractDownloadUrls sPlaylist - Assert.Equal(result.Length, 1) - Assert.Equal(expectedUrls.Length, result.Length) - Assert.Equal(expectedUrls.Head, result.Head) - - [] - let ``Generates expected URL for release playlist`` () = - let rPlaylist = ReleasePlaylist "OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" - let expectedUrls = ["https://www.youtube.com/playlist?list=OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L"] - let result = extractDownloadUrls rPlaylist - Assert.Equal(result.Length, 1) - Assert.Equal(expectedUrls.Length, result.Length) - Assert.Equal(expectedUrls.Head, result.Head) - - [] - let ``Generates expected URL for channel`` () = - let channel = Channel "www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" - let expectedUrls = ["https://www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg"] - let result = extractDownloadUrls channel - Assert.Equal(result.Length, 1) - Assert.Equal(expectedUrls.Length, result.Length) - Assert.Equal(expectedUrls.Head, result.Head) +module DownloadEntityTests + +open Xunit +open CCVTAC.Main.Downloading.Downloading + +module MediaTypeWithIdsTests = + let incorrectMediaType = "Incorrect media type" + let unexpectedError e = $"Unexpected error: {e}" + let unexpectedOk = "Unexpectedly parsed a MediaType" + + [] + let ``Detects video URL with its ID`` () = + let url = "https://www.youtube.com/watch?v=12312312312" + let expectedId = "12312312312" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | Video actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects playlist video URL with its ID`` () = + let url = "https://www.youtube.com/watch?v=12312312312&list=OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg&index=1" + let expectedVideoId = "12312312312" + let expectedPlaylistId = "OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | PlaylistVideo (actualVideoId, actualPlaylistId) -> + Assert.Equal(expectedVideoId, actualVideoId) + Assert.Equal(expectedPlaylistId, actualPlaylistId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects standard playlist URL with its ID`` () = + let url = "https://www.youtube.com/playlist?list=PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L&index=1" + let expectedId = "PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | StandardPlaylist actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects release playlist URL with its ID`` () = + let url = "https://www.youtube.com/playlist?list=OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L&index=1" + let expectedId = "OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | ReleasePlaylist actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects channel URL type 1 with its ID`` () = + let url = "https://www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" + let expectedId = "www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | Channel actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects channel URL type 2 with its ID`` () = + let url = "https://www.youtube.com/@NicknameBasedYouTubeChannelName" + let expectedId = "www.youtube.com/@NicknameBasedYouTubeChannelName" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | Channel actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects channel URL type 2 with encoded Japanese characters`` () = + let url = "https://www.youtube.com/@%E3%81%8A%E3%81%E3%81%8A%E3%81%E3%81%8A%E3%81%E3%81%8A%E3%81" + let expectedId = "www.youtube.com/@%E3%81%8A%E3%81%E3%81%8A%E3%81%E3%81%8A%E3%81%E3%81%8A%E3%81" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | Channel actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects channel videos URL with encoded Japanese characters`` () = + let url = "https://www.youtube.com/@%E3%81%8A%91%E3%83%A9HJ/videos" + let expectedId = "www.youtube.com/@%E3%81%8A%91%E3%83%A9HJ/videos" + let result = mediaTypeWithIds url + + match result with + | Ok mediaType -> match mediaType with + | Channel actualId -> Assert.Equal(expectedId, actualId) + | _ -> failwith incorrectMediaType + | Error e -> failwith (unexpectedError e) + + [] + let ``Detects unsupported channel URL type 2 with unencoded Japanese characters`` () = + let url = "https://www.youtube.com/@日本の文字" + let result = mediaTypeWithIds url + + match result with + | Error _ -> Assert.True true + | Ok _ -> Assert.True (false, unexpectedOk) + + [] + let ``Detects unsupported channel videos URL with unencoded Japanese characters`` () = + let url = "https://www.youtube.com/@日本語の文字/videos" + let result = mediaTypeWithIds url + + match result with + | Error _ -> Assert.True true + | Ok _ -> Assert.True (false, unexpectedOk) + + [] + let ``Error result when an invalid URL is passed`` () = + let url = "INVALID URL" + let result = mediaTypeWithIds url + + match result with + | Error _ -> Assert.True true + | Ok _ -> Assert.True (false, unexpectedOk) + +module DownloadUrlsTests = + [] + let ``Generates expected URL for video`` () = + let video = Video "12312312312" + let expectedUrl = ["https://www.youtube.com/watch?v=12312312312"] + let result = generateDownloadUrl video + Assert.Equal(result.Length, 1) + Assert.Equal(expectedUrl.Length, result.Length) + Assert.Equal(expectedUrl.Head, result.Head) + + [] + let ``Generates expected URL pair for playlist video`` () = + let playlistVideo = PlaylistVideo ("12312312312", "OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg") + let videoUrl = "https://www.youtube.com/watch?v=12312312312" + let playlistUrl = "https://www.youtube.com/playlist?list=OLZK5uy_kgsbf_bzaknqCjNbb2BtnfylIvHdNlKzg" + let expectedUrls = [videoUrl; playlistUrl] + let result = generateDownloadUrl playlistVideo + Assert.Equal(result.Length, 2) + Assert.Equal(expectedUrls.Length, result.Length) + Assert.Equal(expectedUrls.Head, result.Head) + Assert.Equal(expectedUrls[1], result[1]) + + [] + let ``Generates expected URL for standard playlist`` () = + let sPlaylist = StandardPlaylist "PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" + let expectedUrls = ["https://www.youtube.com/playlist?list=PLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L"] + let result = generateDownloadUrl sPlaylist + Assert.Equal(result.Length, 1) + Assert.Equal(expectedUrls.Length, result.Length) + Assert.Equal(expectedUrls.Head, result.Head) + + [] + let ``Generates expected URL for release playlist`` () = + let rPlaylist = ReleasePlaylist "OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L" + let expectedUrls = ["https://www.youtube.com/playlist?list=OLaB53ktYgG5CBaIe-otRu41Wop8Ji8C2L"] + let result = generateDownloadUrl rPlaylist + Assert.Equal(result.Length, 1) + Assert.Equal(expectedUrls.Length, result.Length) + Assert.Equal(expectedUrls.Head, result.Head) + + [] + let ``Generates expected URL for channel`` () = + let channel = Channel "www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg" + let expectedUrls = ["https://www.youtube.com/channel/UBMmt12UKW571UWtJAgWkWrg"] + let result = generateDownloadUrl channel + Assert.Equal(result.Length, 1) + Assert.Equal(expectedUrls.Length, result.Length) + Assert.Equal(expectedUrls.Head, result.Head) diff --git a/src/CCVTAC.Tests/ExtensionsTests.fs b/src/CCVTAC.Tests/ExtensionsTests.fs new file mode 100644 index 00000000..506958c5 --- /dev/null +++ b/src/CCVTAC.Tests/ExtensionsTests.fs @@ -0,0 +1,288 @@ +module ExtensionsTests + +open CCVTAC.Main +open Xunit +open System + +module NumericsTests = + open CCVTAC.Main.Numerics + + [] + let ``isZero returns true for any zero value`` () = + Assert.True <| isZero 0 + Assert.True <| isZero 0u + Assert.True <| isZero 0us + Assert.True <| isZero 0. + Assert.True <| isZero 0L + Assert.True <| isZero 0m + Assert.True <| isZero -0 + Assert.True <| isZero -0. + Assert.True <| isZero -0L + Assert.True <| isZero -0m + + [] + let ``isZero returns false for any non-zero value`` () = + Assert.False <| isZero 1 + Assert.False <| isOne -1 + Assert.False <| isOne Int64.MinValue + Assert.False <| isOne Int64.MaxValue + Assert.False <| isOne 2 + Assert.False <| isZero 1u + Assert.False <| isZero 1us + Assert.False <| isZero -0.0000000000001 + Assert.False <| isZero 0.0000000000001 + Assert.False <| isZero 1. + Assert.False <| isZero 1L + Assert.False <| isZero 1m + + [] + let ``isOne returns true for any one value`` () = + Assert.True <| isOne 1 + Assert.True <| isOne 1u + Assert.True <| isOne 1us + Assert.True <| isOne 1. + Assert.True <| isOne 1L + Assert.True <| isOne 1m + + [] + let ``isOne returns false for any non-one value`` () = + Assert.False <| isOne 0 + Assert.False <| isOne -1 + Assert.False <| isOne Int64.MinValue + Assert.False <| isOne Int64.MaxValue + Assert.False <| isOne 2 + Assert.False <| isOne 0u + Assert.False <| isOne 16u + Assert.False <| isOne 0us + Assert.False <| isOne -0. + Assert.False <| isOne 0.001 + Assert.False <| isOne 0L + Assert.False <| isOne 0m + + module FormatNumberTests = + + // A tiny custom type that implements the required ToString signature. + type MyCustomNum(i: int) = + member _.ToString(fmt: string, provider: IFormatProvider) = + i.ToString(fmt, provider) + + [] + let ``format int`` () = + let actual = formatNumber 123456 + Assert.Equal("123,456", actual) + + [] + let ``format negative int`` () = + let actual = formatNumber -1234 + Assert.Equal("-1,234", actual) + + [] + let ``format zero`` () = + let actual = formatNumber 0 + Assert.Equal("0", actual) + + [] + let ``format int64`` () = + let actual = formatNumber 1234567890L + Assert.Equal("1,234,567,890", actual) + + [] + let ``format decimal rounds to integer display`` () = + let actual = formatNumber 123456.78M + Assert.Equal("123,457", actual) + + [] + let ``format float rounds to integer display`` () = + let actual = formatNumber 123456.78 + Assert.Equal("123,457", actual) + + [] + let ``format negative float rounds to integer display`` () = + let actual = formatNumber -1234.56 + Assert.Equal("-1,235", actual) + + [] + let ``format custom numeric type`` () = + let myNum = MyCustomNum 1234 + let actual = formatNumber myNum + Assert.Equal("1,234", actual) + +module StringTests = + open CCVTAC.Main.String + + [] + let ``fileLabel formats correctly`` () = + Assert.True <| (fileLabel 0 = "0 files") + Assert.True <| (fileLabel 1 = "1 file") + Assert.True <| (fileLabel 2 = "2 files") + Assert.True <| (fileLabel 1_000_000 = "1,000,000 files") + + [] + let ``fileLabelWithDescriptor formats correctly`` () = + Assert.True <| (fileLabelWithDescriptor "audio" 0 = "0 audio files") + Assert.True <| (fileLabelWithDescriptor " temporary " 1 = "1 temporary file") + Assert.True <| (fileLabelWithDescriptor "deleted" 2 = "2 deleted files") + Assert.True <| (fileLabelWithDescriptor "image" 1_000_000 = "1,000,000 image files") + + module ReplaceInvalidPathCharsTests = + open System.IO + + [] + let ``Default replacement '_' replaces invalid path chars`` () = + let invalids = Path.GetInvalidFileNameChars() |> Array.except ['\000'] + if Array.isEmpty invalids then + Assert.True(false, "Unexpected environment: not enough invalid filename chars") + let invalid = invalids[0] + let input = $"start%c{invalid}end" + let result = replaceInvalidPathChars None None input + + Assert.DoesNotContain(invalid.ToString(), result) + Assert.Contains("_", result) + Assert.StartsWith("start", result) + Assert.EndsWith("end", result) + + [] + let ``Custom invalid chars are replaced with provided replacement`` () = + let custom = ['#'; '%'] + let input = "abc#def%ghi" + let result = replaceInvalidPathChars (Some '-') (Some custom) input + + Assert.DoesNotContain("#", result) + Assert.DoesNotContain("%", result) + Assert.Equal("abc-def-ghi", result) + + [] + let ``Throws when replacement char itself is invalid`` () = + let invalidReplacement = Array.head <| Path.GetInvalidFileNameChars() + let input = "has-no-invalid-chars" + Assert.Throws(fun () -> + replaceInvalidPathChars (Some invalidReplacement) None input |> ignore) + + [] + let ``No invalid chars returns identical string`` () = + let input = "HelloWorld123" + let result = replaceInvalidPathChars None None input + Assert.Equal(input, result) + + [] + let ``All invalid path and filename chars are replaced`` () = + let fileInvalid = Path.GetInvalidFileNameChars() |> Array.truncate 3 + let pathInvalid = Path.GetInvalidPathChars() |> Array.truncate 3 + let extras = [| Path.PathSeparator + Path.DirectorySeparatorChar + Path.AltDirectorySeparatorChar + Path.VolumeSeparatorChar |] + let charsToTest = Array.concat [ fileInvalid; pathInvalid; extras ] + let input = String charsToTest + + let result = replaceInvalidPathChars None None input + + result.ToCharArray() |> Array.iter (fun ch -> Assert.Equal(ch, '_')) + Assert.Equal(input.Length, result.Length) + +module SeqTests = + + [] + let ``caseInsensitiveContains returns true when exact match exists`` () = + let input = ["Hello"; "World"; "Test"] + Assert.True <| Seq.caseInsensitiveContains "Hello" input + Assert.True <| Seq.caseInsensitiveContains "World" input + Assert.True <| Seq.caseInsensitiveContains "Test" input + + [] + let ``caseInsensitiveContains returns true when exists but case differs`` () = + let input = ["hello"; "WORLD"; "test"] + Assert.True <| Seq.caseInsensitiveContains "Hello" input + Assert.True <| Seq.caseInsensitiveContains "hello" input + Assert.True <| Seq.caseInsensitiveContains "HELLO" input + Assert.True <| Seq.caseInsensitiveContains "wOrLd" input + Assert.True <| Seq.caseInsensitiveContains "tESt" input + Assert.True <| Seq.caseInsensitiveContains "TEST" input + + [] + let ``caseInsensitiveContains returns false when text not in sequence`` () = + let input = ["Hello"; "World"; "Test"] + Assert.False <| Seq.caseInsensitiveContains "Missing" input + + [] + let ``caseInsensitiveContains works with empty sequence`` () = + Assert.False <| Seq.caseInsensitiveContains "Any" [] + + [] + let ``caseInsensitiveContains handles null or empty strings`` () = + let input = [String.Empty; null; "Test"] + Assert.True <| Seq.caseInsensitiveContains String.Empty input + Assert.True <| Seq.caseInsensitiveContains null input + + [] + let ``caseInsensitiveContains handles Japanese strings`` () = + let input = ["関数型プログラミング"; "楽しいぞ"] + Assert.True <| Seq.caseInsensitiveContains "関数型プログラミング" input + Assert.False <| Seq.caseInsensitiveContains "いや、楽しくないや" input + +module ListTests = + + [] + let ``caseInsensitiveContains returns true when exact match exists`` () = + let input = ["Hello"; "World"; "Test"] + Assert.True <| List.caseInsensitiveContains "Hello" input + Assert.True <| List.caseInsensitiveContains "World" input + Assert.True <| List.caseInsensitiveContains "Test" input + + [] + let ``caseInsensitiveContains returns true when exists but case differs`` () = + let input = ["hello"; "WORLD"; "test"] + Assert.True <| List.caseInsensitiveContains "Hello" input + Assert.True <| List.caseInsensitiveContains "hello" input + Assert.True <| List.caseInsensitiveContains "HELLO" input + Assert.True <| List.caseInsensitiveContains "wOrLd" input + Assert.True <| List.caseInsensitiveContains "tESt" input + Assert.True <| List.caseInsensitiveContains "TEST" input + + [] + let ``caseInsensitiveContains returns false when text not in sequence`` () = + let input = ["Hello"; "World"; "Test"] + Assert.False <| List.caseInsensitiveContains "Missing" input + + [] + let ``caseInsensitiveContains works with empty sequence`` () = + Assert.False <| List.caseInsensitiveContains "Any" [] + + [] + let ``caseInsensitiveContains handles null or empty strings`` () = + let input = [String.Empty; null; "Test"] + Assert.True <| List.caseInsensitiveContains String.Empty input + Assert.True <| List.caseInsensitiveContains null input + + [] + let ``caseInsensitiveContains handles Japanese strings`` () = + let input = ["関数型プログラミング"; "楽しいぞ"] + Assert.True <| List.caseInsensitiveContains "関数型プログラミング" input + Assert.False <| List.caseInsensitiveContains "いや、楽しくないや" input + +module ArrayTests = + + module HasMultiple = + + [] + let ``hasMultiple returns true for array with more than one element`` () = + Assert.True <| Array.hasMultiple [| 1; 2; 3 |] + + [] + let ``hasMultiple returns false for empty array`` () = + Assert.False <| Array.hasMultiple [||] + + [] + let ``hasMultiple returns false for single-element array`` () = + Assert.False <| Array.hasMultiple [| 0 |] + + [] + let ``hasMultiple works with different types of arrays`` () = + Assert.True <| Array.hasMultiple [| "hello"; "world" |] + Assert.True <| Array.hasMultiple [| 1.0; 2.0; 3.0 |] + Assert.True <| Array.hasMultiple [| false; true; true |] + Assert.True <| Array.hasMultiple [| Array.sum; Array.length |] + + [] + let ``hasMultiple handles large arrays`` () = + Assert.True <| Array.hasMultiple (Array.init 100 id) diff --git a/src/CCVTAC.FSharp.Tests/Program.fs b/src/CCVTAC.Tests/Program.fs similarity index 97% rename from src/CCVTAC.FSharp.Tests/Program.fs rename to src/CCVTAC.Tests/Program.fs index fdc31cde..0695f84c 100644 --- a/src/CCVTAC.FSharp.Tests/Program.fs +++ b/src/CCVTAC.Tests/Program.fs @@ -1 +1 @@ -module Program = let [] main _ = 0 +module Program = let [] main _ = 0 diff --git a/src/CCVTAC.Tests/RenamerTests.fs b/src/CCVTAC.Tests/RenamerTests.fs new file mode 100644 index 00000000..84535978 --- /dev/null +++ b/src/CCVTAC.Tests/RenamerTests.fs @@ -0,0 +1,46 @@ +module RenamerTests + +open CCVTAC.Main +open CCVTAC.Main.Settings.Settings +open CCVTAC.Main.PostProcessing +open System +open System.Text +open Xunit + +module UpdateTextViaPatternsTests = + + [] + let ``Renames files per specified rename patterns`` () = + let patterns : RenamePattern list = + [ + { RegexPattern = "\s\[[\w_-]{11}\](?=\.\w{3,5})" + ReplaceWithPattern = String.Empty + Summary = "Remove trailing video IDs (run first)" } + { RegexPattern = "\s{2,}" + ReplaceWithPattern = " " + Summary = "Remove multiple spaces" } + { RegexPattern = " \(字幕\)" + ReplaceWithPattern = String.Empty + Summary = "Remove 字幕 label" } + { RegexPattern = "^(.+?)「(.+?)」（([12]\d{3})）" + ReplaceWithPattern = "%<1>s - %<2>s [%<3>s]" + Summary = "ARTIST「TITLE」（YEAR）" } + { RegexPattern = "\s+(?=\.\w{3,5})" + ReplaceWithPattern = String.Empty + Summary = "Remove trailing spaces before the file extension" } + ] + + let fileName = StringBuilder "ARTIST「TITLE」（1923）　　　 (字幕) [5B1rB894B1U].m4a" + + let expected = "ARTIST - TITLE [1923].m4a" + + let actual = + List.fold + (fun sb pattern -> Renamer.updateTextViaPatterns true (Printer false) sb pattern) + fileName + patterns + |> _.ToString() + + Assert.Equal(expected, actual) + + diff --git a/src/CCVTAC.Tests/TagDetectionTests.fs b/src/CCVTAC.Tests/TagDetectionTests.fs new file mode 100644 index 00000000..ed060049 --- /dev/null +++ b/src/CCVTAC.Tests/TagDetectionTests.fs @@ -0,0 +1,160 @@ +module TagDetectionTests + +open CCVTAC.Main +open CCVTAC.Main.PostProcessing.Tagging +open CCVTAC.Main.PostProcessing +open CCVTAC.Main.Settings.Settings +open System +open Xunit + +let emptyVideoMetadata = { + Id = String.Empty + Title = String.Empty + Thumbnail = String.Empty + Description = String.Empty + ChannelId = String.Empty + ChannelUrl = String.Empty + Duration = None + ViewCount = None + AgeLimit = None + WebpageUrl = String.Empty + Categories = [] + Tags = [] + PlayableInEmbed = None + LiveStatus = String.Empty + ReleaseTimestamp = None + FormatSortFields = [] + Album = String.Empty + Artist = String.Empty + Track = String.Empty + CommentCount = None + LikeCount = None + Channel = String.Empty + ChannelFollowerCount = None + ChannelIsVerified = None + Uploader = String.Empty + UploaderId = String.Empty + UploaderUrl = String.Empty + UploadDate = String.Empty + Creator = String.Empty + AltTitle = String.Empty + Availability = String.Empty + WebpageUrlBasename = String.Empty + WebpageUrlDomain = String.Empty + Extractor = String.Empty + ExtractorKey = String.Empty + PlaylistCount = None + Playlist = String.Empty + PlaylistId = String.Empty + PlaylistTitle = String.Empty + NEntries = None + PlaylistIndex = None + DisplayId = String.Empty + Fulltitle = String.Empty + DurationString = String.Empty + ReleaseDate = String.Empty + ReleaseYear = None + IsLive = None + WasLive = None + Epoch = None + Asr = None + Filesize = None + FormatId = String.Empty + FormatNote = String.Empty + SourcePreference = None + AudioChannels = None + Quality = None + HasDrm = None + Tbr = None + Url = String.Empty + LanguagePreference = None + Ext = String.Empty + Vcodec = String.Empty + Acodec = String.Empty + Container = String.Empty + Protocol = String.Empty + Resolution = String.Empty + AudioExt = String.Empty + VideoExt = String.Empty + Vbr = None + Abr = None + Format = String.Empty + Type = String.Empty +} + +let newLine = String.newLine + +[] +let ``Tag detection patterns detect metadata in video metadata`` () = + let testArtist = "Test Artist Name (日本語入り）" + let testAlbum = "Test Album Name (日本語入り）" + let testTitle = "Test Title (日本語入り）" + let testComposer = "Test Composer (日本語入り）" + let testYear = 1945u + + let videoMetadata = { + emptyVideoMetadata with + Title = $"{testArtist}「{testTitle}」" + Description = $"album: {testAlbum}{newLine}℗ %d{testYear}{newLine}Composed by: {testComposer}" } + + let artistPattern = { + RegexPattern = "^(.+?)「(.+)」" + MatchGroup = 1 + SearchField = "title" + Summary = Some "Find artist in video title" + } + + let titlePattern = { + artistPattern with + MatchGroup = 2 + Summary = Some "Find title in the video title" + } + + let albumPattern = { + RegexPattern = "(?<=[Aa]lbum: ).+" + MatchGroup = 0 + SearchField = "description" + Summary = Some "Find album in description" + } + + let composerPattern = { + RegexPattern = "(?<=[Cc]omposed by |[Cc]omposed by: |[Cc]omposer: |作曲[:：・]).+" + MatchGroup = 0 + SearchField = "description" + Summary = Some "Find composer in description" + } + + let yearPattern = { + RegexPattern = "(?<=℗ )[12]\d{3}" + MatchGroup = 0 + SearchField = "description" + Summary = Some "Find year in description" + } + + let tagDetectionPatterns = { + Title = [ titlePattern ] + Artist = [ artistPattern ] + Album = [ albumPattern ] + Composer = [ composerPattern ] + Year = [ yearPattern ] + } + + match TagDetection.detectArtist videoMetadata None tagDetectionPatterns with + | Some artistResult -> Assert.Equal(testArtist, artistResult) + | None -> Assert.Fail $"Expected artist \"{testArtist}\" was not found." + + match TagDetection.detectAlbum videoMetadata None tagDetectionPatterns with + | Some albumResult -> Assert.Equal(testAlbum, albumResult) + | None -> Assert.Fail $"Expected album \"{testAlbum}\" was not found." + + match TagDetection.detectTitle videoMetadata None tagDetectionPatterns with + | Some titleResult -> Assert.Equal(testTitle, titleResult) + | None -> Assert.Fail $"Expected title \"{testTitle}\" was not found." + + match TagDetection.detectComposers videoMetadata tagDetectionPatterns with + | Some composerResult -> Assert.Equal(testComposer, composerResult) + | None -> Assert.Fail $"Expected composer \"{testComposer}\" was not found." + + match TagDetection.detectReleaseYear videoMetadata None tagDetectionPatterns with + | Some yearResult -> Assert.Equal(testYear, yearResult) + | None -> Assert.Fail $"Expected year \"%d{testYear}\" was not found." diff --git a/src/CCVTAC.sln b/src/CCVTAC.sln index de22aa50..ab72356d 100644 --- a/src/CCVTAC.sln +++ b/src/CCVTAC.sln @@ -3,13 +3,9 @@ Microsoft Visual Studio Solution File, Format Version 12.00 # Visual Studio Version 16 VisualStudioVersion = 25.0.1706.0 MinimumVisualStudioVersion = 10.0.40219.1 -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CCVTAC.Console", "CCVTAC.Console\CCVTAC.Console.csproj", "{3A6C143A-C91A-43F8-8849-11C1DF100F7C}" +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "CCVTAC.Main", "CCVTAC.Main\CCVTAC.Main.fsproj", "{44860E27-2F04-428C-8444-C774627A019F}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "CCVTAC.Console.Tests", "CCVTAC.Console.Tests\CCVTAC.Console.Tests.csproj", "{8CA8220D-5E89-4F6D-9AA3-AFA534BC258B}" -EndProject -Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "CCVTAC.FSharp", "CCVTAC.FSharp\CCVTAC.FSharp.fsproj", "{44860E27-2F04-428C-8444-C774627A019F}" -EndProject -Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "CCVTAC.FSharp.Tests", "CCVTAC.FSharp.Tests\CCVTAC.FSharp.Tests.fsproj", "{D9BD0637-C4C1-435B-B202-1447FD85DCDE}" +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "CCVTAC.Tests", "CCVTAC.Tests\CCVTAC.Tests.fsproj", "{D9BD0637-C4C1-435B-B202-1447FD85DCDE}" EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution @@ -17,14 +13,6 @@ Global Release|Any CPU = Release|Any CPU EndGlobalSection GlobalSection(ProjectConfigurationPlatforms) = postSolution - {3A6C143A-C91A-43F8-8849-11C1DF100F7C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {3A6C143A-C91A-43F8-8849-11C1DF100F7C}.Debug|Any CPU.Build.0 = Debug|Any CPU - {3A6C143A-C91A-43F8-8849-11C1DF100F7C}.Release|Any CPU.ActiveCfg = Release|Any CPU - {3A6C143A-C91A-43F8-8849-11C1DF100F7C}.Release|Any CPU.Build.0 = Release|Any CPU - {8CA8220D-5E89-4F6D-9AA3-AFA534BC258B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {8CA8220D-5E89-4F6D-9AA3-AFA534BC258B}.Debug|Any CPU.Build.0 = Debug|Any CPU - {8CA8220D-5E89-4F6D-9AA3-AFA534BC258B}.Release|Any CPU.ActiveCfg = Release|Any CPU - {8CA8220D-5E89-4F6D-9AA3-AFA534BC258B}.Release|Any CPU.Build.0 = Release|Any CPU {44860E27-2F04-428C-8444-C774627A019F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {44860E27-2F04-428C-8444-C774627A019F}.Debug|Any CPU.Build.0 = Debug|Any CPU {44860E27-2F04-428C-8444-C774627A019F}.Release|Any CPU.ActiveCfg = Release|Any CPU