Goodbye Perl

DEVGUIDE.md

@@ -158,7 +158,6 @@ You can find all test options as separate flags. For example `build -testAll`:
   -testDesktop              Run tests against full .NET Framework
   -testCoreClr              Run tests against CoreCLR
   -testFSharpCore           Run FSharpCore unit tests
-  -testFSharpQA             Run F# Cambridge tests
   -testScripting            Run Scripting tests
   -testVs                   Run F# editor unit tests
 ```
@@ -230,7 +229,7 @@ Or if you are on Linux:
 ## Updating baselines in tests
 
 Some tests use "baseline" (.bsl) files.  There is sometimes a way to update these baselines en-masse in your local build,
-useful when some change affects many baselines.  For example, in the `fsharpqa` and `FSharp.Compiler.ComponentTests` tests the baselines
+useful when some change affects many baselines.  For example, in the `FSharp.Compiler.ComponentTests` tests the baselines
 are updated using scripts or utilities that allow the following environment variable to be set:
 
 Windows:

TESTGUIDE.md

@@ -20,7 +20,6 @@ build -testCompiler -c Release
 build -testCompilerService -c Release
 build -testCompilerComponentTests -c Release
 build -testCambridge -c Release -ci -nobl
-build -testFSharpQA -c Release -ci -nobl
 build -testFSharpCore -c Release
 build -testScripting -c Release
 build -testVs -c Release
@@ -36,7 +35,6 @@ build -testAll -c Release
 | testCoreClr  | Linux/Mac/Windows | Runs all .NetStandard and .NETCore tests in 64 bit processes, this includes tests from other groups |
 | testFSharpCore | Windows | Runs all test for FSharp.Core.dll |
 | testCambridge | Windows | Runs the Cambridge suite tests |
-| testFSharpQA  | Windows | Runs the FSharpQA tests, requires Perl |
 | testVS        | Windows + VS | Runs all VS integration tests |
 | testAOT       | Windows | Run AOT/Trimming tests |
 | testCompiler  | Windows | Runs a few quick compiler tests |
@@ -68,7 +66,6 @@ build -testCoreClr -c Release
 
 The following testsets open other windows and may interfere with you using your workstation, or change focus while you're doing something else:
 
-* FSharpQA
 * Cambridge
 
 ### Running tests online in CI
@@ -86,9 +83,8 @@ Finding the logs in the online CI results can be tricky, a small video can be fo
 
 ## Prerequisites
 
-The prerequisites are the same as for building the `FSharp.sln`, plus, at a minimum:
+The prerequisites are the same as for building the `FSharp.sln`, plus:
 
-* An installation of Perl, required for running FSharpQA tests
 * Run `git clean -xdf -e .vs` before running tests when:
   * Making changes to the lexer or parser
   * Between switching git branches
@@ -100,13 +96,11 @@ The F# tests are split as follows:
 
 * [FSharp Suite](tests/fsharp) - Older suite with broad coverage of mainline compiler and runtime scenarios.
 
-* [FSharpQA Suite](tests/fsharpqa/Source) - Broad and deep coverage of a variety of compiler, runtime, and syntax scenarios.
-
 * [FSharp.Core.UnitTests](tests/FSharp.Core.UnitTests) - Validation of the core F# types and the public surface area of `FSharp.Core.dll`.
 
 * [FSharp.Compiler.Service.Tests](tests/FSharp.Compiler.Service.Tests) - Validation of compiler internals.
 
-* [FSharp.Compiler.ComponentTests](tests/FSharp.Compiler.ComponentTests) - Validation of compiler APIs.
+* [FSharp.Compiler.ComponentTests](tests/FSharp.Compiler.ComponentTests) - Validation of compiler APIs and language conformance. This is the primary test suite for compiler functionality.
 
 * [VisualFSharp.UnitTests](vsintegration/tests/unittests) - Validation of a wide range of behaviors in the F# Visual Studio project system and language service (including the legacy one).
 
@@ -125,39 +119,12 @@ If the compilation and execution encounter no errors, the test is considered to
 
 There are also negative tests checking code expected to fail compilation. See note about baseline under "Other Tips" below for tests checking expectations against "baseline" (.bsl) files.
 
-### FSharpQA Suite
-
-The FSharpQA suite relies on [Perl](http://www.perl.org/get.html), StrawberryPerl package from https://strawberryperl.com.
-
-These tests use the `RunAll.pl` framework to execute, however the easiest way to run them is via the `.\build` script, see [usage examples](#quick-start-running-tests).
-
-Tests are grouped in folders per area. Each folder contains a number of source code files and a single `env.lst` file. The `env.lst` file defines a series of test cases, one per line.
-
-Each test case runs an optional "pre command," compiles a given set of source files using given flags, optionally runs the resulting binary, then optionally runs a final "post command".
-
-If all of these steps complete without issue, the test is considered to have passed.
-
-Read more at [tests/fsharpqa/readme.md](tests/fsharpqa/readme.md).
-
-#### Test lists
-
-For the FSharpQA suite, the list of test areas and their associated "tags" is stored at
-
-```shell
-tests\fsharpqa\source\test.lst   // FSharpQA suite
-```
-
-Tags are in the left column, paths to to corresponding test folders are in the right column.  If no tags are specified, all tests will be run.
-
-If you want to re-run a particular test area, the easiest way to do so is to set a temporary tag for that area in test.lst (e.g. "RERUN") and adjust `ttags` [run.fsharpqa.test.fsx script](tests/fsharpqa/run.fsharpqa.test.fsx) and run it.
-
 ### Logs and output
 
 All test execution logs and result files will be dropped into the `tests\TestResults` folder, and have file names matching
 
 ```shell
     net40-fsharp-suite-*.*
-    net40-fsharpqa-suite-*.*
     net40-compilerunit-suite-*.*
     net40-coreunit-suite-*.*
     vs-ideunit-suite-*.*
@@ -192,11 +159,7 @@ If you have the `VisualFSharp.sln` open, or if you recently debugged it through
 
 ### Finding the logs on CI
 
-Finding the proper logs in the CI system can be daunting, this video shows you where to look once you have an open PR. It shows you how to get the `FsharpQA` logs, but the same method applies to getting any other test logs.
-
-![b51e0ea3-e12b-4ee8-b26a-3b98c11dae33](https://user-images.githubusercontent.com/16015770/91355183-1a6ff900-e7ee-11ea-8fb4-e3627cc9b811.gif)
-
-The console output of the CI runs do not contain output of the FSharpQA tests, but for most other tests the console output contains enough info and can be found by clicking <kbd>Raw output</kbd> in the CI window, or clicking <kbd>download logs</kbd>:
+Finding the proper logs in the CI system can be daunting. The console output contains enough info for most tests and can be found by clicking <kbd>Raw output</kbd> in the CI window, or clicking <kbd>download logs</kbd>:
 
 ![download logs](https://user-images.githubusercontent.com/6309070/89307267-b9596900-d625-11ea-86e9-a1657ce2a368.png)
 
@@ -233,20 +196,13 @@ Some tests are known to fail on these older branches when run using one of the `
 * Adding the `-norestore` flag to the commandline speeds up the build part a little bit.
 * When using the `-ci` flag (mandatory for some testsets), adding the `-nobl` flag prevents creating the binary log files.
 
-Some tests run in parallel by default, or use a hosted compiler to speed things up:
-
-* The FSharp and FSharpQA suites will run test cases in parallel by default. You can comment out the relevant line (look for `PARALLEL_ARG`) to disable this.
-* By default, tests from the FSharpQA suite are run using a persistent, hosted version of the compiler. This speeds up test execution, as there is no need for the `fsc.exe` process to spin up repeatedly. To disable this, uncomment the relevant line (look for `HOSTED_COMPILER`).
-
 ## Solving common errors
 
 The following are common errors that users have encountered while running tests on their system.
 
 ### Error that a file cannot be accessed
 
-The build often leaves dangling processes like `HostedCompilerServer.exe`, `VBCSCompiler.exe` or `MSBuild.exe`. In [Process Explorer](https://learn.microsoft.com/sysinternals/downloads/process-explorer) you can see these processes having no parent process anymore. You can also use this to kill such processes. A typical error looks like and contains the process IDs (here 23152, 25252 and 24704):
-
-> C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(4364,5): error MSB3026: Could not copy "D:\Projects\FSharp\artifacts\bin\FSharp.Core\Debug\net45\FSharp.Core.dll" to "D:\Projects\FSharp\tests\fsharpqa\testenv\bin\FSharp.Core.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'D:\Projects\FSharp\tests\fsharpqa\testenv\bin\FSharp.Core.dll' because it is being used by another process. The file is locked by: "HostedCompilerServer (23152), HostedCompilerServer (25252), HostedCompilerServer (24704)" [D:\Projects\OpenSource\FSharp\tests\fsharpqa\testenv\src\ILComparer\ILComparer.fsproj]
+The build often leaves dangling processes like `VBCSCompiler.exe` or `MSBuild.exe`. In [Process Explorer](https://learn.microsoft.com/sysinternals/downloads/process-explorer) you can see these processes having no parent process anymore. You can also use this to kill such processes.
 
 ### StackOverflow exception
 
@@ -266,7 +222,6 @@ To get an idea of how long it may take, or how much coffee you'll need while wai
 | `-testDesktop` | 5 min | ? |
 | `-testCoreClr` | 36 min | ? |
 | `-testCambridge` | 72 min | 35 min |
-| `-testFSharpQA`  | 13 min | ? |
 | `-testCompiler` | 30 seconds | n/a |
 | `-testFSharpCore` | 2 min | ? |
 | `-testScripting` | 2 min | 1.5 min |

VisualFSharp.sln

@@ -105,8 +105,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TextFile", "vsintegration\I
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "XMLFile", "vsintegration\ItemTemplates\XMLFile\XMLFile.csproj", "{1FB1DD07-06AA-45B4-B5AC-20FF5BEE98B6}"
 EndProject
-Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "HostedCompilerServer", "tests\fsharpqa\testenv\src\HostedCompilerServer\HostedCompilerServer.fsproj", "{4239EFEA-E746-446A-BF7A-51FCBAB13946}"
-EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "FSharp.UIResources", "vsintegration\src\FSharp.UIResources\FSharp.UIResources.csproj", "{C4586A06-1402-48BC-8E35-A1B8642F895B}"
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CSharp_Analysis", "tests\service\data\CSharp_Analysis\CSharp_Analysis.csproj", "{887630A3-4B1D-40EA-B8B3-2D842E9C40DB}"
@@ -619,18 +617,6 @@ Global
 		{1FB1DD07-06AA-45B4-B5AC-20FF5BEE98B6}.Release|Any CPU.Build.0 = Release|Any CPU
 		{1FB1DD07-06AA-45B4-B5AC-20FF5BEE98B6}.Release|x86.ActiveCfg = Release|Any CPU
 		{1FB1DD07-06AA-45B4-B5AC-20FF5BEE98B6}.Release|x86.Build.0 = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Debug|x86.ActiveCfg = Debug|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Debug|x86.Build.0 = Debug|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Proto|Any CPU.ActiveCfg = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Proto|Any CPU.Build.0 = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Proto|x86.ActiveCfg = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Proto|x86.Build.0 = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Release|Any CPU.Build.0 = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Release|x86.ActiveCfg = Release|Any CPU
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946}.Release|x86.Build.0 = Release|Any CPU
 		{C4586A06-1402-48BC-8E35-A1B8642F895B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
 		{C4586A06-1402-48BC-8E35-A1B8642F895B}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{C4586A06-1402-48BC-8E35-A1B8642F895B}.Debug|x86.ActiveCfg = Debug|Any CPU
@@ -1119,7 +1105,6 @@ Global
 		{E3FDD4AC-46B6-4B9F-B672-317D1202CC50} = {F6DAEE9A-8BE1-4C4A-BC83-09215517C7DA}
 		{D11FC318-8F5D-4C8C-9287-AB40A016D13C} = {F6DAEE9A-8BE1-4C4A-BC83-09215517C7DA}
 		{1FB1DD07-06AA-45B4-B5AC-20FF5BEE98B6} = {F6DAEE9A-8BE1-4C4A-BC83-09215517C7DA}
-		{4239EFEA-E746-446A-BF7A-51FCBAB13946} = {CFE3259A-2D30-4EB0-80D5-E8B5F3D01449}
 		{C4586A06-1402-48BC-8E35-A1B8642F895B} = {4C7B48D7-19AF-4AE7-9D1D-3BB289D5480D}
 		{887630A3-4B1D-40EA-B8B3-2D842E9C40DB} = {35636A82-401A-4C3A-B2AB-EB7DC5E9C268}
 		{FF76BD3C-5E0A-4752-B6C3-044F6E15719B} = {35636A82-401A-4C3A-B2AB-EB7DC5E9C268}

azure-pipelines-PR.yml

@@ -436,11 +436,6 @@ stages:
                 _configuration: Release
                 _testKind: testCoreclr
                 transparentCompiler: # Empty display name part.
-              fsharpqa_release:
-                _configuration: Release
-                _testKind: testFSharpQA
-                FSharp_CacheEvictionImmediate: true
-                transparentCompiler:
               vs_release:
                 _configuration: Release
                 _testKind: testVs
@@ -481,7 +476,7 @@ stages:
               testResultsFiles: '*.xml'
               searchFolder: '$(Build.SourcesDirectory)/artifacts/TestResults/$(_configuration)'
             continueOnError: true
-            condition: succeededOrFailed() # ne(variables['_testKind'], 'testFSharpQA')
+            condition: succeededOrFailed()
           - task: PublishBuildArtifacts@1
             displayName: Publish Tests BinLog
             condition: always()

eng/Build.ps1

@@ -55,7 +55,6 @@ param (
     [switch]$testCompilerService,
     [switch]$testCompilerComponentTests,
     [switch]$testFSharpCore,
-    [switch]$testFSharpQA,
     [switch]$testIntegration,
     [switch]$testScripting,
     [switch]$testVs,
@@ -116,7 +115,6 @@ function Print-Usage() {
     Write-Host "  -testDesktop                  Run tests against full .NET Framework"
     Write-Host "  -testCoreClr                  Run tests against CoreCLR"
     Write-Host "  -testFSharpCore               Run FSharpCore unit tests"
-    Write-Host "  -testFSharpQA                 Run F# Cambridge tests"
     Write-Host "  -testIntegration              Run F# integration tests"
     Write-Host "  -testScripting                Run Scripting tests"
     Write-Host "  -testVs                       Run F# editor unit tests"
@@ -162,7 +160,6 @@ function Process-Arguments() {
     if ($testAll) {
         $script:testDesktop = $True
         $script:testCoreClr = $True
-        $script:testFSharpQA = $True
         $script:testIntegration = $True
         $script:testVs = $True
         $script:testAOT = $True
@@ -171,7 +168,6 @@ function Process-Arguments() {
     if ($testAllButIntegration) {
         $script:testDesktop = $True
         $script:testCoreClr = $True
-        $script:testFSharpQA = $True
         $script:testIntegration = $False
         $script:testVs = $True
         $script:testAOT = $True
@@ -180,7 +176,6 @@ function Process-Arguments() {
     if($testAllButIntegrationAndAot) {
         $script:testDesktop = $True
         $script:testCoreClr = $True
-        $script:testFSharpQA = $True
         $script:testIntegration = $False
         $script:testVs = $True
         $script:testEditor = $True
@@ -200,7 +195,6 @@ function Process-Arguments() {
         $script:testDesktop = $False
         $script:testCoreClr = $False
         $script:testFSharpCore = $False
-        $script:testFSharpQA = $False
         $script:testIntegration = $False
         $script:testVs = $False
         $script:testpack = $False
@@ -546,13 +540,6 @@ try {
 
     $nativeTools = InitializeNativeTools
 
-    if (-not (Test-Path variable:NativeToolsOnMachine)) {
-        $env:PERL5Path = Join-Path $nativeTools "perl\5.38.2.2\perl\bin\perl.exe"
-        write-host "variable:NativeToolsOnMachine = unset or false"
-        $nativeTools
-        write-host "Path = $env:PERL5Path"
-    }
-
     $dotnetPath = InitializeDotNetCli
     $env:DOTNET_ROOT = "$dotnetPath"
     Get-Item -Path Env:
@@ -609,33 +596,6 @@ try {
         TestUsingMSBuild -testProject "$RepoRoot\FSharp.sln" -targetFramework $script:desktopTargetFramework
     }
 
-    if ($testFSharpQA) {
-        Push-Location "$RepoRoot\tests\fsharpqa\source"
-        $nugetPackages = Get-PackagesDir
-        $resultsRoot = "$ArtifactsDir\TestResults\$configuration"
-        $resultsLog = "test-net40-fsharpqa-results.log"
-        $errorLog = "test-net40-fsharpqa-errors.log"
-        $failLog = "test-net40-fsharpqa-errors"
-        Create-Directory $resultsRoot
-        UpdatePath
-        $env:HOSTED_COMPILER = 1
-        $env:CSC_PIPE = "$nugetPackages\Microsoft.Net.Compilers\4.3.0-1.22220.8\tools\csc.exe"
-        $env:FSCOREDLLPATH = "$ArtifactsDir\bin\fsc\$configuration\$script:desktopTargetFramework\FSharp.Core.dll"
-        $env:LINK_EXE = "$RepoRoot\tests\fsharpqa\testenv\bin\link\link.exe"
-        $env:OSARCH = $env:PROCESSOR_ARCHITECTURE
-
-        if (-not (Test-Path variable:NativeToolsOnMachine)) {
-            Exec-Console $env:PERL5Path """$RepoRoot\tests\fsharpqa\testenv\bin\runall.pl"" -resultsroot ""$resultsRoot"" -results $resultsLog -log $errorLog -fail $failLog -cleanup:no -procs:$env:NUMBER_OF_PROCESSORS"
-        }
-        else
-        {
-            Exec-Console "perl.exe" """$RepoRoot\tests\fsharpqa\testenv\bin\runall.pl"" -resultsroot ""$resultsRoot"" -results $resultsLog -log $errorLog -fail $failLog -cleanup:no -procs:$env:NUMBER_OF_PROCESSORS"
-        }
-
-        write-host "Exec-Console finished"
-        Pop-Location
-    }
-
     if ($testFSharpCore) {
         TestUsingMSBuild -testProject "$RepoRoot\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj" -targetFramework $script:coreclrTargetFramework
         TestUsingMSBuild -testProject "$RepoRoot\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj" -targetFramework $script:desktopTargetFramework

global.json

@@ -18,9 +18,6 @@
     },
     "xcopy-msbuild": "18.0.0"
   },
-  "native-tools": {
-    "perl": "5.38.2.2"
-  },
   "msbuild-sdks": {
     "Microsoft.DotNet.Arcade.Sdk": "10.0.0-beta.26062.3",
     "Microsoft.DotNet.Helix.Sdk": "8.0.0-beta.23255.2"

src/Compiler/Legacy/LegacyHostedCompilerForTesting.fs

@@ -1,6 +1,6 @@
 // Copyright (c) Microsoft Corporation.  All Rights Reserved.  See License.txt in the project root for license information.
 
-// This component is used by the 'fsharpqa' tests for faster in-memory compilation.  It should be removed and the
+// This component is used for faster in-memory compilation in some tests.  It should be removed and the
 // proper compiler service API used instead.
 
 namespace FSharp.Compiler.CodeAnalysis.Hosted