📘 How to Run Tests in a CI/CD Pipeline using VSTest in Azure DevOps

This guide provides a step-by-step walkthrough for setting up automated tests in your CI/CD pipeline using the built-in VSTest@3 task in Azure DevOps.

---

🤔 What is VSTest@3?

VSTest@3 is an Azure DevOps Pipelines task that runs automated tests as part of your CI/CD workflow. It leverages Microsoft's VSTest.Console.exe to execute unit tests in Visual Studio projects.

In a CI/CD context, VSTest@3 provides:

• ✅ Automatic discovery of test DLLs (using wildcards such as *Tests.dll)
• ✅ Remote test execution on both hosted and self-hosted build agents
• ✅ Generation of detailed test reports in the .trx format that integrate with Azure DevOps reporting
• ✅ Advanced configurations including:
  • Parallel test execution
  • Filtering or excluding specific tests
  • Collecting code coverage data
  • Diagnostic logging

---

📌 How is VSTest@3 Different from dotnet test?

The following table compares features between dotnet test (typically used in VS Code) and VSTest@3 (designed for Azure DevOps):

Feature	dotnet test (VS Code)	VSTest@3 (Azure DevOps)
✅ Runs tests?	Yes	Yes
🚫 Runs in CI/CD?	No (intended for local development)	Yes (specifically for pipelines)
✅ Automatically finds tests?	Yes	No (requires specifying *Tests.dll)
🚫 Generates test reports?	No (by default)	Yes (produces .trx reports for Azure DevOps)
⚠️ Can exclude tests?	Limited options	Yes (supports testFiltercriteria)
🚫 Supports code coverage?	No (by default)	Yes (codeCoverageEnabled: true)
⚠️ Supports parallel execution?	Limited	Yes

---

Step 1: 🗂️ Organize Your Repository Structure

A well-structured repository is essential for maintainability and testability. Below are some guidelines for organizing your projects.

📁 Add Deployment and Pipeline Structure

In the root directory of your repository:

1. Create a deployments/ folder to store all DevOps-related assets.
2. Inside deployments/, create a folder for your Logic App (e.g., MyLogicAppDeployment).
3. Within that folder, create a subfolder named pipelines/.
4. Place your CI configuration YAML file (e.g., CI-Pipeline.yml) inside the pipelines/ directory.

Example structure:

deployments/
└── MyLogicAppDeployment/
    └── pipelines/
        └── CI-Pipeline.yml

This structure promotes separation of concerns, simplifies CI configuration management, and scales well for multiple workflows or environments.

🔧 Project Layout Guidelines

• Separate projects: Keep your Logic App project and Unit Test project in distinct directories.

├── MyLogicApp/                      # Azure Logic App Standard project
│   ├── .vscode/                    
│   ├── Artifacts/                  
│   ├── lib/                        
│   ├── MyWorkflowSample/           # Workflow definitions folder
│   │   └── workflow.json           # Workflow JSON (triggers, actions, etc.)
│   ├── workflow-designtime/       
│   │   ├── host.json               
│   │   └── local.settings.json     
│   ├── .funcignore                 
│   ├── .gitignore                  
│   ├── connections.json            
│   ├── host.json                   
│   ├── local.settings.json         
│   └── parameters.json             
├── Tests/                           # Unit and integration test project
│   ├── MyLogicAppTestProject/       # Main test project folder
│   │   ├── bin/                     # Compiled binary output (autogenerated)
│   │   ├── obj/                     # Intermediate build files (autogenerated)
│   │   ├── MySampleWorkflow/        # Grouping tests for a specific workflow
│   │   │   └── MockOutputs/         # Contains mock data for test execution
│   │   │   └── MyWorkflowTest.cs    # C# test class for workflow testing
│   ├── TestExecutor.cs              # Shared utility for executing tests
│   ├── MyLogicAppTestProject.csproj # Project file for the test project
│   └── Tests.sln                    # Visual Studio solution file for tests
├── deployments/                     # CI/CD deployment folder
│   ├── MyLogicAppDeployment/        # Deployment artifacts for the Logic App
│   │   ├── pipelines/               # CI/CD pipeline definitions
│   │   │   └── CI-Pipeline.yml      # Pipeline configuration file

Note: Ensure that the unit test project references the main Logic App project to access workflow definitions and shared components.

---

Creating the Azure DevOps Pipeline (YAML)

Follow these steps to create your azure-pipelines.yml file:

1. Define the Trigger

Specify which branches will trigger the pipeline:

trigger:
  branches:
    include:
      - main              # Run pipeline on push to main branch
      - devFeatureBranch  # Run pipeline on push to feature branches

2. Define the Build Agent

Select a Microsoft-hosted agent with the latest Windows environment:

pool:
  vmImage: 'windows-latest'  # Latest Windows VM for build/test

3. Set Up the .NET SDK

Install the required .NET SDK to ensure the correct build environment:

steps:
- task: UseDotNet@2
  displayName: 'Install .NET SDK'
  inputs:
    packageType: 'sdk'
    version: '6.x'  # Adjust to your project’s .NET version

4. Restore Dependencies

Restore all project dependencies to ensure necessary packages are available:

- script: dotnet restore
  displayName: 'Restore NuGet Packages'
  workingDirectory: ./Tests

5. Build the Project

Compile the solution in Release mode. The --no-restore flag is used since dependencies have been restored:

- script: dotnet build --configuration Release --no-restore
  displayName: 'Build Solution'
  workingDirectory: ./Tests

6. Run Unit Tests Using VSTest@3

Configure the pipeline to run your unit tests:

- task: VSTest@3
  displayName: 'Run Unit Tests'
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: '**/*Tests.dll'    # Runs all test assemblies
    searchFolder: '$(Build.SourcesDirectory)/LogicAppProject/tests'  # Update as necessary
    codeCoverageEnabled: true
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'

Explanation:

• Test Selection: Uses a wildcard (*Tests.dll) to discover test assemblies.
• Search Folder: Specifies where test assemblies are located.
• Code Coverage: Enables code coverage reporting to provide insight into test coverage.

7. Publish Test Results

Finally, publish the test results so that they appear in the Azure DevOps pipeline summary:

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: 'VSTest'
    testResultsFiles: '**/TestResults/*.trx'
    mergeTestResults: true

---