diff --git a/.github/instructions/generic.instructions.md b/.github/instructions/generic.instructions.md index cf8498c6..9285662f 100644 --- a/.github/instructions/generic.instructions.md +++ b/.github/instructions/generic.instructions.md @@ -43,3 +43,8 @@ Provide project context and coding guidelines that AI should follow when generat - When using `getConfiguration().inspect()`, always pass a scope/Uri to `getConfiguration(section, scope)` — otherwise `workspaceFolderValue` will be `undefined` because VS Code doesn't know which folder to inspect (1) - **path.normalize() vs path.resolve()**: On Windows, `path.normalize('\test')` keeps it as `\test`, but `path.resolve('\test')` adds the current drive → `C:\test`. When comparing paths, use `path.resolve()` on BOTH sides or they won't match (2) +- **Path comparisons vs user display**: Use `normalizePath()` from `pathUtils.ts` when comparing paths or using them as map keys, but preserve original paths for user-facing output like settings, logs, and UI (1) +- **CI test jobs need webpack build**: Smoke/E2E/integration tests run in a real VS Code instance against `dist/extension.js` (built by webpack). CI jobs must run `npm run compile` (webpack), not just `npm run compile-tests` (tsc). Without webpack, the extension code isn't built and tests run against stale/missing code (1) +- **Use inspect() for setting checks with defaults from other extensions**: When checking `python.useEnvironmentsExtension`, use `config.inspect()` and only check explicit user values (`globalValue`, `workspaceValue`, `workspaceFolderValue`). Ignore `defaultValue` as it may come from other extensions' package.json even when not installed (1) +- **API is flat, not nested**: Use `api.getEnvironments()`, NOT `api.environments.getEnvironments()`. The extension exports a flat API object (1) +- **PythonEnvironment has `envId`, not `id`**: The environment identifier is `env.envId` (a `PythonEnvironmentId` object with `id` and `managerId`), not a direct `id` property (1) diff --git a/.github/skills/debug-failing-test/SKILL.md b/.github/skills/debug-failing-test/SKILL.md new file mode 100644 index 00000000..e9b5983f --- /dev/null +++ b/.github/skills/debug-failing-test/SKILL.md @@ -0,0 +1,89 @@ +--- +name: debug-failing-test +description: Debug a failing test using an iterative logging approach, then clean up and document the learning. +--- + +Debug a failing unit test by iteratively adding verbose logging, running the test, and analyzing the output until the root cause is found and fixed. + +## Workflow + +### Phase 1: Initial Assessment + +1. **Run the failing test** to capture the current error message and stack trace +2. **Read the test file** to understand what is being tested +3. **Read the source file** being tested to understand the expected behavior +4. **Identify the assertion that fails** and what values are involved + +### Phase 2: Iterative Debugging Loop + +Repeat until the root cause is understood: + +1. **Add verbose logging** around the suspicious code: + - Use `console.log('[DEBUG]', ...)` with descriptive labels + - Log input values, intermediate states, and return values + - Log before/after key operations + - Add timestamps if timing might be relevant + +2. **Run the test** and capture output + +3. **Assess the logging output:** + - What values are unexpected? + - Where does the behavior diverge from expectations? + - What additional logging would help narrow down the issue? + +4. **Decide next action:** + - If root cause is clear → proceed to fix + - If more information needed → add more targeted logging and repeat + +### Phase 3: Fix and Verify + +1. **Implement the fix** based on findings +2. **Run the test** to verify it passes +3. **Run related tests** to ensure no regressions + +### Phase 4: Clean Up + +1. **Remove ALL debugging artifacts:** + - Delete all `console.log('[DEBUG]', ...)` statements added + - Remove any temporary variables or code added for debugging + - Ensure the code is in a clean, production-ready state + +2. **Verify the test still passes** after cleanup + +### Phase 5: Document and Learn + +1. **Provide a summary** to the user (1-3 sentences): + - What was the bug? + - What was the fix? + +2. **Record the learning** by following the learning instructions (if you have them): + - Extract a single, clear learning from this debugging session + - Add it to the "Learnings" section of the most relevant instruction file + - If a similar learning already exists, increment its counter instead + +## Logging Conventions + +When adding debug logging, use this format for easy identification and removal: + +```typescript +console.log('[DEBUG] :', ); +console.log('[DEBUG] before :', { input, state }); +console.log('[DEBUG] after :', { result, state }); +``` + +## Example Debug Session + +```typescript +// Added logging example: +console.log('[DEBUG] getEnvironments input:', { workspaceFolder }); +const envs = await manager.getEnvironments(workspaceFolder); +console.log('[DEBUG] getEnvironments result:', { count: envs.length, envs }); +``` + +## Notes + +- Prefer targeted logging over flooding the output +- Start with the failing assertion and work backwards +- Consider async timing issues, race conditions, and mock setup problems +- Check that mocks are returning expected values +- Verify test setup/teardown is correct diff --git a/.github/skills/run-e2e-tests/SKILL.md b/.github/skills/run-e2e-tests/SKILL.md new file mode 100644 index 00000000..f4f34ec8 --- /dev/null +++ b/.github/skills/run-e2e-tests/SKILL.md @@ -0,0 +1,125 @@ +--- +name: run-e2e-tests +description: Run E2E tests to verify complete user workflows like environment discovery, creation, and selection. Use this before releases or after major changes. +--- + +Run E2E (end-to-end) tests to verify complete user workflows work correctly. + +## When to Use This Skill + +- Before submitting a PR with significant changes +- After modifying environment discovery, creation, or selection logic +- Before a release to validate full workflows +- When user reports a workflow is broken + +**Note:** Run smoke tests first. If smoke tests fail, E2E tests will also fail. + +## Quick Reference + +| Action | Command | +| ----------------- | -------------------------------------------------------------- | +| Run all E2E tests | `npm run compile && npm run compile-tests && npm run e2e-test` | +| Run specific test | `npm run e2e-test -- --grep "discovers"` | +| Debug in VS Code | Debug panel → "E2E Tests" → F5 | + +## How E2E Tests Work + +Unlike unit tests (mocked) and smoke tests (quick checks), E2E tests: + +1. Launch a real VS Code instance with the extension +2. Exercise complete user workflows via the real API +3. Verify end-to-end behavior (discovery → selection → execution) + +They take longer (1-3 minutes) but catch integration issues. + +## Workflow + +### Step 1: Compile and Run + +```bash +npm run compile && npm run compile-tests && npm run e2e-test +``` + +### Step 2: Interpret Results + +**Pass:** + +``` + E2E: Environment Discovery + ✓ Can trigger environment refresh + ✓ Discovers at least one environment + ✓ Environments have required properties + ✓ Can get global environments + + 4 passing (45s) +``` + +**Fail:** Check error message and see Debugging section. + +## Debugging Failures + +| Error | Cause | Fix | +| ---------------------------- | ---------------------- | ------------------------------------------- | +| `No environments discovered` | Python not installed | Install Python, verify it's on PATH | +| `Extension not found` | Build failed | Run `npm run compile` | +| `API not available` | Activation error | Debug with F5, check Debug Console | +| `Timeout exceeded` | Slow operation or hang | Increase timeout or check for blocking code | + +For detailed debugging: Debug panel → "E2E Tests" → F5 + +## Prerequisites + +E2E tests have system requirements: + +- **Python installed** - At least one Python interpreter must be discoverable +- **Extension builds** - Run `npm run compile` before tests +- **CI needs webpack build** - Run `npm run compile` (webpack) before tests, not just `npm run compile-tests` (tsc) + +## Adding New E2E Tests + +Create files in `src/test/e2e/` with pattern `*.e2e.test.ts`: + +```typescript +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { waitForCondition } from '../testUtils'; +import { ENVS_EXTENSION_ID } from '../constants'; + +suite('E2E: [Workflow Name]', function () { + this.timeout(120_000); // 2 minutes + + let api: ExtensionApi; + + suiteSetup(async function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension, 'Extension not found'); + if (!extension.isActive) await extension.activate(); + api = extension.exports; + }); + + test('[Test description]', async function () { + // Use real API (flat structure, not nested!) + // api.getEnvironments(), not api.environments.getEnvironments() + await waitForCondition( + async () => (await api.getEnvironments('all')).length > 0, + 60_000, + 'No environments found', + ); + }); +}); +``` + +## Test Files + +| File | Purpose | +| ----------------------------------------------- | ------------------------------------ | +| `src/test/e2e/environmentDiscovery.e2e.test.ts` | Discovery workflow tests | +| `src/test/e2e/index.ts` | Test runner entry point | +| `src/test/testUtils.ts` | Utilities (`waitForCondition`, etc.) | + +## Notes + +- E2E tests are slower than smoke tests (expect 1-3 minutes) +- They may create/modify files - cleanup happens in `suiteTeardown` +- First run downloads VS Code (~100MB, cached in `.vscode-test/`) +- For more details on E2E tests and how they compare to other test types, refer to the project's testing documentation. diff --git a/.github/skills/run-integration-tests/SKILL.md b/.github/skills/run-integration-tests/SKILL.md new file mode 100644 index 00000000..d3d68ee2 --- /dev/null +++ b/.github/skills/run-integration-tests/SKILL.md @@ -0,0 +1,112 @@ +--- +name: run-integration-tests +description: Run integration tests to verify that extension components work together correctly. Use this after modifying component interactions or event handling. +--- + +Run integration tests to verify that multiple components (managers, API, settings) work together correctly. + +## When to Use This Skill + +- After modifying how components communicate (events, state sharing) +- After changing the API surface +- After modifying managers or their interactions +- When components seem out of sync (UI shows stale data, events not firing) + +## Quick Reference + +| Action | Command | +| ------------------------- | ---------------------------------------------------------------------- | +| Run all integration tests | `npm run compile && npm run compile-tests && npm run integration-test` | +| Run specific test | `npm run integration-test -- --grep "manager"` | +| Debug in VS Code | Debug panel → "Integration Tests" → F5 | + +## How Integration Tests Work + +Integration tests run in a real VS Code instance but focus on **component interactions**: + +- Does the API reflect manager state? +- Do events fire when state changes? +- Do different scopes return appropriate data? + +They're faster than E2E (which test full workflows) but more thorough than smoke tests. + +## Workflow + +### Step 1: Compile and Run + +```bash +npm run compile && npm run compile-tests && npm run integration-test +``` + +### Step 2: Interpret Results + +**Pass:** + +``` + Integration: Environment Manager + API + ✓ API reflects manager state after refresh + ✓ Different scopes return appropriate environments + ✓ Environment objects have consistent structure + + 3 passing (25s) +``` + +**Fail:** Check error message and see Debugging section. + +## Debugging Failures + +| Error | Cause | Fix | +| ------------------- | --------------------------- | ------------------------------- | +| `API not available` | Extension activation failed | Check Debug Console | +| `Event not fired` | Event wiring issue | Check event registration | +| `State mismatch` | Components out of sync | Add logging, check update paths | +| `Timeout` | Async operation stuck | Check for deadlocks | + +For detailed debugging: Debug panel → "Integration Tests" → F5 + +## Adding New Integration Tests + +Create files in `src/test/integration/` with pattern `*.integration.test.ts`: + +```typescript +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { waitForCondition, TestEventHandler } from '../testUtils'; +import { ENVS_EXTENSION_ID } from '../constants'; + +suite('Integration: [Component A] + [Component B]', function () { + this.timeout(120_000); + + let api: ExtensionApi; + + suiteSetup(async function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension, 'Extension not found'); + if (!extension.isActive) await extension.activate(); + api = extension.exports; + }); + + test('[Interaction test]', async function () { + // Test component interaction + }); +}); +``` + +## Test Files + +| File | Purpose | +| -------------------------------------------------------- | -------------------------------------------------- | +| `src/test/integration/envManagerApi.integration.test.ts` | Manager + API tests | +| `src/test/integration/index.ts` | Test runner entry point | +| `src/test/testUtils.ts` | Utilities (`waitForCondition`, `TestEventHandler`) | + +## Prerequisites + +- **CI needs webpack build** - Run `npm run compile` (webpack) before tests, not just `npm run compile-tests` (tsc) +- **Extension builds** - Run `npm run compile` before tests + +## Notes + +- Integration tests are faster than E2E (30s-2min vs 1-3min) +- Focus on testing component boundaries, not full user workflows +- First run downloads VS Code (~100MB, cached in `.vscode-test/`) diff --git a/.github/skills/run-smoke-tests/SKILL.md b/.github/skills/run-smoke-tests/SKILL.md new file mode 100644 index 00000000..247d8568 --- /dev/null +++ b/.github/skills/run-smoke-tests/SKILL.md @@ -0,0 +1,126 @@ +--- +name: run-smoke-tests +description: Run smoke tests to verify extension functionality in a real VS Code environment. Use this when checking if basic features work after changes. +--- + +Run smoke tests to verify the extension loads and basic functionality works in a real VS Code environment. + +## When to Use This Skill + +- After making changes to extension activation code +- After modifying commands or API exports +- Before submitting a PR to verify nothing is broken +- When the user asks to "run smoke tests" or "verify the extension works" + +## Quick Reference + +| Action | Command | +| ------------------- | ---------------------------------------------------------------- | +| Run all smoke tests | `npm run compile && npm run compile-tests && npm run smoke-test` | +| Run specific test | `npm run smoke-test -- --grep "Extension activates"` | +| Debug in VS Code | Debug panel → "Smoke Tests" → F5 | + +## How Smoke Tests Work + +Unlike unit tests (which mock VS Code), smoke tests run inside a **real VS Code instance**: + +1. `npm run smoke-test` uses `@vscode/test-cli` +2. The CLI downloads a standalone VS Code binary (cached in `.vscode-test/`) +3. It launches that VS Code with your extension installed +4. Mocha runs test files inside that VS Code process +5. Results are reported back to your terminal + +This is why smoke tests are slower (~10-60s) but catch real integration issues. + +## Workflow + +### Step 1: Compile and Run + +```bash +npm run compile && npm run compile-tests && npm run smoke-test +``` + +### Step 2: Interpret Results + +**Pass:** `4 passing (2s)` → Extension works, proceed. + +**Fail:** See error message and check Debugging section. + +### Running Individual Tests + +To run a specific test instead of the whole suite: + +```bash +# By test name (grep pattern) +npm run smoke-test -- --grep "Extension activates" + +# Or temporarily add .only in code: +test.only('Extension activates without errors', ...) +``` + +## Debugging Failures + +| Error | Cause | Fix | +| --------------------------------- | ----------------------------- | ------------------------------------------- | +| `Extension not installed` | Build failed or ID mismatch | Run `npm run compile`, check extension ID | +| `Extension did not become active` | Error in activate() | Debug with F5, check Debug Console | +| `Command not registered` | Missing from package.json | Add to contributes.commands | +| `Timeout exceeded` | Slow startup or infinite loop | Increase timeout or check for blocking code | + +For detailed debugging, use VS Code: Debug panel → "Smoke Tests" → F5 + +## Adding New Smoke Tests + +Create a new file in `src/test/smoke/` with the naming convention `*.smoke.test.ts`: + +```typescript +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { waitForCondition } from '../testUtils'; +import { ENVS_EXTENSION_ID } from '../constants'; + +suite('Smoke: [Feature Name]', function () { + this.timeout(60_000); + + test('[Test description]', async function () { + // Arrange + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension, 'Extension not found'); + + // Ensure extension is active + if (!extension.isActive) { + await extension.activate(); + } + + // Act + const result = await someOperation(); + + // Assert + assert.strictEqual(result, expected, 'Description of what went wrong'); + }); +}); +``` + +**Key patterns:** + +- Use `waitForCondition()` instead of `sleep()` for async assertions +- Set generous timeouts (`this.timeout(60_000)`) +- Include clear error messages in assertions + +## Test Files + +| File | Purpose | +| ----------------------------------------- | ---------------------------------- | +| `src/test/smoke/activation.smoke.test.ts` | Extension activation tests | +| `src/test/smoke/index.ts` | Test runner entry point | +| `src/test/testUtils.ts` | Utilities (waitForCondition, etc.) | + +## Prerequisites + +- **CI needs webpack build**: The extension must be built with `npm run compile` (webpack) before tests run. The test runner uses `dist/extension.js` which is only created by webpack, not by `npm run compile-tests` (tsc) +- **Extension builds**: Run `npm run compile` before tests + +## Notes + +- First run downloads VS Code (~100MB, cached in `.vscode-test/`) +- Tests auto-retry once on failure diff --git a/.github/workflows/pr-check.yml b/.github/workflows/pr-check.yml index 30588559..03efdd0d 100644 --- a/.github/workflows/pr-check.yml +++ b/.github/workflows/pr-check.yml @@ -73,3 +73,153 @@ jobs: - name: Run Tests run: npm run unittest + + smoke-tests: + name: Smoke Tests + runs-on: ${{ matrix.os }} + needs: [build-vsix] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run Smoke Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run smoke-test + + - name: Run Smoke Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run smoke-test + + e2e-tests: + name: E2E Tests + runs-on: ${{ matrix.os }} + needs: [smoke-tests] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run E2E Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run e2e-test + + - name: Run E2E Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run e2e-test + + integration-tests: + name: Integration Tests + runs-on: ${{ matrix.os }} + needs: [smoke-tests] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run Integration Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run integration-test + + - name: Run Integration Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run integration-test diff --git a/.github/workflows/push-check.yml b/.github/workflows/push-check.yml index 80d6ec31..b6dd35d1 100644 --- a/.github/workflows/push-check.yml +++ b/.github/workflows/push-check.yml @@ -74,3 +74,153 @@ jobs: - name: Run Tests run: npm run unittest + + smoke-tests: + name: Smoke Tests + runs-on: ${{ matrix.os }} + needs: [build-vsix] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run Smoke Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run smoke-test + + - name: Run Smoke Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run smoke-test + + e2e-tests: + name: E2E Tests + runs-on: ${{ matrix.os }} + needs: [smoke-tests] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run E2E Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run e2e-test + + - name: Run E2E Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run e2e-test + + integration-tests: + name: Integration Tests + runs-on: ${{ matrix.os }} + needs: [smoke-tests] + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, windows-latest, macos-latest] + python-version: ['3.9', '3.12', '3.14'] + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install Node + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + cache: 'npm' + + - name: Install Python + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install Dependencies + run: npm ci + + - name: Compile Extension + run: npm run compile + + - name: Compile Tests + run: npm run compile-tests + + - name: Configure Test Settings + run: | + mkdir -p .vscode-test/user-data/User + echo '{"python.useEnvironmentsExtension": true}' > .vscode-test/user-data/User/settings.json + shell: bash + + - name: Run Integration Tests (Linux) + if: runner.os == 'Linux' + uses: GabrielBB/xvfb-action@v1 + with: + run: npm run integration-test + + - name: Run Integration Tests (non-Linux) + if: runner.os != 'Linux' + run: npm run integration-test diff --git a/.vscode-test.mjs b/.vscode-test.mjs index b62ba25f..f6a5edf7 100644 --- a/.vscode-test.mjs +++ b/.vscode-test.mjs @@ -1,5 +1,75 @@ import { defineConfig } from '@vscode/test-cli'; +import * as path from 'path'; -export default defineConfig({ - files: 'out/test/**/*.test.js', -}); +// Explicit user data directory - ensures VS Code reads our settings.json +const userDataDir = path.resolve('.vscode-test/user-data'); + +export default defineConfig([ + { + label: 'smokeTests', + files: 'out/test/smoke/**/*.smoke.test.js', + mocha: { + ui: 'tdd', + timeout: 120000, + retries: 1, + }, + env: { + VSC_PYTHON_SMOKE_TEST: '1', + }, + launchArgs: [ + `--user-data-dir=${userDataDir}`, + // Don't open any folder with Python files to prevent premature activation + '--disable-workspace-trust', + ], + // NOTE: Do NOT install ms-python.python for smoke tests! + // It defines python.useEnvironmentsExtension=false by default, which + // causes our extension to skip activation. Smoke tests only verify + // our extension works - we don't need the Python extension. + }, + { + label: 'e2eTests', + files: 'out/test/e2e/**/*.e2e.test.js', + mocha: { + ui: 'tdd', + timeout: 180000, + retries: 1, + }, + env: { + VSC_PYTHON_E2E_TEST: '1', + }, + launchArgs: [ + `--user-data-dir=${userDataDir}`, + '--disable-workspace-trust', + ], + // ms-python.python is installed via CLI flag (--install-extensions) for + // the native Python tools (pet binary). We use inspect() for + // useEnvironmentsExtension check, so Python extension's default is ignored. + }, + { + label: 'integrationTests', + files: 'out/test/integration/**/*.integration.test.js', + mocha: { + ui: 'tdd', + timeout: 60000, + retries: 1, + }, + env: { + VSC_PYTHON_INTEGRATION_TEST: '1', + }, + launchArgs: [ + `--user-data-dir=${userDataDir}`, + '--disable-workspace-trust', + ], + // ms-python.python is installed via CLI flag (--install-extensions) for + // the native Python tools (pet binary). We use inspect() for + // useEnvironmentsExtension check, so Python extension's default is ignored. + }, + { + label: 'extensionTests', + files: 'out/test/**/*.test.js', + mocha: { + ui: 'tdd', + timeout: 60000, + }, + }, +]); diff --git a/.vscode/launch.json b/.vscode/launch.json index af8e2306..3e59c2bd 100644 --- a/.vscode/launch.json +++ b/.vscode/launch.json @@ -42,6 +42,48 @@ ], "outFiles": ["${workspaceFolder}/out/**/*.js", "${workspaceFolder}/dist/**/*.js"], "preLaunchTask": "${defaultBuildTask}" + }, + { + "name": "Smoke Tests", + "type": "extensionHost", + "request": "launch", + "env": { + "VSC_PYTHON_SMOKE_TEST": "1" + }, + "args": [ + "--extensionDevelopmentPath=${workspaceFolder}", + "--extensionTestsPath=${workspaceFolder}/out/test/smoke" + ], + "outFiles": ["${workspaceFolder}/out/**/*.js", "${workspaceFolder}/dist/**/*.js"], + "preLaunchTask": "tasks: build" + }, + { + "name": "E2E Tests", + "type": "extensionHost", + "request": "launch", + "env": { + "VSC_PYTHON_E2E_TEST": "1" + }, + "args": [ + "--extensionDevelopmentPath=${workspaceFolder}", + "--extensionTestsPath=${workspaceFolder}/out/test/e2e" + ], + "outFiles": ["${workspaceFolder}/out/**/*.js", "${workspaceFolder}/dist/**/*.js"], + "preLaunchTask": "tasks: build" + }, + { + "name": "Integration Tests", + "type": "extensionHost", + "request": "launch", + "env": { + "VSC_PYTHON_INTEGRATION_TEST": "1" + }, + "args": [ + "--extensionDevelopmentPath=${workspaceFolder}", + "--extensionTestsPath=${workspaceFolder}/out/test/integration" + ], + "outFiles": ["${workspaceFolder}/out/**/*.js", "${workspaceFolder}/dist/**/*.js"], + "preLaunchTask": "tasks: build" } ] } diff --git a/package.json b/package.json index a7ba5683..dedcee6b 100644 --- a/package.json +++ b/package.json @@ -682,6 +682,9 @@ "pretest": "npm run compile-tests && npm run compile", "lint": "eslint --config=eslint.config.mjs src", "unittest": "mocha --config=./build/.mocha.unittests.json", + "smoke-test": "vscode-test --label smokeTests", + "e2e-test": "vscode-test --label e2eTests --install-extensions ms-python.python", + "integration-test": "vscode-test --label integrationTests --install-extensions ms-python.python", "vsce-package": "vsce package -o ms-python-envs-insiders.vsix" }, "devDependencies": { diff --git a/src/extension.ts b/src/extension.ts index abbf1677..09e9b3d1 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -1,4 +1,14 @@ -import { commands, ExtensionContext, extensions, l10n, LogOutputChannel, ProgressLocation, Terminal, Uri, window } from 'vscode'; +import { + commands, + ExtensionContext, + extensions, + l10n, + LogOutputChannel, + ProgressLocation, + Terminal, + Uri, + window, +} from 'vscode'; import { PythonEnvironment, PythonEnvironmentApi, PythonProjectCreator } from './api'; import { ENVS_EXTENSION_ID } from './common/constants'; import { ensureCorrectVersion } from './common/extVersion'; @@ -18,7 +28,7 @@ import { onDidChangeTerminalShellIntegration, withProgress, } from './common/window.apis'; -import { getConfiguration } from './common/workspace.apis'; +import { getConfiguration, getWorkspaceFolders } from './common/workspace.apis'; import { createManagerReady } from './features/common/managerReady'; import { AutoFindProjects } from './features/creators/autoFindProjects'; import { ExistingProjects } from './features/creators/existingProjects'; @@ -87,7 +97,31 @@ import { registerPoetryFeatures } from './managers/poetry/main'; import { registerPyenvFeatures } from './managers/pyenv/main'; export async function activate(context: ExtensionContext): Promise { - const useEnvironmentsExtension = getConfiguration('python').get('useEnvironmentsExtension', true); + // Only skip activation if user explicitly set useEnvironmentsExtension to false. + // When disabled, the main Python extension handles environments instead (legacy mode). + const config = getConfiguration('python'); + const inspection = config.inspect('useEnvironmentsExtension'); + + // Check global and workspace-level explicit disables + let explicitlyDisabled = inspection?.globalValue === false || inspection?.workspaceValue === false; + + // Also check folder-scoped settings in multi-root workspaces + // (inspect() on an unscoped config won't populate workspaceFolderValue reliably) + if (!explicitlyDisabled) { + const workspaceFolders = getWorkspaceFolders(); + if (workspaceFolders) { + for (const folder of workspaceFolders) { + const folderConfig = getConfiguration('python', folder.uri); + const folderInspection = folderConfig.inspect('useEnvironmentsExtension'); + if (folderInspection?.workspaceFolderValue === false) { + explicitlyDisabled = true; + break; + } + } + } + } + + const useEnvironmentsExtension = !explicitlyDisabled; traceInfo(`Experiment Status: useEnvironmentsExtension setting set to ${useEnvironmentsExtension}`); if (!useEnvironmentsExtension) { traceWarn( diff --git a/src/test/constants.ts b/src/test/constants.ts index 4d33a6ee..e930ad60 100644 --- a/src/test/constants.ts +++ b/src/test/constants.ts @@ -2,3 +2,34 @@ import * as path from 'path'; export const EXTENSION_ROOT = path.dirname(path.dirname(__dirname)); export const EXTENSION_TEST_ROOT = path.join(EXTENSION_ROOT, 'src', 'test'); + +// Extension identifiers +export const ENVS_EXTENSION_ID = 'ms-python.vscode-python-envs'; + +// Test type detection via environment variables +// These are set by the test runner scripts before launching tests +export const IS_SMOKE_TEST = process.env.VSC_PYTHON_SMOKE_TEST === '1'; +export const IS_E2E_TEST = process.env.VSC_PYTHON_E2E_TEST === '1'; +export const IS_INTEGRATION_TEST = process.env.VSC_PYTHON_INTEGRATION_TEST === '1'; + +// Test timeouts (in milliseconds) +export const MAX_EXTENSION_ACTIVATION_TIME = 60_000; // 60 seconds for extension activation +export const TEST_TIMEOUT = 30_000; // 30 seconds default test timeout +export const TEST_RETRYCOUNT = 3; // Number of retries for flaky tests + +/** + * Detect if running in a multi-root workspace. + * Returns false during smoke tests (don't want multi-root complexity). + */ +export function isMultiRootTest(): boolean { + if (IS_SMOKE_TEST) { + return false; + } + try { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const vscode = require('vscode'); + return Array.isArray(vscode.workspace.workspaceFolders) && vscode.workspace.workspaceFolders.length > 1; + } catch { + return false; + } +} diff --git a/src/test/e2e/environmentDiscovery.e2e.test.ts b/src/test/e2e/environmentDiscovery.e2e.test.ts new file mode 100644 index 00000000..efba8ce7 --- /dev/null +++ b/src/test/e2e/environmentDiscovery.e2e.test.ts @@ -0,0 +1,184 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * E2E Test: Environment Discovery + * + * PURPOSE: + * Verify that the extension can discover Python environments on the system. + * This is a fundamental workflow - if discovery fails, users can't select interpreters. + * + * WHAT THIS TESTS: + * 1. Extension API is accessible + * 2. Environment discovery runs successfully + * 3. At least one Python environment is found (assumes Python is installed) + * 4. Environment objects have expected properties + */ + +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { ENVS_EXTENSION_ID } from '../constants'; +import { waitForCondition } from '../testUtils'; + +suite('E2E: Environment Discovery', function () { + this.timeout(90_000); + + // The API is FLAT - methods are directly on the api object, not nested + let api: { + getEnvironments(scope: 'all' | 'global'): Promise; + refreshEnvironments(scope: undefined): Promise; + }; + + suiteSetup(async function () { + // Get and activate the extension + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension, `Extension ${ENVS_EXTENSION_ID} not found`); + + if (!extension.isActive) { + await extension.activate(); + await waitForCondition(() => extension.isActive, 30_000, 'Extension did not activate'); + } + + api = extension.exports; + assert.ok(api, 'Extension API not available'); + assert.ok(typeof api.getEnvironments === 'function', 'getEnvironments method not available'); + }); + + /** + * Test: Can trigger environment refresh + * + */ + test('Can trigger environment refresh', async function () { + // Skip if API doesn't have refresh method + if (typeof api.refreshEnvironments !== 'function') { + this.skip(); + return; + } + + // Capture state before refresh to verify API is callable + const beforeRefresh = await api.getEnvironments('all'); + assert.ok(Array.isArray(beforeRefresh), 'Should get environments before refresh'); + + // Trigger refresh - this should complete without throwing + await api.refreshEnvironments(undefined); + + // Verify API still works after refresh (observable side effect: API remains functional) + const afterRefresh = await api.getEnvironments('all'); + assert.ok(Array.isArray(afterRefresh), 'Should get environments after refresh'); + + // Environments should still be available after refresh + // (count may change if discovery finds more, but should not lose all) + if (beforeRefresh.length > 0) { + assert.ok(afterRefresh.length > 0, 'Should not lose all environments after refresh'); + } + }); + + /** + * Test: Discovers at least one environment + * + */ + test('Discovers at least one environment', async function () { + // Wait for discovery to find at least one environment + let environments: unknown[] = []; + + await waitForCondition( + async () => { + environments = await api.getEnvironments('all'); + return environments.length > 0; + }, + 60_000, // 60 seconds for discovery + 'No Python environments discovered. Ensure Python is installed on the test machine.', + ); + + assert.ok(environments.length > 0, `Expected at least 1 environment, found ${environments.length}`); + }); + + /** + * Test: Discovered environments have required properties + * + */ + test('Environments have required properties', async function () { + const environments = await api.getEnvironments('all'); + + // Skip if no environments (previous test would have caught this) + if (environments.length === 0) { + this.skip(); + return; + } + + const env = environments[0] as Record; + + // Check required properties exist AND have valid values + // PythonEnvironment has envId (a PythonEnvironmentId object), not id directly + assert.ok('envId' in env, 'Environment should have an envId property'); + assert.ok(env.envId !== null && env.envId !== undefined, 'envId should not be null/undefined'); + + // Verify envId structure + const envId = env.envId as Record; + assert.strictEqual(typeof envId, 'object', 'envId should be an object'); + assert.ok('id' in envId, 'envId should have an id property'); + assert.strictEqual(typeof envId.id, 'string', 'envId.id should be a string'); + assert.ok((envId.id as string).length > 0, 'envId.id should not be empty'); + assert.ok('managerId' in envId, 'envId should have a managerId property'); + + // Verify name exists and is a string + assert.ok('name' in env, 'Environment should have a name property'); + assert.strictEqual(typeof env.name, 'string', 'name should be a string'); + + // Verify displayName exists and is a string + assert.ok('displayName' in env, 'Environment should have a displayName property'); + assert.strictEqual(typeof env.displayName, 'string', 'displayName should be a string'); + + // If execInfo exists, it should have expected shape with valid values + if ('execInfo' in env && env.execInfo) { + const execInfo = env.execInfo as Record; + assert.strictEqual(typeof execInfo, 'object', 'execInfo should be an object'); + assert.ok( + 'run' in execInfo || 'activatedRun' in execInfo, + 'execInfo should have run or activatedRun property', + ); + + // Verify run command structure if present + if ('run' in execInfo && execInfo.run) { + const run = execInfo.run as Record; + assert.ok('executable' in run, 'run should have an executable property'); + assert.strictEqual(typeof run.executable, 'string', 'executable should be a string'); + assert.ok((run.executable as string).length > 0, 'executable should not be empty'); + } + } + }); + + /** + * Test: Can get global environments + * + */ + test('Can get global environments', async function () { + // This should not throw, even if there are no global environments + const globalEnvs = await api.getEnvironments('global'); + + // Verify it returns an array + assert.ok(Array.isArray(globalEnvs), 'getEnvironments should return an array'); + + // If there are global envs, verify they have valid structure + if (globalEnvs.length > 0) { + const env = globalEnvs[0] as Record; + + // Global environments should have the same structure as all environments + assert.ok('envId' in env || 'id' in env, 'Global environment should have an identifier'); + + // Verify envId is properly structured if present + if ('envId' in env && env.envId) { + const envId = env.envId as Record; + assert.strictEqual(typeof envId, 'object', 'envId should be an object'); + assert.ok('id' in envId, 'envId should have an id property'); + } + } + + // Global envs should be a subset of all envs + const allEnvs = await api.getEnvironments('all'); + assert.ok( + globalEnvs.length <= allEnvs.length, + `Global envs (${globalEnvs.length}) should not exceed all envs (${allEnvs.length})`, + ); + }); +}); diff --git a/src/test/e2e/index.ts b/src/test/e2e/index.ts new file mode 100644 index 00000000..fa7822e8 --- /dev/null +++ b/src/test/e2e/index.ts @@ -0,0 +1,51 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * E2E Test Runner Entry Point + * + * This file is loaded by the VS Code Extension Host test runner. + * It configures Mocha and runs all E2E tests. + */ + +import * as glob from 'glob'; +import Mocha from 'mocha'; +import * as path from 'path'; + +export async function run(): Promise { + // Set the environment variable so tests know they're running as E2E tests + process.env.VSC_PYTHON_E2E_TEST = '1'; + + const mocha = new Mocha({ + ui: 'tdd', + color: true, + timeout: 180_000, // 3 minutes - E2E workflows can be slow + retries: 1, // Retry once on failure + slow: 30_000, // Mark tests as slow if they take > 30s + }); + + const testsRoot = path.resolve(__dirname); + + // Find all .e2e.test.js files + const files = glob.sync('**/*.e2e.test.js', { cwd: testsRoot }); + + // Add files to the test suite + for (const file of files) { + mocha.addFile(path.resolve(testsRoot, file)); + } + + return new Promise((resolve, reject) => { + try { + mocha.run((failures: number) => { + if (failures > 0) { + reject(new Error(`${failures} E2E tests failed`)); + } else { + resolve(); + } + }); + } catch (err) { + console.error('Error running E2E tests:', err); + reject(err); + } + }); +} diff --git a/src/test/integration/envManagerApi.integration.test.ts b/src/test/integration/envManagerApi.integration.test.ts new file mode 100644 index 00000000..596b2a66 --- /dev/null +++ b/src/test/integration/envManagerApi.integration.test.ts @@ -0,0 +1,165 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * Integration Test: Environment Manager + API + * + * PURPOSE: + * Verify that the environment manager component correctly exposes data + * through the extension API. This tests the integration between internal + * managers and the public API surface. + * + * WHAT THIS TESTS: + * 1. API reflects environment manager state + * 2. Changes through API update manager state + * 3. Events fire when state changes + * + */ + +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { ENVS_EXTENSION_ID } from '../constants'; +import { TestEventHandler, waitForCondition } from '../testUtils'; + +suite('Integration: Environment Manager + API', function () { + // Shorter timeout for faster feedback + this.timeout(45_000); + + // The API is FLAT - methods are directly on the api object, not nested + let api: { + getEnvironments(scope: 'all' | 'global'): Promise; + refreshEnvironments(scope: undefined): Promise; + onDidChangeEnvironments?: vscode.Event; + }; + + suiteSetup(async function () { + // Set a shorter timeout for setup specifically + this.timeout(20_000); + + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension, `Extension ${ENVS_EXTENSION_ID} not found`); + + if (!extension.isActive) { + await extension.activate(); + await waitForCondition(() => extension.isActive, 15_000, 'Extension did not activate'); + } + + api = extension.exports; + assert.ok(typeof api?.getEnvironments === 'function', 'getEnvironments method not available'); + }); + + /** + * Test: API and manager stay in sync after refresh + * + */ + test('API reflects manager state after refresh', async function () { + // Get initial state (verify we can call API before refresh) + await api.getEnvironments('all'); + + // Trigger refresh + await api.refreshEnvironments(undefined); + + // Get state after refresh + const afterRefresh = await api.getEnvironments('all'); + + // Verify we got an actual array back (not undefined, null, or other type) + assert.ok(Array.isArray(afterRefresh), `Expected environments array, got ${typeof afterRefresh}`); + + // Verify the API returns consistent data on repeated calls + const secondCall = await api.getEnvironments('all'); + assert.strictEqual(afterRefresh.length, secondCall.length, 'Repeated API calls should return consistent data'); + }); + + /** + * Test: Events fire when environments change + * + */ + test('Change events fire on refresh', async function () { + // Skip if event is not available + if (!api.onDidChangeEnvironments) { + this.skip(); + return; + } + + const handler = new TestEventHandler(api.onDidChangeEnvironments, 'onDidChangeEnvironments'); + + try { + // Trigger a refresh which should fire events + await api.refreshEnvironments(undefined); + + // Assert that at least one event fired during refresh + await handler.assertFiredAtLeast(1, 5000); + const event = handler.first; + assert.ok(event !== undefined, 'Event should have a value'); + } finally { + handler.dispose(); + } + }); + + /** + * Test: Global vs all environments are different scopes + * + */ + test('Different scopes return appropriate environments', async function () { + const allEnvs = await api.getEnvironments('all'); + const globalEnvs = await api.getEnvironments('global'); + + // Both should return arrays + assert.ok(Array.isArray(allEnvs), 'all scope should return array'); + assert.ok(Array.isArray(globalEnvs), 'global scope should return array'); + + // Global should be subset of or equal to all + // (all includes global + workspace-specific) + assert.ok( + globalEnvs.length <= allEnvs.length, + `Global envs (${globalEnvs.length}) should not exceed all envs (${allEnvs.length})`, + ); + }); + + /** + * Test: Environment objects are properly structured + * + */ + test('Environment objects have consistent structure', async function () { + const environments = await api.getEnvironments('all'); + + if (environments.length === 0) { + this.skip(); + return; + } + + // Check each environment has basic required properties with valid values + for (const env of environments) { + const e = env as Record; + + // Must have some form of identifier + assert.ok('id' in e || 'envId' in e, 'Environment must have id or envId'); + + // If it has an id, it should be a non-empty string + if ('id' in e) { + assert.strictEqual(typeof e.id, 'string', 'Environment id should be a string'); + assert.ok((e.id as string).length > 0, 'Environment id should not be empty'); + } + + // If it has envId, verify it's a valid object with required properties + if ('envId' in e && e.envId !== null && e.envId !== undefined) { + const envId = e.envId as Record; + assert.strictEqual(typeof envId, 'object', 'envId should be an object'); + assert.ok('id' in envId, 'envId should have an id property'); + assert.ok('managerId' in envId, 'envId should have a managerId property'); + assert.strictEqual(typeof envId.id, 'string', 'envId.id should be a string'); + assert.ok((envId.id as string).length > 0, 'envId.id should not be empty'); + } + + // Verify name is a non-empty string if present + if ('name' in e && e.name !== undefined) { + assert.strictEqual(typeof e.name, 'string', 'Environment name should be a string'); + } + + // Verify displayName is a non-empty string if present + if ('displayName' in e && e.displayName !== undefined) { + assert.strictEqual(typeof e.displayName, 'string', 'Environment displayName should be a string'); + } + } + }); +}); diff --git a/src/test/integration/index.ts b/src/test/integration/index.ts new file mode 100644 index 00000000..55454103 --- /dev/null +++ b/src/test/integration/index.ts @@ -0,0 +1,47 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * Integration Test Runner Entry Point + */ + +import * as glob from 'glob'; +import Mocha from 'mocha'; +import * as path from 'path'; + +export async function run(): Promise { + // Set environment variable for test type detection + process.env.VSC_PYTHON_INTEGRATION_TEST = '1'; + + const mocha = new Mocha({ + ui: 'tdd', + color: true, + timeout: 120_000, // 2 minutes + retries: 1, + slow: 15_000, // Mark as slow if > 15s + }); + + const testsRoot = path.resolve(__dirname); + + // Find all .integration.test.js files + const files = glob.sync('**/*.integration.test.js', { cwd: testsRoot }); + + for (const file of files) { + mocha.addFile(path.resolve(testsRoot, file)); + } + + return new Promise((resolve, reject) => { + try { + mocha.run((failures: number) => { + if (failures > 0) { + reject(new Error(`${failures} integration tests failed`)); + } else { + resolve(); + } + }); + } catch (err) { + console.error('Error running integration tests:', err); + reject(err); + } + }); +} diff --git a/src/test/smoke/activation.smoke.test.ts b/src/test/smoke/activation.smoke.test.ts new file mode 100644 index 00000000..922b7a70 --- /dev/null +++ b/src/test/smoke/activation.smoke.test.ts @@ -0,0 +1,162 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * Smoke Test: Extension Activation + * + * PURPOSE: + * Verify that the extension activates successfully in a real VS Code environment. + * This is the most basic smoke test - if this fails, nothing else will work. + * + * WHAT THIS TESTS: + * 1. Extension can be found and loaded by VS Code + * 2. Extension activates without throwing errors + * 3. Extension API is exported and accessible + * + */ + +import * as assert from 'assert'; +import * as vscode from 'vscode'; +import { ENVS_EXTENSION_ID, MAX_EXTENSION_ACTIVATION_TIME } from '../constants'; +import { waitForCondition } from '../testUtils'; + +suite('Smoke: Extension Activation', function () { + // Smoke tests need longer timeouts - VS Code startup can be slow + this.timeout(MAX_EXTENSION_ACTIVATION_TIME); + + /** + * Test: Extension is installed and VS Code can find it + */ + test('Extension is installed', function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + + // Specific assertion: Extension must exist + // If undefined, there's a packaging problem + assert.ok( + extension !== undefined, + `Extension ${ENVS_EXTENSION_ID} is not installed. ` + + 'Check that the extension ID matches package.json and the build ran successfully.', + ); + }); + + /** + * Test: Extension activates successfully + * + * ASSERTION STRATEGY: + * 1. First verify extension exists (prerequisite) + * 2. Trigger activation if not already active + * 3. Wait for activation to complete (with timeout) + * 4. Verify no errors occurred + */ + test('Extension activates without errors', async function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + + // Prerequisite check + assert.ok(extension !== undefined, `Extension ${ENVS_EXTENSION_ID} not found`); + + // If already active, we're done + if (extension.isActive) { + return; + } + + // Activate the extension + // This can take time on first activation as it: + // - Discovers Python environments + // - Initializes managers + // - Sets up views + try { + await extension.activate(); + } catch (error) { + // Activation threw an error - test fails + assert.fail( + `Extension activation threw an error: ${error instanceof Error ? error.message : String(error)}`, + ); + } + + // Wait for activation to complete + // The activate() promise resolves when activation starts, but + // isActive becomes true when activation finishes + await waitForCondition( + () => extension.isActive, + 30_000, // 30 second timeout + 'Extension did not become active after activation', + ); + + // Final verification + assert.strictEqual(extension.isActive, true, 'Extension should be active after activation completes'); + }); + + /** + * Test: Extension exports its API + * + * ASSERTION STRATEGY: + * - Verify exports is not undefined + * - Verify exports is not null + */ + test('Extension exports API', async function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension !== undefined, `Extension ${ENVS_EXTENSION_ID} not found`); + + // Ensure extension is active first + if (!extension.isActive) { + await extension.activate(); + await waitForCondition(() => extension.isActive, 30_000, 'Extension did not activate'); + } + + // Verify API is exported + const api = extension.exports; + + assert.ok( + api !== undefined, + 'Extension exports should not be undefined. ' + + 'Check that extension.ts returns an API object from activate().', + ); + + assert.ok( + api !== null, + 'Extension exports should not be null. ' + 'Check that extension.ts returns a valid API object.', + ); + + // Optional: Verify API shape + // Uncomment and customize when your API is stable + // assert.ok(typeof api.getEnvironments === 'function', 'API should have getEnvironments()'); + }); + + /** + * Test: Extension commands are registered + * + * ASSERTION STRATEGY: + * - Get all registered commands from VS Code + * - Check that our expected commands exist + * - Use includes() for each command to get specific feedback + */ + test('Extension commands are registered', async function () { + const extension = vscode.extensions.getExtension(ENVS_EXTENSION_ID); + assert.ok(extension !== undefined, `Extension ${ENVS_EXTENSION_ID} not found`); + + // Ensure extension is active + if (!extension.isActive) { + await extension.activate(); + await waitForCondition(() => extension.isActive, 30_000, 'Extension did not activate'); + } + + // Get all registered commands + const allCommands = await vscode.commands.getCommands(true); + + // List of commands that MUST be registered + // Add your critical commands here + const requiredCommands = [ + 'python-envs.set', // Set environment + 'python-envs.create', // Create environment + 'python-envs.refreshAllManagers', // Refresh managers + ]; + + for (const cmd of requiredCommands) { + assert.ok( + allCommands.includes(cmd), + `Required command '${cmd}' is not registered. ` + + 'Check that the command is defined in package.json and registered in extension.ts.', + ); + } + }); +}); diff --git a/src/test/smoke/index.ts b/src/test/smoke/index.ts new file mode 100644 index 00000000..dbdafafc --- /dev/null +++ b/src/test/smoke/index.ts @@ -0,0 +1,54 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * Smoke Test Runner Entry Point + * + * This file is loaded by the VS Code Extension Host test runner. + * It configures Mocha and runs all smoke tests. + * + * IMPORTANT: Smoke tests run INSIDE VS Code with REAL APIs. + * They are NOT mocked like unit tests. + */ + +import * as glob from 'glob'; +import Mocha from 'mocha'; +import * as path from 'path'; + +export async function run(): Promise { + // Set the environment variable so tests know they're running as smoke tests + process.env.VSC_PYTHON_SMOKE_TEST = '1'; + + const mocha = new Mocha({ + ui: 'tdd', + color: true, + timeout: 120_000, // 2 minutes - smoke tests can be slow + retries: 1, // Retry once on failure to handle flakiness + slow: 10_000, // Mark tests as slow if they take > 10s + }); + + const testsRoot = path.resolve(__dirname); + + // Find all .smoke.test.js files + const files = glob.sync('**/*.smoke.test.js', { cwd: testsRoot }); + + // Add files to the test suite + for (const file of files) { + mocha.addFile(path.resolve(testsRoot, file)); + } + + return new Promise((resolve, reject) => { + try { + mocha.run((failures: number) => { + if (failures > 0) { + reject(new Error(`${failures} smoke tests failed`)); + } else { + resolve(); + } + }); + } catch (err) { + console.error('Error running smoke tests:', err); + reject(err); + } + }); +} diff --git a/src/test/testUtils.ts b/src/test/testUtils.ts new file mode 100644 index 00000000..8e550bb5 --- /dev/null +++ b/src/test/testUtils.ts @@ -0,0 +1,207 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +/** + * Test utilities for E2E and smoke tests. + * + * These utilities are designed to work with REAL VS Code APIs, + * not the mocked APIs used in unit tests. + */ + +import type { Disposable, Event } from 'vscode'; + +/** + * Sleep for a specified number of milliseconds. + * Use sparingly - prefer waitForCondition() for most cases. + */ +export function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +/** + * Wait for a condition to become true within a timeout. + * + * This is the PRIMARY utility for smoke/E2E tests. Use this instead of sleep() + * for any async assertion that depends on VS Code state. + * + * @param condition - Async function that returns true when condition is met + * @param timeoutMs - Maximum time to wait (default: 10 seconds) + * @param errorMessage - Error message if condition is not met + * @param pollIntervalMs - How often to check condition (default: 100ms) + * + * @example + * // Wait for extension to activate + * await waitForCondition( + * () => extension.isActive, + * 10_000, + * 'Extension did not activate within 10 seconds' + * ); + * + * @example + * // Wait for file to exist + * await waitForCondition( + * async () => fs.pathExists(outputFile), + * 30_000, + * `Output file ${outputFile} was not created` + * ); + */ +export async function waitForCondition( + condition: () => boolean | Promise, + timeoutMs: number = 10_000, + errorMessage: string = 'Condition not met within timeout', + pollIntervalMs: number = 100, +): Promise { + return new Promise((resolve, reject) => { + const startTime = Date.now(); + + const checkCondition = async () => { + try { + const result = await condition(); + if (result) { + resolve(); + return; + } + } catch { + // Condition threw - keep waiting + } + + if (Date.now() - startTime >= timeoutMs) { + reject(new Error(`${errorMessage} (waited ${timeoutMs}ms)`)); + return; + } + + setTimeout(checkCondition, pollIntervalMs); + }; + + checkCondition(); + }); +} + +/** + * Retry an async function until it succeeds or timeout is reached. + * + * Similar to waitForCondition but captures the return value. + * + * @example + * const envs = await retryUntilSuccess( + * () => api.getEnvironments(), + * (envs) => envs.length > 0, + * 10_000, + * 'No environments discovered' + * ); + */ +export async function retryUntilSuccess( + fn: () => T | Promise, + validate: (result: T) => boolean = () => true, + timeoutMs: number = 10_000, + errorMessage: string = 'Operation did not succeed within timeout', +): Promise { + const startTime = Date.now(); + let lastError: Error | undefined; + + while (Date.now() - startTime < timeoutMs) { + try { + const result = await fn(); + if (validate(result)) { + return result; + } + } catch (e) { + lastError = e as Error; + } + await sleep(100); + } + + throw new Error(`${errorMessage}: ${lastError?.message || 'validation failed'}`); +} + +/** + * Helper class to test events. + * + * Captures events and provides assertion helpers. + * + * @example + * const handler = new TestEventHandler(api.onDidChangeEnvironments, 'onDidChangeEnvironments'); + * // ... trigger some action that fires events ... + * await handler.assertFiredAtLeast(1, 5000); + * assert.strictEqual(handler.first.type, 'add'); + * handler.dispose(); + */ +export class TestEventHandler implements Disposable { + private readonly handledEvents: T[] = []; + private readonly disposable: Disposable; + + constructor( + event: Event, + private readonly eventName: string = 'event', + ) { + this.disposable = event((e) => this.handledEvents.push(e)); + } + + /** Whether any events have been fired */ + get fired(): boolean { + return this.handledEvents.length > 0; + } + + /** The first event fired (throws if none) */ + get first(): T { + if (this.handledEvents.length === 0) { + throw new Error(`No ${this.eventName} events fired yet`); + } + return this.handledEvents[0]; + } + + /** The last event fired (throws if none) */ + get last(): T { + if (this.handledEvents.length === 0) { + throw new Error(`No ${this.eventName} events fired yet`); + } + return this.handledEvents[this.handledEvents.length - 1]; + } + + /** Number of events fired */ + get count(): number { + return this.handledEvents.length; + } + + /** All events fired */ + get all(): T[] { + return [...this.handledEvents]; + } + + /** Get event at specific index */ + at(index: number): T { + return this.handledEvents[index]; + } + + /** Reset captured events */ + reset(): void { + this.handledEvents.length = 0; + } + + /** Wait for at least one event to fire */ + async assertFired(waitMs: number = 1000): Promise { + await waitForCondition(() => this.fired, waitMs, `${this.eventName} was not fired`); + } + + /** Wait for exactly N events to fire */ + async assertFiredExactly(count: number, waitMs: number = 2000): Promise { + await waitForCondition( + () => this.count === count, + waitMs, + `Expected ${this.eventName} to fire ${count} times, but fired ${this.count} times`, + ); + } + + /** Wait for at least N events to fire */ + async assertFiredAtLeast(count: number, waitMs: number = 2000): Promise { + await waitForCondition( + () => this.count >= count, + waitMs, + `Expected ${this.eventName} to fire at least ${count} times, but fired ${this.count} times`, + ); + } + + dispose(): void { + this.disposable.dispose(); + } +}