`vscode-test-cli` not excluding files from coverage report

[rudyflores]

I have the following configuration file:

/** @type {import('@vscode/test-cli').IConfigurationWithGlobalOptions} */
export default defineConfig({
  /** @type {import('@vscode/test-cli').TestConfiguration[]} */
  tests: [
    {
      files: [
        `__tests__/__integration__/out/__tests__/__integration__/**/*.test.js`,
      ],
      version: "stable",
      extensionDevelopmentPath: __dirname,
      workspaceFolder: `${__dirname}/__tests__/__integration__/resources`,
      srcDir: "./src",
    },
  ],
  /** @type {import('@vscode/test-cli').ICoverageConfiguration} */
  coverage: {
    includeAll: true,
    reporter: ["lcov", "text-summary"],
    output: `${__dirname}/__tests__/__results__/integration`,
    exclude: ["**/node_modules/**"],
  },
});

And I am unable to generate a coverage report that excludes my node_modules event though I specifically mention to exclude those in coverage.exclude any idea if i'm missing something in my configuration?

[g-arjones]

I am experiencing the same behavior. Files are not excluded from coverage report (they are when c8 is called from command line instead)

[rudyflores]

@g-arjones I found doing this produced a better coverage report! But I wish there was a simpler way to have this setup since it still doesn't provide all the proper files:

import { defineConfig } from "@vscode/test-cli";
import { fileURLToPath } from "url";
import { dirname } from "path";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

/** @type {import('@vscode/test-cli').IConfigurationWithGlobalOptions} */
export default defineConfig({
  /** @type {import('@vscode/test-cli').TestConfiguration[]} */
  tests: [
    {
      files: "./__tests__/__integration__/out/**/*.test.js",
      version: "stable",
      extensionDevelopmentPath: __dirname,
      workspaceFolder: `${__dirname}/__tests__/__integration__/resources`,
      srcDir: "./__tests__/__integration__/out/src",
      mocha: {
        timeout: 5000,
      },
    },
  ],
  coverage: {
    exclude: [
      `${__dirname}/out`,
      `${__dirname}/__tests__/__integration__/out/__tests__`,
    ],
  },
});

[g-arjones]

Yep. That does work indeed.

[wenytang-ms]

is there any update?

[rH4rtinger]

Any update on this?

I have a extension that is packed with webpack and it gives me the dist folder in my coverage, even if I try to exclude the folder via any of the following dist patterns

import { defineConfig } from "@vscode/test-cli";

export default defineConfig({
  files: ["out/test/suite/**/*.test.js"],
  version: "insiders",
  workspaceFolder: "./out/temp/workspace",
  launchArgs: ["--disable-extensions", "--profile-temp"],
  coverage: {
    exclude: ["dist", "**/dist/**", /dist/],
  },
});

used vscode-test-cli version: 0.0.10

[kidtsunami]

I wonder if this is related to this line?

vscode-test-cli/src/cli/coverage.mts

Line 70 in ca11885

(report as any).exclude.relativePath = false;

That would jive with the workaround here #40 (comment)

For now if you fully qualify the paths in your coverage.exclude then they are properly excluded.

[stephen-carter-at-sf]

Wow nothing seems to work for me in this coverage space for vscode-test. Can we please have better documentation, error handling, etc. I can't get anything to work the way I need it to work. Is anyone working on this?

[mhagnumdw]

For me, nothing I set in defineConfig > coverage works. Should it work with and without the --coverage parameter in the vscode-test command?

vscode-test --coverage even generates the coverage, but could I customize it through the defineConfig > coverage setting?!

export default defineConfig({
  files: 'out/test/**/*.test.js',
  launchArgs: [
    '--user-data-dir=' + tempUserDataDir,
  ],
  coverage: { // doesn't change anything
    reporter: ['text', 'html', 'lcov', 'json', 'cobertura'], // doesn't change anything
  }
});

"@vscode/test-cli": "^0.0.10"

[ssbarnea]

Apparently there is no way to configure coverage inside .vscode-test.mjs file, only the options passed from the cli seems to be working. Even attempts to make use of launchArgs to inject coverage options will not work. That is sad for multiple reasons, first as it silently ignores these options instead of giving an error and secondly as it prevents use from using lots of coverage related options like skipCovered: true ones.