[BUG] Enable GitHub reporter in default configs and internal tests.

[sidharthv96]

Next step would be probably to automatically add it as a reporter in the default configuration if the GITHUB_ACTION environment variable is set + activate it on our configs so we can dogfood it.

Originally posted by @mxschmitt in #9191 (comment)

Relates: #9147

[sidharthv96]

Should we just add github in here in addition to dot if the env var is present?

playwright/src/test/cli.ts

Line 29 in e674d87

const defaultReporter: BuiltInReporter = process.env.CI ? 'dot' : 'list';

[mxschmitt]

Yeah I think:

• defaultReporter -> defaultReporters
• Keep dot reporter how it is
• Add GitHub Action reporter if the env var is set.

This is then for projects which don't have a config. In our scenario we need to add it manually to the configs tests/config.

[sidharthv96]

I think we forgot to add GitHub to https://github.com/microsoft/playwright/blob/b126a5685b0e047b19bbf54ec7bc753cba1efa8d/utils/generate_types/overrides-test.d.ts as github isn't listed as an option in

playwright/types/test.d.ts

Lines 23 to 30 in 1971530

	export type ReporterDescription =
	['dot'] |
	['line'] |
	['list'] |
	['junit'] | ['junit', { outputFile?: string, stripANSIControlSequences?: boolean }] |
	['json'] | ['json', { outputFile?: string }] |
	['null'] |
	[string] | [string, any];

Is that the proper procedure to add to the types?

[JoelEinbinder]

I think we forgot to add GitHub to https://github.com/microsoft/playwright/blob/b126a5685b0e047b19bbf54ec7bc753cba1efa8d/utils/generate_types/overrides-test.d.ts as github isn't listed as an option in

playwright/types/test.d.ts

Lines 23 to 30 in 1971530

	export type ReporterDescription =
	['dot'] |
	['line'] |
	['list'] |
	['junit'] | ['junit', { outputFile?: string, stripANSIControlSequences?: boolean }] |
	['json'] | ['json', { outputFile?: string }] |
	['null'] |
	[string] | [string, any];

Is that the proper procedure to add to the types?

These are the types for users. Is there are reason someone manually specifies the github reporter in their config? I figured it would just be automatically turned on by checking for the environment variable.

[sidharthv96]

The current approach is to inject github into the default set if they haven't specified any reporters. So they'd need the type if they are overriding in the config and need to have github in CI.

If we are enabling it regardless of user preference, we'd have to add an ENV var they could set, if they want to disable the annotations.

[sidharthv96]

@JoelEinbinder we could push GitHub in here based on an both GITHUB_ACTION and PW_DISABLE_GITHUB_ANNOTATION(?) env variables so it will be available regardless of the user config or cli flags.

playwright/src/test/runner.ts

Lines 67 to 88 in be30f9f

	private async _createReporter(list: boolean) {
	const defaultReporters: {[key in BuiltInReporter]: new(arg: any) => Reporter} = {
	dot: list ? ListModeReporter : DotReporter,
	line: list ? ListModeReporter : LineReporter,
	list: list ? ListModeReporter : ListReporter,
	github: GitHubReporter,
	json: JSONReporter,
	junit: JUnitReporter,
	null: EmptyReporter,
	};
	const reporters: Reporter[] = [];
	for (const r of this._loader.fullConfig().reporter) {
	const [name, arg] = r;
	if (name in defaultReporters) {
	reporters.push(new defaultReporters[name as keyof typeof defaultReporters](arg));
	} else {
	const reporterConstructor = await this._loader.loadReporter(name);
	reporters.push(new reporterConstructor(arg));
	}
	}
	return new Multiplexer(reporters);
	}

[pavelfeldman]

Why was this issue closed?

We are prioritizing the bugs based on the upvotes, recency and our ability to act. It looks like this issue only has a handful of upvotes, has not been touched recently and/or we lack sufficient feedback to act on it. We are closing issues like this one to keep our bug database maintainable. Please feel free to open a new issue and link this one to it if you think this is a mistake.