[Feature] Add GitHub Action failed test annotation for failed tests

[sidharthv96]

I'm writing a Github Action which reports failures in Playwright tests using the JSON Reporter.

But the JSON Output does not have the exact location of the error, just the test which failed.

From the generated annotation:

When running locally:

If we could add an optional location parameter to JSONReportTestResult, that would be helpful in showing the exact location.

[sidharthv96]

https://github.com/sidharthv96/playwright-report-action

This is my current implementation.

[mxschmitt]

Workaround for now: You should be able to use stack-utils to parse the exception stack and place it at the corresponding position in the source-code: https://www.npmjs.com/package/stack-utils

[sidharthv96]

Thanks, I was looking at https://github.com/microsoft/playwright/blob/master/src/test/reporters/base.ts and

Edit: Seems also possible without the GitHub SDK: https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-error-message

The cleaner solution than a separate github action would be a github-action-reporter like you suggested in #7038

I will try to play around with it and maybe create a POC with proper log format for annotations.

[mxschmitt]

I think it would also make sense to enable such a reporter by default when you are using GitHub Actions with the test-runner.
A lot of users might benefit form it. So it might be a candidate for a contribution if you are interested in it. Also It would be great to not use any dependencies if possible to stay lightweight for our users.

[sidharthv96]

Yup, the POC is working. No dependencies needed.

[sidharthv96]

I can raise a PR soon after cleaning it up.

All the warnings and errors in epilogue can be added as annotations.