[Bug]: `merge-report` does not generate any output for the HTML reporter

[OptimusCrime]

Version

1.44.1

Steps to reproduce

We have copied the blueprint for combining reports into a single report in a GitHub action:

      - name: Download blob reports from GitHub Actions Artifacts
        uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true

      - name: Merge into HTML Report
        env:
          CI: true
          PW_TEST_HTML_REPORT_OPEN: never
        run: npx playwright merge-reports --reporter=html ./all-blob-reports

      - name: Upload HTML report
        uses: actions/upload-artifact@v4
        with:
          name: html-report--attempt-${{ github.run_attempt }}
          path: playwright-report
          retention-days: 14

From: https://playwright.dev/docs/test-sharding#github-actions-example

The output:

merging reports from /home/runner/path/to/workspace/all-blob-reports
extracting: all-blob-reports/report-1.zip
extracting: all-blob-reports/report-2.zip
extracting: all-blob-reports/report-3.zip
-----
Warning: No files were found with the provided path: playwright-report. No artifacts will be uploaded.

Expected behavior

I expect the report to be in the playwright-report directory.

Actual behavior

There's no output from the merged report, without any explanation as to why.

Additional context

This problem seems to happen sporadically, but I can't narrow down when it works and when it does not work. I originally though that it had something to do with rerunning jobs, but I don't think so.

I have also tried to sleep after generating the report, in case this was a timing issue. No luck. I have also tried to ls the contents of the work directory and the playwright-report directory, but sometimes it is just outright missing.

There is nothing in the merge-report output that ever explains what went wrong.

Environment

N/A. Running on GitHub Actions.

[pavelfeldman]

I expect the report to be in the playwright-report directory.

This is a wrong expectation. See https://playwright.dev/docs/test-reporters#blob-reporter for more details.

[OptimusCrime]

@pavelfeldman Not sure if I understand. The fragmented reports were generated by the blob reporter, but I am trying to combine these reports into a single report, using this command:

npx playwright merge-reports --reporter=html ./all-blob-reports

This uses the html reporter as far as I can tell? Maybe I have misunderstood? We're using the setup found here: https://playwright.dev/docs/test-sharding#github-actions-example

[pavelfeldman]

I misunderstood your case. If it happens sporadically, that would mean that someone is removing this folder. Playwright itself removed the folder when the tests are starting. Cold it be that you are running Playwright tests on the same machine?

[pavelfeldman]

We need more information to act on this report. As long as we can't repro it, it is unlikely with can make progress with it. Allow me to close it, but please file a new one and link to this issue when you get back to it!

[OptimusCrime]

You're right, there is something else weird that was going on. It seems to be working as it should now. I'll post here if we encounter the problem again. I suspect that it could have something to do with force pushing a rebase, but I can not get it to fail consistently 🤷‍♂️

Thanks for following up.