Skip to content

Add Lighthouse benchmarks#1944

Merged
MackinnonBuck merged 18 commits intomainfrom
mbuck/lighthouse-metrics
Mar 4, 2024
Merged

Add Lighthouse benchmarks#1944
MackinnonBuck merged 18 commits intomainfrom
mbuck/lighthouse-metrics

Conversation

@MackinnonBuck
Copy link
Member

@MackinnonBuck MackinnonBuck commented Feb 27, 2024

Add Lighthouse benchmarks

This PR includes the following changes:

  • Added Lighthouse testing job
    • Runs against the specified URL, so it's a general-purpose job that's not tied to a specific application
    • Allows for running Lighthouse multiple times and averaging the results
    • Allows an additional "warm-up" run so that the tests run against a fully initialized server
      • Note: The warm-up does not affect the client. We currently clear the Chrome cache between each run, but we could consider adding tests that run against a client cached assets.
    • Tests the following metrics:
      • First Contentful Paint
      • Interaction to Next Paint †
      • Largest Contentful Paint
      • Total Blocking Time
      • Time to Interactive
  • Added a "Run template" job
    • Creates, publishes, and runs a project created from an arbitrary project template
    • It's set up in a way that makes it easy to add additional template scenarios (only .yml config changes required)
  • Added Lighthouse scenarios for the Blazor Web App template:
    • Includes:
      • With Server interactivity
      • With WebAssembly interactivity

Still TODO:

  • Add more Lighthouse scenarios
    • Blazor Web App with Auto interactivity
    • Blazor WebAssembly standalone
  • † Add support for Lighthouse user flows. This will allow us to collect metrics for scenarios other than loading pages (e.g., typing into a text field, clicking a button/link, etc.).
  • Utilize the changes from Add measurements api dotnet/crank#703
  • Document how to run these benchmarks locally

Fixes dotnet/aspnetcore#29303

@MackinnonBuck MackinnonBuck force-pushed the mbuck/lighthouse-metrics branch 8 times, most recently from 499ab95 to 4444b53 Compare February 29, 2024 22:28
Copy link
Member

@sebastienros sebastienros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending changing source branches to main
Next PR will be adding it to the CI

@MackinnonBuck MackinnonBuck force-pushed the mbuck/lighthouse-metrics branch 5 times, most recently from 727ff73 to fc9ce09 Compare March 1, 2024 19:39
@MackinnonBuck MackinnonBuck force-pushed the mbuck/lighthouse-metrics branch 10 times, most recently from b1730ad to 3070847 Compare March 2, 2024 00:48
@MackinnonBuck MackinnonBuck force-pushed the mbuck/lighthouse-metrics branch from 3070847 to 545f097 Compare March 2, 2024 01:04
@MackinnonBuck MackinnonBuck marked this pull request as ready for review March 2, 2024 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Blazor] Add lighthouse score to automated perf tests

2 participants