Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Playwright performance tests to Product editor #47590

Merged
merged 8 commits into from
May 21, 2024

Conversation

octaedro
Copy link
Contributor

@octaedro octaedro commented May 17, 2024

Submission Review Guidelines:

Changes proposed in this Pull Request:

This PR adds some Playwright performance tests to the Product editor.

The following are the metrics we will be measuring:

  • totalBlockingTime
    (TBT) is a performance metric that measures the total amount of time that a page is blocked from responding to user input.

  • cumulativeLayoutShift
    (CLS) is a metric that measures visual stability, indicating how much visible content shifts around on the screen during the viewport's lifetime.

  • largestContentfulPaint.
    (LCP) measures perceived load speed and marks the point in the page load timeline when the page's main content has likely loaded.

  • typing
    Measures the typing performance in a text input field.

  • serverResponse.
    Time taken by the server to start sending a response to a client's request.

  • firstPaint
    The time when any visual change happens on the screen—anything that differs from the previous view.

  • domContentLoaded
    Indicates that the initial HTML document has been completely loaded and parsed, without waiting for stylesheets, images, and subframes to finish loading.

  • loaded
    Refers to the time it takes for the entire page to be fully loaded, including all sub-resources such as stylesheets, images, and scripts.

  • firstContentfulPaint
    (FCP) is used to measure when the first piece of content from the DOM is rendered on the screen.

  • firstBlock
    Indicates the duration from when the browser has finished receiving the complete HTML response from the server to the current time. It shows how much time has passed since the server has finished sending the HTML until a particular point in the page load.

Screenshot 2024-05-20 at 1 49 34 PM

So far, we are only measuring Type in the editor and First block in the editor. As far as I know these metrics need to be also defined in the database of codevitals.run, so I think we will need the help of @youknowriad to create them.

Closes #47291.

How to test the changes in this Pull Request:

Using the WooCommerce Testing Instructions Guide, include your detailed testing instructions:

  1. To run this process, you will have to run the following commands
cd plugins/woocommerce
pnpm test:metrics

Changelog entry

  • Automatically create a changelog entry from the details below.

Significance

  • Patch
  • Minor
  • Major

Type

  • Fix - Fixes an existing bug
  • Add - Adds functionality
  • Update - Update existing functionality
  • Dev - Development related task
  • Tweak - A minor adjustment to the codebase
  • Performance - Address performance issues
  • Enhancement - Improvement to existing functionality

Message

Comment

@octaedro octaedro self-assigned this May 17, 2024
@github-actions github-actions bot added the plugin: woocommerce Issues related to the WooCommerce Core plugin. label May 17, 2024
@octaedro octaedro added focus: new product ux revamped product management experience team: Mothra labels May 17, 2024
@github-actions github-actions bot added the focus: monorepo infrastructure Issues and PRs related to monorepo tooling. label May 20, 2024
@octaedro octaedro requested review from a team and youknowriad May 20, 2024 17:16
Copy link
Contributor

github-actions bot commented May 20, 2024

Hi @youknowriad, @woocommerce/mothra

Apart from reviewing the code changes, please make sure to review the testing instructions as well.

You can follow this guide to find out what good testing instructions should look like:
https://github.com/woocommerce/woocommerce/wiki/Writing-high-quality-testing-instructions

@octaedro octaedro marked this pull request as ready for review May 20, 2024 17:16
@youknowriad
Copy link
Contributor

Happy to add these to codevitals.run database when it's ready to merge.

Also, worth nothing that in the Gutenberg repository, we added a markdown summary for the performance job that can be seen on the job's summary page on GitHub. It might be a good addition to the woo repo too.

Copy link
Contributor

@mdperez86 mdperez86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @octaedro nice job!!!

@youknowriad
Copy link
Contributor

I've added two of these metrics to the woo dashboard https://www.codevitals.run/project/woo (let me know if you want to add more there) They'll start filling as soon as this PR is merged.

@octaedro octaedro merged commit c4e1ceb into trunk May 21, 2024
24 checks passed
@octaedro octaedro deleted the dev/47291_add_playwright_performance_tests branch May 21, 2024 20:24
@github-actions github-actions bot added this to the 9.0.0 milestone May 21, 2024
@github-actions github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label May 21, 2024
@youknowriad
Copy link
Contributor

The result of the "type" metric is a bit suspicious to me. I wonder if you're really testing the right thing here (typing on the right thing for instance), you might want to try to run it in "headed" mode to check.

@Stojdza Stojdza added status: analysis complete Indicates if a PR has been analysed by Solaris and removed needs: analysis Indicates if the PR requires a PR testing scrub session. labels May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
focus: monorepo infrastructure Issues and PRs related to monorepo tooling. focus: new product ux revamped product management experience plugin: woocommerce Issues related to the WooCommerce Core plugin. status: analysis complete Indicates if a PR has been analysed by Solaris team: Mothra
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spike: Analyze key metrics and how to measure them with Playwright
4 participants