Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for WebAgg using Playwright #23540

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

QuLogic
Copy link
Member

@QuLogic QuLogic commented Aug 3, 2022

PR Summary

This may make WebAgg the most tested backend...

I'm not sure what to do about the screenshot test, as I really want to confirm that the figure is sent to the browser, but browsers differ in how they show the resize grip, which causes te comparison to fail.

PR Checklist

Tests and Styling

  • Has pytest style unit tests (and pytest passes).
  • Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

  • [n/a] New features are documented, with examples if plot related.
  • [n/a] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • [n/a] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
  • [n/a] Documentation is sphinx and numpydoc compliant (the docs should build without error).

@timhoffm
Copy link
Member

timhoffm commented Aug 3, 2022

I'm not sure what to do about the screenshot test, as I really want to confirm that the figure is sent to the browser, but browsers differ in how they show the resize grip, which causes te comparison to fail.

Can you mask the resize grip by drawing a white rectangle on top using imagemagic?

@tacaswell
Copy link
Member

result = next(iter(done)).result()
E       playwright._impl._api_types.Error: Executable doesn't exist at /home/vsts/.cache/ms-playwright/chromium-1015/chrome-linux/chrome
E       ╔════════════════════════════════════════════════════════════╗
E       ║ Looks like Playwright was just installed or updated.       ║
E       ║ Please run the following command to download new browsers: ║
E       ║                                                            ║
E       ║     playwright install                                     ║
E       ║                                                            ║
E       ║ <3 Playwright Team                                         ║
E       ╚════════════════════════════════════════════════════════════╝

looks like we need some extra steps in the CI setup?

@QuLogic
Copy link
Member Author

QuLogic commented Aug 3, 2022

I'm not sure what to do about the screenshot test, as I really want to confirm that the figure is sent to the browser, but browsers differ in how they show the resize grip, which causes te comparison to fail.

Can you mask the resize grip by drawing a white rectangle on top using imagemagic?

Actually, I forgot we have access to which browser is running the test, so I could just have two test images... I'll see what the diff looks like.

@QuLogic
Copy link
Member Author

QuLogic commented Oct 5, 2022

So this is nearly working on GitHub Actions. The current failure is on macOS. For some reason, the canvas gets a border and it's black on Linux, but blue on macOS. There's also some slight difference in the resize grip size.

AppVeyor only fails one test, so it's likely a similar deal.

Azure passes on Linux, but fails on macOS and Windows, but on several more tests. I'm not sure why that is yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants