Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge multiple functional tests together to reduce test startup time #797

Draft
wants to merge 1 commit into
base: 3.x
Choose a base branch
from

Conversation

mstenta
Copy link
Member

@mstenta mstenta commented Feb 9, 2024

I was looking into ways of improving the performance of our functional tests, and found this suggestion by @Berdir:

Another thing to consider is to avoid having many test methods. Each runs in a completely new, isolated environment. A trick that is often used is to have a single testAll() method and many doTestSomething() methods, then they are faster by almost factor count(test methods), since set up is usually by far the slowest part.

https://drupal.stackexchange.com/a/213561

This PR tests out this suggestion. It seems to shave off about 30 seconds from our tests overall, based on the few times that I've run them so far.

@Berdir
Copy link

Berdir commented Feb 9, 2024

I have been summoned ;)

FWIW, consider whether this is actually worth. It also reduces test isolation, which might cause weird bugs in tests and makes them harder/slower to run when debugging a specific test, you need to comment out things or so or you also always need to run a bulk of tests.

@mstenta
Copy link
Member Author

mstenta commented Feb 9, 2024

Thank you for your input and insight @Berdir, and for taking the time to offer it here! 😄 🙏

These are great considerations. I agree we should weigh the costs of doing something like this. I was curious how big a difference it would make. In the process I also discovered some tests that clearly did not need to be separate, so it was a good exercise. Perhaps that would be a good next step: look at the tests that were modified and decide on a case by case basis if there is value in keeping them separate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants