JSEnvs are only closed after all tests are run in aggregate project

[sbrunk]

As stated in the README

Executing test will open several browser windows and close them all before the end of the tests.

This behavior seems to cause spawning a lot of browser processes when running test in an aggregate project, which are only stopped after all tests have finished.

Since switching to Firefox in softwaremill/tapir#860 due to scala-js/scala-js-env-selenium#119/#105, this causes issues on memory constrained CI runners. Firefox seems to be a bit more memory hungry than Chrome, so we're only seeing this now but the behavior is the same in Chrome as well.

Is there a way to terminate browser processes after each test run or after the tests of single subproject have finished, when running test on an aggregate project in sbt?

Note that in parallel execution is already disabled in the build:

concurrentRestrictions in Global += Tags.limit(Tags.Test, 1),
Test / parallelExecution := false

[sjrd]

AFAIK, there is no way to force that using configuration. The only way is to execute the tests of each project in a separate sbt command. Browsers will be cleaned after each sbt command completes (literally in the onComplete event of sbt).

[gzm0]

This is unfortunately another shortcoming of sbt.testing._ (although a much more subtle one): There is no resource cleanup hook for resources shared between multiple Frameworks.

In Scala.js' sbt plugin, we might have the option to intercept sbt's testing infrastructure more deeply and get a better point to call TestAdapter#close().

The obvious downside of this would of course be that we'd rely on much more internal details of how sbt executes tests.

IMHO, this is a Scala.js core issue (notably, it is, in my opinion, not an issue with the JSEnv interface). Unless there are objections, I'll transfer this issue to scala-js/scala-js.

[sbrunk]

Hmm seems I have a talent to always open issues in the wrong repo. ;)

Thanks @sjrd @gzm0. Executing test explicitly for each subproject works fine for now.

[gzm0]

I have asked about this on sbt/sbt: sbt/sbt#6603

[gzm0]

Note to self: We could build a heuristic that stops all JSEnvs (per TestAdapter) when all ongoing runs have terminated and a timeout has elapsed (whichever happens last). In the (unlikely) case where more testing gets invoked, we could just start the runners again.

This is less clean, but likely solves the problem.

[gzm0]

After having attempted to build said heuristic, I'm not comfortable proceeding: From what I can see, it has the potential to worsen currently working use cases (notably the workaround to this very issue).