When one subcase fails, other independent sibling subcases are not executed anymore

[svdHero]

Description

The documentation states:

For each SUBCASE() the TEST_CASE() is executed from the start

and

Each leaf subcase (a subcase that contains no nested subcases) will be executed exactly once on a separate path of execution from any other leaf subcase (so no leaf subcase can interfere with another).

However, when one leaf subcase fails, other, independent, sibling subcase leafs are not executed anymore. This is highly unsatisfying and not what I expect migrating from Catch2 and based on the documentation above.

Steps to reproduce

Execute the following tests.

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include "doctest.h"

SCENARIO("My Scenario")
{
    GIVEN("Something")
    {
        WHEN("A")
        {
            THEN("Expected result A-1")
            {
                REQUIRE(0==0);  // Succeeds
            }
            THEN("Expected result A-2")
            {
                REQUIRE(0==1);  // Fails
            }
            THEN("Expected result A-3")
            {
                REQUIRE(0==0);  // Is Not executed
            }
        }
        WHEN("B")
        {
            THEN("Expected result B-1")
            {
                REQUIRE(0==1);  // Is not executed
            }
            THEN("Expected result B-2")
            {
                REQUIRE(0==0);  // Is not executed
            }
        }
    }
}

Test output is:

2 test cases: 1 failed, 1 succeeded

So only 2 test cases are executed instead of 5.

When "A-2" fails, the whole scenario is stopped. Why are "A-3", "B-1" and "B-2" not executed?

They should be totally independent of "A-2".

Expected / desired behaviour

When one leaf fails, still continue with the execution of ancestor and sibling paths to other independent leafs. Compare Catch2 behaviour.

Extra information

• doctest version: 2.4.9
• Operating System: Ubuntu 20.04.6 LTS x86_64
• Compiler+version: Platformio Core 6.1.13 Home 3.4.4 via Platformio IDE 3.3.3 (I assume gcc 9.4.0 for native platform)

[mitchgrout]

Issue still exists on 2.4.11.

As a note, you have your desired behaviour if your REQUIRE is replaced with CHECK. However, REQUIRE is good practice for things like asserting valid memory prior to dereferencing it in a CHECK. Can't immediately see what the issue is but I can try having a dig around.

Unsure if the repo has any active maintainers at the moment though, so even if it's patchable, unsure if it'll be merged.

[svdHero]

@mitchgrout thanks for your comment. Yes, I know about CHECK. However, I do need the REQUIRE functionality in my case, because within the test case, I want to fail fast and not continue. Other independent test cases should still be unaffected, as specified by the documentation.

Have you had any luck, digging around?