Pytest 8 broke sorting for package-scoped fixtures

[bmerry]

Pytest 8 has a regression where the test reordering no longer favours reuse of package-scoped fixtures. If a parametrised package-scoped fixture is used in multiple modules within the same package, then instead of running the whole package with one fixture instantiation before switching to the next, it runs a module at a time, causing the fixture to be torn down and recreated each time.

MWE

conftest.py:

import pytest

@pytest.fixture(scope="package", params=["a", "b", "c"])
def fixture(request):
    return request.param

pkg/__init__.py: empty

pkg/test_things1.py:

def test1(fixture):
    pass

pkg/test_things2.py:

def test2(fixture):
    pass

Output of pytest --setup-plan from pytest 8.2.0:

======================================= test session starts ========================================
platform linux -- Python 3.12.3, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/bmerry/work/experiments/NGC-1318-pytest-ordering/simple
collected 6 items                                                                                  

pkg/test_things1.py 
  SETUP    P fixture['a']
        pkg/test_things1.py::test1[a] (fixtures used: fixture, request)
  TEARDOWN P fixture['a']
  SETUP    P fixture['b']
        pkg/test_things1.py::test1[b] (fixtures used: fixture, request)
  TEARDOWN P fixture['b']
  SETUP    P fixture['c']
        pkg/test_things1.py::test1[c] (fixtures used: fixture, request)
pkg/test_things2.py 
  TEARDOWN P fixture['c']
  SETUP    P fixture['a']
        pkg/test_things2.py::test2[a] (fixtures used: fixture, request)
  TEARDOWN P fixture['a']
  SETUP    P fixture['b']
        pkg/test_things2.py::test2[b] (fixtures used: fixture, request)
  TEARDOWN P fixture['b']
  SETUP    P fixture['c']
        pkg/test_things2.py::test2[c] (fixtures used: fixture, request)
  TEARDOWN P fixture['c']

Whereas pytest 7.4.4 produces:

platform linux -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0
rootdir: /home/bmerry/work/experiments/NGC-1318-pytest-ordering/simple
collected 6 items                                                                                  

pkg/test_things1.py 
  SETUP    P fixture['a']
        pkg/test_things1.py::test1[a] (fixtures used: fixture, request)
pkg/test_things2.py 
        pkg/test_things2.py::test2[a] (fixtures used: fixture, request)
pkg/test_things1.py 
  TEARDOWN P fixture['a']
  SETUP    P fixture['b']
        pkg/test_things1.py::test1[b] (fixtures used: fixture, request)
pkg/test_things2.py 
        pkg/test_things2.py::test2[b] (fixtures used: fixture, request)
pkg/test_things1.py 
  TEARDOWN P fixture['b']
  SETUP    P fixture['c']
        pkg/test_things1.py::test1[c] (fixtures used: fixture, request)
pkg/test_things2.py 
        pkg/test_things2.py::test2[c] (fixtures used: fixture, request)
  TEARDOWN P fixture['c']

====================================== no tests ran in 0.01s =======================================

If the fixture is not parametric (or has only one parameter), it does correctly re-use the fixture across the whole package, so the issue seems to be related to sorting rather than re-use.

Changing the scope to session gives sensible behaviour, so it seems to be specific to package scope.

System information

pip list:

Package    Version
---------- -------
iniconfig  2.0.0
packaging  24.0
pip        24.0
pluggy     1.5.0
pytest     8.2.0
setuptools 69.5.1
wheel      0.43.0

OS: Ubuntu 24.04

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[bluetech]

I think I agree this is a regression. This broke with a21fb87 (released in 8.0.0). Specifically, the reorder_items function stopped reordering the items in this case. I will take a look.