Dispatched nx-loopback tests performance degradation

[rossbar]

This is something I noticed while investigating #7354. The test suite when run with the nx-loopback backend is significantly slower than when run without loopback. I'd expect it to be a little slower due to the extra conversion overhead, but AIUI that should be limited to adding a single attribute to the graph which should not be very expensive. The total performance degradation is signficantly greater than what I'd expect from additional call/conversion overhead.

For example, I've noticed that test_omega is an order of magnitude slower with loopback enabled:

Without loopback

pytest -k test_omega networkx/algorithms/tests/test_smallworld.py =========================================================== test session starts ===========================================================
platform linux -- Python 3.11.6, pytest-8.1.1, pluggy-1.4.0
Matplotlib: 3.8.3
Freetype: 2.6.1
configfile: pyproject.toml
plugins: mpl-0.17.0, cov-5.0.0, xdist-3.5.0
collected 6 items / 5 deselected / 1 selected                                                                                             

networkx/algorithms/tests/test_smallworld.py .                                                                                      [100%]

===================================================== 1 passed, 5 deselected in 2.07s =====================================================

With loopback

=========================================================== test session starts ===========================================================
platform linux -- Python 3.11.6, pytest-8.1.1, pluggy-1.4.0
Matplotlib: 3.8.3
Freetype: 2.6.1
configfile: pyproject.toml
plugins: mpl-0.17.0, cov-5.0.0, xdist-3.5.0
collected 6 items / 5 deselected / 1 selected                                                                                             

networkx/algorithms/tests/test_smallworld.py .                                                                                      [100%]

==================================================== 1 passed, 5 deselected in 36.31s =====================================================

I haven't done an exhaustive comparison to see which tests are most affected, but this is probably a good place to start looking as it's the worst offender (at least on my system).

[eriknw]

_dispatchable._convert_and_call_for_tests does a lot more when running tests. Some examples:

• Avoid creating results with numpy scalars (re: NEP 51) #7282 checks results for numpy scalars
  • this is brutal
• Compare graphs for generator functions when running tests with backend #7066 compares results for functions that return graphs
  • i.e., it calls the the backend func and the default networkx func, converts the backend graph to nx graph, compares them, then returns nx graph
• Add @nx._dispatch decorator to most algorithms #6688 made the nx-loopback backend actually copy the graphs using the dispatchable metadata

_convert_and_call_for_tests is generally regarded as (pragmatic) technical debt. It would be great to make it do less (with the aim to delete it entirely) and find other ways to e.g. check for numpy scalars. I aim to pay off some technical debt this year (and probably beyond).

Maybe CI can run dispatchable tests in parallel (pytest-xdist)

Looking at an arbitrary recent PR, these are the slowest tests:

============================= slowest 10 durations =============================
98.13s call     networkx/algorithms/tests/test_smallworld.py::test_omega
44.24s call     networkx/algorithms/isomorphism/tests/test_tree_isomorphism.py::test_positive
35.57s call     networkx/algorithms/connectivity/tests/test_cuts.py::test_node_cutset_random_graphs
33.96s call     networkx/algorithms/connectivity/tests/test_connectivity.py::test_articulation_points
23.16s call     networkx/algorithms/tests/test_smallworld.py::test_sigma
21.21s call     networkx/algorithms/isomorphism/tests/test_tree_isomorphism.py::test_negative
19.85s call     networkx/algorithms/connectivity/tests/test_kcutsets.py::test_random_gnp
19.47s call     networkx/algorithms/tree/tests/test_branchings.py::test_arborescence_iterator_min
18.35s call     networkx/algorithms/flow/tests/test_gomory_hu.py::TestGomoryHuTree::test_karate_club_graph
17.42s call     networkx/algorithms/flow/tests/test_gomory_hu.py::TestGomoryHuTree::test_davis_southern_women_graph
===== 6151 passed, 59 skipped, 9 xfailed, 9 warnings in 708.94s (0:11:48) ======

[aaronzo]

The expensive numpy results checks are effectively a type check, from what I can see. This could be another argument for using more type checking in the codebase instead (related: #3988)