Coverage measurement for Cython code

[oscarbenjamin]

Currently spin test supports --coverage and --gcov arguments for measuring coverage of python or C code respectively. For python-flint the one outstanding item in moving from setuptools to spin/meson is coverage measurement of Cython code.

A meson project needs to have some sort of configuration option for coverage of Cython e.g.:

if get_option('coverage')
   add_project_arguments('-X', 'linetrace=True', language : 'cython')
   add_project_arguments('-DCYTHON_TRACE=1', language : 'c')
endif

In python-flint's case we need this separately from b_coverage=true and spin's --gcov option because we explicitly do not want to measure coverage of C code (which is all Cython-generated).

I'm not sure what the best approach would be for spin to implement spin test --coverage-cython given that there is not a standard meson option for this.

Maybe spin could configure the CYTHON_TRACE and linetrace directly?

Currently it seems that Cython's coverage plugin does not work in a spin build-install directory which I think is because the plugin tries to look for Cython source files adjacent to the C files. Possibly that is something that needs to be fixed in Cython or maybe it is possible for spin to tell coverage where to find the source files.

[oscarbenjamin]

I've made a repo that demonstrates how to get working Cython coverage with setuptools but also shows what does not work with spin:
https://github.com/oscarbenjamin/cython_coverage_demo

At least some change in Cython's coverage plugin is needed.

Also possibly the way that spin/meson generate the .c files results in them containing the wrong path.

[stefanv]

@oscarbenjamin I'd like to see Cython coverage work. I'm a bit tied up for the next few days, but will take a look after that, if you haven't found a solution by then already.

[oscarbenjamin]

The primary blocker to Cython code coverage with spin/meson is that changes are needed in Cython's coverage plugin. I opened a related Cython issue cython/cython#6186.

[oscarbenjamin]

I have described in cython/cython#6186 (comment) how I think we can make coverage work with Cython for a spin/meson-based project.

I think that the proper solution is that spin/meson needs to make something like a json file that tells the coverage plugin where all the files are.