TST: Turn off codecov project reporting

[pllim]

Description

This pull request is to turn off codecov project reporting because it is meaningless and almost always fails now for no reason. This was requested by @astrofrog

Ref: https://docs.codecov.com/docs/commit-status#disabling-a-status

• By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

[github-actions]

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

[neutrinoceros]

is there any value in keeping this configuration file at all if everything is turned off ? For the record: I support ditching codecov, it hasn't been reliable for a while, and has degraded so much that we almost always just ignore it.

[pllim]

I think we still need it to turn things off, right? Otherwise, it pulls codecov defaults?

Ditching it is an upstream issue, please see OpenAstronomy/github-actions-workflows#189

[neutrinoceros]

If I understand correctly the upstream issue is about supporting report uploads natively in github actions, but it seems already possible to disable report upload altogether without preserving codecov.yml:

diff --git a/.github/workflows/ci_workflows.yml b/.github/workflows/ci_workflows.yml
index 9980b78510..f244528364 100644
--- a/.github/workflows/ci_workflows.yml
+++ b/.github/workflows/ci_workflows.yml
@@ -64,7 +64,7 @@ jobs:
         #       run only with minimal dependencies.
         - name: Python 3.12 with minimal dependencies and full coverage
           linux: py312-test-cov
-          coverage: codecov
+          coverage: ''
 
         - name: Python 3.11 in Parallel with all optional dependencies
           linux: py311-test-alldeps-fitsio
@@ -79,7 +79,7 @@ jobs:
         - name: Python 3.10 with oldest supported version of all dependencies
           linux: py310-test-oldestdeps-alldeps-cov-clocale
           posargs: --remote-data=astropy
-          coverage: codecov
+          coverage: ''
 
         - name: Python 3.11 with all optional dependencies (Windows)
           windows: py311-test-alldeps

[neutrinoceros]

I also wonder if CI jobs that have the -cov tox factor aren't just wasted if we keep them around with upload disabled. (the tox factor itself might still be useful for local coverage measurements)

[pllim]

@neutrinoceros , no I think the OpenAstronomy issue is about ditching codecov and upload coverage.xml only with Actions. But maybe @Cadair can confirm.

[astrofrog]

I think we still want to keep parch reporting right?

[pllim]

Yes, I think patch report still useful (for now). And you can still see that in this PR.

[Cadair]

@neutrinoceros is correct if you don't want to upload coverage in a job you can disable it as in the example.

However, I don't know if just not uploading anything to codecov is enough to shut it up for if you need to remove the app or disable it in config.

[nstarman]

It's funny. In many of my repos I have similar issues with Codecov, but it's the patch cov that's flaky and the project coverage that I still use. IDK what's up with codecov.

[mhvk]

👍 to removing codecov altogether. Couldn't agree more with https://hynek.me/articles/ditch-codecov-python/

Unfortunately, Codecov has grown very flaky. I have lost any confidence in the fact when it fails a build and my first reaction is always to restart the build and only then investigate. Sometimes the upload fails, sometimes Codecov fails to report its status back to GitHub, sometimes it can’t find the build, and sometimes it reports an outdated status. What a waste of computing power. What a waste of my time, clicking through their web application, seeing everything green, yet the build is failing due to missing coverage.

[pllim]

Codecov does have a nice UI though.