Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move FSFile to separate module and add some reader type annotations #2626

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Nov 2, 2023

This is me playing with type annotations for some of the ugliness that is reader creation. This got...interesting. If you thought you hated type annotations before, just look at these changes.

Oh yeah, I moved FSFile to a sub-module.

  • Closes #xxxx
  • Tests added
  • Fully documented
  • Add your name to AUTHORS.md if not there already

@djhoese djhoese added component:readers cleanup Code cleanup but otherwise no change in functionality labels Nov 2, 2023
@djhoese djhoese requested review from mraspaud and pnuu November 2, 2023 20:30
@djhoese djhoese self-assigned this Nov 2, 2023
Copy link

codecov bot commented Nov 2, 2023

Codecov Report

Merging #2626 (dd7052a) into main (86c075a) will decrease coverage by 0.01%.
Report is 24 commits behind head on main.
The diff coverage is 92.18%.

@@            Coverage Diff             @@
##             main    #2626      +/-   ##
==========================================
- Coverage   95.18%   95.18%   -0.01%     
==========================================
  Files         354      355       +1     
  Lines       51270    51331      +61     
==========================================
+ Hits        48803    48858      +55     
- Misses       2467     2473       +6     
Flag Coverage Δ
behaviourtests 4.26% <45.31%> (+0.02%) ⬆️
unittests 95.81% <92.18%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
satpy/readers/_fsfile.py 95.74% <95.74%> (ø)
satpy/readers/__init__.py 97.70% <82.35%> (-0.59%) ⬇️

... and 6 files with indirect coverage changes

@djhoese
Copy link
Member Author

djhoese commented Nov 3, 2023

I'll put a summary of my feelings and the short discussion I had with @mraspaud on this PR. In general I am more a fan of type annotations than @mraspaud or @pnuu (other core contributors who I've talked to about them). I say that because even I hate what this PR turned into. I decided to add type annotations where I did in this PR because of how annoyed I was that it wasn't clear what should be passed or what was returned. As I started adding type annotations it got messier and messier as I tried to inform mypy of what this load_readers function was actually doing.

The initial ReaderName TypeAlias was a "make it more clear that this str in the type annotations is a reader name, not just any string". And I think it kind of did that, but it also made it appear that ReaderName was something more complex and just added to the confusion of the function.

Next I wanted to encapsulate all of the different types that filenames can be. This wasn't too bad (although I learned that mypy was confused by str being the first type in a union in a type alias) and overall made it clear that we were passing a collection of path-like things.

Lastly, mypy was very confused by what reader_kwargs is/was and what was being done to it. I tried and tried to define it with dict upon dict, but it wasn't helping. I assumed I was missing something and made the type alias and when I did that it not only cleared it up for me, but made mypy finally understand what I was trying to define.

But all of this just made the type annotations decently ugly. One of those things where when you already know what it means it looks clean, but that's not who these annotations are for. They're for new users/contributors.

Martin and I basically agreed that this complexity is not a sign that we need more type annotations, but that the interfaces and this particular function is extremely bad and poorly designed. It should be rewritten or restructured. One suggestion I had was something along the lines of:

def some_func(..., reader_kwargs: Any) -> ...:
    sane_reader_kwargs = _sanitize_reader_kwargs(reader_kwargs)
    ...

Basically telling mypy don't worry about it, we'll sanitize the input ourselves. This however has the major downside of not informing the user in a IDE-friendly way if they're input is incorrect.

We'll hold off on this PR for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup Code cleanup but otherwise no change in functionality component:readers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant