Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support of masking olci l1b products by using quality flags #2687

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

yufeizhu600
Copy link
Contributor

This PR added support of using the quality flags from the qualityFlags.nc file to mask the OCLI L1b products, similar as using wqsf file masking OLCI L2 products.

Copy link

codecov bot commented Dec 13, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (338817d) 95.32% compared to head (fd5dc92) 95.35%.
Report is 24 commits behind head on main.

Files Patch % Lines
satpy/readers/olci_nc.py 94.11% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2687      +/-   ##
==========================================
+ Coverage   95.32%   95.35%   +0.02%     
==========================================
  Files         371      371              
  Lines       52435    52440       +5     
==========================================
+ Hits        49982    50002      +20     
+ Misses       2453     2438      -15     
Flag Coverage Δ
behaviourtests 4.15% <0.00%> (-0.02%) ⬇️
unittests 95.96% <97.26%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

coveralls commented Dec 13, 2023

Pull Request Test Coverage Report for Build 7212041175

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report.
To ensure accuracy in future PRs, please see these guidelines.
A quick fix for this PR: rebase it; your next report should be accurate.

  • 71 of 73 (97.26%) changed or added relevant lines in 2 files are covered.
  • 49 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.03%) to 95.915%

Changes Missing Coverage Covered Lines Changed/Added Lines %
satpy/readers/olci_nc.py 32 34 94.12%
Files with Coverage Reduction New Missed Lines %
satpy/tests/test_resample.py 2 99.57%
satpy/composites/init.py 4 93.07%
satpy/resample.py 43 88.74%
Totals Coverage Status
Change from base Build 7188190011: 0.03%
Covered Lines: 50128
Relevant Lines: 52263

💛 - Coveralls

Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't pretend to understand everything that's going on here, but I had a couple questions and suggestions to start with.

satpy/readers/olci_nc.py Show resolved Hide resolved
satpy/tests/reader_tests/test_olci_nc.py Outdated Show resolved Hide resolved
satpy/tests/reader_tests/test_olci_nc.py Outdated Show resolved Hide resolved
"""Init the file handler."""
super().__init__(filename, filename_info, filetype_info, engine)
self.cal = cal.nc
if cal is not None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In what conditions is cal None? Should this be required (but keep it as a kwarg for consistency) and raise an error if it is None?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cal will be None when loading mask into the scene. In that case, that is alright since we only care about the mask_items.

Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know enough about this reader to have the final say, I think @mraspaud should do that. This looks good enough to me though.

@djhoese djhoese added enhancement code enhancements, features, improvements component:readers labels Dec 14, 2023
Comment on lines +149 to +155
mocked_dataset.return_value = xr.Dataset({"quality_flags": (["rows", "columns"],
np.array([1 << (x % 32) for x in range(35)]).reshape(5, 7))},
coords={"rows": np.arange(5),
"columns": np.arange(7)})
ds_id = make_dataid(name="mask")
filename_info = {"mission_id": "S3A", "dataset_name": "mask", "start_time": 0, "end_time": 0}
test = NCOLCI1B("somedir/somefile.nc", filename_info, "c")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on how bored you are, you could change this to not using mocking but instead create a real file on disk. I realize you're just copying what was here before which is why I won't push it.

@yufeizhu600
Copy link
Contributor Author

I don't know enough about this reader to have the final say, I think @mraspaud should do that. This looks good enough to me though.

Thanks Dave.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:readers enhancement code enhancements, features, improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OLCI reader can't process quality flags
3 participants