Skip to content

Latest commit

 

History

History
1219 lines (1050 loc) · 57 KB

CHANGELOG.rst

File metadata and controls

1219 lines (1050 loc) · 57 KB
Raw

What's New

python

import climpred from climpred import HindcastEnsemble import matplotlib as mpl

mpl.rcdefaults() mpl.use("Agg") # cut border when saving (for maps) mpl.rcParams["savefig.bbox"] = "tight"

climpred v2.5.0 (unreleased)

Internals/Minor Fixes

  • Fixed some issues with the documentation build to address rendering errors and reduce the number of warnings on ReadTheDocs. (pr:843) Trevor James Smith
  • Fixed some issues with the typing hints of classes functions. (pr:850) Trevor James Smith

climpred v2.4.0 (2023-11-09)

Internals/Minor Fixes

climpred v2.3.0 (2022-11-25)

Note

As both maintainers moved out of academia into industry, this will be probably the last release for a while. If you are interested in maintaining climpred, please ping us.

Bug Fixes

  • Fix reference="persistence" for resampled init. (730, 731) Aaron Spring.
  • :py.HindcastEnsemble.verify (comparison="m2o", reference="uninitialized", dim="init"). (735, 731) Aaron Spring.
  • :py.HindcastEnsemble.remove_bias does not drop single item lead dimension. (771, 773) Aaron Spring.

New Features

  • Refactored :py.HindcastEnsemble.bootstrap and :py.PerfectModelEnsemble.bootstrap based on :py.HindcastEnsemble.verify and :py.PerfectModelEnsemble.verify, which makes them more comparable. pers_sig is removed. Also reference=["climatology", "persistence"] skill has variance if resample_dim='init'. bootstrap relies on either set_option(resample_skill_func="..."):

    • "loop": calls :pyclimpred.bootstrap.resample_skill_loop which loops over iterations and calls verify every single time. Most understandable and stable, but slow.
    • "exclude_resample_dim_from_dim": calls :pyclimpred.bootstrap.resample_skill_exclude_resample_dim_from_dim which calls verify(dim=dim_without_resample_dim), resamples over resample_dim and then takes a mean over resample_dim if in dim. Enables HindcastEnsemble.bootstrap(resample_dim="init", alignment="same_verifs"). Fast alternative for resample_dim="init".
    • "resample_before": calls :pyclimpred.bootstrap.resample_skill_resample_before which resamples iteration dimension and then calls verify vectorized. Fast alternative for resample_dim="member".
    • "default": climpred decides which to use

    (relates to 375, 731) Aaron Spring.

  • climpred.set_option(resample_skill_func='exclude_resample_dim_from_dim') allows HindcastEnsemble.bootstrap(alignment='same_verifs', resample_dim='init'). Does not work for pearson_r-derived metrics. (582, 731) Aaron Spring.
  • :pyclimpred.utils.convert_init_lead_to_valid_time_lead converts data(init, lead) to data(valid_time, lead) to visualize predictability barrier and the reverse :pyclimpred.utils.convert_valid_time_lead_to_init_lead. (774, 775, 783) Aaron Spring.

Internals/Minor Fixes

  • Refactor asv benchmarking. Add run-benchmarks label to PR to run asv via Github Actions. (664, 718) Aaron Spring.
  • Remove ipython from requirements.txt. (720) Aaron Spring.
  • Calculating np.isin on asi8 instead of xr.CFTimeIndex speeds up :py.HindcastEnsemble.verify and :py.HindcastEnsemble.bootstrap with large number of inits. (414, 724) Aaron Spring.
  • Add option bootstrap_resample_skill_func for they what skill is resampled in :py.HindcastEnsemble.bootstrap and :py.PerfectModelEnsemble.bootstrap, see :py~climpred.options.set_options. (731) Aaron Spring.
  • Add option resample_iterations_func to decide whether :pyxskillscore.resampling.resample_iterations or :pyxskillscore.resampling.resample_iterations should be used, see :py~climpred.options.set_options. (731) Aaron Spring.

    - Add option bootstrap_uninitialized_from_iterations_mean to exchange uninitialized skill with the iteration mean uninitialized. Defaults to False., see :py~climpred.options.set_options. (731) Aaron Spring.

  • alignment="same_verifs" will not result in NaNs in valid_time. (777) Aaron Spring.
  • :py.HindcastEnsemble.plot_alignment (return_xr=True) contains valid_time coordinate. (779) Aaron Spring.

Bug Fixes

  • Fix PerfectModel_persistence_from_initialized_lead_0=True with multiple references. (732, 733) Aaron Spring.

Documentation

  • Add verify dim example showing how :py.HindcastEnsemble.verify and :py.PerfectModelEnsemble.verify are sensitive to dim and how dim answers different research questions. (740) Aaron Spring.

climpred v2.2.0 (2021-12-20)

Bug Fixes

  • Fix when creating valid_time from lead.attrs["units"] in ["seasons", "years"] with multi-month stride in init. (698, 700) Aaron Spring.
  • Fix seasonality="season" in reference="climatology". (641, 703) Aaron Spring.

New Features

  • Upon instantiation, :py.PredictionEnsemble generates new 2-dimensional coordinate valid_time for initialized from init and lead, which is matched with time from verification during alignment. (575, 675, 678) Aaron Spring.

>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST") >>> hind.lead.attrs["units"] = "years" >>> climpred.HindcastEnsemble(hind).get_initialized() <xarray.Dataset> Dimensions: (lead: 10, member: 10, init: 64) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * member (member) int32 1 2 3 4 5 6 7 8 9 10 * init (init) object 1954-01-01 00:00:00 ... 2017-01-01 00:00:00 valid_time (lead, init) object 1955-01-01 00:00:00 ... 2027-01-01 00:00:00 Data variables: SST (init, lead, member) float64 ...

  • Allow lead as float also if calendar="360_day" or lead.attrs["units"] not in ["years","seasons","months"]. (564, 675) Aaron Spring.
  • Implement :py.HindcastEnsemble.generate_uninitialized resampling years without replacement from initialized. (589, 591) Aaron Spring.
  • Implement Logarithmic Ensemble Skill Score :py~climpred.metrics._less. (239, 687) Aaron Spring.
  • :py.HindcastEnsemble.remove_seasonality and :py.PerfectModelEnsemble.remove_seasonality remove the seasonality of all climpred datasets. (530, 688) Aaron Spring.
  • Add keyword groupby in :py.HindcastEnsemble.verify, :py.PerfectModelEnsemble.verify, :py.HindcastEnsemble.bootstrap and :py.PerfectModelEnsemble.bootstrap to group skill by initializations seasonality. (635, 690) Aaron Spring.

>>> import climpred >>> hind = climpred.tutorial.load_dataset("NMME_hindcast_Nino34_sst") >>> obs = climpred.tutorial.load_dataset("NMME_OIv2_Nino34_sst") >>> hindcast = climpred.HindcastEnsemble(hind).add_observations(obs) >>> # skill for each init month separated >>> skill = hindcast.verify( ... metric="rmse", ... dim="init", ... comparison="e2o", ... skipna=True, ... alignment="maximize", ... groupby="month", ... ) >>> skill <xarray.Dataset> Dimensions: (month: 12, lead: 12, model: 12) Coordinates: * lead (lead) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0 * model (model) object 'NCEP-CFSv2' 'NCEP-CFSv1' ... 'GEM-NEMO' skill <U11 'initialized' * month (month) int64 1 2 3 4 5 6 7 8 9 10 11 12 Data variables: sst (month, lead, model) float64 0.4127 0.3837 0.3915 ... 1.255 3.98 >>> skill.sst.plot(hue="model", col="month", col_wrap=3)

  • :py.HindcastEnsemble.plot_alignment shows how forecast and observations are aligned based on the alignment keyword. This may help understanding which dates are matched for the different alignment approaches. (701, 702) Aaron Spring.

    python

    from climpred.tutorial import load_dataset

    hindcast = climpred.HindcastEnsemble(

    load_dataset("CESM-DP-SST")

    ).add_observations(load_dataset("ERSST")) @savefig plot_alignment_example.png width=100% hindcast.plot_alignment(edgecolor="w")

  • Add attrs to new coordinates created by climpred. (695, 697) Aaron Spring.
  • Add seasonality="weekofyear" in reference="climatology". (703) Aaron Spring.
  • Compute reference="persistence" in :py.PerfectModelEnsemble from initialized first lead if :py~climpred.options.set_options (PerfectModel_persistence_from_initialized_lead_0=True) (False by default) using :py~climpred.reference.compute_persistence_from_first_lead. (637, 706) Aaron Spring.

Internals/Minor Fixes

  • Reduce dependencies. (686) Aaron Spring.
  • Add typing. (685, 692) Aaron Spring.
  • refactor add_attrs into :py.HindcastEnsemble.verify and :py.HindcastEnsemble.bootstrap. Now all keywords are captured in the skill dataset attributes .attrs. (475, 694) Aaron Spring.
  • docstrings formatting with blackdocs. (708) Aaron Spring.

Documentation

  • Refresh all docs with sphinx_book_theme and myst_nb. (707, 708, 709, 710) Aaron Spring.

climpred v2.1.6 (2021-08-31)

Adding on to v2.1.5, more bias reduction methods wrapped from xclim are implemented.

Bug Fixes

  • Fix results="p" in :py.HindcastEnsemble.bootstrap and :py.PerfectModelEnsemble.bootstrap when reference='climatology'. (668, 670) Aaron Spring.
  • :py.HindcastEnsemble.remove_bias for how in ["modified_quantile", "basic_quantile", "gamma_mapping", "normal_mapping"] from bias_correction takes all member to create model distribution. (667) Aaron Spring.

New Features

  • allow more bias reduction methods wrapped from xclim in :py.HindcastEnsemble.remove_bias:

    • how="EmpiricalQuantileMapping": :pyxclim.sdba.adjustment.EmpiricalQuantileMapping
    • how="DetrendedQuantileMapping": :pyxclim.sdba.adjustment.DetrendedQuantileMapping
    • how="PrincipalComponents": :pyxclim.sdba.adjustment.PrincipalComponents
    • how="QuantileDeltaMapping": :pyxclim.sdba.adjustment.QuantileDeltaMapping
    • how="Scaling": :pyxclim.sdba.adjustment.Scaling
    • how="LOCI": :pyxclim.sdba.adjustment.LOCI

    These methods do not respond to OPTIONS['seasonality'] like the other methods. Provide group="init.month" to group by month or group='init' to skip grouping. Provide group=None or skip group to use init.{OPTIONS['seasonality']}. (525, 662, 666, 671) Aaron Spring.

climpred v2.1.5 (2021-08-12)

While climpred has used in the ASP summer colloquium 2021, many new features in :py.HindcastEnsemble.remove_bias were implemented.

Breaking changes

  • renamed cross_validate to cv=False in :py.HindcastEnsemble.remove_bias. Only used when train_test_split='unfair-cv'. (648, 655). Aaron Spring.

Bug Fixes

  • Shift back init by lead after :py.HindcastEnsemble.verify. (644, 645) Aaron Spring.

New Features

  • :py.HindcastEnsemble.remove_bias accepts new keyword train_test_split='fair/unfair/unfair-cv' (default unfair) following Risbey et al. 2021. (648, 655) Aaron Spring.

  • allow more bias reduction methods in :py.HindcastEnsemble.remove_bias:

    • how="additive_mean": correcting the mean forecast additively (already implemented)
    • how="multiplicative_mean": correcting the mean forecast multiplicatively
    • how="multiplicative_std": correcting the standard deviation multiplicatively

    Wrapped from bias_correction:

  • :py.HindcastEnsemble.remove_bias now does leave-one-out cross validation when passing cv='LOO' and train_test_split='unfair-cv'. cv=True falls back to cv='LOO'. (643, 646) Aaron Spring.
  • Add new metrics :py~climpred.metrics._spread and :py~climpred.metrics._mul_bias (638) Aaron Spring.

  • Add new tutorial datasets: (651) Aaron Spring.

    • NMME_OIv2_Nino34_sst and NMME_hindcast_Nino34_sst with monthly leads
    • Observations_Germany and ECMWF_S2S_Germany with daily leads
  • Metadata from CF convenctions are automatically attached by cf_xarray. (639, 656) Aaron Spring.
  • Raise warning when dimensions time, init or member are chunked to show user how to circumvent xskillscore chunking ValueError when passing these dimensions as dim in :py.HindcastEnsemble.verify or :py.HindcastEnsemble.bootstrap. (509, 658) Aaron Spring.
  • Implement PredictionEnsemble.chunks. (658) Aaron Spring.

Documentation

climpred v2.1.4 (2021-06-28)

New Features

  • Allow hours, minutes and seconds as lead.attrs['units']. (404, 603) Aaron Spring.
  • Allow to set seasonality via :py~climpred.options.set_options to specify how to group in verify(reference='climatology' or in :py.HindcastEnsemble.remove_bias. (529, 593, 603) Aaron Spring.
  • Allow weekofyear via datetime in :py.HindcastEnsemble.remove_bias, but not yet implemented in verify(reference='climatology'). (529, 603) Aaron Spring.
  • Allow more dimensions in initialized than in observations. This is particular useful if you have forecasts from multiple models (in a model dimension) and want to verify against the same observations. (129, 528, 619) Aaron Spring.

  • Automatically rename dimensions to CLIMPRED_ENSEMBLE_DIMS ["init", "member", "lead"] if CF standard_names in coordinate attributes match: (613, 622) Aaron Spring.

    • "init": "forecast_reference_time"
    • "member": "realization"
    • "lead": "forecast_period"
  • If lead coordinate is pd.Timedelta, :py.PredictionEnsemble converts lead coordinate upon instantiation to integer lead and corresponding lead.attrs["units"]. (606, 627) Aaron Spring.
  • Require xskillscore >= 0.0.20. :py~climpred.metrics._rps now works with different category_edges for observations and forecasts, see daily ECMWF example. (629, 630) Aaron Spring.
  • Set options warn_for_failed_PredictionEnsemble_xr_call, warn_for_rename_to_climpred_dims, warn_for_init_coords_int_to_annual, climpred_warnings via :py~climpred.options.set_options. (628, 631) Aaron Spring.
  • :py.PredictionEnsemble acts like :pyxarray.Dataset and understands data_vars, dims, sizes, coords, nbytes, equals, identical, __iter__, __len__, __contains__, __delitem__. (568, 632) Aaron Spring.

Documentation

Internals/Minor Fixes

  • Add weekly upstream CI, which raises issues for failures. Adapted from xarray. Manually trigger by git commit -m '[test-upstream]'. Skip climpred_testing CI by git commit -m '[skip-ci]' (518, 596) Aaron Spring.

climpred v2.1.3 (2021-03-23)

New Features

  • :py.HindcastEnsemble.verify, :py.PerfectModelEnsemble.verify, :py.HindcastEnsemble.bootstrap and :py.PerfectModelEnsemble.bootstrap accept reference climatology. Furthermore, reference persistence also allows probabilistic metrics (202, 565, 566) Aaron Spring.
  • Added new metric :py~climpred.metrics._roc Receiver Operating Characteristic as metric='roc'. (566) Aaron Spring.

Bug fixes

  • :py.HindcastEnsemble.verify and :py.HindcastEnsemble.bootstrap accept dim as list, set, tuple or str (519, 558) Aaron Spring.
  • :py.PredictionEnsemble.map now does not fail silently when applying a function to all xr.Datasets of :py.PredictionEnsemble. Instead, UserWarnings are raised. Furthermore, PredictionEnsemble.map(func, *args, **kwargs) applies only function to Datasets with matching dims if dim="dim0_or_dim1" is passed as **kwargs. (417, 437, 552) Aaron Spring.
  • :py~climpred.metrics._rpc was fixed in xskillscore>=0.0.19 and hence is not falsely limited to 1 anymore (562, 566) Aaron Spring.

Internals/Minor Fixes

  • Docstrings are now tested in GitHub actions continuous integration. (545, 560) Aaron Spring.
  • Github actions now cancels previous commits, instead of running the full testing suite on every single commit. (560) Aaron Spring.
  • :py.PerfectModelEnsemble.verify does not add climpred attributes to skill by default anymore. (560) Aaron Spring.
  • Drop python==3.6 support. (573) Aaron Spring.
  • Notebooks are now linted with nb_black using %load_ext nb_black or %load_ext lab_black for Jupyter notebooks and Jupyter lab. (526, 572) Aaron Spring.
  • Reduce dependencies to install climpred. (454, 572) Aaron Spring.
  • Examples from documentation available via Binder. Find further examples in the examples folder. (549, 578) Aaron Spring.
  • Rename branch master to main. (579) Aaron Spring.

climpred v2.1.2 (2021-01-22)

This release is the fixed version for our Journal of Open Source Software (JOSS) article about climpred, see review.

New Features

  • Function to calculate predictability horizon :py~climpred.predictability_horizon.predictability_horizon based on condition. (46, 521) Aaron Spring.

Bug fixes

  • :py.PredictionEnsemble.smooth now carries lead.attrs (527, pr:521) Aaron Spring.
  • :py.PerfectModelEnsemble.verify now works with references also for geospatial inputs, which returned NaN before. (522, pr:521) Aaron Spring.
  • :py.PredictionEnsemble.plot now shifts composite lead frequencies like days, pentads, seasons correctly. (532, 533) Aaron Spring.
  • Adapt to xesmf>=0.5.2 for spatial xesmf smoothing. (543, 548) Aaron Spring.
  • :py.HindcastEnsemble.remove_bias now carries attributes. (531, 551) Aaron Spring.

climpred v2.1.1 (2020-10-13)

Breaking changes

This version introduces a lot of breaking changes. We are trying to overhaul climpred to have an intuitive API that also forces users to think about methodology choices when running functions. The main breaking changes we introduced are for :py.HindcastEnsemble.verify and :py.PerfectModelEnsemble.verify. Now, instead of assuming defaults for most keywords, we require the user to define metric, comparison, dim, and alignment (for hindcast systems). We also require users to designate the number of iterations for bootstrapping.

  • User now has to designate number of iterations with iterations=... in :py.HindcastEnsemble.bootstrap (384, 436) Aaron Spring and Riley X. Brady.
  • Make metric, comparison, dim, and alignment required (previous default None) arguments for :py.HindcastEnsemble.verify (384, 436) Aaron Spring and Riley X. Brady.
  • Metric :py~climpred.metrics._brier_score and :py~climpred.metrics._threshold_brier_score now requires callable keyword argument logical instead of func (388) Aaron Spring.
  • :py.HindcastEnsemble.verify does not correct dim automatically to member for probabilistic metrics. (282, 407) Aaron Spring.
  • Users can no longer add multiple observations to :py.HindcastEnsemble. This will make current and future development much easier on maintainers (429, 453) Riley X. Brady.
  • Standardize the names of the output coordinates for :py.PredictionEnsemble.verify and :py.PredictionEnsemble.bootstrap to initialized, uninitialized, and persistence. initialized showcases the metric result after comparing the initialized ensemble to the verification data; uninitialized when comparing the uninitialized (historical) ensemble to the verification data; persistence is the evaluation of the persistence forecast (460, 478, 476, 480) Aaron Spring.
  • reference keyword in :py.HindcastEnsemble.verify should be choosen from [uninitialized, persistence]. historical no longer works. (460, 478, 476, 480) Aaron Spring.
  • :py.HindcastEnsemble.verify returns no skill dimension if reference=None (480) Aaron Spring.
  • comparison is not applied to uninitialized skill in :py.HindcastEnsemble.bootstrap. (352, 418) Aaron Spring.

New Features

This release is accompanied by a bunch of new features. Math operations can now be used with our :py.PredictionEnsemble objects and their variables can be sub-selected. Users can now quick plot time series forecasts with these objects. Bootstrapping is available for :py.HindcastEnsemble. Spatial dimensions can be passed to metrics to do things like pattern correlation. New metrics have been implemented based on Contingency tables. We now include an early version of bias removal for :py.HindcastEnsemble.

  • Use math operations like +-*/ with :py.HindcastEnsemble and :py.PerfectModelEnsemble. See demo Arithmetic-Operations-with-PredictionEnsemble-Objects. (377) Aaron Spring.
  • Subselect data variables from :py.PerfectModelEnsemble as from :pyxarray.Dataset: PredictionEnsemble[["var1", "var3"]] (409) Aaron Spring.
  • Plot all datasets in :py.HindcastEnsemble or :py.PerfectModelEnsemble by :py.PredictionEnsemble.plot if no other spatial dimensions are present. (383) Aaron Spring.
  • Bootstrapping now available for :py.HindcastEnsemble as :py.HindcastEnsemble.bootstrap, which is analogous to the :py.PerfectModelEnsemble method. (257, 418) Aaron Spring.
  • :py.HindcastEnsemble.verify allows all dimensions from initialized ensemble as dim. This allows e.g. spatial dimensions to be used for pattern correlation. Make sure to use skipna=True when using spatial dimensions and output has NaNs (in the case of land, for instance). (282, 407) Aaron Spring.
  • Allow binary forecasts at when calling :py.HindcastEnsemble.verify, rather than needing to supply binary results beforehand. In other words, hindcast.verify(metric='bs', comparison='m2o', dim='member', logical=logical) is now the same as hindcast.map(logical).verify(metric='brier_score', comparison='m2o', dim='member'. (431) Aaron Spring.
  • Check calendar types when using :py.HindcastEnsemble.add_observations, :py.HindcastEnsemble.add_uninitialized, :py.PerfectModelEnsemble.add_control to ensure that the verification data calendars match that of the initialized ensemble. (300, 452, 422, 462) Riley X. Brady and Aaron Spring.

  • Implement new metrics which have been ported over from https://github.com/csiro-dcfp/doppyo/ to xskillscore by Dougie Squire. (439, 456) Aaron Spring

    • rank histogram :py~climpred.metrics._rank_histogram
    • discrimination :py~climpred.metrics._discrimination
    • reliability :py~climpred.metrics._reliability
    • ranked probability score :py~climpred.metrics._rps
    • contingency table and related scores :py~climpred.metrics._contingency
  • Perfect Model :py.PerfectModelEnsemble.verify no longer requires control in :py.PerfectModelEnsemble. It is only required when reference=['persistence']. (461) Aaron Spring.
  • Implemented bias removal :py~climpred.classes.HindcastEnsemble.remove_bias. remove_bias(how='mean') removes the mean bias of initialized hindcasts with respect to observations. See example. (389, 443, 459) Aaron Spring and Riley X. Brady.

Depreciated

  • spatial_smoothing_xrcoarsen no longer used for spatial smoothing. (391) Aaron Spring.
  • compute_metric, compute_uninitialized and compute_persistence no longer in use for :py.PerfectModelEnsemble in favor of :py.PerfectModelEnsemble.verify with the reference keyword instead. (436, 468, 472) Aaron Spring and Riley X. Brady.
  • 'historical' no longer a valid choice for reference. Use 'uninitialized' instead. (478) Aaron Spring.

Bug Fixes

  • :py.PredictionEnsemble.verify and :py.PredictionEnsemble.bootstrap now accept metric_kwargs. (387) Aaron Spring.
  • :py.PerfectModelEnsemble.verify now accepts 'uninitialized' as a reference. (395) Riley X. Brady.
  • Spatial and temporal smoothing :py.PredictionEnsemble.smooth now work as expected and rename time dimensions after :py~climpred.classes.PredictionEnsembleEnsemble.verify. (391) Aaron Spring.
  • PredictionEnsemble.verify(comparison='m2o', references=['uninitialized', 'persistence'] does not fail anymore. (385, 400) Aaron Spring.
  • Remove bias using dayofyear in :py.HindcastEnsemble.reduce_bias. (443) Aaron Spring.
  • climpred works with dask=>2.28. (479, 482) Aaron Spring.

Documentation

  • Updates climpred tagline to "Verification of weather and climate forecasts." (420) Riley X. Brady.
  • Adds section on how to use arithmetic with :py.HindcastEnsemble. (378) Riley X. Brady.
  • Add docs section for similar open-source forecasting packages. (432) Riley X. Brady.
  • Add all metrics to main API in addition to metrics page. (438) Riley X. Brady.
  • Add page on bias removal Aaron Spring.

Internals/Minor Fixes

  • :py.PredictionEnsemble.verify replaces deprecated PerfectModelEnsemble.compute_metric() and accepts reference as keyword. (387) Aaron Spring.
  • Cleared out unnecessary statistics functions from climpred and migrated them to esmtools. Add esmtools as a required package. (395) Riley X. Brady.
  • Remove fixed pandas dependency from pandas=0.25 to stable pandas. (402, 403) Aaron Spring.
  • dim is expected to be a list of strings in :py~climpred.prediction.compute_perfect_model and ~climpred.prediction.compute_hindcast. (282, 407) Aaron Spring.
  • Update cartopy requirement to 0.0.18 or greater to release lock on matplotlib version. Update xskillscore requirement to 0.0.18 to cooperate with new xarray version. (451, 449) Riley X. Brady
  • Switch from Travis CI and Coveralls to Github Actions and CodeCov. (471) Riley X. Brady
  • Assertion functions added for :py.PerfectModelEnsemble: :py~climpred.testing.assert_PredictionEnsemble. (391) Aaron Spring.
  • Test all metrics against synthetic data. (388) Aaron Spring.

climpred v2.1.0 (2020-06-08)

Breaking Changes

  • Keyword bootstrap has been replaced with iterations. We feel that this more accurately describes the argument, since "bootstrap" is really the process as a whole. (354) Aaron Spring.

New Features

  • :py.HindcastEnsemble and :py.PerfectModelEnsemble now use an HTML representation, following the more recent versions of xarray. (371) Aaron Spring.
  • HindcastEnsemble.verify() now takes reference=... keyword. Current options are 'persistence' for a persistence forecast of the observations and 'uninitialized' for an uninitialized/historical reference, such as an uninitialized/forced run. (341) Riley X. Brady.
  • We now only enforce a union of the initialization dates with observations if reference='persistence' for :py.HindcastEnsemble. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (341) Riley X. Brady.
  • :py~climpred.prediction.compute_perfect_model now accepts initialization (init) as cftime and int. cftime is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (332) Aaron Spring.
  • New explicit keywords in bootstrap functions for resampling_dim and reference_compute (320) Aaron Spring.
  • Logging now included for compute_hindcast which displays the inits and verification dates used at each lead (324) Aaron Spring, (338) Riley X. Brady. See (logging).

  • New explicit keywords added for alignment of verification dates and initializations. (324) Aaron Spring. See (alignment)

    • 'maximize': Maximize the degrees of freedom by slicing hind and verif to a common time frame at each lead. (338) Riley X. Brady.
    • 'same_inits': slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (328) Riley X. Brady.
    • 'same_verifs': slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (331) Riley X. Brady.

Performance

The major change for this release is a dramatic speedup in bootstrapping functions, led by Aaron Spring. We focused on scalability with dask and found many places we could compute skill simultaneously over all bootstrapped ensemble members rather than at each iteration.

  • Bootstrapping uninitialized skill in the perfect model framework is now sped up significantly for annual lead resolution. (332) Aaron Spring.

  • General speedup in ~climpred.bootstrap.bootstrap_hindcast and ~climpred.bootstrap.bootstrap_perfect_model: (285) Aaron Spring.

    • Properly implemented handling for lazy results when inputs are chunked.
    • User gets warned when chunking potentially unnecessarily and/or inefficiently.

Bug Fixes

  • Alignment options now account for differences in the historical time series if reference='historical'. (341) Riley X. Brady.

Internals/Minor Fixes

  • Added a Code of Conduct (285) Aaron Spring.
  • Gather pytest.fixture inconftest.py`. (:pr:`313) Aaron Spring.
  • Move x_METRICS and COMPARISONS to metrics.py and comparisons.py in order to avoid circular import dependencies. (315) Aaron Spring.
  • asv benchmarks added for :py.HindcastEnsemble (285) Aaron Spring.
  • Ignore irrelevant warnings in pytest and mark slow tests (333) Aaron Spring.
  • Default CONCAT_KWARGS now in all xr.concat to speed up bootstrapping. (330) Aaron Spring.
  • Remove member coords for m2c comparison for probabilistic metrics. (330) Aaron Spring.
  • Refactored ~climpred.prediction.compute_hindcast and :py~climpred.prediction.compute_perfect_model. (330) Aaron Spring.
  • Changed lead0 coordinate modifications to be compliant with xarray=0.15.1 in :py~climpred.reference.compute_persistence. (348) Aaron Spring.
  • Exchanged my_quantile with xr.quantile(skipna=False). (348) Aaron Spring.
  • Remove sig from :py~climpred.graphics.plot_bootstrapped_skill_over_leadyear. (351) Aaron Spring.
  • Require xskillscore v0.0.15 and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady.
  • Faster bootstrapping without replacement used in threshold functions of climpred.stats (354) Aaron Spring.
  • Require cftime v1.1.2, which modifies their object handling to create 200-400x speedups in some basic operations. (356) Riley X. Brady.
  • Resample first and then calculate skill in ~climpred.bootstrap.bootstrap_perfect_model and ~climpred.bootstrap.bootstrap_hindcast (355) Aaron Spring.

Documentation

climpred v2.0.0 (2020-01-22)

New Features

  • Add support for days, pentads, weeks, months, seasons for lead time resolution. climpred now requires a lead attribute "units" to decipher what resolution the predictions are at. (294) Kathy Pegion and Riley X. Brady.
  • :py.HindcastEnsemble now has :py.HindcastEnsemble.add_observations and :py.HindcastEnsemble.get_observations methods. These are the same as .add_reference() and .get_reference(), which will be deprecated eventually. The name change clears up confusion, since "reference" is the appropriate name for a reference forecast, e.g. "persistence". (310) Riley X. Brady.
  • :py.HindcastEnsemble now has .verify() function, which duplicates the .compute_metric() function. We feel that .verify() is more clear and easy to write, and follows the terminology of the field. (310) Riley X. Brady.
  • e2o and m2o are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (310) Riley X. Brady.

Documentation

  • New example pages for subseasonal-to-seasonal prediction using climpred. (294) Kathy Pegion

    • Calculate the skill of the MJO index as a function of lead time (link).
    • Calculate the skill of the MJO index as a function of lead time for weekly data (link).
    • Calculate ENSO skill as a function of initial month vs. lead time (link).
    • Calculate Seasonal ENSO skill (link).
  • Comparisons page rewritten for more clarity. (310) Riley X. Brady.

Bug Fixes

  • Fixed m2m broken comparison issue and removed correction. (290) Aaron Spring.

Internals/Minor Fixes

  • Updates to xskillscore v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (309) Riley X. Brady.
  • Stacking dims is handled by comparisons, no need for internal keyword stack_dims. Therefore comparison now takes metric as argument instead. (290) Aaron Spring.
  • assign_attrs now carries dim (290) Aaron Spring.
  • reference changed to verif throughout hindcast compute functions. This is more clear, since reference usually refers to a type of forecast, such as persistence. (310) Riley X. Brady.
  • Comparison objects can now have aliases. (310) Riley X. Brady.

climpred v1.2.1 (2020-01-07)

Depreciated

  • mad no longer a keyword for the median absolute error metric. Users should now use median_absolute_error, which is identical to changes in xskillscore version 0.0.10. (283) Riley X. Brady
  • pacc no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (283) Riley X. Brady
  • msss no longer a keyword for the Murphy's MSSS, since it is reserved for the standard MSSS. (283) Riley X. Brady

New Features

  • Metrics pearson_r_eff_p_value and spearman_r_eff_p_value account for autocorrelation in computing p values. (283) Riley X. Brady
  • Metric effective_sample_size computes number of independent samples between two time series being correlated. (283) Riley X. Brady

  • Added keywords for metrics: (283) Riley X. Brady

    • 'pval' for pearson_r_p_value
    • ['n_eff', 'eff_n'] for effective_sample_size
    • ['p_pval_eff', 'pvalue_eff', 'pval_eff'] for pearson_r_eff_p_value
    • ['spvalue', 'spval'] for spearman_r_p_value
    • ['s_pval_eff', 'spvalue_eff', 'spval_eff'] for spearman_r_eff_p_value
    • 'nev' for nmse

Internals/Minor Fixes

  • climpred now requires xarray version 0.14.1 so that the drop_vars() keyword used in our package does not throw an error. (276) Riley X. Brady
  • Update to xskillscore version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (283) Riley X. Brady
  • doc8 added to pre-commit to have consistent formatting on .rst files. (283) Riley X. Brady
  • Remove proper attribute on Metric class since it isn't used anywhere. (283) Riley X. Brady
  • Add testing for effective p values. (283) Riley X. Brady
  • Add testing for whether metric aliases are repeated/overwrite each other. (283) Riley X. Brady
  • ppp changed to msess, but keywords allow for ppp and msss still. (283) Riley X. Brady

Documentation

climpred v1.2.0 (2019-12-17)

Depreciated

  • Abbreviation pval depreciated. Use p_pval for pearson_r_p_value instead. (264) Aaron Spring.

New Features

>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST") >>> ref = climpred.tutorial.load_dataset("ERSST") >>> hindcast = climpred.HindcastEnsemble(hind) >>> hindcast = hindcast.add_reference(ref, "ERSST") >>> print(hindcast) <climpred.HindcastEnsemble> Initialized Ensemble: SST (init, lead, member) float64 ... ERSST: SST (time) float32 ... Uninitialized: None >>> print(hindcast.get_initialized()) <xarray.Dataset> Dimensions: (init: 64, lead: 10, member: 10) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * member (member) int32 1 2 3 4 5 6 7 8 9 10 * init (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0 Data variables: SST (init, lead, member) float64 ... >>> print(hindcast.get_reference("ERSST")) <xarray.Dataset> Dimensions: (time: 61) Coordinates: * time (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015 Data variables: SST (time) float32 ...

  • metric_kwargs can be passed to :py~climpred.metrics.Metric. (264) Aaron Spring.

Bug Fixes

  • :py.HindcastEnsemble.compute_metric doesn't drop coordinates from the initialized hindcast ensemble anymore. (258) Aaron Spring.
  • Metric uacc does not crash when ppp negative anymore. (264) Aaron Spring.
  • Update xskillscore to version 0.0.9 to fix all-NaN issue with pearson_r and pearson_r_p_value when there's missing data. (269) Riley X. Brady.

Internals/Minor Fixes

Documentation

climpred v1.1.0 (2019-09-23)

Features

  • Write information about skill computation to netcdf attributes(213) Aaron Spring
  • Temporal and spatial smoothing module (224) Aaron Spring
  • Add metrics brier_score, threshold_brier_score and crpss_es (232) Aaron Spring
  • Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (232) Aaron Spring

Bug Fixes

  • Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (232) Aaron Spring

Internals/Minor Fixes

  • Rename .stats.DPP to dpp (232) Aaron Spring
  • Add matplotlib as a main dependency so that a direct pip installation works (211) Riley X. Brady.
  • climpred is now installable from conda-forge (212) Riley X. Brady.
  • Fix erroneous descriptions of sample datasets (226) Riley X. Brady.
  • Benchmarking time and peak memory of compute functions with asv (231) Aaron Spring

Documentation

  • Add scope of package to docs for clarity for users and developers. (235) Riley X. Brady.

climpred v1.0.1 (2019-07-04)

Bug Fixes

  • Accomodate for lead-zero within the lead dimension (196) Riley X. Brady.
  • Fix issue with adding uninitialized ensemble to :py.HindcastEnsemble object (199) Riley X. Brady.
  • Allow max_dof keyword to be passed to compute_metric and compute_persistence for :py.HindcastEnsemble. (199) Riley X. Brady.

Internals/Minor Fixes

  • Force xskillscore version 0.0.4 or higher to avoid ImportError (204) Riley X. Brady.
  • Change max_dfs keyword to max_dof (199) Riley X. Brady.
  • Add tests for :py.HindcastEnsemble and PerfectModelEnsemble. (199) Riley X. Brady

climpred v1.0.0 (2019-07-03)

climpred v1.0.0 represents the first stable release of the package. It includes :py.HindcastEnsemble and PerfectModelEnsemble objects to perform analysis with. It offers a suite of deterministic and probabilistic metrics that are optimized to be run on single time series or grids of data (e.g., lat, lon, and depth). Currently, climpred only supports annual forecasts.

Features

  • Bootstrap prediction skill based on resampling with replacement consistently in ReferenceEnsemble and PerfectModelEnsemble. (128) Aaron Spring
  • Consistent bootstrap function for climpred.stats functions via bootstrap_func wrapper. (167) Aaron Spring
  • many more metrics: _msss_murphy, _less and probabilistic _crps, _crpss (128) Aaron Spring

Bug Fixes

  • compute_uninitialized now trims input data to the same time window. (193) Riley X. Brady
  • rm_poly now properly interpolates/fills NaNs. (192) Riley X. Brady

Internals/Minor Fixes

  • The climpred version can be printed. (195) Riley X. Brady
  • Constants are made elegant and pushed to a separate module. (184) Andrew Huang
  • Checks are consolidated to their own module. (173) Andrew Huang

Documentation

  • Documentation built extensively in multiple PRs.

climpred v0.3 (2019-04-27)

climpred v0.3 really represents the entire development phase leading up to the version 1 release. This was done in collaboration between Riley X. Brady, Aaron Spring, and Andrew Huang. Future releases will have less additions.

Features

  • Introduces object-oriented system to climpred, with classes ReferenceEnsemble and PerfectModelEnsemble. (86) Riley X. Brady
  • Expands bootstrapping module for perfect-module configurations. (78, 87) Aaron Spring
  • Adds functions for computing Relative Entropy (73) Aaron Spring

  • Sets more intelligible dimension expectations for climpred (98, 105) Riley X. Brady and Aaron Spring:

    • init: initialization dates for the prediction ensemble
    • lead: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations
    • time: time dimension for control runs, references, etc.
    • member: ensemble member dimension.
  • Updates open_dataset to display available dataset names when no argument is passed. (123) Riley X. Brady
  • Change ReferenceEnsemble to :py.HindcastEnsemble. (124) Riley X. Brady
  • Add probabilistic metrics to climpred. (128) Aaron Spring
  • Consolidate separate perfect-model and hindcast functions into singular functions (128) Aaron Spring
  • Add option to pass proxy through to open_dataset for firewalled networks. (138) Riley X. Brady

Bug Fixes

  • xr_rm_poly can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (94) Andrew Huang
  • Travis CI, treon, and pytest all run for automated testing of new features. (98, 105, 106) Riley X. Brady and Aaron Spring
  • Clean up check_xarray decorators and make sure that they work. (142) Andrew Huang
  • Ensures that help() returns proper docstring even with decorators. (149) Andrew Huang
  • Fixes bootstrap so p values are correct. (170) Aaron Spring

Internals/Minor Fixes

  • Adds unit testing for all perfect-model comparisons. (107) Aaron Spring
  • Updates CESM-LE uninitialized ensemble sample data to have 34 members. (113) Riley X. Brady
  • Adds MPI-ESM hindcast, historical, and assimilation sample data. (119) Aaron Spring
  • Replaces check_xarray with a decorator for checking that input arguments are xarray objects. (120) Andrew Huang
  • Add custom exceptions for clearer error reporting. (139) Riley X. Brady
  • Remove "xr" prefix from stats module. (144) Riley X. Brady
  • Add codecoverage for testing. (152) Riley X. Brady
  • Update exception messages for more pretty error reporting. (156) Andrew Huang
  • Add pre-commit and flake8/black check in CI. (163) Riley X. Brady
  • Change loadutils module to tutorial and open_dataset to load_dataset. (164) Riley X. Brady
  • Remove predictability horizon function to revisit for v2. (165) Riley X. Brady
  • Increase code coverage through more testing. (167) Aaron Spring
  • Consolidates checks and constants into modules. (173) Andrew Huang

climpred v0.2 (2019-01-11)

Name changed to climpred, developed enough for basic decadal prediction tasks on a perfect-model ensemble and reference-based ensemble.

climpred v0.1 (2018-12-20)

Collaboration between Riley Brady and Aaron Spring begins.