np.polyfit() error in EarthRef JupyterHub environment

[Swanson-Hysell]

The MPMS notebooks that use np.polyfit() are running fine locally for me, but are causing this issue on the EarthRef JupyterHub (perhaps needs an updated environment?).

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/opt/tljh/user/lib/python3.6/site-packages/ipywidgets/widgets/interaction.py in observer(change)
     77         with out:
     78             clear_output(wait=True)
---> 79             f(**kwargs)
     80             show_inline_matplotlib_plots()
     81     for k,w in controls.items():

~/rockmag/PmagPy-RockmagPy-notebooks-2144031/rockmag.py in <lambda>(background_temp_range, excluded_temp_range, poly_deg)
    543             background_temp_range[0], background_temp_range[1],
    544             excluded_temp_range[0], excluded_temp_range[1],
--> 545             poly_deg
    546         ), {
    547             'background_temp_range': background_temp_range_slider,

~/rockmag/PmagPy-RockmagPy-notebooks-2144031/rockmag.py in verwey_estimate(temps, mags, t_range_background_min, t_range_background_max, excluded_t_min, excluded_t_max, poly_deg, plot_zero_crossing, plot_title)
    396     dM_dT_filtered = dM_dT_df['dM_dT'][temps_dM_dT_filtered_indices]
    397 
--> 398     poly_background_fit = np.polyfit(temps_dM_dT_filtered, dM_dT_filtered, poly_deg)
    399     dM_dT_filtered_polyfit = np.poly1d(poly_background_fit)(temps_dM_dT_filtered)
    400 

/opt/tljh/user/lib/python3.6/site-packages/numpy/lib/polynomial.py in polyfit(x, y, deg, rcond, full, w, cov)

[rminnett]

Ah, ok, I'll take a look - the JupyterHub packages are definitely getting out of date!

[rminnett]

@Swanson-Hysell, I tried the notebooks on JupyterHub and couldn't reproduce the error above (I got a few other errors, but not that TypeError). Does running this as part of the rmag setup change the results in the MPMS notebooks?

pip install --user --upgrade numpy

[Swanson-Hysell]

Even with numpy upgraded:

I am still getting this issue associated with np.polyfit():

[rminnett]

@Swanson-Hysell, I added a Python 3.10 option as a kernel on JupyterHub and that seems to fix the numpy issue using the latest version of MPMS_goethite_fit.ipynb:

[Swanson-Hysell]

@rminnett I am giving this a try. The first issue I bump into when using the Python 3.10 kernel is that pandas isn't available as a package. Now I can pip install it, but that seems non-ideal as it makes sense to have the standard scientific python packages available.

[rminnett]

It seems to work at the moment:

Do you still get the error? Are there any other packages missing?

[Swanson-Hysell]

I am still getting errors in this 3.10 environment. When I import pandas it gives this error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import rockmag as rmag
      2 import pmagpy.ipmag as ipmag
      3 import pmagpy.contribution_builder as cb

File ~/rockmag/PmagPy-RockmagPy-notebooks-acc3fc0/rockmag.py:1
----> 1 import pandas as pd
      2 import numpy as np
      3 import matplotlib.pyplot as plt

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/pandas/__init__.py:19
     16         _missing_dependencies.append(f"{_dependency}: {_e}")
     18 if _missing_dependencies:  # pragma: no cover
---> 19     raise ImportError(
     20         "Unable to import required dependencies:\n" + "\n".join(_missing_dependencies)
     21     )
     22 del _hard_dependencies, _dependency, _missing_dependencies
     24 try:
     25     # numpy compat

ImportError: Unable to import required dependencies:
numpy: No module named 'numpy'
pytz: No module named 'pytz'

[rminnett]

Ok, I think it's better now after installing these packages in the 3.10 shared environment instead of just for the user:

• numpy
• pytz
• matplotlib
• pillow
• pyparsing
• cycler
• kiwisolver
• plotly
• ipywidgets
• future
• scipy

@Swanson-Hysell are you able to run the latest version of the MPMS_goethite_fit.ipynb notebook now?

[Swanson-Hysell]

We are running into a few errors associated with the 3.10 environment when trying to run the RockmagPy notebooks:

• cartopy is necessary for ipmag functions and makes sense to be an available package
• the ipywidget functionality that was working in the 3.6 environment is broken in the 3.10 environment as documented in this issue: the ipywidget sliding bars are not visible in JupyterHub #26
• in the JupyterHub 3.10 environment, we get a ModuleNotFoundError: No module named 'tenacity' associated with trying to use plotly. More details on that error can be found below.

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[8], line 1
----> 1 rmag.plot_mpms_data(fc_data, zfc_data, rtsirm_cool_data, rtsirm_warm_data, use_plotly=True)

File ~/rockmag/PmagPy-RockmagPy-notebooks-fefd070/rockmag.py:185, in plot_mpms_data(fc_data, zfc_data, rtsirm_cool_data, rtsirm_warm_data, fc_color, zfc_color, rtsirm_cool_color, rtsirm_warm_color, fc_marker, zfc_marker, rtsirm_cool_marker, rtsirm_warm_marker, symbol_size, use_plotly, plot_derivative, return_figure, drop_first, drop_last)
    181         fig.update_layout(height=450)
    183     fig.update_layout(title="MPMS Data and Derivatives" if plot_derivative else "MPMS Data")
--> 185     fig.show()
    187 else:
    188     # Matplotlib plotting
    189     if plot_derivative:

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/plotly/basedatatypes.py:3410, in BaseFigure.show(self, *args, **kwargs)
   3377 """
   3378 Show a figure using either the default renderer(s) or the renderer(s)
   3379 specified by the renderer argument
   (...)
   3406 None
   3407 """
   3408 import plotly.io as pio
-> 3410 return pio.show(self, *args, **kwargs)

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/_plotly_utils/importers.py:36, in relative_import.<locals>.__getattr__(import_name)
     34     rel_module = ".".join(rel_path_parts[:-1])
     35     class_name = import_name
---> 36     class_module = importlib.import_module(rel_module, parent_name)
     37     return getattr(class_module, class_name)
     39 raise AttributeError(
     40     "module {__name__!r} has no attribute {name!r}".format(
     41         name=import_name, __name__=parent_name
     42     )
     43 )

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    124             break
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/plotly/io/_renderers.py:9
      5 from packaging.version import Version
      7 from plotly import optional_imports
----> 9 from plotly.io._base_renderers import (
     10     MimetypeRenderer,
     11     ExternalRenderer,
     12     PlotlyRenderer,
     13     NotebookRenderer,
     14     KaggleRenderer,
     15     AzureRenderer,
     16     ColabRenderer,
     17     JsonRenderer,
     18     PngRenderer,
     19     JpegRenderer,
     20     SvgRenderer,
     21     PdfRenderer,
     22     BrowserRenderer,
     23     IFrameRenderer,
     24     SphinxGalleryHtmlRenderer,
     25     SphinxGalleryOrcaRenderer,
     26     CoCalcRenderer,
     27     DatabricksRenderer,
     28 )
     29 from plotly.io._utils import validate_coerce_fig_to_dict
     31 ipython = optional_imports.get_module("IPython")

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/plotly/io/_base_renderers.py:10
      8 from plotly import utils, optional_imports
      9 from plotly.io import to_json, to_image, write_image, write_html
---> 10 from plotly.io._orca import ensure_server
     11 from plotly.io._utils import plotly_cdn_url
     12 from plotly.offline.offline import _get_jconfig, get_plotlyjs

File /opt/tljh/user/envs/Python-3.10/lib/python3.10/site-packages/plotly/io/_orca.py:14
     11 from pathlib import Path
     12 from shutil import which
---> 14 import tenacity
     16 import plotly
     17 from plotly.files import PLOTLY_DIR, ensure_writable_plotly_dir

ModuleNotFoundError: No module named 'tenacity'

[Swanson-Hysell]

Documenting issues with current JupyterHub 3.10 kernel:

• Anisotropy_plot_AMS.ipynb : no issues
• high_T_susceptibility.ipynb : no issues
• MPMS_goethite_fit.ipynb : ipywidget sliders do not appear, these missing sliders provide important interactive functionality (they worked in the previous kernel)
• MPMS_plot_dc.ipynb : any attempt to use plotly throws the error ModuleNotFoundError: No module named 'tenacity'
• MPMS_verwey_fit.ipynb : ipywidget sliders do not appear, these missing sliders provide important interactive functionality they worked in the previous kernel)

Here is what the sliders should look like (running in local JupyterLab with ipywidgets 7.6.5):

vs what it looks like on the JupyterHub (with ipywidgets 8.1.2):

[rminnett]

The sliders and plotly appear to be working on 3.10 now:

[Swanson-Hysell]

The np.polyfit() error and the plotly import error have been resolved. The ipywidget slider issue documented in issue #26 remains.