Skip to content

Learn how to handle model drift and perform test-based model monitoring

License

Notifications You must be signed in to change notification settings

pyladiesams/model-drift-beginner-dec2022

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

An introduction to model drift

Level: Beginner

Workshop description

During this workshop you will learn how to:

  • measure your model performance
  • handle model drift
  • perform test-based model monitoring

All this with the help of evidently

Requirements

  • Python 3.6 or higher
  • VS Code

Note: the workshop has been tested with Jupyter notebooks running in the browser and VS Code. The widgets will not display correctly if opened in PyCharm.

Usage

  • Clone the repository and navigate to its root
  • Install the required libraries with pip: $ pip install -r requirements.txt
  • Run $ jupyter nbextension install --sys-prefix --symlink --overwrite --py evidently (installs the Evidently nbextension)
  • Run $ jupyter nbextension enable evidently --py --sys-prefix (activates it)

Note: if you use Jupyter Lab, the reports might not display in the notebook. However, you can still save them as HTML files.

  • Open VS Code settings, choose Workspace tab, update Jupyter: Notebook File Root with ${workspaceFolder}. If this step is skipped, workshop and solution notebooks will run from their folder instead of the root one. Additionaly you will not be able to save reports without ugly hacks.

Note for Windows users:

Unfortunately, building reports inside a Jupyter notebook is not yet possible for Windows. The reason is Windows requires administrator privileges to create symlink. In later versions Evidently will address this issue. You can still generate the HTML to view externally.

Video record

Re-watch YouTube stream here

Credits

This workshop was set up by @pyladiesams and @EzheZhezhe