Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[new feature] add (another model-free) data valuation method: complexity-gap score #972

Open
jwmueller opened this issue Jan 30, 2024 · 0 comments
Labels
enhancement New feature or request help-wanted We need your help to add this, but it may be more challenging than a "good first issue"

Comments

@jwmueller
Copy link
Member

Cleanlab's data valuation module: https://docs.cleanlab.ai/master/cleanlab/datalab/internal/issue_manager/data_valuation.html

Currently solely supports Data Shapely based on K-nearest-neighbor models.

Goal: add more efficient Data Valuation methods as options for this module.
One good one to consider (should benchmark against Cleanlab's current KNN Data Shapely) is the complexity-gap method published in this paper:

DATA VALUATION WITHOUT TRAINING OF A MODEL
https://arxiv.org/abs/2301.00930

@jwmueller jwmueller added enhancement New feature or request help-wanted We need your help to add this, but it may be more challenging than a "good first issue" labels Jan 30, 2024
@jwmueller jwmueller changed the title add (another model-free) data valuation method: complexity-gap score [new feature] add (another model-free) data valuation method: complexity-gap score Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help-wanted We need your help to add this, but it may be more challenging than a "good first issue"
Projects
None yet
Development

No branches or pull requests

1 participant