* Added base structure outline for get_ood_scores()
Added base test for get_ood_scores()
* Added warning for illogical param combo
* Addressed PR comments
* Added better unit tests
* TODO: test for correctly identifying OOD example
* Moved logic from get_ood_scores to _subtract_confident_thresholds
* TODO: Is this a cleaner way of doing this? Minimal code repeat but strange change to _subtract function
* Addressed type issue
* Switched logic for getting confident thresholds
* Fixing typecheck issues wiht labels parameter being None
* Simplified helper function. Testing type
* Fixed mypy static typing issue
* Mypy typecheck logic test
* removed uncessesary imports in util file
* typechecker debugging (add assert)
* Fixed type logic and removed confident_thresholds=None return
* Added extra arg in helper func to end of func
* Added zero-index checking for label param
* Added ood examples to outlier score notebook
* Added skeleton file structure for implementing outliers
* Make adjust_pred_probs=True by default not false
* Added base Outlier class functionality
TODO:
* test_outlier.py
* Added logic tests for function
* Updated get_ood_scores to always return confident_thresholds
* Even if none were calculated (then return is None)
* Added warning for fit that doesn't calculate confident_thresholds
* Moved get_outlier_scores and get_ood_scores to outlier.py
* Changed param dicts to dicts
* Added docstring to outlier.py
* Added proper return types
* Fixed mypy typing issues
* Switched outliers -> features; ood -> predictions naming conv
* Switched docstring to stem from fit and score functions
* Changed return of helper functions
* Fixed tutorials notebook to use OutOfDistribution class
* Moved imports to top of file
* Added option for different knn objects, addressed pr comments
* Switched params arg to init only
* Addressed PR comment for notebook, cleared notebook
* Fixed PR comments, wording.
* Testing relative links on build
* Changed ood_pred_probs scores to reflect 0 = most ood 1 = least
* Added MLP for detection outliers with pred_probs
* Fixed typos
* Added MLP Import
* Improved warning when fit call unnecessary
* Changed referenced to params dict in warnings/errors
* Added bagging+MLP classifier into notebook
* Reverted tutorial wording
* OOD tutorial improvements
* cleanup OOD documentation
* adopting->using
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* Added runnable package versioning for tutorials
Added quickstart at top of tutorials
* Addressed PR comments
* Fixed quickstart to include cleanlearning
* Set correct requirements.txt
* Updated y to labels
* Fixed y-> labels labeling issues
* Updated quickstart message/removed extra dependencies
* remove keras from package-versions
* more concise
* remove pathlib as explicit requirement
* remove pathlib version
* rewording
* true label -> given label
* remove venv from kernelspec
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* cleanlearning tips
* addessed comments
* add a few general tips at the end
* minor edits
* format pred_probs as variable type
* bug fix to pass lint (missing comma)
* fix format for linter
Co-authored-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
* Added outlier detection tutorial into docs
* Switched to pytorch model/dataset implementation
* Fixed image normalization
* Fixed outlier detection for test set
* Added outlier thresholding into tutorial.
* Cleaned output
* Removed unused imports and renamed notebook
* Added outliers notebook to PR
* Added quickstart into tutorial
* Best quickstart header
* Cleaned cell output
* Changed to use subset of original data for speed
* Fixed randomness
* Cleared outputs
* fixed metadata tags
* Fixed metadata
* Cleaned kernel and verified output
* Improved unit test
Changed labels references to classes where apropriate
* Create FAQ page in cleanlab docs
* Add FAQ notebook to store answers to frequently asked questions
* Improved formatting issues for faq page
- added faq to sidebar
* Improved formatting issues for faq page
- added faq to sidebar
- changed email to comply with CLA
* Removed faq numbering
- Removed notebook output metadata
- Added notebook to index for left table link
* Text and example rewritten for clarity
- How do I format labels for cleanlab-- example improved
- How do I format labels for cleanlab-- text amended
- Can't find an answer to your question-- added links
* add links to issues/slack
Co-authored-by: Ulyana Tkachenko <uly@ulyana.lan>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* example label formatting
* example label format in text tutorial
* example label format for tabular tutorial
* y -> full_labels
* example label format for audio tutorial
* clarify input shapes in image tutorial
* clarify input shapes in text tutorial
* clarify input shapes in tabular tutorial
* clarify input shapes for audio tutorial
* generalize text tutorial to multiclass datasets
* fixed typo
* fixed misspelling
* separate cell for num_classes
less likely to be missed if users skip the previous cell to load their own datasets
* removing pycharm metadata
* print classes
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
The anchor already says "Run in Google Colab", so if the logo fails to
load, it will say "Google Colab Logo Run in Google Colab"; it's better
to have the alt text just be empty.
* dataset health tutorial pulldown format
* tutorial ordering in sidebar
* pulldown format, bullet format
* reformat pulldown indepth tutorial
* shorten data fetching header
* adjust spacing
Jonas Mueller