Enhance numerical stability and scoring accuracy for datasets with identical features
- Adjust minimum median values to 100x machine epsilon to prevent unstable operations.
- Implement specific scoring logic for outlier and near-duplicate issues in datasets with identical features, ensuring correct scores are assigned in such edge cases.
- Introduce unit tests to validate the expected behavior for datasets comprising entirely identical examples. Check an additional case with an extra unique example.
Related to issue #1055.
* Added base structure outline for get_ood_scores()
Added base test for get_ood_scores()
* Added warning for illogical param combo
* Addressed PR comments
* Added better unit tests
* TODO: test for correctly identifying OOD example
* Moved logic from get_ood_scores to _subtract_confident_thresholds
* TODO: Is this a cleaner way of doing this? Minimal code repeat but strange change to _subtract function
* Addressed type issue
* Switched logic for getting confident thresholds
* Fixing typecheck issues wiht labels parameter being None
* Simplified helper function. Testing type
* Fixed mypy static typing issue
* Mypy typecheck logic test
* removed uncessesary imports in util file
* typechecker debugging (add assert)
* Fixed type logic and removed confident_thresholds=None return
* Added extra arg in helper func to end of func
* Added zero-index checking for label param
* Added ood examples to outlier score notebook
* Added skeleton file structure for implementing outliers
* Make adjust_pred_probs=True by default not false
* Added base Outlier class functionality
TODO:
* test_outlier.py
* Added logic tests for function
* Updated get_ood_scores to always return confident_thresholds
* Even if none were calculated (then return is None)
* Added warning for fit that doesn't calculate confident_thresholds
* Moved get_outlier_scores and get_ood_scores to outlier.py
* Changed param dicts to dicts
* Added docstring to outlier.py
* Added proper return types
* Fixed mypy typing issues
* Switched outliers -> features; ood -> predictions naming conv
* Switched docstring to stem from fit and score functions
* Changed return of helper functions
* Fixed tutorials notebook to use OutOfDistribution class
* Moved imports to top of file
* Added option for different knn objects, addressed pr comments
* Switched params arg to init only
* Addressed PR comment for notebook, cleared notebook
* Fixed PR comments, wording.
* Testing relative links on build
* Changed ood_pred_probs scores to reflect 0 = most ood 1 = least
* Added MLP for detection outliers with pred_probs
* Fixed typos
* Added MLP Import
* Improved warning when fit call unnecessary
* Changed referenced to params dict in warnings/errors
* Added bagging+MLP classifier into notebook
* Reverted tutorial wording
* OOD tutorial improvements
* cleanup OOD documentation
* adopting->using
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Jonas Mueller