* update N in classification.py
* minor docstring changes on K-1 classes
* use K in shape
* minor grammar fixes
* docs language improvements
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* Added support to build KNN graph with only training data
- Added default classifier for get_knn_distance_ood_scores()
* Added unit test for checking default KNN model is used when nbrs=None
* Changed default num neighbors from 10 to K.
* Changed unit test to check score sums
* Fixed k=15
* Added support to build KNN graph with only training data
- Added default classifier for get_knn_distance_ood_scores()
- Added unit test for checking default KNN model is used when nbrs=None
- Changed default num neighbors from 10 to K.
- Changed unit test to check score sums
* Improved code readibility, extended unit test to check for default k
* Changed naming convention from get_knn_distance_ood_scores to get_outlier_scores
- Changed nbrs to knn
- Improved unit test readability
* Made unit test more robust to check if user-set value is passed
* changed classifier -> estimator
* fixed test warning handle
* Updated function headers to better definition
* Improved header writing added functionality to avoid training/testing with identical features
* Updated docstring, updated handling of features=None and test for it
* Added ValueError for k>len(features) and test to catch ValueError
* Updated argument types to Optional, added runtime typecheck.
* Added unit test to check TypeError
* Changed Exception type thrown when knn=None, features=None
* TypeError to ValueError
* Reversed scoring for outlier severity
* Added test to check t parameter
* Added t parameter for global rescaler
* Improved function definition concerning 't' parameter
* Fixed tests
* Removed repeated calls
* Added assertion to make sure X_ood is always smallest outlier score
* Improved logic for checking X_ood has the smallest score
Our package doesn't have type annotations everywhere, so we can't use
mypy in strict mode just yet. Still, adding type checking in CI is
valuable, so we don't have unchecked annotations in our code.
This patch includes basic fixes to make type checking pass, including
switching the incorrect `np.array` type annotation for `np.ndarray` and
adding some assertions for flow-sensitive typing.
* Create FAQ page in cleanlab docs
* Add FAQ notebook to store answers to frequently asked questions
* Improved formatting issues for faq page
- added faq to sidebar
* Improved formatting issues for faq page
- added faq to sidebar
- changed email to comply with CLA
* Removed faq numbering
- Removed notebook output metadata
- Added notebook to index for left table link
* Text and example rewritten for clarity
- How do I format labels for cleanlab-- example improved
- How do I format labels for cleanlab-- text amended
- Can't find an answer to your question-- added links
* add links to issues/slack
Co-authored-by: Ulyana Tkachenko <uly@ulyana.lan>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* cleanlearning w dfs
* work for sparse matrix as well
* simplify logic of labels_to_array and extend types
* address pr feedback
* add unit test
* rare label dataframe
* modularize subsetting code
* series rarelabel test
* replace cal.com with slack/email
* Add general method to find num_classes from labels
* compute num_classes with pred_probs.shape[1]
* fix broken commits, address 2nd round of comments
Co-authored-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
Add clarification of the labels format requirements for all major API functions.
* Fix broken link
* Clarify reqs for labels format. rank.py does not yet support multi_label
* add double ticks to code in docstrings
* clarify docstring
* further clarify multi_label vs single label labels reqs
* also add new docstring to confident joint
* example label formatting
* example label format in text tutorial
* example label format for tabular tutorial
* y -> full_labels
* example label format for audio tutorial
* clarify input shapes in image tutorial
* clarify input shapes in text tutorial
* clarify input shapes in tabular tutorial
* clarify input shapes for audio tutorial
* Add KNN distance OOD scoring function and unit tests
* Update KNN distance OOD scoring function
* Change query_features to features in unit tests for KNN distance OOD scoring function
* Update KNN distance OOD scoring function
* Update tests for KNN distance OOD scoring function to use auto for algo
* Allow k=None for KNN Distance OOD score
* added class to use cleanlab with tensorflow and huggingface models
* - Documentation refactoring
- predict and predict_proba functions work on new test data too
* Add log_loss_search weighting method for ensemble label quality scoring function
* Update log_loss_search weighting method
* Add test for log_loss_search method
* Add parameter to Ensemble label quality scoring function for t values in log_loss_search method
* Update ensemble label quality scoring function docstring
* Update ensemble label quality scoring function comments
* Update docstring in ensemble label quality scoring function
* Update docstring in ensemble label quality scoring function
* Update docstring in ensemble label quality scoring function
* Modify verbose printout for log_loss_search
* Add clipping of pred_prob when calculating weights for log_loss_search
* Add clipping of pred_prob and renormalization when calculating weights for log_loss_search
* Add comments for log_loss_search weighting scheme
The issue that was introduced in coverage 6.3 has been fixed:
https://github.com/nedbat/coveragepy/issues/1310#issuecomment-1129894701.
We can't just upgrade to `coverage` or `coverage>=6.4` because the
former could install bad versions of coverage (e.g. 6.3), and the latter
is unsupported on Python 3.6, which we want to continue supporting.
This patch just bans coverage 6.3 / 6.3.x, so with Python 3.7+, we'll
use the latest version of coverage, and with Python 3.6, we'll use the
latest supported version of coverage that's not a 6.3 release.
For some reason, GitHub is returning a 403 to the GitHub actions runner
when it tries to access these URLs. Maybe docs.github.com doesn't allow
access from GitHub actions.
* Allow user to pass custom_weights to ensemble scoring method
* Add tests for ensemble scoring with custom_weights
* Add check to make sure length of custom_weights matches len(pred_probs_list)
* Update tests for usage of custom_weights in ensemble scoring
Hui Wen