* 🏷️ restrict parameters for list types
* 🐛 only process characters in input token
Example: process_token("Cleanlab", [("C", "a"), ("a", "C")]) should return "aleCnlCb", not "CleCnlCb".
* ✅ use all sentences in test_get_sentence
* ✅ add test cases to test_filter_sentence
* ✅ extent test cases in test_mapping
* 📝 clean up docstrings
Restrict arg types based on docstrings. Fix punctuation and typos. Add examples to docstring.
* ✅ split tests for filter_sentence
* ✅ extend test_merge_probs
* ✅ test merge_probs with ignored/normalized columns in probs
* ✅ extend test cases for get_sentences
* ⚰️ remove unused pandas import
* 🏷️ pass strict mypy check
We ignore np.max as it is untyped.
No issues found in token_classification_utils.py by running
```
mypy --install-types --non-interactive --strict cleanlab/internal/token_classification_utils.py
```
* 👷 add strict type-checking in CI
* 💚 use strict type-check for single file
* ⏪ remove strict type check in CI
* refactor: 🏷️ use np.ndarray type instead of npt.NDArray
* ⏪ go back to generic np.ndarray type
* ♻️ always return tuple in filter_sentence
Remove unused argument+docstring. Simplify relevant unit tests.
* 🔥 resolve comments on typing
Remove ignore-comments. Remove duplicate tag in docstring. Remove unused imports.
* 🔥 remove duplicate tag in docstring
* Changed all instances of np.array in docstring to np.ndarray
* np.array is NOT a proper class name it is just a function to
create np.ndarrays and therefore should not be parameter class
* Black formatting compliance
* validation_func docstring
* torch,tf compatibility+tests
* keras test
* skip tests if python < 3.7
* pytorch numpy int bug on windows
* make tensorflow test work on windows
* move tf env variable setting
* pytorch test increase epochs
* install cpu-tensorflow on windows CI
* torch test optimizer to adam
* fix bugs in shuffled TF dataset
* dummy unit test for TF on windows
* dummy code for TF windows testing
* deal with np.int bug on windows
* remove windows debugging code
* docstrings for new functionality
* address merge conflicts
* reformat after merge
* addressed comments
* 🏷️ annotate function args and return values
Starting with the validation module:
- I think (X, y) might need some custom Union type to handle both numpy arrays and pandas dataframe, etc.
- All of the "assert" functions return None.
Ref #307
* refactor: 🏷️ swap npt.NDArray -> np.ndarray
np.ndarray seems more consistent with the rest of the repo.
Maybe it's necessary to go back to npt.NDArray when disallowing generics?
See numpy docs: https://numpy.org/devdocs/reference/typing.html#numpy.typing.NDArray
* refactor: 🔥 remove unused import
* 🏷️ unconstrain X input types
* 🐛 handle label type issues
- Have to restrict the output of labels_to_array to pass mypy checks.
- Returning the values of pd.Series isn't type-stable.
* 🏷️ include np.generic in arg-type union
* test: ✅ test labels_to_array
* 🏷️ add type aliases for X and y
* 🚨 ignore type-checks for pandas indexing assertions
CI typechecker runs on Python 3.10 which gives this error:
'cleanlab/internal/validation.py:125: error: No overload variant of "__getitem__" of "_iLocIndexerSeries" matches argument type "List[int]"'
It should be fine to let mypy ignore these expressions as they don't return anything.
* 🥅 specify errors to ignore
"type: ignore" doesn't pass strict mypy type-checks unless the specific errors are provided
* 🏷️ annotate label series to array
Our package doesn't have type annotations everywhere, so we can't use
mypy in strict mode just yet. Still, adding type checking in CI is
valuable, so we don't have unchecked annotations in our code.
This patch includes basic fixes to make type checking pass, including
switching the incorrect `np.array` type annotation for `np.ndarray` and
adding some assertions for flow-sensitive typing.
* cleanlearning w dfs
* work for sparse matrix as well
* simplify logic of labels_to_array and extend types
* address pr feedback
* add unit test
* rare label dataframe
* modularize subsetting code
* series rarelabel test
* replace cal.com with slack/email
* Add general method to find num_classes from labels
* compute num_classes with pred_probs.shape[1]
* fix broken commits, address 2nd round of comments
Co-authored-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
* df return type, need tests still
* Add pandas as a dependency
We already decided that pandas will be a dependency of cleanlab (also
used in the dataset module, see
https://github.com/cleanlab/cleanlab/pull/182).
* Tweak documentation
* addressed comments
* remove lazy import
* address 2nd round comments
* unit tests
* improve codecov
* Fix typo
* methods to save more space
* nocover statements for prints
* extra nocover
* nocover warnings
* test docstring formatting
* test docstring formatting2
* test docstring formatting2
* move compress to helper, find-label docs params
* readded stuff lost in merge conflict
* addressed remaining PR review comments
* docs formatting
* docs formatting2
* docs formatting3
* docs formatting4
* docs formatting5
* docs formatting5
* docs formatting6
* docs formatting7
* docs formatting8
* docs formatting9
* docs formatting19
* docs formatting20
* docs formatting20
* docs formatting21
* code formatting
* fix a bug where confident joint isnt computed
The confident joint wasn't getting computed if noise_matrix was passed in and pred_probs was not passed in. But that's bad because it stops workflows like:
```python
cl = CleanLearning()
cl.fit(data, labels, noise_matrix=noise_matrix)
cleanlab.dataset.health_summary(labels, confident_joint=cl.confident_joint)
```
* fixed bug from last commit. code in wrong place.
* print overwrite bugfix
Co-authored-by: Anish Athalye <me@anishathalye.com>
Co-authored-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
* added LearningWithNoisyLabels.find_label_issues instance method
* LNL.find_label_issues no longer memoizes
* verbose unit test coverage
* Fixed all issues in PR. Fixed confident joint usage in LNL.find_label_issues. Fixed other minor bugs. Added tests
Co-authored-by: Curtis G. Northcutt <curtis.northcutt@gmail.com>
Co-authored-by: Anish Athalye <me@anishathalye.com>
This was used to skip e.g. importing PyTorch on some tests. However,
PyTorch supports all versions of Python that Cleanlab supports, so
there's no need to issue this warning. Furthermore, having this kind of
code in our codebase increases maintenance burden and contributes to
user frustration. If we mark a version of some library as unsupported,
but the library adds support for it in the future, there's no way users
can use the two packages together without waiting for us to push a new
version of cleanlab that marks that it's compatible and then switching
to the new version, which may be painful for unrelated reasons. It's
better to not have these kinds of checks; if users are using a version
of Python that's unsupported by some other library that they want to
use, that's not our problem, and they'll have issues installing or
importing it and figure it out themselves.
This patch also removes mentions of Python 2.
LearningWithNoisyLabels used `sklearn.utils.check_X_y` to enforce that
`X` was 2D, to be in line with what sklearn's standard estimators
expect. However, LearningWithNoisyLabels is dataset-agnostic: it doesn't
look at the data points themselves. If the underlying classifier
supports data in a different shape, there's no reason
LearningWithNoisyLabels should prohibit it. Users have requested that we
relax this unnecessary restriction [1] so LearningWithNoisyLabels will
more naturally support e.g. image datasets and CNN models.
Thanks to @kothari1997narayan for suggesting this change.
[1]: https://github.com/cleanlab/cleanlab/issues/86
* utilities -> internals
* docbuilding instructions improvement
* fixup formatting of contributing.md
* change contributor guidelines language to be optional
* line formatting
Elías Snorrason