76dfdd23fa
Fix flaky test of outlier scores for identical examples ( #1079 )
2024-04-06 04:06:02 +00:00
a0f24add61
Use existing Datalab to audit additional new data ( #1049 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-04-06 02:51:59 +00:00
b751269fdc
Add user-facing cleanlab.data_valuation module ( #1050 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
2024-04-04 01:48:20 +00:00
2f2bc1fe18
Refine Scoring and Enhance Stability for Datasets with Identical Examples ( #1056 )
...
Enhance numerical stability and scoring accuracy for datasets with identical features
- Adjust minimum median values to 100x machine epsilon to prevent unstable operations.
- Implement specific scoring logic for outlier and near-duplicate issues in datasets with identical features, ensuring correct scores are assigned in such edge cases.
- Introduce unit tests to validate the expected behavior for datasets comprising entirely identical examples. Check an additional case with an extra unique example.
Related to issue #1055 .
2024-03-19 12:41:49 +00:00
ac184552b6
Convert DataFrame features to numpy arrays in null value check ( #1045 )
2024-03-08 15:43:41 +00:00
90ac796213
Transform regression label quality scores ( #1032 )
2024-03-05 17:15:21 +00:00
85823cad3e
Validate cleanvision issue types correctly in Datalab.get_issues ( #1028 )
2024-02-26 11:09:47 +00:00
a2dec09d20
check that lab.get_issues("label") works if 0 label issues found in a successful label issue check ( #1026 )
2024-02-24 03:57:58 +00:00
51a7d218e2
Fix error handling in get_issues method with empty issue dataframes ( #1025 )
2024-02-24 02:06:30 +00:00
6c7705a45b
Support features as input for data valuation check in Datalab ( #1023 )
2024-02-23 00:15:22 -08:00
79b16e68d1
Single Class Span Classification Support ( #982 )
...
* add single class span classification support
2024-02-14 20:17:04 +05:30
fd0dcd2c97
Improve the datalab.display module ( #1005 )
2024-02-12 19:02:42 -08:00
1ed51aa35a
Fix issue summary in report.py to omit undetected issue types ( #1003 )
...
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
2024-02-12 16:53:10 -08:00
d3aba50c4e
Multilabel class order ( #997 )
...
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
2024-02-12 16:09:08 -08:00
b640fb5b8f
Add underperforming_group issue type among the Datalab defaults ( #990 )
...
Closes #910
2024-02-12 08:27:14 -08:00
16c5866a8b
Add Task enum for specifying machine learning task within Datalab ( #971 )
2024-02-08 15:35:38 -07:00
bc9fd539fb
Increase deadline for test_find_label_issues for multilabel data ( #985 )
2024-02-08 08:33:54 -07:00
5540959173
Add existing issue managers to more tasks ( #979 )
...
Co-authored-by: Sanjana <sanjanag@users.noreply.github.com >
2024-02-07 22:05:21 -07:00
387ffa3504
Add a show_all_issues optional argument to Datalab.report() ( #970 )
...
Co-authored-by: Sanjana <sanjanag@users.noreply.github.com >
2024-02-07 14:52:00 -07:00
bad83457f3
Drop Python 3.7 support and add Python 3.11 support ( #980 )
2024-02-07 14:46:58 -07:00
070b6e5b42
Fix dev dependencies (pytest and black formatter) ( #975 )
2024-02-01 17:42:56 +00:00
22dd845c48
add bbox area computation ( #968 )
2024-01-25 20:21:06 -08:00
51de77768d
Multilabel Issue Manager for Classification ( #929 )
...
* Add Multilabel label issue manager
* Add task validation in Datalab constructor
* Add type checks for labels in issue managers
* Refactor code to use Datalab for finding label issues in multi-label classification tutorial
---------
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-24 00:14:04 +00:00
89866d53b4
make data valuation issue manager user facing ( #925 )
...
Co-authored-by: Ganesh Tata <tataganesh95@gmail.com >
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-17 17:49:58 +00:00
15bec56103
Rescale ood scores in outlier package ( #953 )
...
Co-authored-by: Ulyana <ulya.tkach@gmail.com >
2024-01-17 17:42:23 +00:00
b2de6bbefb
add property based test for multilabel classification ( #945 )
...
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-10 14:55:09 +00:00
0a03742f52
Refactor batching logic in cleanlab.segmentation.filter.find_label_issues ( #918 )
2024-01-08 11:30:25 +00:00
5e078339a7
Make Null Issue Manager User-Facing ( #927 )
...
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-05 14:15:29 +00:00
23af72a2bd
Near duplicate score transformation ( #943 )
2024-01-05 01:04:16 +00:00
9ecfea1316
switch rarest class column to given label column ( #940 )
...
follow-up to #933
2024-01-03 10:00:56 +00:00
b806cba4aa
Add imbalance class label to Issues Dataframe ( #933 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-03 08:35:02 +00:00
7eb9967a0b
class imbalance scores to still show rarest proportion even when is_i… ( #934 )
...
Co-authored-by: Ganesh Tata <tataganesh95@gmail.com >
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2024-01-02 16:39:27 +00:00
bc10585352
Patch: epsilon for near duplicate ( #919 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-12-21 13:03:51 +00:00
03d5c968b9
Report class metrics for object detection dataset ( #877 )
...
* Add support to per-class metrics for Object detection, thanks @Steven-Yiran
2023-12-19 09:22:20 +05:30
15d39c5703
FEAT: Vectorize loops, bug fix ( #917 )
...
- Vectorize for loops for speedup
- Get null rows correctly
2023-12-18 15:24:38 +00:00
f3a65b8c18
Issue 873 improve mem find label issues ( #885 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-12-15 12:20:40 +00:00
3fb20e9596
CI: Run slowest tests at the end and fail early ( #900 )
2023-12-14 17:59:28 +00:00
4ad0b6e001
Add Class Imbalance issue type to Datalab defaults ( #912 )
2023-12-14 17:52:41 +00:00
2e3608a2d8
Predictions for regression in Datalab ( #902 )
...
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com >
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
2023-12-07 04:26:16 +00:00
dee32ad9a1
Underperforming Group Issue Type ( #838 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-30 16:27:27 +00:00
e9ee35f075
Test properties of near duplicate sets ( #895 )
...
* create and test strategy for generating knn_graph as test inputs
* test strategy for generating knn_graphs
- Each row has the distances sorted in ascending order in the csr-format.
- The indices of the neighbors are unique within a column and don't have the query point as a neighbor.
- If points a and b are mutual neighbors, they have the same distance between them.
* add property based tests for near-duplicate sets
* flag near-duplicate issues based on items in near duplicate sets.
2023-11-28 15:06:48 +00:00
09d95dc47f
add false negative support for ObjectLab2D ( #883 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-21 18:39:18 +00:00
a6d1319327
Introduce regression support to Datalab ( #796 )
...
Preleminary work to ensure that default task (classification) will work smoothly.
Minimal implementation of label issue manager for regression
- issue_finder.py list_possible_issue_types, list_default_issue_types no longer a static methods as they depend on task and modification of get_available_issue_types for regression
* DataIssues with strategy pattern
- moving list_possible_issue_types and list_default_issue_types from the IssueFinder class to cleanlab/datalab/internal/issue_manager_factory.py
* make different strategies for getting available issue types
add test class for the IssueFinder when the task is regression
* apply black formatter
* avoid mapping labels column for regression in Datalab
* Pass in features to LabelIssueManager for regression
---------
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-20 20:28:53 +00:00
0903c2a361
Non-IID: Test that Datalab prioritizes knn-graph over pred-probs ( #897 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-16 15:21:59 +00:00
aeab1d1577
Extract Common Setup Code in Non-IID Tests for Reusability ( #896 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-16 10:48:30 +00:00
5fbf6c0493
Add Data Valuation Issue Manager ( #850 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-11-06 17:04:57 +01:00
ccb4a90e38
Extend non-iid issue check to run if only pred_probs are provided ( #857 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-10-30 02:55:53 +00:00
69c8c5318c
Add Null Issue Manager ( #856 )
...
Co-authored-by: Elías Snorrason <eliassno@gmail.com >
2023-10-30 02:42:54 +00:00
a28f0f1dad
Uly dataset health ( #875 )
...
* Remove datasets from the tutorial that are logits
* Update dataset code
* Change hardcoded expected score in datalab tests
2023-10-25 01:21:58 +00:00
902fb487c9
Functions to explore object detection dataset ( #840 )
...
Add dataset summary functions+plots to object detection.
---------
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com >
Co-authored-by: Ulyana <ulyana@cleanlab.ai >
2023-10-16 13:17:05 -07:00
Martin Zornoza Auñon