mirrors / cleanlab--cleanlab

文件历史

1,485 提交 54 分支 21 Git 标签
jwmueller-quickstarttutorial
提交图

272 次代码提交

作者 SHA1 备注 提交日期
Kyle Gallatin f3a65b8c18 Issue 873 improve mem find label issues (#885) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-12-15 12:20:40 +00:00
Ganesh Tata 3fb20e9596 CI: Run slowest tests at the end and fail early (#900) 2023-12-14 17:59:28 +00:00
Ganesh Tata 4ad0b6e001 Add Class Imbalance issue type to Datalab defaults (#912) 2023-12-14 17:52:41 +00:00
Elías Snorrason 2e3608a2d8 Predictions for regression in Datalab (#902) 
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
 2023-12-07 04:26:16 +00:00
Ganesh Tata dee32ad9a1 Underperforming Group Issue Type (#838) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-30 16:27:27 +00:00
Elías Snorrason e9ee35f075 Test properties of near duplicate sets (#895) 
* create and test strategy for generating knn_graph as test inputs

* test strategy for generating knn_graphs

- Each row has the distances sorted in ascending order in the csr-format.
- The indices of the neighbors are unique within a column and don't have the query point as a neighbor.
- If points a and b are mutual neighbors, they have the same distance between them.

* add property based tests for near-duplicate sets

* flag near-duplicate issues based on items in near duplicate sets.
 2023-11-28 15:06:48 +00:00
Aditya Thyagarajan 09d95dc47f add false negative support for ObjectLab2D (#883) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-21 18:39:18 +00:00
OrdoAbChao a6d1319327 Introduce regression support to Datalab (#796) 
Preleminary work to ensure that default task  (classification) will work smoothly.

Minimal implementation of label issue manager for regression

- issue_finder.py list_possible_issue_types, list_default_issue_types no longer a static methods as they depend on task and modification of get_available_issue_types for regression

* DataIssues with strategy pattern


- moving list_possible_issue_types and list_default_issue_types from the IssueFinder class to cleanlab/datalab/internal/issue_manager_factory.py

* make different strategies for getting available issue types

add test class for the IssueFinder when the task is regression

* apply black formatter

* avoid mapping labels column for regression in Datalab

* Pass in features to LabelIssueManager for regression

---------

Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-20 20:28:53 +00:00
Ganesh Tata 0903c2a361 Non-IID: Test that Datalab prioritizes knn-graph over pred-probs (#897) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-16 15:21:59 +00:00
Ganesh Tata aeab1d1577 Extract Common Setup Code in Non-IID Tests for Reusability (#896) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-16 10:48:30 +00:00
ChG 5fbf6c0493 Add Data Valuation Issue Manager (#850) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-11-06 17:04:57 +01:00
ABHIJIT PAL ccb4a90e38 Extend non-iid issue check to run if only pred_probs are provided (#857) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-10-30 02:55:53 +00:00
ABHIJIT PAL 69c8c5318c Add Null Issue Manager (#856) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-10-30 02:42:54 +00:00
Ulyana a28f0f1dad Uly dataset health (#875) 
* Remove datasets from the tutorial that are logits
* Update dataset code
* Change hardcoded expected score in datalab tests
 2023-10-25 01:21:58 +00:00
Yiran Shi 902fb487c9 Functions to explore object detection dataset (#840) 
Add dataset summary functions+plots to object detection.
---------

Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Ulyana <ulyana@cleanlab.ai>
 2023-10-16 13:17:05 -07:00
Aditya Thyagarajan ffbddc3a9e mypy linting (#864) 
* mypy linting

* summary without type check

* Add test to confirm np.array can be passed into visualize

---------

Co-authored-by: Ulyana <ulya.tkach@gmail.com>
 2023-10-05 15:54:53 -07:00
Sanjana Garg e843938185 Added unit test for max prevalence 2023-09-01 16:54:33 +01:00
Sanjana Garg 0a7c05cb8e Fixed test 2023-08-31 18:24:19 +01:00
ChG 4250f5925d Add GEN out-of-distribution detection method (#800) 
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-08-29 22:30:59 -07:00
Elías Snorrason 4ce9f771c8 Unify softmax implementations (#826) 
* unify softmax implementations

* Refine mypy type hint in _set_fine_search_range function

- Update type hint for `min_entropy_ind` from built-in `int` to `np.intp`.
  - This refinement addresses a type compatibility warning.
  - `np.intp` is the integer type used by numpy for indexing and can differ in size from the built-in Python `int` depending on the platform (32-bit vs 64-bit).
  - Mypy highlighted this type hint discrepancy.

* add unit tests for softmax

---------

Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-08-30 00:22:05 +00:00
Tata Ganesh fd2506c6b0 Use rare class proportion instead of mean values for imbalance score (#828) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-08-28 14:24:24 +00:00
Aditya Thyagarajan a497b38c28 change thresholding class-wise for ObjectLab (#804) 
* Adding support for per-class thresholding for ObjectLab
 2023-08-26 05:06:00 +05:30
Tata Ganesh d37a7d8d85 Expose low memory option for token and multi-label classification (#822) 2023-08-23 01:14:29 -04:00
Hui Wen 0adce8290f 50x speedup for multiannotator module (#821) 2023-08-16 18:39:06 -07:00
Ulyana 396bd68e66 Improve indexing for Object Detection Badloc Score (#807) 2023-08-14 09:29:56 -07:00
Tata Ganesh 83731599be Expose low memory option for finding label issues in classification (#791) 2023-08-11 00:35:51 -04:00
Elías Snorrason ee65553a05 Increase hypothesis deadline for testing near duplicate sets (#802) 2023-08-08 14:46:02 -04:00
Tata Ganesh 3d50b9859d Add imbalance issue to datalab (#758) 
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-07-28 18:42:15 +00:00
Elías Snorrason b0fb8e5f77 Move Datalab internals to dedicated internal module (#783) 2023-07-28 16:18:51 +00:00
Yiran Shi bc78816341 Label Issue Detection Extention (#760) 
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-07-27 15:43:34 -04:00
Elías Snorrason cc0fb470d9 Update near-duplicate sets (#781) 2023-07-26 17:12:20 +00:00
Sanjana 944c61379d Cleanvision Integration (#679) 
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
 2023-07-25 15:51:53 +00:00
Hui Wen d84d00e3f4 add test make sure correct examples are not flagged (#778) 2023-07-21 21:18:21 -07:00
Ulyana dcd9ea3a16 Improve OD scores (#770) 
* Add updated badloc, overlooked scores
* Improve code clarity for all scores
* Add codecov tests
* Remove unnecessary logic for swap score
 2023-07-21 12:37:51 -07:00
Aditya Thyagarajan cb51891b06 lowering swap score for multiple labels associated with a box (#750) 
* Lower swap scores for overlapping annotations + Optional arguments correction

---------

Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Ulyana <ulya.tkach@gmail.com>
 2023-07-12 20:29:20 +05:30
Yiran Shi cb4c05cbff Add support for high dimensional arrays in CleanLearning(#749) 
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-06-27 00:12:52 -04:00
Mayank Kumar 9f302200a4 Regression label quality scores (#572) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: huiwengoh <45724323+huiwengoh@users.noreply.github.com>
 2023-06-23 17:51:46 -07:00
Vedang Lad 35606fc401 label error detection in semantic segmentation datasets (#677) 
Co-authored-by: Ulyana <ulyana@cleanlab.ai>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
 2023-06-23 00:32:08 -07:00
Ulyana 299d66f3ec Update object detection tutorial to clarify out-of-sample predictions (#745) 
* Add out-of-sample clarification to notebook, fix filename check, add unit tests
 2023-06-15 13:14:30 -07:00
Ulyana da65a9727f identifying label errors in Object Detection data (#676) 
Co-authored-by: Aditya Thyagarajan <aditya1593@icloud.com>
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-06-07 01:12:46 -07:00
Elías Snorrason 43e24ba3f6 Make labels optional in Datalab (#730) 
* feat(datalab): ✨ make labels optional

* test(datalab): ✅ fix test case for comparing empty list/array
 2023-06-05 16:04:13 +00:00
DerWeh 114201bf76 ENH: make clipping unnecessary for entropy (#703) 
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-05-25 00:49:30 -04:00
Elías Snorrason 375c76fcea Include non-iid in default issue checks (#723) 
* flag no more than one example having non-iid issue

* feat(datalab): ✨ add non-iid to default issue checks
 2023-05-19 20:30:01 +00:00
Elías Snorrason 7400051a95 Fix unbound knn variable in NonIIDIssueManager (#709) 2023-05-10 21:38:47 -07:00
Elías Snorrason fcc9417a3d Simplify reporting for outlier issues (#705) 2023-05-10 01:49:17 +00:00
Elías Snorrason d16cdd67cf Suppress too_slow health check for dataset generation strategy (#706) 2023-05-10 01:48:45 +00:00
Elías Snorrason c41c837e43 Set non iid num issues with an extra rule (#707) 
* feat(datalab): 👔 count number of non-iid issues differently

* fix broken seeding test

* add significance threshold as a parameter

* docs(datalab): 📝 add docs on significance_threshold argument for non-iid issue manager

* test(datalab): ✅ add more end-to-end tests for non-iid issue checks with Datalab

---------

Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
 2023-05-10 01:33:25 +00:00
Sanjana cf5d2a02eb Added installation instructions for package extras (#697) 
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
Co-authored-by: huiwengoh <45724323+huiwengoh@users.noreply.github.com>
 2023-05-09 15:19:24 -04:00
Elías Snorrason be1bd6903d Seed noniid permutation tests (#694) 2023-05-08 22:50:11 -07:00
Elías Snorrason 5ee2eb8be9 Drop summary scores from Datalab.report() (#699) 
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
 2023-05-04 22:15:41 -04:00