Preleminary work to ensure that default task (classification) will work smoothly.
Minimal implementation of label issue manager for regression
- issue_finder.py list_possible_issue_types, list_default_issue_types no longer a static methods as they depend on task and modification of get_available_issue_types for regression
* DataIssues with strategy pattern
- moving list_possible_issue_types and list_default_issue_types from the IssueFinder class to cleanlab/datalab/internal/issue_manager_factory.py
* make different strategies for getting available issue types
add test class for the IssueFinder when the task is regression
* apply black formatter
* avoid mapping labels column for regression in Datalab
* Pass in features to LabelIssueManager for regression
---------
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
* feat(datalab): 👔 count number of non-iid issues differently
* fix broken seeding test
* add significance threshold as a parameter
* docs(datalab): 📝 add docs on significance_threshold argument for non-iid issue manager
* test(datalab): ✅ add more end-to-end tests for non-iid issue checks with Datalab
---------
Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
* issuemanager.get_summary->make_summary
* test(datalab): ♻️ move tests for issue managers into their respective test modules
* test(datalab): ✅ add tests for custom issue manager
- Validate scores provided to IssueManager.make_summary
- Move fixture for custom issue manager to conftest.py
- Test make_summary on a "custom" IssueManager
* test(datalab): add __init__.py for discoverability of issue manager tests
---------
Co-authored-by: Elías Snorrason <eliassno@gmail.com>
Add a new method `Reporter.report()` with fewer restrictions. The existing `get_report()` method is a pure function that returns a string as output. The new `report()` method can have side-effects, such as printing the report.
Ganesh Tata