文件历史

26 次代码提交

作者 SHA1 备注 提交日期
Jonas Mueller 289872efb0 Remove sklearn install from tutorials (#1091) 2024-04-10 21:16:06 +00:00
Ulyana e460385027 Improve Multiannotator Tutorial (#827)
* Improve spelling + sentence structure

* Add omitted information

* Address PR comments
---------

Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
2023-08-25 11:21:54 -07:00
Jonas Mueller e081acc0d1 mention datalab for checking features at end of crowdlab tutorial (#816) 2023-08-15 00:00:02 -04:00
Hui Wen 0cd40abbb2 Remove explicit sklearn dependency from tutorial notebooks (#794) 2023-08-02 14:34:00 -04:00
Ulyana 7f12abf885 Add improved package import check for tutorials (#777)
Co-authored-by: Hui Wen <45724323+huiwengoh@users.noreply.github.com>
2023-07-27 17:56:47 -04:00
Jonas Mueller 44081c6538 add activelab name to docs (#648)
Co-authored-by: huiwengoh <45724323+huiwengoh@users.noreply.github.com>
2023-03-12 20:26:04 -07:00
Jonas Mueller 266d9479db shorten notebook name in link (#617) 2023-02-01 22:53:02 -08:00
Jonas Mueller 5f9fd95c45 update crowdlab paper name 2023-01-29 23:48:49 -08:00
Jonas Mueller c0b4e58471 multiannotator explanation improvements (#570)
more clarification when to use active learning vs fixed dataset analysis functions
2022-12-22 00:35:18 -08:00
Jonas Mueller 3a6f33c1ff typo: probabilites -> probabilities (#557) 2022-12-05 21:36:56 -08:00
Jonas Mueller 7dd06d14a0 remove unnecessary TF warning suppression statements in tutorials (#544) 2022-11-26 11:25:58 -08:00
Jonas Mueller aceaa4f25e Improve tutorials intro language/formatting (#526) 2022-11-06 00:34:54 -07:00
Hui Wen 738acae810 Point to format label function for multiannotator (#506) 2022-10-25 10:52:21 -07:00
Jonas Mueller 00781ce984 update paper links (#503)
Co-authored-by: huiwengoh <45724323+huiwengoh@users.noreply.github.com>
2022-10-18 00:01:00 -07:00
Jonas Mueller aedd0da886 mention crowdlab in multiannotator tutorial (#491) 2022-09-30 00:05:49 -07:00
Anish Athalye d8cc2e5056 Fix details disclosure elements in docs (#456)
This patch makes the following fixes:

- Make summary text appear on the same line as the ">" arrow marker;
  prior to this patch, it was being pushed down to the next line because
  there were nested block elements inside the <summary> element
- Style details element to visually delineate collapsible content with
  dashed lines above and below the content
- Add syntax highlighting for code blocks in details disclosure elements
  (some were missing syntax highlighting)
- Hide a cell that was meant to be hidden in the audio tutorial

The summary element looked ugly due to a block element being present
inside the <summary>. This was present in the rendered docs even though
it was not explicitly in the source. Even though our source (in a
Markdown cell in a Jupyter notebook) looked like:

    <summary>Example text</summary>

The rendered docs had:

    <summary><p>Example text</p></summary>

The easy fix was to leave this as-is and fix the styling with CSS,
making all elements inside the summary to be displayed as inline
elements.

Where was the extra <p> tag coming from? Different Markdown parsers
handle mixed Markdown/HTML differently. We are using nbsphinx, which
does something especially weird [1]: converting from Markdown ->
reStructuredText -> HTML.

For example, consider the following Markdown:

    <details><summary>Summary</summary>

    Body.

    </details>

nbsphinx was first converting it to this reST:

    .. raw:: html

       <details>

    .. raw:: html

       <summary>

    Summary

    .. raw:: html

       </summary>

    Body.

    .. raw:: html

       </details>

And then this reST was being converted to HTML. This is where the extra
<p> was coming from.

One workaround for this behavior is to switch to raw HTML cells in the
notebook, rather than Markdown cells, and write pure HTML there. This
has the disadvantage that it's inconvenient when the content inside the
<details> has extra markup: Markdown is very convenient for that (e.g.,
for styling text), and having the contents being parsed as Markdown also
allows us to easily use syntax highlighting inside the body.

Another workaround is to use raw reST cells [2] and write the proper
reST directly (avoiding the bad Markdown -> reST conversion). This
preserves the ability to use convenient formatting (though with reST
instead of Markdown) as well as syntax-highlighted code blocks. For
example:

    .. raw:: html

        <details><summary>Summary</summary>

    .. code-block:: python

        def meaning_of_life():
            return 42

    .. raw:: html

        </details>

The above snippet produces the expected HTML, with syntax-highlighted
code in the body, and without the extra <p> inside the <summary>.

Rather than either of these workarounds, the most convenient one for
developers is to fix the issue using CSS, so that the Markdown cells in
the tutorials can be left as-is and no special care is required when
writing details disclosure elements, which is what this patch does.

[1]: https://github.com/spatialaudio/nbsphinx/blob/fc79fd1ac5ab6df225b256ef4ca203c58c5ad92b/src/nbsphinx.py#L1309
[2]: https://nbsphinx.readthedocs.io/en/0.8.9/raw-cells.html#reST
2022-09-16 12:11:05 -07:00
Jonas Mueller a24dda5da8 clarify features for multiannotator tutorial 2022-09-16 11:22:02 -07:00
Jonas Mueller 5d900200e5 mention make_data in multiannotator tutorial 2022-09-16 03:46:03 -07:00
Elías Snorrason 5167d23b71 Update links to example notebooks (#444)
Related to cleanlab/examples#21
2022-09-15 11:10:33 -04:00
Hui Wen d74dda0ed9 fix code and md inconsistency (#442) 2022-09-14 14:45:09 -07:00
Jonas Mueller b9ab4ba57b suppress tensorflow warning logs in tutorials if not properly installed (#432)
* remove ineffective TF suppression lines

* suppress tensorflow logs in workflows tutorial

* formatting

* hidden cell comment

* move suppress code to hidden cell

* move suppression code to first hidden cell

* move suppression code to first cell

* supress tf logs in tabular tutorial

* optional code to hidden cells in audio tutorial

* suppress tf logs in multiannotator tutorial

* suppress tf logs in token tutorial

* move cross-val tutorial to appear last

* move cross-val tutorial to be last in main index

* move outlier before multiannotator in API list

* remove newline at end of cell
2022-09-13 00:47:21 -07:00
Jonas Mueller f7102db902 Add links to examples notebooks where appropriate (#428)
* links to OOD examples notebook

* iterative notebook title in its link
2022-09-11 12:24:51 -07:00
Jonas Mueller 78e8496363 Small tutorial notebook fixes/improvements (#420) 2022-09-09 17:36:27 +01:00
Hui Wen d8147caded Polish multiannotator docs (#422)
* docstring formatting

* add tiebreak info

* add example link to tutorial

* add missing backticks

* minor docstring text edits

* clarify "task"

Co-authored-by: Jonas Mueller <1390638+jwmueller@users.noreply.github.com>
2022-09-09 09:15:54 -07:00
Jonas Mueller 6ab83fc37c language improvements (#380) 2022-08-30 16:38:56 -07:00
Hui Wen 960c2b4ac1 CL functionality for multiannotator data (#333)
* setup multiannotator functionality

* add docstring

* add unittests

* edit get label quality with nan helper func

* docstring edits

* handle non overlapping annotators in get_annotator_agreement_with_annotators

* return NaN annotator quality scores for non-overlapping annotators

* add unittest

* add unittests for get_consensus_label tiebreaks

* np.array -> np.ndarray

* add new annotator quality method

* docstring edits

* address comments, docstring changes

* add method to compute improved consensus labels

* separate detailed_label_quality return

* elif statement typo

* changed get_majority_vote_label

* add 'best_quality' as a consensus method

* change lqs kwargs naming

* change lqs kwargs naming

* change unittest function names

* docstring edits

* change get_worst_class to return class with lowest agreement with consensus

* edit method to get annotator lqs, change quality_of_consensus -> consensus_quality_score

* remove unnecessary args

* satisfy mypy checks

* fix mypy issue

* add unittests

* reshuffled order of functions

* address comments

* docstring edit

* change return to always return dict

* add clipping to prevent negative pred_probs

* add tutorial

* add multiannotator to tutorial index

* add newline

* add tiebreaks for best_quality labels and worst_class

* change auto method name to crowdlab

* fix indexing for non numeric anno names

* add test for nonnumeric anno names

* update unittest

* address comments [partially complete]

* edit metadata

* missing /details tag

* bugfix to allow pd Int64 types

* update tutorial

* address comments
2022-08-30 11:29:06 -07:00