Among other changes, this patch also disables HTML validation. This is a
bit tricky to get right, because when the Markdown is processed by
GitHub, it does some transformations that aren't mimicked by Pandoc. The
result of converting the existing Markdown to HTML using Pandoc is in
fact invalid HTML5.
The best way to fix this would be to use the [GitHub Markdown
API](https://docs.github.com/en/rest/markdown) to render the Markdown,
but that doesn't seem worth the effort right now. Instead, this patch
just disables HTML validation but preserves the link checking
functionality (and makes it start passing again).
Previously, the CI cron schedule, copy-pasted from a personal project,
ran jobs at 8am UTC (3am EST or 4am EDT) on Saturday. This isn't a great
time to get emails about CI breakages.
This patch switches the schedule so it runs on Monday morning (7am EST /
8am EDT).
[skip ci]
This patch moves the `--allow-redefinition` into the `.mypi.ini` file.
Centralizing our mypy config in a single file makes sense, and this way,
if people want to run mypy locally, they don't need to remember to
supply this command-line argument.
This patch pins the Pandoc version to a more recent release (2.19.2).
As explained in d8cc2e5056, nbconvert
translates Markdown -> reST -> HTML, and it uses Pandoc for this
Markdown -> reST conversion. Older versions of Pandoc don't do a
high-fidelity conversion; in particular, they don't have great support
for mixed Markdown/HTML content.
Consider the following Markdown, the core of how we use the details
disclosure element in several tutorial notebooks:
<details><summary>Summary</summary>
Body.
</details>
Using Pandoc 2.19.2, this converts to the following reST:
.. raw:: html
<details>
.. raw:: html
<summary>
Summary
.. raw:: html
</summary>
Body.
.. raw:: html
</details>
This is a correct conversion. However, using Pandoc 2.5, which is the version
of Pandoc that was being used by CI prior to this patch (the version of Pandoc
available in the package repository for Ubuntu 20.04), it converts as follows:
.. raw:: html
<details>
Summary
Body.
.. raw:: html
</details>
The <summary> tag is lost in this conversion.
This patch switches to a more recent version of Pandoc to fix this issue.
* validation_func docstring
* torch,tf compatibility+tests
* keras test
* skip tests if python < 3.7
* pytorch numpy int bug on windows
* make tensorflow test work on windows
* move tf env variable setting
* pytorch test increase epochs
* install cpu-tensorflow on windows CI
* torch test optimizer to adam
* fix bugs in shuffled TF dataset
* dummy unit test for TF on windows
* dummy code for TF windows testing
* deal with np.int bug on windows
* remove windows debugging code
* docstrings for new functionality
* address merge conflicts
* reformat after merge
* addressed comments
Our package doesn't have type annotations everywhere, so we can't use
mypy in strict mode just yet. Still, adding type checking in CI is
valuable, so we don't have unchecked annotations in our code.
This patch includes basic fixes to make type checking pass, including
switching the incorrect `np.array` type annotation for `np.ndarray` and
adding some assertions for flow-sensitive typing.
For some reason, GitHub is returning a 403 to the GitHub actions runner
when it tries to access these URLs. Maybe docs.github.com doesn't allow
access from GitHub actions.
Preserving the output cells in Jupyter notebooks that are checked in to
this repo bloat the repo size, and keeping the output is not necessary
because the notebooks are run as part of the CI for producing docs.
For example, in https://github.com/cleanlab/cleanlab/pull/165, the
`audio.ipynb` is 112K (and now part of the repo history forever), while
clearing the output cells reduces that size down to 24K. I suspect that
Git's delta compression will also work better without the output cells
there as we make changes to the notebook. Also, the diffs will be more
readable.
Ganesh Tata