/docs

CI/CD for cleanlab docs

In the cleanlab repository, we've configured GitHub Actions to perform the following automatically:

1. When a commit is pushed to the master branch, a new version of the master docs will be built and deployed to the cleanlab-docs repository.

2. When a release is published, a new version of the docs with the corresponding release tag will be built and deployed as a new folder in the cleanlab-docs repository. Redirection to the stable version of the docs will be changed to this newly released one, accessible via a link on the docs' site sidebar. All the older versions will remain available in the cleanlab-docs repo, accessible by manually entering the subdirectory in the URL.

3. When a user manually runs the workflow, one of the above will happen depending on the user's selection to run from a branch or tag.

If you'd like to build our docs locally or remotely yourself, or want to know more about the steps taken in the GitHub Pages workflow, read on!

Build the cleanlab docs locally

1. Fork and clone the cleanlab repository.

2. Install the required packages to build the docs:

pip install -r docs/requirements.txt

3. Build the docs with sphinx-multiversion:

   • If you're building from a branch (usually the master branch):

   sphinx-multiversion docs/source docs/build -D smv_branch_whitelist=YOUR_BRANCH_NAME -D smv_tag_whitelist=None
   

   • If you're building from a tag (usually the tag of the stable release):

   sphinx-multiversion docs/source docs/build -D smv_branch_whitelist=None -D smv_tag_whitelist=YOUR_TAG_NAME
   

   Note: If you have more than one branch or tag, run the above command again changing only the YOUR_BRANCH_NAME or YOUR_TAG_NAME placeholder.

4. [Optional] To show dynamic versioning and version warning banners:

   • Copy the docs/_templates/versioning.js file to the docs/build directory.

   • In the copied versioning.js file:

     • find placeholder_version_number and replace it with the latest release tag name, and

     • find placeholder_commit_hash and replace it with the master branch commit hash.

5. [Optional] To redirect site visits from the port address to the stable version of the docs:

   • Copy the docs/_templates/redirect-to-stable.html file to the docs/build directory and rename it as index.html.

   • In this index.html file, find stable_url and replace it with ./YOUR_LATEST_RELEASE_TAG_NAME/index.html. Note the single period, ., before the subdirectory.

6. [Optional] To redirect site visits from the subdirectory PORT_ADDRESS/stable to the stable version of the docs:

   • Create a folder called stable in docs/build/stable

   • Copy the docs/_templates/redirect-to-stable.html file to the docs/build directory and rename it as index.html.

   • In this index.html file, find stable_url and replace it with ../YOUR_LATEST_RELEASE_TAG_NAME/index.html. Note the double period, .., before the subdirectory.

7. The docs for each branch and/or tag can be found in the docs/build directory, open any of the index.html in your browser to view the docs:

docs
│
└───build
│   |   index.html (redirects to stable release of the docs)
|   |   versioning.js (for dynamic versioning and version warning banner)
|   |
│   └───YOUR_BRANCH_NAME (e.g. master)
│   │       index.html
│   │       ...
│   │
│   └───YOUR_TAG_NAME_1 (e.g. your stable release tag name)
│   │       index.html
│   │       ...
│   │
│   └───YOUR_TAG_NAME_2 (e.g. an old release tag name)
│   │       index.html
│   │       ...
│   │
│   └───stable
│   │       index.html (redirects to stable release of the docs)
│   │
│   └───...
│
└───...
    │   ...

Build the cleanlab docs remotely on GitHub

1. Fork the cleanlab repository.

2. Create a new repository named cleanlab-docs and a new branch named master.

3. In the cleanlab-docs repo, configure GitHub Pages; under the Source section, select the master branch and /(root) folder. Take note of the URL where your site is published.

4. Generate SSH deploy key and add them to your repos as such:

   • In the cleanlab-docs repo, go to Settings > Deploy Keys > Add deploy key and add your public key with the Allow write access
   • In the cleanlab repo, go to Settings > Secrets > New repository secrets and add your private key named ACTIONS_DEPLOY_KEY

5. In the cleanlab repo, check that you have the GitHub Pages workflow under the repo's Actions tab. This should be created automatically from .github\workflows\gh-pages.yaml. This workflow can be activated by any of the 3 triggers below:

   • A push to the master branch in the cleanlab repo.
   • Publish of a new release in the cleanlab repo.
   • Manually run from the Run workflow option and select either the master branch or one of the release tag.

6. Activate the workflow with any of the 3 triggers listed above and wait for it to complete.

7. Navigate to the URL where your GitHub Pages site is published in step 3. The default URL should have the format https://repository_owner.github.io/cleanlab-docs/.

Behind-the-scenes of the GitHub Pages workflow

We've configured GitHub Actions to run the GitHub Pages workflow (gh-pages.yaml) to build and deploy our docs' static files. Here's a breakdown of what this workflow does in the background:

Spin up and configure the CI/CD server

1. Spin up a Ubuntu server.

2. Install Pandoc, a document converter required by nbsphinx to generate static sites from notebooks (.ipynb).

3. Check-out the cleanlab repository.

4. Setup Python and cache dependencies.

5. Install dependencies for the docs from docs/requirements.txt.

Build the docs' static site files

6. Run Sphinx with the sphinx-multiversion wrapper to build the doc's static site files. These files will be outputted to the docs/build directory.

Generate the versioning.js file used to store the latest release tag name and commit hash

7. Get the latest release tag name and insert it in the versioning.js file. The index.html of each doc version will read this as a variable and display it beside the stable hyperlink.

8. Insert the latest commit hash in the versioning.js file. The index.html of each doc version will read this as a variable and display it beside the developer hyperlink.

9. Copy the versioning.js file to the docs/build folder.

If the workflow is triggered by a new release, generate the redirecting HTML which redirects site visits to the stable version

10. Insert the repository owner name in the redirect-to-stable.html file AKA the redirecting HTML.

11. Create a copy of the redirect-to-stable.html file to docs/build then insert the stable release docs's index.html file path prefixed with .

12. Create a copy of the redirect-to-stable.html file to docs/build/stable then insert the stable release docs's index.html file path prefixed with ..

Deploy the static files

13. Deploy docs/build folder to the cleanlab/cleanlab-docs repo's master branch.