{
 "cells": [
  {
   "cell_type": "raw",
   "id": "0cd68b28",
   "metadata": {},
   "source": [
    "---\n",
    "skip_exec: true\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c859632",
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.vision.all import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08a1554e",
   "metadata": {},
   "source": [
    "# Pure PyTorch to fastai\n",
    "\n",
    "> Incrementally adding fastai goodness to your PyTorch models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78ed48e1",
   "metadata": {},
   "source": [
    "We're going to use the MNIST training code from the official PyTorch examples, slightly reformatted for space, updated from AdaDelta to AdamW, and converted from a script to a module. There's a lot of code, so we've put it into migrating_pytorch.py!\n",
    "\n",
    ":::{.callout-note}\n",
    "\n",
    "The source script for `migrating_pytorch` is in the `examples` subdirectory of this folder if you checked out the `fastai` repo from git, or can be downloaded from [here](https://github.com/fastai/fastai/blob/master/nbs/examples/migrating_pytorch.py) if you're using an online viewer such as Colab.\n",
    "\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab9c37c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from migrating_pytorch import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a053a4b8",
   "metadata": {},
   "source": [
    "We can entirely replace the custom training loop with fastai's. That means you can get rid of `train()`, `test()`, and the epoch loop in the original code, and replace it all with just this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4818e127",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = DataLoaders(train_loader, test_loader)\n",
    "learn = Learner(data, Net(), loss_func=F.nll_loss, opt_func=Adam, metrics=accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee72bfbe",
   "metadata": {},
   "source": [
    "Data is automatically moved to the GPU or CPU depending on what's available, without the need of extra Callbacks or overhead.\n",
    "\n",
    "fastai supports many schedulers. We recommend fitting with one cycle training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52ddd1fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.130664</td>\n",
       "      <td>0.049394</td>\n",
       "      <td>0.984200</td>\n",
       "      <td>01:16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(epochs, lr)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22015776",
   "metadata": {},
   "source": [
    "As you can see, migrating from pure PyTorch allows you to remove a lot of code, and doesn't require you to change any of your existing data pipelines, optimizers, loss functions, models, etc.\n",
    "\n",
    "Once you've made this change, you can then benefit from fastai's rich set of callbacks, transforms, visualizations, and so forth.\n",
    "\n",
    "Note that fastai is much more than just a training loop (although we're only using the training loop in this example) - it is a complete framework including GPU-accelerated transformations, end-to-end inference, integrated applications for vision, text, tabular, and collaborative filtering, and so forth. You can use any part of the framework on its own, or combine them together, as described in the [fastai paper](https://arxiv.org/abs/2002.04688)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6253f7dd",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true
  },
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}