![](\"https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/pytorch-logo-dark.svg\"){kind=logo}
![](\"https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/docs/source/_static/images/logo.svg\"){kind=logo}
\n",
"\n",
"# Model Zoo -- Basic Convolutional Network With Custom Weight Initialization (He) Trained on MNIST"
]
},
{
"cell_type": "markdown",
"id": "7ac40ea0",
"metadata": {},
"source": [
"## General settings and hyperparameters"
]
},
{
"cell_type": "markdown",
"id": "482e3be0",
"metadata": {},
"source": [
"- Here, we specify some general hyperparameter values and general settings\n",
"- Note that for small datatsets, it is not necessary and better not to use multiple workers as it can sometimes cause issues with too many open files in PyTorch. So, if you have problems with the data loader later, try setting `NUM_WORKERS = 0` instead."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "0f7e034b",
"metadata": {},
"outputs": [],
"source": [
"BATCH_SIZE = 128\n",
"NUM_EPOCHS = 20\n",
"LEARNING_RATE = 0.005\n",
"NUM_WORKERS = 4"
]
},
{
"cell_type": "markdown",
"id": "12555a5d",
"metadata": {},
"source": [
"## Implementing a Neural Network using PyTorch Lightning's `LightningModule`"
]
},
{
"cell_type": "markdown",
"id": "e4a5614b",
"metadata": {},
"source": [
"- In this section, we set up the main model architecture using the `LightningModule` from PyTorch Lightning.\n",
"- We start with defining our neural network model in pure PyTorch, and then we use it in the `LightningModule` to get all the extra benefits that PyTorch Lightning provides."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "9b73b854",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torch.nn.functional as F\n",
"\n",
"\n",
"# Regular PyTorch Module\n",
"class PyTorchCNN(torch.nn.Module):\n",
" def __init__(self, num_classes):\n",
" super().__init__()\n",
"\n",
" # Calculate \"same\" padding:\n",
" # (w - k + 2*p)/s + 1 = o\n",
" # => p = (s(o-1) - w + k)/2\n",
" \n",
" self.features = torch.nn.Sequential(\n",
" # 28x28x1 => 28x28x8\n",
" torch.nn.Conv2d(\n",
" in_channels=1,\n",
" out_channels=8,\n",
" kernel_size=(3, 3),\n",
" stride=(1, 1),\n",
" padding=1), # (1(28-1) - 28 + 3) / 2 = 1\n",
" \n",
" # 28x28x8 => 14x14x8\n",
" torch.nn.MaxPool2d(\n",
" kernel_size=(2, 2),\n",
" stride=(2, 2),\n",
" padding=0), # (2(14-1) - 28 + 2) = 0\n",
" \n",
" # 14x14x8 => 14x14x16\n",
" torch.nn.Conv2d(\n",
" in_channels=8,\n",
" out_channels=16,\n",
" kernel_size=(3, 3),\n",
" stride=(1, 1),\n",
" padding=1), # (1(14-1) - 14 + 3) / 2 = 1 \n",
" \n",
" # 14x14x16 => 7x7x16 \n",
" torch.nn.MaxPool2d(\n",
" kernel_size=(2, 2),\n",
" stride=(2, 2),\n",
" padding=0) # (2(7-1) - 14 + 2) = 0\n",
" )\n",
"\n",
" self.output_layer = torch.nn.Linear(7*7*16, num_classes)\n",
" \n",
" ###############################################\n",
" # Reinitialize weights using He initialization\n",
" ###############################################\n",
" for m in self.modules():\n",
" if isinstance(m, torch.nn.Conv2d):\n",
" torch.nn.init.kaiming_normal_(m.weight.detach())\n",
" m.bias.detach().zero_()\n",
" elif isinstance(m, torch.nn.Linear):\n",
" torch.nn.init.kaiming_normal_(m.weight.detach())\n",
" m.bias.detach().zero_()\n",
"\n",
" def forward(self, x):\n",
" x = self.features(x)\n",
" x = torch.flatten(x, start_dim=1)\n",
" x = self.output_layer(x)\n",
" return x # x are the model's logits"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "87f85fa6",
"metadata": {},
"outputs": [],
"source": [
"import pytorch_lightning as pl\n",
"import torchmetrics\n",
"\n",
"\n",
"# LightningModule that receives a PyTorch model as input\n",
"class LightningCNN(pl.LightningModule):\n",
" def __init__(self, model, learning_rate):\n",
" super().__init__()\n",
"\n",
" self.learning_rate = learning_rate\n",
" # The inherited PyTorch module\n",
" self.model = model\n",
"\n",
" # Save settings and hyperparameters to the log directory\n",
" # but skip the model parameters\n",
" self.save_hyperparameters(ignore=['model'])\n",
"\n",
" # Set up attributes for computing the accuracy\n",
" self.train_acc = torchmetrics.Accuracy()\n",
" self.valid_acc = torchmetrics.Accuracy()\n",
" self.test_acc = torchmetrics.Accuracy()\n",
" \n",
" # Defining the forward method is only necessary \n",
" # if you want to use a Trainer's .predict() method (optional)\n",
" def forward(self, x):\n",
" return self.model(x)\n",
" \n",
" # A common forward step to compute the loss and labels\n",
" # this is used for training, validation, and testing below\n",
" def _shared_step(self, batch):\n",
" features, true_labels = batch\n",
" logits = self(features)\n",
" loss = torch.nn.functional.cross_entropy(logits, true_labels)\n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
"\n",
" return loss, true_labels, predicted_labels\n",
"\n",
" def training_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"train_loss\", loss)\n",
" self.train_acc(predicted_labels, true_labels)\n",
" self.log(\"train_acc\", self.train_acc, on_epoch=True, on_step=False)\n",
" return loss # this is passed to the optimzer for training\n",
"\n",
" def validation_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"valid_loss\", loss)\n",
" self.valid_acc(predicted_labels, true_labels)\n",
" self.log(\"valid_acc\", self.valid_acc,\n",
" on_epoch=True, on_step=False, prog_bar=True)\n",
"\n",
" def test_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.test_acc(predicted_labels, true_labels)\n",
" self.log(\"test_acc\", self.test_acc, on_epoch=True, on_step=False)\n",
"\n",
" def configure_optimizers(self):\n",
" optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n",
" return optimizer"
]
},
{
"cell_type": "markdown",
"id": "0cdfc165",
"metadata": {},
"source": [
"## Setting up the dataset"
]
},
{
"cell_type": "markdown",
"id": "381f2443",
"metadata": {},
"source": [
"- In this section, we are going to set up our dataset."
]
},
{
"cell_type": "markdown",
"id": "d9b43c84",
"metadata": {},
"source": [
"### Inspecting the dataset"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f50a28e3",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"from torchvision import datasets\n",
"from torchvision import transforms\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"train_dataset = datasets.MNIST(root='./data', \n",
" train=True, \n",
" transform=transforms.ToTensor(),\n",
" download=True)\n",
"\n",
"train_loader = DataLoader(dataset=train_dataset, \n",
" batch_size=BATCH_SIZE, \n",
" num_workers=NUM_WORKERS,\n",
" drop_last=True,\n",
" shuffle=True)\n",
"\n",
"test_dataset = datasets.MNIST(root='./data', \n",
" train=False,\n",
" transform=transforms.ToTensor())\n",
"\n",
"test_loader = DataLoader(dataset=test_dataset, \n",
" batch_size=BATCH_SIZE,\n",
" num_workers=NUM_WORKERS,\n",
" drop_last=False,\n",
" shuffle=False)\n",
"\n",
"# Checking the dataset\n",
"all_train_labels = []\n",
"all_test_labels = []\n",
"\n",
"for images, labels in train_loader: \n",
" all_train_labels.append(labels)\n",
"all_train_labels = torch.cat(all_train_labels)\n",
" \n",
"for images, labels in test_loader: \n",
" all_test_labels.append(labels)\n",
"all_test_labels = torch.cat(all_test_labels)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "361b33ae",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Training label distribution: tensor([5919, 6733, 5949, 6119, 5833, 5414, 5907, 6250, 5842, 5938])\n",
"\n",
"Test labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Test label distribution: tensor([ 980, 1135, 1032, 1010, 982, 892, 958, 1028, 974, 1009])\n"
]
}
],
"source": [
"print('Training labels:', torch.unique(all_train_labels))\n",
"print('Training label distribution:', torch.bincount(all_train_labels))\n",
"\n",
"print('\\nTest labels:', torch.unique(all_test_labels))\n",
"print('Test label distribution:', torch.bincount(all_test_labels))"
]
},
{
"cell_type": "markdown",
"id": "31a45979",
"metadata": {},
"source": [
"### Performance baseline"
]
},
{
"cell_type": "markdown",
"id": "a1f29a77",
"metadata": {},
"source": [
"- Especially for imbalanced datasets, it's quite useful to compute a performance baseline.\n",
"- In classification contexts, a useful baseline is to compute the accuracy for a scenario where the model always predicts the majority class -- you want your model to be better than that!"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "5ec92aba",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Baseline ACC: 11.35%\n"
]
}
],
"source": [
"majority_prediction = torch.argmax(torch.bincount(all_test_labels))\n",
"baseline_acc = torch.mean((all_test_labels == majority_prediction).float())\n",
"print(f'Baseline ACC: {baseline_acc*100:.2f}%')"
]
},
{
"cell_type": "markdown",
"id": "063ad85a",
"metadata": {},
"source": [
"### Setting up a `DataModule`"
]
},
{
"cell_type": "markdown",
"id": "42626ebd",
"metadata": {},
"source": [
"- There are three main ways we can prepare the dataset for Lightning. We can\n",
" 1. make the dataset part of the model;\n",
" 2. set up the data loaders as usual and feed them to the fit method of a Lightning Trainer -- the Trainer is introduced in the next subsection;\n",
" 3. create a LightningDataModule.\n",
"- Here, we are going to use approach 3, which is the most organized approach. The `LightningDataModule` consists of several self-explanatory methods as we can see below:\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "c068349e",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from torch.utils.data.dataset import random_split\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"class DataModule(pl.LightningDataModule):\n",
" def __init__(self, data_path='./'):\n",
" super().__init__()\n",
" self.data_path = data_path\n",
" \n",
" def prepare_data(self):\n",
" datasets.MNIST(root=self.data_path,\n",
" download=True)\n",
" return\n",
"\n",
" def setup(self, stage=None):\n",
" # Note transforms.ToTensor() scales input images\n",
" # to 0-1 range\n",
" train = datasets.MNIST(root=self.data_path, \n",
" train=True, \n",
" transform=transforms.ToTensor(),\n",
" download=False)\n",
"\n",
" self.test = datasets.MNIST(root=self.data_path, \n",
" train=False, \n",
" transform=transforms.ToTensor(),\n",
" download=False)\n",
"\n",
" self.train, self.valid = random_split(train, lengths=[55000, 5000])\n",
"\n",
" def train_dataloader(self):\n",
" train_loader = DataLoader(dataset=self.train, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=True,\n",
" shuffle=True,\n",
" num_workers=NUM_WORKERS)\n",
" return train_loader\n",
"\n",
" def val_dataloader(self):\n",
" valid_loader = DataLoader(dataset=self.valid, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return valid_loader\n",
"\n",
" def test_dataloader(self):\n",
" test_loader = DataLoader(dataset=self.test, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return test_loader"
]
},
{
"cell_type": "markdown",
"id": "4b8b5862",
"metadata": {},
"source": [
"- Note that the `prepare_data` method is usually used for steps that only need to be executed once, for example, downloading the dataset; the `setup` method defines the the dataset loading -- if you run your code in a distributed setting, this will be called on each node / GPU. \n",
"- Next, lets initialize the `DataModule`; we use a random seed for reproducibility (so that the data set is shuffled the same way when we re-execute this code):"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "5cf11291",
"metadata": {},
"outputs": [],
"source": [
"torch.manual_seed(1) \n",
"data_module = DataModule(data_path='./data')"
]
},
{
"cell_type": "markdown",
"id": "6c2f8da1",
"metadata": {},
"source": [
"## Training the model using the PyTorch Lightning Trainer class"
]
},
{
"cell_type": "markdown",
"id": "e45c3c4a",
"metadata": {},
"source": [
"- Next, we initialize our model.\n",
"- Also, we define a call back so that we can obtain the model with the best validation set performance after training.\n",
"- PyTorch Lightning offers [many advanced logging services](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) like Weights & Biases. Here, we will keep things simple and use the `CSVLogger`:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "1ba71aa8",
"metadata": {},
"outputs": [],
"source": [
"from pytorch_lightning.callbacks import ModelCheckpoint\n",
"from pytorch_lightning.loggers import CSVLogger\n",
"\n",
"\n",
"pytorch_model = PyTorchCNN(num_classes=10)\n",
"\n",
"lightning_model = LightningCNN(\n",
" pytorch_model, learning_rate=LEARNING_RATE)\n",
"\n",
"callbacks = [ModelCheckpoint(\n",
" save_top_k=1, mode='max', monitor=\"valid_acc\")] # save top 1 model \n",
"logger = CSVLogger(save_dir=\"logs/\", name=\"my-model\")"
]
},
{
"cell_type": "markdown",
"id": "59445fc4",
"metadata": {},
"source": [
"- Now it's time to train our model:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "1d7904fb",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GPU available: True, used: True\n",
"TPU available: False, using: 0 TPU cores\n",
"IPU available: False, using: 0 IPUs\n",
"LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
"\n",
" | Name | Type | Params\n",
"-----------------------------------------\n",
"0 | model | PyTorchCNN | 9.1 K \n",
"1 | train_acc | Accuracy | 0 \n",
"2 | valid_acc | Accuracy | 0 \n",
"3 | test_acc | Accuracy | 0 \n",
"-----------------------------------------\n",
"9.1 K Trainable params\n",
"0 Non-trainable params\n",
"9.1 K Total params\n",
"0.036 Total estimated model params size (MB)\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validation sanity check: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c681a1170051487b8975985933d57f9a",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training took 1.89 min in total.\n"
]
}
],
"source": [
"import time\n",
"\n",
"\n",
"trainer = pl.Trainer(\n",
" max_epochs=NUM_EPOCHS,\n",
" callbacks=callbacks,\n",
" progress_bar_refresh_rate=50, # recommended for notebooks\n",
" accelerator=\"auto\", # Uses GPUs or TPUs if available\n",
" devices=\"auto\", # Uses all available GPUs/TPUs if applicable\n",
" logger=logger,\n",
" deterministic=True,\n",
" log_every_n_steps=100)\n",
"\n",
"start_time = time.time()\n",
"trainer.fit(model=lightning_model, datamodule=data_module)\n",
"\n",
"runtime = (time.time() - start_time)/60\n",
"print(f\"Training took {runtime:.2f} min in total.\")"
]
},
{
"cell_type": "markdown",
"id": "2d25c4ce",
"metadata": {},
"source": [
"## Evaluating the model"
]
},
{
"cell_type": "markdown",
"id": "54e93b6c",
"metadata": {},
"source": [
"- After training, let's plot our training ACC and validation ACC using pandas, which, in turn, uses matplotlib for plotting (you may want to consider a [more advanced logger](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) that does that for you):"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "cff635af",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃ Test metric ┃ DataLoader 0 ┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│ test_acc │ 0.9797000288963318 │\n",
"└───────────────────────────┴───────────────────────────┘\n",
"\n"
],
"text/plain": [
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.9797000288963318 \u001b[0m\u001b[35m \u001b[0m│\n",
"└───────────────────────────┴───────────────────────────┘\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"[{'test_acc': 0.9797000288963318}]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trainer.test(model=lightning_model, datamodule=data_module, ckpt_path='best')"
]
},
{
"cell_type": "markdown",
"id": "048fb8ed",
"metadata": {},
"source": [
"## Predicting labels of new data"
]
},
{
"cell_type": "markdown",
"id": "0a469bbd",
"metadata": {},
"source": [
"- You can use the `trainer.predict` method on a new `DataLoader` or `DataModule` to apply the model to new data.\n",
"- Alternatively, you can also manually load the best model from a checkpoint as shown below:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "4493c25d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"logs/my-model/version_14/checkpoints/epoch=17-step=7721.ckpt\n"
]
}
],
"source": [
"path = trainer.checkpoint_callback.best_model_path\n",
"print(path)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "687cd03a",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"3:23: E703 statement ends with a semicolon\n"
]
}
],
"source": [
"pytorch_model = PyTorchCNN(num_classes=10)\n",
"lightning_model = LightningCNN.load_from_checkpoint(path, model=pytorch_model)\n",
"lightning_model.eval();"
]
},
{
"cell_type": "markdown",
"id": "b180bb27",
"metadata": {},
"source": [
"- Note that our PyTorch model, which is passed to the Lightning model requires input arguments. However, this is automatically being taken care of since we used `self.save_hyperparameters()` in our PyTorch model's `__init__` method.\n",
"- Now, below is an example applying the model manually. Here, pretend that the `test_dataloader` is a new data loader."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "74c6c831",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([7, 2, 1, 0, 4])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_dataloader = data_module.test_dataloader()\n",
"\n",
"all_true_labels = []\n",
"all_predicted_labels = []\n",
"for batch in test_dataloader:\n",
" features, labels = batch\n",
" \n",
" with torch.no_grad(): # since we don't need to backprop\n",
" logits = lightning_model(features)\n",
" \n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
" all_predicted_labels.append(predicted_labels)\n",
" all_true_labels.append(labels)\n",
" \n",
"all_predicted_labels = torch.cat(all_predicted_labels)\n",
"all_true_labels = torch.cat(all_true_labels)\n",
"all_predicted_labels[:5]"
]
},
{
"cell_type": "markdown",
"id": "324643e8",
"metadata": {},
"source": [
"Just as an internal check, if the model was loaded correctly, the test accuracy below should be identical to the test accuracy we saw earlier in the previous section."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "ed5e1d1d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test accuracy: 0.9797 (97.97%)\n"
]
}
],
"source": [
"test_acc = torch.mean((all_predicted_labels == all_true_labels).float())\n",
"print(f'Test accuracy: {test_acc:.4f} ({test_acc*100:.2f}%)')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}