mirrors / rasbt--deeplearning-models

项目文件夹

rasbt--deeplearning-models
文件
test
T
Sebastian Raschka 3a1265474c lower refresh rate to avoid crashing notebooks
2022-02-18 15:38:01 -08:00

1175 行
80 KiB
Plaintext
原始文件 永久链接 Blame 文件历史

{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "f115186a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch : 1.10.1\n",
"pytorch_lightning: 1.6.0.dev0\n",
"torchmetrics : 0.6.2\n",
"matplotlib : 3.3.4\n",
"\n"
]
}
],
"source": [
"%load_ext watermark\n",
"%watermark -p torch,pytorch_lightning,torchmetrics,matplotlib"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "c3704577",
"metadata": {},
"outputs": [],
"source": [
"%load_ext pycodestyle_magic\n",
"%flake8_on --ignore W291,W293,E703"
]
},
{
"cell_type": "markdown",
"id": "386be16a",
"metadata": {},
"source": [
"<a href=\"https://pytorch.org\"><img src=\"https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/pytorch-logo-dark.svg\" width=\"90\"/></a> &nbsp; &nbsp;&nbsp;&nbsp;<a href=\"https://www.pytorchlightning.ai\"><img src=\"https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/docs/source/_static/images/logo.svg\" width=\"150\"/></a>\n",
"\n",
"# Model Zoo -- Basic Convolutional Network Trained on MNIST"
]
},
{
"cell_type": "markdown",
"id": "51feb31c",
"metadata": {},
"source": [
"## General settings and hyperparameters"
]
},
{
"cell_type": "markdown",
"id": "b4a3c0af",
"metadata": {},
"source": [
"- Here, we specify some general hyperparameter values and general settings\n",
"- Note that for small datatsets, it is not necessary and better not to use multiple workers as it can sometimes cause issues with too many open files in PyTorch. So, if you have problems with the data loader later, try setting `NUM_WORKERS = 0` instead."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "07ef6ebf",
"metadata": {},
"outputs": [],
"source": [
"BATCH_SIZE = 128\n",
"NUM_EPOCHS = 20\n",
"LEARNING_RATE = 0.005\n",
"NUM_WORKERS = 4"
]
},
{
"cell_type": "markdown",
"id": "542e29a6",
"metadata": {},
"source": [
"## Implementing a Neural Network using PyTorch Lightning's `LightningModule`"
]
},
{
"cell_type": "markdown",
"id": "d3b1c087",
"metadata": {},
"source": [
"- In this section, we set up the main model architecture using the `LightningModule` from PyTorch Lightning.\n",
"- We start with defining our neural network model in pure PyTorch, and then we use it in the `LightningModule` to get all the extra benefits that PyTorch Lightning provides."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "8fc60e06",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torch.nn.functional as F\n",
"\n",
"\n",
"# Regular PyTorch Module\n",
"class PyTorchCNN(torch.nn.Module):\n",
" def __init__(self, num_classes):\n",
" super().__init__()\n",
"\n",
" self.num_classes = num_classes\n",
" \n",
" # Calculate \"same\" padding:\n",
" # (w - k + 2*p)/s + 1 = o\n",
" # => p = (s(o-1) - w + k)/2\n",
" \n",
" self.features = torch.nn.Sequential(\n",
" # 28x28x1 => 28x28x8\n",
" torch.nn.Conv2d(\n",
" in_channels=1,\n",
" out_channels=8,\n",
" kernel_size=(3, 3),\n",
" stride=(1, 1),\n",
" padding=1), # (1(28-1) - 28 + 3) / 2 = 1\n",
" \n",
" # 28x28x8 => 14x14x8\n",
" torch.nn.MaxPool2d(\n",
" kernel_size=(2, 2),\n",
" stride=(2, 2),\n",
" padding=0), # (2(14-1) - 28 + 2) = 0\n",
" \n",
" # 14x14x8 => 14x14x16\n",
" torch.nn.Conv2d(\n",
" in_channels=8,\n",
" out_channels=16,\n",
" kernel_size=(3, 3),\n",
" stride=(1, 1),\n",
" padding=1), # (1(14-1) - 14 + 3) / 2 = 1 \n",
" \n",
" # 14x14x16 => 7x7x16 \n",
" torch.nn.MaxPool2d(\n",
" kernel_size=(2, 2),\n",
" stride=(2, 2),\n",
" padding=0) # (2(7-1) - 14 + 2) = 0\n",
" )\n",
"\n",
" self.output_layer = torch.nn.Linear(7*7*16, num_classes)\n",
"\n",
" def forward(self, x):\n",
" x = self.features(x)\n",
" x = torch.flatten(x, start_dim=1)\n",
" x = self.output_layer(x)\n",
" return x # x are the model's logits"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "cca100e8",
"metadata": {},
"outputs": [],
"source": [
"import pytorch_lightning as pl\n",
"import torchmetrics\n",
"\n",
"\n",
"# LightningModule that receives a PyTorch model as input\n",
"class LightningCNN(pl.LightningModule):\n",
" def __init__(self, model, learning_rate):\n",
" super().__init__()\n",
"\n",
" self.learning_rate = learning_rate\n",
" # The inherited PyTorch module\n",
" self.model = model\n",
"\n",
" # Save settings and hyperparameters to the log directory\n",
" # but skip the model parameters\n",
" self.save_hyperparameters(ignore=['model'])\n",
"\n",
" # Set up attributes for computing the accuracy\n",
" self.train_acc = torchmetrics.Accuracy()\n",
" self.valid_acc = torchmetrics.Accuracy()\n",
" self.test_acc = torchmetrics.Accuracy()\n",
" \n",
" # Defining the forward method is only necessary \n",
" # if you want to use a Trainer's .predict() method (optional)\n",
" def forward(self, x):\n",
" return self.model(x)\n",
" \n",
" # A common forward step to compute the loss and labels\n",
" # this is used for training, validation, and testing below\n",
" def _shared_step(self, batch):\n",
" features, true_labels = batch\n",
" logits = self(features)\n",
" loss = torch.nn.functional.cross_entropy(logits, true_labels)\n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
"\n",
" return loss, true_labels, predicted_labels\n",
"\n",
" def training_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"train_loss\", loss)\n",
" self.train_acc(predicted_labels, true_labels)\n",
" self.log(\"train_acc\", self.train_acc, on_epoch=True, on_step=False)\n",
" return loss # this is passed to the optimzer for training\n",
"\n",
" def validation_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.log(\"valid_loss\", loss)\n",
" self.valid_acc(predicted_labels, true_labels)\n",
" self.log(\"valid_acc\", self.valid_acc,\n",
" on_epoch=True, on_step=False, prog_bar=True)\n",
"\n",
" def test_step(self, batch, batch_idx):\n",
" loss, true_labels, predicted_labels = self._shared_step(batch)\n",
" self.test_acc(predicted_labels, true_labels)\n",
" self.log(\"test_acc\", self.test_acc, on_epoch=True, on_step=False)\n",
"\n",
" def configure_optimizers(self):\n",
" optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n",
" return optimizer"
]
},
{
"cell_type": "markdown",
"id": "dd15cc14",
"metadata": {},
"source": [
"## Setting up the dataset"
]
},
{
"cell_type": "markdown",
"id": "279535f4",
"metadata": {},
"source": [
"- In this section, we are going to set up our dataset."
]
},
{
"cell_type": "markdown",
"id": "4ac5f526",
"metadata": {},
"source": [
"### Inspecting the dataset"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "57c55ed6",
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"from torchvision import datasets\n",
"from torchvision import transforms\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"train_dataset = datasets.MNIST(root='./data', \n",
" train=True, \n",
" transform=transforms.ToTensor(),\n",
" download=True)\n",
"\n",
"train_loader = DataLoader(dataset=train_dataset, \n",
" batch_size=BATCH_SIZE, \n",
" num_workers=NUM_WORKERS,\n",
" drop_last=True,\n",
" shuffle=True)\n",
"\n",
"test_dataset = datasets.MNIST(root='./data', \n",
" train=False,\n",
" transform=transforms.ToTensor())\n",
"\n",
"test_loader = DataLoader(dataset=test_dataset, \n",
" batch_size=BATCH_SIZE,\n",
" num_workers=NUM_WORKERS,\n",
" drop_last=False,\n",
" shuffle=False)\n",
"\n",
"# Checking the dataset\n",
"all_train_labels = []\n",
"all_test_labels = []\n",
"\n",
"for images, labels in train_loader: \n",
" all_train_labels.append(labels)\n",
"all_train_labels = torch.cat(all_train_labels)\n",
" \n",
"for images, labels in test_loader: \n",
" all_test_labels.append(labels)\n",
"all_test_labels = torch.cat(all_test_labels)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "900a9a2c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Training label distribution: tensor([5911, 6731, 5949, 6120, 5834, 5412, 5911, 6253, 5844, 5939])\n",
"\n",
"Test labels: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
"Test label distribution: tensor([ 980, 1135, 1032, 1010, 982, 892, 958, 1028, 974, 1009])\n"
]
}
],
"source": [
"print('Training labels:', torch.unique(all_train_labels))\n",
"print('Training label distribution:', torch.bincount(all_train_labels))\n",
"\n",
"print('\\nTest labels:', torch.unique(all_test_labels))\n",
"print('Test label distribution:', torch.bincount(all_test_labels))"
]
},
{
"cell_type": "markdown",
"id": "ebc44192",
"metadata": {},
"source": [
"### Performance baseline"
]
},
{
"cell_type": "markdown",
"id": "8530dcc7",
"metadata": {},
"source": [
"- Especially for imbalanced datasets, it's quite useful to compute a performance baseline.\n",
"- In classification contexts, a useful baseline is to compute the accuracy for a scenario where the model always predicts the majority class -- you want your model to be better than that!"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "86580b1d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Baseline ACC: 11.35%\n"
]
}
],
"source": [
"majority_prediction = torch.argmax(torch.bincount(all_test_labels))\n",
"baseline_acc = torch.mean((all_test_labels == majority_prediction).float())\n",
"print(f'Baseline ACC: {baseline_acc*100:.2f}%')"
]
},
{
"cell_type": "markdown",
"id": "fdabdc42",
"metadata": {},
"source": [
"### Setting up a `DataModule`"
]
},
{
"cell_type": "markdown",
"id": "784cf131",
"metadata": {},
"source": [
"- There are three main ways we can prepare the dataset for Lightning. We can\n",
" 1. make the dataset part of the model;\n",
" 2. set up the data loaders as usual and feed them to the fit method of a Lightning Trainer -- the Trainer is introduced in the next subsection;\n",
" 3. create a LightningDataModule.\n",
"- Here, we are going to use approach 3, which is the most organized approach. The `LightningDataModule` consists of several self-explanatory methods as we can see below:\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "d7feb0a5",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from torch.utils.data.dataset import random_split\n",
"from torch.utils.data import DataLoader\n",
"\n",
"\n",
"class DataModule(pl.LightningDataModule):\n",
" def __init__(self, data_path='./'):\n",
" super().__init__()\n",
" self.data_path = data_path\n",
" \n",
" def prepare_data(self):\n",
" datasets.MNIST(root=self.data_path,\n",
" download=True)\n",
" return\n",
"\n",
" def setup(self, stage=None):\n",
" # Note transforms.ToTensor() scales input images\n",
" # to 0-1 range\n",
" train = datasets.MNIST(root=self.data_path, \n",
" train=True, \n",
" transform=transforms.ToTensor(),\n",
" download=False)\n",
"\n",
" self.test = datasets.MNIST(root=self.data_path, \n",
" train=False, \n",
" transform=transforms.ToTensor(),\n",
" download=False)\n",
"\n",
" self.train, self.valid = random_split(train, lengths=[55000, 5000])\n",
"\n",
" def train_dataloader(self):\n",
" train_loader = DataLoader(dataset=self.train, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=True,\n",
" shuffle=True,\n",
" num_workers=NUM_WORKERS)\n",
" return train_loader\n",
"\n",
" def val_dataloader(self):\n",
" valid_loader = DataLoader(dataset=self.valid, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return valid_loader\n",
"\n",
" def test_dataloader(self):\n",
" test_loader = DataLoader(dataset=self.test, \n",
" batch_size=BATCH_SIZE, \n",
" drop_last=False,\n",
" shuffle=False,\n",
" num_workers=NUM_WORKERS)\n",
" return test_loader"
]
},
{
"cell_type": "markdown",
"id": "cc0115cb",
"metadata": {},
"source": [
"- Note that the `prepare_data` method is usually used for steps that only need to be executed once, for example, downloading the dataset; the `setup` method defines the the dataset loading -- if you run your code in a distributed setting, this will be called on each node / GPU. \n",
"- Next, lets initialize the `DataModule`; we use a random seed for reproducibility (so that the data set is shuffled the same way when we re-execute this code):"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "070ce880",
"metadata": {},
"outputs": [],
"source": [
"torch.manual_seed(1) \n",
"data_module = DataModule(data_path='./data')"
]
},
{
"cell_type": "markdown",
"id": "1b562ab6",
"metadata": {},
"source": [
"## Training the model using the PyTorch Lightning Trainer class"
]
},
{
"cell_type": "markdown",
"id": "4a234512",
"metadata": {},
"source": [
"- Next, we initialize our model.\n",
"- Also, we define a call back so that we can obtain the model with the best validation set performance after training.\n",
"- PyTorch Lightning offers [many advanced logging services](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) like Weights & Biases. Here, we will keep things simple and use the `CSVLogger`:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "47b86b85",
"metadata": {},
"outputs": [],
"source": [
"from pytorch_lightning.callbacks import ModelCheckpoint\n",
"from pytorch_lightning.loggers import CSVLogger\n",
"\n",
"\n",
"pytorch_model = PyTorchCNN(num_classes=10)\n",
"\n",
"lightning_model = LightningCNN(\n",
" pytorch_model, learning_rate=LEARNING_RATE)\n",
"\n",
"callbacks = [ModelCheckpoint(\n",
" save_top_k=1, mode='max', monitor=\"valid_acc\")] # save top 1 model \n",
"logger = CSVLogger(save_dir=\"logs/\", name=\"my-model\")"
]
},
{
"cell_type": "markdown",
"id": "8b90016d",
"metadata": {},
"source": [
"- Now it's time to train our model:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "1798f3c1",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"GPU available: True, used: True\n",
"TPU available: False, using: 0 TPU cores\n",
"IPU available: False, using: 0 IPUs\n",
"LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
"\n",
" | Name | Type | Params\n",
"-----------------------------------------\n",
"0 | model | PyTorchCNN | 9.1 K \n",
"1 | train_acc | Accuracy | 0 \n",
"2 | valid_acc | Accuracy | 0 \n",
"3 | test_acc | Accuracy | 0 \n",
"-----------------------------------------\n",
"9.1 K Trainable params\n",
"0 Non-trainable params\n",
"9.1 K Total params\n",
"0.036 Total estimated model params size (MB)\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validation sanity check: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "038cac69150141329102d77ade875cdf",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Training: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Validating: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training took 1.88 min in total.\n"
]
}
],
"source": [
"import time\n",
"\n",
"\n",
"trainer = pl.Trainer(\n",
" max_epochs=NUM_EPOCHS,\n",
" callbacks=callbacks,\n",
" progress_bar_refresh_rate=50, # recommended for notebooks\n",
" accelerator=\"auto\", # Uses GPUs or TPUs if available\n",
" devices=\"auto\", # Uses all available GPUs/TPUs if applicable\n",
" logger=logger,\n",
" deterministic=True,\n",
" log_every_n_steps=100)\n",
"\n",
"start_time = time.time()\n",
"trainer.fit(model=lightning_model, datamodule=data_module)\n",
"\n",
"runtime = (time.time() - start_time)/60\n",
"print(f\"Training took {runtime:.2f} min in total.\")"
]
},
{
"cell_type": "markdown",
"id": "37de19e0",
"metadata": {},
"source": [
"## Evaluating the model"
]
},
{
"cell_type": "markdown",
"id": "02391244",
"metadata": {},
"source": [
"- After training, let's plot our training ACC and validation ACC using pandas, which, in turn, uses matplotlib for plotting (you may want to consider a [more advanced logger](https://pytorch-lightning.readthedocs.io/en/latest/extensions/logging.html) that does that for you):"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "4532580f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<AxesSubplot:xlabel='Epoch', ylabel='ACC'>"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABN4klEQVR4nO3dd3hUZfrw8e+T3gNJIIQEktB7DQmCQBCVIkUBERAVFbFhZy277+7qruu6/lx13UUQFFEsiCBVFBWJqPSW0CH0EEogEBKSkDLP+8cZMIRJnzMzkPtzXXNlMqfMPSeTuefpSmuNEEIIUZqbswMQQgjhmiRBCCGEsEkShBBCCJskQQghhLBJEoQQQgibPJwdgD2FhYXpmJiYah174cIF/P397RuQHUl8NSPx1YzEVzOuHN+mTZtOa63r2dyotb5ubl27dtXVtXLlymof6wgSX81IfDUj8dWMK8cHbNRlfKZKFZMQQgibJEEIIYSwSRKEEEIIm66rRmohxPWlsLCQtLQ08vPzy90vODiYXbt2OSiqqnOF+Hx8fIiKisLT07PSx0iCEEK4rLS0NAIDA4mJiUEpVeZ+2dnZBAYGOjCyqnF2fFprzpw5Q1paGrGxsZU+TqqYhBAuKz8/n9DQ0HKTg6iYUorQ0NAKS2KlSYIQQrg0SQ72UZ3rWOsTRLFFM2VlKtsyipwdihBCuJRa3wbh7qaYvuoAXcNkXQwhhCip1pcgAGJC/TiZa3F2GEIIF3Pu3Dnee++9Kh83aNAgzp07V+Xjxo8fz7x586p8nFkkQQAxYf6czJUShBDiSmUliOLi4nKPW7ZsGXXq1DEpKsep9VVMADGh/izO01wsKsbbw93Z4QghbHhlyQ52pp+3ua24uBh396r/77ZpGMRfh7Qtc/uLL77I/v376dSpE56engQEBBAREcHWrVvZuXMnt99+O0ePHiU/P5+nnnqKiRMnAhATE8PGjRvJyclh4MCBJCQksGHDBiIjI1m0aBG+vr4VxrZixQomT55MUVER3bp1Y+rUqXh7e/Piiy+yePFiPDw8uPXWW3nzzTf56quveOWVV3B3dyc4OJhVq1ZV+VrYIgkCiAnzQwNHM/NoVj/A2eEIIVzE66+/zvbt29m6dStJSUncdtttbN++/fJYgpkzZxISEkJeXh7dunVjxIgRhIaGXnGOffv28cEHHzBr1ixGjRrF/PnzGTduXLnPm5+fz/jx41mxYgUtWrTg3nvvZerUqdx7770sWLCA3bt3o5S6XI31t7/9jeXLlxMZGVmtqq2ySILAKEEAHDp9QRKEEC6qvG/6jhqIFh8ff8VAs3fffZcFCxYAcPToUfbt23dVgoiNjaVDhw4AdO3alUOHDlX4PHv27CE2NpYWLVoAcN999zFlyhQmTZqEj48PEyZM4LbbbmPw4MEA9OzZk/HjxzNq1CiGDx9uj5cKSBsEUCJBnLng5EiEEK6s5JoOSUlJ/Pjjj6xZs4bk5GQ6d+5scyCat7f35fvu7u4UFVXcpd6YhftqHh4erF+/nhEjRrBw4UIGDBgAwLRp03j11Vc5evQonTp14syZM1V9abafzy5nucbV9ffC31MShBDiSoGBgWRnZ9vclpWVRd26dfHz82P37t2sXbvWbs/bqlUrDh06RGpqKs2aNWP27Nn06dOHnJwccnNzGTRoEN27d6dZs2YA7N+/n4SEBBISEliyZAlHjx69qiRTHZIgrOr7uXH4TK6zwxBCuJDQ0FB69uxJu3bt8PX1JTw8/PK2AQMGMG3aNDp06EDLli3p3r273Z7Xx8eHjz76iDvvvPNyI/UjjzxCZmYmw4YNIz8/H601b7/9NgB/+MMf2LdvH1pr+vXrR8eOHe0ShyQIq3A/xcHTUoIQQlzp888/t/m4t7c33377rc1tl9oZwsLC2L59++VSyOTJk8t9rlmzZl2+369fP7Zs2XLF9oiICNavX3/VcV9//XW5560uaYOwCvdzI/1cHheLyu/fLIQQtYUkCKtwfzcs2ujqKoQQZnr88cfp1KnTFbePPvrI2WFdRaqYrOr7GTMdSldXIYTZpkyZ4uwQKsXUEoRSaoBSao9SKlUp9aKN7a2UUmuUUheVUldVziml3JVSW5RSS82ME6CBn3EppCeTEEIYTEsQSil3YAowEGgDjFFKtSm1WybwJPBmGad5CnDIOn3+nhDk4yEJQgghrMwsQcQDqVrrA1rrAmAOMKzkDlrrU1rrDUBh6YOVUlHAbcAHJsZY8vmIDfOXrq5CCGFlZoKIBI6W+D3N+lhlvQM8DzhsHu6YMH/p6iqEEFZmNlLbWt+uUnNqK6UGA6e01puUUokV7DsRmAgQHh5OUlJS1aK0ysnJgewCjp0t5IefVuLp5lrLHObk5FT7tTmCxFczEp9twcHBZY5kLqm4uLhS+5ktIiKC48ePc/z4cZ5//nlmz54NXBnfoEGDePXVV+nSpYvNc7Rr146ff/7ZLiOhS8vPz6/S39HMBJEGNCrxexSQXsljewJDlVKDAB8gSCn1qdb6qikQtdbTgekAcXFxOjExsVrBJiUlkRjXjEX7k4lt183lejIlJSVR3dfmCBJfzUh8tu3atatSk/A5arK+yggMDCQwMJCFCxdefqxkfO7u7vj7+5cZr1KKgIAAU16Pj48PnTt3rvT+ZiaIDUBzpVQscAwYDYytzIFa65eAlwCsJYjJtpKDvUVbJ+07fEa6ugrhcr59EU5ss7nJt7gI3KvxcdagPQx8vczNL7zwAtHR0Tz22GMAvPzyyyilWLVqFWfPnqWwsJBXX32VYcOuaF7l0KFDDB48mO3bt5OXl8f48ePZt28frVu3Ji+v8mOt3nrrLWbOnAnAhAkTePrpp7lw4QKjRo0iLS2N4uJi/vznP3PXXXfZXCeipkxLEFrrIqXUJGA54A7M1FrvUEo9Yt0+TSnVANgIBAEWpdTTQButte1VQUwWa00Q0g4hhAAYPXo0Tz/99OUEMXfuXL777jueeeYZgoKCOH36NN27d2fo0KEoZbtaeurUqfj5+ZGSkkJKSkqZVUulbdq0iY8++oh169ahtSYhIYE+ffpw4MABGjZsyDfffAMYkwZmZmbaXCeipkwdKKe1XgYsK/XYtBL3T2BUPZV3jiQgyYTwrlLHz5MgHw/pySSEKyrnm36eSVVMnTt35tSpU6Snp5ORkUHdunWJiIjgmWeeYdWqVbi5uXHs2DFOnjxJgwYNbJ5j1apVTJgwAYAOHTpcXhuiIr/++it33HHH5SnGhw8fzi+//MKAAQOYPHkyL7zwAoMHD6ZXr14UFRXZXCeipmSqjRIudXWVsRBCiEtGjhzJvHnz+PLLLxk9ejSfffYZGRkZbNq0ia1btxIeHm5zHYiSyipdlKesNSFatGjBpk2baN++PS+99BJ/+9vfylwnoqYkQZQSHSoJQgjxu9GjRzNnzhzmzZvHyJEjycrKon79+nh6erJy5UoOHz5c7vG9e/dm7ty5AGzfvp2UlJRKPW/v3r1ZuHAhubm5XLhwgQULFtCrVy/S09Px8/Nj3LhxTJ48mc2bN5OTk0NWVhaDBg3inXfeYevWrTV92YDMxXSVmDB/lqakU1BkwctD8qcQtV3btm3Jzs4mMjKSiIgI7r77boYMGUJcXBydOnWiVatW5R7/6KOPMm7cODp06ECnTp2Ij4+v1PN26dKF8ePHX95/woQJdO7cmeXLl/OHP/wBNzc3PD09mTp1KtnZ2TbXiagpSRClxIT6YdFwJDNXejIJIQDYtu333lNhYWGsWbPG5n45OTkAxMTEsH37dgB8fX2ZNWtWpdtISq5Z/eyzz/Lss89esb1///7079//quNsrRNRU/IVuZSYsN+7ugohRG0mJYhSYqSrqxDCARISErh48eIVj82ePZv27ds7KaKrSYIopa50dRXCpWitq9ULyNWtW7fOoc9XVq+o8kgVUynS1VUI1+Hj48OZM2eq9eEmfqe15syZM/j4+FTpOClB2BAd6s+Wo2edHYYQtV5UVBRpaWlkZGSUu19+fn6VP/wcyRXi8/HxISqq3HHJV5EEYYN0dRXCNXh6ehIbG1vhfklJSVWahM7RXD2+ssinnw2XuroePSvtEEKI2ksShA2Xuroekp5MQohaTBKEDZe6uh6SnkxCiFpMEoQNl7q6SglCCFGbSYKwQSlFjHR1FULUcpIgyhAjs7oKIWo5SRBliAn149jZPAqKLM4ORQghnEISRBliwvylq6sQolaTBFGG6FDp6iqEqN0kQZQhNky6ugohajdTE4RSaoBSao9SKlUp9aKN7a2UUmuUUheVUpNLPN5IKbVSKbVLKbVDKfWUmXHaIl1dhRC1nWlzMSml3IEpwC1AGrBBKbVYa72zxG6ZwJPA7aUOLwKe01pvVkoFApuUUj+UOtZU0tVVCFHbmVmCiAdStdYHtNYFwBxgWMkdtNantNYbgMJSjx/XWm+23s8GdgGRJsZqk3R1FULUZsqsedaVUiOBAVrrCdbf7wEStNaTbOz7MpCjtX7TxrYYYBXQTmt93sb2icBEgPDw8K5z5sypVrw5OTkEBFy5BvXX+wpYsr+QGbf64eHm3AVLbMXnSiS+mpH4akbiq76+fftu0lrH2dpm5nTftj5Rq5SNlFIBwHzgaVvJAUBrPR2YDhAXF6cTExOrGKYhKSmJ0sdmBqWxeH8yse270bSec/+4tuJzJRJfzUh8NSPxmcPMKqY0oFGJ36OA9MoerJTyxEgOn2mtv7ZzbJVyqavrYalmEkLUQmYmiA1Ac6VUrFLKCxgNLK7MgcpYgPZDYJfW+i0TYyzXpa6uB09LV1chRO1jWhWT1rpIKTUJWA64AzO11juUUo9Yt09TSjUANgJBgEUp9TTQBugA3ANsU0pttZ7yj1rrZWbFa0tdP08CfTykBCGEqJVMXXLU+oG+rNRj00rcP4FR9VTar9huw7C/glxY+CgNihsDiVdsUkoRG+bPQRkLIYSohWRNak9fOLmD+paDNjdHh/qTfPScY2MSQggXIFNtKAWth1D37DbIzbxqc2yoH2lnc2VWVyFErSMJAqD1EBQW2PvdVZuiQ41ZXdNkVlchRC0jCQKgYWfyvevBriVXbYq5PGmftEMIIVyQ1lCYb8qpJUEAKEVGve6QugIuZl+xSbq6CiFc2k9/h1m3wcUcu59aEoTV6bAboPgi7Pvhiselq6sQwmX9+jb88m8IbwNe/nY/vSQIq6zgVuB/dTWTdHUVQrik9TPgx5eh3QgY/I7R4cbOJEFcotyh1W2w7/ur6vOiQ/05LAsHCSFcRfKXsGwytBgAd7wPbu6mPI0kiJJaD4WCHDiw8oqHpaurEMJl7FoKCx+FmF5w5yxw9zTtqSRBlBTTC3yCr6pmkq6uQgiXsP8nmHc/NOwMY74wBvqaSBJESR5e0GIg7FkGxb+vYSRdXYUQTndkHcy5G0Kbw91fgXeg6U8pCaK01kMg7ywc/u3yQzGhfgAckq6uQghnOJ4Mn90JgRFwzwLwC3HI00qCKK1ZP/D0g52/z0we4u9FoI+HlCCEEI6XsRdmDzdKDPcugsBwhz21JIjSPH2h+S2weylYjEZppZR1fWopQQghHOjsYfhkmNGF9d5FUKdRxcfYkSQIW1oPhZyTkLbh8kMxYf4ckrEQQojScjPh1G77nzf7hJEcCi/APQshrJn9n6MCkiBsaX4ruHvBrt+rmWKkq6sQorS8czCzP7yXANP7wqaP7TPlRW4mfHI75JyCu+dDg3Y1P2c1SIKwxScImvQ1EoTWAMRIV1chREnFhfDVeMg8AL2eg6J8WPIk/LslLHka0rdW77z55+HTEcZ5x3wBjbrZMeiqkQRRltZD4NwROJECSFdXIUQJWsO3zxuDaof8B/r9BR5dDQ/+AG2GQfIcmN4H3u8DGz/CvaiSXywL8+CL0cbnzqiPoUkfc19HBSRBlKXlIFBulwfNSVdXIcRla6fCxpnQ8ynoPM54TCloFA+3vwfP7YZBbxqljKVPc8Oa+2HJU5C+pexzFhXA3Hvh8Gpj+oyWAx3zWsphaoJQSg1QSu1RSqUqpV60sb2VUmqNUuqiUmpyVY41nX8oRPe8nCCkq6sQAoA938HyP0KrwdDvZdv7+NaB+Ifg0d/gwR/JqNfTmD9peiK839tILiWXFrAUw9cPGXPBDXkH2o80/3VUgmkJQinlDkwBBgJtgDFKqTaldssEngTerMax5ms9FDJ2Q8Ze6eoqhIAT22DeAxDRAYZPB7cKPkKVgkbd2NPqSZi8xyhVWIph6TPwZktY/ASkbTLaLnYuhFtfha7jHfFKKsXMEkQ8kKq1PqC1LgDmAMNK7qC1PqW13gAUVvVYh2g92Php7c0kXV2FqMWyT8Dno4352sZ8WfX1F3yCjVLFI7/ChBXQ7g7YNg8+uAm2fAp9XoAeT5gTezV5mHjuSOBoid/TgAQHHGs/QQ0hqptRzdR7MjGhfnyTkk5BkQUvD2m+EaLWKMiFL8ZAXibc/y0ERVT/XEpBVJxx6/8abPvKGJQb/5D94rUTMxOErdUrtL2PVUpNBCYChIeHk5SUVMmnuFJOTo7NYxt5taHpgY9Z891c8s/UxaLh6+VJNPB3bIIoKz5XIfHVjMRXM6bGpy202fl/1MvYwvZ2L3Fm7znYW7XnKj8+6wC4n3+uQZDmMDNBpAElx4VHAen2PlZrPR2YDhAXF6cTExOrHChAUlISNo/NbAzvfswNwRl4tb6ZGdvWEN60HYmt6lfreaqrzPhchMRXMxJfzZga34q/Q8ZquOXvtO/5ZLVO4erXryxmfg3eADRXSsUqpbyA0cDiCo6xx7H2FdIEwtvDriXEhBp1jrL8qBC1xNYv4Jc3ocu9Ltc+4AimlSC01kVKqUnAcsAdmKm13qGUesS6fZpSqgGwEQgCLEqpp4E2Wuvzto41K9YKtR4CSf8kRJ8l0NuDw9LVVYjr3+HVRi+jmF4w6N+mrPns6sysYkJrvQxYVuqxaSXun8CoPqrUsU7TeggkvYbas4yYsBYclK6uQlzfzuw3FuepGw13zTYWE6uFpCtOZdRvDSFNYediokP9rr8SxPnjVw7aEaI2yzsLn98FaBg7F3zrOjsip5EEURlKQZuhcOgXWgcXkXY2j8Li62RW14JcY2Tnl+MuT0woRK1VXAhz74Ozh+CuzyC0qbMjcipJEJXVeghYiogv3ECxRZN2Ns/ZEdlH8hdw4RQcSDLW4haittIalk2Ggz8bE/DF9HR2RE4nCaKyGnaBoEiaZ64EuD5GVFsssGYKRHSCeq2N+WUK850dlRDOsWYKbJoFNz4Dne92djQuQRJEZSkFrYcQnP4LfuRfH11d934Lmfuh55Mw4J9GsXrte86OSgjH270Mvv9/xvxrN/3F2dG4DEkQVdF6KKoon4He266PhurV/4PgxtB6GDTtCy1vg1VvGo3WQtQWxzbB/AnQsJMxzXZFE/DVInIlqqJxd/ALY6j3pmu/q2vaJjiyGro/Cu7W3s63/h0shbDib86NTQhHyDoGiybBBzcb03OPmQNefs6OyqWYOg7iuuPmDq1uI2HLXF49fdbZ0dTMmv+CdzB0uef3x0KbQvfH4Ld3oNsEiOrqtPCEySwWSN9szFS8a6mxkln0DRDdw1gHJazl9ftNOu8s/Po2rHsftAUSHoFek401YMQVJEFUVeuh+Gz+mOisDRQW34Kn+zX4T3T2EOxcZEwd4B145bbek42eTd8+byyfeL1+SNRGlmJjdPCuJbB7KZw/Bm4eENvH+AZ9eDVsn2/s6xsCjS8ljB4oS7FTQ7eLwjwjKfz6lrHuc4e7oO8fjcFwwiZJEFUV25sCj0BuLVpP2tk8YsOqOCe8K1g71VhONeGRq7d5B0K/v8Kix4xpiDve5fj4hP0UFcDBVbBrkdEQm3saPHyg2c3GOsot+v8+EExr48vD4dXW22+w5xsAerr7wLGev5cwIruAh7fzXldVFBdB8uew8p+QnQ7NbzXe4w3aOTsylycJoqo8vMiO7sctqT+SfCrr2ksQeWdh82xoN9JY78KWjmNgwwz48a/Q6jbwDnBsjNcai8Xo5eYqc/UU5ML+FUZJYc93cDELvAKMZNB6qJEcbP1NlYKQWON2qZvn+eNwZDUnV39FZPZh+OnvxuPu3sZ6BtYSBo0Sqr6Ajtm0ht3fGG1qp/dAZByMmAExNzo7smuGJIhq8Gw3jKD9C8nf/wu0uca+YW/8CAovQI9JZe/j5gYD34APbzHqavv92XHxXWtyMuDT4cZqYWPnOq+RM/+8sZ7xzkWQ+iMU5holg9ZDjFuTRPD0qfp5gyKg3Qj2nQ4lMjERcjPhyFqjdHF4NfzyFqz6P/D0N2Yb6DgaYno7v2ry8Gr44a+Qth5Cm8NdnxprSLtKEr9GVCpBKKX8gTyttUUp1QJoBXyrtS69VGitENi2P3kLvQg5/B1wDSWIogKjDrZJIjRoX/6+jeKh/ShY/V+jIbtujCMivLbkZMDHQ+DsQSi6CF/dB6M/B3dPx8VgKYbvXoJNH0FxAQSEGyXANkONqiB7x+IXAq0GGTcw5vA6ug52LoYdC4z2q6Ao6DAKOo2FsOb2ff6KnNwJK16Bvd9BYIQxIrrTuN976okqqWyaXwX4KKUigRXA/cAss4JydcrLn83ecTQ/m2RUL1SXxQI7FsKMfsYHsdm2z4OcE5Wf1/7ml42eW99LCeIqOafg48FGnf3YuTD4LeMb/KLHa/aeqAqLBRY/CevfNz6QH1gOz+42YmmS6JhE5R1oVFkNfRcm74WRMyG8Dfz2H/hfHMy4CdbPMEoeZoaRnwELHoWpPeDwGqON4YnN0HW8JIcaqOyVU1rrXKXUg8B/tdZvKKW2mBmYq9sX0peeJ1bDsY3Gt+2qsFhg9xJI+hec2mEUz9O3GL1GouLMCVhrY2Bc/TbQtF/ljgmOhBufhZWvGg2dsb3Nie1ak3PKWnI4DHfPNa5Lkz5w4YxxrfzCoP8/zK3OsFhg6VOw1brYfd8/mvdcleXpC+1GGLfsk0Ynh+QvjPmNvnsJWg6AjmOh+S01S14FF+D0XsjYa/25m4Q9y41qrR6TjPesX4j9XlctVukEoZS6AbgbeLCKx16Xshv3o+D467jvXIR7ZROExWJ0L/z5X3Byu1E3OvwDaNbPmFH164nwyC/mNPbt/8lIRsPeq9oHV49JsPkT4x984s/ybSz7pJEcso7C3V9BbK/ft/WebPQSWjsF/MOg17PmxKA1LHvO+Lv0eg4SXzLneWoiMNx47/SYBCe2GSuzbZtrNJz7hUL7O432iohOZb8fL5yGjD1GA/OlZHB6r3HtL1HuENKEEw1uouFd/4Y6jWyfS1RLZf/bnwZeAhZYV4VrAqw0LaprQER4OL9Z2nHjjsW43/pq+R+6WhuJIelfcHIbhDaD4TOMb1pu7sY+t79nfPD88Be47d/2D3j1fyGgAbQfWbXjPH2NEdZf3QebP4ZuD1Z8zPUq+4Q1OaQZyaF0bxiloP8/IfeMUQ/uFwpd77NvDFrDsj/AxpnQ82m46c+u3/DaoD0MaA+3vGJ8UUn+wugssW6aMUlkx9FGyfZ0iSSQsQfySlRLefoZ7RmNu0PYfVCvhTGYL6QJeHixNymJhpIc7K5SCUJr/TPwM4BSyg04rbWu3urd14nYMD/mWuLpe36G8Q0posPVO13qZvfz68Y+IU2NuV7ajbz6m3hsb+j+uPHts+VAo17XXk5shwMrjX7v1em73mYYRN8IP70K7YbXzgVUsk/ArMFwPh3unlf2VNBubkYpLe8sLH3auFZthtonBq2NktyGGXDDJKONyNWTQ0nunkZX2xb9jeuzY6GRLH786+/7+IUaH/yth0C9lsb9ei2Mhm9n94yqhSrbi+lz4BGgGNgEBCul3tJa/5+Zwbmy6FB/fijuyj89P8Rt15IrE4TWxtoKSa/DiRTjW87t04xidXlVNP3+YvRfX/g4PLbGfvWoa/5ntHN0vb96xytlzPY6vY9RChr4un3iulaUTA7j5hn9/svj4QWjPoFPhsH8B8F3fs3bb7Q2ZhtdN9WYDqWiUqur860Lcfcbt8wDRtVdWAuZ7sLFVDYlt9Fanwdux1gnujFwT7lHXOdC/b0o9A7hcEBHo14VrInhW+ODdM5Yowvg7VPh8Q3QaUzF9feePjB8ulGP/c2z9lnh7Xw6bJsHncfVLOFEdIAu98H66XBqd83julacPw6zbqt8crjEy9/o3RTSBL4YC+lbqx+D1kbV45r/QfxE6P/atZ0cSgtpYswDJcnB5VQ2QXgqpTwxEsQi6/iHCj+9lFIDlFJ7lFKpSqkXbWxXSql3rdtTlFJdSmx7Rim1Qym1XSn1hVKqGqN8zKOUIjrMj189e0DGLqMr3/RE+GI05GcZ1QyTNhp9wavSsBvR0Wh03LHA+GCvqXXvgy42Zm2tqZv+nzEid/lLtWN50vPHja6s2Sdg3PzKJ4dL/EJg3NfGPEefjoAz+6seg9bGSODV70Lcg8YAxuspOQiXVtkE8T5wCPAHVimlooHz5R2glHIHpgADgTbAGKVUm1K7DQSaW28TganWYyOBJ4E4rXU7wB0YXclYHSYm1J9F+Z2NX5ZNNupVh00xEkPnu6vf46fn0xAVb/RUyTpW/QAvZhuNga2HGNMn1JR/GCS+aDQ07l1e8/O5svPpRsnhcnK4oXrnCY6EexYAGmbfXvW1Nla+Zkwu13U8DHpTkoNwqEolCK31u1rrSK31IG04DPSt4LB4IFVrfUBrXQDMAYaV2mcY8In1nGuBOkqpCOs2D8BXKeUB+AHplX1RjhIT6s+WLH+K+78OQ/8HT2wyqnJqOkDJ3QPumGZMMrboseoPvNryqTEPTw879ieIf8ioK17+kjEy+3qUdcxIDjmnjBJA4+41O19Yc6NhOzfTmJYjr5JTxSf9C1a9AZ3vgdvelkZa4XBKV6KqQCkVDPwVuNTS9jPwN611VjnHjAQGaK0nWH+/B0jQWk8qsc9S4HWt9a/W31cAL2itNyqlngL+AeQB32utbS4Sq5SaiFH6IDw8vOucOXMqfD225OTkEBBQtUnpfj1WyAfbCni9ly8N/O3/zxuR/h0t905lX7OH2FMnsUrxKUsxCese4aJ3KFu62LdROeTMZjpse4X9TcZztPEdQPWunyNVNj7v/NN02vonPAuzSOnwMueDW9kthjpnk+mQ8jeyA5uT3PEVLO6/9ygrHV/jw3NpcvAzjje4iT0tnzBm33Wi6+Xv6yyuHF/fvn03aa1tj9DVWld4A+YDrwBNrLe/Al9XcMydwAclfr8HYxR2yX2+AW4s8fsKoCtQF/gJqAd4AguBcRXF2bVrV11dK1eurPIxGw6e0dEvLNU/7T5Z7ectl8Wi9acjtf57fb1u6eyqHbttvtZ/DdJ65xJzYvv0Tq3/Eal1tvHaq3P9HKlS8Z07qvU7HbV+LUrrI+vNCWT7Aq3/Gmxcv6IC2/H98pbxt5v/kNbFRebEUUXXxd/XiVw5PmCjLuMztbJfS5pqrf+qjeqiA1rrS8miPGlAyZErUVxdTVTWPjcDB7XWGdpoEP8aqGILoflirFN9Hzpt0vrUSsHQ/4KnH612vw3FlZwbUWtjYFxIU2NMhRn6vwZFedfP8qRZaUa1Uu4Zo82gUTdznqft7cZAyH3LjeUuS1cf/vYu/PiyMVbm9qm/D6QUwgkqmyDylFKXh40qpXpiVP2UZwPQXCkVq5TywmhkXlxqn8XAvdbeTN2BLK31ceAI0F0p5aeUUkA/YFclY3WYUH8vArw9OGzm+tSBDWDIOwRlp8KqNyt3zOHVxnKSNzxm3gdMWDNjwaEtnxrzSLk6bTHaTArzjMb7vLPGVA7ZJ+HkDmtyyDSSg1nzYV3S7UHo+ydImQM//Pn3HmFr3jN+b3uHMaBSkoNwssp2s3kE+MTaFgFwFih3DgGtdZFSahKwHKMX0kxtTNPxiHX7NIwxFYOAVCAXY5ZYtNbrlFLzgM1AEbAFmF6VF+YISiliwvw4aFYJ4pI2wzgRnkiDVf9nrIZV0VrRa/5nLBnZcay5cfV5HpLnwLcvQpOrejE7RtFFWPiYkRQtRUaXXkuRMQ12iZ+JaOtcAGXwDoZ7FjpuHe7efzAS1Jr/gX8YkWnHIHWGsaDP8Bky55VwCZWdaiMZ6KiUCrL+fl4p9TSQUsFxyzCSQMnHppW4r4HHyzj2rxhtHS4tJtSf7cfKbKu3m33NJ9IgLxUWTISHfyl7YZrT+4xR3L2fN3/xGp9gY/T3kicJ9/+Ziju22ZnFAgsegR1fG1Uy3oHGGstu7r//VMb9g0eOEtukqfVx6025W/d1h5heENrUcbErBQNeN6q0fnyZ5mAsaDNypmPXkxCiHFX6mqKN0dSXPAu8Y9dorkExof58u/0EhcUWPN3N62lS7OFvTOj3yVDrhH5lVDetmWIsBxn/kGmxXKHzONgymxZ7p8D+PtDUQUlCa/juRSM53PwK3Ph0ubsfTkoitneiQ0KrNDc3o50BzYmMMzQY+ZEkB+FSavKJJiN2MBqqiy2aY2crapKxgyZ9jHl4NsyA1BVXb8/JMCY/63gXBNQ3Px4wvn2PnUueb0P4Ygwc/MUxz/vrW8ZCOTdMgp5POeY5zeDhBSNnsrv1M8Z9IVxITRJELZhroWIxoUY1zsEzJrdDXNLvL1CvlbFyWelVujZ8AEX5xoemI/mFkNzxb8aypJ+PMtoDzLT5E6P3VPtRcMvfZXSxECYpN0EopbKVUudt3LKBhg6K0aWZ3tW1NE9fY0K/CxnG9B6XFOYZJYsWA4xpkh2s0CsY7lsMwVHw2Z1wZJ05T7R7GSx5ylgVb9gUGV0shInK/e/SWgdqrYNs3AK11tLNAqOrax0/T9YeOOO4J43oaMyJtH3+7xP6JX9hNHg6uvRQUkB9uG8JBIQbk9OlbbTv+Q+vgXn3Q8POxnTaUiUjhKnk61cNKaW4t3s0y3ecZEe6+b2ZLuv5DER1M6YFz0ozGqcjOl29ypmjBTaA8UuNif1mD4djm+1z3pM74Iu7ILgRjP0KvF1z2gIhrieSIOzgwV5NCPLx4O0f9jruSd09jMFUxYUwcyCcSYUeT7hGfXxQQ6Mk4RsMs++A48k1O9+5I0aJxNMP7vla1g0QwkEkQdhBsK8nD/dpyo+7TrHlSCVn6rSH0KbGymJZR4xv1m1KT5brRHUawX1LjbEJnwwzlj2tjgtnjJJIYa4x7XadxvaNUwhRJkkQdjK+Rwwh/l685chSBEDcA9BrMgx+2/X60NeNNhquPXyN8RunqjhbysUc+PxOyDoKY+ZAeFtz4hRC2CQJwk78vT14LLEpv+w77dgGa6Wg35+h+S2Oe86qCGlitEm4ecLHQyBjT+WOKy6Eufca8zyNnFn11dyEEDUmCcKOxnWPJjzIm7e+33tp+nIBRlXY+KWAMpLE6dTy97dYjHEe+1fA4Heg1W2OiFIIUYokCDvy8XRn0k3NWX8ok1X7Tjs7HNcS1txouLYUG0ki84Dt/bQ2ZjRN+dJYA7truXNCCiFMJAnCzu6Ka0RkHV/+/f0eKUWUVr+V0SZRlA+zhsDZQ1fvs/pdY4bT+IeNthUhhNNIgrAzLw83nrq5OSlpWfyw86Szw3E94W3h3kVQkGMkiXNHft+29XNjIsK2dxgznbpCl10hajFJECYY3jmS2DB/3vphLxaLlCKuEtEB7l0I+VlGdVNWGuy1rrAW28e6WI68NYVwNvkvNIGHuxtP39yc3Sey+WbbcWeH45oadjZWb8vNhI8Gwtz7oEF7GP0ZeHg7OzohBJIgTDOkQ0Nahgfy9o97KSq2VHxAbRTV1Rj8lpsJQRFw9zxjYJ0QwiVIgjCJm5vimVtacCDjAgu3pjs7HNfVKB4eXw8PrYSAes6ORghRgiQIE/VvG067yCD+s2IvBUVSiihTcCT41nF2FEKIUkxNEEqpAUqpPUqpVKXUVavaK8O71u0pSqkuJbbVUUrNU0rtVkrtUkrdYGasZlBK8dytLTmamcdXm446OxwhhKgS0xKEUsodmAIMBNoAY5RSbUrtNhBobr1NBKaW2PYf4DutdSugI1DFiXxcQ2KLenSNrst/V6SSX1js7HCEEC7mwsUiTp3Pd3YYNplZgogHUrXWB7TWBcAcoPR0o8OAT7RhLVBHKRWhlAoCegMfAmitC7TW50yM1TRGKaIFJ87n8/m6IxUfIISoVf7fwu3c8d5qlxxYa+aqcJFAyXqVNCChEvtEAkVABvCRUqojsAl4Smt91bqeSqmJGKUPwsPDSUpKqlawOTk51T62MlqHuPHO9zuJzD+Et0fVB4CZHV9NSXw1I/HVzLUaX36R5pvkXAos8NW3K6nv51rNwmYmCFufgqVTZFn7eABdgCe01uuUUv8BXgT+fNXOWk8HpgPExcXpxMTEagWblJREdY+tjMDYTEZMXcMBj8Y8mti0ysebHV9NSXw1I/HVzLUa36KtxyiwbAXALbwFiV2jHBtYBcxMV2lAoxK/RwGl+3uWtU8akKa1Xmd9fB5GwrhmdY0OoW/Lekz7eT/n8wudHY4QwgUsSU6nQZAPwb6ebDyU6exwrmJmgtgANFdKxSqlvIDRwOJS+ywG7rX2ZuoOZGmtj2utTwBHlVItrfv1A3aaGKtDPHtLS7LyCvnwl4PODkUI4WTncgv4eW8GQzpGEBddlw21KUForYuAScByjB5Ic7XWO5RSjyilHrHutgw4AKQCM4DHSpziCeAzpVQK0Al4zaxYHaV9VDAD2jbgw18PcvZCgbPDEUI40fIdJygs1gzp2JC4mBD2Z1zgTM5FZ4d1BTPbINBaL8NIAiUfm1bivgYeL+PYrUCcmfE5w7O3tmD5zhO8v+oALw5s5exwhBBOsjg5nZhQP9pHBl8eSLvp8FlubdvAyZH9zrWazGuBFuGBDOvYkFmrD3Iq2zX7PgshzHUqO581+88wtGNDlFK0jwrGy8ONjYfPOju0K0iCcIKnbm5BYbFmatJ+Z4cihHCCZSnHsWgY0rEhAN4e7nSMCna5dghJEE4QG+bPyC5RfLb2COnn8pwdjhBOpbWudeumLE5Op1WDQJqH/z57cVxMCNuPZZFX4DozLkiCcJIn+jVDo/nfylRnhyKEU03+KoXRM9Y6OwyHOZqZy+Yj5xjaqeEVj3eLqUthsSY57ZxzArNBEoSTRNX1Y0x8Y+ZuOMqRM7nODkcIpzh7oYDFycdYfzCTbWlZzg7HIZamGIuIDelwZYLo2jgEwKXGQ0iCcKLH+zbD3U3xnxX7nB2KEE6xJCWdwmKNh5vi8/WHnR2OQyxOTqdz4zo0CvG74vFgP09ahgey4ZDrNFRLgnCi8CAf7r0hmgVb0kg9lePscIRwuPmb0mgdEcQdnSNZtDWd7Ot8loHUU9nsOn7+qtLDJXExddl8+CzFLtImIwnCyR7p0xRfT3eem7uVnItFzg5HCIdJPZVNcloWI7pEMjahMbkFxSy6zldfXJx8HDcFgztE2NzeLSaE7ItF7DmR7eDIbJME4WShAd68M7oz29PPM+HjDbJmhKg15m06hrubYlinSDo1qkPriCA+X3fEJae9tgetNUuS0+neJJT6QT4294mLqQvAxsOu0Q4hCcIF3NImnLdGdWTdwUwe+2wzhcWyPKm4vhVbNAu2pJHYoh71Ar1RSjE2oTE7j58n+TptrN5+7DwHT1+4PPbBlsg6vkQE+7hMO4QkCBcxrFMkr97ejp92n+KZL7e6TB2kEGb4LfU0J89fZESJ6a1v79QQPy93Plt7fTZWL04+hqe7YmC7sqfSUEoRFxPiMj2ZJEG4kLsTovnjoFYsTTnOnxZsu26L2kLM35xGsK8n/VrXv/xYoI8nQzs2ZElKOll511djtUVrlqYcp3fzetTx8yp3324xdTmelc8xFxhEKwnCxUzs3ZQnb2rGnA1HefWbXZIkxHXnfH4hy3ecYEjHCLw93K/YNjahMfmFFhZuOeak6Myx76yF41n55VYvXRIX7TrjISRBuKBnbmnB+B4xfPjrQRkjIa47y1KOk19oYUSXq1dP6xBVh3aR119j9brjRfh4unFLm/AK923ZIJBAbw+XmJfJ1Om+RfUopfjL4DZcuFjEOz/uI8Dbg2bODkoIO5m/OY0m9fzp1KiOze1j46P544JtbD5ylq7Wb9PXsqJiCxtOFNGvdQT+3hV/5Lq7KTpH12WjCzRUSwnCRbm5KV4f0YHb2kfw6je7+Pno9VUnK2qnw2cusOHQWUZ0iUIpW0vSw9BODfH3cuezdUccHJ05ftt/huzCq6fWKE+36LrsOZlNVq5z/+8lQbgwdzfF23d1IrFlPWbtKGBJ8vU9iEhc/+ZvPoZSMLxLZJn7BHh7MKxzJN+kHHf6B6Q9LN6ajq8HJLasV+lj4mJC0Bo2H3FuKUIShIvz8nBj6t1daVHXjWe+3MqKXSedHZIQ1WKxaL7enEbPpmFEBPuWu+/Y+MZcLLIwf3Oag6IzR35hMd/vOEHXcA98PN0rPsCqU6M6eLgpp7dDSIK4Bvh6ufN0Vx/aNAzi0c82s3r/aWeHJESVrT+USdrZPEZ0Lbv0cEm7yGA6RgXz+fpru7E6aU8G2ReLSGhQ+eQAxv98u8hgp7dDSIK4Rvh6KD6+P56YUD8e+ngjW5xc9BSiquZvSsPfy53+lVxzeWxCY1JP5bjMqOLqWJKcTqi/F21Cq5YgwBgPsTXtHBeLnDf9jqkJQik1QCm1RymVqpR60cZ2pZR617o9RSnVpdR2d6XUFqXUUjPjvFbU9ffi0wcTCAv0ZvxHG9h1/LyzQxKiUnILili27TiD2kfg51W5zpNDOjYk0NuDz9ddmyOrcy4WsWL3SQa1j8DdzXaDfHniYkIoKLKw/Zjzph4xLUEopdyBKcBAoA0wRinVptRuA4Hm1ttEYGqp7U8Bu8yK8VpUP8iHTx9MwNfTnXs+XM/B0xecHZLDWSyao5m5rNqbwazfDvLy4h38sFPaZlzZ8h0nuFBQfMXUGhXx8/Lg9s6RLNt+grMXCkyMzhw/7jxJfqGlUoPjbImLNibuc2YJysxxEPFAqtb6AIBSag4wDNhZYp9hwCfaqGRcq5Sqo5SK0FofV0pFAbcB/wCeNTHOa06jED8+nZDAXe+vYdwH65j7yA1E1im/0e9alJVbyP7TORzIuMDByz+N28Wi3yc0dHdTfLnhKN893YvoUH8nRizKMn/TMaLq+hIfU7VxDWMTGjN77WHmb05jQq8mJkVnjsXJ6UQE+xAXXZdV1SgEhQZ406SevzGiuk9T+wdYCcqsBiCl1EhggNZ6gvX3e4AErfWkEvssBV7XWv9q/X0F8ILWeqNSah7wTyAQmKy1HlzG80zEKH0QHh7edc6cOdWKNycnh4CAgGod6wi24jt8vpjX1+cT5KV4sosPDf1VmX3LnRFfZZ2/qNl3rpgTFyycuKCtPy1kl+jh6K6gnq+igb8bDfzdiPD//X6x1vzp1zwaBbrxYrwPbjauwbX493UlNYnvTJ6FyT/nMbSpJ3c0L38eIlv+viaPC0Waf97oW+b729WuX06B5qmVudwS7cnoVl7Vjm/m9otsOlnEf2/ys/m+toe+fftu0lrH2dpmZgnC1qspnY1s7qOUGgyc0lpvUkollvckWuvpwHSAuLg4nZhY7u5lSkpKorrHOkJZ8bXrmMm4D9bzp1/zCPTxoF3DYDpEBdMuMpj2kcFEh/o5JGlU9/odOn2B4VNXk2mtQqgX6E1sWBBxLfxpEhZAbJg/Ter50yjED0/3smtEdb00nvsqmQMe0Ta/aV6rf19XUZP4pqxMRbOHZ4f3rFYJLyPgKH+Yl4JP4w7c0DTU7vGZ4Yv1RyjW23h8cALto4KrHV9GwFFWzUuhUZs4mocH2j/QCpiZINKARiV+jwJKj/Qqa5+RwFCl1CDABwhSSn2qtR5nYrzXpK7RIXz/TG9+Sz3NtmNZbD+WxUe/HaLAuqaEM5NGRc5eKOD+WRvQWvPFQ91pGxlEkI9ntc41vEsk324/wRvL95DYsh7N6jv+n0lcTWvN/E1pxMeEVLv6b3CHhvxt6U4+X3+kzAThahZvTSc2zJ92kUE1Ok83a5XchkNnr7sEsQForpSKBY4Bo4GxpfZZDEyytk8kAFla6+PAS9Yb1hLEZEkOZWsU4sfo+MaMtv5eUGRh78lsth/LYpv1ZitptI8yEoYzkkZ+YTETZ2/k2Lk8Pp+QQFwV66ZLU0rx2vB29H97Fc/NTWb+oz3wKKfEIRxjy9FzHDh9gYf7VL/9wNfLnRFdovhs3WHO5LQhNMDbjhHa36nz+aw9eIYnbmpe4/+p6FA/wgK82Xgok7EJje0UYeWZliC01kVKqUnAcsAdmKm13qGUesS6fRqwDBgEpAK5wP1mxVObeHm40S7SKDHYShop1pLGrBJJo1+r+ky5u0uVRntWl8WieX5eChsOneW/YzrXODlcUj/Qh7/f3o5Jn2/h/VUHeLyvTHHobPM3peHj6cag9rbXYK6ssQmNmbX6EPM2pfGwkxpsK2tpynG0hqEda/aawfji0y2mLhuctASpqbO5aq2XYSSBko9NK3FfA49XcI4kIMmE8GqV8pLGT7tP8faPe7n/ow18cF9cpWacrIm3ftjL4uR0nh/QstpdAMsyuENDvt1+gnd+3EvflvVp07BmRXxRffmFxSxJTqd/2wYEVrPq8JIW4YHERdfli/VHeKhXE9yqMa7AURYnp9M6Ishu1ZxxMSF8u/0EJ7LyaRBsey1rs0gZvBa7lDSe7Nect0Z1ZP2hTO6dud7U1bzmbjjK/1amMia+EY+a9E3w78PaEezrxXNfJVNQJOt7O8uKXac4n19kc92H6hib0JhDZ3JZc+CMXc5nhqOZuWw9eo6hdvzi0y3GGA+x0QmlCEkQAoA7OkcxZWxnUtLOcfcHay/3KrKnX/Zl8McF2+jVPIy/DWtnWptHiL8X/xzenl3Hz/Pfn2TBJWeZvzmNBkE+9GwWZpfzDWofQbCvJ5+78DTgi60zLg/uUPPqpUtaRwTh6+nulHmZJEGIywa0i2D6PXHsO5nD6OlrOJWdb7dz7z5xnsc+3Uyz+gG8d3eXcrus2sMtbcIZ0SWK95L2k3z0nKnPJa6WkX2Rn/dmcHvnyGpNM2GLj6fRWL18xwkysi/a5Zz2tiQ5nS6N69AoxM9u5/R0d6Nz4zpSghDO17dVfT4a3420s3nc9f5a0u2wcPrJ8/k88NEG/LzdmTm+W43royvrL0PaUC/A26hqKr52ZwQ12/c7TjB341G7zpq6aOsxii2akZWYubUqxiY0osii+WrTUbue1x72nsxm94lsu1YvXRIXE8LO9PPkXCyy+7nLIwlCXKVHszBmPxjP6eyL3DltDYfPVH++pwsXi3jw4w2cyyvkw/u60dCBU4IE+3ryxsgOpJ7K4et9195cPo5w8PQFnvhiC8/PS+GPC7ZTWGyfNpt5m9LoGBVs9/EozeoHEh8bwpz1R7FYXCvpL0lOx03BIDtWL13SLaYuFo3DZ3GWBCFs6hodwucPdedCQRGj3l9D6qnsKp+j2KJ58ost7Ew/z5SxXWgXGWxCpOXr3aIeYxMas/xQkdMXX3E1Wmte+joFLw83xveI4Yv1R3hg1gbO59esk8KO9Cx2n8iu0sR8VXF3QmOOZObya6rrrIuitWZxcjo3NA2lfqD9exp1blwXN+X4ifskQYgytY8K5suJN1BsgbveX8vO9MpPL6615pUlO1ix+xSvDG1L31b1TYy0fH8c1JowX8Xkr5LJLbB/Ef1iUbHdvnk70tyNR1l7IJM/DmrNy0Pb8sbIDqzZf4YR763maGZutc87f9MxPN1VldZgrooB7RpQ18+1Gqu3Hcvi8JlcU6qXwFiGtU3DIGPiPgeSBCHK1bJBIHMf7o6Xhxujp69hayUbfD/89SCfrDnMxN5NuOeGGFNjrEiAtwcPtvfm8JlcXv92t93OW2zRfLH+CD1f/4nB7/7KiSz7Neqb7VR2Pv/4ZhcJsSHcFWfMdjMqrhGfPBjPyfP53PHeb9WqzigstrBo6zH6tQqnrn/VJ+arDG8Pd0Z2jeKHXSc5dd41rvnirel4uisGtLV/9dIlcdEhbDlyzqFfRiRBiAo1qRfA3IdvoI6fF+M+WMe6Cvqhf7f9BP9YtouB7Rrw4oBWDoqyfK1C3HmgZyyfrDnMb3aomlideprb3v2Fl77eRqMQP46dy2PE1NWknsqxQ7Tme2XxTvKLLPxzePsrBp31aBrG14/1xM/Lg9HT17Js2/EqnffnPRmcuVBgWvXSJWPiG1Ns0czd6PzGaotFszTlOH1a1CPYz7wOGN1iQsgrLK5SSb6mJEGISmkU4sfch28gPMib+z5az6q9GTb323r0HE9/uYVOjerw9l2dXGrE6/MDWtIkzJ/n56VUu5794OkLPPTJRsZ+sI7s/CKmjO3C14/2YM7E7lwsKubOaatdfjnY73ec4Jttx3mqX3Oa1Lt6Cupm9QNY8FgP2kUG89hnm3kvKbXSPZzmbUoj1N+LxJb17B32FZrUC+CGJqF8sf4oxU5urF5/KJMT5/PtPitAaXExlxYQclw1kyQIUWkNgn348uEbiA0LYMLHG69axe1oZi4TPt5AvUBvZtwb55B5narCx9OdN0d15HhWHq8u3VnxASVk5RXy6tKd3Pr2z6xOPc3zA1qy4rk+3NYhAqUU7SKDmf9oDwJ9PBk7Yx0r95wy6VXUzPn8Qv68aDutGgQysXfZE+iFBnjz2YQEhnZsyBvf7WHm9oIKR6WfvVDAit0nGdqpoenjXMAYWX3sXB6r9tn+suIIp3Mu8tb3e/HxdOPm1uGmPld4kA+NQ/wcOmBOEoSokrAAb754KIHWEYE8+ukmllhHjl4o1Iz/aD2FxZqPxscT5qIzbnZpXJeH+zRl7sY0ftpd8TKlRcUWZq89TN83k/jwt4MM7xzFyj8k8lhis6sSYHSoP/Mf7UGTev489PFGvt6cZtbLqLY3vttNRvZFXh/RocIPcR9Pd/4zuhNP9mvOL8eKGP/RerJyyy55LUlJp7BY221qjYr0b9uAUH8vpzVWrz1whkH/+YWtaed49fb2ps9hBkYpYuPhTLuOWSmPJAhRZXX8vPh0QgJdGtflqTlb+HzdEf67JZ8jmbm8f09XmtV3nZW9bHn65ua0DA/khfnbOJdb9viIVXszGPTuL/x54Xaa1w9gyaQb+dfIDuV2Y6wX6M2cid2Jjw3h2bnJTF+134yXUC0bDmXy6doj3N8zlk6N6lTqGKUUz97Sgofae7HhUCbDp/7GkTO2ezjN35RGqwaBtHXQBIleHm6MjIvip92nOJvvuIbbYovmPz/uY+yMtQR4e7DwsZ6MNLnN5ZJuMSGczingUBl/A3uTBCGqJdDHk48fiKdnszD+uGAbuzMt/N/IjnRv4voLunh7uPPvUR05e6GAvy7ecdX2/Rk5PDBrA/fOXE9+oYVp47oyZ2L3So/jCPTx5KP7u3Fb+wheW7abf3yz0+mDui4WFfPi/BQi6/jy7C0tqnx8z0hPPn0wgTMXCrj9vd/YVGrah9RT2SSnZTGiS5RD1xUZ081orP45zTEjjE+dz+eeD9fx9o97GdYpkiVP3OjQGYO7ObgdQhKEqDZfL3dm3BvH2ITGjGvtxe2d7TutgpnaRQYz6aZmLNqazrfWnjrncgt4ZckO+r+9ivUHM3lpYCt+eLY3A9o1qPKHnreHO++O6cy9N0Qz45eDTP4q2aljJaas3M/+jAv844521a4KSWgSyoLHehLk48GYGetYtPXY5W3zNh3D3U0xrLO5DbWlxYT506dFPRalFvLs3K0cOl39Uf8V+WWfUaLcfOQsb4zswFujOjqkWqmkpvUCqOvn6bDxEI59deK64+Ppzmt3tCcpyXWnYC7L432b8eOuk/xp4XaOZOYy9ef9nM8rZHR8Y569pUWN21Hc3RSvDG1L/UBv3vx+L5m5Bbx3dxf8vBz7b7fnRDZTk1K5o3MkiS1rNmAxNsyfBY/15OHZm3hqzlaOnMnl0cSmLNiSRp8W9UwZRVyRd+7qxEuzV/JNynEWbU1nZJconujXjKi69pkwr6jYwts/7uW9pP00rx/A5w91p4UTlv8Eo8qva3SIwxqqpQQhai1PdzfeGtWJnPwi/vntbto2DOKbJ3vx2h3t7dbIrpRi0k3NeX14e1btzWDMjHWmTKVelmKL5sWvUwj08eTPg9vY5Zx1/b2YPSGeOzpH8u8f9nLX9LWcPH/RYY3TtuIZ3cqbX57vyz3do1mw5Rh930ziTwu2cTyrZpNNHs/KY8yMtUxZuZ9RXRux6PEbnZYcLukWU5cDpy9wOsf8GW2lBCFqtRbhgXw4Po4iiyaxRT3T6s9HxzcmxN+LJ77Ywshpq/nkgXi7fcMtz6drD7PlyDnevqsjIXYc2ezt4c5bozoSG+bPWz/sJcjHg36tnTedCkD9IB9eHtqWh/s04X8/pTJ341G+2pjG2ITGPJbYlPpBVSvd/LT7JM/NNRadeueuTi5ThXppPMTGQ2cZ0K6Bqc8lJQhR6/VqXo++Leub3rh6a9sGzH4wgYzsi4yYupo9J6o+AWJVHDuXxxvf7aZ3i3rc3sn+H25KKZ7s15yPxnfjP2M6u8y4l4hgX/5xR3t+ei6ROzpHMnvtYXq9sZJ/fLOzUt+6C4osvLZsFw/M2kiDYF+WPHGjyyQHMNrPvDzcHNIOYWqCUEoNUErtUUqlKqVetLFdKaXetW5PUUp1sT7eSCm1Uim1Sym1Qyn1lJlxCuEo8bEhfPXIDQDcOW21ab1RtNb8vwXbsGj4x+3mrd4HxhoifWvYtmGGRiF+/GtkB1Y8awxo/PDXg/R+YyX/+m43Z8uo5juamcuo99cwfdUBo7rqsR42R5s7k7eHO52i6rDhsPntEKYlCKWUOzAFGAi0AcYopUpXgg4EmltvE4Gp1seLgOe01q2B7sDjNo4V4prUqkEQ8x/tQVigN+M+WHfViHR7WJJynJV7Mpjcv6VdVze7FsWE+fPWqE58/0wf+rUOZ9rP++n1xkre+n7PFeuvf7f9BLe9+wv7T+Xw3t1d+Pvt7VymVFRaXExddhzLMmV24pLMLEHEA6la6wNa6wJgDjCs1D7DgE+0YS1QRykVobU+rrXeDKC1zgZ2Aa5TxhOihqLq+jHvkR60igji4dkbWbq/gOwarsNwydkLBbyyeAcdo4IZ3yPGLue8HjSrH8B/x3Tmu6d606t5GO/+lMqN//qJd1fs4+XFO3jk003EhvnzzZO9GNTevFlZ7aFbTAhFFl3p2ZWrS5k1ZFspNRIYoLWeYP39HiBBaz2pxD5Lgde11r9af18BvKC13lhinxhgFdBOa33VNIZKqYkYpQ/Cw8O7zpkzp1rx5uTkEBDgWkXJkiS+mnHV+C4WaaalXGTLqWJ83KF3lAc3R3tS36/6390+2HaRNelFvNzDl0aB9vkO6KrX75LqxHf4fDELUwvZcqoYgP7RHtzZ0gsPEyaYtPf1u1CombQilzuaezK0ac06H/Tt23eT1jrO1jYzezHZusqls1G5+yilAoD5wNO2kgOA1no6MB0gLi5OJyYmVivYpKQkqnusI0h8NePK8fW/GWYuWkFKXghLU47zw5EibmkdzgM3xpIQG1Kl9oNf953m1+/W8XjfptzT335Trbvy9YPqx3cfsP1YFhcuFpFg4iwAZly//+5YxWnlQ2JivF3PW5KZCSINaFTi9yggvbL7KKU8MZLDZ1rrr02MUwinaxLszgPDOvPSoNbMXnOYz9Yd5vudJ2kTEcQDN8YypGME3h7l14fnFRTz0oIUmoT588RNzR0U+bXPGUvh2kNcTF0Wbkmn2KJxN2lafTPbIDYAzZVSsUopL2A0sLjUPouBe629mboDWVrr48r4yvQhsEtr/ZaJMQrhUsKDfJjcvyVrXurH68PbU2SxMPmrZHq+/hPv/LiXjOyyu2m+/eNejmbm8drw9i7buCrsp1tMCDkXi9h9wrwFhEwrQWiti5RSk4DlgDswU2u9Qyn1iHX7NGAZMAhIBXKB+62H9wTuAbYppbZaH/uj1nqZWfEK4Up8PN0ZHd+Yu7o14rfUM8z87SDv/LiP91buZ0jHhjxwYwxtG/7+zXf7sSw++OUAY+IbXxMTJoqai4sJAYwBcyXfC/Zk6khq6wf6slKPTStxXwOP2zjuV2y3TwhRqyiluLF5GDc2D+NARg4frz7EV5vSmL85jYTYEB64MZbElvV4fl4KYQHevDjQNZZ4FeaLrONLw2AfNhzK5D6TeqvJVBtCXCOa1AvglWHtePbWlszdcJRZqw/x8OxNBPt6kpVXyLRxXQj2NW9NZOF64mJCWHfwDFprUwZDSoIQ4hoT7OvJQ72bcH/PGH7YeZKP1xwiNsyfAe1cu+++sL9uMXVZnJxO2tk8UwZESoIQ4hrl4e7GwPYRDHTxQV3CPJfbIQ5nmpIgZLI+IYS4RrUIDyTQx4MNJq0PIQlCCCGuUe5uiq7RdU2b2VWqmIQQ4ho2oG0DktPOYbFo3Ow8YE4ShBBCXMNGxzdmdHxjU84tVUxCCCFskgQhhBDCJkkQQgghbJIEIYQQwiZJEEIIIWySBCGEEMImSRBCCCFskgQhhBDCJmUsyXB9UEplAIereXgYcNqO4dibxFczEl/NSHw148rxRWut69nacF0liJpQSm3UWsc5O46ySHw1I/HVjMRXM64eX1mkikkIIYRNkiCEEELYJAnid9OdHUAFJL6akfhqRuKrGVePzyZpgxBCCGGTlCCEEELYJAlCCCGETbUqQSilBiil9iilUpVSL9rYrpRS71q3pyilujg4vkZKqZVKqV1KqR1Kqads7JOolMpSSm213v7i4BgPKaW2WZ97o43tTruGSqmWJa7LVqXUeaXU06X2cej1U0rNVEqdUkptL/FYiFLqB6XUPuvPumUcW+771cT4/k8ptdv691uglKpTxrHlvhdMjO9lpdSxEn/DQWUc66zr92WJ2A4ppbaWcazp16/GtNa14ga4A/uBJoAXkAy0KbXPIOBbQAHdgXUOjjEC6GK9HwjstRFjIrDUidfxEBBWznanXsNSf+8TGIOAnHb9gN5AF2B7icfeAF603n8R+FcZ8Zf7fjUxvlsBD+v9f9mKrzLvBRPjexmYXIm/v1OuX6nt/wb+4qzrV9NbbSpBxAOpWusDWusCYA4wrNQ+w4BPtGEtUEcpFeGoALXWx7XWm633s4FdQKSjnt9OnHoNS+gH7NdaV3dkvV1orVcBpVeUHwZ8bL3/MXC7jUMr8341JT6t9fda6yLrr2uBKHs/b2WVcf0qw2nX7xKllAJGAV/Y+3kdpTYliEjgaInf07j6w7cy+ziEUioG6Ayss7H5BqVUslLqW6VUW8dGhga+V0ptUkpNtLHdVa7haMr+x3Tm9QMI11ofB+NLAVDfxj6uch0fwCgR2lLRe8FMk6xVYDPLqKJzhevXCziptd5XxnZnXr9KqU0JQtl4rHQf38rsYzqlVAAwH3haa32+1ObNGNUmHYH/AgsdHF5PrXUXYCDwuFKqd6ntTr+GSikvYCjwlY3Nzr5+leUK1/FPQBHwWRm7VPReMMtUoCnQCTiOUY1TmtOvHzCG8ksPzrp+lVabEkQa0KjE71FAejX2MZVSyhMjOXymtf669Hat9XmtdY71/jLAUykV5qj4tNbp1p+ngAUYRfmSnH4NMf7hNmutT5be4OzrZ3XyUrWb9ecpG/s49Toqpe4DBgN3a2uFeWmVeC+YQmt9UmtdrLW2ADPKeF5nXz8PYDjwZVn7OOv6VUVtShAbgOZKqVjrN8zRwOJS+ywG7rX2xOkOZF2qCnAEa53lh8AurfVbZezTwLofSql4jL/hGQfF56+UCrx0H6Mxc3up3Zx6Da3K/ObmzOtXwmLgPuv9+4BFNvapzPvVFEqpAcALwFCtdW4Z+1TmvWBWfCXbtO4o43mddv2sbgZ2a63TbG105vWrEme3kjvyhtHDZi9G74Y/WR97BHjEel8BU6zbtwFxDo7vRoxicAqw1XobVCrGScAOjF4Za4EeDoyvifV5k60xuOI19MP4wA8u8ZjTrh9GojoOFGJ8q30QCAVWAPusP0Os+zYElpX3fnVQfKkY9feX3oPTSsdX1nvBQfHNtr63UjA+9CNc6fpZH5916T1XYl+HX7+a3mSqDSGEEDbVpiomIYQQVSAJQgghhE2SIIQQQtgkCUIIIYRNkiCEEELYJAlCiCpQShWrK2eMtdssoUqpmJKzggrhbB7ODkCIa0ye1rqTs4MQwhGkBCGEHVjn9v+XUmq99dbM+ni0UmqFdWK5FUqpxtbHw61rLSRbbz2sp3JXSs1Qxnog3yulfJ32okStJwlCiKrxLVXFdFeJbee11vHA/4B3rI/9D2P68w4Yk969a338XeBnbUwa2AVjNC1Ac2CK1rotcA4YYeqrEaIcMpJaiCpQSuVorQNsPH4IuElrfcA64eIJrXWoUuo0xlQQhdbHj2utw5RSGUCU1vpiiXPEAD9orZtbf38B8NRav+qAlybEVaQEIYT96DLul7WPLRdL3C9G2gmFE0mCEMJ+7irxc431/mqMmUQB7gZ+td5fATwKoJRyV0oFOSpIISpLvp0IUTW+pRah/05rfamrq7dSah3GF68x1seeBGYqpf4AZAD3Wx9/CpiulHoQo6TwKMasoEK4DGmDEMIOrG0QcVrr086ORQh7kSomIYQQNkkJQgghhE1SghBCCGGTJAghhBA2SYIQQghhkyQIIYQQNkmCEEIIYdP/Bzo70cFo09B9AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6xElEQVR4nO3deXyU1b348c83+x5CAmHfESTsYVNcwBWxrnVBrVrrUv1J63LtrV1ubW9Xe1tv1bpctda6Fa2otYpLVXDDBVDEIKhsSiABEiD7ZJvv748zwRAmIZPMk8mE7/v1mldmnvU7z0ye75zznHMeUVWMMcaYlmIiHYAxxpjuyRKEMcaYoCxBGGOMCcoShDHGmKAsQRhjjAkqLtIBhFNOTo4OGzasQ+tWVVWRmpoa3oDCwOIKjcUVGosrND0xrlWrVpWoap+gM1W1xzzy8/O1o5YuXdrhdb1kcYXG4gqNxRWanhgXsFJbOadaFZMxxpigLEEYY4wJyhKEMcaYoCxBGGOMCcoShDHGmKAsQRhjjAnKEoQxxpigelRHOWOM6c5UlcraBnZX1VFSWcfuqjpKK2spraojMS6GQVnJDMpKYVBWMpnJ8YhIROO1BGGMMZ3U6Fc+31FBUVnNASf+0so6Sqtq2V1ZR0lVHXUN/nZtMy0xjkFZyQzslbxf4mj62yvF+wRiCcIYY0LU6FfWFZXz3qZS3ttUygebd1Pua9hvmaT4GLJTE8lJS6BPWiJj+2WQnZZAdmoC2amJ9E5LICc1key0BHqnJlBb72frnmoK99RQuO+ve/7B5t1U1O6//ZSE2H0JI66mljlzwv8+LUEYY8xBNPqVT7eX8+Lmeh7ZsoIPtuymIpAQhuekcurE/swY3pvhOWkuAaQlkJIQ2uk1KT6WzJRMxg/MDDq/rKb+gMTR9Lyqsn2lklBZgjDGmBYaGv18uq+EsJsVzX7Bj8ip4hsTBzBrRG9mjcgmNyOpS2LKTI4nMzmTvAEHJpBly5Z5sk9LEMaYqOH3KxW1DZTX1FNWU0+5r57ymnrKaxr2va6qbSRGICZGEIEYEWJFiBEQEWICz5vPjwn89dU3surLPazYsofKpoTQJ5XTJg9g1ohs/EXrOXPenMgehC5kCcIY02Ua/Uqlr8Gd2H3uxF7hq6ciMK3C507+n22uZdHWVftO+mU1LhFU1Dag2vr2YwSS42MB8Cv4VVGFRtV9zw9mZJ9UzggkhJkjetM3/esSwrI9n3f2EEQVSxDGmLCoqWtkXXE5a7eVsXZ7OTvKfVT4GvY7+Ve2uNAaTEpCLAnip29DJZnJ8fTLSOKw3HQyk+PJSIojIzmejOT4wOvA3+Q4MpPjSU2IIyam9ZY9GkgSftX9Eog/kEBiREhNtNNiEzsSxpiQVdU28GlROQXbyvhkWxlrt5WzYVcljX73Ez0rJZ5BWSmkJ8UxPCeV9MCJPT0pjvQkd6JPT3In9owkd6JPT4ojLSmO+NgYli1bxpw5x4Y9bpFAtRKR7V8QLTxNECIyD7gdiAUeUNXftZifBTwIjAR8wHdUtSAw7zrgSkCA+1X1T17GaowJrtxXz9ptLhkUbC+jYFsZm0qq9lXX9ElPZMLATE7OyyVvYCYTBmbSPzMp4p28TOd5liBEJBa4CzgRKARWiMhzqvpps8V+DKxW1bNEZGxg+eNFZDwuOcwA6oCXROQFVf3Cq3iNOZT5/UpRuY8vS6v4qrSaL3dX82VpFWu3l/NlafW+5fpnJjF+YCanTxrIhEEZjB+QSd8uasVjup6XJYgZwAZV3QQgIouAM4DmCWIc8FsAVV0vIsNEJBc4HHhPVasD674BnAX83sN4jYkYX30jxWU+isp8FJfXUFxWS3FZDUVlPqrrGumVEk9WSgJZqQlkpcTTOzWBXikJ9E5JICvVzUtJiG3zV7uvvpHCPdV8WeoeX+2u5sMvfPz3qmUU7q6hrvHrtvRxMcKgrGTG9c/gvGmDGT8wk7wBGeSkJXbF4TDdhJcJYiCwtdnrQmBmi2U+Bs4G3haRGcBQYBBQAPxaRLKBGmA+sNLDWI3xTFVtA9sq/Lzx+S527JcEmp772Ftdf8B6GUlx9M9MJiUxlu17a9hdXUdZTX2rLXES4mLIakokKa53bmJ8DNv21PDV7mqKy337rZuaEEvvRCVvSDonHp7LkOwUhvZOZWh2Cv0zk4iLtbE8D3Wi7Wn31ZENi5wLnKyqVwReXwzMUNXvNVsmA3eNYgrwCTAWuEJVPxaRy4FrgUpcqaNGVW8Isp+rgKsAcnNz8xctWtSheCsrK0lLS+vQul6yuEITqbj8qpTWKEVVfoqrlOIq/77ne2oP/B/LSICspBh6JwlZSUJWogSeB6YlColxB5YG/KpU1UNlnVJZ7x4VTc/r2Pe6KvC3thGyk4S+KTH0TRH6pMTQN9m9Tk+Aqqoq+xxD0BPjmjt37ipVnRZsnpcliEJgcLPXg4DtzRdQ1XLgMgBxZePNgQeq+hfgL4F5vwls7wCqeh9wH8C0adN0TgcHJHGtJjq2rpcsrtB4HVdZTT2bdlWyaVcVm0oCf3dVsbm0ar9B2DKS4hjRJ4O5w1IZ2SeNiuItHH/EVPplJNE3I5HEuFjPYgzFofo5dtShFpeXCWIFMFpEhgPbgAXAhc0XEJFeQLWq1gFXAG8GkgYi0ldVd4rIEFw11BEexmrMARr9yifbynhnQwnvbixlfXEFJZW1++bHxghDeqcwIieVYw7LYUSfNEb2SWNEn1SyUxP2ux6wbFkh04f1jsTbMKbDPEsQqtogIguBl3HNXB9U1bUicnVg/r24i9EPi0gjrhrp8mabWBy4BlEPXKuqe7yK1Rhwnag2l1TxzoYS3g4khaYROg/vn8FxY/swok8aI3JSGdEnjSG9U0iIs3p603N52g9CVZcAS1pMu7fZ83eB0a2se7SXsRkDsLPCx/INpby9oYR3NpRQVOYDYGCvZOZP6M/sUTkcOTKbbGu9Yw5B1pPaHFIqaxt4f1Mp72wo5Z0NJXy2owKAXinxzB6Zw+xROcwelc2Q3inW0csc8ixBmB6ttqGR1V/t5Z2NpSzfUMLqrXtp8CuJcTHMGN6bs6YO5KhROYzrn9HmGD7GHIosQZgexa/KJ4VlvLPRVRmt2LIbX72fGIEJg3rx3WNHMHtUDlOHZJEU3z1aEhnTXVmCMFFNVdm4q4rlG0tYvqGUtz6vpurltwE4LDeNBdOHcOTIbGaOyCYzOT7C0RoTXSxBmKhTVFbDOxtcldE7G0vYUe6ang7slczUvnGcc/R4jhiZvd84/saY0FmCMN1afaOfz4orWL11Lx9v3cuqL/ewqaQKgOzUBI4Yme0uLI/MYXDvZN544w3mTB4Y4aiN6RksQZhuQ1Up3FPDR4FksHrrXgq2lVEb6KGcnZrAlCG9uHDmEGaPymFMbrpdWDbGQ5YgTMTsra4LlAzKWL11Dx8XlrG7qg6AxLgYJgzM5OJZQ5k0uBeTB/diUFayNT01pgtZgjCeqG/0s6e6jj1V9ZRW1bK7qo49VXWUVtWxpaSK1Vv3siVwnwERGN03jePH9mXykF5MGtSLMf3SibfRRI2JKEsQJmSfbi/n7W31fPHmJkqbnfh3V9Wyp7qe0srafUNUBNMvI4lJgzM5b/pgJg/uxYSBmaQnWQsjY7obSxCm3TbsrOB/Xv6Ml9fucBM+WUd8rOy790Dv1ATyBiSTnepubNP0t3ezR1ZKgpUMjIkSliDMQW3fW8OfXv2cp1YVkpIQxw0nHEZu7VbmH3806Ylxdl3AmB7KEoRp1Z6qOu55YyMPLd8CCt8+cjjXzh1Jdloiy5ZtI8OqhYzp0SxBmANU1zXw13e2cO+yjVTWNXD2lEFcf8JoBvdOiXRoxpguZAnC7FPf6GfRiq3c8doX7Kqo5YTD+/KDk8cypl96pEMzxkSAJQiD36+88EkRf3zlM7aUVjN9WBb3XDSVaXYHNGMOaZYgDmGqyltflPD7l9dTsK2cMbnp/OXSaRw3tq9deDbGWII4VK3eupdbX1zPu5tKGdgrmdvOm8QZkwcSa0NXGGMCLEEcYjbuquQPL3/GiwXFZKcmcMtp47hw5hAS4+zeCMaY/VmCOEQUl/m4/bXPeXJlIUlxMVx3/GiuPGYEaYn2FTDGBGdnhx6urLqeu9/YwEPvbMGvysWzhrLwuFHkpCVGOjRjTDdnCaKHqqlr5K/LN3Pvso1U1DZw1uSB3HDiYdaXwRjTbpYgepj6Rj//WFnI7a99zo7yWo4b25cfnDyGw/tnRDo0Y0yUsQTRQ6gqSz4p5o+vfMamkiryh2Zx5wVTmTHc+jIYYzrGEkQP8M6GEm59aT1rCss4LDeN+y+ZxgmHW18GY0zneJogRGQecDsQCzygqr9rMT8LeBAYCfiA76hqQWDeDcAVgAKfAJepqs/LeKPNJ4Vl/P7l9bz1RQkDeyXzh3MncdYU68tgjAkPzxKEiMQCdwEnAoXAChF5TlU/bbbYj4HVqnqWiIwNLH+8iAwEvg+MU9UaEXkSWAA85FW80eaBtzbxqxfWkZUSz09PPZxvzRpKUrz1ZTDGhI+XJYgZwAZV3QQgIouAM4DmCWIc8FsAVV0vIsNEJLdZbMkiUg+kANs9jDVqqCp/fn0Df/z358yf0I/ffXOiDbttjPGEqKo3GxY5B5inqlcEXl8MzFTVhc2W+Q2QpKo3isgMYHlgmVUich3wa6AGeEVVL2plP1cBVwHk5ubmL1q0qEPxVlZWkpaW1qF1vdQ8LlVl8Rf1PL+pntkD4vjO+ISIVSdFw/HqTiyu0HQoLlWSa7YT46+nKm1Y94mrC3Qmrrlz565S1WlBZ6qqJw/gXNx1h6bXFwN3tlgmA/grsBp4BFgBTAKygNeBPkA88CzwrYPtMz8/Xztq6dKlHV7XS01x+f1+veWfBTr0h8/rj59eo42N/m4RV3djcYUmquOqKVPd8Lrqst+rPnqO6u+Gqt6S4R4v3KRa74tMXBHQmbiAldrKOdXLKqZCYHCz14NoUU2kquXAZQDimtxsDjxOBjar6q7AvKeBI4FHPYy322r0Kz999hP+/sFWLj9qOD899fDobKHkb4Rd693ffhMgGt+DiQy/H0o+h8IVUPgBFK6EnetwbViAPmNh7KkwaIb7jr13N2x9H859CHqPiGTkUc3LBLECGC0iw4FtuIvMFzZfQER6AdWqWodrsfSmqpaLyFfALBFJwVUxHQ+s9DDWbqvRr/zHk6t5dvV2vnfcKG488bDoSQ5VpbBtJWz9wP1jb/sQ6ircvPQBMOYUGDsfhh0DcQmRjdV0K3H1lfDFq80SwiqoLXMzkzJh0HQYdyYMmgYD8yG51/4bGHYUPHsN/N+xcPqdkHdmF7+DnsGzBKGqDSKyEHgZ18z1QVVdKyJXB+bfCxwOPCwijbiL15cH5r0vIk8BHwINwEfAfV7F2l3VNfi55+NaVu7Yzg9OHsO1c0dFOqTWNTbAzrXuH3rrCvd390Y3T2IhNw8mne/+sf2N8NkS+PjvsPIvkJAOo0+AMafC6BMP/GePZr5yWHwFR216E5Z38N8tNdudDCecC/3GhzW8bqO2Ar5cDpvegE3LOGrnWngHkBjoOw7Gn+2+O4NnQO+REBPT9vbGngrffQue+g7841LYciWc/GuIszHIQuFpPwhVXQIsaTHt3mbP3wVGt7LuLcAtXsbXnfnqG7nm0VWs3NHIz74xju8cNXz/Bcq3w+rHYNb/g4TULo8vvm4vrH8hUDpYCds/hPpqNzO1r/tHnnqx+6ceMOXAGKdcBPU17oTw2Qvw2Yuw9hmIiYOhs90/+Jj50GvwAfuOGlUl8OjZsGMtO/qdwMAhIzu2nZLPYfmd8M6f3Mlywjkw/hzIGhrWcLtUY7373mxaBpvfcD8o/A0QlwRDZrEp9SJGHH0+DJwKiR285W3WULjsRXjtF/Dun11JxKqcQmI9qbuh6roGrnx4Jcs3lvLtvITgyeGhU2H3JqjeA/N+03XBNTbA389n9oZXXZuzmDjoNxGmXuKSwaDp0GtI+64vxCfDmHnu4fe76qj1L7jHi//pHv0muJLF2PluP9FSvbZ3KzxyJpRtgwV/54vtCQycM6fj26sqcQn0k6fgtf92j8GzXLLIOwtSc8IVuTdUYeenLiFsegO+fAfqKl0Jof9kOPL7MGIODJ4J8Ul8tWwZI0Yc2/n9xiW4ksPQ2c2qnO5wxyxc9n4FHz4CBYshvZ97HyPmwICpEBvdp9jojr4HKvfV852/ruDDr/Zw23mTyCrbsP8CFcXwt9OgcheMOhHevwcmnut+pXeF9+6GDa/y1eCzGXLid6H/JHei76yYGFfqGDwDTvwFlHzhEsVnS+CNW+GN30HmYBj7DZjyre5d1bLrc5ccaivh4mdg6BGwfVnntpmaAzOudI89X0LBU7DmH7DkJnjxhzDyOJh4nit1JXaTZph7t35dQtj0BlTtdNOzR8GkBe4kOuwoSM7yPpax8+Hqt+Afl8E/vg1b3oGTfgXxSR3bXmM9fP4SrPobbHjVTRt+DPj2wtLfwNJfu6rTYUd9nTD6jImeHzgBliC6kb3VdVzy4Ad8ur2cP184lfkT+rNsWbMEUbnTJYfyIrj4addy464Z8K/r4IrXvf+1UrrRffHHnMqmfpcwZMgs7/aVMxqOut49Kne6f8b1S9w1i/fvgYHTIP9SyDu7+5wQwV2If+wcd93lshdcCSjcsobC0f/hHjvWwpon3a/Xp6+EuGR3MpxwnksaXX3xv3q3u7b04SOwa52bltr365PkiGMhc1DXxtSk15BOVzkl1RTDq79w1buVO1xji2P/0/1o6TXELVRVClve/Lq09PmLbnpaP/f+R8yB4cdC5sBwv8OwswTRTeyqqOXiv7zPppIq7rskn+PG5u6/QOUulxzKCuFbi6Hp5HzKre4X0fv3wpELD9hu2Pj98Nz3ITYRTv0jfPiZd/tqKa2vq8KaekngBLQIVj0Ez30PXvqxq2bJ/zYMmNx1MQWz+U34+wWQ0hsufhayO3jNIRS5ea7EdfwtrlnnJ0+6qqiCxe6X+bgz4PDTYNjR3l2gVYUtb7lf0+ueg8Y6V9V48m/dybDv4d3nl3NHqpwa6tx1slV/Y9ampa5abPTJ7gfKqBMP/GGWmu2217TNPVtcotj8Bmx4DdY84aZnj/46cQ47qls2zrAE0Vk1e0H97qTQQUVlNVz0wPsU7fXx4KXTOWp0i/rkqlJ4+HRXtXDRP2DokV/PG3em+7Iu/TWMO/3rXzHh9uHf4Mu34bQ7IKM/0IUJormU3nDE/4NZ17gT4qqH3C/WVX+F/pPon34k+KZCUhff/2Ld867FTO8Rrlopo3/X7j8mxlVlDT0C5t0Km5a6ksWaJ90x8qKlWOUu+Phxlxh2b3TNT/MvcyfO3LzOb99L7alyKt3ojt3qx6G6BDIHs3nYhQw/66eh/frPGgb5w9xx8fubXYtZ5koiK+53SWfiAjjrnnC+y06zBNFZiy6EojVw0n+7f44Qfylt3V3NhQ+8x56qeh6+fAbTh+2faOLqy11y2L0JLnwChh+9/wZE4NQ/wF2z4IWb3DLh/rVWtg3+/TP3K3TqJeHddkeJuFLUkFkw77euPn7VQ4z5/B7448OuWWT+Za4VjNe/Xj96DJ5b6NrjX/hkp34shEVcAhx2snvs11LspQNaiiX6QozV74fNy1xSWP8C+OthyBGummXcGeG5HtVVWlY5bX0fzr7PVdutesiViiTW9dfJvwxGzuXLN99ieGeqhmJi3PWzfuNdib+hzjXO2LTMlZS7GUsQnbFni2uNkdoXnr8B1j7rOuW0s/nh1t3VnPd/71Jd18hjV8xk0uBe+y9QvZtJH/8MarbDhYtcUTSYXkPguJ/Ayz92J4DxZ3fiTbWgCi/8h7sod9rt3aeqoLnkLJh5Fcy4klX/uo98XeOqWD56BHLHu+qnCed6U4Rf/md45ScwYi6c/2j3uh4CrbcU+2wJvPifHAGw5faDtxSrKHa/dlf9DfZ+6Y75jKvcr+I+Y7r6XYVPyyqnuwNVt1nD4PifweSLXMskL/c/9Mj9awW6EUsQnbH2Gff3in/DxqXwyn/B3Ue4OuFplx+0M88j731JaWUdz147m3EDWlSJ1OyFR84itWqrKxWMPK7tWGZ819VtvnSzWzZcJ8O1T7uLbCf9qmvq1DtDhIqMMTDnu67++xNXqmDJTe6zyTvT1QsPP7bjrVeaqMLrv4S3/uiq+c6+r/t3wjqgpdgGNi65g5H1n+3fUmzMKa411JAjAtcWHnL9VLTRlSKP/5lrTdbZY9idNFU5ffiIO1kPP/bgnfEOAZYgOqNgsbsYlzUMpl0Go06Af33fnZDWPgtn3NlmC4lte2sYmJV8YHLwlcEjZ8GOtRSM/xETR59w8Fhi49z1gfvnwqs/h9P+1Ik3FlBVCkv+07XnnnlN57fXlZIyYPrl7rH9I3eS+2Sxu14RnwqjjnO/mg87OfQqIX+jK1Wt+qsrnZx6G8RE4b04ckaxdcjZjJwzx11P+PwlV7L48GH44D5XvaKNkJIDR1wLUy+FnG7cm7+zmkriZh9LEB2163Mo/gTmNbtJXq/B8K2n4aNHXXXPPbPdr60Z3w36a2RHmY9+GS1+hfnK4dFvum2f/wi7i0Ko0x0w2fWsfvfPMPF8d8GyM17+kWvXfcZz0d3hZ8AU9zjl966l0fpAz+11/3InwSFHuJ7bY+e7ZN+Whjp45ruuZHXUje7z7Y7VbqFK6+N6vk+9GOqq3UXuLW+70saYU22srEOUlaE6au3TgLjqheZE3D/Z/3vPNV176WZ4aL5rEdFCUZmP/pnNEkRthWtDv/0j1z57zCmhxzXnR66a4Pnr3cmsoz5/xVVZHf0f3b9FSnvFJboWPKf9CW5c5/qOHHU9VJe6ZHj7JLj7SHj9V64/Q8t7pdRVwd8XuM/+xF/CCbf0jOTQUkKKS5jzfuuq5Cw5HLIsQXSEqqteGnZU680ZMwe6Fi1n3uuatd1zpBtPx98IgN+v7Cj30a8pQdRWwmPnuvFpznkQDv9Gx2JLTHP9FHath3du79g2fOXuonufsS5B9EQxMTAo35UArn0Pvv8RnPRrd+3mrT+6qrrbxsHzN7qespU74eEz3S/r0/8Ms78f6XdgjOeiuN4ggnYUuAHUZh2kXl4EJl/gWh+9cCO88lP49J9wxt2UJA2hwa+uBFFXBY+f7wa+++YDrrlgZxx2svvl9+b/uL+h1hu/9gso3waXv9L9L7yGS+8RrtnhkQvdtZcvXnZVUU0jzgLEJsC5f3P9TYw5BFiC6IiCxa7u+vB2nsgz+sOCx91Aay/+AO49isapNxBLHgNScdUWXy2Hs+8PXxPVebfChtddVdOl/2p/VciXy2HFA+6i9OAZ4Ykl2qRmw+QL3aOpH8GWt1y1SzdtjmiMFyxBhKqpemnkXHciaS8RN6je8GNgyX/Qf8XvWJwwglHv9oOid+Gs/3NDRoRLei6c+HNXVbT6cTe89sHU+9zwFb2GwHE/DV8s0ax5PwJjDjF2DSJU21a54X3Hf7Nj66fnwnmPsGzi7xksu0gtehfOvNvdTCfcpn7bDQn9yk/ccNEH88atULrBdYjrbh2+jDFdzhJEqAoWu7rosad2fBsivJd8LPMb/oheudRVZXghJsa12KmthJcP0r676GN3UXvytw7eKc8Yc0iwBBEKfyMUPA2jT3IDk3VCcVkN8Rl9iBno8X0c+h7umnKuWQQbXw++TGM9/PNad8+Bk3/lbTzGmKhhCSIUX70LlcVhuZB8QB8ILx19k7uP7/M3uouuLS2/03XMm/+Hrrl5izEmKliCCEXBYohPgcM6f8HS9YHoopEv45PgG/8LezbDG7/ff17JBlj2O3fPAGu+aYxpxhJEezXWuz4MY06BhNRObUpVu7YEAe5OVpMvguV3uOGMIXAToO+5BDL/D10XizEmKliCaK/Nb7ghGfI6X720t7qe2gb/geMwee2kX7lrJ/+6ziWHVQ+6/hcn/8bbIY2NMVHJEkR7FTwNiRluxNZOKirzAXRtCQLcqKUn/wYKV7ihqv99i+vlPbkdfSSMMYccSxDt0VDrRv4M0xj4xeXuQnG/rk4Q4EZ5HX4svH2bu1Vqd70JkDEm4ixBtMeGV6G2vOOd41r4ugQRgdszirgL1mn93J20Dja8tTHmkOVpghCReSLymYhsEJGbg8zPEpFnRGSNiHwgIuMD08eIyOpmj3IRud7LWNtUsBiSe7sLvWFQXOYjNkbokx6hgfCyR7rhrqd9JzL7N8ZEBc8ShIjEAncBpwDjgAtEZFyLxX4MrFbVicAlwO0AqvqZqk5W1clAPlANPONVrG2qq3I3lxl3BsTGh2WTRWU++qYnEhsTwaodu52iMeYgvDxLzAA2qOomVa0DFgEthz8dB7wGoKrrgWEikttimeOBjar6pYextu7zl6G+OmzVS+BKELld3YLJGGNCJNryrlnh2rDIOcA8Vb0i8PpiYKaqLmy2zG+AJFW9UURmAMsDy6xqtsyDwIeq+udW9nMVcBVAbm5u/qJFizoUb2VlJWlpBw5Ql1fwWzLKP+PdI/7ihvgOgx+9Vc3AtBgWTjl4kmgtrkizuEJjcYXG4gpNZ+KaO3fuKlWdFnSmqnryAM4FHmj2+mLgzhbLZAB/BVYDjwArgEnN5icAJUBue/aZn5+vHbV06dIDJ9bsVf3vPqpLftjh7QYz7r9e1J8/V9DxuLoBiys0FldoLK7QdCYuYKW2ck718n4QhcDgZq8HAdubL6Cq5cBlACIiwObAo8kpuNLDDg/jbN36JdBYG9bqpQpfPVV1jV3fB8IYY0Lk5TWIFcBoERkuIgnAAuC55guISK/APIArgDcDSaPJBcDfPYyxbQWLIXMIDApe+uqI4kAT1y4bh8kYYzrIswShqg3AQuBlYB3wpKquFZGrReTqwGKHA2tFZD2utHBd0/oikgKcCDztVYxtqip1N6gff3ZYO5JFrBe1McaEyNNbjqrqEmBJi2n3Nnv+LjC6lXWrgRDu6Rlm654Df0NYq5egWQnCWjEZY7o5awzfmoLFkD0a+k0I62abShDWzNUY091Zggimohi2vO1KD2Eep6i4vIactEQS4uzQG2O6NztLBbP2WUDDcue4lrr8PhDGGNNBliCCKVgMuROgz5iwb9p6URtjooUliJb2fAmFH3hSegArQRhjoocliJbWBsYE9CBB1NQ1UlZTH5n7QBhjTIgsQbRUsBgGTvPkPgnF5dYHwhgTPVpNECLy+2Yd2ppPv0FEbvU2rAgp+QKK14S970OTorII3knOGGNC1FYJ4hvAfUGm3w6c6k04EVbwNCCQd6Ynmy+O5J3kjDEmRG0lCFVVf5CJfqDn3cRYFQqegqGzIWOAJ7sosl7Uxpgo0laCqBaRA4bBCEyr8S6kyEit+hJKPves9RK4EkSvlHiSE8JzXwljjPFSW2Mx/Qx4UUR+BTTdwGca8CPgeo/j6nJ9d77lbgg0ruVN78KnqMxnpQdjTNRoNUGo6osicibwA+B7gckFwDdV9ZMuiK3rqLoEMWIOpOZ4tpvi8hprwWSMiRqtJggRSQJ2qOqlLab3FZEkVfV5Hl1X2fYhyb4dMP4WT3dTXOZjwsBMT/dhjDHh0tY1iDuAo4NMPxH4X2/CiZCCxfglDsZ61zirtqGRkso6+mVYCyZjTHRoK0EcpaoH3KxHVR8DjvEupC7m98PaZ9jdeyok9/JsNzvLawHrJGeMiR5tXaRuqylrz+mB3eCDqZewfXcC3l19aNbE1RKEMSZKtHWi3ykiM1pODEzb5V1IXSwhBeb+iN3Z+Z7uxobZMMZEm7ZKED8AnhSRh9i/meslwAKP4+pxim2YDWNMlGm1BKGqHwAzcVVN3waaWjNdiksSJgRFZT7SEuNIT4qPdCjGGNMubZUgUNUdwC0iMgW4AJccjgEWd0FsPUpxmc9KD8aYqNJWP4jDcFVJFwClwBOAqOrcLoqtR7EbBRljok1bF6nXA8cDp6nqUap6J9DYNWH1PMU2zIYxJsq0lSC+CRQDS0XkfhE5np44imsXaGj0s7PCShDGmOjS1kXqZ1T1fGAssAy4AcgVkXtE5KT2bFxE5onIZyKyQURuDjI/S0SeEZE1IvKBiIxvNq+XiDwlIutFZJ2IHBHyu+smdlXW4lfItQRhjIkiB+3wpqpVqvqYqn4DGASsBg442bckIrHAXcApwDjgAhEZ12KxHwOrVXUirmXU7c3m3Q68pKpjgUnAuoO/ne6pqMz6QBhjok9IPaJVdbeq/p+qHteOxWcAG1R1k6rWAYuAlmNpjwNeC2x7PTBMRHJFJAPXWuovgXl1qro3lFi7k+J9NwqycZiMMdFDVNWbDYucA8xT1SsCry8GZqrqwmbL/AZIUtUbAz20l+P6XjTibnf6Ka70sAq4TlWrguznKuAqgNzc3PxFixZ1KN7KykrS0tI6tO7BvLKlnsfX1/Hn41JISwjtMo6XcXWGxRUaiys0FldoOhPX3LlzV6nqtKAzVdWTB3Au8ECz1xcDd7ZYJgP4K67a6hFgBS4hTAMacAkFXHXTLw+2z/z8fO2opUuXdnjdg/n1C5/qYT9Zon6/P+R1vYyrMyyu0FhcobG4QtOZuICV2so5tc2Ocp1UCAxu9noQsL35AqpaDlwGICICbA48UoBCVX0/sOhTtOO6R3fV1AfCvUVjjIkOXo7KugIYLSLDRSQB1+nuueYLBFoqJQReXgG8qarlqloMbBWRMYF5x+Oqm6JScVmN9aI2xkQdz0oQqtogIguBl4FY4EFVXSsiVwfm3wscDjwsIo24BHB5s018D3gskEA2EShpRKOiMh/Th/WOdBjGGBMSL6uYUNUlwJIW0+5t9vxdYHQr667GXYuIan6/sqPcxmEyxkSfnnPjn26qtKqO+ka1PhDGmKhjCcJjTX0gcm0cJmNMlLEE4bGiwI2CrARhjIk2liA81nSrUbsGYYyJNpYgPFZU5iMuRshJTYx0KMYYExJLEB7bUeYjNyOJmBjrJGeMiS6WIDxmd5IzxkQrSxAeK7Y+EMaYKGUJwkOqSlFZjZUgjDFRyRKEh8pq6vHV++mXafeBMMZEH0sQHrI7yRljopklCA9ZL2pjTDSzBOEhK0EYY6KZJQgPFZfVECPQJ906yRljoo8lCA8Vlfnok55IfKwdZmNM9LEzl4dcHwhrwWSMiU6WIDxUXOajv12gNsZEKUsQHious17UxpjoZQnCIxW+eipqG6wFkzEmalmC8MgOuw+EMSbKWYLwyNd9IOwitTEmOlmC8EhTguhnF6mNMVHKEoRHmobZ6JthneSMMdHJEoRHisp8ZKcmkBQfG+lQjDGmQyxBeKS4rMYuUBtjopqnCUJE5onIZyKyQURuDjI/S0SeEZE1IvKBiIxvNm+LiHwiIqtFZKWXcXrBbjVqjIl2niUIEYkF7gJOAcYBF4jIuBaL/RhYraoTgUuA21vMn6uqk1V1mldxemWH3WrUGBPlvCxBzAA2qOomVa0DFgFntFhmHPAagKquB4aJSK6HMXUJX30je6rrrYmrMSaqiap6s2GRc4B5qnpF4PXFwExVXdhsmd8ASap6o4jMAJYHllklIpuBPYAC/6eq97Wyn6uAqwByc3PzFy1a1KF4KysrSUtL69C6Le2o8vPDt2q4ckICswfGd2pb4YwrnCyu0FhcobG4QtOZuObOnbuq1VoaVfXkAZwLPNDs9cXAnS2WyQD+CqwGHgFWAJMC8wYE/vYFPgaOOdg+8/PztaOWLl3a4XVbWr6hRIf+8Hl954tdnd5WOOMKJ4srNBZXaCyu0HQmLmCltnJOjetQymmfQmBws9eDgO3NF1DVcuAyABERYHPggapuD/zdKSLP4Kqs3vQw3rApLq8BbJgNY0x08/IaxApgtIgMF5EEYAHwXPMFRKRXYB7AFcCbqlouIqkikh5YJhU4CSjwMNaw2teL2hKEMSaKeVaCUNUGEVkIvAzEAg+q6loRuTow/17gcOBhEWkEPgUuD6yeCzzjChXEAY+r6ktexRpuxWU+MpLiSEnwsoBmjDHe8vQMpqpLgCUtpt3b7Pm7wOgg620CJnkZm5dcHwhrwWSMiW7Wk9oDdqMgY0xPYAnCA9aL2hjTE1iCCLO6Bj+lVbVWgjDGRD1LEGG2s8KHKlaCMMZEPUsQYVa8r4mrXaQ2xkQ3SxBh9vWtRq0EYYyJbpYgwqzYOskZY3oISxBhVlTmIzUhlvRE6yRnjIluliDCrLi8htzMJAK9wI0xJmpZgggz6wNhjOkpLEGEWXGZj34Z1oLJGBP9LEGEUUOjn50VtVaCMMb0CJYgwqikso5Gv1oLJmNMj2AJIoyKy60PhDGm57AEEUbFZXYnOWNMz2EJIoy+7kVtF6mNMdHPenOFUXGZj4S4GLJS4iMdijE9Sn19PYWFhfh8vojGkZmZybp16yIaQzDtiSspKYlBgwYRH9/+85MliDBq6gNhneSMCa/CwkLS09MZNmxYRP+/KioqSE9Pj9j+W3OwuFSV0tJSCgsLGT58eLu3a1VMYVRc5iM3w64/GBNuPp+P7Oxs+/HVQSJCdnZ2yCUwSxBhVFReYy2YjPGIJYfO6cjxswQRJn6/sqPM7iRnjOk5LEGEye7qOuoa/fS3KiZjTA9hCSJM7E5yxvRce/fu5e677w55vfnz57N3797wB9RFrBVTmBTbneSM6RK/+NdaPt1eHtZtjhuQwS2n5bU6vylBXHzxxftNb2xsJDY2ttX1lixZErYYI8HTEoSIzBORz0Rkg4jcHGR+log8IyJrROQDERnfYn6siHwkIs97GWc4FNkwG8b0WDfffDMbN25k9uzZTJ8+nblz53LhhRcyYcIEAM4880zy8/PJy8vjvvvu27fesGHDKCkpYcuWLRx++OFceeWV5OXlcdJJJ1FTU9Pq/u6//36mT5/OpEmT+OY3v0l1dTUAO3bs4KyzzmLSpElMmjSJ5cuXA/D4448zceJEJk2adEAS6xRV9eQBxAIbgRFAAvAxMK7FMv8D3BJ4PhZ4rcX8G4HHgefbs8/8/HztqKVLl3Z4XVXV37+0Tkf+6AVtaPR3ajstdTYur1hcobG4QtMyrk8//TQygQRs3rxZ8/LytLy8XJcuXaopKSm6adOmffNLS0tVVbW6ulrz8vK0pKREVVWHDh2qu3bt0s2bN2tsbKx+9NFHqqp67rnn6iOPPNLq/prWV1X9yU9+onfccYeqqp533nn6v//7v6qq2tDQoHv37tWCggIdNWqU7tq1a79Yggl2HIGV2so51csSxAxgg6puUtU6YBFwRotlxgGvBRLVemCYiOQCiMgg4FTgAQ9jDJuiQB+I2BhrimdMTzdjxoz9OpzdcccdTJo0iVmzZrF161a++OKLA9YZPnw4kydPBiA/P58tW7a0uv2CggKOPvpoJkyYwGOPPcbatWsBeP3117nmmmsAiI2NJTMzk9dff50zzzyTnJwcAHr37h2md+ntNYiBwNZmrwuBmS2W+Rg4G3hbRGYAQ4FBwA7gT8B/Am12WxSRq4CrAHJzc1m2bFmHgq2srOzwugDrttSQDJ3aRjCdjcsrFldoLK7QtIwrMzOTioqKiMbj9/tpbGykurqaxMTEffG89dZbvPzyy7zyyiukpKQwf/58du/eTUVFBapKZWUllZWVxMfH71unoaGBqqqqVt/TpZdeyuOPP74vQbz11lv7tldRUUFdXd2+ZWtqavZNPxifzxfS5+1lggj2U1pbvP4dcLuIrAY+AT4CGkTkG8BOVV0lInPa2omq3gfcBzBt2jSdM6fNxVu1bNkyOrouwH+vXMbhAzOYM2dqh7cRTGfj8orFFRqLKzQt41q3bl1Eh7jo378/VVVVxMbGkpKSQlxc3L546uvrycnJITc3l/Xr17NixQpSUlJIT09HREhLSwMgJiZm3zqJiYnU19e3+p4qKysZNWoUSUlJLF68mIEDB5Kens4JJ5zAo48+yvXXX09jYyNVVVWceuqpnHHGGfzoRz8iOzub3bt3t1qKSEpKYsqUKe1+315WMRUCg5u9HgRsb76Aqpar6mWqOhm4BOgDbAZmA6eLyBZc1dRxIvKoh7F2iqpSVOazTnLG9FDZ2dnMnj2bmTNn8oMf/GC/efPmzaOhoYGJEyfyX//1X8yaNavT+/vlL3/JzJkzOfHEExk7duy+6bfffjtLly5lwoQJ5Ofns3btWvLy8rjppps49thjmTRpEjfeeGOn99/EyxLECmC0iAwHtgELgAubLyAivYDqwDWKK4A3VbUc+FHgQaAEcZOqfsvDWDulvKaBmvpGa8FkTA/2+OOPBx0ULzExkRdffDHoOk3XGXJycigoKNg3/aabbmpzX9dcc82+aw3N5ebm8s9//vOA6RdddBFXX331wd5CyDxLEKraICILgZdxLZoeVNW1InJ1YP69wOHAwyLSCHwKXO5VPF4qKrcbBRljeh5PO8qp6hJgSYtp9zZ7/i4w+iDbWAYs8yC8sCmyTnLGmA649tpreeedd/abdt1113HZZZdFKKL9WU/qMNhhw2wYYzrgrrvuinQIbbKxmMKgqMyHCPRNT4x0KMYYEzaWIMKguMxHn7RE4mPtcBpjeg47o4VBUbnPrj8YY3ocSxBhUFxWYy2YjDE9jiWIMCgq89HPbhRkjAlo6j29fft2zjnnnKDLzJkzh5UrV3ZlWCGzVkydVFnbQIWvwVowGdNVXrwZij8J7zb7TYBTfhfebQIDBgzgqaeeCvt2u4qVIDrJbhRkTM/3wx/+cL87yv385z/nF7/4BccffzxTp05lwoQJQXs4b9myhfHj3W1uampqWLBgARMnTuT8889v834Q4HpTT5s2jby8PG655ZZ901esWMGRRx7JpEmTmDFjBhUVFTQ2NnLTTTcxYcIEJk6cyJ133hmW920liE76+lajliCM6RIe/NI/mAULFnD99dfvuxnPk08+yUsvvcQNN9xARkYGJSUlzJo1i9NPPx2R4EP+33PPPaSkpLBmzRrWrFnD1KltD+z561//mt69e9PY2Mjxxx/PmjVrGDt2LOeffz5PPPEE06dPp7y8nOTkZO688042b97MRx99RFxcHLt37w7L+7YE0UlFZe5XgJUgjOm5pkyZws6dOykqKmLTpk1kZWXRv39/brjhBt58801iYmLYtm0bO3bsoF+/fkG38eabb/L9738fgIkTJzJx4sQ29/nkk09y33330dDQQFFREZ9++ikiQv/+/Zk+fToAGRkZgBv9duHChcTFuVN6uO4JYQmik5pKELl2kdqYHu2cc87h2WefZe/evSxYsIDHHnuMXbt2sWrVKuLj4xk2bBg+n6/NbbRWumhp8+bN/OEPf2DFihVkZWXx7W9/G5/Ph6oG3UZr0zvLrkF0UnG5j96pCSTFt37jcmNM9FuwYAGLFy/mqaee4pxzzqGsrIy+ffsSHx/P0qVL+fLLL9tc/5hjjuGxxx4D3B3j1qxZ0+qy5eXlpKamkpmZyY4dO/aNFjt27Fi2b9/OihUrAKioqKChoYHjjjuOe++9l4aGBgCrYuouiq2JqzGHhLy8PCorKxk4cCD9+/fnoosu4rTTTmPatGlMnjx5v/s2BHPNNddw2WWXMXHiRCZPnsyMGTNaXXbSpElMmTKFvLw8RowYwezZswFISEjgiSee4Hvf+x41NTUkJyfz6quvcumll/LVV18xceJE4uPjufLKK1m4cGGn37MliE6aOKgXeQP8kQ7DGNMF3nvvvX33g8jJyeHdd98NulxlZSUAw4YN23cfiOTkZBYtWtTufT300ENBp0+fPp333ntvv2kVFRXcdttt3Hbbbe3efntYguik605oc7RyY4yJWpYgjDEmgmbOnEltbe1+0x555BEmTJgQoYi+ZgnCGBMVvGqpE2nvv/9+l+xHVUNex1oxGWO6vaSkJEpLSzt0kjMuOZSWlpKUFFqDGitBGGO6vUGDBlFYWMiuXbsiGofP5wv5JNsV2hNXUlISgwYNCmm7liCMMd1efHw8w4cPj3QYLFu2jClTpkQ6jAN4FZdVMRljjAnKEoQxxpigLEEYY4wJSnpSqwAR2QW0PSBK63KAkjCGEy4WV2gsrtBYXKHpiXENVdU+wWb0qATRGSKyUlWnRTqOliyu0FhcobG4QnOoxWVVTMYYY4KyBGGMMSYoSxBfuy/SAbTC4gqNxRUaiys0h1Rcdg3CGGNMUFaCMMYYE5QlCGOMMUEdUglCROaJyGciskFEbg4yX0TkjsD8NSIytYviGiwiS0VknYisFZHrgiwzR0TKRGR14PGzLopti4h8EtjnyiDzu/yYiciYZsdhtYiUi8j1LZbpkuMlIg+KyE4RKWg2rbeI/FtEvgj8zWpl3Ta/jx7E9T8isj7wOT0jIr1aWbfNz9yDuH4uItuafVbzW1m3q4/XE81i2iIiq1tZ18vjFfTc0GXfMVU9JB5ALLARGAEkAB8D41osMx94ERBgFvB+F8XWH5gaeJ4OfB4ktjnA8xE4bluAnDbmR+SYtfhci3Gdfbr8eAHHAFOBgmbTfg/cHHh+M3BrR76PHsR1EhAXeH5rsLja85l7ENfPgZva8Tl36fFqMf+PwM8icLyCnhu66jt2KJUgZgAbVHWTqtYBi4AzWixzBvCwOu8BvUSkv9eBqWqRqn4YeF4BrAMGer3fMInIMWvmeGCjqna0B32nqOqbwO4Wk88A/hZ4/jfgzCCrtuf7GNa4VPUVVW0IvHwPCG3sZ4/iaqcuP15NxN2l6Dzg7+HaX3u1cW7oku/YoZQgBgJbm70u5MCTcHuW8ZSIDAOmAMFuM3WEiHwsIi+KSF4XhaTAKyKySkSuCjI/0sdsAa3/40bieAHkqmoRuH9woG+QZSJ93L6DK/kFc7DP3AsLA1VfD7ZSXRLJ43U0sENVv2hlfpccrxbnhi75jh1KCSLYvQpbtvFtzzKeEZE0YDFwvaqWt5j9Ia4aZRJwJ/BsF4U1W1WnAqcA14rIMS3mR+yYiUgCcDrwjyCzI3W82iuSx+0nQAPwWCuLHOwzD7d7gJHAZKAIV53TUiT/Ny+g7dKD58frIOeGVlcLMi2kY3YoJYhCYHCz14OA7R1YxhMiEo/7Ajymqk+3nK+q5apaGXi+BIgXkRyv41LV7YG/O4FncMXW5iJ2zHD/kB+q6o6WMyJ1vAJ2NFWzBf7uDLJMRI6biFwKfAO4SAMV1S214zMPK1XdoaqNquoH7m9lf5E6XnHA2cATrS3j9fFq5dzQJd+xQylBrABGi8jwwC/PBcBzLZZ5Drgk0DJnFlDWVIzzUqCO8y/AOlW9rZVl+gWWQ0Rm4D67Uo/jShWR9KbnuIucBS0Wi8gxC2j1l10kjlczzwGXBp5fCvwzyDLt+T6GlYjMA34InK6q1a0s057PPNxxNb9mdVYr++vy4xVwArBeVQuDzfT6eLVxbuia75gXV9676wPX4uZz3JX9nwSmXQ1cHXguwF2B+Z8A07oorqNwRb81wOrAY36L2BYCa3EtEd4DjuyCuEYE9vdxYN/d6Zil4E74mc2mdfnxwiWoIqAe94vtciAbeA34IvC3d2DZAcCStr6PHse1AVcn3fQdu7dlXK195h7H9Ujgu7MGdwLr3x2OV2D6Q03fqWbLduXxau3c0CXfMRtqwxhjTFCHUhWTMcaYEFiCMMYYE5QlCGOMMUFZgjDGGBOUJQhjjDFBWYIwJgQi0ij7jyQbtlFFRWRY89FEjYm0uEgHYEyUqVHVyZEOwpiuYCUIY8IgcE+AW0Xkg8BjVGD6UBF5LTAQ3WsiMiQwPVfcPRk+DjyODGwqVkTuD4z9/4qIJEfsTZlDniUIY0KT3KKK6fxm88pVdQbwZ+BPgWl/xg2HPhE3ON4dgel3AG+oG0xwKq4XLsBo4C5VzQP2At/09N0Y0wbrSW1MCESkUlXTgkzfAhynqpsCg6sVq2q2iJTgho6oD0wvUtUcEdkFDFLV2mbbGAb8W1VHB17/EIhX1V91wVsz5gBWgjAmfLSV560tE0xts+eN2HVCE0GWIIwJn/Ob/X038Hw5bhRNgIuAtwPPXwOuARCRWBHJ6KogjWkv+3ViTGiSZf+b17+kqk1NXRNF5H3cD68LAtO+DzwoIj8AdgGXBaZfB9wnIpfjSgrX4EYTNabbsGsQxoRB4BrENFUtiXQsxoSLVTEZY4wJykoQxhhjgrIShDHGmKAsQRhjjAnKEoQxxpigLEEYY4wJyhKEMcaYoP4/LIfjty+iVjoAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import pandas as pd\n",
"\n",
"\n",
"metrics = pd.read_csv(f\"{trainer.logger.log_dir}/metrics.csv\")\n",
"\n",
"aggreg_metrics = []\n",
"agg_col = \"epoch\"\n",
"for i, dfg in metrics.groupby(agg_col):\n",
" agg = dict(dfg.mean())\n",
" agg[agg_col] = i\n",
" aggreg_metrics.append(agg)\n",
"\n",
"df_metrics = pd.DataFrame(aggreg_metrics)\n",
"df_metrics[[\"train_loss\", \"valid_loss\"]].plot(\n",
" grid=True, legend=True, xlabel='Epoch', ylabel='Loss')\n",
"df_metrics[[\"train_acc\", \"valid_acc\"]].plot(\n",
" grid=True, legend=True, xlabel='Epoch', ylabel='ACC')"
]
},
{
"cell_type": "markdown",
"id": "65863b5e",
"metadata": {},
"source": [
"- The `trainer` automatically saves the model with the best validation accuracy automatically for us, we which we can load from the checkpoint via the `ckpt_path='best'` argument; below we use the `trainer` instance to evaluate the best model on the test set:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "6fabd8a1",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Restoring states from the checkpoint path at logs/my-model/version_13/checkpoints/epoch=16-step=7292.ckpt\n",
"LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
"Loaded model weights from checkpoint at logs/my-model/version_13/checkpoints/epoch=16-step=7292.ckpt\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "037b802d27cc4baaa4c1a12a0995dd7a",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Testing: 0it [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃<span style=\"font-weight: bold\"> Test metric </span>┃<span style=\"font-weight: bold\"> DataLoader 0 </span>┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│<span style=\"color: #008080; text-decoration-color: #008080\"> test_acc </span>│<span style=\"color: #800080; text-decoration-color: #800080\"> 0.9815999865531921 </span>│\n",
"└───────────────────────────┴───────────────────────────┘\n",
"</pre>\n"
],
"text/plain": [
"┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
"┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n",
"┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
"│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.9815999865531921 \u001b[0m\u001b[35m \u001b[0m│\n",
"└───────────────────────────┴───────────────────────────┘\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"[{'test_acc': 0.9815999865531921}]"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trainer.test(model=lightning_model, datamodule=data_module, ckpt_path='best')"
]
},
{
"cell_type": "markdown",
"id": "4f447b27",
"metadata": {},
"source": [
"## Predicting labels of new data"
]
},
{
"cell_type": "markdown",
"id": "954ece41",
"metadata": {},
"source": [
"- You can use the `trainer.predict` method on a new `DataLoader` or `DataModule` to apply the model to new data.\n",
"- Alternatively, you can also manually load the best model from a checkpoint as shown below:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "6800950e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"logs/my-model/version_13/checkpoints/epoch=16-step=7292.ckpt\n"
]
}
],
"source": [
"path = trainer.checkpoint_callback.best_model_path\n",
"print(path)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "179f91a0",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"3:23: E703 statement ends with a semicolon\n"
]
}
],
"source": [
"pytorch_model = PyTorchCNN(num_classes=10)\n",
"lightning_model = LightningCNN.load_from_checkpoint(path, model=pytorch_model)\n",
"lightning_model.eval();"
]
},
{
"cell_type": "markdown",
"id": "4d5f4a56",
"metadata": {},
"source": [
"- Note that our PyTorch model, which is passed to the Lightning model requires input arguments. However, this is automatically being taken care of since we used `self.save_hyperparameters()` in our PyTorch model's `__init__` method.\n",
"- Now, below is an example applying the model manually. Here, pretend that the `test_dataloader` is a new data loader."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "cbdad102",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([7, 2, 1, 0, 4])"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_dataloader = data_module.test_dataloader()\n",
"\n",
"all_true_labels = []\n",
"all_predicted_labels = []\n",
"for batch in test_dataloader:\n",
" features, labels = batch\n",
" \n",
" with torch.no_grad(): # since we don't need to backprop\n",
" logits = lightning_model(features)\n",
" \n",
" predicted_labels = torch.argmax(logits, dim=1)\n",
" all_predicted_labels.append(predicted_labels)\n",
" all_true_labels.append(labels)\n",
" \n",
"all_predicted_labels = torch.cat(all_predicted_labels)\n",
"all_true_labels = torch.cat(all_true_labels)\n",
"all_predicted_labels[:5]"
]
},
{
"cell_type": "markdown",
"id": "05e1d31e",
"metadata": {},
"source": [
"Just as an internal check, if the model was loaded correctly, the test accuracy below should be identical to the test accuracy we saw earlier in the previous section."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "83afc1d0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test accuracy: 0.9816 (98.16%)\n"
]
}
],
"source": [
"test_acc = torch.mean((all_predicted_labels == all_true_labels).float())\n",
"print(f'Test accuracy: {test_acc:.4f} ({test_acc*100:.2f}%)')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
在新工单中引用 查看 Git Blame 复制永久链接