week04/optimization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Классификация и градиентные спуски\n",
    "\n",
    "В этой тетрадке мы попробуем немного посмотреть на то, как работают разные градиентные спуски. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import torch                     # Низкоуровневые штуки\n",
    "import torch.nn as nn            # Высокоуровневые штуки\n",
    "import torch.nn.functional as F  # Тоже высокоуровневые штуки, но с интерфейсом функций, а не классов"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Выборка\n",
    "\n",
    "Делать всё это мы будем на животных. Ежегодно около 7.6 миллионов бедных животных в США оказываются в приютах. Часть из них находит себе новую семью, часть возвращается к старому (бывает, что питомец потерялся и его нашли на улице), а часть погибает. Ужегодно усыпляется около 2.7 млн. собак и кошек.  \n",
    "\n",
    "Используя датасет с входной информацией (цвет, пол, возраст и т.п.) из одного из приютов, мы попытаемся спрогнозировать что произойдёт с новыми животными, которые попадут в этот приют. Данные, используемые в тетрадке уже были предварительно обработаны и приведены в удобную для построения моделей форму. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_path = Path('X_cat.csv')\n",
    "y_path = Path('y_cat.csv')\n",
    "\n",
    "if not X_path.exists():\n",
    "    !wget -q https://github.com/dniku/neural_nets_dpo/raw/master/week04/X_cat.csv -O $X_path\n",
    "\n",
    "if not y_path.exists():\n",
    "    !wget -q https://github.com/dniku/neural_nets_dpo/raw/master/week04/y_cat.csv -O $y_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = pd.read_csv('X_cat.csv', sep = '\\t', index_col=0)\n",
    "target = pd.read_csv('y_cat.csv', sep = '\\t', index_col=0, header=None, names=['status'])['status']\n",
    "\n",
    "print(X.shape)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В датасете находится около 27 тысяч наблюдений и 37 фичей. Посмотрим на то как выглядит распределение того, что произошло со зверятами по особям."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Видим, что классы несбалансированы. Попробуем оставить четыре класса и объединить класс умерших животных с классом животных, которых усыпили. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target[target == 'Died'] = 'Euthanasia'\n",
    "\n",
    "target.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Закодируем классы."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "le = LabelEncoder()\n",
    "y = le.fit_transform(target)\n",
    "\n",
    "print(dict(zip(range(len(le.classes_)), le.classes_)))\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разобьём выборку на тренировочную и тестовую. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)\n",
    "\n",
    "print(X_train.shape)\n",
    "print(X_valid.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Отшкалируем данные. Если это не сделать, модель будет учиться хуже."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "ss = StandardScaler()\n",
    "\n",
    "X_train = ss.fit_transform(X_train)\n",
    "X_valid = ss.transform(X_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Архитектурка"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Соберём что-то простенькое и более-менее с потолка."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_new_model():\n",
    "    dim_in = X_train.shape[1]\n",
    "    dim_out = len(le.classes_)\n",
    "    \n",
    "    model = <YOUR CODE>\n",
    "    \n",
    "    return model\n",
    "\n",
    "print(make_new_model())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Данные"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Завернём датасет в даталоадер, чтобы бесплатно получить перемешивание и нарезку на батчи."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepare_data(X, y, batch_size=1000, shuffle=False):\n",
    "    X_torch = torch.tensor(X.values if isinstance(X, pd.DataFrame) else X, dtype=torch.float32)\n",
    "    y_torch = torch.tensor(y, dtype=torch.int64)\n",
    "    \n",
    "    dataset = torch.utils.data.TensorDataset(X_torch, y_torch)\n",
    "    loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=shuffle)\n",
    "    \n",
    "    return X_torch, y_torch, dataset, loader\n",
    "\n",
    "X_train_torch, y_train_torch, train_dataset, train_loader = prepare_data(X_train, y_train, shuffle=True)\n",
    "X_valid_torch, y_valid_torch, valid_dataset, valid_loader = prepare_data(X_valid, y_valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Training loop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Решаем задачу классификации, поэтому в качестве лосса берём кросс-энтропию. Если будет время, в следующий раз напишем `CrossEntropyLoss` самостоятельно."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Всё как в прошлый раз."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(model, opt, criterion, train_loader, valid_loader, num_epochs):\n",
    "    # Заводим словарь, куда будем писать логи\n",
    "    history = {'loss_train': [], 'loss_valid': []}\n",
    "    \n",
    "    with tqdm(range(num_epochs)) as progress_bar:\n",
    "        for epoch in progress_bar:  # цикл по эпохам\n",
    "            epoch_losses_train = []\n",
    "            epoch_losses_valid = []\n",
    "\n",
    "            for X_batch, y_batch in train_loader:  # цикл по train-датасету (его за нас перемешивает DataLoader)\n",
    "                logits = <YOUR CODE>\n",
    "                loss = <YOUR CODE>\n",
    "                \n",
    "                # Посчитайте градиенты и сделайте шаг оптимизатора (он передаётся в функцию параметром opt)\n",
    "                <YOUR CODE>\n",
    "\n",
    "                # .item() конвертирует тензор из одного элемента в питоновское число\n",
    "                epoch_losses_train.append(loss.item())\n",
    "\n",
    "            with torch.no_grad():  # отключаем построение вычислительного графа на время валидации\n",
    "                for X_batch, y_batch in valid_loader:  # цикл по valid-датасету\n",
    "                    logits = <YOUR CODE>\n",
    "                    loss = <YOUR CODE>\n",
    "\n",
    "                    epoch_losses_valid.append(loss.item())\n",
    "            \n",
    "            # Записываем логи\n",
    "            history['loss_train'].append(np.mean(epoch_losses_train))\n",
    "            history['loss_valid'].append(np.mean(epoch_losses_valid))\n",
    "            \n",
    "            # Обновляем прогресс-бар\n",
    "            progress_bar.set_postfix_str(\n",
    "                f'Train loss: {history[\"loss_train\"][-1]:.3f}, ' +\n",
    "                f'Validation loss: {history[\"loss_valid\"][-1]:.3f}')\n",
    "\n",
    "    return history"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Эксперименты!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_epochs = 500\n",
    "learning_rate = 0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "histories = {}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Функция для графиков:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_history(histories):\n",
    "    plt.figure(figsize=(16, 10))\n",
    "\n",
    "    for name, history in histories.items():\n",
    "        train = plt.plot(history['loss_train'], label=f'{name} train')\n",
    "        plt.plot(history['loss_valid'], color=train[0].get_color(), linestyle='--', label=f'{name} valid')\n",
    "\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.ylabel('Log loss')\n",
    "    plt.legend()\n",
    "    plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SGD "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как и раньше, используем класс `torch.optim.SGD`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "histories['SGD'] = train(model, opt, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SGD with momentum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Снова используем `torch.optim.SGD`, но указываем параметр `momentum`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "histories['Momentum'] = train(model, opt, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RMSprop "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Понадобится класс `torch.optim.RMSprop`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "histories['RMSProp'] = train(model, opt, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adam "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Наконец, попробуем `torch.optim.Adam`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "histories['Adam'] = train(model, opt, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Стратегии с постепенным понижением lr \n",
    "\n",
    "![](https://raw.githubusercontent.com/FUlyankin/neural_nets_econ/master/2019/sem_2/ahaha.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем уменьшать learning rate ступеньками: например, в 2 раза каждые 50 эпох. Для этого нам понадобится класс `torch.optim.lr_scheduler.StepLR`. Будем использовать его вместе с Адамом."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Добавили параметр scheduler\n",
    "def train(model, opt, scheduler, criterion, train_loader, valid_loader, num_epochs):\n",
    "    history = {'loss_train': [], 'loss_valid': [], 'lr': []}  # будем записывать lr\n",
    "    with tqdm(range(num_epochs)) as progress_bar:\n",
    "        for epoch in progress_bar:\n",
    "            epoch_losses_train = []\n",
    "            epoch_losses_valid = []\n",
    "\n",
    "            for X_batch, y_batch in train_loader:\n",
    "                logits = model(X_batch)\n",
    "                loss = criterion(logits, y_batch)\n",
    "\n",
    "                opt.zero_grad()\n",
    "                loss.backward()\n",
    "                opt.step()\n",
    "                \n",
    "                epoch_losses_train.append(loss.item())\n",
    "\n",
    "            with torch.no_grad():\n",
    "                for X_batch, y_batch in valid_loader:\n",
    "                    logits = model(X_batch)\n",
    "                    loss = criterion(logits, y_batch)\n",
    "\n",
    "                    epoch_losses_valid.append(loss.item())\n",
    "                    \n",
    "            history['loss_train'].append(np.mean(epoch_losses_train))\n",
    "            history['loss_valid'].append(np.mean(epoch_losses_valid))\n",
    "            \n",
    "            # вызываем scheduler\n",
    "            scheduler.step()\n",
    "            \n",
    "            # записываем lr\n",
    "            history['lr'].append(opt.param_groups[0]['lr'])\n",
    "\n",
    "            progress_bar.set_postfix_str(\n",
    "                f'Train loss: {history[\"loss_train\"][-1]:.3f}, ' +\n",
    "                f'Validation loss: {history[\"loss_valid\"][-1]:.3f}, ' +\n",
    "                f'LR: {history[\"lr\"][-1]:.5f}')  # показываем lr\n",
    "\n",
    "    return history\n",
    "\n",
    "\n",
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "scheduler = <YOUR CODE>\n",
    "histories['Adam + StepLR'] = train(model, opt, scheduler, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# функция для картинок, чтобы видеть как скорость обучения меняется от эпохи к эпохе\n",
    "def plot_learning_rate(loss_history):\n",
    "    epochs = len(loss_history)\n",
    "    plt.plot(range(1, epochs + 1), loss_history, label='learning rate')\n",
    "    plt.xlabel(\"epoch\")\n",
    "    plt.xlim([1, epochs + 1])\n",
    "    plt.ylabel(\"learning rate\")\n",
    "    plt.legend(loc=0)\n",
    "    plt.grid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_learning_rate(histories['Adam + StepLR']['lr'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем ещё вариант: `torch.optim.lr_scheduler.ReduceLROnPlateau`. Этот класс умножает learning rate на параметр `factor`, когда в течение `patience` эпох лосс на валидации не уменьшается."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(model, opt, scheduler, criterion, train_loader, valid_loader, num_epochs):\n",
    "    history = {'loss_train': [], 'loss_valid': [], 'lr': []}\n",
    "    with tqdm(range(num_epochs)) as progress_bar:\n",
    "        for epoch in progress_bar:\n",
    "            epoch_losses_train = []\n",
    "            epoch_losses_valid = []\n",
    "\n",
    "            for X_batch, y_batch in train_loader:\n",
    "                logits = model(X_batch)\n",
    "                loss = criterion(logits, y_batch)\n",
    "\n",
    "                opt.zero_grad()\n",
    "                loss.backward()\n",
    "                opt.step()\n",
    "                \n",
    "                epoch_losses_train.append(loss.item())\n",
    "\n",
    "            with torch.no_grad():\n",
    "                for X_batch, y_batch in valid_loader:\n",
    "                    logits = model(X_batch)\n",
    "                    loss = criterion(logits, y_batch)\n",
    "\n",
    "                    epoch_losses_valid.append(loss.item())\n",
    "                    \n",
    "            history['loss_train'].append(np.mean(epoch_losses_train))\n",
    "            history['loss_valid'].append(np.mean(epoch_losses_valid))\n",
    "                    \n",
    "            scheduler.step(history['loss_valid'][-1])  # вызываем scheduler от валидационного лосса\n",
    "\n",
    "            history['lr'].append(opt.param_groups[0]['lr'])\n",
    "\n",
    "            progress_bar.set_postfix_str(\n",
    "                f'Train loss: {history[\"loss_train\"][-1]:.3f}, ' +\n",
    "                f'Validation loss: {history[\"loss_valid\"][-1]:.3f}, ' +\n",
    "                f'LR: {history[\"lr\"][-1]:.5f}')\n",
    "\n",
    "    return history\n",
    "\n",
    "\n",
    "model = make_new_model()\n",
    "opt = <YOUR CODE>\n",
    "scheduler = <YOUR CODE>\n",
    "histories['Adam + ReduceLROnPlateau'] = train(\n",
    "    model, opt, scheduler, criterion, train_loader, valid_loader, num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_history(histories)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_learning_rate(histories['Adam + ReduceLROnPlateau']['lr'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Что дальше?\n",
    "\n",
    "* Другие расписания изменения скорости обучения (например, циклически меняющееся; для вдохновения см., например, [эту статью](https://www.jeremyjordan.me/nn-learning-rate/))\n",
    "* Другая архитектура сетки (добавить слои или сделать глубже)\n",
    "* ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Авторские права и почиташки \n",
    "\n",
    "Ноутбук основан на [ноутбуке](https://github.com/FUlyankin/neural_nets_dpo/blob/e296fc1/week03_grad/Keras_SGD_experiments_semisolve.ipynb) от Филиппа Ульянкина, который для его создания использовал [этот мануал](https://github.com/sukilau/Ziff-deep-learning/blob/master/3-CIFAR10-lrate/CIFAR10-lrate.ipynb)."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}