Day4_20230308_Cell_Classification_Using_Deep_Learning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hiroalchem/data_science_lecture_2023/blob/main/Day4_20230308_Cell_Classification_Using_Deep_Learning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HPio0OjPnu2X"
      },
      "source": [
        "# 今日の取り組み   \n",
        "今日はTensorFlowを用いた基本的な画像分類を行ってみます\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B0aEQ0jsml_q"
      },
      "outputs": [],
      "source": [
        "# まずデータをダウンロードして解凍します\n",
        "import gdown\n",
        "gdown.download('https://drive.google.com/uc?id=1ftQvsAllb457U_bWKVIOLHhr_fr31sRu', 'CellCycle_for_classification.zip', quiet=False)\n",
        "!unzip CellCycle_for_classification.zip"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8234geiFoAhb"
      },
      "outputs": [],
      "source": [
        "# lectureのディレクトリに移動します\n",
        "%cd /content/CellCycle_for_classification"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XGVPG90N1AwM"
      },
      "outputs": [],
      "source": [
        "!pwd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fjMLmbdZnw2p"
      },
      "source": [
        "# **0. 準備**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gKHufjJ21Lke"
      },
      "source": [
        "## 0-1: 必要なライブラリを読み込みます"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install tensorflow==2.8.3"
      ],
      "metadata": {
        "id": "ZMDBk-jRsldA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ww1tXvALn6SD"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from pathlib import Path\n",
        "\n",
        "from skimage import io\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras.preprocessing import image_dataset_from_directory\n",
        "from tensorflow.keras.utils import plot_model\n",
        "from sklearn.metrics import confusion_matrix, classification_report"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EuYl61pm3owd"
      },
      "source": [
        "## 0-2:  フォルダ構造を確認します\n",
        "train, val, testのフォルダがあり、その中にG1, G2, Prophaseのフォルダがあることを確認します。    \n",
        "Day4    \n",
        "&emsp;  |- train   \n",
        "&emsp;  |&emsp;    |-G1   \n",
        "&emsp;  |&emsp;    |-G2  \n",
        "&emsp;  |&emsp;    |-Prophase  \n",
        "&emsp;  |   \n",
        "&emsp;  |- val   \n",
        "&emsp;  |&emsp;    |-G1   \n",
        "&emsp;  |&emsp;    |-G2  \n",
        "&emsp;  |&emsp;    |-Prophase  \n",
        "&emsp;  |   \n",
        "&emsp;  |- test   \n",
        "&emsp;   &emsp;    |-G1   \n",
        "&emsp;   &emsp;    |-G2  \n",
        "&emsp;   &emsp;    |-Prophase  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UT68T0Yp3d4n"
      },
      "outputs": [],
      "source": [
        "# train, validation, testフォルダのパスを変数に格納しておきます\n",
        "root_dir = ('./')\n",
        "train_dir = os.path.join(root_dir, 'train')\n",
        "val_dir = os.path.join(root_dir, 'val')\n",
        "test_dir = os.path.join(root_dir, 'test')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vxHn5cFdjV2h"
      },
      "source": [
        "# **1. データとモデルの準備**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rD9DlLg9jISW"
      },
      "source": [
        "## 1-1: データをTensorFlowが使える形に読み込みます\n",
        "Tensorflowは大きく2つの方法を用意してくれています。   \n",
        "・ フォルダから直接読み込む   \n",
        "・ 先に画像を別のライブラリで読み込んでおいてからTensorFlowの形式に変換する   \n",
        "今回は前者でやってみます\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HGBfECtpHlDa"
      },
      "outputs": [],
      "source": [
        "# まずバッチサイズと画像サイズを指定します。\n",
        "# バッチサイズとは一度にモデルに渡す画像の枚数で、これが大きいほど学習は安定します (epochごとに渡す枚数ではなく、epoch内で小分けにする画像の枚数です)\n",
        "# バッチサイズはメモリが許す限り大きい方が望ましいです。今回は32にしてみます。\n",
        "\n",
        "BATCH_SIZE = 32\n",
        "\n",
        "# 画像サイズはリサイズした後のサイズになります。今回は元画像が 66x66になっているのでそのまま66x66で使用します。\n",
        "\n",
        "IMG_SIZE = (66, 66)\n",
        "\n",
        "\n",
        "\n",
        "train_dataset = image_dataset_from_directory(train_dir,\n",
        "                                             shuffle=True,\n",
        "                                             batch_size=BATCH_SIZE,\n",
        "                                             image_size=IMG_SIZE,\n",
        "                                             label_mode='categorical')\n",
        "val_dataset = image_dataset_from_directory(val_dir,\n",
        "                                             shuffle=True,\n",
        "                                             batch_size=BATCH_SIZE,\n",
        "                                             image_size=IMG_SIZE,\n",
        "                                             label_mode='categorical')\n",
        "test_dataset = image_dataset_from_directory(test_dir,\n",
        "                                             shuffle=False,\n",
        "                                             batch_size=1,\n",
        "                                             image_size=IMG_SIZE,\n",
        "                                             label_mode='categorical')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZoB-YQXl6eY"
      },
      "source": [
        "## 1-2: 画像のチェック"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j4X8Ouo3m0N4"
      },
      "outputs": [],
      "source": [
        "# class名を取得します\n",
        "class_names = train_dataset.class_names\n",
        "print(class_names)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZbRehf6LHv1E"
      },
      "outputs": [],
      "source": [
        "# 学習用データセットから9枚表示してみます。\n",
        "plt.figure(figsize=(10, 10)) # 描画するfigの準備\n",
        "for images, labels in train_dataset.take(1): # データセットからバッチを取り出します\n",
        "  for i in range(9):\n",
        "    ax = plt.subplot(3, 3, i + 1) # 3x3マスのどこに描画するか指定\n",
        "    plt.imshow(images[i].numpy().astype(\"uint8\")) # 画像をnumpy形式に変換し、かつuint8に変換して表示\n",
        "    plt.title(class_names[np.argmax(labels[i])]) # その画像のラベルを取得する\n",
        "    plt.axis(\"off\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4IY_4ryqQWTd"
      },
      "outputs": [],
      "source": [
        "# 学習時間を短縮するために、学習時のデータの読み込みを並列化させる設定を行なっておきます。\n",
        "AUTOTUNE = tf.data.AUTOTUNE\n",
        "\n",
        "train_dataset = train_dataset.prefetch(buffer_size=AUTOTUNE)\n",
        "val_dataset = val_dataset.prefetch(buffer_size=AUTOTUNE)\n",
        "test_dataset = test_dataset.prefetch(buffer_size=AUTOTUNE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ruRAa0f_oHRy"
      },
      "source": [
        "## 1-3: 画像のaugmentation (水増し) の準備\n",
        "深層学習ではaugmentationという手法を使ってデータの水増しを行います。   \n",
        "データになんらかの画像処理を加えて見た目を変えた画像を学習に追加することで、モデルの汎化性能を向上させることができます。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3fty9zSCIuIN"
      },
      "outputs": [],
      "source": [
        "# ここではhorizontal flipとrotationを追加してみます\n",
        "data_augmentation = tf.keras.Sequential([\n",
        "  tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),\n",
        "  tf.keras.layers.experimental.preprocessing.RandomRotation(0.2),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hIRUDH-zQn9t"
      },
      "outputs": [],
      "source": [
        "# augmentationの結果を確認してみます\n",
        "for image, _ in train_dataset.take(1): \n",
        "  plt.figure(figsize=(10, 10))\n",
        "  first_image = image[0] # 最初の画像を取り出します\n",
        "  for i in range(9):\n",
        "    ax = plt.subplot(3, 3, i + 1)\n",
        "    augmented_image = data_augmentation(tf.expand_dims(first_image, 0)) # 最初の画像をaugmentationにかけて結果を得ます\n",
        "    plt.imshow(augmented_image[0] / 255) # 結果を表示\n",
        "    plt.axis('off')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xrEPV42700ZY"
      },
      "source": [
        "## 1-4: モデルの準備\n",
        "今回はResNet50というモデルを使用します。   \n",
        "ResNetは古くからあるモデルですが現在も改良されながら使われています\b。   \n",
        "検索するとたくさんの情報が得られるので、興味のある方は調べてみてください。   "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-w2t1NVMRazJ"
      },
      "outputs": [],
      "source": [
        "# まずResNet50を呼び出します。\n",
        "# TensorFlow (Keras) では、いくつかのモデルが簡単に使えるようになっています。\n",
        "# 入力する画像の形状の準備します。\n",
        "IMG_SHAPE = IMG_SIZE + (3,)\n",
        "print(IMG_SHAPE)\n",
        "\n",
        "# モデルをbase_modelとして準備します。\n",
        "# weights='imagenet' とすることで、imagenetで事前学習された重みを取得できます。\n",
        "base_model = tf.keras.applications.ResNet50(input_shape=IMG_SHAPE,\n",
        "                                               include_top=False,\n",
        "                                               weights='imagenet')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "laoetPu5R9yZ"
      },
      "outputs": [],
      "source": [
        "# base_modelを通して出てくる特徴量の形状を確認してみましょう。\n",
        "image_batch, label_batch = next(iter(train_dataset))\n",
        "feature_batch = base_model(image_batch)\n",
        "print(feature_batch.shape)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fqkvH4NCSIgX"
      },
      "outputs": [],
      "source": [
        "# modelのsummaryを表示することもできます。\n",
        "base_model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pVykHQ9l5KGX"
      },
      "outputs": [],
      "source": [
        "# モデル構造をみやすいプロットにすることもできます。\n",
        "plot_model(base_model,show_shapes=True, to_file='base_model.png')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WLmx3Dz11VfL"
      },
      "outputs": [],
      "source": [
        "# 次に前処理を準備します。\n",
        "# 画像の前処理としてimagenetで学習された際のresnet50の前処理を取得します。\n",
        "preprocess_input = tf.keras.applications.resnet50.preprocess_input"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dUIXtkBhSWjF"
      },
      "outputs": [],
      "source": [
        "# 最終的な出力までモデルを構築します。\n",
        "inputs = tf.keras.Input(shape=(66, 66, 3)) # inputの指定\n",
        "x = data_augmentation(inputs) # まずaugmentationをします。\n",
        "x = preprocess_input(x) # 前処理をします。\n",
        "x = base_model(x, training=False) # base_modelを通します。\n",
        "x = tf.keras.layers.GlobalAveragePooling2D()(x) # gobal average poolingという手法で3x3x2048次元を 2048次元に落とします。\n",
        "x = tf.keras.layers.Dropout(0.2)(x) # dropoutを追加します。\n",
        "outputs = tf.keras.layers.Dense(3, activation='softmax')(x) # 最後にクラス数と同じ3次元まで落とし、softmax関数を通します。\n",
        "model = tf.keras.Model(inputs, outputs) # モデルのinputとoutputを指定して終了です。"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 最終的なモデルを確認しましょう。\n",
        "plot_model(model,show_shapes=True, to_file='model.png')"
      ],
      "metadata": {
        "id": "S6oQHwGr8J3_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# まずbase_modelは学習をさせず、追加した部分だけを学習させてみます。\n",
        "base_model.trainable = False"
      ],
      "metadata": {
        "id": "orHxsJMj8J6P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V2OWnywhSg-K"
      },
      "outputs": [],
      "source": [
        "# 学習率を指定\n",
        "base_learning_rate = 0.0001\n",
        "\n",
        "# モデルをコンパイルする必要があります。\n",
        "# 最適化アルゴリズムはAdamにしてみます。こ\n",
        "# 損失関数は Categorical Crossentropyです。これは3クラス以上の分類に使用する基本的な損失関数です。\n",
        "# metricsは評価のために使用する評価関数です。今回はaccuracyを指定してみましょう。\n",
        "model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),\n",
        "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),\n",
        "              metrics=['accuracy'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f6W35CQrTOnx"
      },
      "outputs": [],
      "source": [
        "# 最後にモデルをチェックします。\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JhWA9y4sTRTr"
      },
      "outputs": [],
      "source": [
        "# 学習に使用するパラメータ (層の数) をチェック\n",
        "len(model.trainable_variables)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **2. 学習と評価**"
      ],
      "metadata": {
        "id": "o45pFLkhCANh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2-1 準備したモデルを学習させてみます"
      ],
      "metadata": {
        "id": "6Msc_W28CEVb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# まず10epoch学習させてみましょう。\n",
        "initial_epochs = 10"
      ],
      "metadata": {
        "id": "POfY1SqHBV71"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ud0a9jIjTViN"
      },
      "outputs": [],
      "source": [
        "# 学習前のlossとaccuracyを調べます。\n",
        "# 学習前なのでvalidtaion datasetに対するaccuracyはほぼランダムな場合の数値になると思います。\n",
        "\n",
        "loss0, accuracy0 = model.evaluate(val_dataset)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-kGbhObjTeWj"
      },
      "outputs": [],
      "source": [
        "# では学習させましょう。\n",
        "# 学習結果はhistoryに保存されます。\n",
        "history = model.fit(train_dataset, # train datasetの指定\n",
        "                    epochs=initial_epochs, # 学習するepoch数の指定\n",
        "                    validation_data=val_dataset) # validation datasetの指定"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r7MNZTvgViEC"
      },
      "outputs": [],
      "source": [
        "# 学習結果をプロットしてみます。\n",
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.subplot(2, 1, 1)\n",
        "plt.plot(acc, label='Training Accuracy')\n",
        "plt.plot(val_acc, label='Validation Accuracy')\n",
        "plt.legend(loc='lower right')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.ylim([min(plt.ylim()),1])\n",
        "plt.title('Training and Validation Accuracy')\n",
        "\n",
        "plt.subplot(2, 1, 2)\n",
        "plt.plot(loss, label='Training Loss')\n",
        "plt.plot(val_loss, label='Validation Loss')\n",
        "plt.legend(loc='upper right')\n",
        "plt.ylabel('Cross Entropy')\n",
        "plt.ylim([0,2.0])\n",
        "plt.title('Training and Validation Loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2-2: テストデータで評価します"
      ],
      "metadata": {
        "id": "A7dxYMVkC43D"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6dV2NSoTn8M1"
      },
      "outputs": [],
      "source": [
        "# test_datasetから画像とlabelを取り出して、モデルで推論させます。\n",
        "predictions = np.array([])\n",
        "labels =  np.array([])\n",
        "for x, y in test_dataset:\n",
        "  predictions = np.concatenate([predictions, np.argmax(model.predict(x), axis = -1)])\n",
        "  labels = np.concatenate([labels, np.argmax(y.numpy(), axis=-1)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xFGvGqk3n8PI"
      },
      "outputs": [],
      "source": [
        "# 混同行列を表示します\n",
        "cm = confusion_matrix(labels, predictions)\n",
        "pd.DataFrame(cm,columns=[\"pred_\" + str(n) for n in class_names], index=[\"GT_\" + str(n) for n in class_names])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "g8XlIrhZn8RW"
      },
      "outputs": [],
      "source": [
        "# 行ごとに割合にして表示してみましょう。\n",
        "pd.DataFrame(cm,columns=[\"pred_\" + str(n) for n in class_names], index=[\"GT_\" + str(n) for n in class_names]).apply(lambda x:x/sum(x),axis=1)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 次に各々のクラスについて、precision, recall, f1-scoreを表示してみましょう。\n",
        "print(classification_report(labels, predictions, target_names=class_names))"
      ],
      "metadata": {
        "id": "ltSdOZiFGk2i"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2-3: 畳み込み層も学習させてみる\n",
        "ここまではResNetの中は学習させませんでした。   \n",
        "今度はResNetの一部の層も学習させてみましょう。"
      ],
      "metadata": {
        "id": "lRQtV_67IHto"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gOG3qw-CX3Rd"
      },
      "outputs": [],
      "source": [
        "# base_modelを学習可能にします。\n",
        "base_model.trainable = True\n",
        "print(\"base_modelの層の数は \", len(base_model.layers))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 最後の10層だけ学習せてみます。\n",
        "fine_tune_at = -10\n",
        "\n",
        "# 最後の10層よりも前の層は学習させません。\n",
        "for layer in base_model.layers[:fine_tune_at]:\n",
        "  layer.trainable = False\n",
        "for layer in base_model.layers[fine_tune_at:]:\n",
        "  print(layer, layer.trainable)"
      ],
      "metadata": {
        "id": "XhLTfdPoIfIO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o9a-7gHOh5cM"
      },
      "outputs": [],
      "source": [
        "# コンパイルします。\n",
        "model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),\n",
        "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),\n",
        "              metrics=['accuracy'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UAi5jApOh5kS"
      },
      "outputs": [],
      "source": [
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b7usMsUdiiS7"
      },
      "outputs": [],
      "source": [
        "len(model.trainable_variables)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 10epoch追加で学習させます。\n",
        "fine_tune_epochs = 10\n",
        "\n",
        "# 前回のepoch数と足します。\n",
        "# モデルの総epoch数は20になります。\n",
        "total_epochs =  initial_epochs + fine_tune_epochs\n",
        "print(total_epochs)"
      ],
      "metadata": {
        "id": "an88QzzvJJHA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "T8Jgcm41ilOz"
      },
      "outputs": [],
      "source": [
        "# 学習\n",
        "history_fine = model.fit(train_dataset,\n",
        "                         epochs=total_epochs, # epoch数はtotal epochsです。\n",
        "                         initial_epoch=history.epoch[-1], # 前回の学習結果から再開するため、ここでhistoryに保存されているepoch数を呼び出します。\n",
        "                         validation_data=val_dataset)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Cu8RIQ5cm-01"
      },
      "outputs": [],
      "source": [
        "acc_fine = acc + history_fine.history['accuracy']\n",
        "val_acc_fine = val_acc + history_fine.history['val_accuracy']\n",
        "\n",
        "loss_fine = loss + history_fine.history['loss']\n",
        "val_loss_fine = val_loss + history_fine.history['val_loss']\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.subplot(2, 1, 1)\n",
        "plt.plot(acc_fine, label='Training Accuracy')\n",
        "plt.plot(val_acc_fine, label='Validation Accuracy')\n",
        "plt.legend(loc='lower right')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.ylim([min(plt.ylim()),1])\n",
        "plt.title('Training and Validation Accuracy')\n",
        "\n",
        "plt.subplot(2, 1, 2)\n",
        "plt.plot(loss_fine, label='Training Loss')\n",
        "plt.plot(val_loss_fine, label='Validation Loss')\n",
        "plt.legend(loc='upper right')\n",
        "plt.ylabel('Cross Entropy')\n",
        "plt.ylim([0,2.0])\n",
        "plt.title('Training and Validation Loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# test_datasetから画像とlabelを取り出して、モデルで推論させます。\n",
        "predictions = np.array([])\n",
        "labels =  np.array([])\n",
        "for x, y in test_dataset:\n",
        "  predictions = np.concatenate([predictions, np.argmax(model.predict(x), axis = -1)])\n",
        "  labels = np.concatenate([labels, np.argmax(y.numpy(), axis=-1)])"
      ],
      "metadata": {
        "id": "ga9aOIjmKT2J"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 混同行列を表示します\n",
        "cm = confusion_matrix(labels, predictions)\n",
        "pd.DataFrame(cm,columns=[\"pred_\" + str(n) for n in class_names], index=[\"GT_\" + str(n) for n in class_names])"
      ],
      "metadata": {
        "id": "MtjyWxhMKWBB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 行ごとに割合にして表示してみましょう。\n",
        "pd.DataFrame(cm,columns=[\"pred_\" + str(n) for n in class_names], index=[\"GT_\" + str(n) for n in class_names]).apply(lambda x:x/sum(x),axis=1)"
      ],
      "metadata": {
        "id": "CZpwRMn9KWDI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 次に各々のクラスについて、precision, recall, f1-scoreを表示してみましょう。\n",
        "print(classification_report(labels, predictions, target_names=class_names))"
      ],
      "metadata": {
        "id": "mvOcLSPNKfFK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 課題 もっと学習させてみましょう \n",
        "\n",
        "\n",
        "1.   base_model (ResNet50) の全ての層を学習可能に\n",
        "2.   最適化関数はNadam (tf.keras.optimizers.Nadam)\n",
        "3.   学習率はこれまでの10分の1に\n",
        "4.   追加のepoch数は20epoch"
      ],
      "metadata": {
        "id": "kJ_kM0A9Lj6b"
      }
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMdIQeboQyO5SKu12dxaT9h",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}