solar_model_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8nk6F-xeuXbL"
   },
   "outputs": [],
   "source": [
    "#NO NEED TO RUN\n",
    "#Code to split \"data\" folder into \"train\", \"val\", and \"test\" folders from \"Data\" folder on Google Drive\n",
    "\n",
    "#pip install split_folders\n",
    "#import split_folders\n",
    "\n",
    "#split_folders.ratio('/content/drive/My Drive/Data', output='/content/drive/My Drive/test_train_validate', seed=133, ratio=(.8, .15, .05))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "IMPORTANT: ***Click Kernel>Change kernel>solar-kernel to select kernel created in readme***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "**The cells below download the validation and test images from the pre-split dataset, from which Keras data generators are created.**\n",
    "<br/><br/>\n",
    "<font size=\"4\"> \n",
    "(Code used to split original dataset is shown above.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "    \n",
    "**First, the validation and test image sets must be downloaded.**\n",
    "        <br/><br/>\n",
    "Link for test set: https://www.dropbox.com/sh/3yilkr8dpbbbfui/AABrLOrvMXxjvmDRdIXDMN2ha?dl=0\n",
    "        <br/><br/>\n",
    "Link for validation set: https://www.dropbox.com/sh/lgf0j8s3hbdi5i0/AAC3jEhaoBRm8zboUDRF6lFla?dl=0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\">\n",
    "    \n",
    "**Folders should be placed in the same directory as this notebook.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Do3EG0CBAcHc"
   },
   "outputs": [],
   "source": [
    "#Loading in images from train_test_validate\n",
    "\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "#Create data generator\n",
    "datagen = ImageDataGenerator(\n",
    "    rescale=1./255,\n",
    "    horizontal_flip=True,\n",
    "    vertical_flip=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1261,
     "status": "ok",
     "timestamp": 1588393948217,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "lze5gEEiOWl6",
    "outputId": "9c3f0802-3f3f-4cf9-acfd-d1dc1b644485"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2470 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "#Validation generator\n",
    "validation_generator = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/test_train_validate/val',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary',\n",
    "        shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 892,
     "status": "ok",
     "timestamp": 1588393949897,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "la8fMMHoOWui",
    "outputId": "6cf386a8-287b-41f0-8f18-469c3178b50b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 826 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "#Test generator\n",
    "test_generator = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/test_train_validate/test',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary',\n",
    "        shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "    \n",
    "NOT RECOMMENDED: Run the following cells to load in baseline Inception V3 model, re-train single layer across 2 epochs, and then unfreeze all layers beyond \"mixed6\" and re-train across 42 epochs\n",
    "<br/><br/>\n",
    "    \n",
    "***Training will take a signifigant amount of time due to the size of the training set.*** \n",
    "<br/><br/>\n",
    "<font size=\"4\"> \n",
    "    \n",
    "**It is recommended to instead skip to the markdown cell below (towards the end of the notebook) where the pre-trained model weights are loaded in, in order to avoid having to re-run the training process.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "Download training set: https://www.dropbox.com/sh/gcv9ljkf4la930s/AADg8jfItbZzuS3XM81hIAQXa?dl=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Train generator\n",
    "train_generator = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/test_train_validate/train',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Pdxqg3sshn1M"
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras.applications.inception_v3 import InceptionV3\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras import Model\n",
    "from tensorflow.keras.optimizers import RMSprop\n",
    "from tensorflow.keras.optimizers import SGD\n",
    "from tensorflow.keras import metrics\n",
    "\n",
    "#Loading in Inception V3 pre-trained model\n",
    "pre_trained_model=InceptionV3(input_shape=(300,300,3),\n",
    "                              include_top=False,\n",
    "                              weights='imagenet')\n",
    "#Freezing layers\n",
    "for layer in pre_trained_model.layers:\n",
    "    layer.trainable=False\n",
    "#Adding mixed7 layer\n",
    "last_layer = pre_trained_model.get_layer('mixed7')\n",
    "last_output = last_layer.output\n",
    "x = layers.Flatten()(last_output)\n",
    "#Adding fully connected layer\n",
    "x = layers.Dense(1024, activation='relu')(x)\n",
    "# Set dropout rate\n",
    "x = layers.Dropout(0.2)(x) #Default is 0.2\n",
    "x = layers.Dense(1, activation='sigmoid')(x)\n",
    "#Setting metrics\n",
    "metrics = [\n",
    "  metrics.BinaryAccuracy(name='accuracy'),\n",
    "  metrics.Precision(name='precision'),\n",
    "  metrics.Recall(name='recall'),\n",
    "  metrics.TruePositives(name='tp'),\n",
    "  metrics.FalsePositives(name='fp'),\n",
    "  metrics.TrueNegatives(name='tn'),\n",
    "  metrics.FalseNegatives(name='fn')]\n",
    "#Compiling model: here we use an RMSprop optimizer\n",
    "model= Model(pre_trained_model.input,x)\n",
    "model.compile(optimizer=RMSprop(lr=0.001,momentum=0.9,decay=0.9),\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 105
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 7265518,
     "status": "ok",
     "timestamp": 1588401263610,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "afhhmIXF_Nwo",
    "outputId": "8c57c8ac-782d-42dd-f425-bbbd75f14f99"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "659/659 [==============================] - 7093s 11s/step - loss: 33.4850 - accuracy: 0.8193 - precision: 0.4581 - recall: 0.7430 - tp: 1558.0000 - fp: 1843.0000 - tn: 9240.0000 - fn: 539.0000 - val_loss: 15.1891 - val_accuracy: 0.8354 - val_precision: 0.4812 - val_recall: 0.7363 - val_tp: 282.0000 - val_fp: 304.0000 - val_tn: 1773.0000 - val_fn: 101.0000\n",
      "Epoch 2/2\n",
      "659/659 [==============================] - 156s 237ms/step - loss: 29.3147 - accuracy: 0.8304 - precision: 0.4798 - recall: 0.7632 - tp: 1602.0000 - fp: 1737.0000 - tn: 9339.0000 - fn: 497.0000 - val_loss: 12.9609 - val_accuracy: 0.8463 - val_precision: 0.5045 - val_recall: 0.7389 - val_tp: 283.0000 - val_fp: 278.0000 - val_tn: 1799.0000 - val_fn: 100.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model with 2 epochs, allowing single layer of Inception V3 model to be re-trained\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=2,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pbD2-VBGAUqT"
   },
   "outputs": [],
   "source": [
    "#Fine tune convolutional layers\n",
    "unfreeze = False\n",
    "#Unfreeze all models after \"mixed6\"\n",
    "for layer in pre_trained_model.layers:\n",
    "    if unfreeze:\n",
    "        layer.trainable = True\n",
    "    if layer.name == 'mixed6':\n",
    "        unfreeze = True\n",
    "#Compile model again, this time using a Stochastic Gradient Descent (SGD) optimizer with a low learning rate (0.000075)\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=SGD(lr=0.000075),\n",
    "              metrics=metrics)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code below trains the model across 42 epochs. Training was done over multiple cells, and the model was saved at several points thorughout the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 105
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 318538,
     "status": "ok",
     "timestamp": 1588401772265,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "ypfCxfhHAU5T",
    "outputId": "0d0ff804-15aa-4c5c-969f-688abd413869"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "659/659 [==============================] - 158s 239ms/step - loss: 19.0429 - accuracy: 0.8517 - precision: 0.5207 - recall: 0.8070 - tp: 1999.0000 - fp: 1840.0000 - tn: 11318.0000 - fn: 478.0000 - val_loss: 3.0517 - val_accuracy: 0.9455 - val_precision: 0.8430 - val_recall: 0.7990 - val_tp: 306.0000 - val_fp: 57.0000 - val_tn: 2020.0000 - val_fn: 77.0000\n",
      "Epoch 2/2\n",
      "659/659 [==============================] - 154s 234ms/step - loss: 8.4704 - accuracy: 0.8935 - precision: 0.6181 - recall: 0.8701 - tp: 1829.0000 - fp: 1130.0000 - tn: 9943.0000 - fn: 273.0000 - val_loss: 2.7127 - val_accuracy: 0.9549 - val_precision: 0.9359 - val_recall: 0.7624 - val_tp: 292.0000 - val_fp: 20.0000 - val_tn: 2057.0000 - val_fn: 91.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=2,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 156837,
     "status": "ok",
     "timestamp": 1588402416297,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "oeaMSe3y6w_d",
    "outputId": "6f86f102-5032-4589-bcb9-57d80485cd86"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 155s 235ms/step - loss: 5.6536 - accuracy: 0.9121 - precision: 0.6671 - recall: 0.8926 - tp: 1870.0000 - fp: 933.0000 - tn: 10147.0000 - fn: 225.0000 - val_loss: 1.7409 - val_accuracy: 0.9634 - val_precision: 0.9322 - val_recall: 0.8251 - val_tp: 316.0000 - val_fp: 23.0000 - val_tn: 2054.0000 - val_fn: 67.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 156134,
     "status": "ok",
     "timestamp": 1588402850988,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "-av9AoZbCIi7",
    "outputId": "9c1b4547-966b-42a5-a7ca-dbe80b1851f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 155s 235ms/step - loss: 4.5751 - accuracy: 0.9170 - precision: 0.6817 - recall: 0.8964 - tp: 1878.0000 - fp: 877.0000 - tn: 10203.0000 - fn: 217.0000 - val_loss: 1.4327 - val_accuracy: 0.9573 - val_precision: 0.8174 - val_recall: 0.9347 - val_tp: 358.0000 - val_fp: 80.0000 - val_tn: 1997.0000 - val_fn: 25.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 157698,
     "status": "ok",
     "timestamp": 1588403052799,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "cARsbgUSC1ru",
    "outputId": "67c89de8-777e-4785-e3c0-c24316a6f97a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 156s 237ms/step - loss: 3.7298 - accuracy: 0.9301 - precision: 0.7219 - recall: 0.9133 - tp: 1918.0000 - fp: 739.0000 - tn: 10336.0000 - fn: 182.0000 - val_loss: 6.6553 - val_accuracy: 0.7561 - val_precision: 0.3883 - val_recall: 0.9843 - val_tp: 377.0000 - val_fp: 594.0000 - val_tn: 1483.0000 - val_fn: 6.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4}) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 160080,
     "status": "ok",
     "timestamp": 1588403237901,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "EBvClIfrDhJ6",
    "outputId": "2cb2d675-269a-4c42-eba5-d65a0163c994"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 159s 241ms/step - loss: 3.3032 - accuracy: 0.9327 - precision: 0.7293 - recall: 0.9175 - tp: 1924.0000 - fp: 714.0000 - tn: 10364.0000 - fn: 173.0000 - val_loss: 1.1789 - val_accuracy: 0.9626 - val_precision: 0.8299 - val_recall: 0.9556 - val_tp: 366.0000 - val_fp: 75.0000 - val_tn: 2002.0000 - val_fn: 17.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1eFhf0-yNolf"
   },
   "outputs": [],
   "source": [
    "#model.save('/content/drive/My Drive/Models/solar_model_5_2_v0.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 193224,
     "status": "ok",
     "timestamp": 1588403512870,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "OTCxoB83EP7o",
    "outputId": "659ebb4b-b09a-4a86-8c12-4a519943c5c3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 191s 290ms/step - loss: 3.5545 - accuracy: 0.9312 - precision: 0.7246 - recall: 0.9151 - tp: 1918.0000 - fp: 729.0000 - tn: 10350.0000 - fn: 178.0000 - val_loss: 1.0552 - val_accuracy: 0.9630 - val_precision: 0.8460 - val_recall: 0.9321 - val_tp: 357.0000 - val_fp: 65.0000 - val_tn: 2012.0000 - val_fn: 26.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CTT5xayzOmhx"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v1.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 172161,
     "status": "ok",
     "timestamp": 1588403849092,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "TbqftIeURS1c",
    "outputId": "3168027d-dcff-4531-d7fa-29e0041cd2a5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 169s 256ms/step - loss: 2.6690 - accuracy: 0.9425 - precision: 0.7624 - recall: 0.9275 - tp: 1944.0000 - fp: 606.0000 - tn: 10473.0000 - fn: 152.0000 - val_loss: 1.0028 - val_accuracy: 0.9661 - val_precision: 0.8507 - val_recall: 0.9567 - val_tp: 376.0000 - val_fp: 66.0000 - val_tn: 1991.0000 - val_fn: 17.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hNgIi2-wQErk"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v2.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 185317,
     "status": "ok",
     "timestamp": 1588404651062,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "A_V1bqbkR59L",
    "outputId": "bd6792ad-10cc-4aaf-e91b-07a54deb2c3a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 183s 278ms/step - loss: 2.6579 - accuracy: 0.9415 - precision: 0.7584 - recall: 0.9274 - tp: 1943.0000 - fp: 619.0000 - tn: 10461.0000 - fn: 152.0000 - val_loss: 0.8488 - val_accuracy: 0.9755 - val_precision: 0.9258 - val_recall: 0.9211 - val_tp: 362.0000 - val_fp: 29.0000 - val_tn: 2028.0000 - val_fn: 31.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4}) #HERE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rCb6a2t7TBym"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v3.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 183849,
     "status": "ok",
     "timestamp": 1588406674042,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "CJUQZ81kSpof",
    "outputId": "bb3e4722-9d5f-4742-f657-c941cee73eec"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 182s 277ms/step - loss: 2.1026 - accuracy: 0.9495 - precision: 0.7868 - recall: 0.9365 - tp: 1963.0000 - fp: 532.0000 - tn: 10547.0000 - fn: 133.0000 - val_loss: 2.4033 - val_accuracy: 0.8812 - val_precision: 0.5763 - val_recall: 0.9796 - val_tp: 385.0000 - val_fp: 283.0000 - val_tn: 1774.0000 - val_fn: 8.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 163369,
     "status": "ok",
     "timestamp": 1588406872722,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "utkyMYAmap9E",
    "outputId": "12e36c3f-040d-45d2-a3bd-645287c60b56"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 161s 245ms/step - loss: 2.3561 - accuracy: 0.9431 - precision: 0.7634 - recall: 0.9309 - tp: 1952.0000 - fp: 605.0000 - tn: 10473.0000 - fn: 145.0000 - val_loss: 1.0326 - val_accuracy: 0.9714 - val_precision: 0.9575 - val_recall: 0.8601 - val_tp: 338.0000 - val_fp: 15.0000 - val_tn: 2042.0000 - val_fn: 55.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 163105,
     "status": "ok",
     "timestamp": 1588407066940,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "dnTYxsq_aqQT",
    "outputId": "61974f54-1f24-4e52-e910-6587cda364cf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 162s 245ms/step - loss: 2.1865 - accuracy: 0.9468 - precision: 0.7776 - recall: 0.9323 - tp: 1954.0000 - fp: 559.0000 - tn: 10520.0000 - fn: 142.0000 - val_loss: 0.9685 - val_accuracy: 0.9616 - val_precision: 0.8286 - val_recall: 0.9593 - val_tp: 377.0000 - val_fp: 78.0000 - val_tn: 1979.0000 - val_fn: 16.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 163370,
     "status": "ok",
     "timestamp": 1588407275771,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "77FJJUsebbQ-",
    "outputId": "ff39ef64-0b17-41b7-dc3e-6bc1be656fd4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 162s 245ms/step - loss: 2.0566 - accuracy: 0.9493 - precision: 0.7862 - recall: 0.9356 - tp: 1960.0000 - fp: 533.0000 - tn: 10547.0000 - fn: 135.0000 - val_loss: 4.5123 - val_accuracy: 0.8082 - val_precision: 0.4548 - val_recall: 0.9847 - val_tp: 387.0000 - val_fp: 464.0000 - val_tn: 1593.0000 - val_fn: 6.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 163327,
     "status": "ok",
     "timestamp": 1588407454103,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "mnm1D_-FbbjO",
    "outputId": "5f0f3388-c624-4fd5-f24c-a1d33e2c3039"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 162s 246ms/step - loss: 1.7894 - accuracy: 0.9542 - precision: 0.8027 - recall: 0.9437 - tp: 1977.0000 - fp: 486.0000 - tn: 10594.0000 - fn: 118.0000 - val_loss: 0.6941 - val_accuracy: 0.9780 - val_precision: 0.9104 - val_recall: 0.9567 - val_tp: 376.0000 - val_fp: 37.0000 - val_tn: 2020.0000 - val_fn: 17.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PvTZt6WCZ-3c"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v4.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 169258,
     "status": "ok",
     "timestamp": 1588407793509,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "Xs_NpDKGZ_DH",
    "outputId": "e5cd09f4-cf8f-4ea1-c0ea-cbc166131373"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 168s 255ms/step - loss: 1.5658 - accuracy: 0.9547 - precision: 0.8046 - recall: 0.9447 - tp: 1981.0000 - fp: 481.0000 - tn: 10597.0000 - fn: 116.0000 - val_loss: 0.7630 - val_accuracy: 0.9731 - val_precision: 0.8884 - val_recall: 0.9517 - val_tp: 374.0000 - val_fp: 47.0000 - val_tn: 2010.0000 - val_fn: 19.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 163859,
     "status": "ok",
     "timestamp": 1588407984116,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "vc49v9M9agsY",
    "outputId": "ec7aec1d-e32c-437b-be0b-06612d2e4139"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 162s 245ms/step - loss: 1.4672 - accuracy: 0.9568 - precision: 0.8138 - recall: 0.9452 - tp: 1984.0000 - fp: 454.0000 - tn: 10622.0000 - fn: 115.0000 - val_loss: 0.7533 - val_accuracy: 0.9727 - val_precision: 0.8844 - val_recall: 0.9542 - val_tp: 375.0000 - val_fp: 49.0000 - val_tn: 2008.0000 - val_fn: 18.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 161267,
     "status": "ok",
     "timestamp": 1588408172947,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "V64UDa9mag7u",
    "outputId": "7909fde5-8bff-4599-e501-f16275d0f848"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 160s 243ms/step - loss: 1.2607 - accuracy: 0.9624 - precision: 0.8340 - recall: 0.9533 - tp: 2000.0000 - fp: 398.0000 - tn: 10679.0000 - fn: 98.0000 - val_loss: 0.6442 - val_accuracy: 0.9722 - val_precision: 0.8842 - val_recall: 0.9517 - val_tp: 374.0000 - val_fp: 49.0000 - val_tn: 2008.0000 - val_fn: 19.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eeeAlG6SfRUZ"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v5.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 171939,
     "status": "ok",
     "timestamp": 1588408389924,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "PYz7SQ65fRkt",
    "outputId": "abee001f-2c1f-4f9d-e1ca-e9667f4c4745"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 170s 257ms/step - loss: 1.4366 - accuracy: 0.9565 - precision: 0.8124 - recall: 0.9446 - tp: 1979.0000 - fp: 457.0000 - tn: 10623.0000 - fn: 116.0000 - val_loss: 0.5537 - val_accuracy: 0.9804 - val_precision: 0.9389 - val_recall: 0.9389 - val_tp: 369.0000 - val_fp: 24.0000 - val_tn: 2033.0000 - val_fn: 24.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zSJmuahFahL0"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v6.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 175417,
     "status": "ok",
     "timestamp": 1588408700135,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "kdvkQzuZgfr2",
    "outputId": "85738cce-2f91-4639-a5cf-bc12102e377f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 174s 264ms/step - loss: 1.4835 - accuracy: 0.9577 - precision: 0.8165 - recall: 0.9470 - tp: 1985.0000 - fp: 446.0000 - tn: 10633.0000 - fn: 111.0000 - val_loss: 0.8757 - val_accuracy: 0.9616 - val_precision: 0.8300 - val_recall: 0.9567 - val_tp: 376.0000 - val_fp: 77.0000 - val_tn: 1980.0000 - val_fn: 17.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 182310,
     "status": "ok",
     "timestamp": 1588409107985,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "7wL2-r_mgf9V",
    "outputId": "74d97ac0-f50a-4cdf-c40e-2caeaabdc4f3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 181s 275ms/step - loss: 1.3827 - accuracy: 0.9592 - precision: 0.8220 - recall: 0.9500 - tp: 1995.0000 - fp: 432.0000 - tn: 10643.0000 - fn: 105.0000 - val_loss: 0.5214 - val_accuracy: 0.9812 - val_precision: 0.9437 - val_recall: 0.9389 - val_tp: 369.0000 - val_fp: 22.0000 - val_tn: 2035.0000 - val_fn: 24.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WOnMfk3oahem"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v7.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 181545,
     "status": "ok",
     "timestamp": 1588409531858,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "xnqHHWC9jV-E",
    "outputId": "e09a0b11-3540-4b57-94fa-92aa3ff5748f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 179s 272ms/step - loss: 1.1195 - accuracy: 0.9634 - precision: 0.8386 - recall: 0.9538 - tp: 2001.0000 - fp: 385.0000 - tn: 10692.0000 - fn: 97.0000 - val_loss: 2.4117 - val_accuracy: 0.8580 - val_precision: 0.5308 - val_recall: 0.9873 - val_tp: 388.0000 - val_fp: 343.0000 - val_tn: 1714.0000 - val_fn: 5.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 161114,
     "status": "ok",
     "timestamp": 1588409713338,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "2UU_pvFpahcQ",
    "outputId": "00f63f93-fff0-4027-fe9a-b34a85871da1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 160s 242ms/step - loss: 1.2838 - accuracy: 0.9587 - precision: 0.8200 - recall: 0.9490 - tp: 1991.0000 - fp: 437.0000 - tn: 10640.0000 - fn: 107.0000 - val_loss: 0.5852 - val_accuracy: 0.9788 - val_precision: 0.9273 - val_recall: 0.9415 - val_tp: 370.0000 - val_fp: 29.0000 - val_tn: 2028.0000 - val_fn: 23.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Qq-l8nNFlY9Y"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v8.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 174625,
     "status": "ok",
     "timestamp": 1588410114159,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "1exWExWplZSJ",
    "outputId": "b4a4065f-8fbc-48c8-a91d-70fdaa5f9305"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 173s 263ms/step - loss: 1.0790 - accuracy: 0.9654 - precision: 0.8457 - recall: 0.9571 - tp: 2006.0000 - fp: 366.0000 - tn: 10713.0000 - fn: 90.0000 - val_loss: 0.7952 - val_accuracy: 0.9763 - val_precision: 0.9772 - val_recall: 0.8728 - val_tp: 343.0000 - val_fp: 8.0000 - val_tn: 2049.0000 - val_fn: 50.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 160932,
     "status": "ok",
     "timestamp": 1588410308462,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "qSkOMsGAlZjp",
    "outputId": "e0f05b06-aa2e-4fd4-c908-4748ab3ab8d7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 159s 242ms/step - loss: 0.9888 - accuracy: 0.9668 - precision: 0.8517 - recall: 0.9585 - tp: 2010.0000 - fp: 350.0000 - tn: 10728.0000 - fn: 87.0000 - val_loss: 0.8340 - val_accuracy: 0.9743 - val_precision: 0.9825 - val_recall: 0.8550 - val_tp: 336.0000 - val_fp: 6.0000 - val_tn: 2051.0000 - val_fn: 57.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 162178,
     "status": "ok",
     "timestamp": 1588410477888,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "zWLEZWAxmsEr",
    "outputId": "53a2cbd6-923d-4d58-9919-ec3447f95958"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 161s 244ms/step - loss: 1.0108 - accuracy: 0.9646 - precision: 0.8431 - recall: 0.9557 - tp: 2004.0000 - fp: 373.0000 - tn: 10705.0000 - fn: 93.0000 - val_loss: 0.5916 - val_accuracy: 0.9796 - val_precision: 0.9549 - val_recall: 0.9160 - val_tp: 360.0000 - val_fp: 17.0000 - val_tn: 2040.0000 - val_fn: 33.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 158619,
     "status": "ok",
     "timestamp": 1588411186975,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "Bexu2WBImsc8",
    "outputId": "1cf96201-b152-49b7-b2e1-015e471c8ef5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 157s 239ms/step - loss: 1.1678 - accuracy: 0.9639 - precision: 0.8404 - recall: 0.9542 - tp: 2001.0000 - fp: 380.0000 - tn: 10698.0000 - fn: 96.0000 - val_loss: 0.6178 - val_accuracy: 0.9816 - val_precision: 0.9703 - val_recall: 0.9135 - val_tp: 359.0000 - val_fp: 11.0000 - val_tn: 2046.0000 - val_fn: 34.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 479
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2023724,
     "status": "ok",
     "timestamp": 1588413330474,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "ddKexAEDohSK",
    "outputId": "bae8d17f-30f9-4937-f9ea-9f8d3988d730"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/13\n",
      "659/659 [==============================] - 158s 240ms/step - loss: 0.9178 - accuracy: 0.9700 - precision: 0.8640 - recall: 0.9633 - tp: 2021.0000 - fp: 318.0000 - tn: 10759.0000 - fn: 77.0000 - val_loss: 0.7705 - val_accuracy: 0.9780 - val_precision: 0.9748 - val_recall: 0.8855 - val_tp: 348.0000 - val_fp: 9.0000 - val_tn: 2048.0000 - val_fn: 45.0000\n",
      "Epoch 2/13\n",
      "659/659 [==============================] - 156s 236ms/step - loss: 0.8774 - accuracy: 0.9681 - precision: 0.8568 - recall: 0.9594 - tp: 2011.0000 - fp: 336.0000 - tn: 10748.0000 - fn: 85.0000 - val_loss: 0.4942 - val_accuracy: 0.9808 - val_precision: 0.9413 - val_recall: 0.9389 - val_tp: 369.0000 - val_fp: 23.0000 - val_tn: 2034.0000 - val_fn: 24.0000\n",
      "Epoch 3/13\n",
      "659/659 [==============================] - 154s 234ms/step - loss: 0.6739 - accuracy: 0.9760 - precision: 0.8891 - recall: 0.9705 - tp: 2037.0000 - fp: 254.0000 - tn: 10822.0000 - fn: 62.0000 - val_loss: 0.5786 - val_accuracy: 0.9829 - val_precision: 0.9631 - val_recall: 0.9288 - val_tp: 365.0000 - val_fp: 14.0000 - val_tn: 2043.0000 - val_fn: 28.0000\n",
      "Epoch 4/13\n",
      "659/659 [==============================] - 155s 235ms/step - loss: 0.8456 - accuracy: 0.9694 - precision: 0.8621 - recall: 0.9613 - tp: 2013.0000 - fp: 322.0000 - tn: 10759.0000 - fn: 81.0000 - val_loss: 0.4867 - val_accuracy: 0.9812 - val_precision: 0.9437 - val_recall: 0.9389 - val_tp: 369.0000 - val_fp: 22.0000 - val_tn: 2035.0000 - val_fn: 24.0000\n",
      "Epoch 5/13\n",
      "659/659 [==============================] - 154s 233ms/step - loss: 0.8979 - accuracy: 0.9709 - precision: 0.8678 - recall: 0.9638 - tp: 2022.0000 - fp: 308.0000 - tn: 10769.0000 - fn: 76.0000 - val_loss: 0.6191 - val_accuracy: 0.9792 - val_precision: 0.9803 - val_recall: 0.8880 - val_tp: 349.0000 - val_fp: 7.0000 - val_tn: 2050.0000 - val_fn: 44.0000\n",
      "Epoch 6/13\n",
      "659/659 [==============================] - 155s 235ms/step - loss: 0.6591 - accuracy: 0.9736 - precision: 0.8793 - recall: 0.9666 - tp: 2025.0000 - fp: 278.0000 - tn: 10802.0000 - fn: 70.0000 - val_loss: 0.5969 - val_accuracy: 0.9735 - val_precision: 0.8868 - val_recall: 0.9567 - val_tp: 376.0000 - val_fp: 48.0000 - val_tn: 2009.0000 - val_fn: 17.0000\n",
      "Epoch 7/13\n",
      "659/659 [==============================] - 153s 232ms/step - loss: 0.8414 - accuracy: 0.9674 - precision: 0.8532 - recall: 0.9605 - tp: 2016.0000 - fp: 347.0000 - tn: 10729.0000 - fn: 83.0000 - val_loss: 0.4271 - val_accuracy: 0.9849 - val_precision: 0.9660 - val_recall: 0.9389 - val_tp: 369.0000 - val_fp: 13.0000 - val_tn: 2044.0000 - val_fn: 24.0000\n",
      "Epoch 8/13\n",
      "659/659 [==============================] - 155s 235ms/step - loss: 0.8904 - accuracy: 0.9687 - precision: 0.8601 - recall: 0.9594 - tp: 2011.0000 - fp: 327.0000 - tn: 10752.0000 - fn: 85.0000 - val_loss: 0.4827 - val_accuracy: 0.9829 - val_precision: 0.9512 - val_recall: 0.9415 - val_tp: 370.0000 - val_fp: 19.0000 - val_tn: 2038.0000 - val_fn: 23.0000\n",
      "Epoch 9/13\n",
      "659/659 [==============================] - 154s 234ms/step - loss: 0.6998 - accuracy: 0.9739 - precision: 0.8805 - recall: 0.9676 - tp: 2034.0000 - fp: 276.0000 - tn: 10797.0000 - fn: 68.0000 - val_loss: 0.5715 - val_accuracy: 0.9816 - val_precision: 0.9628 - val_recall: 0.9211 - val_tp: 362.0000 - val_fp: 14.0000 - val_tn: 2043.0000 - val_fn: 31.0000\n",
      "Epoch 10/13\n",
      "659/659 [==============================] - 155s 236ms/step - loss: 0.5809 - accuracy: 0.9771 - precision: 0.8938 - recall: 0.9714 - tp: 2037.0000 - fp: 242.0000 - tn: 10836.0000 - fn: 60.0000 - val_loss: 0.4593 - val_accuracy: 0.9824 - val_precision: 0.9605 - val_recall: 0.9288 - val_tp: 365.0000 - val_fp: 15.0000 - val_tn: 2042.0000 - val_fn: 28.0000\n",
      "Epoch 11/13\n",
      "659/659 [==============================] - 156s 236ms/step - loss: 0.5206 - accuracy: 0.9790 - precision: 0.9017 - recall: 0.9737 - tp: 2037.0000 - fp: 222.0000 - tn: 10861.0000 - fn: 55.0000 - val_loss: 0.3967 - val_accuracy: 0.9820 - val_precision: 0.9440 - val_recall: 0.9440 - val_tp: 371.0000 - val_fp: 22.0000 - val_tn: 2035.0000 - val_fn: 22.0000\n",
      "Epoch 12/13\n",
      "659/659 [==============================] - 158s 240ms/step - loss: 0.6348 - accuracy: 0.9746 - precision: 0.8834 - recall: 0.9680 - tp: 2030.0000 - fp: 268.0000 - tn: 10810.0000 - fn: 67.0000 - val_loss: 0.5322 - val_accuracy: 0.9812 - val_precision: 0.9677 - val_recall: 0.9135 - val_tp: 359.0000 - val_fp: 12.0000 - val_tn: 2045.0000 - val_fn: 34.0000\n",
      "Epoch 13/13\n",
      "659/659 [==============================] - 158s 239ms/step - loss: 0.6002 - accuracy: 0.9759 - precision: 0.8885 - recall: 0.9704 - tp: 2031.0000 - fp: 255.0000 - tn: 10827.0000 - fn: 62.0000 - val_loss: 0.4071 - val_accuracy: 0.9800 - val_precision: 0.9155 - val_recall: 0.9644 - val_tp: 379.0000 - val_fp: 35.0000 - val_tn: 2022.0000 - val_fn: 14.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=13,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Enr8OSBU0ej0"
   },
   "outputs": [],
   "source": [
    "#model.save('solar_model_5_2_v9.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 174855,
     "status": "ok",
     "timestamp": 1588413698122,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "P8DXj5rUmsyM",
    "outputId": "bcc0c8d7-7cb9-4d0d-f900-166241ce1a6c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 173s 263ms/step - loss: 0.5583 - accuracy: 0.9762 - precision: 0.8913 - recall: 0.9696 - tp: 2041.0000 - fp: 249.0000 - tn: 10821.0000 - fn: 64.0000 - val_loss: 0.4472 - val_accuracy: 0.9776 - val_precision: 0.9043 - val_recall: 0.9618 - val_tp: 378.0000 - val_fp: 40.0000 - val_tn: 2017.0000 - val_fn: 15.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 159662,
     "status": "ok",
     "timestamp": 1588413883429,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "kHNZ_xBJsp7p",
    "outputId": "bfe7b5cf-8ee1-482f-a84e-251412f39cf8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 158s 240ms/step - loss: 0.6228 - accuracy: 0.9743 - precision: 0.8809 - recall: 0.9689 - tp: 2026.0000 - fp: 274.0000 - tn: 10810.0000 - fn: 65.0000 - val_loss: 0.5919 - val_accuracy: 0.9706 - val_precision: 0.8673 - val_recall: 0.9644 - val_tp: 379.0000 - val_fp: 58.0000 - val_tn: 1999.0000 - val_fn: 14.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 161675,
     "status": "ok",
     "timestamp": 1588414097595,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "lwq88nT_sqVK",
    "outputId": "94881293-0d6f-48fb-9eec-98e00b56131b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "659/659 [==============================] - 160s 243ms/step - loss: 0.4354 - accuracy: 0.9806 - precision: 0.9082 - recall: 0.9766 - tp: 2048.0000 - fp: 207.0000 - tn: 10871.0000 - fn: 49.0000 - val_loss: 0.4696 - val_accuracy: 0.9837 - val_precision: 0.9561 - val_recall: 0.9415 - val_tp: 370.0000 - val_fp: 17.0000 - val_tn: 2040.0000 - val_fn: 23.0000\n"
     ]
    }
   ],
   "source": [
    "#Fit model\n",
    "history = model.fit(\n",
    "      train_generator,\n",
    "      steps_per_epoch=13195//20,\n",
    "      epochs=1,\n",
    "      validation_data=validation_generator,\n",
    "      validation_steps=2470//20,\n",
    "      verbose=1,\n",
    "      workers=1,\n",
    "      use_multiprocessing=False,\n",
    "      class_weight={0:1,\n",
    "                    1:4})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C8H-QOpb2BTS"
   },
   "outputs": [],
   "source": [
    "#Saving final model\n",
    "model.save('final_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hVIXe95R1ij4"
   },
   "source": [
    "<font size=\"4\"> \n",
    "Final model was trained over 42 epochs, shown above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "    \n",
    "***Continue here if loading in pre-trained model***\n",
    "<br/><br/>\n",
    "The code below downloads an h5 file containing the pre-trained model weights, and then loads in the file as a Keras model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\n",
      "  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0\n",
      "100  1034    0  1034    0     0    252      0 --:--:--  0:00:04 --:--:--   524\n",
      "100  901M  100  901M    0     0  2324k      0  0:06:37  0:06:37 --:--:-- 2162k:16  0:10:14 1425k:07:48 1752k0:00:30  0:07:28 1929k:17  0:01:01  0:07:16 1713kk      0  0:06:52  0:02:36  0:04:16 1838k    0  0:06:56  0:03:53  0:03:03  860k:57  0:04:01  0:02:56 2613k  0:02:45 3146k:04:30  0:02:23 3371k0:04:41  0:02:04 3468k  0:04:48  0:01:57 2150k 3057k  2323k      0  0:06:37  0:05:58  0:00:39 1852k    0     0  2332k      0  0:06:35  0:06:07  0:00:28 2993k 0:06:15  0:00:21 1949k 0  0:06:36  0:06:33  0:00:03 2179k\n"
     ]
    }
   ],
   "source": [
    "#Download final version of model, used to generate results reported in presentation\n",
    "!curl -L -o final_model.h5 https://www.dropbox.com/s/mprqnfq3prmqun4/final_model.h5?dl=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eG8G485-YB0d"
   },
   "outputs": [],
   "source": [
    "#Load in downloaded model\n",
    "from tensorflow.keras.models import load_model\n",
    "loaded_model=load_model('final_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "    \n",
    "***The code below generates metrics (accuracy, precision, recall, and confusion matrices) for the validation and test sets.***\n",
    "    \n",
    "<font size=\"4\"> \n",
    "    \n",
    "Because the validation results influence the setting of hyperparameters, test results offer the best possible approximation of the model's expected real-world performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 29040,
     "status": "ok",
     "timestamp": 1588414789051,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "_8aimtW0YCn6",
    "outputId": "706d3142-1b32-4480-ca30-249934b7065d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "124/124 [==============================] - 27s 218ms/step\n"
     ]
    }
   ],
   "source": [
    "#Evaluate model\n",
    "\n",
    "#Generate predictions from validation set\n",
    "validation_generator.reset()\n",
    "predictions=loaded_model.predict(validation_generator,steps=2470//20+1,verbose=1)\n",
    "predictions_classes=predictions>0.5\n",
    "predictions_classes=predictions_classes.flatten().astype(int)\n",
    "actual=validation_generator.classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "***Validation results*** (validation set contains 2470 images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 989,
     "status": "ok",
     "timestamp": 1588414798458,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "v_hJbqddYCbn",
    "outputId": "6dc6b4bc-12b7-47a9-ff89-e23697f7f321"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9817813765182186"
      ]
     },
     "execution_count": 170,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "#Validation accuracy\n",
    "metrics.accuracy_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 977,
     "status": "ok",
     "timestamp": 1588414798987,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "O8fegZrdYCYg",
    "outputId": "faec3ac9-cb04-4553-be08-f70b668b0052"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2059,   18],\n",
       "       [  27,  366]])"
      ]
     },
     "execution_count": 171,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Confusion matrix\n",
    "metrics.confusion_matrix(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 787,
     "status": "ok",
     "timestamp": 1588414799408,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "vuIghmINYCWD",
    "outputId": "0de07561-624f-41d3-8224-bdd3be2f1127"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.953125"
      ]
     },
     "execution_count": 172,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Precision\n",
    "metrics.precision_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1352,
     "status": "ok",
     "timestamp": 1588414800569,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "Eacqad7XYCTb",
    "outputId": "2187ac10-75f0-4df6-cd5e-5bf757ac3d90"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9312977099236641"
      ]
     },
     "execution_count": 173,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Recall\n",
    "metrics.recall_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 9772,
     "status": "ok",
     "timestamp": 1588414809542,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "77Nf4zZtYCIF",
    "outputId": "0232b5b2-6604-4df1-d306-8c9f0e023aff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42/42 [==============================] - 9s 208ms/step\n"
     ]
    }
   ],
   "source": [
    "#Generate predictions from test set\n",
    "test_generator.reset()\n",
    "predictions=loaded_model.predict(test_generator,steps=826//20+1,verbose=1)\n",
    "predictions_classes=predictions>0.5\n",
    "predictions_classes=predictions_classes.flatten().astype(int)\n",
    "actual=test_generator.classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "***Test results*** (test set contains 826 images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 9080,
     "status": "ok",
     "timestamp": 1588414809543,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "hCk47tnCYCFG",
    "outputId": "c887e226-5865-4601-cbfe-a65763eb018d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9794188861985472"
      ]
     },
     "execution_count": 175,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Test accuracy\n",
    "metrics.accuracy_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 8489,
     "status": "ok",
     "timestamp": 1588414809544,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "pjNTBTqgYnVl",
    "outputId": "173d1fc4-d34e-4da1-f398-4627e447ebf1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[685,   9],\n",
       "       [  8, 124]])"
      ]
     },
     "execution_count": 176,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Confusion matrix\n",
    "metrics.confusion_matrix(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 7923,
     "status": "ok",
     "timestamp": 1588414809544,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "GXKSCE-dYnka",
    "outputId": "01a0097d-cad7-440f-f4ac-660f9690b008"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9323308270676691"
      ]
     },
     "execution_count": 177,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Precision\n",
    "metrics.precision_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 7236,
     "status": "ok",
     "timestamp": 1588414809545,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "VvzPPPb_Ynh_",
    "outputId": "30423df2-fd7b-401e-a111-43be7d75ffa2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9393939393939394"
      ]
     },
     "execution_count": 178,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Recall\n",
    "metrics.recall_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RSQictS3TagF"
   },
   "outputs": [],
   "source": [
    "#Generate predictions from training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "***Training results (not reported in presentation)***\n",
    "<br/><br/>\n",
    "Please note that due to the large size of the training set (13195 images), this code will take significant time to run."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "Download training set (if not already done): https://www.dropbox.com/sh/gcv9ljkf4la930s/AADg8jfItbZzuS3XM81hIAQXa?dl=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 2010,
     "status": "ok",
     "timestamp": 1588414882018,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "uulWEzVyX6Jt",
    "outputId": "5d1fc196-4b01-4d97-8f28-8403baff0b93"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 13195 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "#Read in training data with shuffle=False\n",
    "train_generator_no_shuffle = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/test_train_validate/train',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary',\n",
    "        shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 154934,
     "status": "ok",
     "timestamp": 1588415143370,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "eWLcIsOk55WW",
    "outputId": "01f52ce3-19f1-4690-c1a6-e02013f690f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "660/660 [==============================] - 153s 232ms/step\n"
     ]
    }
   ],
   "source": [
    "train_generator_no_shuffle.reset()\n",
    "predictions=loaded_model.predict(train_generator_no_shuffle,steps=13195//20+1,verbose=1)\n",
    "predictions_classes=predictions>0.5\n",
    "predictions_classes=predictions_classes.flatten().astype(int)\n",
    "actual=train_generator_no_shuffle.classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 146369,
     "status": "ok",
     "timestamp": 1588415143371,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "2Lbtq-B_6JsF",
    "outputId": "2f346b4a-f50a-4d11-d6ed-993772c35ab2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.993179234558545"
      ]
     },
     "execution_count": 181,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Training accuracy\n",
    "metrics.accuracy_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 145925,
     "status": "ok",
     "timestamp": 1588415143372,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "Y-RDWLC36KSU",
    "outputId": "13a3e15c-6a65-4d5c-aa8a-15f942cd521f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[11023,    72],\n",
       "       [   18,  2082]])"
      ]
     },
     "execution_count": 182,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Confusion matrix\n",
    "metrics.confusion_matrix(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 145589,
     "status": "ok",
     "timestamp": 1588415143372,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "hrggm0Zx6Kcm",
    "outputId": "7800b60f-9396-44f3-9406-5b269d17864d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9665738161559888"
      ]
     },
     "execution_count": 183,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Precision\n",
    "metrics.precision_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 145141,
     "status": "ok",
     "timestamp": 1588415143373,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "i2XICqqW6Kly",
    "outputId": "60842892-9487-45cb-9df0-ef86c4bcca04"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9914285714285714"
      ]
     },
     "execution_count": 184,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Recall\n",
    "metrics.recall_score(actual,predictions_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\">\n",
    "    \n",
    "**Create diagram of network**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.utils import plot_model\n",
    "plot_model(loaded_model, to_file='solar_model_plot.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"5\"> \n",
    "    \n",
    "**The code below was used to generate solar predictions (both binary classifications and probabilities) for the buildings in the Hayward dataset, which are displayed on the maps included in our presentation.**\n",
    "<br/><br/>\n",
    "<font size=\"4\"> \n",
    "The final outputs of the code below are stored as Geojson files entitled \"hayward_with_predictions.geojson\" and \"hayward_with_probabilities.geojson\" located in the Github repository.\n",
    "<br/><br/>\n",
    "Due to the large number of images (over 50,000), running the code below takes many hours, and is thus not recommended to be re-run. If desired, the images can be found at https://drive.google.com/drive/folders/19Y3U00xGvSpq4IcsRoyG6LMpzsG0Swve?usp=sharing. Please note that the file paths below will need to be modified in order for the functions to work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "S5N_ak6Q6WYj"
   },
   "outputs": [],
   "source": [
    "#Generate predictions for Hayward dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "y2b9UZOc6gmL"
   },
   "outputs": [],
   "source": [
    "#Loads in a portion of the Hayward images (split into 10 groups, labelled 0-9) denoted by input \"num\"\n",
    "#Generates binary classifications for the selected images, and saves output as a CSV\n",
    "def generate_hayward_predictions(num):\n",
    "  #Model must be loaded in as \"loaded_model\"\n",
    "    import os\n",
    "    import pandas as pd\n",
    "    hayward_generator = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/Google_API_images/hayward_all/hayward_'+str(num)+'_container',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary',\n",
    "        shuffle=False)\n",
    "    num_files=len(os.listdir('/content/drive/My Drive/Google_API_images/hayward_all/hayward_'+str(num)+'_container/hayward_'+str(num)))\n",
    "    predictions=loaded_model.predict(hayward_generator,steps=num_files//20+1,verbose=1)\n",
    "    predictions_classes=predictions>0.5\n",
    "    predictions_classes=predictions_classes.flatten().astype(int)\n",
    "    predictions_frame=pd.DataFrame({'File Name':hayward_generator.filenames,'Prediction':predictions_classes})\n",
    "    predictions_frame.to_csv('/content/drive/My Drive/hayward_predictions/hayward_'+str(num)+'_predictions.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tZa-i9k_FiXw"
   },
   "outputs": [],
   "source": [
    "#Loads in a portion of the Hayward images (split into 10 groups, labelled 0-9) denoted by input \"num\"\n",
    "#Generates raw model output (probabilities from 0-1) for the selected images, and saves output as a CSV\n",
    "def generate_hayward_probabilities(num):\n",
    "  #Model must be loaded in as \"loaded_model\"\n",
    "    import os\n",
    "    import pandas as pd\n",
    "    hayward_generator = datagen.flow_from_directory(\n",
    "        '/content/drive/My Drive/Google_API_images/hayward_all/hayward_'+str(num)+'_container',\n",
    "        target_size=(300, 300),\n",
    "        batch_size=20,\n",
    "        class_mode='binary',\n",
    "        shuffle=False)\n",
    "    num_files=len(os.listdir('/content/drive/My Drive/Google_API_images/hayward_all/hayward_'+str(num)+'_container/hayward_'+str(num)))\n",
    "    predictions=loaded_model.predict(hayward_generator,steps=num_files//20+1,verbose=1)\n",
    "    predictions=predictions.flatten().astype(float)\n",
    "    predictions_frame=pd.DataFrame({'File Name':hayward_generator.filenames,'Probability':predictions})\n",
    "    predictions_frame.to_csv('/content/drive/My Drive/hayward_probabilities/hayward_'+str(num)+'_probabilities.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=\"4\"> \n",
    "**Running functions above for all 10 sets of images**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 357
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 21448696,
     "status": "ok",
     "timestamp": 1588523646298,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "6Rutzv1IGWkj",
    "outputId": "234fefdd-9c38-4e8e-ec15-9c6b4d368bc3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2521s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2532s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2601s 10s/step\n",
      "Found 0 images belonging to 1 classes.\n",
      "264/264 [==============================] - 1s 3ms/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2586s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2584s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2602s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2786s 11s/step\n",
      "Found 0 images belonging to 1 classes.\n",
      "264/264 [==============================] - 1s 3ms/step\n",
      "Found 5267 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2635s 10s/step\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "for elem in list(range(0,10)):\n",
    "    generate_hayward_probabilities(elem)\n",
    "    time.sleep(15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Skipped portions above were re-ran"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 289
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 20198156,
     "status": "ok",
     "timestamp": 1588442208859,
     "user": {
      "displayName": "Eric Phillips",
      "photoUrl": "",
      "userId": "17074296885058998542"
     },
     "user_tz": 420
    },
    "id": "EdXuPyKl-sJQ",
    "outputId": "38b8ea01-3bd4-4205-b11e-49ab7a4b3b10"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2501s 9s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2497s 9s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2507s 9s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2495s 9s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2503s 9s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2516s 10s/step\n",
      "Found 5268 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2481s 9s/step\n",
      "Found 5267 images belonging to 1 classes.\n",
      "264/264 [==============================] - 2480s 9s/step\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "for elem in list(range(0,10)):\n",
    "    generate_hayward_predictions(elem)\n",
    "    time.sleep(15)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Clean_Solar_ID.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "solar-kernel",
   "language": "python",
   "name": "solar-kernel"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}