02-node-classification-exercise.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "02-node-classification-exercise.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TDQ3oMUkuVvh"
      },
      "source": [
        "# Node Classification\n",
        "\n",
        "Build a GNN to predict the subject area for each paper in a citation graph of scientific papers. This is an example of a GNN that does node classification.\n",
        "\n",
        "Most scientific research is usually based heavily on previous work, and it is common for authors to give credit for such work in their papers in the form of citations. This forms a citation graph. In this graph each node represents a scientific paper, and each edge represents a citation from one paper to another. In addition, each node has a feature vector associated with it.\n",
        "\n",
        "For this example, use the [CORA Dataset](https://graphsandnetworks.com/the-cora-dataset/) available from the `torch_geometric.datasets.Planetoid` package.\n",
        "\n",
        "## Setup\n",
        "\n",
        "The following two cells import Pytorch Geometric (PyG) and a couple of supporting Pytorch packages that are customized against the torch version."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "3vDtMJCIJ4fQ",
        "outputId": "3f5178b6-9733-4a3e-c9fe-9fbc2b6d605c"
      },
      "source": [
        "import torch\n",
        "torch.__version__"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'1.9.0+cu111'"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tmqSDymYK8sp",
        "outputId": "b105c1db-021b-4d0a-d230-39b87a95041d"
      },
      "source": [
        "!pip install -q torch-scatter -f https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html\n",
        "!pip install -q torch-sparse -f https://pytorch-geometric.com/whl/torch-1.9.0+cu111.html\n",
        "!pip install -q torch-geometric"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[K     |████████████████████████████████| 10.4 MB 4.1 MB/s \n",
            "\u001b[K     |████████████████████████████████| 3.7 MB 4.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 325 kB 4.0 MB/s \n",
            "\u001b[K     |████████████████████████████████| 407 kB 36.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 45 kB 3.6 MB/s \n",
            "\u001b[?25h  Building wheel for torch-geometric (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QHfAtdl6uYd9"
      },
      "source": [
        "## Dataset\n",
        "\n",
        "The website for the CORA dataset provides the following information:\n",
        "\n",
        "> The CORA dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words.\n",
        "\n",
        "* Load the dataset from the `torch_geometric.datasets.Planetoid` package using the defaults.\n",
        "* The CORA dataset is a single graph consisting of multiple nodes. Verify that the number of graphs (given by the `torch_geometric.data.Data` object) is 1.\n",
        "* Each node is associated with a feature vector that is basically a word vector indicating presence or absence of a word from a fixed size dictionary. Find and report the size of each node's feature vector.\n",
        "* Find the number of subject areas the nodes are classified into, i.e. the number of classes.\n",
        "* Inspect the underlying Data object representing the graph.\n",
        "* Count and report the number of nodes in the training, validation, and test sets (Hint: count the number of `True` entries in each mask in the `Data` object)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tmGpFBkQLHSE"
      },
      "source": [
        "import torch_geometric\n",
        "from torch_geometric.datasets import Planetoid"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4nbFabVX0BXO"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ewbxj6N_Txg"
      },
      "source": [
        "## DataLoader\n",
        "\n",
        "Create the train, validation and test DataLoaders (`torch_geometric.loader.DataLoader`). You will need them when building your training and evaluation steps later.\n",
        "\n",
        "Set the `batch_size` attribute of your `DataLoader` objects to `BATCH_SIZE`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TkCzTvXwAS5r"
      },
      "source": [
        "BATCH_SIZE = 32"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u-QgH_zo_2X8"
      },
      "source": [
        "from torch_geometric.loader import DataLoader"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jX-vY-Zx0GEJ"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a8jQHj08CTMv"
      },
      "source": [
        "## Model\n",
        "\n",
        "Now define the GNN network -- like any Pytorch network, it should subclass `torch.nn.Module`.\n",
        "\n",
        "As shown in the image below, the GNN should consist of 3 GCN ( `torch_geometric.nn.GCNConv`) layers, followed by a classifier head consisting of 2 Linear (`torch.nn.Linear`) layers. Notice the similarity to a CNN.\n",
        "\n",
        "Each GCNConv layer is followed by a ReLU activation and a Dropout function. The dropout percentage is provided via the constructor.\n",
        "\n",
        "The two linear layers have a Dropout layer in between them. The linear layer is followed by a `log_softmax` activation function.\n",
        "\n",
        "Your constructor should accept the `input_dim` (the number of node features), `hidden_dim`, `output_dim` (the number of classes) and the `dropout_pct`. You can use the same `dropout_pct` value for all Dropout layers in the network.\n",
        "\n",
        "Note that in addition to the node feature vectors (the `Data.x` values) in each batch, the graph layers takes the `edge_index` as well (the `Data.edge_index`).\n",
        "\n",
        "The diagram below shows the input and output dimensionality for each layer.\n",
        "\n",
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tU8IkdhoA3y_"
      },
      "source": [
        "![fig-02-01.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlEAAAC5CAYAAAAI9GoYAAAAAXNSR0IArs4c6QAAB1t0RVh0bXhmaWxlACUzQ214ZmlsZSUyMGhvc3QlM0QlMjJhcHAuZGlhZ3JhbXMubmV0JTIyJTIwbW9kaWZpZWQlM0QlMjIyMDIxLTExLTAyVDE5JTNBMDMlM0EzOS45MTVaJTIyJTIwYWdlbnQlM0QlMjI1LjAlMjAoTWFjaW50b3NoJTNCJTIwSW50ZWwlMjBNYWMlMjBPUyUyMFglMjAxMF8xNV83KSUyMEFwcGxlV2ViS2l0JTJGNTM3LjM2JTIwKEtIVE1MJTJDJTIwbGlrZSUyMEdlY2tvKSUyMENocm9tZSUyRjk1LjAuNDYzOC41NCUyMFNhZmFyaSUyRjUzNy4zNiUyMiUyMGV0YWclM0QlMjJ3aFFNOFlPX1NfZXBLc0IyclJlZSUyMiUyMHZlcnNpb24lM0QlMjIxNS42LjYlMjIlMjB0eXBlJTNEJTIyZGV2aWNlJTIyJTNFJTNDZGlhZ3JhbSUyMGlkJTNEJTIyVWo1ZzZyQzNqVWNaU2hrNU11MlQlMjIlMjBuYW1lJTNEJTIyUGFnZS0xJTIyJTNFN1ZwYmM5c29HUDAxZm94SFY5dDU5SzFKcHRuZFRQeXc3Wk9IV05oaUZvR0tVQ3p2cjE4d1lGMVFFcnV4VTdYZEZ3OGNFS0J6RHBjUHVlZFBrJTJCS0dnVFQlMkJnMFlROXp3bktuciUyQnJPZDVReThVdnhMWUtXQVUlMkJBcllNQlFweUMyQkJmb1hhdERSYUk0aW1OVXFja294UjJrZFhGRkM0SXJYTU1BWTNkYXJyU211OTVxQ0RiU0F4UXBnRyUyRjBiUlR6V2J4RTZKWDRMMFNZMlBidU9Ma21BcWF5QkxBWVIzVllnZjk3enA0eFNybEpKTVlWWWNtZDRVYzk5ZXFIME1EQUdDVCUyRm1BYllva21nSWgxOXZIJTJCOCUyQjMlMkJPJTJGSHFJbHVOS3RQQU9jNnhlJTJCbWY0NXBlUlpqNW52REJIYkdIRzRTTUZLNXJkQzY1NCUyRmlYbUNSYzRWU1pDbGl2NDFLcURvY1dJUHolMkZRRkdZZEZCZExEdllFMGdaenRSQlZkZXEyWjA5YnhCenElMkZyUWlob2JpaWdjR0FsbjV6YUxoa1J5UTBRU2VRNVZsa1dTeEJFbzJsNjBSdWhVR1dvVldkSmtFRzIzMFJHY2RrdnNwTVB6VFpXVkV0bk8xMDdrVTJPV0FieU45V0dFWTFuOXVjVjBnTlcwZzFHSU1ZY1BSY254MXRUT3NlSGlnU0l6NUlHdFlsRFp5R1Zobk4yUXJxaDZwbWJyVGpEaHJlYUlxdWVMRWEydXQlMkJlT3Z2dDRMZjVYbmpqc0p1VFp5Z3kyejVUc2ZZQ2x1V21RRVd2VTZlUkdMRDk2JTJCdGdEWGRUNHFTd01HM25KcUNxMnklMkZtNDVGQmQ5Smk3TFF0Tkx2OTAxRFlxQ3FyWHI3QXE3MDJkQks4TXZyOG1TYzBYJTJGZ2xHTEtCRUlvZ1hJb0NPTUdCRERhRUxsRUN0V2d3Q2RTTFNRMjNiRXVTRkFVeVc0bWJYNWdOQ2VSVlAlMkYxbGZGNEIzaURoZ05HdGdPQ0ZnZDRsM0xBd0hJQUltbk9sWllrVDVackNIak94TG5JR0lNWmxXTEJIQ1NxcGtvdkk1VDh1dHFONnZ0QTZOblN1ZTVIYWpkOFRidXFJciUyQjdjbUxoN1poMEkwdTZlMFFnWUYzWXBRNG5wYTdzVWthWjg3TlZ0VzRFMXlEZlQ1UzZ5MHRjYmxzMmVnN0dyenQyTG5EdFlNM2klMkJzMzRvMEQ4U3lWZGlUNUVyZ3clMkJaTWJFSGhmVlF4MzRqJTJGQlpSd0lZdnpFUGclMkJacWRHd0VNM0FiRFRWRG9RdEhNTzVwMGF6ZVE5NXBwYk9IdjIlMkZicDNkY21PeDN5MlhYWjNKWkVEUzJqZUVIdTh3T2xIOUZsd1ZIdW16MHY4c3U0akw3Z3VGRWx4bkh1S2M1eHJqVE9jR2RMN3JzSjNOUDgzejAzWGQ1UDl3OXI0YmJuUWpabUZMdlBSSGJ4YzdKZnVkQ2NQY3NNYmk4WjltZm9lVTF5OWtWeFhEZFZVR2JFN3RWMEElMkI5RDNQdHlOeVNSQWxndnRGNURYVmlrTXA2U2JHUlh5bjdhMHkzcXhndzNnZUVVQzZXUVVxV25pMFBCazhRUDlBTXlRcVZlWmpLdFFpeSUyQmJOZ09kTjlSSWlKT0ZiVkk1VEpZWFFsbEhXdmglMkYzR1loMjJhRG9NWDk0czNxTXBwU2h6c3JtM25GODl6YjNVbVQyNnNPWEw0JTJCOG1xYVZmaThwSDN3ZjllRUh0ZUhKNlAxNHM3ajdkelI4RmZqc2Z6eTZ3anA3ak12TnlJcm5XdkFzR3g2MmwlMkZ1a3lpV3o1YndKMU9DciUyRmt1SFAlMkZ3TSUzRCUzQyUyRmRpYWdyYW0lM0UlM0MlMkZteGZpbGUlM0USGK5SAAAgAElEQVR4Xu2dB/gFRXX2XxsRLFGxxd6CQWOLfGJAP9RE/USjwRbsLZbY8JPYRYqioiJqjAV7L58aQDHWSCygQSyxt9gLdiFYgsr3/C47uCy7e+fes3dn773vPA8P8L979sy858yZd87MzpxL613OWO/qT6L251qyFsZ+SeBqYstiH9c8/w2273yM5j2xrH2N/Txk5/9u7OdjtKonlsV+VfVZ6XvXvbFnnHGG482yHnKuc83Mv6wPGPtlgQf0GPYBzdmitm82VOd8MGhfY2/sAwiUEw36fbmKBzQvO4AGVA4q6mATgDPo8Ma+HPYBzdmitm82VCZRAagGF3VcGxzS7BcGsc/WM6UHTaKmZI2R6xJ0eA+yAXsFsQ9ozha1fbOhMokKQDW4aLBv2e8DFgliH9BcTtQkqhz2xTUHHd7BJmDBIPYBzdmitm82VCZRAagGFw32Lft9wCJB7AOay4maRJXDvrjmoMM72AQsGMQ+oDlb1PbNhsokKgDV4KLBvmW/D1gkiH1AczlRk6hy2BfXHHR4B5uABYPYBzRni9q+2VCZRAWgGlw02Lfs9wGLBLEPaC4nahJVDvvimoMO72ATsGAQ+4DmbFHbNxsqk6gAVIOLBvuW/T5gkSD2Ac3lRE2iymFfXHPQ4R1sAhYMYh/QnC1q+2ZDZRIVgGpw0WDfst8HLBLEPqC5nKhJVDnsi2sOOryDTcCCQewDmrNFbd9sqEyiAlANLhrsW/b7gEWC2Ac0lxM1iSqHfXHNQYd3sAlYMIh9QHO2qO2bDZVJVACqwUWDfct+H7BIEPuA5nKiJlHlsC+uOejwDjYBCwaxD2jOFrV9s6EyiQpANbhosG/Z7wMWCWIf0FxO1CSqHPbFNQcd3sEmYMEg9gHN2aK2bzZUJlEBqAYXDfYt+33AIkHsA5rLiZpElcO+uOagwzvYBCwYxD6gOVvU9s2GyiQqANXgosG+Zb8PWCSIfUBzOVGTqHLYF9ccdHgHm4AFg9gHNGeL2r7ZUJlEBaAaXDTYt+z3AYsEsQ9oLidqElUO++Kagw7vYBOwYBD7gOZsUds3GyqTqABUg4sG+5b9PmCRIPYBzeVETaLKYV9cc9DhHWwCFgxiH9CcLWr7ZkNlEhWAanDRYN+y3wcsEsQ+oLmcqElUOeyLaw46vINNwIJB7AOas0Vt32yoTKICUA0uGuxb9vuARYLYBzSXEzWJKod9cc1Bh3ewCVgwiH1Ac7ao7ZsNlUlUAKrBRYN9y34fsEgQ+4DmcqImUeWwL6456PAONgELBrEPaM4WtX2zoTKJCkA1uGiwb9nvAxYJYh/QXE7UJKoc9sU1Bx3ewSZgwSD2Ac3ZorZvNlQmUQGoBhcN9i37fcAiQewDmsuJmkSVw7645qDDO9gELBjEPqA5W9T2zYbKJCoA1eCiwb5lvw9YJIh9QHM5UZOoctgX1xx0eAebgAWD2Ac0Z4vavtlQmUQFoBpcNNi37PcBiwSxD2guJ2oSVQ774pqDDu9gE7BgEPuA5mxR2zcbKpOoAFSDiwb7lv0+YJEg9gHN5URNosphX1xz0OEdbAIWDGIf0JwtavtmQ2USFYBqcNFg37LfBywSxD6guZyoSVQ57ItrDjq8g03AgkHsA5qzRW3fbKhMogJQDS4a7Fv2+4BFgtgHNJcTNYkqh31xzUGHd7AJWDCIfUBztqjtmw2VSVQAqsFFg33Lfh+wSBD7gOZyoiZR5bAvrjno8A42AQsGsQ9ozha1fbOhMokKQDW4aLBv2e8DFgliH9BcTtQkqhz2xTUHHd7BJmDBIPYBzdmitm82VCZRAagGFw32Lft9wCJB7AOay4maRJXDvrjmoMM72AQsGMQ+oDlb1PbNhsokKgDV4KLBvmW/D1gkiH1AczlRk6hy2BfXHHR4B5uABYPYBzRni9q+2VCZRAWgGlw02Lfs9wGLBLEPaC4nahJVDvvimoMO72ATsGAQ+4DmbFHbNxsqk6gAVIOLBvuW/T5gkSD2Ac3lRE2iymFfXHPQ4R1sAhYMYh/QnC1q+2ZDZRIVgGpw0WDfst8HLBLEPqC5nKhJVDnsi2sOOryDTcCCQewDmrNFbd9sqEyiAlANLhrsW/b7gEWC2Ac0lxM1iSqHfXHNQYd3sAlYMIh9QHO2qO2bDZVJVACqwUWDfct+H7BIEPuA5nKiW02iTj31VJ188sm60pWupPOe97ytVvjlL3+pb37zm7rEJS6hi1/84ktZ6re//a2+/vWva8cdd9RlLnMZnfvc517qPUMLBR0+FGyM/azrTbn/2b6BDleybwWqvRGixr6cGYPYl6t4QPOUgzjN2l/SkZJO7WjjUoH+29/+tu5///vr3e9+91mvvdOd7qQXv/jFuuhFLzr7289+9jM96EEP0pvf/OaznrnlLW+pl7/85TMi9JrXvEb3vOc99YpXvEL3vve9z3rmbne7m654xSvqqU99qiBPT3va0/SkJz3prN+vcpWrzGT32GOPgNmGEQ06vLEPmCET+0Ml7VpTc7qkv6v9/76S7tyoxqMkfa362w6SXi+pztpPlPS0jKrbvhkgdT2Sad8u8aWwz63upS51KR1++OG6+93vfjaR973vfSJ+MbFc52Lsy1kviP2iFZ/HDxZ931LPT51E/UrSeSQdLumpLWRq4WDzm9/8Rte+9rV1rWtdS8973vNEQPnc5z6nffbZR9e85jV1zDHH6IwzztBee+2l3//+93r1q18tiM+XvvQl3e9+99Pvfvc7nXDCCbO/3+te99KFLnShWZZp5513nhmAIHSFK1xhRp4OPvhgveAFL9Bb3vIW3ehGN5oFp0MPPVSvetWr9NWvflWXvOQllzLaUEJBhzf2AUNkYn+CpBs21NT77CGSDmj8vruk/6j+tpOk0xq/HyvpNhlVt30zQFpXEvWsZz1L97jHPc5WfSaOxLkb3rDpcgEgCohm9q1iBNbYD+YU8/jBYIr6XjR1ErWfpKdXM+kzJB3RIFMLB3oySY94xCP0ne98Rxe+8IXPwuZjH/uYXvnKV+o5z3mOPvrRj+omN7mJvvjFL+rqV7/6Wc9AtshWQZDe+ta3zggRy3wQshe+8IVnI1FPfOITdcELXnCWubrPfe5z1jtYHuQ3sleQOUjb4x73OJEdu9nNbqbnP//5utzlLjfT893vfndG0N7znvfo+te//ox8oefnP/+5nv3sZ8/eSZ14P89c5CIXWchpxg42xv4P5snEfq1IlO27sH0nNZB//OMfn2XQ3/a2t3XGH+LdD37wA+2333467rjjdJ3rXEeHHXaYrne9683acvTRR8+yXJCxW9ziFnr605+uy172srOMPbGMmMbWCeLbqkpm3zL2KzBAEPtFazSPHyz6vqWenzqJolE/kXSxqnX/I6lOpk4ha7RIofOzx+moo46aibHkRsYplfOd73yzDn7ggQf2prXJRBE83vCGN8wCCdkpZnApE8W/IVdf+cpXdLWrXa21ip///Odn2S9I3b777qsnP/nJM4L0oQ99SAcddJAOOeSQWTbrGte4hh7ykIfM/uGdt7/97XXaaadpp512msm8973v1Qc/+MFFYJg9G3T4hQmssV94kF0rEmX7LmzfSQ3k9eU84l9b/DnggAO0++67zyZsj370o/X+979/RpTIYhGPLn/5y8+2L/DM4x//+Fl8O+KII2YTTiaPbIl4zGMeo5ve9KYLx6tcgbHjWm69eI6Vj7ZMlLFfBMWzPdvHD7q2AS2trE1wHok6SNKBg2oc5mXsDTlG0h0WJVEQEDJQZJ0oZHHSf/P/EBj+4W/MprpKIlFkgh75yEfOgslJJ500W+JjOe82t7nNbAmPWRsdp60QZD7wgQ/MCBiFzNeuu+46I3kve9nLZr8lcsSz/P0lL3mJLnCBC+id73ynbnWrW82CFPpZaly0VMFmUbGznjf2Yezn9T+Y8Y0bBqrLsNnu4Mbvu0k6qfrbjpJ+2fj9aEl/m2H0hUmy+9Y5SFQGzO2PLNq3FlGUO5C3xR9izW677TbLKvFBDvW89KUvPdu2ADEiO4Uf/OhHP5qRsE9+8pP68Ic/PCNRbJ/4/ve/v/IPa8aOa8Z+OL9fBMueZxM/uONA7+t9zbwgPkYd5un4saQzNxxJ4UzUwx/+cB177LH62tfO3Hv7wx/+UHwp9qtf/WqW5YFAsdR3l7vcRT/5yU90sYulJJh0yimn6Mgjj9R973tfveMd75hloiBRv/jFL7TLLrvMZmDHH3/8jESRNWJW9va3v31GqOoFIsRMjcBCqjstzVEHZnOk1Vnm+9a3vjVLg1PQxZIjqXaI3/nPf/4ZeUJvs57zAE2/jz1jM/bnCDZT7n8LkyjbdzD7Lox9bp/nuVwS1RZ/iIt3vnPzWwbNthkQl57ylKfouc997iymsi2Bj2wSiSLz/rrXvW6Rqi717NhxbZFKGvtF0Mp6to8fTCITldWKFT7EmidfErG5fJA9UZCgBz7wgbOsDmQnFTr6jW984xmJYrP4da973RlhYcN5KmSfWF5jie61r33tWSSK31nW473sc+I9ECRS3mSISGenAnljee8b3/jG7Cu9T33qU7ON55TPfOYzM/lf//rXs70J7B946Utfeg4S9a53vWuW8dp///1n+7eo5zJl7GBj7AcbZJcx96IyCw/ktu9g9l0Y+0WMmzuQt8UfvlhmSY4MO3GSAjlKZIksFJNA4ifbIl7/+tebRNWMY+wX8dS5z87jB3NfMMQDU54J0755u+8XDjZs7L7yla8826j9ohe9aHb2E1kf9gB8+ctfnpEoluGYbbE8BzmC2Jx44om69a1vPduDxN6P+nIeFU1f9CH/2Mc+drb5nNnZgx/84NkxCXvvvfcslc0XMXQk9mR94hOfmG1gZ2kOnWwwJ7OV6tNFok4//fTZ14DM9iBgd7jDHZbyhbFJlLEfbJBdyt4LCrlvrVHfWsS2xJ+HPvShut3tbneWGFsEWKJLRxwQD9viD0SZzeXPfOYzReaR/Zhk2j/72c/OjoxhG8KnP/1p/fSnP51tN9hhhx1m2xWYVDoTdWYW0Ngv4q29z87jB4Mp6nvR1EnUvHMgFg70gAE54ownOjWFGdUTnvCE2R4kNk7uueees03lnCXFclwqBA02UXJoZpNE8QyBhCXBRKLYtM57n/GMZ5z1jpvf/OazL/ZIdUO87nrXu+qNb3zjrA68F303uMENZqSuGcQgcilr9bCHPWz2JV/aYL6Mt4xNooz9ZpMo23cw+y4V13JjAAM52xjqhbjERvE+EpXiz5ve9KbZhzCpkDVnAvi9731vloVnPxSFLQ3ESzJSP/7xj/WFL3xhln1fdSkR13LbZOxzkcp6bh4/yHpJ9KGpk6h57Vs62EBg6PTsQ7rqVa+avlQ7hz72G5G6JnvFfqVlChkYZnkc0pkO86y/h3qQVaIeXSenL6N3nkypYGPsw19GzjPtEL+7bwVQLNW3AlVeSJTJGzGNPZ31mEbfZstCugWC/aJko5ggjlWM/Zk3cKwh9mO5yKB6tpZEDYrimr5s04PNlM0SxH6Mpi1Nosao3NR1BO1r7AMGNvYB8IKiQeyD2suIm0SVwX0SWoMO70AfsGIQ+4DmbFHbNxuqcz4YtK+xN/YBBMqJBv2+XMUDmk2iAuCtu2jQ4R3oAw4QxD6gOVvU9s2GyiQqANXgosG+Zb8PWCSIfUBzOVGTqHLYF9ccdHgHm4AFg9gHNGeL2r7ZUJlEBaAaXDTYt+z3AYsEsQ9oLidqElUO++Kagw7vYBOwYBD7gOZsUds3GyqTqABUg4sG+5b9PmCRIPYBzeVETaLKYV9cc9DhHWwCFgxiH9CcLWr7ZkNlEhWAanDRYN+y3wcsEsQ+oLmcqElUOeyLaw46vINNwIJB7AOas0Vt32yoTKICUA0uGuxb9vuARYLYBzSXEzWJKod9cc1Bh3ewCVgwiH1Ac7ao7ZsNlUlUAKrBRYN9y34fsEgQ+4DmcqImUeWwL6456PAONgELBrEPaM4WtX2zoTKJCkA1uGiwb9nvAxYJYh/QXE7UJKoc9sU1Bx3ewSZgwSD2Ac3ZorZvNlQmUQGoBhcN9i37fcAiQewDmsuJmkSVw7645qDDO9gELBjEPqA5W9T2zYbKJCoA1eCiwb5lvw9YJIh9QHM5UZOoctgX1xx0eAebgAWD2Ac0Z4vavtlQmUQFoBpcNNi37PcBiwSxD2guJ2oSVQ774pqDDu9gE7BgEPuA5mxR2zcbKpOoAFSDiwb7lv0+YJEg9gHN5URNosphX1xz0OEdbAIWDGIf0JwtavtmQ2USFYBqcNFg37LfBywSxD6guZyoSVQ57ItrDjq8g03AgkHsA5qzRW3fbKhMogJQDS4a7Fv2+4BFgtgHNJcTNYkqh31xzUGHd7AJWDCIfUBztqjtmw2VSVQAqsFFg33Lfh+wSBD7gOZyoiZR5bAvrjno8A42AQsGsQ9ozha1fbOhMokKQDW4aLBv2e8DFgliH9BcTtQkqhz2xTUHHd7BJmDBIPYBzdmitm82VCZRAagGFw32Lft9wCJB7AOay4maRJXDvrjmoMM72AQsGMQ+oDlb1PbNhsokKgDV4KLBvmW/D1gkiH1AczlRk6hy2BfXHHR4B5uABYPYBzRni9q+2VCZRAWgGlw02Lfs9wGLBLEPaC4nahJVDvvimoMO72ATsGAQ+4DmbFHbNxsqk6gAVIOLBvuW/T5gkSD2Ac3lRE2iymFfXHPQ4R1sAhYMYh/QnC1q+2ZDZRIVgGpw0WDfst8HLBLEPqC5nKhJVDnsi2sOOryDTcCCQewDmrNFbd9sqEyiAlANLhrsW/b7gEWC2Ac0lxM1iSqHfXHNQYd3sAlYMIh9QHO2qO2bDZVJVACqwUWDfct+H7BIEPuA5nKiJlHlsC+uOejwDjYBCwaxD2jOFrV9s6EyiQpANbhosG/Z7wMWCWIf0FxO1CSqHPbFNQcd3sEmYMEg9gHN2aK2bzZUJlEBqAYXDfYt+33AIkHsA5rLiZpElcO+uOagwzvYBCwYxD6gOVvU9s2GyiQqANXgosG+Zb8PWCSIfUBzOVGTqHLYF9ccdHgHm4AFg9gHNGeL2r7ZUJlEBaAaXDTYt+z3AYsEsQ9oLidqElUO++Kagw7vYBOwYBD7gOZsUds3GyqTqABUg4sG+5b9PmCRIPYBzeVETaLKYV9cc9DhHWwCFgxiH9CcLWr7ZkNlEhWAanDRYN+y3wcsEsQ+oLmcqElUOeyLaw46vINNwIJB7AOas0Vt32yoTKICUA0uGuxb9vuARYLYBzSXEzWJKod9cc1Bh3ewCVgwiH1Ac7ao7ZsNlUlUAKrBRYN9y34fsEgQ+4DmcqImUeWwL6456PAONgELBrEPaM4WtX2zoTKJCkA1uGiwb9nvAxYJYh/QXE7UJKoc9sU1Bx3ewSZgwSD2Ac3ZorZvNlQmUQGoBhcN9i37fcAiQewDmsuJrj2JKgfdxmhe1gfO2BgEyjVkWezHqLHtG0d5Wfsae2MfR6DcG5b1+3I1DmjeqsYGcLKoETACRsAIGAEjYATOhoBJlB3CCBgBI2AEjIARMAJLIGAStQRoFjECRsAIGAEjYASMgEmUfcAIGAEjYASMgBEwAksgYBK1BGgWMQJGwAgYASNgBIyASZR9wAgYASNgBIyAETACSyBgErUEaBYxAkbACBgBI2AEjIBJlH3ACBgBI2AEjIARMAJLIGAStQRoFjECRsAIGAEjYASMgEmUfWAsBA6qFKV/j6XXeoxAFIExfbfeP9xXopaz/JgIbKXvmkSN6WLbrStdZWGf224/WMfWj+m7DEQHSjpYkknUOnrL9tZ5K33XA9r2OvzYLR9zIBq7bda32QiM6btbORBttvtsTeu20ndNorbGv4s3dMyBqHhjXYGNQmBM393KgWijvGV7G7OVvmsStb0OP3bLxxyIxm6b9W02AmP67lYORJvtPlvTuq30XZOorfHv4g0dcyAq3lhXYKMQGNN3t3Ig2ihv2d7GbKXvmkRtr8OP3fIxB6Kx22Z9m43AmL475YHoYpIuJenbkv57YJPvLOn3kn7W8d55vw9cHb9uCQSm7LtLNCdPxCQqDyc/FUdgzIEoXlu/wQj8AYExfXeKA9E1JL1G0l/UnOJ5kv5R0umSTpa0v6TX9jjNpyVdWdIlJP2m9txfS3q0pJtXf/uIpKdLekf1//N+h9DxNePLJX1Y0p4tddhb0uclfUPSn0n6lqRfdtT1zpLeJum3jd9PlXScpEdI+q/Gb5eV9B1Jfy7pc7XfHibpQZKuKemvJL2vQyfy36t+u2SF59sl3bb2/D5VvdKfqM/xkh4n6ZMT6axT9N2VQ2MStXKIraBCYMyByKAbgSERGNN3pzYQXbwiDZCox0s6TdINJL1L0nMkPaka9CFUPNNWri0JEkX5mxpB2ql6HyTqBZLIdD1E0mOq/4Zsoa/rd7JWkCgwe1lFoj4q6Z8blYDkQU6+XpEoZHgvxCTVK4n8UNKvKxJ1J0kfrH4gEwaxg4hBiOrlclU9miRqP0kPqEgUZPC9kq7SAhCk7nfV3x9ckcgLVXX+UfX320t6a5UJ5E83lPT3FZ6QNEhi6TI13x0FD5OoUWC2EkljDkQG3AgMicCYvju1geiAiggw+JN1SuV2kshQPS2DRJFZumKVgfojSXepXnI1SV+RdDNJH6j+tqOkx0p6sSRIVt/vZG+aJOrYqk5N+1+phURBBk9scZTzVCQK4vP+2u8QxvtLunyARM0bc0+S9NKKGIL9kQ0S1ZQ/ocp+QahKl6n57ih4zDPoKJWwkq1AYMyBaCsAdSNHQ2BM353aQPSmaunrPj1ok+npykSdV9J3q6wJe57I5vyxpFMk8duXqiW+V0j6tyrzk/ZFzfudKjVJFMTqX2p1/X61DNdGolgGhKSlgp3fKCmRqDtK+ndJjJMsA0JuIFVki+olZaLIzP2g9sNeFXkkU5QyUXdtyKb68eddq4wSWTPwZGnyRnNIFG24dZUdHK1DdCiamu+OgodJ1CgwW4kzUfaBNUZgm0nUeyR9QRJLU12lj0RBHthjxF6oc1eE7O6SXle9jI3q95TE31j2oxxRkQhI17zfmySKd0BMUoGY/YOkNhLF3qaf155lSY3sVCJRbe0le/a1DhLFfi7IYSpXrTJadRL1iYYs+7gStodIum61F4rlOrJM7CNjCTEt5zXH7IdW7UNH6WISVdoC1r/RCIw5EG00kG7c6AiM6btTG4heKGkPSddpoM5S3k2qvUx9JOrVku5R24zNsuC/SmKz9/mrbFT60u/SVcbqyZJYLoTAkY3q+v2YFS/nsScqZaKuUGXKIH+QsnpZZE9UV+IC4kYWjeVM9kHtIIn3PkrSs3pI1LMlsW8NIlq6TM13R8HDmahRYLYSZ6LsA2uMwDaTKDZ6P18SBAeylArLb/8j6YE9e6LYHE1mhq/U0sZnyNc/Ve+7jaSn1jZLp3eTXTpcEl+g9f3O8tmYe6LY28XyWlpiS/UdgkRBSNkXdsval4Es/f1ltTG9LRMF4SJLBUbPnUD/MomagBFchc1FYMyBaHNRdMtKIDCm77LHhcFoKhcQX0DSF6uv2CBDfL22b7U/6PqSWJ6CXEG0jq4Zh6/fIAB8dceXbWlTOhmWH1df4JHl4UgAjg0g48WxApCId1ZfwLG/qO93luoiJIoM2X82HIpsEHuyqEtzYznLbvdtycotQqKaGT3Usy8LYnkRSezDSoUjJdhozhLln1Zf57E3i0L26TBJfyIJO9SXJUv0EXQmEpX8t1Q9RtXrTNSocG+1sjEHoq0G2o0fHIExfXeKs/ndqj1Mu9SQ5Ss1NlpT0hECdeD5nJ89TRAEyFe9cBzB1auMzv1q7+EZzluCePHVH2Xe77kkiq8Dc86JYo/RiyoSxVEGELVU2LfFMQ5pn1L6e9c5UfUjDvrOidpd0scksXz4lpo+xmeOPwAvjmJgb1kqZOmOknRotTl/cKdf4oVT9N0lmrGYiEnUYnj56eURGHMgWr6WljQC50RgTN+dWiYqocFYAVlgCYnzlpqHUUb8hqMM2CsFMfhmy4vm/R7RbdnhEHAmajgs/SYjcA4ExhyIDL8RGBKBMX13qiRqSDz9rs1EwCRqM+3qVk0EgTEHook02dXYEATG9N2tXBLZED/Z9mZspe96OW/b3X689o85EI3XKmvaBgTG9N2tHIi2wYm2oI1b6bsmUVvg2RNp4pgD0USa7GpsCAJj+u5WDkQb4ifb3oyt9F2TqG13+/HaP+ZANF6rrGkbEBjTd70nahs8ajPb6D1Rm2lXt2oiCIw5EE2kya7GhiAwpu+aRG2I02xhM0yittDobvJ4CIw5EI3XKmvaBgTG9N2tXBLZBifagjZupe96OW8LPHsiTUwDEScxuxiBdUKA7BBljHhZH4jWCSPX1QiAAH1lKqftj2KRMYLCKA2xkskjkEjU5CvqChqBDgTGiJdpOc9GMALrioCvfVlXy7nek0aAjuWyGgT2ksQFpsdVt86vRovfOoYPj6Fjmy2ZsorOiK/OC5gwbw2+Y8ysVmcqv9kIGAEQ2Mq9CDa9EVgCgTH3ty1RPYusGwImUetmMdfXCJwTAZMoe4URyEPAJCoPJz+ViYBJVCZQfswITBgBk6gJG8dVmxQCJlGTMsf6V8Ykav1t6BYYAZMo+4ARyEPAJCoPJz+ViYBJVCZQfswITBgBk6gJG8dVmxQCJlGTMsf6V8Ykav1t6BYYAZMo+4ARyEPAJCoPJz+ViYBJVCZQfswITBgBk6gJG8dVmxQCJlGTMsf6V8Ykav1t6BYYAZMo+4ARyEPAJCoPJz+ViYBJVCZQfswITBgBk6gJG8dVmxQCJlGTMsf6V8Ykav1t6BYYAZMo+4ARyEPAJCoPJz+ViYBJVCZQfswITBgBk6gJG8dVmxQCJlGTMsf6V8Ykav1t6BYYAZMo+4ARyEPAJCoPJ42LcWoAABsSSURBVD+ViYBJVCZQfswITBiBRKK26vb0CdvDVZsuAiZR07XNWtbMJGotzeZKG4GzIeBMlB3CCOQhYBKVh5OfykTAJCoTKD9mBCaMgDNREzaOqzYpBEyiJmWO9a+MSdT629AtMALORNkHjEAeAiZReTj5qUwETKIygfJjRmDCCJhETdg4rtqkEDCJmpQ51r8yJlHrb0O3wAh4Oc8+YATyEDCJysPJT2UiYBKVCZQfMwITRsAkasLGcdUmhYBJ1KTMsf6VMYlafxu6BUbAy3n2ASOQh4BJVB5OfioTAZOoTKD8mBGYMALORE3YOK7apBAwiZqUOda/MiZR629Dt8AIOBNlHzACeQiYROXh5KcyETCJygTKjxmBCSPgTNSEjeOqTQoBk6hJmWP9K2MStf42dAuMACSKwgBxsOEwAkagE4HUV9K/DZURCCFgEhWCz8JGwAgYASNgBIzAtiJgErWtlne7jYARMAJGwAgYgRACJlEh+CxsBIyAETACRsAIbCsCJlHbanm32wgYASNgBIyAEQghYBIVgs/CRsAIGAEjYASMwLYiYBK1rZZ3u42AETACRsAIGIEQAiZRIfgsbASMgBEwAkbACGwrAiZR22p5t9sIGAEjYASMgBEIIWASFYLPwkbACBgBI2AEjMC2ImASta2Wd7uNgBEwAkbACBiBEAImUSH4LGwEjIARMAJGwAhsKwImUdtqebfbCBgBI2AEjIARCCFgEhWCz8JGwAgYASNgBIzAtiJgErWtlne7jYARMAJGwAgYgRACJlEh+CxsBIyAETACRsAIbCsCJlHbanm32wgYASNgBIyAEQghYBIVgs/CRsAIGAEjYASMwLYiYBI1fctfVNLPpl9N13AgBM4n6fQF33VuSfzz2xa583b8fUEVWY+PqSurQn6oFwHi/3mW8I8+O4/lA30+vylmxzaU3y3YoC4bjIkZdfh99c+C1V+vx6dOov5a0uskXWoFsF5R0g0k/b8VvHuIV0Ke3i7p2pIuvOQL95H0n5K+tqT8tokdJOlKku69gobn2GInSadJuoqkrzfq0NcXHiTpFpJu31LvUyTdWNKnV9Cm+iv/XNJnJBFTVonjipsx2uuTrf9U0lcH1vpHku4m6Y2SftnzbnzqFZIu3/JMnw27fKruAwM36RyvSz5/96rPDI3jFMaegyVdVtLfN1rf5zt9NuiLE0Pb602SPijpSyscw4eu81LvmzqJgkhcXdJHl2pdv9DfSHqOpKuu4N1DvPJ/S/p3SXSYXy35QgbOp0yYKC7ZrJWJXVkSA9AXV6AhxxZ9JKqvL0yNRK0SxxWYpsgryQrcSNLH5xCdZSp3MUk/qQbg7y1JovpsOCUSdccV4TiFsaeLRPX5ztRI1OtXOIYv0zcGl5k6idpN0uOrGfZLJX1SEp3mGpLeLOkRknaW9BZJb5P0DxVC+0t6h6Q9JD1W0m2rv+8p6f9KepSk91Uz/rdW73ySpB9LekED5QdWwYigwmz/JEn3qgLfByT9rSQC1QUl/ZukW0vi2SdWzzKLOLEibE+VxIzpeZIO7bHmZSS9t2rnv0rau8qaPatyyHdX7SBQUh4niRkZKdRjq/Yxk6QO35F0n0qOwACpojxU0gUk8c4TJL1Q0j9W7TlV0nMl3aTKYDymwh45/vu+Fbl7cdWOMwb3zDIvvJ+kS0s6vJpF0b5HSjq/pKdJwgf/StJdqtnvvpI+LwkSw4zrIZLaMMY36rbA9/DXIyR9qNbURKKw3Z0rvU+W9EpJ9b5AED1A0l0lfVPSdyX9cdVPCKLY7nJVJgK/vm5lRzKvbT7U5eM/mmMG+iK+x3IDvo9vEFOaOJJNxmfwq0Mqf7xZJcPfl50klPGSYbRC1unj2JD+Tsz6nKT7S/ph5UtMHsEOAk4MI3ZgW3wRX+mKP8RG+i5ZaDIqZCJvXouPqQUp2/IySfeo9OILELtkQ+JUn091+QA6yOgSb/H/10rCl4lRZCja+lYfsl0+T1+s4xiJu3X99f7W1T/w9VWOPZCo61exBnsy9mCn/661mfjeZYO+OLFD1RfvJOnnkp5ZxQtiRJcv9tnnIpL+qRpzj6/iD9h8rDaGg+Muki4h6abVGM7Yk8ZCYi3j11qVqZOoekr1w5IgQTjRxasBCEOQCv92RVQgRxCdh1XkCvkja+lqCA4E5lqS9qtI2K0kfULSM6ogwiBTLzgyA9GB1YD5z5L4h2CG86WlF5yIvUukXyF5dGwIGmnN50u6ZKWPd5MBI4VOB2grDNo4HEGH9rC084PKSXHMR1dLfGSraAuD8gMq/SxPMqgzuL9T0qsqYghBAreUGn66pAtVOLAHhwGOtoPPe6q2gQmkASKaZmbHSLpdtewFtmD8H2vl9d2VxdYQYPBjWe3LlS9BaBhUdqxI5hskvUTSqytbsPYPmQbTNozBtG4LyDoDCYQC26WSSBR6ISf4N2QXf2AgTEvbkDewxycJhuj9l4pEsXT7FUkMjPg4fQYShf90+VCXj1O/rsJy+DcqfyG4E4TxcWJKE0d8EWL44GpwBzuC5VGSaAt9ZNtKsjWDClgSL7Ahkziwwqfo39iTQQeyQIyBjDCw8veu+APpYaLF8i4436Hyz79rgEzfTXpfXhEeBl78LtnwnpWuNp9Cf5cPEFfx+YdL+kK1bIjdiTFdfevXPU7Q5fNpOa+O47Jxt66+PvZ09Q8wG2PsYfJNAoFJF+Tz2RWGtPl/emzQFyeIGRAoJj74IuMEdofkdvliXx9lEs7EiDELwke8JHbVl/MSjsQVxj5iGBMG4hQkH7mprgx1tn3dSBSDewrsDNwY4V2VI/9ltexHmyBWZF/ItLSRKAyVu5yH4XEuAhoFhyboMSDMI1HoZ08CAxrLkqTvCVLM3PeqBtIu46CTPQ3sByPjxuzgCpLI+vAulpz+pCI3EBwYP9kHSBSBk3rXl5C6BngcGBIFXmQ8CNDMRCETBEjwZPClvdSdWS7EChIAWSRb8f0NGQGbgz9Bgdk+ZJMljF0rQgKJgtj8piLD36p8DbLdRqLIUC2ynEdAgVyRXcLH0IttE4mCvIE7PkEhq0UhIELAGHQhaterJgiQKPypy4fI4Lb5OPtqugr+gl9ep3qAgMlMtI1EMSC9v5oAUXf2+EHaIeT045Qd3RA3ympGG4nCz8gy/J+qn4ETZImBkwkNhcGKzCNYd8UfyEjuch4DJpkiiA3EB0KL3tQXGBS7fOovenwA2zJApkkbGUf8E59EV1vf6ltG7/L5NhIVibvJeE0S1dY/aA8kapVjD7GWcYNCDGfyTbYSDCFR/NbVD7swI3PFGERCAlJO4Vn8CbvhE22+2OXYaYygXshDxH5aTQSbJKrenpOrSUPKuDHuMClc9MOarA63qofWjUSxnMJAT4E8sfH66MqRGbzS8hYzIJZJPtUgUWRQYPGLkijIC4NGGqh2r/6/HsTQz8CWMlH1DfEEPL6cIrVNYUAmc8ESSFepkyjkGaSahSzUL6p0KB2CgYnCctQ8EgUOLCkkEgUhYsbI7ASi1CwMtMwkIYQsZ6KLTalkTPo2r67Kd1fx3iaJqm9WhbxCRiA0tDmRh0SKCaTYtE6iEsaLkqikN305hV6IUfIp/IdBiawkhSwjS9eQZ7KUkGxKXZ7BrMuHsHmbj7dtVE+4v6YKumR9KfQJlp/aSFRqD3rIslyzksHPWMIC920rTRJVjxfY8iMVlpAo4gaDE4VsOyQ7LcOkTHg9/ixCoup6GZCJm3Ubsi+zy6fwO2Jgmw8weDLI1wsxg+VyCEBb3+r7+KHL55skKhp3U32bJKqtf5Blg0Stcuxhwp4+dGFrC8uM9TYzceuyQRdmxAGIeLPQH4nxXb7Y1Ufrk3om3BTGYOJRk0TV28NvrHKQgWUsY/mZbH9fRnJycWLdSNSLasy5SaLqmROIBR2f/SKQrvR1H50d1r4oiap/IcHso06iUjBIs/5EoupfvUCCYNes+VIWJVHMBm9ZpTuR5zN4UvbstTpM0g2rZRHaWx+YmpkocEhkkKUUOlIiUZADZoIsH7IUQLBLpAyCxbsJ/GRf+IcgwzIlexDoeJtQmiSq/pVcnUSxdJW+aPqzinziUyyptmG8KIlKertIFEvbZBxZUqaQbSWQYw8yPvgHpD0tuUHCmH12+dATGl8BJR/vI1EsJeLzEEcKWStmtW0kKrXHJOoPvaRJourxokmiwDplDBjgmOnjawyebfFnERJV19tGooifXT6F7bt8gL0uELC0ckC2nJjCRK35BWrqW30kqsvnmyQqGne7SFTbGJBI1FhjTxuJYk9dlw26MEv7qthnh20pxDOyU3yd3OWLXTE+ZepZxWBrDAVySeasSaLqOPIbsYaxyCRqRSNoc09UH4lisyWsnHVgZsmQDAYhOiZLZ+zLoFOzDMOAx/4o1rQxKgMOn6BDGup7VGhW8wuJNMCwzwCyxvIeJIkBjXXgVZCoRGxoB8SJ2SgDM7pwQPZMQRBJrx9X7bkCC/aqMMCyHMByHMs5YEpA4zf2XzRJVJrRooO9PMx62aQPnsiS8md/Bm1HF4M57d+EkkuiWM5jsz/+RKBgrwm2YGBrwxhb1W0BVtiL5WkCSSrNr/O6SBSZCQYP9JJNJKNJ5gLfZxmPYItv4wP4KyQKcgc5bvMhNqm3DRJ9JIoZMXrJ7nK0AV/h4CsmUXk9YRESxWQHAgxhx+fIADFx6Yo/ZIbZn0mfJdOHrfjvlMWvE4V5JIoMZpdPQda7fICJH3sJ6SfEVeIQHxAQI5chUV0+PwUSNdbY00aiWPLvskEXZvRrSC7bENh7y4cNxAZs9l9LkCj8iUwpS3joJMaQXWrbE2USlRcfBntqERIFA4ZE0GHZY8EeAmZspBUJ7vwdZ4PxQqJwHDbrMZsjHQlRwKkgG/XSRqL+V+2LvrQUAXGD4eeSKAYf9tt0FTbZ8c6U8WCJLmWy2GsAkaE9EKy0rMNyIu3FeRk4GWRZeiKYsWeKgZa9NWzqg3gx+CUSRYBOAzpy7MdKhc5LoGAgZt2ad4An6VdmM3SeTSjYMu13awv0LOGRlUsDAhupwZLlTWxByr8NY0gUZDvZAuLJ7Lu5qbqLRKEXXSnNzl44yD51oZA2J/tAtglyBvmlUBfOGSPDgK26fKjPx7vsmvbHMZmgsKyOTycS1YZjWybqs7VsxSb4UG4b6mf99M3+Wc5jnxR2pNAv2exNZhuS3BZ/+Fo4fYhDPCJWENeam3ab50Q1M1Fp6aXLp7Adme82H2ATPBu82ftEYfKHf0LIuvoWPtpVunweLNLyYBPHthWAeXE36W8u5zUnGYwBrGqQcRlr7CEOM8bV2wzp6bJBX5zAn4gbxHIKpIe4DwFaNBOFPL7D2ENWivGJepE9nZeJYsWI2JEyUWmvaW4/Kv7c1JfzcgDCCXBkOi2sHCLR/GSaNDIDPV8y1AskizXYtGyVo6/5DDopzAoXKWQZ6BBtheW+eoYiPcO+GIgMG+fr68ZkIyCFbARncOY52kt6ljNjqBv/zcBHxyLIzjuWgM2ZpKmZBddPTEcXyzP8La1/L9LudX8W4gOZZQ8UQRsyypdUqfRhXLdFFAd8l0ERWzb9l2wifkkga9q5y4fa6gNJww+ahXdCpilggC9uoy9EbZgjD4liUsQGfOI1JKRe+uLPkLcd9PlUlw9QX4gYMYPN6X0xJyce9vl8DpZDP1N67Km3p8sGfZjx29Uqkt71pXjSwfjK0nFbQZaPi5gYMJHEZ9dqc3jEMTaJRBFMIB/rUtKm7rb6MjBP9ST1dcF3VfVMJIrzVDa9sKGVDFizMBiyL8xl9QgkEsV5cZtc1jEeJhK1bmPPMn4E2epa3udL7VUciL1MPUeX2QQSRfqQVDVnaDQzTaMDaoUbjwDZGQhUc3/JxjfcDSyCAHtW2Pw79NUwRRqzYUo99myYQZdpziaQqGXabRkjYASMgBEwAkbACIQQWAcStU43WfO1yqJrwX03a491IzpONKaukNMOJGy/GgjIOa/ZNr8CjmXiQN+t96vCkPifvgBdxBv66rOqujbr1xc3F2lL17PrjM0Q7fc7MhFYBxK1LjdZ910e23cj+NQuj+27vT3TrdbiMfvVmcd/rLLUL0PdFr9aNg6kW++5UqpZui78jdqu+XVe/X199prSBcTpiIP64Z1RXJBfZ2yGaH/kHXxFyLmDfK2+8WWdSdTUbrLuC559N4JPjUT13d6+SR1iXW5It1+tl9cta6+pkai+ODAlEsWxCXxez9eiQ96c0Eeipo5N6R7D4b3sHe27Nqp0HQfTvy4kalU3WXP0AYdkciYTs3IOJuTsqHk3WXNXGRvZuSIhlRQ8mcFxLhPnXXB4GRuQ6zeC992s3XdbOnXi8k7OtOJgNA4145qbrhvGOeqhr3Td/F2/vZ1rdvhMnvuO+BKFs6L4ZJkD+DjBnPOR+HpoHcsqb0i3X515pAZnZ9GniDPb4le5caDr1nsyUUPHAb6q4qw8rm6ql5Qh5yonsgac70M8gZDU7dVXn644gh6+7iROcTcfZwYRD1nu42suDgDmqBDiJHGFWNNXuuImZ2Bx1xsnd3PMCzcocLAtMYqzqbjsnTPayFRxftqhmcFqithwLhhHXHBIcr30jQGMPxD09HUnd+TxD0Q4ihW25B34FfXiXkduTaiTqJ0r/DndnkusqU+yNbGB66voM/gDtuHL366/d42BJCkYd9Nhw1wlw5Vwo5R1IVE4z9A3WXNAHAdQctgmB3NCFAAfg0B6+m6yJghwnUH9dPMUPDkPhUMVufuO810IEjeuHZTYd7M2hKTttnTO4OEfDo/jYE0OKOOSUC5F7rphPF230OZI6ToQggoBh8/V+ZS9fto0B7qlA/sISBzeSXsgjpC51AGbgXkUxx1AScLNfmW/GsCdzjGZmhcHum69h0QNHQcgGgxiHKbYJFEpznHCPYQHskLsSqf3Ewe66kPs5Gy6tjjCzQYcvsjVKFz1wgGOnLJO7OBwTPDhEE8mnBC2eXemdcXNtguIOeQT4sCATlzjUEwKhIrDi+edicSzkKipYcM4BdEFw3rpGwOI70z407VB/D+3LHDOH+2LYAV5Y+zAvxg3ORyaMwQhtCkThW9wHiJjK2dZ8QwHl2IHTjnn8FP+zgHGYM4Y1PZ3Ju1dYyAn+CMLeePybg7bHu3QznUhUau4yRrWy4yrfu8RRuK4BA6nXPQm60SimPFBrjg7hCDD2jBZnHTadNfN2rDvrtvSCWpcJ8JBZjD1+oWPkJi2G8b7Uqncn9d183c9eEKiaAssn85BMMUWZBiQpwOgex0L7bRf2a+G9t2cOECmkljTdus9VwGNFQcSUSBTBLGB+EA+mKClOED2qKs+DI5dcYSBkAGfCRiFjAMxjnve0MVJ5tzYkO5dS3d3dtmjK262kSgOiGVpjwwbsZLlvnRROCdyMwmeV9YJG2zVNQbMI1ERrDgAlps+0oSdrSlcr0bSIJEo7MxNIByIzd843Z24S4aKk9b5b+zBb6yeYKu2vzMB6BoDubaK+2NZGeGaoz0rPUMu73b6y7qQqFXcZM39ZxirWSAlnMmy6E3W9WsckO+696zrZm2O4O+6LZ1ARBaoWTgqn6s02m4Y77v3jNlA183fTRJF6pUULKlyTi/nxHSCIzhB1HD6dSzpipehb0i3X52ZYaBwUTeH8LVlODfVr3LiABlkLvvm9oDmrfec/j5WHGh+8NJ2ATF39XXVhwxAVxzhxoVdGoGBk/UhkOmalnT2Vc4FxF1xs0mi6nGbzD33opJho+Rc/J6qvE7YEMu6xoAmiYJkMCkmExXFCjzJIjYPhq4v57FVhi0zXDNDBpCEAsQJv4LkQnzwCzKVrOBwuX3b3w/rGQPxQa5Dgjyx/YTMG1nIUcq6kKj6vUVD3WTNZcbsLaJTp2szYMOkDbmrbNH7g3Ivj+26WRujd92Wzr4DLiDFISl8Qs0+Bdb8cdhFL4/lVvium7+bJCpd+pxIFOlxZgybQKLsV/aroQNtThzg9GcGoLZb71n6GisOzLs7jyw9E6iu+rBE2BVHuOCWgTJlKVjuIdbSvmUuIO6Km6u8gHje5cxTwabt7ksmMEykIVGQSerKeM8VVSyhQqLq7Wu7ZxD7surQVcggsRwISaKgj6wTYxVjKUvBLPceVd3zCNlhlYf3sqoBYeIf/JDxjy0iZGLb/g5J7BoDIevc8kF2i9URlhDRj6+tvGwKiVrmJusHVGTgURXoLMORmoScMEPsI1HMtDF2/X67nOBJFqfrZu20KQ6SyP4E9oGR/mYvEnIQPlLRECfqzKW2kABSmYuSKPZ8dd38bRJ1ZpCnY9qv/rAcgy+mwNwVmOxXZ26SrZOErox016337Btik+6QcQC7ENeap+znkCiy4F31YTLXFUdYBmSw3ruapKZLu8lcLEOiuuJmSRI1Njb7VFjW9+LSF/tIFKstbABnOZV9wGSAhyJRySa8l2VBLqQmbvJ+SAy2ZtWCf0O2IFWMq4y116yWj9lPBbE6rsposezKsnLz74y1XWMg+4S5vBqfY3mYD65YauYjsZWXdSVRQ91kzToracBUCFx8KdIMLntI+kjF5HmW9DOkh/0DqXQFT7JaZG9S6rTvZu2u29JxkMOrr1nQh2PiZASwtg7EDeNkr7oKjtp183d9mYvZXzMTxQW2BFUyUdSBpYl1LE3chrwh3X51pkdwOzszw7Scl5blN9mvcuNA1633bCwfOg6wl4S9nlxYXS/zSFSyV1d9GDS74ghfHzJwsieGwuSPmETsaCNRxEniXFfpipsMzGl5kA3K87Ir+CN7seaVKWLDJJ9MD7asl74x4BYV+eB5lta5BJ2MT04mah5WjGt89cd4zGoOGSkm9fVzolii4+Mdfj++yoTh+3wtyZ5klvf4Ddvftlplafv7T3vGQPaDMRamFSV8Mu3Fm2fn8O/rQKIWaeQyN1nDoElbs98Hxxqj9N2s3XdbOgSGrBR7Cdg70VeYCdC2ZoEA4qSULrzGwGCddNiv/mAt+9Vwntt36/1QcWCo2vbVp6t/ML5AxLhcmM3pxJ6uQsaOwbitsPRJJqIvbg7VzmXes2pslqlTXQb8IbUnL/iiHJvwSogQE/uuu2shbmQt0Y9PQL74b+rFB0uMu2lvIO/r+ju/dY2BfAyBH7JMCKkerWwaiRoNuDVQxIZpnLVZCGR8lupiBJZBwH61DGqWmYcAA+d+HQ+x36W5eXne+/x7HAHbJANDk6gMkPyIETACRsAIGAEjYASaCPx/vbZtbdZzpSoAAAAASUVORK5CYII=)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VFrPdmofA538"
      },
      "source": [
        "---\n",
        "Finally, sanity test your model by running a batch of training data through it and verify the output dimensionality."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HBsckSK90JI0"
      },
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "from torch_geometric.nn import GCNConv"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "86q9G2rf5jzH"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tCfJmvEJCf3Y"
      },
      "source": [
        "## Training Loop\n",
        "\n",
        "Now define the training loop. One way to do so is to define the training loop as follows#\n",
        "\n",
        "```python\n",
        "for epoch in epochs:\n",
        "    train_step()\n",
        "    eval_step()\n",
        "```\n",
        "\n",
        "where the `train_step()` trains the model in batches for one epoch, i.e. the batches run through the data once. The function is defined as follows:\n",
        "\n",
        "```python\n",
        "def train_step(model: torch.nn.Module,\n",
        "               optimizer: torch.optim.Optimizer,\n",
        "               train_loader: torch_geometric.loaders.DataLoader,\n",
        "               device: torch.Device\n",
        "               ) -> \n",
        "               train_loss: Float, \n",
        "               train_accuracy: Float\n",
        "```\n",
        "\n",
        "Compute the loss and accuracy for each batch, then return the loss and accuracy over the entire epoch.\n",
        "\n",
        "Similarly, the `eval_step()` evaluates the model in batches for one epoch, i.e. for the entire evaluation dataset. The function can be defined as shown below. As with the `train_step()` function, compute and return the eval loas and accuracy over each epoch.\n",
        "\n",
        "Note that you can reuse the same function to report validation and test loss and accuracy, by appropriately setting the `eval_loader` and setting the `is_validation` flag.\n",
        "\n",
        "```python\n",
        "def eval_step(model: torch.nn.Module,\n",
        "              eval_loader: torch_geometric.loaders.DataLoader,\n",
        "              device: torch.Device,\n",
        "              is_validation=False: bool\n",
        "             ) ->\n",
        "             val_or_test_loss: Float,\n",
        "             val_or_test_acc: Float \n",
        "```\n",
        "\n",
        "In the `train_loop()` method, accumulate the training and validation accuracies and loss and return them as a list of tuples. Run the loop for `num_epochs` epochs, and report the losses and accuracies every N epochs. A suggested signature for the `train_loop()` method is as follows.\n",
        "\n",
        "```python\n",
        "def train_loop(model: torch.nn.Module, \n",
        "               optimizer: torch.optim.Optimizer, \n",
        "               train_loader: torch_gemetric.loaders.DataLoader,\n",
        "               val_loader: torch_gemetric.loaders.DataLoader,\n",
        "               device: torch.Device, \n",
        "               num_epochs: Long, \n",
        "               log_every=50: Long\n",
        "              ) ->\n",
        "              List[(Float, Float, Float, Float)] \n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GxxjVgoNJQjA"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zk0sJtuTJN14"
      },
      "source": [
        "## Training\n",
        "\n",
        "Now train the network using the hyperparameters provided in the cell below.\n",
        "\n",
        "1. At this point, if your code runs without errors, you can switch from using CPU to GPU. To do so, go to Runtime :: Change Runtime Type and setting Hardware Accelerator to GPU. You will need to rerun the notebook from the beginning once you switch from CPU to GPU.\n",
        "2. Set the device to `torch.device(\"cuda\")` if GPU is available otherwise set it to `torch.device(\"cpu\")`.\n",
        "3. Instantiate the GNN model and set the model to the appropriate device using `model.to(device)`.\n",
        "4. Instantiate the `torch.optim.AdamW` optimizer with model parameters and the appropriate `LEARNING_RATE` and `WEIGHT_DECAY` parameters.\n",
        "5. Run the training loop for `NUM_EPOCHS` epochs and capture the training and validation losses and accuracies returned.\n",
        "6. Use the training and validation losses and accuracies tuple to plot the loss and accuracy curves for the model training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dNdFYNw2Ni3G"
      },
      "source": [
        "# model parameters\n",
        "INPUT_DIM = dataset.num_features\n",
        "HIDDEN_DIM = 64\n",
        "OUTPUT_DIM = dataset.num_classes\n",
        "NUM_GCN_LAYERS = 3\n",
        "DROPOUT_PCT = 0.5\n",
        "\n",
        "# optimizer\n",
        "LEARNING_RATE = 5e-4\n",
        "WEIGHT_DECAY = 5e-3\n",
        "\n",
        "NUM_EPOCHS = 500"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UPfSoJ7u0XZg"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UmmpqgtF0Uz8"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AkAhbCUORQul"
      },
      "source": [
        "## Evaluation\n",
        "\n",
        "Now evaluate your trained model on the held out test set. You can reuse the `eval_step()` function that you used for evaluating the model with the validation set during the training.\n",
        "\n",
        "Report the accuracy value on the test set to 5 decimal places."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qapm9mDnQR_J"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j5AsqR1aRi2N"
      },
      "source": [
        "## Repeat classification with GAT\n",
        "\n",
        "Repeat the classification exercise using a Graph Attention layer (`torch_geometric.nn.GATConv`) instead of the GCN layer. Here are the steps.\n",
        "\n",
        "1. Either copy-paste the previous network and make the necessary changes, or refactor the previous network so it can be subclassed for the new convolution layer.\n",
        "2. Instantiate the model and set it to the correct `device`.\n",
        "3. Instantiate the `AdamW` optimizer with the new model parameters.\n",
        "4. Run the training loop for `NUM_EPOCHS` epochs, capturing the training and validation loss and accuracies.\n",
        "5. Use the list of tuples to plot the training curves for training and validation loss and accuracies.\n",
        "6. Compute and report the accuracy of the GAT based GNN against the held-out test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ht2ZD2Sz0fzt"
      },
      "source": [
        "from torch_geometric.nn import GATConv"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fvrt-FwXRc7-"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cd6Sq8dXTOtk"
      },
      "source": [
        "## Repeat Classification with GraphSAGE\n",
        "\n",
        "Repeat the classification exercise using a GraphSAGE layer (`torch_geometric.nn.SAGEConv`) instead of the GCN layer. As before, steps are as follows.\n",
        "\n",
        "1. Either copy-paste the previous network and make the necessary changes, or refactor the previous network so it can be subclassed for the new convolution layer.\n",
        "2. Instantiate the model and set it to the correct `device`.\n",
        "3. Instantiate the `AdamW` optimizer with the new model parameters.\n",
        "4. Run the training loop for `NUM_EPOCHS` epochs, capturing the training and validation loss and accuracies.\n",
        "5. Use the list of tuples to plot the training curves for training and validation loss and accuracies.\n",
        "6. Compute and report the accuracy of the GAT based GNN against the held-out test set.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BddkpK5H0nOR"
      },
      "source": [
        "from torch_geometric.nn import SAGEConv"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "95weFkBcS3gM"
      },
      "source": [
        "# your code here"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}