BERT_extractive_summarization.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "BERT extractive summarization.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "a635a56a0fbc4fbe89fce126f85581df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_c2824da7bd4e420b8e53b81c5292686e",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_ee7ec61caa684b6889e620c110e36f11",
              "IPY_MODEL_3a4b9b5c4d0c40ae983b5a2f066b989b"
            ]
          }
        },
        "c2824da7bd4e420b8e53b81c5292686e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ee7ec61caa684b6889e620c110e36f11": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_d60734a66bdf43ea9431d9508e6e9ea8",
            "_dom_classes": [],
            "description": "Downloading: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 231508,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 231508,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_ff82763f04af41c8a6a9676e8fa1267a"
          }
        },
        "3a4b9b5c4d0c40ae983b5a2f066b989b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_ee6155867eae4db6b2b1519df2923c16",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 232k/232k [00:00&lt;00:00, 1.16MB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_35d68fdcda004bb4be57e4c55668a0b9"
          }
        },
        "d60734a66bdf43ea9431d9508e6e9ea8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "ff82763f04af41c8a6a9676e8fa1267a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ee6155867eae4db6b2b1519df2923c16": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "35d68fdcda004bb4be57e4c55668a0b9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "ac195b0b1358441c8125399eec8d7029": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_4dea90beeb6f42c8b11db84926732aec",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_88b69fffc0f14828abd69929e5687a50",
              "IPY_MODEL_3736cbd82892435899c41576b71c1fa0"
            ]
          }
        },
        "4dea90beeb6f42c8b11db84926732aec": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "88b69fffc0f14828abd69929e5687a50": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_a29ffc90783f4c39a4ef6f181816647f",
            "_dom_classes": [],
            "description": "Downloading: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 28,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 28,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8b65ab94183f438faf12a804cc174745"
          }
        },
        "3736cbd82892435899c41576b71c1fa0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_4e72af7a236f4befa32f4dfcd370b693",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 28.0/28.0 [00:00&lt;00:00, 117B/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_c16a22ebcdf64535b4cbb75e3e068b65"
          }
        },
        "a29ffc90783f4c39a4ef6f181816647f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "8b65ab94183f438faf12a804cc174745": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "4e72af7a236f4befa32f4dfcd370b693": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "c16a22ebcdf64535b4cbb75e3e068b65": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "fee69f74e79e459c875d6a450e184ea0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_06cd415af14a4fc3a7247db511d0b0fa",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_772f2c1a5ec0483689f3400e6ce03a2f",
              "IPY_MODEL_a7c5c1bc0de440679811e32f99fd4936"
            ]
          }
        },
        "06cd415af14a4fc3a7247db511d0b0fa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "772f2c1a5ec0483689f3400e6ce03a2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_8805e240428a4c0db64dc5db5926dd45",
            "_dom_classes": [],
            "description": "Downloading: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 466062,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 466062,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_5f27e1ce748545f29d3ebcaad08b8bb8"
          }
        },
        "a7c5c1bc0de440679811e32f99fd4936": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_3cbe0efc6cfd46ca9a0be14cc04409d4",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 466k/466k [00:00&lt;00:00, 4.49MB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_a7f1e3701ed04726ace0897f52dbbdf3"
          }
        },
        "8805e240428a4c0db64dc5db5926dd45": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "5f27e1ce748545f29d3ebcaad08b8bb8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        },
        "3cbe0efc6cfd46ca9a0be14cc04409d4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          }
        },
        "a7f1e3701ed04726ace0897f52dbbdf3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          }
        }
      }
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/aashitadutta/FinancialNarrativeSummarization/blob/main/BERT_extractive_summarization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "biKCFW6WS396"
      },
      "source": [
        "# 1. Packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nVLI5G2zMXyV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "71739ce3-c068-497b-db0a-3630b03bac90"
      },
      "source": [
        "!pip install transformers # insall the huggingface library"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: transformers in /usr/local/lib/python3.7/dist-packages (4.6.1)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from transformers) (20.9)\n",
            "Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.10.3)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.0.12)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.41.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2019.12.20)\n",
            "Requirement already satisfied: huggingface-hub==0.0.8 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.0.8)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.19.5)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from transformers) (4.0.1)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers) (0.0.45)\n",
            "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->transformers) (2.4.7)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.25.11)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2020.12.5)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->transformers) (3.4.1)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->transformers) (3.7.4.3)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (7.1.2)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.15.0)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.0.1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b9F7N7DGNGdF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4e244a8d-c50f-49ad-aeed-1aaa9de81d11"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "from nltk import sent_tokenize\n",
        "%tensorflow_version 1.x\n",
        "from keras.preprocessing.sequence import pad_sequences\n",
        "\n",
        "from scipy.spatial.distance import pdist,squareform\n",
        "from sklearn.decomposition import PCA\n",
        "\n",
        "import torch\n",
        "import transformers as ppb\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "TensorFlow 1.x selected.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f2HpiJ6pS9b1"
      },
      "source": [
        "# 2. Pre-trained Model\n",
        "Distill Bert\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "346ecnmYS09M",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "097d3322-63e7-48bd-94b4-6753efc71fed"
      },
      "source": [
        "model_class, tokenizer_class, pretrained_weights = (ppb.DistilBertModel, ppb.DistilBertTokenizer, 'distilbert-base-uncased')\n",
        "\n",
        "## Want BERT instead of distilBERT? Uncomment the following line:\n",
        "#model_class, tokenizer_class, pretrained_weights = (ppb.BertModel, ppb.BertTokenizer, 'bert-base-uncased')\n",
        "\n",
        "# Load pretrained model/tokenizer\n",
        "tokenizer = tokenizer_class.from_pretrained(pretrained_weights)\n",
        "model = model_class.from_pretrained(pretrained_weights)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertModel: ['vocab_transform.weight', 'vocab_projector.bias', 'vocab_layer_norm.weight', 'vocab_projector.weight', 'vocab_transform.bias', 'vocab_layer_norm.bias']\n",
            "- This IS expected if you are initializing DistilBertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing DistilBertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cvPa_QNi9YTf"
      },
      "source": [
        "# 3. Data preprocessing\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KJ0I6K9qQV_T"
      },
      "source": [
        "## 3.1 Paragraph spliting\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m2mh_6iCQUOI"
      },
      "source": [
        "paragraph =\"\"\" We have laid the foundations for the future this year through  sizeable investment in research and development. \n",
        "We are pleased to report that we  have achieved success in both endeavours. \n",
        "We have  also made signiﬁ cant investment in our product portfolio,  \n",
        "both by licensing new technologies to commercialise  and by developing in-house proprietary technologies \n",
        "to  address the needs of mass consumer markets.\n",
        "\n",
        "  02\n",
        "Z GROUP plc \n",
        "ANNUAL REPORT AND ACCOUNTS 2006\n",
        "Z GROUP AT A GLANCE\n",
        "Z GROUP product portfolio\n",
        "OVERVIEW\n",
        "Bridging the gap between new technology and marketing, Z GROUP combines a highly effective direct-to-consumer \n",
        "marketing approach, a rapid time-to-market and a growing portfolio of innovative products. The Group’s strength lies \n",
        "in identifying new technological opportunities with well-deﬁ ned commercial potential and developing these by licensing, \n",
        "acquiring and creating IP . It has achieved signiﬁ cant growth by producing a number of strong consumer brands, establishing \n",
        "key partnerships and delivering effective, analytical marketing strategies.\n",
        "The Group focuses on low-risk, pre-paid cash generation and its product portfolio emphasises low-cost products, low-cost \n",
        "customer acquisition and high levels of customer retention, achieved through the provision of excellent customer service \n",
        "and product upgrades. In addition, all the Group’s products have a global market base, which continues to grow, with \n",
        "customers in the USA, Russia, the rest of Europe, Africa and Asia.\n",
        "“ Net2Roam was brilliant during a two week stay in Portugal. \n",
        "It meant I could continue my work during my trip at \n",
        "a reasonable cost . . . avoiding expensive hotel phone \n",
        "bills and no one actually knew I was away!”\n",
        "Joanna Rice, Sky News Presenter\n",
        "Net2Roam is a pre-paid service purchased online that \n",
        "delivers Dial-up and ISDN access in over 160 countries \n",
        "and from over 22,000 Dial-up locations, permitting users \n",
        "to browse the web and send and receive emails using \n",
        "their existing email accounts. \n",
        "“ I frequently travel the world, and often to far-ﬂ ung places, \n",
        "so a reliable and efﬁ cient means of always getting online \n",
        "– backed up by helpful customer service advice when \n",
        "I need it – is absolutely essential to me.”\n",
        "Mike Harna, Status Quo\n",
        "NetAway is a cut-down version of Net2Roam, which is sold \n",
        "in retail outlets throughout the UK. \n",
        "Net2Roam addresses the global need for low-cost easy \n",
        "Internet access when travelling abroad, and with increasing \n",
        "international and national business travel as well as \n",
        "increased laptop sales both in the UK and Europe, \n",
        "Net2Roam continues to satisfy that demand.\n",
        "Net2Roam and NetAway are Net2Roam Limited brands.\n",
        "“ Onbidder is one of the easiest ways to take control of – \n",
        "and win – eBay auctions.”\n",
        "Computer Shopper Magazine\n",
        "Onbidder is a software download that seamlessly integrates \n",
        "into eBay, making a user’ s bid at the last second of an auction \n",
        "– giving the users the best chance to win their item at the best \n",
        "possible price. This product was released in October 2005 \n",
        "and is also available in retail outlets throughout the UK. \n",
        "Onbidder is a CallPal Limited brand.\n",
        "“   Once you’ve used ONSPEED you won’t want to surf the \n",
        "net without it.” \n",
        " Laptop Magazine\n",
        "ONSPEED is the leading Internet acceleration software \n",
        "in the market. An award-winning product, it relies upon \n",
        "patented compression technology that delivers Internet \n",
        "speed increases of up to ten times for Dial-up and up to \n",
        "ﬁ ve times for Broadband connections. It is available online \n",
        "and in retail outlets in the UK and abroad. \n",
        "Dial-up users still make up the majority of Internet users in the \n",
        "UK, much of Europe and the rest of the world. ONSPEED \n",
        "delivers the only viable means to increase web browsing \n",
        "speeds and signiﬁ cantly enhance the sending and receiving \n",
        "of emails. For users who are on a Broadband connection \n",
        "of up to two Megabytes (2Mb) and who frequently do not \n",
        "receive the promised speeds, as a result of line quality, \n",
        "distance from the exchange and contention ratios, ONSPEED \n",
        "delivers speed increases of up to ﬁ ve times. A number of \n",
        "major Broadband providers now offer ONSPEED as an \n",
        "enhancement to their 1Mb and 2Mb connections. \n",
        "Z GROUP CURRENTLY COMPRISES SIX BRANDED PRODUCTS WITH FOUR ACTIVE BUSINESSES:   03\n",
        "“It’s no exaggeration to say that ONSPEED Mobile could \n",
        "revolutionise mobile web browsing.” \n",
        "PC Advisor magazine\n",
        "Z GROUP’s success in identifying new technologies that \n",
        "satisfy a growing consumer need has seen it develop its \n",
        "portfolio through the launch of ONSPEED Mobile, which \n",
        "was released in March 2006. Using compression technology \n",
        "and licensed page reformatting software, ONSPEED Mobile \n",
        "has been commercialised for a mass consumer audience \n",
        "to offer enhanced mobile phone surﬁ ng, which is faster, \n",
        "easier and more affordable. By compressing web pages \n",
        "by up to 80% and resizing pages to ﬁ t speciﬁ c mobile \n",
        "screens, users enjoy speed increases of up to ten times, \n",
        "web pages that are easy to view and use and data savings \n",
        "of up to 75%. As most operators charge per Mb of data, \n",
        "by compressing the data, bills are reduced.\n",
        "There is a rapidly growing market for mobile Internet \n",
        "access, and this unique technology will dramatically \n",
        "improve the customer perception and experience of \n",
        "mobile phone browsing, which has, since the advent \n",
        "of Wireless Application Protocol (WAP), been incredibly \n",
        "poor. This new product is aimed at the 700 million \n",
        "java-enabled mobile phone users across the world.\n",
        "ONSPEED and ONSPEED Mobile are \n",
        "Turbodial Limited brands.\n",
        "OnShare is a recent addition to the Group’s portfolio. \n",
        "A software service, it uses proprietary next generation \n",
        "ﬁ le sharing technology known as Friend-to-Friend (F2F) \n",
        "and delivers the highest level of security possible. The \n",
        "Group has made a signiﬁ cant investment in research and \n",
        "development over the last 18 months and, in OnShare, \n",
        "it now owns valuable IP . This represents a new and exciting \n",
        "departure for the Group, in addition to the other \n",
        "technologies it licenses and acquires.\n",
        "The beta version of OnShare was launched on 31 March 2006 \n",
        "and addresses the growing market for ﬁ le sharing, which \n",
        "currently accounts for around 35% of all Internet trafﬁ c. \n",
        "This is predicted to rise to around 65% by 2007 and \n",
        "OnShare is set to satisfy the increasing demand while \n",
        "addressing the inherent problems associated with other \n",
        "kinds of ﬁ le sharing programs. \n",
        "Offering secure, private one-to-one ﬁ le sharing, it has been \n",
        "developed to integrate into the very heart of the Windows \n",
        "application. It enables secure ﬁ le sharing, without the bundled \n",
        "spyware and malware associated with most other programs. \n",
        "It also integrates directly into Windows, so users can create \n",
        "Virtual Private Networks (VPNs) on-the-fly, without \n",
        "the requirement for hardware and set-up. Known as \n",
        "Instant Private Networks (IPNs) OnShare enables friends, \n",
        "family and small to medium enterprises to establish the kind \n",
        "of networks that have commonly been the preserve of big \n",
        "business. The Group is extremely excited about the potential \n",
        "for OnShare for both the consumer and business audiences. \n",
        "OnShare is an OnShare Limited brand.\n",
        "Z GROUP plc \n",
        "ANNUAL REPORT AND ACCOUNTS 2006\n",
        "\n",
        "\"\"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mc5UGzricKt2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e4a7e8b6-0d12-47a5-c18f-cd68a9d3ecc1"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rqCDnDWqRnAs"
      },
      "source": [
        "paragraph_split = sent_tokenize(paragraph) # split the paragraph"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ib-JgZ50RpmV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1b975913-7929-4bfa-cdd7-7ed3b50de90c"
      },
      "source": [
        "len(paragraph_split)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "45"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6appU6ewaPQV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ddadf2b7-7528-417a-e943-4d459d9ff02a"
      },
      "source": [
        "print(paragraph_split)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[' We have laid the foundations for the future this year through  sizeable investment in research and development.', 'We are pleased to report that we  have achieved success in both endeavours.', 'We have  also made signiﬁ cant investment in our product portfolio,  \\nboth by licensing new technologies to commercialise  and by developing in-house proprietary technologies \\nto  address the needs of mass consumer markets.', '02\\nZ GROUP plc \\nANNUAL REPORT AND ACCOUNTS 2006\\nZ GROUP AT A GLANCE\\nZ GROUP product portfolio\\nOVERVIEW\\nBridging the gap between new technology and marketing, Z GROUP combines a highly effective direct-to-consumer \\nmarketing approach, a rapid time-to-market and a growing portfolio of innovative products.', 'The Group’s strength lies \\nin identifying new technological opportunities with well-deﬁ ned commercial potential and developing these by licensing, \\nacquiring and creating IP .', 'It has achieved signiﬁ cant growth by producing a number of strong consumer brands, establishing \\nkey partnerships and delivering effective, analytical marketing strategies.', 'The Group focuses on low-risk, pre-paid cash generation and its product portfolio emphasises low-cost products, low-cost \\ncustomer acquisition and high levels of customer retention, achieved through the provision of excellent customer service \\nand product upgrades.', 'In addition, all the Group’s products have a global market base, which continues to grow, with \\ncustomers in the USA, Russia, the rest of Europe, Africa and Asia.', '“ Net2Roam was brilliant during a two week stay in Portugal.', 'It meant I could continue my work during my trip at \\na reasonable cost .', '.', '.', 'avoiding expensive hotel phone \\nbills and no one actually knew I was away!”\\nJoanna Rice, Sky News Presenter\\nNet2Roam is a pre-paid service purchased online that \\ndelivers Dial-up and ISDN access in over 160 countries \\nand from over 22,000 Dial-up locations, permitting users \\nto browse the web and send and receive emails using \\ntheir existing email accounts.', '“ I frequently travel the world, and often to far-ﬂ ung places, \\nso a reliable and efﬁ cient means of always getting online \\n– backed up by helpful customer service advice when \\nI need it – is absolutely essential to me.”\\nMike Harna, Status Quo\\nNetAway is a cut-down version of Net2Roam, which is sold \\nin retail outlets throughout the UK.', 'Net2Roam addresses the global need for low-cost easy \\nInternet access when travelling abroad, and with increasing \\ninternational and national business travel as well as \\nincreased laptop sales both in the UK and Europe, \\nNet2Roam continues to satisfy that demand.', 'Net2Roam and NetAway are Net2Roam Limited brands.', '“ Onbidder is one of the easiest ways to take control of – \\nand win – eBay auctions.”\\nComputer Shopper Magazine\\nOnbidder is a software download that seamlessly integrates \\ninto eBay, making a user’ s bid at the last second of an auction \\n– giving the users the best chance to win their item at the best \\npossible price.', 'This product was released in October 2005 \\nand is also available in retail outlets throughout the UK.', 'Onbidder is a CallPal Limited brand.', '“   Once you’ve used ONSPEED you won’t want to surf the \\nnet without it.” \\n Laptop Magazine\\nONSPEED is the leading Internet acceleration software \\nin the market.', 'An award-winning product, it relies upon \\npatented compression technology that delivers Internet \\nspeed increases of up to ten times for Dial-up and up to \\nﬁ ve times for Broadband connections.', 'It is available online \\nand in retail outlets in the UK and abroad.', 'Dial-up users still make up the majority of Internet users in the \\nUK, much of Europe and the rest of the world.', 'ONSPEED \\ndelivers the only viable means to increase web browsing \\nspeeds and signiﬁ cantly enhance the sending and receiving \\nof emails.', 'For users who are on a Broadband connection \\nof up to two Megabytes (2Mb) and who frequently do not \\nreceive the promised speeds, as a result of line quality, \\ndistance from the exchange and contention ratios, ONSPEED \\ndelivers speed increases of up to ﬁ ve times.', 'A number of \\nmajor Broadband providers now offer ONSPEED as an \\nenhancement to their 1Mb and 2Mb connections.', 'Z GROUP CURRENTLY COMPRISES SIX BRANDED PRODUCTS WITH FOUR ACTIVE BUSINESSES:   03\\n“It’s no exaggeration to say that ONSPEED Mobile could \\nrevolutionise mobile web browsing.” \\nPC Advisor magazine\\nZ GROUP’s success in identifying new technologies that \\nsatisfy a growing consumer need has seen it develop its \\nportfolio through the launch of ONSPEED Mobile, which \\nwas released in March 2006.', 'Using compression technology \\nand licensed page reformatting software, ONSPEED Mobile \\nhas been commercialised for a mass consumer audience \\nto offer enhanced mobile phone surﬁ ng, which is faster, \\neasier and more affordable.', 'By compressing web pages \\nby up to 80% and resizing pages to ﬁ t speciﬁ c mobile \\nscreens, users enjoy speed increases of up to ten times, \\nweb pages that are easy to view and use and data savings \\nof up to 75%.', 'As most operators charge per Mb of data, \\nby compressing the data, bills are reduced.', 'There is a rapidly growing market for mobile Internet \\naccess, and this unique technology will dramatically \\nimprove the customer perception and experience of \\nmobile phone browsing, which has, since the advent \\nof Wireless Application Protocol (WAP), been incredibly \\npoor.', 'This new product is aimed at the 700 million \\njava-enabled mobile phone users across the world.', 'ONSPEED and ONSPEED Mobile are \\nTurbodial Limited brands.', 'OnShare is a recent addition to the Group’s portfolio.', 'A software service, it uses proprietary next generation \\nﬁ le sharing technology known as Friend-to-Friend (F2F) \\nand delivers the highest level of security possible.', 'The \\nGroup has made a signiﬁ cant investment in research and \\ndevelopment over the last 18 months and, in OnShare, \\nit now owns valuable IP .', 'This represents a new and exciting \\ndeparture for the Group, in addition to the other \\ntechnologies it licenses and acquires.', 'The beta version of OnShare was launched on 31 March 2006 \\nand addresses the growing market for ﬁ le sharing, which \\ncurrently accounts for around 35% of all Internet trafﬁ c. \\nThis is predicted to rise to around 65% by 2007 and \\nOnShare is set to satisfy the increasing demand while \\naddressing the inherent problems associated with other \\nkinds of ﬁ le sharing programs.', 'Offering secure, private one-to-one ﬁ le sharing, it has been \\ndeveloped to integrate into the very heart of the Windows \\napplication.', 'It enables secure ﬁ le sharing, without the bundled \\nspyware and malware associated with most other programs.', 'It also integrates directly into Windows, so users can create \\nVirtual Private Networks (VPNs) on-the-fly, without \\nthe requirement for hardware and set-up.', 'Known as \\nInstant Private Networks (IPNs) OnShare enables friends, \\nfamily and small to medium enterprises to establish the kind \\nof networks that have commonly been the preserve of big \\nbusiness.', 'The Group is extremely excited about the potential \\nfor OnShare for both the consumer and business audiences.', 'OnShare is an OnShare Limited brand.', 'Z GROUP plc \\nANNUAL REPORT AND ACCOUNTS 2006']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UqvhcFAI9Ydn"
      },
      "source": [
        "## 3.2 Tokenization\n",
        "\n",
        "Note the start token [CLS] is always denoted numerically as [101] and the end token [SEP] is always [102]."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4b0ffe54R07_"
      },
      "source": [
        "input_tokens = []\n",
        "for i in paragraph_split:\n",
        "  input_tokens.append(tokenizer.encode(i, add_special_tokens=True))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lQ4x96ArSq8k",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "74aa381a-750d-4684-a24e-5df4a7d45135"
      },
      "source": [
        "temp = []\n",
        "for i in input_tokens:\n",
        "  #print(len(i))\n",
        "  temp.append(len(i))\n",
        "np.max(temp) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "83"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LF_xqD0cUOit"
      },
      "source": [
        "## 3.3 Truncating and padding\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DodMuc3PWa_S"
      },
      "source": [
        "input_ids = pad_sequences(input_tokens, maxlen=100, dtype=\"long\", value=0, truncating=\"post\", padding=\"post\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "neQ8BYhJiC-m",
        "outputId": "d06c1538-964d-4ad6-9869-f7c51c64fc8d"
      },
      "source": [
        "print('Original: ', paragraph_split[0])\n",
        "print('Token IDs:', input_ids[0])\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Original:   We have laid the foundations for the future this year through  sizeable investment in research and development.\n",
            "Token IDs: tensor([  101,  2057,  2031,  4201,  1996, 10100,  2005,  1996,  2925,  2023,\n",
            "         2095,  2083,  2946,  3085,  5211,  1999,  2470,  1998,  2458,  1012,\n",
            "          102,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,\n",
            "            0,     0,     0,     0,     0,     0,     0,     0,     0,     0])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tNvezhZ1Wj4g"
      },
      "source": [
        "## 3.4 Attention Mask\n",
        "\n",
        "This is a embedding layer of values [0] or [1]. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pGujN-TbT0EO"
      },
      "source": [
        "def create_attention_mask(input_id):\n",
        "  attention_masks = []\n",
        "  for sent in input_ids:\n",
        "    att_mask = [int(token_id > 0) for token_id in sent]  # create a list of 0 and 1.\n",
        "    attention_masks.append(att_mask)  # basically attention_masks is a list of list\n",
        "  return attention_masks"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sTl4PXfxZEJH"
      },
      "source": [
        "input_masks = create_attention_mask(input_ids)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G8DPpHE-aPNt"
      },
      "source": [
        "After this is done, then we can turn the data to torch tensor object."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CwwYDBJCaIc5"
      },
      "source": [
        "input_ids = torch.tensor(input_ids)  \n",
        "attention_mask = torch.tensor(input_masks)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0SeQM2k7Zjwv"
      },
      "source": [
        "# 4. Embedding\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qzTIH0XiaDTx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8edb441f-93b6-4531-bd0c-79d9612e30eb"
      },
      "source": [
        "# Get all of the model's parameters as a list of tuples.\n",
        "params = list(model.named_parameters())\n",
        "\n",
        "print('The BERT model has {:} different named parameters.\\n'.format(len(params)))\n",
        "\n",
        "print('==== Embedding Layer ====\\n')\n",
        "\n",
        "for p in params[0:5]:\n",
        "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))\n",
        "\n",
        "print('\\n==== First Transformer ====\\n')\n",
        "\n",
        "for p in params[5:21]:\n",
        "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))\n",
        "\n",
        "print('\\n==== Output Layer ====\\n')\n",
        "\n",
        "for p in params[-4:]:\n",
        "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The BERT model has 100 different named parameters.\n",
            "\n",
            "==== Embedding Layer ====\n",
            "\n",
            "embeddings.word_embeddings.weight                       (30522, 768)\n",
            "embeddings.position_embeddings.weight                     (512, 768)\n",
            "embeddings.LayerNorm.weight                                   (768,)\n",
            "embeddings.LayerNorm.bias                                     (768,)\n",
            "transformer.layer.0.attention.q_lin.weight                (768, 768)\n",
            "\n",
            "==== First Transformer ====\n",
            "\n",
            "transformer.layer.0.attention.q_lin.bias                      (768,)\n",
            "transformer.layer.0.attention.k_lin.weight                (768, 768)\n",
            "transformer.layer.0.attention.k_lin.bias                      (768,)\n",
            "transformer.layer.0.attention.v_lin.weight                (768, 768)\n",
            "transformer.layer.0.attention.v_lin.bias                      (768,)\n",
            "transformer.layer.0.attention.out_lin.weight              (768, 768)\n",
            "transformer.layer.0.attention.out_lin.bias                    (768,)\n",
            "transformer.layer.0.sa_layer_norm.weight                      (768,)\n",
            "transformer.layer.0.sa_layer_norm.bias                        (768,)\n",
            "transformer.layer.0.ffn.lin1.weight                      (3072, 768)\n",
            "transformer.layer.0.ffn.lin1.bias                            (3072,)\n",
            "transformer.layer.0.ffn.lin2.weight                      (768, 3072)\n",
            "transformer.layer.0.ffn.lin2.bias                             (768,)\n",
            "transformer.layer.0.output_layer_norm.weight                  (768,)\n",
            "transformer.layer.0.output_layer_norm.bias                    (768,)\n",
            "transformer.layer.1.attention.q_lin.weight                (768, 768)\n",
            "\n",
            "==== Output Layer ====\n",
            "\n",
            "transformer.layer.5.ffn.lin2.weight                      (768, 3072)\n",
            "transformer.layer.5.ffn.lin2.bias                             (768,)\n",
            "transformer.layer.5.output_layer_norm.weight                  (768,)\n",
            "transformer.layer.5.output_layer_norm.bias                    (768,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bpTmWGDrbe3L"
      },
      "source": [
        "with torch.no_grad():\n",
        "    last_hidden_states = model(input_ids, attention_mask=attention_mask)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1Kw2DjpYbtxy"
      },
      "source": [
        "sentence_features = last_hidden_states[0][:,0,:].detach().numpy()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ozMJlunXdaIC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0bf3a052-6b88-4c58-93c6-dc5bf60a1081"
      },
      "source": [
        "sentence_features.shape  \n",
        "# get the embedding sentence data shape = (number of sentence, BERT hidden layer)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(45, 768)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6EsnTnfwdupS"
      },
      "source": [
        "# 5. Machine Learning\n",
        "\n",
        "Using the traditional machine learning tool, such as nearest-beighbors or clustering, to select relevant sentences."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Baw4m7_8gIW7"
      },
      "source": [
        "## 5.1 Visualization"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B96EflyNdHbZ"
      },
      "source": [
        "array_similarity = squareform(pdist(sentence_features, metric='euclidean'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4fEnaiwzeBuJ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 301
        },
        "outputId": "89ead466-55a8-4c16-df2c-0b86f29c0e89"
      },
      "source": [
        "sns.heatmap(array_similarity)\n",
        "plt.title('visualizing sentence semantic similarity')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'visualizing sentence semantic similarity')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVQAAAELCAYAAACGSn6XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2deZxdVZXvf79b85CqVGWozPOcQEhShNAghElxaECfdqsttoDQdj+ndv60vlb7OfZTFNtGRQYbEERwQAVEkDCpCQlTSEhC5nlOJZVKUvN6f5wTvNT97br3pm6lboX15XM+VNbZd599ztl333PWb6+1aWZwHMdxek6irxvgOI5zquADquM4To7wAdVxHCdH+IDqOI6TI3xAdRzHyRE+oDqO4+SIfjmgknyI5D/28jE2kbw4/vvfSN6cwWdWklzYm+1y8g+SY0g2kSzIYZ1NJCec4Gdf7Yckv0Tyzh6044ck/8+Jfv71RmFfN+BEMLM3n+TjfS3DcjN7uy25hKQBmGxm6/q6Lf0JkpsAfNDMHgUAM9sCoDKXxzCzE64vl/3QzD50/O94kL7TzEblqv5TjX75hOo4Tu+Tyyfu1wt5OaCS/CzJ+7rYbiD5vfjvx0l+MP57EsknSB4iuY/kPbF9HEkjWZhUR/LnJpJ8jOT++HM/JTkw0J5XX5tIfj9+HTu+tZP8Urwv2U3wJZI/J3k7ycPxa1h9Up1zST4f77uX5D0kvxI4vjzHeN80ko+QPEByDcm/S9r3E5L/TfKB+DhLSE6M9z0ZF3sxPo+/j+1vI/kCyYMk/0zy9KT6NpH8FMnlcVvuIVmatP/y+LONJNeTvDS2V5O8heROkttJfiX0ZSU5n+SyuI7dJK9P2rcgbtNBki8mu1fie/uVeH8Tyd+SHBTf10aSS0mOSyp/A8mt8b5nSb6hy/2W947kHQDGAPhtfJzPdO1rJGtJ3kZyB8kGkr8+gftqJCcl3ccbGbm6mkj+ieQwkt+N619Nck6X+3Rx4Jj3ktwVH/NJkjOT9v2E5A9IPkjyCIALYttXSFYAeAjACP61748geZTkoKQ65pLcS7JIHf+Ux8zybgMwFsBRAAPifxcA2AlgQfzvxxG9cgHA3QA+j+jHoRTAubF9HAADUJhUb/LnJgG4BEAJgCEAngTw3aSymwBcHP/9JUSvOl3beQaAvQDmBD7TDOAtcfu/DmBxvK8YwGYAHwNQBOAdAFoBfCVwPULnWAFgK4CrELlv5gDYB2BGvP8nAPYDmB/v/ymAnyXVawAmJf17DoA9AM6K2/yP8TmVJJ3fMwBGAKgFsArAh+J98wEciq9pAsBIANPifb8C8KO4vUPjOv4pcK5/AXBl/Hdl0j0fGZ/LW+L6L4n/PSTp3q4DMBFANYCXAbwC4OL43G8HcFvScd4HYFC875MAdgEoTXfvut5n1dcAPADgHgA18f09P5v72vXexPdxH4B5cbnHAGwE8P64fV8BsCiTvgvgagADEPX77wJ4IWnfT+J7eE5Sm36CuF8CWAhgW5dzeBDAPyf9+zsA/quvx5C+2vLyCdXMNgN4DsDbY9OFAI6a2WJRvA3RADzCzJrN7OkMj7HOzB4xsxYz2wvgegDnZ9pGkkMA/BrAR8zs+UCxp83sQTPrAHAHgNmxfQGiL/L3zKzNzH6JaJAJETrHtwHYZGa3mVl73I5fAHhX0md/ZWbPmFk7ogH1jG6Ocx2AH5nZEjPrMLP/AdASt/c43zOzHWZ2AMBvk+q7BsCt8TXtNLPtZraaZB2igenjZnbEzPYg+tK9u5tznURysJk1Jd3z9wF4ML6enWb2CIBlcd3Huc3M1pvZIURPU+vN7NH43O9F9IMBADCzO81sf3zdvo1ogJmaVFfo3nULyeEA3ozoh6Yhvr9PdHOumfbdX5nZs2bWjOgHqtnMbo/bd0/yuXWHmd1qZofNrAXRYDubZHVSkfvN7E/xNW7OoMr/QXRvjrsI3oPoer0uycsBNeYuRDcHAN4b/1vxGQAE8Ez8anZ1JpWTrCP5s/gVtBHAnQAGZ/jZIgD3AbjLzH7WTdFdSX8fBVAavxaOALDd4p/0mK3d1BM6x7EAzopfgQ+SPAjgHwAM66YN3YkdYwF8skt9o+P2pqtvNID1gTqLAOxMqvNHiJ5UFdcAmAJgdfya/raket7VpW3nAhie9NndSX8fE/9+9dxj18Wq+NX3IKKn2uT7H7p36RgN4ICZNWRQNpu+m/G5hSBZQPIbjNwxjYieZIHXnnd3/VBxP4AZJMcjems4ZGbdPRyc0uSzyn8vgG+THIXoSfVsVcjMdgG4FgBIngvgUUb+wUNxkXIAjfHfyQPN1xC9Vp1mZgdIXgHg+xm27b/iOr+Q+em8hp0ARpJk0qAaGpC6O8etAJ4ws0tOsB1d2Qrgq2b21RP87MSAvQXA4PhJsVvMbC2A95BMIHKF3Bf76LYCuMPMrj2Btr2G2F/6GQAXAVhpZp0kGxANbpnQXYq2rQBqSQ40s4PdVhK4r9Z7sy7eC+ByRG6QTYh+RLqed3fnlrLPzJpJ/hzRU+o0vI6fToE8fkKNX8MfB3AbgI1mtkqVI/mueNAFos5hADrjz28H8L74l/lqvPYLPwBAE4BDJEcC+HQm7SL5T4hcA/9gZp3ZnxmAyE/YAeDDJAtJXo7IBxk6pjxHAL8DMIXklSSL4u1MktMzbMduAMlzHX8M4EMkz2JEBcm3khyQQV23ALiK5EUkEyRHkpxmZjsB/AHRj2NVvG8iSeleIfk+kkPia3t8QOpE9AbxtyTfFN/PUpILk65LNgwA0I7I/11I8t8BVGXx+a7X7VXi830IwI0ka+J7cp4q28197S0GIPpx24/oQSOj6YBJ7AYwqIuLAIj80x8AcBl8QM1r7kL0axp63QeAMwEsIdkE4DcAPmZmG+J91yIaKPcDmAngz0mf+zKAuYieZB8A8MsM2/QeRF+mHUlq579l+FkAgJm1Inr6ugbRoPE+RINjS+Aj8hzN7DCANyLyR+5A9Jr6TUT+wEz4EoD/iV+h/87MliG6Zt9H9AVfh+iLksk5PYNIHPsOomv6BKLXdCAST4oRCUUNiNwlw0U1AHApgJXxud4A4N1mdszMtiJ6uvo3RAPhVkT39kT68MMAfo9ItNqMSIDK5lX36wC+EF+3T4n9VyLyj65GJPJ9PFBPd323N7gd0fluR3QvlCYRxMxWIxLSNsTnPiK2/wnRD8Fzsf7xuoWvdeM5fQXJJQB+aGa39XVbHCdbSD6GSFNIG1F4KpPvT6inLCTPZzSXsJBRGO3piJ6aHKdfQfJMRG9796Qre6qTz6LUqc5UAD9HNDdzA4B3xv43x+k3kPwfAFcgclcc7uv29DX+yu84jpMj/JXfcRwnR/iA6jiOkyPS+lBJTkM0XWVkbNoO4DeheaEpBygeKX0Ktw65QJZPBDwQb5yuZ7VsX5uaz6SjU8/PHj1ZB69Yhy5fWKYbE5p9WnZGrd4RoPNAk7S372uV9qLRFbqiROB3MaHPy47p+hNV5dLeuu6AtBcM1PkvCmp0PQUzp0h72+Ll0l54+iRpZ92wFFvjD/4oy5ZNKJb20LUpWnC6tB/5+VJdjT5VFE+UeXZwZNkhaS8Zqe/hjsVl0t5wRNtnvGGftCdKdeKogiG6TyWmqRgN4Nhvn5P2Nct0kOHsq/V5Nb+wR9oHPfBEpsEVQdr2bcjYj1k0eEKPj5dMt0+oJD8L4GeIw+PijQDuJvm5XDbEcRwnJ3R2ZL7lmHRPqNcAmGlmbclGRinVVgL4hvoQyesQJdoAC6qRSASerBzHcXLNCQcw9px0PtROvDYxxnGGo5sQOTO7yczqzazeB1PHcU4qnZ2Zbzkm3RPqxwH8keRa/DU0bwyiXKIfzuQAIV/p1XsXSfvy0Tq7XPkc7ZMa1tKYYgv9QJWfpkPSWzfq6XOHt2r/W0uzvmxVx7rNhZHanon69+zoLu3vqn2jXmKo+dGXpL14yiBpb1mTes0AoKAyFPmqSZTo69C6UfsJy+bqiNiCoV1DwyM4tE4feFCqD7V4sHaFsUz7eRPDA4nFqnU/Kxqh67Fm/dpoTfpaNjfqa1ZU3SbtddO0nz2xVrsJi6fpJF6tq7XPkkd0OwuK9PmWnjVa2mvXaD9724Zj0t58oPcWA7COtDl4eo1uB1Qz+z3JKYgSdySLUkvjPIyO4zj5RR++8qdV+eOsP1klUXAcx+kzekFsyhQPPXUc59Qin59QHcdx+hW9IDZlSq8PqKGJ+iHx6fStL0j7kz9dIO2/LQml1Uxl3r2hObxaFJkzSE+SHjhMO9orzspuYv++h7UYtm+fnhnRdtMaae/sKJX29lW6naXlgQUpd2vz3j16dY2Wdt19Jk7RSxEVP6fFsw2/1J1k7JYnpb1gWGou6EefHilKAmOor8GosdukvbpRi0CrHtb5pwsT+ss7ZKiuZ+cuXc+e3foa7Arc2zXFWtR5y106eIWBCISqwfpeDWjXy6RtWaSFxcfbdN9/1059nTvaei9IM29FKcdxnH6Hv/I7juPkCBelHMdxcoQ/oTqO4+SIfBalSM4HYGa2lOQMRIuorTazBzM5QChLVCjyKSQ+nbdfT4X90+CzUmybTGfiaQtoUvOq90t73VsDmXjGTNb2M+RK10FK//JjaR9VpSOu2lu0I79sqM4exVASqspA5qFBWrio3BhYYj7Qb4sna+GFVfp6jjxth7QXjtFRP6hMbWdFp474GRPIMFY6XItABTOnSvvkeX+W9oIqfS1bd+nXzomDdF8rHqWFwpJFWmAZ1alvbs3wo9JeeYb+TiSGalE3MUJHqdW8rEXj83bqL1flQl1/w106mi4n5OsTKskvAngzoqV2HwFwFoBFAD5Hcs4Jrt/uOI7Ta1iHDuM9GaR7Qn0ngDMQLUu8C8AoM2sk+S0ASwD4gOo4Tn6Rx9mm2s2sw8yOAlhvZo0AYGbH0E22KZLXkVxGctnt233dOcdxTiJ9mG2q20X64rXiLzCzoyQTcVw/SFYDWGRmc9Md4Pkxl8sDDBujMx59b5v2uVzRqicgn7NvibR/fMR5KbYJHdrfFfKtXlig/Ty1w45Ie9U8PQk7xL6n9KvJ7r06K9b4qdr/tm7VEGkfUKr9igOq9bUMHbcj4K/bC52Na3bdXmkf/AbtJ1zza13P+Dna/1k8Uftof3FvaoDGhA59DaZO020sn6X9yKt/pSe0twWuTWWJ9mtvbNbXuI26E+4s1PbdBXowuKZan9fOPfqaDSzXfWH4XN3H1/25RtqXBNJ0Xlgc8L8HmLH+gR5n0G9+9tcZZ+wvnXdFTjP2p3vlP8/MWoBXk6QcpwjAP+ayIblEDabOqY0aTJ3XKfk6D/X4YCrs+wDouEzHcZy+xENPHcdxckS+TptyHMfpd+TzxP6eElrSOdsfkdBkfeUv/e4OnaXoC8MXSvuEdt3GimotaJQN0z6aRI3OyhSi5ZjOSBQSOpoPaVGnOiAsmOnzam3Rt726QtfT1qbFvE5dHC3HAtmsAhQX6utZOFjXw7JUESuwEjgKGeho1LpFaGmU0oC4EpIhyyu0KFVyTLdnMPRralFbYBnswAQdC3zfaip11q1QtqnCOi3OVZTq80KrFqVC36Gjh0PnlQP6cEDtvRxajuM4fYBZR8ZbOkj+K8mVJFeQvJtkt1N5fEB1HOfUIkfzUEmOBPBRAPVmNgtAAYB3d/cZ96E6jnNqkVuVvxBAGck2AOUAdOKJGH9CdRzn1MI6M9+6q8ZsO4BvAdgCYCeAQ2b2h+4+0+tPqKMD2X7KT9MRI6FlSkLRTCr6KSQ+fWXn49J+f40OBBhyvr48LA1k7pmjl3UJMWjxw9JefUwLCIlCLaQwocuXVGofUUGZrqcjIDKFslZhvY6+GTpdR9kkBo+Q9kHDA2uvBAQWFqbe89Ft+qmkbogW/op0sjMgERB1hupzCtEZUMmmDNXr14eEwvJmHU1X3aJFncqhWgSqLtf2gkp9cxPVWmQaUKvbP3ur7jxVE/V9KdjSi8JRFqIUyesAXJdkusnMbor31QC4HMB4AAcB3EvyfWZ2Z6i+dNmmzgKwKk6IUgbgcwDmAngZwNfMrBdzcDmO45wAWUwhigfPmwK7Lwaw0cz2AgDJXwL4GwDBATXdK/+tAI4nWLwB0Wp234xtt2XcasdxnJNF7pKjbAGwgGQ5SQK4CMCq7j6Q7pU/YWbHn9nrk5KhPE1SZ5rFax+j/9+UybhyROYrkzqO4/SIHM1DNbMlJO8D8ByAdgDPI/w0CyD9E+oKklfFf79Ish4ASE4BEMziamY3mVm9mdX7YOo4zkmloz3zLQ1m9kUzm2Zms8zsylB+k+Oke0L9IIAbSH4BUTKUv5DcCmBrvC99gwLO+daNek36yKuQSmiZkoeOpEa2hCKfQuLT5Q06smr1Y7OkfeBoLVCUtOplMkK0NukIpPUbB0n7zHl7pH3ZKr1UyMzhOn/NylU6Gqg4kOK2pkj3oYY2ndLu4BI993nO8OxSuSXKAhFXnami2v4CfS2L9wbS1o0NKHCibgA4ckif67YGXX9B4FpuL9D1jDctLO41XX5LsX4WmhpYJmffpkDkUyCSaWCxFp8qRuhBaPN2fc/tKZ1ackStTt+ZE/I1lj8WnT5AsgqR0lUIYJuZBWRZx3GcPibfY/njTP0v9nJbHMdxek6+PqE6juP0O/L9CdVxHKff0JGnGftzcoBAVM7hrTrSY84gLaTUvTWwZs3PU2MLQo72UORTSHyatm6FtC9uPFPaC1ZlvJQNAKC0RDvyVxRpIaL+kmnSPnz5Lmk/2KAjuuqKtACyrV0LF4c69LWfVa1FpqOB9H2F586T9s5nF0t7YqgOZ1IRadPv1IJgWZWejFIwUItYiVE6mmvI9M3Sfuw5fa6jZuiYl03LR0t7U4euZ1qNvsYDGnSkYZmuHi+u1eVrjum+NmihFp8YWKesLrB0MwNpEoPRd7nAn1Adx3FyhA+ojuM4OcJFKcdxnBzhT6iO4zg5wrLTMnJJ2gGV5AQA7wAwGkAHgFcA3BXPTU1L6Om7pVkfeuCwQOq6MZOlvXbYyym20JpPobR7ocinkPi0YM9SaX9+5FxpDzHs9KPSfu6zWjDhgGHSPmjIemkvq9FCQWcgF2L5dl1+YJ1uZ2mdvrktewOKQ0B9HXimFihZFVijqzhVzBtbr7tjolxfy8LxOrrM9uhotFA9o2dp8SlRrq9xfclBaS+u0iJQ7Rj9fWhrDZzXuFppnzVOn1dIHGKtFucKKnU7Jw/XkYztrfoARaW9uNRze98tI92t1kbyowB+iGgtsjMBlCAaWBeTXNjrrXMcx8mWHCWYPhHSPaFeC+AMM+sgeT2AB81sIckfAbgfwJyct8hxHKcHWCAfw8kgk9lgxwfdEgCVAGBmWwAE1womeR3JZSSX3b5tZ89b6TiOkym5y4eaNemeUG8GsJTkEgBvQJRcGiSHANDpaPDaLNhNn367/LmoOqZ9SRVnaR9Q4oyzpb1q1QZdvibV/xZaoiSUJSo0UT/kK52z/TlpD9H4L2+S9roD2ieaWPhOaR/wO+3TLakfK+1tL2+T9oomPWm7YrL+3S2cqP1stkTXz1kLdD0BnxdP1/e8YHzqi1HlWZeh7Zb/m1o4sKQJp87Q9qF6Znxx0WPS3rFuq7QnhtZI+4D1Oq9Q1Xh9DYqm6MxgI0p1AEzBxZdJe+Wmu6U9UamfixKnB5bzCQSjDNrzO2lvWKb98qXVvejnzNdpU2Z2A8lHAUwH8G0zWx3b9wLQufDyADWYOqc2cjB1Xp/04St/WpXfzFYCWHkS2uI4jtNz+lDl93mojuOcWuTzPFTHcZx+hUdKOY7j5Ih89qE6juP0K/JV5Xccx+lvWPspnGDacRznpOKv/I7jODmiD1/5ab08xaDxmkvkAaxdn3TDizoqpzSwlMWxQ6mRHi2BJTgGjdVZpVqbdOaexn06KiSUJapkwQRpD1H1+Yel/eYhF0j7ebU6Y1Bbm27/xoZqaZ8yVAe5memoooaDOktXqHxtjb4+A+qapX33Br22/YgZOoMURXKqZX8eLsuWUc9JnDBeX4OBl+gsVBvvbNL1l+t+WVKh7UcO6SVHjh7VGbdGTdURhQ3b9D0pKdPne6xJ119Zo+9J5QQ9LjRt0Pd87069xEqo/Z2BqaJ1i57QB8iCI//xDxkPahX//tMeHy8Zf0J1HOfUwqdNOY7j5Ih8zTZFsork10neQfK9Xfbd2M3nXs02ddtqnSjDcRynV+joyHzLMenS990GgAB+AeDdJH9B8rgTSKcOQpRtyszqzaz+qmmjctRUx3Gc9FhnZ8Zbrkn3yj/RzP5X/PevSX4ewGMkdY4wQfu+Vmk/uksLKfv26TXgR1Vp5/buvanO8LZO/TtRfUwvJ7F+4yBpX1GkBYTQEiWhtHshQuLTB/cukvbdcydJ+6PL9I9WQeDnclGDFl5KA/0r9AI1CVp8eqphiLS/fZ7OjVuyQws4hYN191TLlxxarE+2zLT6UVgaeDop1Pe2tFS3sbFRC5c8rPvO8nYtwA0OJfRYo80HWvRxp47bK+17G8ulvTWwlErVLC1WFZXr7/O6Np3hbd9y3c7BJbr+OmnNkjyeNlVCMmEWzUMws6+S3A7gScTJph3HcfKKfPWhAvgtgAuTDWb2EwCfBKB/qhzHcfqSfF1Tysw+E7D/nuTXct4ax3GcnpLHT6jd8eWctcJxHCdHWHtnxluu6fYJleTy0C5k6D8uGq1Fpto36qiitpu0F769RY/946emrgfeLKKnACBRqH+5Zs7TEUj1l0yTdg4YpusPrPkU4rwr/l3aQ+JT3cPrpP3gx/Q6XM3LdTRQyXgdZVMwXkcbNT6wUdpVlBoAzDxfe4O2PqAFihDNW7SI2PCXQym2cQktAhUk9Jcm1EfKD+jorFf26zWi5s3QQlvFAr0W1OQiLT61vKTrKRqrRayjL+i1qVat0oLg7HO1WFVQqyOoOg5qEa7lsB4yzp24Q9oHXnuWtHdu1mtx5YQ8nthfB+BNABq62AlAr2znOI7Tl+Sxyv87AJVm9kLXHSQf75UWOY7j9IR8HVDN7Jpu9r03tM9xHKev6O2ET93hsfyO45xa9ILYlCm9nr7v8EffJg/QtvWwLN+wVgsXA8fryIrVS1Od8NXlumxVrRY5XtqqI4eGF+tIoEFDdBrAAWO1Iz/E3lU6euWFQ1pkeuv7dHsG3vCMtK+eNEvaN+8fKO33a60KV5u+bqUlWmDZcUjHfJz5Hl1P61otBJXODeiepal95MXv63tSVdYi7WPfqNteMHGEtDf9XkfBNe7U/fX6ozp14gUtWgwbQ31tyop1nzraquuZ8VadZnD/M/p7XlisB5+aN2gxuX27/t6ufEJHGz5YqkWvt7frvly/7dc9Tqd36KqLMx7Uqm97tNvjkRwI4GYAsxAFDV5tZn8JlfcnVMdxTi1y60O9AcDvzeydJIsB6KegGB9QHcc5tcjRGz/JagDnAfgAAJhZK9JEiKZL33dpcuUkbyG5nORdJIPzUF+Tvm/FlmzOwXEcp0dYp2W8JY9V8XZdUlXjAewFcBvJ50neTFL7QmK69aGSfM7M5sZ/3wxgF4AfA3gHgPPN7Ip0J3f4438rD8AS7QPafr/2JQ2Zqn0u217UE58pPCNDx2r/z7GDui0HG7RTccSk1InlAFBx4VhpD/HU9/SPXVNCZwBaeLrOLXtwq/bjTVu3Qtq31E/Rx23Qk+MXHdaT1Ge1a1/1hEmpwRYAUHmGvp5/vE/7dOdP0JPdB9TrPv37e1P9liNM+1ArSvS1H3eu9kFu+4t+06upCyyHU6WzWR3aru/V2r06cGDykK5TwCOCWa6ov89HWrQvs6RQ+5LHzdTH3bRSt7OxTdd/2uk6AKFosH6WG3j3oh77UA++54KM3/m7Ox7JegCLAZxjZktI3gCg0cz+T+gz2bzy15vZGfHf3yH5j1l89qSiBlPn1EYNps7rE2vPmQ91G4BtZrYk/vd9AD7X3QfSDahDSX4CUWRUFUnaXx9pe5IHwHEcp3fIkQ/VzHaR3EpyqpmtAXARgJe7+0y6AfXHAI5ncP4fAIMB7CU5DEBK9JTjOE5fY7lV+T8C4Kexwr8BwFXdFU4XKSUzSsUjt04r7ziO05fkcF5/HHZfn2n5nkyb+jKiNae6b9Ax7fxvWROYzF0eyBRVqYWaAdWpwkhriz6tgjL9y7VylRZd6oq0QNbZpp20bS9ntyDhlKG6naElSkJZoja/oP2HIfFpzLJXpP2+2vOl/exSLcK1U3t9vr5Nt/9r8/X1XHh+INPSTD3JHonU6z/MtEAWEmkmvln3y8Kz50v7kO2LpX3FKj3Zpa5CBxpsO6KDHkZVaDGseqQ+rwMNWiQbN0NnGNu7QR+3olqLdsWjtMg0/TwtAq/4b93+55frDGZThmvhUsuT2dELeaMzptfT9zmO45xMAsuInRQ8fZ/jOKcW+fqECk/f5zhOPyNvX/k9fZ/jOP2NvB1Qc0GiSjvPCyq1Mxw6sAIFg3Q9u/emOs+rK7Qjv0ObURx4R9jWro9Zvl1nAKpoCpxTADMtbpUGOkRoiZL7y3S0zuhA5FNIfHrngSek/enBehmLjk4tSp3XrrtVYvwoaW97aaW0FwnxCQASw1JFr+LEJl02IEqxptsIwhQKK3Q9pdQRUS81a6GwoVifU9MxXb4s0NfaAte+dPoAaT+4QveF1nYt9tYU6r5sjVp8KirQ1+HbBVrQ/Nah7JbDyYZTekB1HMc5qQQeVE4GPqA6jnNK0dnedwNqumxT9SQXkbyT5GiSj5A8RHIpyTndfO7VDC63vrgp5412HMcJYZ2Zb7kmXTz+jQD+E8ADiKZJ/cjMqhElCLgx9CEzu8nM6s2s/urZ43LVVsdxnLSYMeMt16R75S8ys4cAgOQ3zey+qMH2R5LfyuQAret05EaIvXt0REflRp1OrKMzdQmUtjbtaA8E9qCmSDvgD3Vo4WJgIGVbxeTs8sWs/aOOfApFIjc+sFHarzYdvRJKuxeKfAqJT+fuWyLttw65QNrnV+h73rFeR8c98PJoab+kSa/1XjUnNcrmmQK9BMeAwFNIw6169veCK56X9peW6eivA9SRfTai4xIAACAASURBVAsn6Lbv2qLv1ZJCLSa1HNNf0SOd2v7yfbr3PF2s+1pNR6APPqRTXZaX62i3fa1aVLtxiFaCy6q0uJUL8lmUaib5RgDVAIzkFWb2a5LnA9CynuM4Th9infkrSn0I0St/J6KIqX8m+RMA2wFc27tNcxzHyZ4+XEU67cT+FxENpMf5WLyB5FXw8FPHcfKMzva+S9XckyPL1H6O4zh9iVnmW67p9WxTBQMD6fhK9KFbAlE2oYQHe5EaKdUZiIjCei0INLTpKJJZ1VoIK63TjSkMrOkewh7V4tYkaPuxQ/palpZogWVWo74QobR7ocinkPh09V6dEveFEj2jLpTKsSyQEHjzJr1+0YhjqeJWdaB/rCvSdc8t0un1rFlfy0MB8WlHkb5m7S3aXlSopYddHbqd5QP0NWtr0n7CUMRSdUDxGBg4blO7Pt9hY7SwWHFQC7ghgah0SO8pR/nsQ/VsU47j9Ct6YzpUpni2KcdxTinydtqUZ5tyHKe/EXJdnQw8lt9xnFOKfPah9piCGp0Cr3WjjtaZOEULKcWTtaA0e9veFFvLMe1QHzpdCxEHl+hUYkcD9bTs1b+AtiS7NaVqa/Rxn2pIjf4CgJnna4Fi6T36Gk+fnHptgPCaT6G0e6HIp5D4dMY2HW3UUDZX2i+9eJe0F82fKu3oSI0Am3ODjkw6MyACjXm3jshLzNLrsZ277zFp3/iiFs462nQf2RFYU+qKUh2ZVDVe3/MRDVq4nHCpjvoreyyQcjKwptTA2br9hdPGS3v1D/ZI+7r9+vpMSpFlImqlNTvydh6q4zhOf6Mvn1BP2NlA8qFu9v0129TyzSd6CMdxnKzpNGa85Zp081D1O1o0beqM0OfM7CYANwFA0ycu68MHcMdxXm/05bQpWjcOB5IdAJ5ANIB2ZYGZ6VQ1SRy75VP6ACV6Mn3Hcy/ptlTpicOd+7QvVpEYrDPidO7S/pzCc+fpijq0X46zFmTcFgBo/Mx3pL1klP6d2/609rmO/tvAi0Z7oJ3F2S1R0rF+q7SHJuqzTK/pXnPjc9K+/++n6fZUaB/25j+ktn/Umdo/3tkcCMIYqvtfZ5P2NW76s84GNfla7dsP9cuCyWN1+S3aB5yo1X22Y+c+aUezbn9o+RzrDMwxCvTx1he1TlA0Wl+HggXaJ925dp20V3z+9h6Phi+Mzfwh7ozNv8np6JvOh7oKwD+Z2dquO0jqb5njOE4fks8T+7+EsJ/1I7ltiuM4Ts/pS5W/W1EqTihNkheR7DrfIxQx7ziO02f0pSiVbk2pjwK4H9HT6AqSlyft/lrOW+M4jtND8nkJlGsBzDOzJpLjANxHcpyZ3QAtVKXQtlgnrCoYqp3tG36pn9dHnqad9hufT504XByYzD1o+G5pD9H57GJpH3imFl0K23WmohC7N2hHfskOLSyEaF2rMwA99exIaV94/k5pb3tppbSHligJZYkKTdQPiU+D7lkt7Q/XnCvtk0enCjJPPTFMlt1RqJfDqWvXYszZ03Q/29yiJ+S3/1Av5TH5gkDGsJe06PrAan2N3zR2k7Tv3qb7zvo2vRRMC3WfOhp4pBrdpvtyKXTQSW2ZfmEd36q/Q9aS3XclG3rjyTNT0g2oCTNrAgAz20RyIaJBdSwyHFAdx3FOJh19OKCmm9i/m+Sr803jwfVtAAYDOK03G+Y4jnMi9OUrf7oB9f0AXvP+ZmbtZvZ+AOflvDWO4zg9pDOLLdekS98XzPZhZn/KfXMcx3F6hvWhN7LXk6MUnj5J2jlUr6AydsuTup4xOkPSeKRmuSkcrCNsEEiakCgLLNMydKC0s0oLFDz9bH3cACNmLJX2wsH6tjRv0Wuil87V13J+gxafimbqpVqKEvr6XNIUEGoCS5SEskR1rNwg7SHx6U0NT0v7Y/ybFFv9JC2E7doSWPbGdKTUmtVadJlQprNBjZ6vRanCMakZsQCg6HQdcXXe7u3SXjld94Xy0VqILFqqBdmh43X7D2zVEYiFAWE3xODZOmqucN6Z0m7Nui/ngoBWelLIekAlOdTMdK4ux3GcPqajR2uP9ox0yVG6pickgGdIzkGUB0AnynQcx+kj+nAFlLRD+T4AzyZtywCMBPBc/LfkNen7/vRyrtrqOI6TFgMz3nJNugH10wDWALjMzMab2XgA2+K/J4Q+ZGY3mVm9mdVffc6MXLbXcRynW3Kt8pMsIPk8yd+lK5tO5f82yXsAfCfOLvVFAFm5fFmnI1gwSNsLhmkRAZV6mY/iianlQ+njGIiaCXmxE3MCKV+LdRq9gvF6SZAQ1M1E4XgtwDX8RaeEqyzV7RlQrwUHBMSnxDB93Ko5+6V9xDEtjKglSgCddg/QkU+AFp8A4MIDqSuY7xmkxc8BB3QEz/Bx+lq2HtV9ZM8uLSYVDNLXHgW6Hg7XafSGnK0n1BTO1dFlnRt0+Zot+rzK5+rFRcoma3Ho2Fp93QoCp1s4Tt9zDNUCKA/qPpULeuGV/2OIMu8FBqe/ktZ7a2bbzOxdAB4H8AgAPbI5juPkAbl85Sc5CsBbAdycybHTqvwkpyHymz6GaECdGNsvNbPfZ3IQx3Gck0U7M/eNkrwOwHVJppviFUeO810AnwGgX1G6kFW2KQBvNLMV8W7PNuU4Tt5h2WxJek+8vTqYknwbgD1m9mymx+71bFOO4zgnkxz6UM8BcBnJtwAoBVBF8k4ze1/oA+nWlFppZjOT/l0J4D4ALwO40MyCC/UdZ/cF58sDFA/W4/GjT+uUcxWdOnJjT2Hqb0JHYKgPpSTbHxAQphfp6JKx9VqMKZmv1ywP8fT1WhA4lNDtGZfQKeHaOvWLxnZoBWGY6bXYixO6Kz5ToN3m1YGeO6dYCyOhqKJQ6r1Q9FPJoNQDD31Ir1H0zWEXSPuF0Pd2ypVaKVx8s77GTdT3amShvldNbToq7yC1vdz0Rd4l+j0AlAQE1lA9o4p1OwfX6Xu1f48WOpd2aL3mgqq90p4o0O2c8NIfevygdt/wf8hYOH/nzp9mdLw4096nzOxt3ZXzbFOO45xSZPPKn2vSvfK/H8BrHuvMrB3A+0n+qBfa4ziO0yN6I1LKzB5HNNOpWzzblOM4pxTZqPy5ptezTTmO45xM+jDZVO8PqGUTAlFLgZR5Y6iFmjGTG6R97brUCI1C6of+uiHa0V68VzvUy6r0OjyJ8kDEVSACKUQZtUhWZtpeEBCNykp0O+2obg8ZiAwL2AcE3qHWFenyZwZSv3U264pC6z6FUu+p6KeQ+PTZXYuk/Ve1Oj/6hB1aRNlaqIWznYFv0LQyLfyVtOhrs61FpxOcPUS3h7t05NOBhG7Q9qJAn23VguOhLfp7O3bYQWmfuEsLoMXlui9XjuzFNaX6cP6RP6E6jnNKkbfZpkg+R/ILJCdmU2lytqnbVgfdsI7jODknn1X+GgADASwiuQvA3QDuMTOdwj0mjja4CQAar7mkL10ajuO8zmjP41f+BjP7FIBPkXwDgPcAeI7kKgB3d4l51YQyGw3X2WlGjdVPtKXD9bg8tVD7mCD8gUV6RRMMHBvIrDNQ+51C2aA4NbtUhRPGP6jrL9V+tuZD2u9cO1/fxvUP6WUpJr5Z21mjJ2033Kr9XXOLjkj7mHfrJWI6dul66tr1S1pomRKVKerCJj1RP+QrffsBvdTOhpd0dqfJHbqPnD9YBzEMuahM2i1wrmWP6n5cOVTfq1kD9KIZq9bqJVxml2n9oLNYfz9Ly7VffsfuamkfO0Rfh8Hv1Vk+rUGXzwV5+8qfjJk9ZWb/gihRyjcBZLeA0skkIK44jnPqY8x8yzXpnlBfSWmsWQeA38eb4zhOXpG3T6hm9m6S00heFMfxvwrJS3u3aY7jONmT64z92ZBO5f8IktL3kbw8aben73McJ+/IZ5X/OvQwfV/RgtP1jmqtEFU3aud5wUy91nvhMy+m2EKCV3DifWgJlFF6+QbbowUBDh2t6w8w8BItbiEw0b38gM5yVTBSCxHjzt2kqz97ftq2JbPgiuel3Zq1yJSYVS/tbesel/azp+lJI2tW6/NSy5SEskSFJuqHxKcJy1dL+6Y5uv/Vvk/XgzYt6nCovudDjzwj7QUj6qS987AOgJk3WS9EXDhWX8u2VTqjV6JSDw0VG/T1rFigv3OcOlPb93Q7UahH5LPKn4gzTMHMNsUprO4jORaeD9VxnDwkb32o8PR9juP0M/L5ld/T9zmO06/I21h+T9/nOE5/oy9f+Xs9OcqRny+V9qIROupn1cM6w9DkealrsQPAhudSs+6UFuvMVDVDdWTPkUM6ImfI9M3SHso2VVz0mLSH2HinFuBKS7Wg8cr+Gmmvn7Ve2vds1As1Dtm+WNoLK/RL0EvLtJByKLBsx7n79HXYukJH2Wxu0ZFVE8p09NOeXanntTawREkoS1Qo8ikkPo17fo20P7Z5kLTPOksLl9aeMrUbALDmOS3qTJy2W9o3rdHHberQ2aNqAkudVFToLFGhzGYHG3UfTKzTAuWkPQ9Ie0ej7uNlV0lzVpzS6fscx3FOJu19OKR2O6CSLARwDYC3Azg+h2g7ormpt5iZ/plxHMfpI/ryCTWdyn8HgDMAfAnAW+LtywBmA7gz9KHk9H23b9uZo6Y6juOkpy8jpdK98s8zsyldbNsALCapnUF4bfq+fW/Sy0g7juP0Bnmr8gM4QPJdAH5hFi3sTTIB4F0AtPLThYT2kcOadYq6woAzvKBKC0FqTXrtZg+zrUELYcee06LL6Fk69VjHuq1ZHbesXF+DxkZ9BvNm6Kf9xp26fE2dFiJWrNLRN6XU7TkQEJ92FOkXnI0vauFi+oe0+NT+Qy3OjZ4fiJoblHq+j/1Sd7TQEiWhtHuhyKeQ+HThAS2Wbt3f9TkkYsM6Xc+WhBZGx7fpa7y/Q0eGDSvWEVRLTQuUUxv0Ui21ZVq0m3yBFgpffEQvybLt8UDKxmk6LWEu6OzDl/50r/zvBvBOALtIvhI/le4C8I54n+M4Tl7RkcWWa9LNQ91E8noA3wawHsA0RHlQXzazjb3QHsdxnB7Rl0+o6VT+LwJ4c1zuEQDzATwO4HMk55jZV3u9hY7jOFmQz/NQ34lI5S9B9Ko/yswaSX4LwBIAPqA6jpNX5HOkVHucof8oyfVm1ggAZnaMZEbtLp6o0/RZk3aGDxmqhYjWXdrjUVmS6twur9AO784OLf8VBG7BqBlauEiUB9bJGqrFmBAlFVpk4mEtUIRSpH35bi3Y/cdoHWVTV6Ejxl5q1pFMCyfoVGvtLdoF3xEQUjr36Xs4+QItnhWOCaRhLEg935GFuo5pZYF+FljzKZR2LxT5FBKfRj+jJ8H8ZYhOnXjOcJ1Gb/dWLSaNqdDfkzGX6PaP36f7MgNrShVN16kr29fr6zy4Qts3NmnBt3K3Fr20ZJcd+SxKtZI8Lp/OO24kWY2+/SFwHMeR5HO2qfPMrAUAjk+biikC8I+90B7HcZwe0ZGvotTxwVTY9wHY1ystchzH6QH57EN1HMfpV+TttKlccGSZdoY3N+pD79ylndgTB+2X9o3NqU77kmP6N2rKUL3ezvYCLQJtWq7XiKovOSjtA9ZrEShE81F93OXtgRSGRTrq5IIWLUod2q4jqLYd0RFLDQGBYtcW3Z6iQi0y7QjUf+5kLTIde+klXf/pWpDh8OEptqY2LQKVtOg2WrvuI6E1n0Jp90KRTyHx6ey9eu2oxzr+RtpDvFik+877j2qhc91iLZgOqtUCZd0kLW4lanRE2rKAoNkRGGF27tDlx+jiWZHP06Ycx3H6FXmr8pMsJ/kZkp8mWUryAyR/Q/I/SerHELw229QdO3pvdUPHcZyudMAy3nJNumlTPwFQB2A8gAcA1AP4f4hWPP1B6ENmdpOZ1ZtZ/ZUj9Hw2x3Gc3iCf0/dNMbO/I0kAOwFcbGZG8mkAL2ZygJKReswuqtY+mj279a9G8Sid8ahtRarfbzC0r7G1RZ/ueNMZepo69DGLq3T9VeO1PcSBpRXSPrhd19PykvaPjaH2463dq/1mowKTwpuOab/WkkLty9zVoe/VFaU6I1HnFv228sBq7as+b/d2aR9ydupSZwep/bPbWrSvsexRvb780CPaxxlaoiSUJSo0UT/kKw1lrVoxbra0j2nUfXPrn3TAwp0Bn2vHYZ216i03aX/6jOF6Qv4U0+0JZY+rG6H7SC6wfBel4kH0QTOzpH97nlPHcfKOfJ42tYxkpZk1mdnVx40kJwLovZ8Yx3GcE6TT8lSUMrMPAphB8kwAIDmD5CcQpfF7w0lon+M4TlbkKvSU5GiSi0i+THIlyY+lO3bG6ftIPgLgLACLAHwWURYqzzblOE5e0ZG7l/52AJ80s+dIDgDwLMlHzOzl0Ad6PX3fjsXaSV43TQsjuzr0ZPSSRVqo2VmY6jwvatOO9vJmLYTtNe2wn1ajV3mpHaNFrKIpgexIAUY1BqJ39RLwKBqrJ9iXPa/Pa3ClnrRdPVILC2XbdT0tx3Q3KR+gs3pVjdf2RO0EaX/T2E3SXjldH7dwbuoyJeW/0SLT7CHaXjlUt7FghF4eZuI0HbQRWqIklCUqREh8mrVJa793DF4o7QOH6KxPnyvSfSFRpAefyou0ULjhFv393Aj9PT9nmBbnWo703hT4XA2nZrYTkRgPMztMchWAkQBOeEDtcfo+x3Gck0lvTOwnOQ7AHEQPkkE8fZ/jOKcUlsV/yUFI8XZd1/riIKZfAPj48YfKEJ6+z3GcU4psnvSSl7xXkCxCNJj+1Mx+ma4+T9/nOM4pheVo2lQc0HQLgFVmdn0mn+n15CgNR7SzOrFWn/SaYp05aVRnwPlfoH6PdNnqFi1WbSnW5Qc0aGGhrVW3cURpdr8xDdv0tTnQoh3/R1/QwsjRVr0mekejPq8DDTpjUFvgGh/p1N2krUlH04xo0MLIxLH6+uzepsW28tH67apzQ2qk1K5CHXXGXfrazBqglzTpPKwFx01rdDTa/g7dp0JLlISyRIUin0Li05X7Hpf250vmSvvOo4GovGJ9vmOKt0p7VU0gQmu/jrJrPKD7eHNr7w097bnzoZ4D4EoAL5F8Ibb9m5k9GPqAZ5tyHOeUIlehp2b2NKK8JRmTLtvUh8koSJrkJJJPkjxIcgnJ03rQVsdxnF6hE5bxlmvSqfz/HPtLAeAGAN8xs4GIJvb/MPShZOXs/qMbctRUx3Gc9JhZxluuSTegJrsEhprZr+IGPw4gOHM5OX3f5eV6MrfjOE5vkM/p++4j+RMA/wHgVyQ/DuBXAC4EsCWTA8x4gxYiiqfppSbecpeOTqoZroWOaw6n2q1Tuz0qh+o12qcG1pcv08EiKBynhY6Ciy/THwhQ8umf6/aM09E9q1YNkfa5l+llZjb8QYtb42bopWBKp+vfyJfv07/kRQV6eZEJl+rrjGZ9X9a3acGnaKmuv2ZL6vmWdGqh7UBCd/FVa/W1nDdZX5umDl3/sJCoc4mOOgstURJKuxeKfAqJT3O2Pyftz9TVS3txkb7Gu9bqvpAIJJk7v023c/y1A6XdDvZebqUchp5mTbppU58n+QEAdwOYiCgE9ToAvwbwD73eOsdxnCzpjVf5TMlE5X8ZwIfNbCnJmQAuRTQvSz8WOY7j9CF5u+qpyDY1H8DjAD5Hco6ZebYpx3HyinzO2N/jbFOO4zgnk75MMN3r2aYSpTqqqHW1jlRhQjv/K8/QTvuNv0i111RqoaC6XIsl+zbpY74YcMzPGqfbXrnpbmkPcaxJn9PeRt2e2edqsWr/Mzp65UggMmzvBr1g7cEVOorn6WLdzmqtZ6DsMS3IjP2gTo3XQl1+6HgtXJTPTRUFyzfq7ri9SPe/2WU6kqlwrBaraoq16LLUdB8Zv097xNYt1ut8hdZ8CqXdC0U+hcSn+buXSfuHRpwr7QtadZ9600wdQdWyRg8lO+/SfXbgWJ1CMriUchb05dpM6QbUVpLlZnYUnm3KcZx+QHu+qvzwbFOO4/Qz8lbl92xTjuP0N/JW5Xccx+lv5LPK32MKhgTSqh3RAlHVYO2sTgwdLu0Dyw9mXEdBpY6IqqjWbak5poUCBgJ2E5XakR+iska3szWQHrCgVotMhcVaHSop1Otwhc63tV0ft6YjEMXToTtuqH7r1L6to4HreWCr7jtlk1NFx1HFAb9Zqxb4Oot11FbbKr0GUkWFjjqb2qDPlYH6B9VqkanjsL63oTWfQmn3QpFPIfHphzuelvZpdRdIe0eLPq+aWi3aFZfrPpjQlzMn5O0rv+M4Tn+jL1/506XvS5C8muQDJF8k+RzJn5FcmOZzr2abunX55pw22HEcpzs6rDPjLdeke0K9BcBmAF9HNMm/EcBTAL5A8jQz+y/1oeR1Wpo+cVlfTgtzHOd1Rj77UOeZ2VXx30+TXGxm/07ySQAvAJADajKJaROlvaBI+xsHtD+v6wmslz587vYUW2Gd9pslqrVPbmCxzjA0aKH2/7B2hK7/9DOkPURl0wPSXjVL+1Y7DuoJ8DVv0EuIlK3UmbuKR2l/XU1hwPf5kJ5g39Su7+HA2YEXnw7t3xvdpq9zYaEuf2xt6vUZXKevzaEt+lxLy3X5RKX+ShQk9NNMbZm+V0XTdR+pm6SP+5abAhnSLtIpz0JLlISyRIUm6od8pR/fvUjar7xwmrRXlGn/e2KU/t4y8P3PBX0ZKZUuH2obyYkAQHIugFbg1elU/uTpOE7ekc0y0rkm3RPqpwEsItkSl303AJAcAuB3OW+N4zhOD8nbWH4ze4zk3yOK6V9KcgbJTwBYbWafOTlNdBzHyZzeEJsyxdP3OY5zSpHPolSP0/cd+61ekqH0LO1s37JIT6avefkFad+xJTV7T0Vpqyw7oFaLTxUjAuJTqRZpCip1eZRkN1u5aYMWIorKdftbDuvbVZXQotGmlTqz0fTztIhljToDU3m5nkQ+bEyjtBdOGy/tLc/oBRtLoTM8hSgQl3n/Oi04jh2WGvgBADt263XkKzbo7EgHG/W1nHyBvvbt6/VE90SNFkxnDNfi1oZbdJ+qqtGiTmiJklCWqNBE/ZD4NOju1dK+fLQWZMf9s25/5y59nXNB3r7yIwfp+xzHcU4m+fyE6un7HMfpV1i++lDh6fscx+ln5G22KU/f5zhOfyNvVf5csGbZYGmvXaMFosfb9Jr35+3UzvMlCSFGtGqBYvZW7fjfvF07zus6dFTL5OH7pX3Qnuym5u7dqaNa1rXphSDOnbhD2lc+ode1bzEdt7Hiv7X4VFSgI5P2tQYEnIP6Olf/QC8RM+4KLYaFoo0Gz9biXOG41D619EWdEWviLn1vxw7RS5RULND9NbFOC5EvPqL76+AKLUota9bXcoppkWkj9Hlt3a/rOb9NHze0REkoS1Qo8ikkPp2+VYvGL9wYiACr0oJvTpZAyWNRynEcp1+Rt6GnJAtI/hPJ/0vynC77vtC7TXMcx8mevgw9TRfL/yMA5wPYD+B7JK9P2veO0IeS0/fdf3RjDprpOI6TGWaW8ZZr0g2o883svWb2XQBnAagk+UuSJQC0cwRR+j4zqzez+svL9SRvx3Gc3qATlvGWa9L5UF/NfWZm7QCui8NRH0OG/uPZV+sxu22Djr55185t0l65UC+BUnhHaoq60BIcVRO1sGBP6UgdBqJO2lv1OTUs0yJWiFFT9TXYt1wLKQOvPUvaH/ziFmn/+OTU1IYA8PxyfS2/XaCFmhuHaNEoJKau26+jiiYumCHt41sXS3vhvDP1AYampsa74P6HZNHQEhyD3ztB2jl1prRP2qNTLW57XNe/sUkLcB2Bb1xhID3gOcP0kiyNB7RYNf7agdK+8y4dmRRcoiSQdi8U+RQSn87YptNxrpuh+0Iu6AgstXMySPeEuozkpckGM/sygNsAjOutRjmO45woffnKn24e6vu62kjebmbvB3BzzlvjOI7TQ/J2Yj/J33Q1AbiA5EAAMLPLeqthjuM4J0I+z0MdDWAloqdRQzSg1gP4di+3y3Ec54TI52xT8wB8DMDnAXzazF4geczMnsj0AM0v6KiZ5gM6EqOjLSD43KUFEzXZ4GhgjfOCLdpZPaJWp6FjwMNcVKod+aXVgbR+AToDxQeXaBGoc7NOwfb2dh3tUjRYn8CUQKTXtw5pwaGsSkdWlQ7R13MS9FpWnWvXSbu1BMTCZi3a8WBq+xMF+ktUOTJQd4PuT9yjo9E6GrXgWDdNR3NV7tb3cOcOHeFUN0KnAWw5or+iza3abgd1PQPH6vYkAhknQ2s+hdLuhSKfQuLTpJdflvbsvkGavA09jROifIfkvfH/d6f7jOM4Tl/Sl6/86VR+AICZbTOzdwF4CMCdvdskx3GcEyeXkVIkLyW5huQ6kp9LVz6rp00zewCAnpDnOI6TB+TqCZVkAYD/BnAJgG0AlpL8jZlpfwUyfEJ1HMfpL+RwHup8AOvMbIOZtQL4GYDLc3bwnm4ArvPyfVM+n9ri5f3e9qR8LjcA1wFYlrRdl7TvnQBuTvr3lQC+3219J7nxy7x835TPp7Z4eb+3PSl/srYTGVD9ld9xHEezHdFc/OOMim1BfEB1HMfRLAUwmeR4ksUA3g2ga/ToazjZc0pv8vJ9Vj6f2uLlc1s+n9pyMsqfFMysneSHATwMoADArWa2srvPMPYNOI7jOD3EX/kdx3FyhA+ojuM4OcIHVMdxnBzRqwMqyWkkP0vye/H2WZLT05S/iGRlF/uloc8klbk9zf6zSFbFf5eR/DLJ35L8JsnqLmWLSb6f5MXxv99L8vsk/zdJnYbH6VVIDs2y/KDeaovjhOi1AZXkZxGFahHAM/FGAHerJAMkPwrgfgAfAbCCZHKI19e6lP1Nl+23AN5x/N+BJt0K4HieuxsAVAP4Zmy7rUvZ2wC8FcDHSN4B4F0AlgA4E3m0UkFfDTIkq0l+g+RqkgdI7ie5KralLGpEsork10neQfK9XfbdKMrXdtkGS0ZIwgAABiFJREFUAXiGZA3JWlH+GyQHx3/Xk9wAYAnJzSTPF+XrSS4ieSfJ0SQfIXmI5FKSc7qULYyXUv89yeXx9hDJD6kf11wsvU7ylW72fTjpXCeRfJLkQZJLSJ4myk8geSvJr5CsJPljkitI3ktynCifV+fb7+jFKINXABQJezGAtcL+EoDK+O9xiMLAPhb/+/kuZZ9DlPVqIaJlrhcC2Bn/fX6gPauSP99l3wtd/r08/n8hgN0ACuJ/8/g+UX81gG8AWA3gAKKlt1fFtoGifBWArwO4A8B7u+y7UZSv7bINArAJQA2AWlH+GwAGx3/XA9gAYB2AzV2vUbx/UXxNRwN4BMAhRPPw5oi6HwbwWQDDkmzDYtsfRPlfxO25AtE8vl8AKFH3IrZ1AtjYZWuL/79B9Z2kvxcBODP+ewpEFA6iH/c3A3gPgK0A3hnbLwLwly5l7wbwAwALEE3sHhX//QMA94i6bwZwF4CPA3gWwPWhfhfbDgNojLfD8dZx3C7Kr0z6+wEAb4//XgjgT6L8kwD+GcDnAKwA8Mn4Hl8D4DFRPq/Ot79tvVdxNLCMFfaxANZ011Hif1cC+D2A65E64CUA/CuiL/4ZsS3li9blM/cCuCr++zYA9fHfUwAs7VJ2BaKBvya+0bWxvRRJA3OXz/TbQQZZDDCxPeX+dbdP3L/PA/gToh8Fda6fjO/9aUm2jd0ccxWAwvjvxaHrkGR7PunvLaF98b9f6ea4KfuQ9IOL6Af5JgC/BFDSte64zPcA3A6gLsNzXZP0d9d+m/Jjn8255uP59ret9yoGLkX0RPRQfJFvir8k6wBcKso/hnhw7HKDbgfQETjGKEQD5fe7dhZRthrATwCsR/T63oboqe0JALO7lP3XeN9mAB8F8EcAP0b0FP3FQP39dpA5gS/dHwB8psuXog7Rj8ejgbYkutg+gGh5nc1p7u31AAagmx9MRG6iPwC4EMCXELl0zgfwZQB3iPJ/AfBGRK6czQCuiO3nI/XHZnFcLpFkSwD4ewBLRN2rhe2L8b1NeTOL98+L+/9H47q7O9evxv14AoB/Q/RkOBbAVQB+J8o/i+hHdD6Affjrg8Qk6AE4r863v229W3l0sRYA+F/xtgDx67MoOwpJT3dd9p2T5jhvBfC1DNtUBWB2fFPruik3AsCI+O+BiBIlzO+mfL8dZLIZYGJ7DSL/82oADYhcHKtim3I//CeAi4X90tCXLqnMZfGXfFeacgsB3APgeUQ/fA8iyiSk3E6zEb1RPARgWnxtDsbX/m+6lB0X17sHkRvrlfjvewCMF3XfCf3A8EEAbWm+Kx8F8BSAHWnO9QOIHgr2IXqDehmRzlAtyl4EYE18f85F9Ca0Nj6Hy0X54+e7Nz7X42X77Hz709bnDThVti6DzIEug0yNKN+Xg0xhl3IZDzBJn5kG4GLEfu/k9ndT/iJR/s3pygMoAzDrBOsPlZ+eaXkAZyF6whsE4BwAnwLwlm6u+3z81cUyA8Ansij/BgD/nkX5mYjeXrorf1aX8t22P+lzg+Ltziy/C7dnUXY4gP3Z1J/Pm4eengRIXmVmt+WyPMkyABPNbEVv1N9d2XhGxv9G9INxBiLx8P5433NmNrdL+Y8A+HAW5bOt/0TK/wuiH79uy5P8IiL/ciEin/18AI8jyuL+sJl9tUvdXcufhciHnWn5bOvPdXk1S+ZCRK/osC5Lx4vyBHBBFuW7rb/f0dcj+uthQxr/bj6XV2WRxYyM/l4+LlsAoByRMl0V28ugfZD9vXxWM2gQvf1kUz7rGTr9afMVTHMEyeWhXYh8qXlbPtu6Efl+mwDAzDaRXAjgPpJjodb17t/l282sA8BRkuvNrDH+3DGSar3i/l6+HtktHZ/tUvPZ1t+v8AE1d9QBeBOQsig9Afw5z8tnW/dukmeY2QsAYGZNJN+GKHgiZXJ5Py/fSrLczI4iGjyiCxNF16kBqV+XtyyXju/t8v2Ovn5EPlU2ALcAODew7658Ln8CdWc1I6M/l0c8N1iUG4ykKWynSnlRLuMZNCejfL5vLko5juPkCM825TiOkyN8QHUcx8kRPqA6juPkCB9QHcdxcoQPqI7jODni/wOphV6T6EAb/AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TMv7U8cWeF1v",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e7712dde-e5b1-4f9a-d871-9c89388472c0"
      },
      "source": [
        "pca = PCA(n_components=2)\n",
        "pca.fit(sentence_features)\n",
        "print(np.sum(pca.explained_variance_ratio_))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.38147822\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3dJqnERseEIj"
      },
      "source": [
        "pca_sentence_features = pca.transform(sentence_features)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5NiljyFJfRo0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c4a26fed-0a93-4a70-dc49-e247575312e2"
      },
      "source": [
        "plt.figure(figsize=(10,10))\n",
        "for i in range(len(pca_sentence_features)):\n",
        "  plt.scatter(pca_sentence_features[i,0],pca_sentence_features[i,1])\n",
        "  plt.annotate('sentence '+ str(i),(pca_sentence_features[i,0],pca_sentence_features[i,1]))\n",
        "plt.title('2D PCA projection of embedded sentences from BERT')\n",
        "plt.xlabel(\"Principal Component 1\")\n",
        "plt.ylabel(\"Principal Component 2\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Principal Component 2')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoQAAAJcCAYAAABg2rUNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde3hU1b3/8fcikIRLxEaDBgMSFCIEkpDECPVACAGDRFBSWmtPS4WKbVUg3kCEUKHlHM750aoHjlWqgKW0RKEKAiJg9RAVwUBSyk0rkATkDkkgkZDb+v0xk+kkJCHcMgnzeT3PPMzsvdfa371mM/OdtddeMdZaRERERMR7tfB0ACIiIiLiWUoIRURERLycEkIRERERL6eEUERERMTLKSEUERER8XJKCEVERES8nBJCkSbGGLPTGDOwudR7gX0aY8xCY0y+MWbLVd7XQGPMwStYnzXG3F7HuoeNMZ9cYr2XXNZbGGPCjDHZxpgzxpgJno5HxBsoIRSPMsb4GWPeMMbkOj/8s40x97qtH2iMqTTGFDkfB40xbxlj7qynzi7OL/OqMjnGmOfc1htjzARjzA5jTLGzzreNMb1r1POCs567rs7R185aG26t/fhy6jDGLDLG/OZK13sJ/g0YAoRYa+Maed/iZIz52BjziKfjuAiTgI+stQHW2v9p7J07/98XOz8/Thhj/mKMud5t/cfGmBK3z5giY8x7znXun1lnjDFfGmPGONe5b19pjDnr9vrfG/s4RdwpIRRPawkcAOKB9sA04C1jTBe3bQ5Za9sBAUBfYA+QYYxJvEDd1zvLPQRMN8YMdS5/GZgITAACge7Au0ByVUFjjAFGA6ec/14RxpiWV6quZuJWIMdaW+zpQKRZuRXYWddKY4xPI8QQ6fz86Ap8B3ihxvonrLXt3B7D3dZVfWZdBzwJ/MEYE+a+PZAHDHdbtqQRjkmkTkoIxaOstcXW2hestTnW2kpr7SpgPxBTy7bWWnvQWjsdeB34rwbuYxOOL5dexphuwOPAQ9bav1lrz1lrv7XWLrHWznYr1h8IxpE0/tAY41tX/c6exGXGmHRnj8A2Y0yk2/ocY8xkY8x2oNgY09IYM8J5CbfA2dvQo8b2g53PWxhjnjPG7DXGnHT2jga6bftvxpjPnPUccF6OfBT4d2BSjZ4L93r9jDEvGWMOOR8vGWP8nOsGOntNnzbGHDPGHK7q4ajj+DsaY1YaY04ZY742xoxzLv+Z833q54xjRh3lxxpjdhvHZeUPjDG3uq2zxpjHjDH/dLbtr40xtzmP+bSzPXxr1Pe8s1cnx73XxXnMc4wxecaYo8aYV40xrd3WP+s81kPGmLE16rzBeYynjePS92011t9hjFnvbIMvjTE/aGjZGvX4G2P+5HyvC4wxXxhjbnKua28cvemHjTHfGGN+U5UYOd/3T5zHl2+M2W+cPe3GmFk4zud5zvdhXgNiXmSM+V9jzGpnu282xtzmtj7crexRY8zzzuV1nq/1HVuNNvgbkOAWb3dnPL83xqwxxhQDCcaYHsbxf6fAOP4vjagR/yvGmPeddXxqjLnZeZ7nG2P2GGP61PU+uLPWngZWAj0bsn2NstZauwbHD8uIiy0v0qistXro0WQewE1ACXCH8/VA4GAt2w0CKoG2tazrAlgcvY8GuBv4FkgEfgHkNiCON4C3gFbASeB79Wz7AlAGjHJu/wyOpLaVc30OkA10Alrj6JEsxnEptRWOy2NfA75u2w92Pp8IfA6EAH7Aa8BfnOtuBc7g6AFtBdwARDnXLQJ+UyNO93pnOuvtAAQBnwG/dmvzcuc2rYBhzvb7Th3HvxF4BfAHooDjwCDnuoeBT+ppu/udx97D+X5NAz5zW2+BFTh6WsKBc8CHOHpt2gO7gJ/WiPt3zraKd7ZzmHP9izi+2ANx9Da/B/ync91Q4CjQC2gL/Nm579ud65c6z4e2zm2+qTou57IDwBjnMfQBTgA9L1S2lvb4uTOuNoAPjh9G1znXveN8/9s637ctwM/d2rkMGOcs90vgEGCc6z8GHnHbz4ViXoTjvI9zrl8CLHWuCwAOA0873/MA4K4GnK91Hlst7VAz3kVAIY7/yy2c+/waeB7wxfF5cMbtvV7kPJ4YZ4x/w/F/crRz37/BcUm6rvPS/b3/DrAOmFlXfDXKDsT5meWMdQSOz6o+df1/1EOPpvDweAB66FH1wJF8bABec1vm+nCtse0dzg/tW2pZ18W5rgDIB3YDE5zrpgKfXyCONsBp4AHn69eAFfVs/4J7nc4vgcNAf+frHGCs2/o04K0a238DDHTbvipx2w0kum0bjOOLvyUwBXinjpgWUX9CuBcY5rYuCcel3ao2Pwu0dFt/DOhby346ARVAgNuy/wQWOZ8/TP0J4fvAz2q0xbfArc7XFrjbbf1WYLLb698CL7nFXY7bjwQciVgajh8GxcBtbuv6AfudzxcAs93WdXfu+3YcCUQZzh8pzvX/wb8SwgeBjBrH9RrwqwuVraU9xuJIziNqLL8JRzLc2m3ZQziTGmc7f13jHLbAzc7XH1M9waozZrfz53W3dcOAPW77zaoj/vrO11qPrY56asa7CPij2+v+wBGghduyvwAvuG3/B7d144Hdbq97AwX17N/i+AwowHF+78Hts8YZ37fO9VUP9x9Ulc5l55zlU2vZRw5KCPVoQg9vG88kTZQxpgWwGCgFnmhAkVv4V9JXlxutteU1lp3E8SVVn5E4Eos1ztdLgA3GmCBr7fE6yhyoemKtrTSOu1071rbeuTy3xvYHcBxTTbcC7xhjKt2WVeBIEDrhSOwuRbUYnM/d4z1Zo+2+BdrVUc8pa+2ZGnXFNjCOW4GXjTG/dVtmcLRFVXxH3dadreX1zW6v82318YpVxxWEI0naaoxx30/VWLSOOJJN93JVgvjXWNfa1t8K3GWMcT8XW+I4ny9UtqbFON7XpcZxE8OfcPyIuRXHD6bDbvG3qFHvkaon1tpvndvV9p5dKObz6qP6+1/feVff+VrrsVlry+qoq6aa/4cOWGvd95NL9f9DFzpv6mqbKtHW2q+NMa2Ax3CMW+5prS1xrp9grX29jrKHrLUhxjEMYzaOHsyXLrA/EY/SGELxOOP45noDx5fG9xr4BTES2GYv/maFD4EQY0x9CctPcXxZ5BljjgBv4/gy/lE9ZTpVPXEmtyE4LtlVsW7PD+H44qza3jjLf1NLvQeAe62117s9/K213zjX1TUezdaxvNYYgM414m2oQ0CgMSagRl21HUttDuC47Ol+fK2ttZ9dQiwA3zHGtK0RyyEclw/PAuFu+2lvHYP7wdGj26lGuSrHcfxAqGv9AeD/ahxDO2vtLxtQthprbZm1doa1tifwXeA+HJc5D+DobbrRbR/XWWvDL9QgVVXXeF1fzBdyAMcl+7rW1Xq+1nNsDVXz/1An5/+1Khdz3jV8p47Po9eBUByX/C+m7DlgMtDbGPPAlY5N5EpSQihNwe9xjCEbbq09W9dGxuEWY8yvgEdwjB+6KNbaf+IY7/YX47h5wtc52P2HzsHwt+AYa3gfjvFwUUAkjhtY6vvyijHGpBjHXcSpOL68P69j27eAZGNMorP34Wnn9rUlQa8Cs4zzRgtjTJAx5n7nuiXAYGPMD4zjRpUbjDFRznVHqftLGxyX16Y567sRmI6jx+aiWGsPOOP+T2c7RgA/u4i6XgWmGGPCwXXjxPcvNo4aZjjf1/443se3nT1JfwBeNMZ0cO7rFmNMkrPMW8DDxpiexpg2OC73Vh1jBfBX4AVjTBtjTE8cPxqqrAK6G2N+Yoxp5XzcaYzp0YCy1RhjEowxvY3jZpHTOC63VlprD+MYx/ZbY8x1zps3bjPGxDewTWqeD3XG3IC6VgHBxphU47hRJ8D8a2qmOs/Xuo6tgfHXtBlHr+UkZ+wDgeE4xmteUc54x+D4QbHvYstba0txDG2YfoVDE7milBCKRzm/OH6OI/E6Ymqfk6ujMaYIKAK+wDH+Z6C1dt0l7nYCMA/4XxyXnPfi6HF8D/gJkG2tXWetPVL1AP4HiDDG1NVDsALHuKx8Zx0pdfV0Wmu/BH4MzMXRczUcRzJcWsvmL+O4EWKdMeYMjiTzLmc9eTjGdj2N4y7GbBzJKzh6XHs678B8t5Z6fwNkAtuBfwDbnMsuxUM4xm0ewnHjw6+stRsaUtBa+w6OZHupMeY0sAO4t/5S9TqC4z04hCNh/oW1do9z3WQcNyJ87tzXBiDMGcf7OC7p/c25zd9q1PsEjl7jIzjGpy10O4YzwD3AD537PeI8Jr8Lla3FzcAyHAnTbuD/+Ndl3NE4bqDY5TzGZVx4+EOVl4FRzjts/6cBMdfJWXYIjvP2CPBPHHcFV+2n1vP1Asd2UZz/V4bjOFdO4PiRN9rtvb4S/u783MnHkcSPtNaecls/z1SfV3Br7dUAjjGqnY0xw+vZRsSjqu5AE5FLZIx5AccdiT++QvXlAT+21m68EvWJiIhciHoIRZoQY0wQjhsRcjwcioiIeBElhCJNhHH8Ob5/AnOdl4NFREQahS4Zi4iIiHg59RCKiIiIeLlmNTH1jTfeaLt06eLpMEREREQuaOvWrSestUGejqMhmlVC2KVLFzIzMz0dhoiIiMgFGWPq+8tETYouGYuIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKiIiIeDklhCIiIiJeTgmhiIiIiJdTQigiIiLi5ZQQioiIiHg5JYQiIiIiXk4JoYiIiIiXU0IoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKyFWVnZ3NmjVrPLLv3NxcoqOjiYqKIjw8nFdffdUjcYiINHUtPR2AiFzbsrOzyczMZNiwYY2+7+DgYDZt2oSfnx9FRUX06tWLESNG0LFjx0aPRUSkKVMPoYjUqri4mOTkZCIjI+nVqxfp6ekAbN26lfj4eGJiYkhKSuLw4cMADBw4kMmTJxMXF0f37t3JyMigtLSU6dOnk56eTlRUFOnp6RQXFzN27Fji4uLo06cPK1asAGDRokWkpKQwdOhQunXrxqRJk1yxrF27lujoaCIjI0lMTHTFV1s97nx9ffHz8wPg3LlzVFZWXtU2ExFptqy1zeYRExNjRaRxLFu2zD7yyCOu1wUFBba0tNT269fPHjt2zFpr7dKlS+2YMWOstdbGx8fbp556ylpr7erVq21iYqK11tqFCxfaxx9/3FXPlClT7OLFi6211ubn59tu3brZoqIiu3DhQhsaGmoLCgrs2bNnbefOnW1eXp49duyYDQkJsfv27bPWWnvy5Ml666kpLy/P9u7d27Zu3drOmzfviraRiEh9gEzbBPKnhjx0yVhEqil87z2OvfgSbXJzeP/QIVJPneJ7qan079+fHTt2sGPHDoYMGQJARUUFwcHBrrIpKSkAxMTEkJOTU2v969atY+XKlcyZMweAkpIS8vLyAEhMTKR9+/YA9OzZk9zcXPLz8xkwYAChoaEABAYG1ltPjx49qu2vU6dObN++nUOHDvHAAw8watQobrrppivRVCIi1wwlhCLiUvjeexxOm44tKaFLK1/eDulERmYmz/3ylyT94AeMHDmS8PBwNm3aVGv5qsuzPj4+lJeX17qNtZbly5cTFhZWbfnmzZtd5S9UR3311KVjx4706tWLjIwMRo0a1aAyIiLeQmMIRcTl2IsvYUtKHM/Ly2htDMNbt+GnLXzYtm0bYWFhHD9+3JUQlpWVsXPnznrrDAgI4MyZM67XSUlJzJ07F8fVFMjKyqq3fN++fdm4cSP79+8H4NSpUw2u5+DBg5w9exaA/Px8PvnkkwYnkCIi3kQJoYi4lDtvEAH46tw5HszNZWTOfub+8yumTZuGr68vy5YtY/LkyURGRhIVFcVnn31Wb50JCQns2rXLdVNJWloaZWVlREREEB4eTlpaWr3lg4KCmD9/PikpKURGRvLggw8CNKie3bt3c9dddxEZGUl8fDzPPPMMvXv3voSWERG5tpmqX9fNQWxsrM3MzPR0GCLXrH8OSqT80KHzlrfs2JFuf/vQAxGJiDRfxpit1tpYT8fREOohFBGXDk+mYvz9qy0z/v50eDLVQxGJiEhj0E0lIuLSfvhwwDGWsPzwYVoGB9PhyVTXchERuTYpIRSRatoPH64EUETEy+iSsYiIiIiXU0IoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKiIiIeDklhCIiIiJeTgmhiIiIiJdTQigiIiLi5ZQQioiIiHg5JYQiIiIiXk4JoYiIiIiXU0IoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKiIiIeDklhCIiIiJeTgmhiIiIiJdTQigiIiLi5ZQQioiIiHg5JYQiIiIiXk4JoYiIiIiXU0IoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl/NYQmiM8TfGbDHG/N0Ys9MYM8NTsYiIiIh4s5Ye3Pc5YJC1tsgY0wr4xBjzvrX2cw/GJCIiIuJ1PJYQWmstUOR82cr5sJ6KR0RERMRbeXQMoTHGxxiTDRwD1ltrN9eyzaPGmExjTObx48cbP0gRERGRa5xHE0JrbYW1NgoIAeKMMb1q2Wa+tTbWWhsbFBTU+EGKiIiIXOOaxF3G1toC4CNgqKdjEREREfE2nrzLOMgYc73zeWtgCLDHU/GIiIiIeCtP3mUcDLxpjPHBkZi+Za1d5cF4RERERLySJ+8y3g708dT+RURERMShSYwhFBERERHPUUIoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKiIiIeDklhCIiIiJeTgmhiIiIiJdTQigiIiLi5ZQQioiIiHg5JYQiIiIiXk4JoYiIiIiXU0IoIiIi4uWUEIqIiIh4OSWEIiIiIl5OCaGIiIiIl1NCKCIiIuLllBCKiIiIeDklhCLSbGRnZ7NmzRqP7T8vL4977rmHHj160LNnT3JycjwWi4jIlaSEUESaDU8nhKNHj+bZZ59l9+7dbNmyhQ4dOngsFhGRK0kJoYhcdcXFxSQnJxMZGUmvXr1IT08HYOvWrcTHxxMTE0NSUhKHDx8GYODAgUyePJm4uDi6d+9ORkYGpaWlTJ8+nfT0dKKiokhPT6e4uJixY8cSFxdHnz59WLFiBQCLFi0iJSWFoUOH0q1bNyZNmuSKZe3atURHRxMZGUliYqIrvtrqcbdr1y7Ky8sZMmQIAO3ataNNmzZXtd1ERBpLS08HICLXvrVr19KxY0dWr14NQGFhIWVlZYwfP54VK1YQFBREeno6U6dOZcGCBQCUl5ezZcsW1qxZw4wZM9iwYQMzZ84kMzOTefPmAfD8888zaNAgFixYQEFBAXFxcQwePBhw9CZmZWXh5+dHWFgY48ePx9/fn3HjxrFx40ZCQ0M5deoUALNmzaq1nrZt27qO4auvvuL6668nJSWF/fv3M3jwYGbPno2Pj09jNqWIyFWhhFBErprirGOc/iCHoH3nWLtsNU+VjWfkz35A//792bFjBzt27HD1uFVUVBAcHOwqm5KSAkBMTEydY/XWrVvHypUrmTNnDgAlJSXk5eUBkJiYSPv27QHo2bMnubm55OfnM2DAAEJDQwEIDAyst54ePXq49lVeXk5GRgZZWVl07tyZBx98kEWLFvGzn/3sSjWXiIjHKCEUkauiOOsYBX/9J7askq6BnVgz+g98lLuZKamTuef+oYwcOZLw8HA2bdpUa3k/Pz8AfHx8KC8vr3Ubay3Lly8nLCys2vLNmze7yl+ojvrqcRcSEkJUVBRdu3YF4IEHHuDzzz9XQigi1wSNIRSRq+L0BznYskoAjpw5QetWfqTcMYRHe41i27ZthIWFcfz4cVdCWFZWxs6dO+utMyAggDNnzrheJyUlMXfuXKy1AGRlZdVbvm/fvmzcuJH9+/cDuC4ZN6SeO++8k4KCAo4fPw7A3/72N3r27HnBdhARaQ7UQygiV0VFwTnX8z3H9zHr41doYVrQskVL3nhvMb6+vixbtowJEyZQWFhIeXk5qamphIeH11lnQkICs2fPJioqiilTppCWlkZqaioRERFUVlYSGhrKqlWr6iwfFBTE/PnzSUlJobKykg4dOrB+/foG1ePj48OcOXNITEzEWktMTAzjxo27/IYSEWkCTNUv4uYgNjbWZmZmejoMEWmAw7O3VEsKq/hc70fwc3EeiEhEpHEZY7Zaa2M9HUdD6JKxiFwV1yV1wbSq/hFjWrXguqQunglIRETqpEvGInJVtO3jmLT59Ac5VBScw+d6P65L6uJaLiIiTYcSQhG5atr26aAEUESkGdAlYxEREREvp4RQRERExMspIRQRERHxckoIRURERLycEkIRERERL6eEUERERMTLKSEUERER8XJKCEVERES8nBJCERERES+nhFBERETEyykhFBEREfFySghFREREvJwSQhEREREvp4RQRERExMspIRQRERHxckoIRURERLycEkIRERERL6eEUERERMTLKSEUEfEi2dnZrFmzxmP7f/PNN+nWrRvdunXjzTff9FgcIlJdS08HICIijSc7O5vMzEyGDRvW6Ps+deoUM2bMIDMzE2MMMTExjBgxgu985zuNHouIVKceQhGRZqC4uJjk5GQiIyPp1asX6enpAGzdupX4+HhiYmJISkri8OHDAAwcOJDJkycTFxdH9+7dycjIoLS0lOnTp5Oenk5UVBTp6ekUFxczduxY4uLi6NOnDytWrABg0aJFpKSkMHToULp168akSZNcsaxdu5bo6GgiIyNJTEx0xVdbPe4++OADhgwZQmBgIN/5zncYMmQIa9euvdpNJyINoB5CEZFmYO3atXTs2JHVq1cDUFhYSFlZGePHj2fFihUEBQWRnp7O1KlTWbBgAQDl5eVs2bKFNWvWMGPGDDZs2MDMmTPJzMxk3rx5ADz//PMMGjSIBQsWUFBQQFxcHIMHDwYcvYlZWVn4+fkRFhbG+PHj8ff3Z9y4cWzcuJHQ0FBOnToFwKxZs2qtp23btq5j+Oabb+jUqZPrdUhICN98802jtJ+I1E8JoYhIE/Zu1jf8vw++JHffSU4se4+TZY/x5M8eon///uzYsYMdO3YwZMgQACoqKggODnaVTUlJASAmJoacnJxa61+3bh0rV65kzpw5AJSUlJCXlwdAYmIi7du3B6Bnz57k5uaSn5/PgAEDCA0NBSAwMLDeenr06HGFW0RErgYlhCIiTdS7Wd8w5a//4GxZBS0DbyFo9Et8nruNX6Q+y4P3D2PkyJGEh4ezadOmWsv7+fkB4OPjQ3l5ea3bWGtZvnw5YWFh1ZZv3rzZVf5CddRXj7tbbrmFjz/+2PX64MGDDBw4sM7tRaTxaAyhiEgT9f8++JKzZRUAlJ85SYtWfvjeEU9lr+Fs27aNsLAwjh8/7koIy8rK2LlzZ711BgQEcObMGdfrpKQk5s6di7UWgKysrHrL9+3bl40bN7J//34A1yXjhtSTlJTEunXryM/PJz8/n3Xr1pGUlNSQphCRq0w9hCIiTdShgrOu52XHczj28UIwBtOiJYvf+zO+vr4sW7aMCRMmUFhYSHl5OampqYSHh9dZZ0JCArNnzyYqKoopU6aQlpZGamoqERERVFZWEhoayqpVq+osHxQUxPz580lJSaGyspIOHTqwfv36BtUTGBhIWload955JwDTp093XXIWEc8yVb/mmoPY2FibmZnp6TBEROqVnZ3NoUOHLntql7tn/41v3JLCKrdc35pPnxtUZzkfHx969+4NQOfOnVm5cuVlxSEil8YYs9VaG+vpOBpCl4xFRK6wKzX587NJYbRu5VNtWetWPjybVPc4PYDWrVuTnZ1Ndna2kkERaRAlhCIiTk1trr+ZPxtOSfpTFCxLwwA3t4GO2xfxHz8fWedcfyIil8Ra22weMTExVkTkalm2bJl95JFHXK8LCgpsaWmp7devnz127Ji11tqlS5faMWPGWGutjY+Pt0899ZS11trVq1fbxMREa621CxcutI8//rirnilTptjFixdba63Nz8+33bp1s0VFRXbhwoU2NDTUFhQU2LNnz9rOnTvbvLw8e+zYMRsSEmL37dtnrbX25MmT9dZTk4+Pj42JibF33XWXfeedd65oG4lIwwGZtgnkTw156KYSEfF6uzM+ImPpH9mXk8M7GV9gi4v46S8fa7Zz/eXm5nLLLbewb98+Bg0aRO/evbntttuuRFOJyDVKCaGIeLXdGR+xbv48ykvPEdSuLRMHfZevcr/mqfFPMDzle81urj9wzPcH0LVrVwYOHEhWVpYSQhGpl8YQiohXy1j6R8pLzwFQeLaEVi19iAq5ibtuCWqWc/3l5+dz7pzjeE6cOMGnn35Kz549L9gOIuLd1EMoIl7tzMkTrudHCs+w6u+7McbQwhjeWbeh2c31t3v3bn7+85/TokULKisree6555QQisgFaR5CEfFq8x8fw5kTx89bHnBjEI/+70IPRCQi1wrNQygi0kz0/+FoWvr6VVvW0teP/j8c7aGIREQany4Zi4hX69E/AXCMJTxz8gQBN9xI/x+Odi0XEfEGSghFxOv16J+gBFBEvJouGYuIiIh4OSWEIiIiIl7OYwmhMaaTMeYjY8wuY8xOY8xET8UiIiIi4s08OYawHHjaWrvNGBMAbDXGrLfW7vJgTCIiIiJex2M9hNbaw9babc7nZ4DdwC2eikdERETEWzWJMYTGmC5AH2BzLeseNcZkGmMyjx8/f/JYEREREbk8Hk8IjTHtgOVAqrX2dM311tr51tpYa21sUFBQ4wcoIlJDdnY2a9as8di++/XrR3h4OBEREaSnp3skDhG5tng0ITTGtMKRDC6x1v7Vk7GIiDSUJxPCNm3a8Mc//pGdO3eydu1aUlNTKSgo8EgsInLt8ORdxgZ4A9htrf2dp+IQEe9RXFxMcnIykZGR9OrVy9W7tnXrVuLj44mJiSEpKYnDhw8DMHDgQCZPnkxcXBzdu3cnIyOD0tJSpk+fTnp6OlFRUaSnp1NcXMzYsWOJi4ujT58+rFixAoBFixaRkq1nmxIAACAASURBVJLC0KFD6datG5MmTXLFsnbtWqKjo4mMjCQxMdEVX231uOvevTvdunUDoGPHjnTo0AENpxGRy+XJu4zvBn4C/MMYk+1c9ry11jM/u0Xkmrd27Vo6duzI6tWrASgsLKSsrIzx48ezYsUKgoKCSE9PZ+rUqSxYsACA8vJytmzZwpo1a5gxYwYbNmxg5syZZGZmMm/ePACef/55Bg0axIIFCygoKCAuLo7BgwcDjt7ErKws/Pz8CAsLY/z48fj7+zNu3Dg2btxIaGgop06dAmDWrFm11tO2bdtaj2fLli2UlpZy2223Xe2mE5FrnMcSQmvtJ4Dx1P5FxHus3real7e9TO7eXPJW5JHfIp+JP55I//792bFjBzt27GDIkCEAVFRUEBwc7CqbkpICQExMDDk5ObXWv27dOlauXMmcOXMAKCkpIS8vD4DExETat28PQM+ePcnNzSU/P58BAwYQGhoKQGBgYL319OjR47x9Hj58mJ/85CdMnTqVtWvXMmzYsMttpouWm5vLyJEjqaysdCXWv/jFLwBH7+rhw4dp3bo14Di2Dh06NHqMItIw+lvGInJNW71vNS989gIlFSX43uxLl191YduObTz2zGN8P/n7jBw5kvDwcDZt2lRreT8/PwB8fHwoLy+vdRtrLcuXLycsLKza8s2bN7vKX6iO+uqp6fTp0yQnJzNr1iyKiopYs2aNRxLC4OBgNm3ahJ+fH0VFRfTq1YsRI0bQsWNHAJYsWUJsbGyjxyUiF8/jdxmLiFxNL297mZKKEgDK8sto4duCtn3b0nJQS7Zt20ZYWBjHjx93JYRlZWXs3Lmz3joDAgI4c+aM63VSUhJz587FWgtAVlZWveX79u3Lxx9/TEJCApGRkfTo0YP09HSSkpJIS0tzjWfs16/feeMZ77zzTm6++WbuvvtuRowY4dHxjL6+vq6E99y5c1RWVtZ73CLSdCkhFJFr2pHiI67nJQdL2DtzL1+nfc2X6V8ybdo0fH19WbZsGZMnTyYyMpKoqCg+++yzeutMSEhg165driQsLS2NsrIyIiIiCA8PJy0trd7yQUFBPPzww2zfvh2AkJAQhg4dynPPPcfGjRs5duwYJSUlnDt3jqlTp7rKlZeXM3HiRM6dO8eiRYuIi4sDHJels7OzefDBB13jELds2cJHH33Es88+S3FxMeAYz5iens4//vEP0tPTOXDgAMePH2fcuHEsX76cv//977z99tsA9dbj7sCBA0RERNCpUycmT57s6h0EGDNmDFFRUfz61792Jcsi0jSZ5vSfNDY21mZmZno6DBFpRu5Zdg+Hiw+ftzy4bTDrRq1r9Hh2Z3xExtI/si8nh9czvuCB+4bz018+5hrP+N3vfpeuXbsC/xrPuG7dOgYOHMisWbO4++67OXr0KHfffTdff/01ixYtqnaDS2xsLCUlJbRs6RgRdOrUKT744AM2b97Mp59+yh/+8AcA7r33XqZOnUp+fj5Lly5lyZIl1eKsq57axjMCHDp0iAceeID33nuPm266iW+++YZbbrmFM2fO8L3vfY8f//jHjB49+qq0qUhTZYzZaq1tFuMmNIZQRK5pE6MnusYQVvH38Wdi9MRGj2V3xkesmz+P8tJzBLVry8RB3+Wr3K95avwTDE/5XrMcz1ilY8eO9OrVi4yMDEaNGsUttzj+EmlAQAA/+tGP2LJlixJCkSZMl4xF5JqW3DWZF777AsFtgzEYgtsG88J3XyC5a3Kjx5Kx9I+Ul54DoPBsCa1a+hAVchN33RLk0fGMGzduZP/+/QCuKXAaUs/Bgwc5e/YsAPn5+XzyySeEhYVRXl7OiRMnXMewatUqevXqVX/jiIhHqYdQRK55yV2TPZIA1nTm5AnX8yOFZ1j1990YY2hhDO+s2+AazzhhwgQKCwspLy8nNTWV8PDwOutMSEhg9uzZREVFMWXKFNLS0khNTSUiIoLKykpCQ0NZtWpVneWDgoKYP38+KSkpVFZW0qFDB9avX9+genbv3s3TTz+NMQZrLc888wy9e/emuLiYpKQkysrKqKioYPDgwYwbN+7yG1BErhqNIRQRaSTzHx/DmRPn/1WRgBuDePR/F3ogIhG5mprTGEJdMhYRaST9fzialr5+1Za19PWj/w81tk5EPEuXjEVEGkmP/gmAYyzhmZMnCLjhRvr/cLRruYiIpyghFBFpRD36JygBFJEmR5eMRURERLycEkIRERERL6eEUERERMTLKSEUERER8XJKCEVERES8nBJCERERES+nhFBERETEyykhFBEREfFySghFREREvJwSQhEREREvp4RQRERExMspIRQRERHxckoIRURERLycEkIRERERL6eEUERERMTLKSEUERER8XJKCEVERES8nBJCERERES+nhFBEpBFkZ2ezZs0aj8Zw+vRpQkJCeOKJJzwah4g0PUoIRUQaQVNICNPS0hgwYIBHYxCRpkkJoYhIPYqLi0lOTiYyMpJevXqRnp4OwNatW4mPjycmJoakpCQOHz4MwMCBA5k8eTJxcXF0796djIwMSktLmT59Ounp6URFRZGenk5xcTFjx44lLi6OPn36sGLFCgAWLVpESkoKQ4cOpVu3bkyaNMkVy9q1a4mOjiYyMpLExERXfLXVU9PWrVs5evQo99xzz9VsLhFpplp6OgARkaZs7dq1dOzYkdWrVwNQWFhIWVkZ48ePZ8WKFQQFBZGens7UqVNZsGABAOXl5WzZsoU1a9YwY8YMNmzYwMyZM8nMzGTevHkAPP/88wwaNIgFCxZQUFBAXFwcgwcPBhy9iVlZWfj5+REWFsb48ePx9/dn3LhxbNy4kdDQUE6dOgXArFmzaq2nbdu2rmOorKzk6aef5k9/+hMbNmxozOYTkWZCCaGISC2+2nyETSv2sndfEe+sXQUlTzD60Qfp378/O3bsYMeOHQwZMgSAiooKgoODXWVTUlIAiImJIScnp9b6161bx8qVK5kzZw4AJSUl5OXlAZCYmEj79u0B6NmzJ7m5ueTn5zNgwABCQ0MBCAwMrLeeHj16uPb1yiuvMGzYMEJCQq5U84jINUYJoYhIDV9tPsJHS/ZQXlrJTdd3YtIDv2fPoS94euIk7nvgXkaOHEl4eDibNm2qtbyfnx8APj4+lJeX17qNtZbly5cTFhZWbfnmzZtd5S9UR331uNu0aRMZGRm88sorFBUVUVpaSrt27Zg9e3adZUTEu2gMoYhIDZtW7KW8tBKAguIT+Lb0J6ZrInffNpJt27YRFhbG8ePHXQlhWVkZO3furLfOgIAAzpw543qdlJTE3LlzsdYCkJWVVW/5vn37snHjRvbv3w/gumTckHqWLFlCXl4eOTk5zJkzh9GjRysZFJFq1EMoIlJD0alzrueHTu3n3c9fw5gW+LRoyfI1f8LX15dly5YxYcIECgsLKS8vJzU1lfDw8DrrTEhIYPbs2URFRTFlyhTS0tJITU0lIiKCyspKQkNDWbVqVZ3lg4KCmD9/PikpKVRWVtKhQwfWr19/0fWIiNTGVP2qbA5iY2NtZmamp8MQkWvcm89/Wi0prNIu0I+f/sfdHohIRJojY8xWa22sp+NoCF0yFhGpod/9t9HSt/rHY0vfFvS7/zYPRSQicnXpkrGISA3d77oZcIwlLDp1jnaBfvS7/zbXchGRa40SQhGRWnS/62YlgCLiNXTJWERERMTLKSEUEWnCmsLfQD59+jQhISE88cQTAHz77bckJydzxx13EB4eznPPPefR+ETk8ikhFBFpwppCQpiWlsaAAQOqLXvmmWfYs2cPWVlZfPrpp7z//vseik5ErgQlhCIiV0FxcTHJyclERkbSq1cv0tPTAdi6dSvx8fHExMSQlJTE4cOHARg4cCCTJ08mLi6O7t27k5GRQWlpKdOnTyc9PZ2oqCjS09MpLi5m7NixxMXF0adPH1asWAHAokWLSElJYejQoXTr1o1Jkya5Ylm7di3R0dFERkaSmJjoiq+2emraunUrR48e5Z577nEta9OmDQkJCQD4+voSHR3NwYMHr3wjikjjsdY2m0dMTIwVEWkOli1bZh955BHX64KCAltaWmr79etnjx07Zq21dunSpXbMmDHWWmvj4+PtU089Za21dvXq1TYxMdFaa+3ChQvt448/7qpnypQpdvHixdZaa/Pz8223bt1sUVGRXbhwoQ0NDbUFBQX27NmztnPnzjYvL88eO3bMhoSE2H379llrrT158mS99birqKiw8fHx9sCBA+fFUSU/P9+GhobavXv3Xn6jiVxjgEzbBPKnhjx0l7GIyBW0et9qXt72Mrl7c8lbkUd+i3wm/ngi/fv3Z8eOHezYsYMhQ4YAUFFRQXBwsKtsSkoKADExMeTk5NRa/7p161i5ciVz5swBoKSkhLy8PAASExNp3749AD179iQ3N5f8/HwGDBhAaGgoAIGBgfXW06NHD9e+XnnlFYYNG0ZISEitsZSXl/PQQw8xYcIEunbtekntJSJNgxJCEZErZPW+1bzw2QuUVJTge7MvXX7VhW07tvHYM4/x/eTvM3LkSMLDw11/A7kmPz8/AHx8fCgvL691G2sty5cvJywsrNryzZs3u8pfqI766nG3adMmMjIyeOWVVygqKqK0tJR27dq5/g7yo48+Srdu3UhNTa2zDhFpHjSGUETkCnl528uUVJQAUJZfRgvfFrTt25aWg1qybds2wsLCOH78uCshLCsrY+fOnfXWGRAQwJkzZ1yvk5KSmDt3Ltb5Z0ezsrLqLd+3b182btzI/v37ATh16lSD61myZAl5eXnk5OQwZ84cRo8e7UoGp02bRmFhIS+99NIF20VEmj4lhCIiV8iR4iOu5yUHS9g7cy9fp33Nl+lfMm3aNHx9fVm2bBmTJ08mMjKSqKgoPvvss3rrTEhIYNeuXa6bStLS0igrKyMiIoLw8HDS0tLqLR8UFMT8+fNJSUkhMjKSBx98EOCi63F38OBBZs2axa5du4iOjiYqKorXX3+9weVFpOkxVb8Om4PY2FibmZnp6TBERGp1z7J7OFx8+LzlwW2DWTdqnQciEhFPMsZstdbGejqOhlAPoYjIFTIxeiL+Pv7Vlvn7+DMxeqKHInLw5FyGH330EVFRUa6Hv78/7777rkdiEZG66aYSEZErJLlrMuAYS3ik+Ag3t72ZidETXcs9JTs7m8zMTIYNG9bo+05ISCA7OxtwjF+8/fbbq81pKCJNg3oIRUSuoOSuyawbtY7tP93OulHrLisZvFYmt66ybNky7r33Xtq0aXPJbSIiV4d6CEVEmqi1a9fSsWNHVq9eDUBhYSFlZWWMHz+eFStWEBQURHp6OlOnTmXBggWAY27ALVu2sGbNGmbMmMGGDRuYOXMmmZmZzJs3D4Dnn3+eQYMGsWDBAgoKCoiLi2Pw4MGAozcxKysLPz8/wsLCGD9+PP7+/owbN46NGzcSGhrqulN51qxZtdbTtm3bWo9n6dKlPPXUU1e72UTkEighFBFpYrZv386HH37Ivn37WL58OeXl5YwdO7ZZTm5d5fDhw/zjH/8gKSnpcptHRK4CJYQiIk3I9u3bee+99ygrK+OGG25g3Lhx7Nu3jyeffJIRI0Y0u8mtq7z11luMHDmSVq1aXXBbEWl8GkMoItKEfPjhh5SVlQFw5swZWrVqRXh4OFFRUc1ycusqf/nLX3jooYfq3Y+IeI56CEVEmpDCwkLX86NHj7J+/XqMMfj4+LBixQrX5NYTJkygsLCQ8vJyUlNTCQ8Pr7POhIQEZs+eTVRUFFOmTCEtLY3U1FQiIiKorKwkNDSUVatW1VnefXLryspKOnTowPr16xtcT05ODgcOHCA+Pv7yGkdErhpNTC0i0oS8+OKL1ZLCKu3bt+fJJ5/0QEQicqk0MbV4BU9OdgswdOhQrr/+eu67775a10+YMIF27do1clQilycxMfG8cXatWrVyTfUiInI1KCGUS+bphPDZZ59l8eLFta7LzMwkPz+/kSMSuXwREREMHz7cdadv+/btGT58OBERER6OTESuZUoIvdC1MtltYmIiAQEB5y2vqKjg2Wef5b//+7+vaLuJNJaIiAiefPJJXnjhBZ588kklgyJy1emmEi90rU12W9O8efMYMWJEtbnZREREpG5KCL3IV5uPsGnFXvbuK+Kdtaug5AlGP/pgs57stqZDhw7x9ttv8/HHH19k64iIN8nOzubQoUMe+fvOVU6fPk3Pnj154IEHXD+sq4wYMYJ9+/axY8cOD0Un3kYJoZf4avMRPlqyh/LSSm66vhOTHvg9ew59wdMTJ3HfA/c228lua8rKyuLrr7/m9ttvB+Dbb7/l9ttv5+uvv77oukTk2pWdnU1mZqZHE8K0tDQGDBhw3vK//vWvuiFOGp3GEHqJTSv2Ul5aCUBB8Ql8W/oT0zWRu28b2awnu60pOTmZI0eOkJOTQ05ODm3atFEyKHINuVbGQG/dupWjR49yzz33VFteVFTE7373O6ZNm3bF206kPuoh9BJFp865nh86tZ93P38NY1rg06Ily9f8qVlOdtu/f3/27NlDUVERISEhvPHGG/o7qSLXuGthDHRlZSVPP/00f/rTn9iwYUO140tLS+Ppp5+mTZs2V70tRdwpIfQS7QL9XElhz0530rPTna7lsbGOOTOjoqLYuHHjeWXdx+PdeOONrjGEgYGBfPHFF9W2fe21184r//DDD/Pwww+7Xrsnd/feey/33ntvte1bt25daz01ZWRkXHCboqKiC24jIk3ftTQG+pVXXmHYsGGEhIRUiyE7O5u9e/fy4osv1hmnyNWihNBL9Lv/NtcYwiotfVvQ7/7bPBiViDQHnrwBIzc3l2H33MfpU2cprygnvtdIJj3we3Yc+Izh942gZSsf2rdvT0BAANnZ2bXW0dTGQG/atImMjAxeeeUVioqKKC0tpV27dtx6661kZmbSpUsXysvLOXbsGAMHDtRNctIo6hxDaIzpbYz53BhzwBgz3xjzHbd1W67Ezo0xC4wxx4wxuo3qKut+180k/PsdtAt0fLC1C/Qj4d/voPtdN3s4MhFp6jw5CX1wcDBP3v8/PPe9+Tw78n/5YOuf+La0iD5dBjK4zw/4t3/7N/7+979z6tQpfve73wFNfwz0kiVLyMvLIycnhzlz5jB69Ghmz57NL3/5Sw4dOkROTg6ffPIJ3bt3VzIojaa+m0p+D7wA9Aa+Aj4xxlR1J7Wqq9BFWgQMvUJ1yQV0v+tmfvofd/P4q4P46X/crWQQz/+1FXBMPRESEsITTzzhWjZ16lQ6deqkOw3lklSd1564AWPo0KFcd911+Pn5ceONN/Lqq68CjrF/UVFR3HDDDbRp04Y77riDJUuWXPAGDF9fX86ddiRXZRWllFeW8er7U/ntigl8sev/mDZtGu3atWPkyJG8+uqrREZGEhUVxWeffVZvGyUkJLBr1y7XMaWlpVFWVkZERATh4eGkpaXVW959DHRkZCQPPvggwEXXI9JkWGtrfQB/r/E6Afgn0BfYVle5i30AXYAdDdk2JibGilxJCxcutI8//rhHY5gwYYJ96KGHqsWxadMme+jQIdu2bVsPRibNVdV5vWzZMvvII4+4lhcUFNjS0lLbr18/e+zYMWuttUuXLrVjxoyx1lobHx9vn3rqKWuttatXr7aJiYnV6qsyZcoUu3jxYmuttfn5+bZbt262qKjILly40Hbp0sUePXrUnj171oaEhNiQkBC7fft2GxISYsePH2+nTp1qT548aSsqKuzEiRNrraem3z6+3HYMDLWtWvrZ79893s77+Yd23s8/tIumfOIqGxoaavfu3Xulm1LksgCZ9grlS1f7UW9CCLSvsSzCmRSevGIBXCAhBB4FMoHMzp07X+JbIteaoqIiO2zYMBsREWHDw8Pt0qVLrbXWZmZm2gEDBtjo6Gh7zz332EOHDllrHV90kyZNsnfeeaft1q2b3bhxoz137pzt1KmTvfHGG21kZKRdunSpLSoqsmPGjLF33nmnjYqKsu+++6611vGFOHLkSJuUlGRvv/12++yzz7pief/9922fPn1sRESEHTRokCu+2uqpKTMz0z744IN1JqZKCL3LlTqvg4KDbKuAVtYv2M+2Cmhl7x9zvx02bJi988477R133GH9/f1tZGSk7dSpk73uuuvsDTfcYG+//XYbEhJiP/nEkWT9+c9/tr6+vjYiIsL26NHDPv74467zuk2bNtbf39926dLFVc+uXbvswoULqyWggwYNsh06dLALFy60P/rRj2xISEi1hC8mJsaGh4fbyMjIavXU9OXnh+2r4z+yv/lxur01KMz+x0/etq+O/8h++flhW1ZWZocOHWpffPHFq/nWXFBWVpZdvXq1x/aflJRk27dvb5OTkz0Wg5zvWkkIfwT0rWV5Z+APVywA9RDKJfBEz0doaKgtKCiwZ8+etZ07d7Z5eXn22LFjNiQkxO7bt89aa+3JkyfrrcddRUWFjY+PtwcOHFBCKNbaK3Ner9q7ynYZ18UGJgbaXot62Tvm3WHb92lvQ7qG2BkzZthPPvnE+vv713pe+/n52ffee88eO3bMduzY0Xbs2NFaa+3cuXPt448/7jqvo6Oj7ZYtW847r6vO47y8PNu7d2/bokULO2HCBLty5Uo7atQoGxISYp988knbp08fO2rUKNu7d2+7Z8+eBrXNl58ftoumfGL7hg21j4/8tf3y88PWWmvHjBljx48ff/mNf5k8fbVhw4YNduXKlUoIm5jmlBDWOYbQWvtna+3ntSzPs9aOu8Qr1CKXZ/tb8GIven/8MOuXv8nkMfeTkZFB+/bt+fLLL11TT0RFRfGb3/yGgwcPuoo2dOqJqrkVBw4cWOvUE/7+/q6pJz7//PM6p56oq54qdU09Id6nOOsYh2dvIeiDc6xdtpqnHh5/yef1y9tepqyyDICy/DJa+LbgXP45Tpw+wW9/+1see+wxysvLWblyJeAYS3fw4EH8/f1p06YNR44c4fPPP6dfv360auUYLh4cHMyZM2dc5/XBgwe59957Xed1zRsnOnXqxPbt20lISGDdunV07dqVTz/9lIMHD/Ld736XDRs20K9fPyorKy94A8bBgwc5e/Ys3e+6mRHP9uRk5V5+PuN+ut91M9OmTaOwsJCXXnrp0tv+GpnoOjExkYCAgEtuBxFNOyPNx/a34L0JUHaW7je0YNs4f9bs+5hpT/6TxBE/bHZ/fq+uqSdmz55dZxm59hRnHaPgr//EllXSNbATa0b/gY9yNzMldTL33D/0os/rI8VHXOtKDpZwJP0IpUdL8e3gy4fvf0hsbCzZ2dlMmDCB/fv3U1xcTN++fQkPD8cYQ0VFxXn7qJqEfteuXcyaNYtf/OIXpKam8tlnnzFq1Kg6J4/39/enS5cu7N69m9dff53hw4czc+ZMXnvtNRYsWMDrr7/uugGjrknod+/ezdNPP40xBmstzzzzDL179+bgwYPMmjWLO+64g+joaACeeOIJHnnkkYtq/2thomuRK8GjCaEx5i/AQOBGY8xB4FfW2jc8GZM0YR/OhLKzABw6U0lga8OPw+H69sW8vm0bzz33nOvP7/Xr14+ysjK++uqrev/aSl1TT8ydOxdjDFlZWfTp06fO8n379uWxxx5j//79rg/xwMDABtWzZMkS1/NFixaRmZmpZNALnf4gB1vmmB/0yJkTXN86gJQ7hnB9++v567aNF31e39z2Zgr8C6gsqSSgdwABvQM4suwIrctaExMTAzh+sGzcuNF13o0b57jo069fP8LCwggPD+exxx6rNlH9F198wfPPP8/evXvx9/fntddeO++8fvjhhxk8eDBnz56ldevWLF68mLvuuouwsDB69+7N97//fR599FEGDRrEokWLCA8Pv+Ak9EOGDGH79u3nLQ8JCXH1LF6S7W/BhzPpvS+Xp5efY3L5Me4b+0yznOha5Eq4YEJojLnbWvvphZZdCmvtQ5dbh3iRwn9dJvvH0UqeXV9CCwOtfIr5/YrlzfLP79Vl0qRJ/PnPf+bbb78lJCSERx55hBdeeKHB5aX5qCj415+V3HN8H7M+foUWpgUtW7TkjfcWX/R5PTF6ItNOTOPEmhN8nfY1QfcF0XlkZzqs79Ao53VdPXoA//Vf/8VPfvITUlNTCQoKYuHChVegBS/BNXa1QeRKMBf6hWWM2Watjb7QssYQGxtrMzMzG3u30lS82AsKD5y/vH0neFJzm0vzdHj2lmpJYRWf6/0Ifi7ukupcvW81L297mSPFR7i57c1MjJ5Ictfkyw312uH2WVJ1tcG/pWHVwet5/UQf3nrrLXr27MnixYvP65UdOHAgc+bMITY2lhMnThAbG0tOTg7Lly9n5cqVvPnmm4DjkvHp06fPu0pQ1StbdWn5vvvu45lnniE8PJzo6Ohql4wDAwPrrKc2H3/8MXPmzLmoH59ydRljtlprYz0dR0PU2UNojOkHfBcIMsY85bbqOsDnagcmcp7E6a5f9S6tWjuWizRT1yV1cY0hrGJateC6pC6XXGdy12QlgPW5Bq829O/fnz179lBUVERISAhvvPEGSUlJl9dO4lXq7CE0xsTjGN/3C+BVt1VngPestf+86tHVoB5CqRr3Q+FBaB/iSAYjfuDpqEQuS3HWMU5/kENFwTl8rvfjuqQutO3TwdNhXbt0tUEaSXPqIWzIJeNbrbW5jRRPvZQQiojIZXMbQ+jSqjUM/x/9wJQrqjklhA25y9jPGDMfxwTSru2ttYOuVlAiIiJXTVXSp6sNIi4NSQjfxnHJ+HXg/AmqRESugv/P3t2HVVmlDf//XiAvZopjaoJoUSEqtNkKoVaISgqKppB3zOSMo6Y9mmOhpWamk9z5G2fyHmv0acpSMO+ewHBGNAw1s4HKVJStYplTqGhYoAEK8s75+wPZI4q4fQEEz89xXEd7r73X2icrhJNrvVksBKioRAAAIABJREFUFrKzsxkxYkSTfPa0adM4e/Ys9vb2zJ8/n8jISKB61ecrr7zCRx99hL29PdOmTeO5555r9BjVDTI9qQmgUhexJSGsEJG/N3gkSil1EYvFQlpaWpMkhHfccQfvv/8+np6eZGdnW0+raN++PbGxsZw4cYLDhw9jZ2dHTk5Oo8enlFI32xWPrrvIJsMwnjUMw9UwjA41V4NHppRqllrCUWA9evTA09MTADc3Nzp37kxubi4Af//731m4cCF2dtU/Pjt31sUfSqkW4GqHHQNH67gym+LgZT8/vzqOjlZK3UoSEhJk8uTJ1uf5+flSVlYmAwYMkJycHBERiYuLk4kTJ4qISFBQkMyaNUtERJKSkiQ4OFhERGJiYmT69OnWdubNmydr164VEZG8vDzx9PSUwsJCiYmJEQ8PD8nPz5fi4mLp3r27ZGVlSU5Ojri7u0tmZqaIiJw5c6bedq5k165d0rNnT6msrBQRkQ4dOshrr70mfn5+EhoaKkeOHLnxTlNKtUhAmjRBvnQ911WHjEXEoyETUqVUy1CzdUqnzFKSE5KYVT6D8KefbNZHgZ06dYrf/e53rFmzxnpHsLS0FGdnZ9LS0vjHP/7BpEmTSE1NvaG+U0qppmbL0XV3ALOA7iLyjGEYnoCXiOhW6EopoDoZrNlc+b4O3dg8/l12HN/FvKi5DBsd2iyPAjt79ixhYWEsXryY/v37W8vd3d2tCWx4eDgTJ06stx2llGoObJlDGAOUUX1qCcCPwGsNFpFSqtk5u+WY9aSNn86dprWDExE9h/KMz1j27duHl5cXubm51oSwvLycQ4cO1dtm27ZtOXfunPV5SEgIy5cvr5nKQnp6er31+/fvT0pKCkePHgXgl19+sbmdsrIywsPDGT9+PGPHjq312pgxY9ixYwcA//rXv+jRo0e9cSilVHNgyyrj+0Uk0jCM3wCIyHnDMIwGjksp1YxcfBbv4dxMFn/+FnaGHa3sWrFq09pmdxTYunXrSElJ4cyZM8TGxgLVi1fMZjMvvfQS48aNY9myZdx555289957N9Z5Sil1C7DlpJKvgGDgSxHpaxjG/cCHInJ9p67fAD2pRKlb06klu2slhTXs2zvh+lKj/6hQSqlbQnM6qcSWIeM/AslAN8MwPgC2A3Pqr6KUup20C7kXw6H2jxPDwY52Ifc2TUBKKaWuiS2rjLcZhrEP6A8YwPMicrrBI1NKNRtt+lTvxXd2yzEq80uxb+9Eu5B7reVKKaVubbbMIQRwBvIuvL+3YRiISErDhaWUam7a9OmsCaBSSjVTtmw782cgEjgEVF0oFkATQqWUUkqpFsCWO4RjqN538PIZ40oppZRSqtmzZVFJJuDQ0IEopZRSSqmmYcsdwvOAxTCM7YD1LqGIPNdgUSmllFJKqUZjS0K48cKllFJKKaVaIFu2nVljGIYjUHM+03ciUt6wYSmllFJKqcZiyyrjQcAa4BjV+xB2Mwzj97rtjFJKKaVUy2DLkPH/AMNE5DsAwzB6AB8Cfg0ZmFJKKaWUahy2rDJ2qEkGAUTkCLrqWCmllFKqxbDlDmGaYRjvAf974fk4IK3hQlJKKaWUUo3JloRwGjAdqNlmJhV4q8EiUkoppZRSjcqWVcalhmGsALZTfXTddyJS1uCRKaWUUkqpRmHLKuMw4G3gB6pXGXsYhvF/ROSThg5OKaWUUko1PFsWlfwPMFhEBolIEDAYWNawYSmllFIth8ViYfPmzU3y2cePH6dv376YzWa8vb15++23ra+Fhobi6+uLt7c3U6dOpbKyskliVE3PloTwnIh8f9HzTOBcA8WjlFJKtThNmRC6urqyc+dOLBYLu3btYsmSJWRnZwOwbt069u/fT0ZGBrm5uXz00UdNEqNqerYkhGmGYWw2DGOCYRi/BzYBewzDiDAMI6KB41NKKaWaTFFREWFhYfj6+uLj40N8fDwAe/fuJSgoCD8/P0JCQjh16hQAgwYNYu7cuQQEBNCjRw9SU1MpKytj4cKFxMfHYzabiY+Pp6ioiEmTJhEQEECfPn1ITEwEIDY2loiICEJDQ/H09GTOnDnWWJKTk+nbty++vr4EBwdb46urnYs5Ojri5OQEQGlpKVVVVdbX2rVrB0BFRQVlZWUYhtEAvaiaA1tWGTsDPwNBF57nAq2BUYAA/2iY0JRSSqmmlZycjJubG0lJSQAUFBRQXl7OjBkzSExMpFOnTsTHxzN//nxWr14NVCdXu3fvZvPmzSxatIhPP/2U6Oho0tLSWLFiBQAvv/wyQ4YMYfXq1eTn5xMQEMBjjz0GVN9NTE9Px8nJCS8vL2bMmIGzszNTpkwhJSUFDw8PfvnlFwAWL15cZztt2rSp9XWcOHGCsLAwvv/+e15//XXc3Nysr4WEhLB7926GDx/O2LFjG7xP1a3JllXGExsjEKWUUupWUbBpEznL3uCO48f4JDubqF9+4YmoKAIDA8nIyCAjI4OhQ4cCUFlZiaurq7VuRET14Jmfnx/Hjh2rs/2tW7eyceNGli5dCkBJSQlZWVkABAcH4+LiAkDv3r05fvw4eXl5DBw4EA8PDwA6dOhQbzu9evWq9XndunXjwIEDZGdnM2bMGMaOHcvdd98NwJYtWygpKWHcuHF89tln1q9L3V5sWWXsAcwA7r34/SLyeMOFpZRSSjWNgk2bOLVgIVJSwr0Ojnzk3o3UtDRemjaNkCefJDw8HG9vb3bu3Fln/ZrhWXt7eyoqKup8j4iwfv16vLy8apXv2rXLWv9qbdTXzpW4ubnh4+NDampqrbuBzs7OjB49msTERE0Ib1O2zCHcABwDllO94rjmUkoppVqcnGVvICUl1Y8rymltGIxqfQe/t7Nn3759eHl5kZuba00Iy8vLOXToUL1ttm3blnPn/rMeMyQkhOXLlyMiAKSnp9dbv3///qSkpHD06FEA65CxLe2cPHmS4uJiAPLy8vjiiy/w8vKisLDQOvexoqKCpKQkevbsWX/nqBbLloSwRET+JiI7RORfNVeDR6bUdWjKlXwWi4UBAwbg7e2NyWSyTj4HGDduHF5eXvj4+DBp0iTKy8ubJEal1NVVXEiSAI6UlhJ5/Djhx46y/N9HeOWVV3B0dCQhIYG5c+fi6+uL2Wzmq6++qrfNwYMH880331gXlSxYsIDy8nJMJhPe3t4sWLCg3vqdOnVi5cqVRERE4OvrS2RkJIBN7Xz77bf069cPX19fgoKCePHFF3nwwQcpKiri8ccfx2QyYTab6dy5M1OnTr2OHlMtgVHzV8UV32AYTwGewFagtKZcRPY1bGiX8/f3l7Q0PUZZXVlsbGytiduN6ciRIxiGgaenJ9nZ2fj5+fHtt9/Svn17Nm/ezPDhwwF46qmnGDhwINOmTWv0GJVSV/fvIcFUXNiW5WKt3Nzw/Gx7E0SkmivDMPaKiH9Tx2ELW+4QPghMAZbwn+HipQ0ZlLr9tIStHXr06IGnpydQPU+nc+fO5ObmAjBixAgMw8AwDAICAjh58mTDdaZS6oZ0nhmF4excq8xwdqbzzKgmikipRiAi9V7A94Dj1d7XGJefn5+olikhIUEmT55sfZ6fny9lZWUyYMAAycnJERGRuLg4mThxooiIBAUFyaxZs0REJCkpSYKDg0VEJCYmRqZPn25tZ968ebJ27VoREcnLyxNPT08pLCyUmJgY8fDwkPz8fCkuLpbu3btLVlaW5OTkiLu7u2RmZoqIyJkzZ+pt50p27dolPXv2lMrKylrlZWVl0qdPH0lJSbn+zlJKNbj8jRvlyOAh8k3PXnJk8BDJ37ixqUNSzRCQJrdA/mTLZcs+hBlAeyCnAfNSdZtKykzizX1vcvyH42QlZpFnl8fzv32+2W7tAHDq1Cl+97vfsWbNGuzsat+Ef/bZZxk4cCCBgYHX1V9KqcbhMmoULqNGNXUYSjUaWxLC9sBhwzD2UHsOoW47o25IUmYSr371KiWVJTh2ceTeP97Lvox9PPvis/xX2H81y60dzp49S1hYGIsXL6Z///61Xlu0aBG5ubm888479bahlFJKNTZb5hD+EQgH/j902xl1E725701KKqu3dijPK8fO0Y42/dvQakirZrm1Q1lZGeHh4YwfP/6y3f7fe+89tmzZwocffnjZXUOllFKqqV31N5NUbzFzGGh74fpWdNsZdRP8VPST9XHJyRJ+iP6B7xd8z3fx3zXLrR3WrVtHSkoKsbGxmM1mzGYzFosFgKlTp/Lzzz8zYMAAzGYz0dHR19pdSimlVIOxZduZJ4HXgc8BAwgEZotIQoNHdwnddqZlGZYwjFNFpy4rd23jytaxW5sgIqWUUurmaWnbzswHHhKR34vIeCAAqP82i1I2eL7v8zjb197awdnemef7Pt9EESmllFK3J1sWldiJyMUrjM9gWyKpVL3C7gsDqucS/lT0E13adOH5vs9by5VSSinVOGxJCJMNw9gCfHjheSTwScOFpG4nYfeFaQKolFJKNbGrJoQiMtswjAjg0QtFK0Xknw0bllJKKaWUaixXTAgNw3gAuFtEvhSRfwD/uFD+qGEY94vID40VpFJKKaWUajj1zQV8AzhbR3nBhdeUUkqpJmexWNi8eXOTfX5oaCjt27dn5MiRtconTJiAh4fHZdtQKXUrqi8hvFtEDl5aeKHs3gaLSCmllLoGTZ0Qzp49m7Vr19b52uuvv47FYsFisWA2mxs5MqVsV19C2L6e11rf7ECUUkrdXoqKiggLC8PX1xcfHx/i4+MB2Lt3L0FBQfj5+RESEsKpU9X7lQ4aNIi5c+cSEBBAjx49SE1NpaysjIULFxIfH2/dkL6oqIhJkyYREBBAnz59SExMBCA2NpaIiAhCQ0Px9PRkzpw51liSk5Pp27cvvr6+BAcHW+Orq51LBQcH07Zt24bsKqUaXH2LStIMw5giIu9eXGgYxmRgb8OGpZRSqqVLTk7Gzc2NpKQkAAoKCigvL2fGjBkkJibSqVMn4uPjmT9/PqtXrwagoqKC3bt3s3nzZhYtWsSnn35KdHQ0aWlprFixAoCXX36ZIUOGsHr1avLz8wkICOCxxx4Dqu8mpqen4+TkhJeXFzNmzMDZ2ZkpU6aQkpKCh4eH9cjKxYsX19lOmzZtbP4a58+fT3R0NMHBwSxZsqTWGepK3UrqSwijgH8ahjGO/ySA/oAj1WcbK6WUUtfs29QdpMa9T+axY/wzdQ9SVMjvpz1LYGAgGRkZZGRkMHToUAAqKytxdXW11o2IiADAz8+PY8eO1dn+1q1b2bhxI0uXLgWgpKSErKwsoPpunouLCwC9e/fm+PHj5OXlMXDgQDw8PADo0KFDve306tXLpq/zT3/6E126dKGsrIxnnnmGP//5zyxcuPBaukqpRnPFhFBEfgYeNgxjMOBzoThJRD5rlMiUUkq1ON+m7mDryhVUlJXS6c42PD/kYY4c/55ZM/7AqIgnCA8Px9vbm507d9ZZv+YOm729PRUVFXW+R0RYv349Xl5etcp37dpV6w5dfW3U146tahJZJycnJk6caE0slboVXfXEERHZISLLL1yaDCqllLpuqXHvU1FWCkBBcQkOrewxu99Nv66d2LdvH15eXuTm5loTwvLycg4dOlRvm23btuXcuXPW5yEhISxfvhwRASA9Pb3e+v379yclJYWjR48CWIeMr7WdS9XMfRQRNmzYgI+Pz1VqKNV0bDmpRCmllLopzp05bX38U8E5Pt7/LYZhYGcY/HPrpzg6OpKQkMBzzz1HQUEBFRUVREVF4e3tfcU2Bw8ezJIlSzCbzcybN48FCxYQFRWFyWSiqqoKDw8PPv744yvW79SpEytXriQiIoKqqio6d+7Mtm3bbG4nMDCQw4cPU1hYiLu7O6tWrSIkJIRx48aRm5uLiGA2m3n77bdvrPOUakBGzV8+zYG/v7+kpaU1dRhKKaWu08rpEzl3Ovey8rYdO/HM/41pgoiUajiGYewVEf+mjsMWVx0yVkoppW6WwF+Pp5Vj7ZW2rRydCPz1+CaKSCkF9R9ddw6o6/ahAYiItGuwqJRSSrVIvQIHA9VzCc+dOU3buzoS+Ovx1nKlVNOob5Wx7rKplFLqpusVOFgTQKVuMTYvKjEMozPgXPNcRLIaJCKllFJKKdWorjqH0DCMxw3D+DdwFPgXcAz4pIHjUkoppZRSjcSWRSX/DfQHjoiIBxAMfN2gUSmllFJKqUZjS0JYLiJnADvDMOxEZAfVR9gppZRSSqkWwJY5hPmGYdwJpAIfGIaRAxQ1bFhKKaWUUqqx2HKHcDRQDEQBycAPwKiGDEqpm81isbB58+Ym+ezjx4/Tt29fzGYz3t7e1tMKzp07h9lstl4dO3YkKiqqSWJUSil1e7vqHUIRKTIMowsQAPwCbLkwhKxUs2GxWEhLS2PEiBGN/tmurq7s3LkTJycnCgsL8fHx4fHHH8fNzQ2LxWJ9n5+fHxEREY0en1JKKWXLKuPJwG4gAhgLfG0YxqSGDkwpgKKiIsLCwvD19cXHx4f4+HgA9u7dS1BQEH5+foSEhFgPkR80aBBz584lICCAHj16kJqaSllZGQsXLiQ+Ph6z2Ux8fDxFRUVMmjSJgIAA+vTpQ2JiIgCxsbFEREQQGhqKp6cnc+bMscaSnJxM37598fX1JTg42BpfXe1czNHRESen6pMZSktLqaqquuw9R44cIScnh8DAwJvbgUoppZQtRKTeC/gOuOui53cB312tni0XEHqh/e+Bl672fj8/P1G3l4SEBJk8ebL1eX5+vpSVlcmAAQMkJydHRETi4uJk4sSJIiISFBQks2bNEhGRpKQkCQ4OFhGRmJgYmT59urWdefPmydq1a0VEJC8vTzw9PaWwsFBiYmLEw8ND8vPzpbi4WLp37y5ZWVmSk5Mj7u7ukpmZKSIiZ86cqbedS2VlZcmDDz4orVu3lhUrVlz2+qJFi+SFF164sc5SSil1SwHS5CbkS41x2bKo5Axw7qLn5y6U3RDDMOyB/wsMBU4CewzD2Cgi39xo26r5O3DgANu3byczM5P169dTUVHBpEmTCAwMJCMjg4yMDIYOHQpAZWUlrq6u1ro1w65+fn4cO3aszva3bt3Kxo0bWbp0KQAlJSVkZVXvtR4cHIyLiwsAvXv35vjx4+Tl5TFw4EA8PDwA6NChQ73t9OrVq9bndevWjQMHDpCdnc2YMWMYO3Ysd999t/X1uLg41q5de0N9druwWCxkZ2c3yfA/QGhoKF9//TWPPvooH3/8sbV83LhxpKWl4eDgQEBAAO+88w4ODg5NEqNSSl0rWxaVfA/sMgzjVcMw/kj1HoRHDMOYZRjGrBv47ADgexHJFJEyII7qBSzqNnfgwAE2bdpEQUEBd911F1OmTKGwsJCZM2cSHR2NiODt7Y3FYsFisXDw4EG2bt1qrV8zPGtvb09FRUWdnyEirF+/3trGxUlcTf2rtXG1duri5uaGj48Pqamp1rL9+/dTUVGBn5+fbR10m2vKBUIAs2fPrjN5HzduHIcPH+bgwYMUFxfz3nvvNUF0Sil1fWxJCH8ANgBy4Xki1aeWtL1wXa+uwImLnp+8UFaLYRjPGIaRZhhGWm5u7g18nGoutm/fTnl5OVC9EtfBwQFvb2/MZjP79u3Dy8uL3Nxcdu7cCUB5eTmHDh2qt822bdty7tx/bnSHhISwfPnymqkLpKen11u/f//+pKSkcPToUQB++eUXm9s5efIkxcXFAOTl5fHFF1/g5eVlff3DDz/kN7/5Tb2f3xK0hPmgUH0HuW3by3/0jRgxAsMwMAyDgIAATp48eVP7TymlGpItq4wXNUYg9Xz+SmAlgL+/v1zl7aoFKCgosD7++eef2bZtG4ZhYG9vT2JiIo6OjiQkJPDcc89RUFBARUUFUVFReHt7X7HNwYMHs2TJEsxmM/PmzWPBggVERUVhMpmoqqrCw8Oj1vDfpTp16sTKlSuJiIigqqqKzp07s23bNpva+fbbb3nhhRcwDAMR4cUXX+TBBx+0vr5u3bomvePVWJKTk3FzcyMpKQmo/v9cXl7OjBkzSExMpFOnTsTHxzN//nxWr14NQEVFBbt372bz5s0sWrSITz/9lOjoaNLS0lixYgUAL7/8MkOGDGH16tXk5+cTEBDAY489BlTfTUxPT8fJyQkvLy9mzJiBs7MzU6ZMISUlBQ8PD2tyv3jx4jrbadOmzTV9neXl5axdu5Y333zzZnWdUko1uCsmhIZhvCEiUYZhbOI/dwetROTxG/zsH4FuFz13v1CmbnMuLi7WpPCBBx7ggQcesJb7+1cfkmM2m0lJSbms7ueff2593LFjR+scwg4dOrBnz55a733nnXcuqz9hwgQmTJhgfX5xcjd8+HCGDx9e6/2tW7eus52LDR06lAMHDlzx9czMzHrrN3dJmUm8ue9Njv9wnKzELPLs8nj+t8832/mgV/Pss88ycOBAXTGulGpW6rtDWDNJZmkDffYewNMwDA+qE8FfA0810GepZiQ4OJhNmzZZh40BHBwcrEN7qvlIykzi1a9epaSyBMcujtz7x3vZl7GPZ198lv8K+y/Cw8Px9va2Dv9f6lrmg148DA+wa9eu65oPemk712LRokXk5uZe9Y8EpZS61VxxDqGI7L3wMA1IFZF/ici/gC+oTuZuiIhUAH8AtgDfAutEpP6JYOq2YDKZGDVqlPXOjouLC6NGjcJkMjVxZOpavbnvTUoqSwAozyvHztGONv3b0GpIq2Y5H7Q+7733Hlu2bOHDDz/Ezs6W6dlKKXXrsGXbme3AY0Dhheetga3Awzf64SKyGWj5k6fUNTOZTJoAtgA/Ff1kfVxysoSf4n+qXnjRyuDdde82u/mgAIGBgRw+fJjCwkLc3d1ZtWoVISEhTJ06lXvuuYcBAwYA1cPdCxcuvIHeU0qpxmPU/DV8xTcYhkVEzFcrawz+/v6SlpbW2B+rlLpOwxKGcaro1GXlrm1c2Tp2ax01lFKq5TAMY6+I+Dd1HLawZVyjyDCMvjVPDMPwA4obLiSlVEvxfN/ncbZ3rlXmbO/M832fb6KIlFJK1cWWIeMo4CPDMLIBA+gCRDZoVEqpFiHsvjCgei7hT0U/0aVNF57v+7y1XCml1K3hqkPGAIZhOAA1S+++E5Hy+t7fUHTIWCmllFLNRXMaMrblDiHAQ8C9F97f98IGu+83WFRKKaWUUqrRXHUOoWEYa6nei/BRqhPDh4Bmke0qpZRqPE19znRoaCjt27dn5MiRtcqffvppfH19MZlMjB07lsLCwiu0oNTty5ZFJf7AIyLyrIjMuHA919CBKaWUal6aOiGcPXs2a9euvax82bJl7N+/nwMHDtC9e3frsYdKqf+wJSHMoHohiVJKqRaoqKiIsLAwfH198fHxIT4+HoC9e/cSFBSEn58fISEhnDpVvYXQoEGDmDt3LgEBAfTo0YPU1FTKyspYuHAh8fHxmM1m4uPjKSoqYtKkSQQEBNCnTx8SExMBiI2NJSIigtDQUDw9PZkzZ441luTkZPr27Yuvr6/1dKIrtXOp4OBg2rZte1l5u3btgOrTaIqLizEM4+Z1nlIthC1zCDsC3xiGsRsorSm8CWcZK6WUugUkJyfj5uZGUlISAAUFBZSXlzNjxgwSExPp1KkT8fHxzJ8/n9WrVwNQUVHB7t272bx5M4sWLeLTTz8lOjqatLQ06x24l19+mSFDhrB69Wry8/MJCAjgscceA6rvJqanp+Pk5ISXlxczZszA2dmZKVOmkJKSgoeHh/UUmcWLF9fZTps2bWz+GidOnMjmzZvp3bs3//M//3Mzu0+pFsGWhPDVhg5CqZbMYrGQnZ3NiBEjGv2zjx8/Tnh4OFVVVdZf8FOnTgWq7/5MmDCB4uJiRowYwZtvvql3Tm4zBZs2kbPsDe44foxPsrOJ+uUXnoiKIjAwkIyMDDIyMhg6dCgAlZWVuLq6WutGREQA4Ofnx7Fjx+psf+vWrWzcuJGlS5cCUFJSQlZWFlB9N6/meMrevXtz/Phx8vLyGDhwIB4eHgB06NCh3nZ69epl89caExNDZWUlM2bMID4+nokTJ9pcV6nbwVUTwgvnFyulrpPFYiEtLa1JEkJXV1d27tyJk5MThYWF+Pj48Pjjj+Pm5sa0adN499136devHyNGjCA5OZnhw4c3eoyqaRRs2sSpBQuRkhLudXDkI/dupKal8dK0aYQ8+STh4eF4e3tbz5m+lJOTEwD29vZUVFTU+R4RYf369Xh5edUq37Vrl7X+1dqor51rZW9vz69//Wv+8pe/aEKo1CWuOIfQMIwvLvz3nGEYZy+6zhmGcbbxQlSqabSEeVWOjo7WX7ylpaVUVVUBcOrUKc6ePUv//v0xDIPx48ezYcOGhutMdcvJWfYGUlJS/biinNaGwajWd/B7O3v27duHl5cXubm51oSwvLycQ4cO1dtm27ZtOXfunPV5SEgIy5cvp2a/2/T09Hrr9+/fn5SUFI4ePQpgHTK+1nYuJiJ8//331scbN26kZ8+eNtdX6rYhIs3m8vPzE6UaS0JCgkyePNn6PD8/X8rKymTAgAGSk5MjIiJxcXEyceJEEREJCgqSWbNmiYhIUlKSBAcHi4hITEyMTJ8+3drOvHnzZO3atSIikpeXJ56enlJYWCgxMTHi4eEh+fn5UlxcLN27d5esrCzJyckRd3d3yczMFBGRM2fO1NvOpbKysuTBBx+U1q1by4oVK0REZM+ePdb4RERSUlIkLCzsJvSaai6+6dlLvvHqKd949ZSV7u7Sw9FJvJycxMfZWfbs2SMiIunp6RIYGCgmk0l69+4tK1euFJHq7/Wa9+Tm5so999wjItXfm/7+/uLr6ytxcXFy/vx5eeaZZ8THx0d69+5t/R679N9EWFiY7NixQ0RENm/eLGazWTw9PcVsNouIXLGdSz366KPSsWNHcXZ2lq5du0pycrJUVlbKww8/LD5+uGFgAAAgAElEQVQ+PuLt7S1PPfWUFBQUXLV/QkJCxMXF5bLPevTRR8XX11d8fX3F1dVVRo8ebWOPq9sRkCa3QP5ky1XvkLFhGPbAIRHRP6fUbWND+o+8vuU7jmee4XTCJs6UP8vMp3/TbOdVdevWjQMHDpCdnc2YMWMYO3bszegm1cy1cnWlIjsbgEfb3MmjHndWl7u54elfvdWs2WwmJSXlsrqff/659XHHjh2t3+sdOnRgz549td77zjvvXFZ/woQJTJgwwfr8448/tj4ePnw4w4cPJzY2lpqTqVq3bl1nO5dKTU2ts/zLL7+8at1LzZ49m/Pnz1/2uRd/xhNPPMHo0aOvuW2lbkX1bjsjIpXAd4ZhdG+keJRqUhvSf2TePw7yY34xrTp0pdP4N/g6/06mRs0mOjoaEcHb2xuLxYLFYuHgwYNs3brVWv9a5lXVtHFxEnc986rqaqcubm5u+Pj4kJqaSteuXTl58qT1tZMnT9K1a1fbOkm1CJ1nRmE4O9cqM5yd6Twz6obabQlTLeDKW9jUOHv2LJ999hljxoy5of5S6lZhyz6EvwIOGYax3TCMjTVXQwemVFN4fct3FJdXAlBx7gx2Dk449gyiymdUs5xXdfLkSYqLiwHIy8vjiy++wMvLC1dXV9q1a8fXX3+NiPD+++/rnY7bjMuoUbj+dzSt3NzAMGjl5obrf0fjMmrUDbVbs4XN/v37ycjIIDQ01LrCPSEhgb179zJp0iTmz59vrVOzhc0bb7zBokWLcHR0JDo6msjISCwWC5GRkdatZ3bv3s2OHTuYPXs2RUVFQPXCrfj4eA4ePEh8fDwnTpwgNzeXKVOmsH79evbv389HH30EUG8712LDhg0EBwdb9zhUqrmzZduZBQ0ehVK3iOz8Yuvj8txj5HweA4aBYdeKtZv+H46OjiQkJPDcc89RUFBARUUFUVFReHt7X7HNwYMHs2TJEsxmM/PmzWPBggVERUVhMpmoqqrCw8Oj1pDZpTp16sTKlSuJiIigqqqKzp07s23bNpva+fbbb3nhhRe4cP44L774Ig8++CAAb731lnXbmZphOnV7cRk16oYTwBoHDhxg+/btZGZmsn79eioqKpg0aVKznWpxNR9++CGTJ0++pjpK3cqumBAahuEMTAUeAA4Cq0TkyuNXSrUAbu1b8+OFpLD1fX60vs8PgK7tW+N/C8yrupgt86qGDh3KgQMH6nzN39+fjIyMeusrZYsDBw6wadMmysvLueuuu5gyZQqZmZnMnDmTxx9/vMVtYXP69Gl2797NP//5z+tuQ6lbTX1DxmuoPsf4IDAc0K3dVYs3O8SL1g72tcpaO9gzO+TG9j9TqiXbvn075eXlAJw7dw4HBwe8vb0xm83NcqrF1SQkJDBy5EicL5mDqVRzVt+QcW8ReRDAMIxVwO7GCUmppjOmT/XCite3fEd2fjFu7VszO8TLWq6UulxBQYH18c8//8y2bdswDAN7e3sSExOb3VQLgMDAQA4fPkxhYSHu7u6sWrWKkJAQAOLi4njppZduoMeUuvUYNX8lXfaCYewTkb5Xet4U/P39pWYbAqWUUreGZcuW1UoKa7i4uDBz5swmiEipW4NhGHtFxL+p47BFfUPGvhefTgKY9KQSpZRSlwoODsbBwaFWmYODg3WrF6XUre+KQ8YiYn+l15RSSqkaJpMJqJ5LWFBQgIuLC8HBwdZypdStz5ZtZ5RSSql6mUwmTQCVasZs2ZhaKaWUUkq1YJoQKqWUUkrd5jQhVEoppW4zFouFzZs3N2kMZ8+exd3dnT/84Q/WstDQUHx9ffH29mbq1KlUVlY2YYS3F00IlVJKqdvMrZAQLliwgIEDB9YqW7dunfUc7NzcXOsZ1KrhaUKolFJKNRNFRUWEhYXh6+uLj48P8fHxAOzdu5egoCD8/PwICQnh1KlTAAwaNIi5c+cSEBBAjx49SE1NpaysjIULFxIfH4/ZbCY+Pp6ioiImTZpEQEAAffr0ITExEYDY2FgiIiIIDQ3F09OTOXPmWGNJTk6mb9+++Pr6WrcYulI7l9q7dy8///wzw4YNq1Xerl07ACoqKigrK8MwjJvbgerKRKTZXH5+fqKUUkrdrhISEmTy5MnW5/n5+VJWViYDBgyQnJwcERGJi4uTiRMniohIUFCQzJo1S0REkpKSJDg4WEREYmJiZPr06dZ25s2bJ2vXrhURkby8PPH09JTCwkKJiYkRDw8Pyc/Pl+LiYunevbtkZWVJTk6OuLu7S2ZmpoiInDlzpt52LlZZWSlBQUFy4sSJy+IQERk2bJi0b99efvOb30hFRcXN6bgmAqTJLZA/2XLpHUKllFK3haYcJrVYLAwYMABvb29MJpP1zh7AihUreOCBBzAMg9OnT9dZv2DTJv49JJg75szhk//9X6KeeILU1FRcXFz47rvvyMjIYOjQoZjNZl577TVOnjxprRsREQGAn58fx44dq7P9rVu3Wo8KHDRoECUlJWRlZQHVG4+7uLjg7OxM7969OX78OF9//TUDBw7Ew8MDgA4dOly1nRpvvfUWI0aMwN3dvc5YtmzZwqlTpygtLeWzzz6zoXfVzaD7ECqllLotWCwW0tLSGDFiRKN/9h133MH777+Pp6cn2dnZ1qHd9u3b88gjjzBy5EgGDRpUZ92CTZs4tWAhUlLCvQ6OfOTejdS0NF6aNo2QJ58kPDwcb29vdu7cWWd9JycnAOzt7amoqKjzPSLC+vXr8fLyqlW+a9cua/2rtVFfOxfbuXMnqampvPXWWxQWFlJWVsadd97JkiVLrO9xdnZm9OjRJCYmMnTo0Cu2pW4evUOolFLqltYS5s316NEDT09PANzc3OjcuTO5ubkA9OnTh3vvvfeKX3/OsjeQkpLqxxXltDYMRrW+g9/b2bNv3z68vLzIzc21JoTl5eUcOnSo3j5t27Yt586dsz4PCQlh+fLlVI9yQnp6er31+/fvT0pKCkePHgXgl19+sbmdDz74gKysLI4dO8bSpUsZP348S5YsobCw0Pr/sKKigqSkJHr27FlvHOrm0YRQKXVLaOpVj6GhobRv356RI0fWKhcR5s+fT48ePejVqxd/+9vfmijC21dycjJubm7W1aehoaGUl5czY8YMEhIS2Lt3L5MmTWL+/PnWOhUVFezevZs33niDRYsW4ejoSHR0NJGRkVgsFiIjI1m8eDFDhgxh9+7d7Nixg9mzZ1NUVARUfz/Gx8dz8OBB4uPjOXHiBLm5uUyZMoX169ezf/9+6wrY+tqpy+7duykrK+P++++36euvuJAkARwpLSXy+HHCjx1l+b+P8Morr+Do6EhCQgJz587F19cXs9nMV199VW+bgwcP5ptvvrEmxwsWLKC8vByTyYS3tzcLFiyot36nTp1YuXIlERER+Pr6EhkZCXDN7VysqKiIxx9/HJPJhNlspnPnzkydOtXm+urGGDVZfHPg7+8vaWlpTR2GUqoBxMbGkpaWxooVK5rk87dv38758+d55513+Pjjj63lMTEx7Nixg9jYWOzs7MjJyaFz585NEuNt58A62B7NkczjDPuglMjRwxk56UUCAwPJyMjg4Ycf5r777gOgsrISV1dXtm7dyqBBg1i8eDGPPPIIP//8M4888gjff//9Zd9j/v7+lJSU0KpV9eypX375hS1btrBr1y6+/PJL3n33XQCGDx/O/PnzycvLIy4ujg8++KBWmFdqp1evXpd9SadOnWLQoEGsWbOG/v3713rt3nvvJS0tjY4dO9Yq//eQYCqysy9rq5WbG56fbb+enlWNxDCMvSLi39Rx2ELvECqlbkhLGM6D6onzbdu2vaz873//OwsXLsTOrvrHpSaDjeTAOtj0HBScoMddduyb4syDhZ/zysz/Q3R0NCKCt7c3FosFi8XCwYMH2bp1q7X6tcybq2kjKyvLmsRdz7y5utq52NmzZwkLC2Px4sWXJYP16TwzCsPZuVaZ4exM55lRNreh1NVoQthAbtXVbBMmTMDDwwOz2YzZbMZisTRJjKrlaGnDeZf64YcfiI+Px9/fn+HDh/Pvf//75nagqtv2aCgvBiD7XBV3OMBvvWG2uahZzpsrKysjPDyc8ePHM3bsWFt6wMpl1Chc/zuaVm5uYBi0cnPD9b+jcRk16praUao+usq4gdyqq9kAXn/99Wv+gaTUpYrSczi75RidMktJTkhiVvkMwp9+0jqcV7MNBvxnOK+GrdtgbNy4kaVLlwLUuQ0GYN0GIy8v74rbYNTVTl13cOpSWlqKs7MzaWlp/OMf/2DSpEmkpqZeS1ep61Hwn21TDv5cxextJdgZ4GBfxN8T11vnzT333HMUFBRQUVFBVFQU3t7eV2xy8ODB1i1R5s2bx4IFC4iKisJkMlFVVYWHh0et6QKXunjeXFVVFZ07d2bbtm02tbNu3TpSUlI4c+YMsbGxQPXdbrPZzN/+9jf+8pe/8NNPP2EymRgxYgTvvfderfouo0ZpAqgaVlNvhHgtV2NsTF1YWCgjRowQk8kk3t7eEhcXJyIiaWlpMnDgQOnbt68MGzZMsrOzRaR60885c+bIQw89JJ6enpKSkiKlpaXSrVs36dixo/j6+kpcXJwUFhbKxIkT5aGHHhKz2SwbNmwQkerNQcPDwyUkJEQeeOABmT17tjWWTz75RPr06SMmk0mGDBlija+udupjMpnkyJEjIiLy+9//Xj766KOb2mfq9lO472c5+coXcmJuipyYmyIHnvtY3hy9QB7pO0AWLVokBw4ckP79+9dZNygoSPbs2SMiIrm5uXLPPfeIyOUb5fbt21cOHz58Wf1L3xcWFiY7duyQjRs3ylNPPXXZ+6/UTl127NghYWFhtcq8vLysm+9WVVVJu3btbGpL3aC/eov8sd3l11+9mzoypWyGbkzdfLW04a+6VrPNnz8fk8nEzJkzKS0tbaCebBy36tD89u3b6du3L2azmUcffZTvv/++SWJsKGe3HEPKqwD46dxpWjs4EdFzKM/4jG2Ww3n1GTNmDDt27ADgX//6Fz169Lim+uo6BS8Eh9a1yxxaV5crpW46HTK+4NvUHaTGvU/msWP8M3UPUlTI76c926yHv06dOsXvfvc71qxZY50Q/6c//YkuXbpQVlbGM888w5///GcWLmy+P2Bv1aH5adOmkZiYSK9evXjrrbd47bXXrMNELUFl/n/+kDicm8niz9/CzrCjlV0rVm1a2+yG8wACAwM5fPgwhYWFuLu7s2rVKkJCQnjppZcYN24cy5Yt484777xsKE81ENOT1f/dHl09fOziXp0M1pQrpW4qTQipTga3rlxBRVkpne5sw/NDHubI8e+ZNeMPjIp4otntAg9XXs1Wk8g6OTkxceJEa2LZ2IqKinjyySc5efIklZWVLFiwgMjISPbu3cusWbMoLCykY8eOxMbG4urqyqBBg+jXrx87duwgPz+fVatW0a9fPxYuXEhxcTFffPEF8+bNY+TIkcyYMYOMjAzKy8t59dVXGT16NLGxsWzcuJHz58/zww8/EB4ezl/+8heg+q7wyy+/TGVlJR07dmT79u0UFRXV2c7FLr5TdPFGs+3bt8cwDM6ePQtAQUEBbm5ujde5jcC+vZM1KRx0XwCD7guwlrv6V++wYDabSUlJuazu559/bn3csWNH6x9RHTp0YM+ePbXe+84771xWf8KECUyYMMH6/OLkbvjw4QwfPrzW+1u3bl1nO5e60rzA9u3bk5SUdNX6qgGYntQEUKlGokPGQGrc+1SUVf9yKyguwaGVPWb3u+nXtVOzHP6qbzVbzdYfIsKGDRvw8fGpN46G0tKH5t977z3rWZ1r167lpZdeasDebHztQu7FcKj948NwsKNdyL1NE5BSSqkboncIgXNn/nOY+E8F5/h4/7cYhoGdYfDPrZ82u+Gv+lazjRs3jtzcXEQEs9nM22+/fWOdd41O/ZRI5g9LKSw8zscf5zD9D2f4deTMFjc0v2zZMjZv3ky/fv14/fXXmTVrVosaamzTp3ovvrNbjlGZX4p9eyfahdxrLVdKKdW8aEIItL2rI+dOV58p6dWlE15dOlWXd+yEfzMc/vrtb3/Lb3/72zpf++yzz+qt25BO/ZTI4cPzqaoqxr2bA2/9vQt79uxmzpxpDB/e/A5oh7qH5nNzc9m/fz/9+vUDIDIyktDQ0HrbaY7a9OmsCaBSSrUQOmQMBP56PK0cnWqVtXJ0IvDX45soopYp84elVFVVbzR7+nQFzs4GwcFOhIdLixqa/9WvfkVBQQFHjhwBYNu2bTbveaeUUko1Bb1DCPQKHAxUzyU8d+Y0be/qSOCvx1vL1c1RUvqfA9qPHi1j5Tu/YGcH9q0MPvjfNS1qaP7dd9/liSeewM7Ojl/96lesXr36xjpPKaWUakBGzR2Q5sDf31/S0tKaOgx1nb78MpCS0ssPaHd2cuORR/TkB6WUUi2LYRh7RcS/qeOwhQ4Zq0Zz3/0vYmdXe6NZO7vW3Hf/i00UkVJKNY6m3EQfIDQ0lPbt2zNy5Mha5S19E31lO00IVaNx7TKanj0X4+zkBhg4O7nRs+diXLuMvmpdpZRqzpo6IZw9ezZr1669rHzatGl88MEHWCwWnnrqKV577bUmiE7dCjQhVI3KtctoHnkkleAh3/PII6maDN4CbtXj/wIDAzGbzZjNZtzc3BgzZkyTxKhub0VFRYSFheHr64uPj4/1e3Tv3r0EBQVZTyiq2eN10KBBzJ07l4CAAHr06EFqaiplZWUsXLiQ+Ph4zGYz8fHxFBUVMWnSJAICAujTpw+JiYlA9TzkiIgIQkND8fT0ZM6cOdZYkpOT6du3L76+vgQHB1vjq6udSwUHB9O2bdvLylv6JvrqGjT1YcrXcvn5+V3lGGml1LWKiYmR6dOnN8lnf/fdd3LkyBEREfnxxx+lS5cukpeXd9n7IiIiZM2aNY0dnlKSkJAgkydPtj7Pz8+XsrIyGTBggOTk5IiISFxcnEycOFFERIKCgmTWrFkiIpKUlCTBwcEicvm/s3nz5snatWtFRCQvL088PT2lsLBQYmJixMPDQ/Lz86W4uFi6d+8uWVlZkpOTI+7u7pKZmSkiImfOnKm3nbrs2LFDwsLCapWlpKRIhw4dpGvXrtKrVy8pKCi4sQ5TtQBpcgvkT7ZceodQqWaqJdy56NGjB56enkDt4/8udvbsWT777DO9Q6ga1amfEvnyy0AKC2fy8cdrmf6HCFJTU3FxceG7776zbqJvNpt57bXXOHnypLWurZvo1+yQMGjQoDo30Xd2drZuov/1119fcRP9K7Vji5pN9E+ePMnEiROZNWvW9XSXagF02xmlmqma4/9qztktKCiwHv+XmJhIp06diI+PZ/78+dZtb2qO/9u8eTOLFi3i008/JTo6mrS0NFasWAHAyy+/zJAhQ1i9ejX5+fkEBATw2GOPAdVDvOnp6Tg5OeHl5cWMGTNwdnZmypQppKSk4OHhYd3Lseb4v0vbadOmTZ1fz6XH/9XYsGEDwcHBtGvXrkH6UalLtcRN9Otyu2yir2yjdwiVamYOHDjAsmXL+Pzzz1m/fj0TJ05s9ncuao7/i4mJsR7/V+PDDz/kN7/5zQ31mVLXoqVton8luom+upjeIVSqGTlw4ACbNm2ivLycu+66iylTppCZmcnMmTN5/PHHm+Wdi7qO/6tx+vRpdu/ezT//+c9621DqZmppm+hD9SKtw4cPU1hYiLu7O6tWrSIkJEQ30VdWujG1Us3IsmXLKCgoAODcuXO0bt2aVq1acfLkSU6fPs26devo3bs3a9euZcCAAZSXl3PkyBG8vb0ZNGgQS5cuxd/fn9OnT+Pv78+xY8dYv349GzduZM2aNUD1kPHZs2dZvnw5hmGQnp5Onz59iI2NrTW0PHLkSF588UW8vb3p27dvrSHjDh06XLGdi5WVlTF8+HBGjRpFVFTUZV/v22+/zc6dO62xNQcWi4Xs7GxGjBjR6J99/PhxwsPDqaqqsk4fmDp1KgDz58/n/fffJy8vj8LCwkaPrTnRTfTVzaIbUyulGkRNMgjw888/8+677/L222/zySef8Morr1jvXMydOxdfX1/MZjNfffVVvW0OHjyYb775xrqoZMGCBZSXl2MymfD29mbBggX11r/4zoWvry+RkZEANrVTc/xfzZF/ZrMZi8VifT0uLq7ZDRc35TY+rq6u7Ny5E4vFwq5du1iyZAnZ2dWJzahRo9i9e3eTxNXc3OxN9Jt6D8K5c+fi4+NTa/GZUpfSO4RKNSMX3yG8mIuLCzNnzmyCiFqOoqIinnzySU6ePEllZSULFiwgMjKSvXv3MmvWLAoLC+nYsSOxsbG4uroyaNAg+vXrx44dO8jPz2fVqlX069ePBx54gOLiYrp27cq8efMYOXIkM2bMICMjg/Lycl599VVGjx5NbGwsGzdu5Pz58/zwww+Eh4fzl7/8BaheMPTyyy9TWVlJx44d2b59O0VFRXW2cyVnzpyhT58+fP3117X2lrvzzjv1DqENTv2USOYPSykpPYWzkyv33f/ide+beund9caUlJTEG2+8wSeffEJpaSmDBg1i+/btukirkegdQqVUgwgODsbBwaFWmYODg3WrF3X9alZt79+/n4yMDEJDQ63DrgkJCezdu5dJkyYxf/58a52aVdtvvPEGixYtwtHRkejoaCIjI7FYLERGRlpXW+/evZsdO3Ywe/ZsioqKgOo7R/Hx8Rw8eJD4+HhOnDhBbm4uU6ZMYf369ezfv5+PPvoIoN52LnbixAlMJhPdunVj7ty5utHwdXLtMhqzOZm//o8Xzz57lqGPzW+WWzt98803DBw4kFatWtGmTRtMJhPJyckN3X2qGdJFJUo1IyaTCag+f7SgoAAXFxeCg4Ot5eo6HFgH26N5MPM4L6wvZW5FDiMnvUhgYCAZGRnWVdsAlZWVuLq6Wqvaump748aNLF26FKDOVduAddV2Xl7eFVdt19XOpatCu3XrxoEDB8jOzmbMmDGMHTuWu++++2b01G2nJWzt5Ovry6JFi3jhhRc4f/48O3bsoHfv3o3ZjaqZ0IRQqWbGZDJpAnizHFgHm56D8mJ63GXHvinObM78nFdm/pvgx3/dLFdt13Bzc8PHx4fU1FTGjh1rUx1Vbf1Pv/CnzFMcL2rF2Y83k/uH53gh8r+a5R8Jw4YNY8+ePTz88MN06tSJAQMGYG9vf7O6SrUgOmSslLrpbtXzkY8ePWqd5xcZGUnZllehvHq/uexzVdzhAL/1htnmoma539zJkycpLq7+evLy8vjiiy+ua8Pi29n6n37hxe9OcLK0HPtu9+Dy9v/jS5cuTJ3zEtHR0YgI3t7eWCwWLBYLBw8eZOvWrdb61/JHQk0bFydx1/NHQl3tXGz+/PlYLBa2bduGiNCjR4/r6hvVsmlCqJS66ZoyIbzjjjt4//33OXToEMnJyURFRZGfnw9Ur7acOXMm33//Pb/61a9Y9flRa72DP1cR8G4R5rcLWfTJj81y1fa3335Lv3798PX1JSgoiBdffJEHH3wQgDlz5uDu7s758+dxd3fn1VdfvZZuvW38KfMUxVXVSXfl6RwMZ2fsHxtBacRTzfKPhMrKSs6cOQNU72N64MABhg0bdtV+ULehpj5M+VouPz+/Kx4grZS6cYWFhTJixAgxmUzi7e0tcXFxIiKSlpYmAwcOlL59+8qwYcMkOztbRESCgoJkzpw58tBDD4mnp6ekpKRIaWmpdOvWTTp27Ci+vr4SFxcnhYWFMnHiRHnooYfEbDbLhg0bREQkJiZGwsPDJSQkRB544AGZPXu2NZZPPvlE+vTpIyaTSYYMGWKNr6526mMymeTIkSNSVVUld911l5SXl4uIyFdffSXDvNqI/LHd5ddfvW9qv6rmo8tn6XL3hav9n/+vtLrPU1rd30NaefWWPXv2iIhIenq6BAYGislkkt69e8vKlStFpPrfQ817cnNz5Z577hERkTNnzoi/v7/138P58+flmWeeER8fH+ndu7eEhYWJSPW/h+nTp1tjCQsLkx07doiIyObNm8VsNovJZJLHHntMROSK7VysuLhYevXqJb169ZJ+/fpJenp6g/SbqhuQJrdA/mTL1eQBXMulCaFSDSshIUEmT55sfZ6fny9lZWUyYMAAycnJERGRuLg4mThxoohU/wKcNWuWiIgkJSVJcHCwiFz+i23evHmydu1aERHJy8sTT09PKSwslJiYGPHw8JD8/HwpLi6W7t27S1ZWluTk5Ii7u7tkZmaKSPUv1PrauZJdu3ZJz549pbKyUnJzc+X++++3vpaVlSXe97uLvHZ37WTwtbtF9sffWEeqZsvvywxrQnjx5fdlRlOHppqh5pQQ6qISpRQb0n/k9S3fcTzzDKcTNnGm/FlmPv2bZjmJvkbN+chr1qy57HxkK2cXGLUQtkdDwUlwcYfghWB68lq6T7Ug8+5z5cXvTliHjQFa2xnMu8+1nlpKNX+aECp1m9uQ/iPz/nGQ4vJKWnXoSqfxb/D18X1MjZpN5OgRzXKlbV3nI991113k5+dTUVFhPe6va9eu1cmfJoDqgie6VP/x8afMU/xYWk5XJwfm3edqLVeqpdJFJUrd5l7f8h3F5ZUAVJw7g52DE449g6jyGdUsJ9GXlZURHh7O+PHja223YhgGgwcPJiEhAYA1a9bUe9KHun090aUDaQ97c2qwmbSHvTUZVLcFTQiVus1l5xdbH5fnHuPU+7PIjplB5tY1zXKlbX3nI//5z3/mr3/9Kw888ABnzpzh6aefvtbuUkqpFknPMlbqNvfIks/48aKksEbX9q358qUhTRCRUs2XxWIhOzubESNGNMnnz5kzh6SkJKqqqhg6dChvvvkmhmE0SSxKzzK+KsMw/sswjEOGYVQZhtEsOkqpGrfqpstPP/00vr6+mEwmxo4dS+H/3969h1dV3fkff3+5JEShWI/TcAkAACAASURBVEYoCbcGSKIkJIGkEWS4Bg2ICkQrnfZXq1YdrQKpKEyg0OqUkVamnRkZL3QKsWprWlDDAIMgVYk0AgEiQkXlLhIMcjUh5HbW748cToNNQlCSQ7I/r+fJ05y9z17nuzbt6Sdrr712cXGD2nwkLYawtuc+uSCsbWseSdOCxiIXKpjfD3/5y19Yv34927ZtY/v27WzatIm33norKLVI8xOsS8bbgXRgXZA+X+RLu1QXXf71r3/Nu+++y7Zt2+jZs2fguannM2FANx5P70+3K8IwqkcGH0/vz4QB3RqxJyKXlpKSEsaNG0dCQgJxcXGBP7Y2b97M8OHDSUpKIi0tjcLCQgBGjBjBjBkzSElJITo6mtzcXMrLy5kzZw7Z2dmB6RIlJSXcddddpKSkMGDAAHJycgDIysoiPT2dMWPGEBUVxfTp0wO1rFq1ioEDB5KQkEBqamqgvtraqcnMOHPmDOXl5ZSVlVFRUaHnWEvDBXPNG+BNILmh79c6hPJVtORFl2vy+Xzuvvvuc/Pmzbso503EC1rKGpzTpk1zHTt2dF/72tfczJkzL+o5kguH1iG8eMzsXuBegJ49ewa5GmnOVq1aRUREBCtWrADg5MmTVFRUMHnyZHJycujcuTPZ2dnMmjWLRYsWAVBZWcnGjRtZuXIljz76KK+//jqPPfYY+fn5gRG4mTNnMmrUKBYtWsSJEydISUlh9OjRQPVo4tatWwkNDSUmJobJkyfTrl077rnnHtatW0dkZGTgDtq5c+fW2s7ll19ea382btxIeXk5ffr0CWy78847WblyJf369ePf//3fG+1cirQUhYdz2LN7PsXF+1m+vIgHHjzKdyb9uFmuwblr1y7ef/99Dh48CMB1111Hbm4uQ4cOvSjnSlq2RguEZvY60LWWXbOcc38/1l0H59xCYCFU31RykcqTJhDMydUFBQXcf//9nDp1CkpKuOfy9sScKeX/Dh3ilp07ee/TTwkJCaFPnz689957zeYL/6y6Fl1evHgxVVVVTJ48mezsbO68884vfQ5FWrrCwzns3DkLn6+U7j3a8tTTXdm0aSPTp9/P2LG3Nbs1OF955RUGDRpE+/btARg7dix5eXkKhNIgjTaH0Dk32jkXV8tPg8OgNG+Xwly7v8ybx9OXt2fuBzvp1Ko12d26szYvj/bOcdtttxEeHk6XLl0oKCigoKCA9957j9WrVwfauZAv/LNt1AxxX+YLv7Z2aqpt0eWaWrduzXe+8x2WLl3asJMl4lF7ds/H56u+w/6zzypp185ITQ1l4kTXLNfg7NmzJ2+99RaVlZVUVFTw1ltv1fodIlIbrUMof6clTK6Ojo4mKiqKol//B52rqviHNm3YVXaGMp+Pr7VqxX1hl7FlyxZuu+02Dh8+3Gy+8OtadNk5x65duwK/L1u2jKuuuqreOkS87kxZYeD3vXvLeeBHh/jnew/y20W7m+UanLfeeit9+vShf//+JCQkkJCQwE033fQlzox4UjAmLgITgYNAGfAp8FpDjtNNJU2jpUyuds65v151tXupZy/XOyTEPdOtm4tqG+LagOsTEuI2bdrkpkyZ4vr27euGDh3q4uPjXb9+/dzChQsD/dq0aZNzzrkjR464Xr16BepITk4O3FRy+vRpd++997q4uDjXr18/N27cuFr7P27cOPfGG28455xbuXKlS0xMdPHx8W706NHOOVdnOzU9//zzrk2bNi4hISHws3XrVldVVeWuvfZaFxcX52JjY913v/tdd/LkyfP8S4t429tv/6N7fW3vv/t5++1/DHZp0kLQjG4q0cLUErD08DEe31PI/l27ODXjR4y95RamTfp2YHL1tddeS+/evYG/zbVbvXo1I0aMYO7cuQwZMoRPP/2UIUOGsGvXLrKyss65+SI5OZkzZ87Qpk311NVjx47x2muvsWHDBtavX89vfvMboHrey6xZszh+/DgvvfQSL7744jl11tVObZdG1g/5R/7fpk08Hh5OQlgYAAWlpfz7yRO0io7m+uuvZ/ny5YEnWYiId9ScQ3hWq1ZhXHXVXMK76rGG8tU1p4WpL/m7jKVpLD18jIc/+JhSn6N1j150fOb3rN+4no+m/wuTxqY1u8nVUD3X7keHC8mIiCChXbvA9gFf/zprFzxJx5tuYvXq1Xz44Yf1tiMiLdPZ0Ldn93zOlBXSLjSc3n0eVhgUT9IcQgHg8T2FlPqqR4urPivC2rWj9egbKEv/brOcXH12rt2dkydzx5P/RZuICDCjTUQEbaY9RMebbqKsrIxf/OIX3HfffQ05RSLSAoV3Hc+QIbmkjtrFkCG5CoPiWQqEAsAnZRWB3yv37uLYj77P0Xsmsf+3TzXLydV//OMfWbduHVlZWQyfPZtvHztK2R9+T9Sf1/Lstm1cffXVxMfHc9NNNzFqlJ7X62XBvBseYMyYMVxxxRXceOON52z/85//zMCBA4mLi+MHP/hBvaPmIiJfleYQCgDJf9nBwRqh8KzuoW3JvzY2CBWJNI0vznVtamvXruX06dM8++yzLF++HACfz0evXr1Yu3Yt0dHRzJkzh169evHDH/4wKDWKyJfTnOYQaoRQAMjsHU5YKztnW1grI7N3eB1HiARXS1geCaoXL+/QocM5244ePUpISAjR0dFA9RMntK6kiDQm3VQiANzStfppGY/vKeSTsgq6hbYls3d4YLvIpaalPYqwpiuvvJLKykry8/NJTk5myZIlfPzxx41xGkVEAAVCqeGWrp0UAOWSF1geqaQNp5av5MiDU85ZHqk5PXu2LmbGSy+9xI9//GPKysq4/vrrad269YWeKhGRBlMgFJFmoyUuj1SXwYMHk5ubC6DlkUSk0WkOoccF8w7LgoICBg8eTGxsLPHx8YE5YDVNmTIl8KB2kZa2PFJ9ioqKALQ8kog0CQVCjwtmILzsssv43e9+x44dO1i1ahUZGRmcOHEisD8/P5/jx48HpTa5NLW05ZEAhg4dyre//W3Wrl1L9+7dee211wB44okntDySiDQZLTvTTJWUlHDbbbdx8OBBqqqqmD17NpMmTWLz5s089NBDFBcXc+WVV5KVlUV4eDgjRozgmmuu4Y033uDEiRP89re/5ZprrqFv376UlpbSrVs3MjMzufHGG5k8eTLbt2+noqKCn/3sZ4wfP56srCyWLVvG6dOn2b17NxMnTuSXv/wlUD25f+bMmVRVVXHllVeydu1aSkpKam2nPgkJCSxZsoSoqCiqqqoYPXo0v//974mKiqK4uLgpTqtc4rQ8kog0J81p2RnNIWymWtodlhs3bqS8vJw+ffoAsGDBAm6++eZzbggQyewdHphDeJaWRxIR+eoUCJuZDzccJi9nN7v3FPPKquVw5kFuv3dSs77DsrCwkO9///s899xztGrVikOHDvGnP/2JN9988yueLWlptDySiEjjUCBsRj7ccJg3XtxJZbmPb1zRg+kTnmbnoU1MmzqdGyeMbZZ3WJ46dYpx48Yxd+5cBg0aBFRPvt+1axd9+/YF4PTp0/Tt25ddu3bV25Z4g5ZHEhG5+HRTSTOSl7ObynIfACdKPiOkTTuSeqcypM/EZnmHZXl5ORMnTuT222/n1ltvDWwfN24chw8fZt++fezbt4/LLrtMYVBERKQRaYSwGSk+Vhb4/dCxvbz6zrOYtaJ1qzYsXflC4A7LKVOmcPLkSSorK8nIyCA2tu7J9iNHjmTevHkkJiaSmZnJ7NmzycjIID4+Hp/PR2RkZOD5qrWpeYelz+ejS5curFmzpkHt/PGPf2TdunUcPXqUrKwsoPrxYImJiV/tRImIiMgF0V3GzchzM9efEwrPat8plB/825AgVCQiIiJ1aU53GeuScTMyeHwf2oSc+0/WJqQVg8f3CVJFIiIi0hLoknEzEn1NV6B6LmHxsTLadwpl8Pg+ge0iIiIiX4YCYTMTfU1XBUARERG5qHTJWERERMTjFAhFREREPE6BUERERMTjFAhFREREPE6BUERERMTjFAhFREREPE6BUERERMTjFAhFRL6CgoICVq5cGZTP3r9/PwMHDiQxMZHY2FieeeaZv3vPzTffTFxcXBCqE5HmRAtTi4h8BQUFBeTn53PDDTc0+WeHh4eTl5dHaGgoxcXFxMXFcfPNNxMREQHAyy+/TPv27Zu8LhFpfjRCKCKeVFJSwrhx40hISCAuLo7s7GwANm/ezPDhw0lKSiItLY3CwkIARowYwYwZM0hJSSE6Oprc3FzKy8uZM2cO2dnZJCYmkp2dTUlJCXfddRcpKSkMGDCAnJwcALKyskhPT2fMmDFERUUxffr0QC2rVq1i4MCBJCQkkJqaGqivtnZqCgkJITQ0FICysjJ8Pl9gX3FxMb/61a/4yU9+0jgnUERaFudcs/lJSkpyIiIXw5IlS9zdd98deH3ixAlXXl7uBg8e7IqKipxzzr300kvuzjvvdM45N3z4cPfQQw8555xbsWKFS01Ndc45t3jxYvfAAw8E2snMzHTPP/+8c86548ePu6ioKFdcXOwWL17sIiMj3YkTJ1xpaanr2bOnO3DggCsqKnLdu3d3e/bscc45d/To0Xrb+aIDBw64/v37u7CwMLdgwYLA9oyMDPfyyy+7vXv3utjY2Itz0kTkggD57hLITw350SVjEfGUDzccJi9nN7v3FPPKquVw5kFuv3cSQ4cOZfv27Wzfvp3rrrsOgKqqKsLDwwPHpqenA5CUlMS+fftqbX/16tUsW7aM+fPnA3DmzBkOHDgAQGpqKh07dgSgX79+7N+/n+PHjzNs2DAiIyMB6NSpU73tXH311ed8Xo8ePdi2bRuHDh1iwoQJ3HrrrRQWFrJ7925+/etf11mniEhNCoQi4hkfbjjMGy/upLLcxzeu6MH0CU+z89Ampk2dzo0TxjJx4kRiY2PJy8ur9fizl2dbt25NZWVlre9xzrF06VJiYmLO2b5hw4bA8edro7526hIREUFcXBy5ubkcOXKE/Px8vvnNb1JZWUlRUREjRozgzTffbFBbIuI9mkMoIp6Rl7ObyvLqeXYnSj4jpE07knqnMqTPRLZs2UJMTAxHjhwJBMKKigp27NhRb5sdOnTg888/D7xOS0vjySefpPpqEWzdurXe4wcNGsS6devYu3cvAMeOHWtwOwcPHqS0tBSA48eP8/bbbxMTE8P999/PoUOH2LdvH2+//TbR0dEKgyJSL40QiohnFB8rC/x+6NheXn3nWcxa0bpVG5aufIGQkBCWLFnClClTOHnyJJWVlWRkZBAbG1tnmyNHjmTevHkkJiaSmZnJ7NmzycjIID4+Hp/PR2RkJMuXL6/z+M6dO7Nw4ULS09Px+Xx06dKFNWvWNKid999/n2nTpmFmOOd4+OGH6d+//1c/USLiOXb2r8/mIDk52eXn5we7DBFppp6buf6cUHhW+06h/ODfhgShIhFpycxss3MuOdh1NIQuGYuIZwwe34c2Ied+7bUJacXg8X2CVJGIyKVBl4xFxDOir+kKVM8lLD5WRvtOoQwe3yewXUTEqxQIRcRToq/pqgAoIvIFumQsIiIi4nEKhCIiIiIep0AoIiIi4nEKhCIiIiIep0AoIiIi4nEKhCIiIiIep0AoIiIi8iVlZ2fz6KOPBu3zzeyImTkz+/QL239qZmf8+14xs5D62lEgFBEREfmS1qxZw9KlS4NZwi+Af61l+xDgEWA/cBL4YX2NKBCKiIiI5xQVFdGlSxfCwsJo164dU6dOBeCFF17giiuu4LLLLuPKK6+koKAAgCuuuIJrrrmG9u3bExISwoIFCyguLiYrK4vt27cTFhbG1KlTKSoqIjo6mvbt2wMMNLO5AGb2P2b2iX9Er9zMNpytxcxmmdlpMys1s2P+bZ3N7EMzK/bvm1tbP5xz84FjNbeZmQEDgaf9m14CJtR3PhQIRURExHOeeOIJOnXqRGlpKWfOnGHatGmcPn2a+++/n7y8PE6fPs33vvc9brvttsAxVVVVFBcXM2vWLObMmUP79u254447iIuLo7S0lP/8z/9kwoQJpKamUlxcDLAdeMTMOvub6EJ1UOsMJJlZipldBTwKpDnnwoBk/3tfBV53zrUHrv5CO+fzD8AJ51yl//UhoFt9BygQioiIiGdsWvYsh3/Wl1GF/82uDz8gLiaSBQsW0LNnT1577TWKi4sZOHAgYWFhPPvssxw9ejRw7A9/WH3V9ZZbbuHzzz+vtf13332XxYsXExYWBhBHddZK8e/e7Zz72Dl3EjgOJAHfBT52zuUCOOf2+N+bCNxlZqXAzi+0c9EpEIqIiIgnbFr2LHGbf0JXjjC2b1s+eLA9gzsUkjljOqNGjcLn89G+fXtKS0sDI4c1A+Hll18OQNu2bXHO1foZzjleeeUVSktLAbY459o451b4d5fXeKsPCD1PyROcc2H+n5rtnM9R4Aoza+N/HQF8Ut8BCoQiIiLiCT22PEGYVWeyLYWV/MNl8JsbQ7knOZT333+ftLQ0zpw5w8KFCwE4ffo0OTk59bb59a9/nZKSksDrxMREZsyYgc/nA8DMvnOesn4P9DCzof739/ZvLwB+6Z8P2JB2Alx1Wn0DuNW/6TtAvR1RIBQRERFP6OKOBH7/v4+qCJ9fTNjcUzybd4Jf/OIXtG/fnhdeeIFHHnmEsLAwOnXqdN47iO+77z4OHz4cuKlk+fLlVFRUcNlll0H1fMH/rO9459xOqucQvua/PLzJv+tGoA1QamZn6mrHzE4CvwK6mFmlmc307/or8DugFzCJv81NrJXVNeR5KUpOTnb5+fnBLkNERESaocM/60tXjvz9djrT9We7Lvrnmdlm51y9QexSoRFCERER8YSPBz5CqTt3feZSF8LHAx8JUkWXDgVCERER8YRv3fzPbE/6OYfpjM8Zh+nM9qSf862b/znYpQWdLhmLiIiINAJdMhYRERGRZkOBUERERMTjFAhFREREPE6BUERERMTjFAhFREREPE6BUETkSyooKGDlypVB+ez9+/czcOBAEhMTiY2N5Zlnngns+8Mf/kD//v2Jj49nzJgxfPbZZ0GpUUSaDwVCEZEvKZiBMDw8nLy8PAoKCtiwYQPz5s3j0KFDVFZWMnXqVN544w22bdtGfHw8CxYsCEqNItJ8KBCKiOeUlJQwbtw4EhISiIuLIzs7G4DNmzczfPhwkpKSSEtLo7CwEIARI0YwY8YMUlJSiI6OJjc3l/LycubMmUN2djaJiYlkZ2dTUlLCXXfdRUpKCgMGDCAnp/pZ8llZWaSnpzNmzBiioqKYPn16oJZVq1YxcOBAEhISSE1NDdRXWzs1hYSEEBoaCkBZWRk+nw8A5xzOOUpKSnDOcerUKSIiIhrvZIpIy3D2y6M5/CQlJTkRka9qyZIl7u677w68PnHihCsvL3eDBw92RUVFzjnnXnrpJXfnnXc655wbPny4e+ihh5xzzq1YscKlpqY655xbvHixe+CBBwLtZGZmuueff94559zx48ddVFSUKy4udosXL3aRkZHuxIkTrrS01PXs2dMdOHDAFRUVue7du7s9e/Y455w7evRove180YEDB1z//v1dWFiYW7BgQWD7n/70J9ehQwfXtWtXN3ToUFdZWXlxTpyIXBAg310C+akhPxohFBHPKDycw/r1Qyku/jHLlz/PAw+mk5ubS8eOHfnggw/Yvn071113HYmJifz85z/n4MGDgWPT09MBSEpKYt++fbW2v3r1aubNm0diYiIjRozgzJkzHDhwAIDU1FQ6duxIu3bt6NevH/v37+edd95h2LBhREZGAtCpU6fztlNTjx492LZtG7t27eK5557j008/paKigqeffpqtW7dy6NAh4uPjefzxxy/maRSRFqhNsAsQEWkKhYdz2LlzFj5fKd17tOWpp7uyadNGpk+/n7Fjb2PixInExsaSl5dX6/FnL8+2bt2aysrKWt/jnGPp0qXExMScs33Dhg2B48/XRn3t1CUiIoK4uDhyc3Pp1asXAH369AHgtttuY968eQ1qR0S8SyOEIuIJe3bPx+crBeCzzypp185ITQ1l4kTHli1biImJ4ciRI4FAWFFRwY4dO+pts0OHDnz++eeB12lpaTz55JM4/zPit27dWu/xgwYNYt26dezduxeAY8eONbidgwcPUlpa3Z/jx4/z9ttvExMTQ7du3fjrX//KkSNHAFizZg1XX311/SdHRDxPI4Qi4glnygoDv+/dW87CZ4/RqhW0bmO8+MJzhISEsGTJEqZMmcLJkyeprKwkIyOD2NjYOtscOXJk4NJuZmYms2fPJiMjg/j4eHw+H5GRkSxfvrzO4zt37szChQtJT0/H5/PRpUsX1qxZ06B23n//faZNm4aZ4Zzj4Ycfpn///gD89Kc/ZdiwYbRt25ZevXqRlZX11U6eiLR4dvYv0Cb9ULMngJuAcmA3cKdz7sT5jktOTnb5+fmNXZ6ItEDr1w/lTNmhv9veLjSCIUNyg1CRiLR0ZrbZOZcc7DoaIliXjNcAcc65eOBDIDNIdYiIR/Tu8zCtWoWds61VqzB693k4SBWJiFw6ghIInXOrnXNnZ1S/A3QPRh0i4h3hXcdz1VVzaRcaARjtQiO46qq5hHcdH+zSRESC7lKYQ3gXkF3XTjO7F7gXoGfPnk1Vk4i0QOFdxysAiojUotECoZm9DnStZdcs51yO/z2zgErgxbracc4tBBZC9RzCRihVRERExNMaLRA650bXt9/M7gBuBFJdMO5sEREREREgSJeMzWwMMB0Y7pw7HYwaRERERKRasO4yXgB0ANaYWYGZPROkOkREREQ8LygjhM65vsH4XBERERH5e3p0nYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKiIiIeJwCoYiIiIjHKRCKSItTUFDAypUrg/LZ+/fvZ+DAgSQmJhIbG8szzzwT2JednU18fDyxsbHMmDEjKPWJiNRGgVBEWpxgBsLw8HDy8vIoKChgw4YNzJs3j0OHDnH06FEeeeQR1q5dy44dOzh8+DBr164NSo0iIl+kQCgil4ySkhLGjRtHQkICcXFxZGdnA7B582aGDx9OUlISaWlpFBYWAjBixAhmzJhBSkoK0dHR5ObmUl5ezpw5c8jOziYxMZHs7GxKSkq46667SElJYcCAAeTk5ACQlZVFeno6Y8aMISoqiunTpwdqWbVqFQMHDiQhIYHU1NRAfbW1U1NISAihoaEAlJWV4fP5ANizZw9RUVF07twZgNGjR7N06dJGOpMiIhemTbALEBE5a9WqVURERLBixQoATp48SUVFBZMnTyYnJ4fOnTuTnZ3NrFmzWLRoEQCVlZVs3LiRlStX8uijj/L666/z2GOPkZ+fz4IFCwCYOXMmo0aNYtGiRZw4cYKUlBRGjx4NVI8mbt26ldDQUGJiYpg8eTLt2rXjnnvuYd26dURGRnLs2DEA5s6dW2s7l19++Tn9+Pjjjxk3bhy7du3iiSeeICIigrCwMD744AP27dtH9+7defXVVykvL2+qUysiUi8FQhEJuqWHj/H4nkL2l7Th1PKVHHlwCtMmfZuhQ4eyfft2tm/fznXXXQdAVVUV4eHhgWPT09MBSEpKYt++fbW2v3r1apYtW8b8+fMBOHPmDAcOHAAgNTWVjh07AtCvXz/279/P8ePHGTZsGJGRkQB06tSp3nauvvrqcz6vR48ebNu2jUOHDjFhwgRuvfVWvvGNb/D0008zadIkWrVqxbXXXsvu3bsvxukTEfnKFAhFJKiWHj7Gwx98TKnP0bpHLzo+83vWb1zPR9P/hUlj05g4cSKxsbHk5eXVevzZy7OtW7emsrKy1vc451i6dCkxMTHnbN+wYUPg+PO1UV87dYmIiCAuLo7c3FxuvfVWbrrpJm666SYAFi5cSOvWrRvUjohIY9McQhEJqsf3FFLqcwBUfVaEtWtH69E3UJb+XbZs2UJMTAxHjhwJBMKKigp27NhRb5sdOnTg888/D7xOS0vjySefxLnqz9m6dWu9xw8aNIh169axd+9egMAl44a0c/DgQUpLSwE4fvw4b7/9diBAFhUVBbY/9dRT3H333fXWISLSVDRCKCJB9UlZReD3yr27KH72P8CMo23a8NILzxESEsKSJUuYMmUKJ0+epLKykoyMDGJjY+tsc+TIkcybN4/ExEQyMzOZPXs2GRkZxMfH4/P5iIyMZPny5XUe37lzZxYuXEh6ejo+n48uXbqwZs2aBrXz/vvvM23aNMwM5xwPP/ww/fv3B2Dq1Km8++67AMyZM4fo6OivcupERC4aO/uXbnOQnJzs8vPzg12GiFxEyX/ZwcEaofCs7qFtyb+27tAnInKpM7PNzrnkYNfRELpkLCJBldk7nLBWds62sFZGZu/wOo4QEZGLTZeMRSSobulafQfv43sK+aSsgm6hbcnsHR7YLiIijU+BUESC7paunRQARUSCSJeMRURERDxOgVBERETE4xQIRURERDxOgVBERETE4xQIRURERDxOgVBERETE4xQIRURERDxOgVBERETE4xQIRURERDxOgVCkgQoKCli5cmVQazh16hTdu3fnwQcfDGzbvHkz/fv3p2/fvkyZMgXnXBArFBGR5kiBUKSBLoVAOHv2bIYNG3bOtvvvv5/f/OY3fPTRR3z00UesWrUqSNWJiEhzpUAoLV5JSQnjxo0jISGBuLg4srOzgeqRteHDh5OUlERaWhqFhYUAjBgxghkzZpCSkkJ0dDS5ubmUl5czZ84csrOzSUxMJDs7pEEo9AAADZ5JREFUm5KSEu666y5SUlIYMGAAOTk5AGRlZZGens6YMWOIiopi+vTpgVpWrVrFwIEDSUhIIDU1NVBfbe180ebNm/n000+5/vrrA9sKCws5deoUgwYNwsy4/fbbefXVVxvlPIqISMvVJtgFiDS2VatWERERwYoVKwA4efIkFRUVTJ48mZycHDp37kx2djazZs1i0aJFAFRWVrJx40ZWrlzJo48+yuuvv85jjz1Gfn4+CxYsAGDmzJmMGjWKRYsWceLECVJSUhg9ejRQPZq4detWQkNDiYmJYfLkybRr14577rmHdevWERkZybFjxwCYO3dure1cfvnlgT74fD6mTZvGCy+8wOuvvx7Y/sknn9C9e/fA6+7du/PJJ5807gkVEZEWR4FQWqxXt37CE699wP49R/lsyf9ytOJH/PiH/8TQoUPZvn0727dv57rrrgOgqqqK8PDwwLHp6ekAJCUlsW/fvlrbX716NcuWLWP+/PkAnDlzhgMHDgCQmppKx44dAejXrx/79+/n+PHjDBs2jMjISAA6depUbztXX3114LOeeuopbrjhhnPCn4iIyMWiQCgt0qtbPyHz5fcoraiiTadudL79P3hn/xbuy3iESeNvYOLEicTGxpKXl1fr8aGhoQC0bt2aysrKWt/jnGPp0qXExMScs33Dhg2B48/XRn3t1JSXl0dubi5PPfUUxcXFlJeX0759e6ZOncrBgwcD7zt48CDdunWrsx0REZHaaA6htEhPvPYBpRVVAFR+fpRWbUMJuWo4vrib2LJlCzExMRw5ciQQCCsqKtixY0e9bXbo0IHPP/888DotLY0nn3wycFfv1q1b6z1+0KBBrFu3jr179wIELhk3pJ0XX3yRAwcOsG/fPubPn8/tt9/OvHnzCA8P52tf+xrvvPMOzjl+97vfMX78+IacIhERkQCNEEqLdOhEaeD3iiP7KHpzMZhhrdrw/P/+npCQEJYsWcKUKVM4efIklZWVZGRkEBsbW2ebI0eOZN68eSQmJpKZmcns2bPJyMggPj4en89HZGQky5cvr/P4zp07s3DhQtLT0/H5fHTp0oU1a9ZccDtf9NRTT3HHHXdQWlrK2LFjGTt2bIOPFRERAbDmtGZZcnKyy8/PD3YZ0gwMmfdnPqkRCs/qdkUY6/9lVBAqEhERrzGzzc655GDX0RC6ZCwt0iNpMYS1bX3OtrC2rXkkre55eiIiIl6lS8bSIk0YUH1jxROvfcChE6VEXBHGI2kxge0iIiLyNwqE0mJNGNBNAVBERKQBdMlYRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOOCEgjN7F/NbJuZFZjZajOLCEYdIiIiIhK8EcInnHPxzrlEYDkwJ0h1iIiIiHheUAKhc+5UjZeXAy4YdYiIiIgItAnWB5vZXOB24CQwsp733QvcC9CzZ8+mKU5ERETEQ8y5xhmcM7PXga617JrlnMup8b5MoJ1z7qfnazM5Odnl5+dfxCpFREREGoeZbXbOJQe7joZotBFC59zoBr71RWAlcN5AKCIiIiIXX7DuMo6q8XI8sDMYdYiIiIhII14yrvdDzZYCMYAP2A/c55z7pAHHHfG/vzFcCXzWSG1fyrzab/Bu39Vv7/Fq373ab/Bu3y+1fvdyznUOdhENEZRAeCkys/zmcp3/YvJqv8G7fVe/vcerffdqv8G7ffdqvy8GPalERERExOMUCEVEREQ8ToHwbxYGu4Ag8Wq/wbt9V7+9x6t992q/wbt992q/vzLNIRQRERHxOI0QioiIiHicAqGIiIiIxykQ1sLMppmZM7Mrg11LUzCzfzWzbWZWYGarzSwi2DU1BTN7wsx2+vv+ipldEeyamoqZfdvMdpiZz8xa/BINZjbGzD4ws11m9i/BrqepmNkiMysys+3BrqUpmVkPM3vDzP7q/+/51GDX1BTMrJ2ZbTSzd/39fjTYNTU1M2ttZlvNbHmwa2luFAi/wMx6ANcDB4JdSxN6wjkX75xLBJYDc4JdUBNZA8Q55+KBD4HMINfTlLYD6cC6YBfS2MysNfDfwFigH/BPZtYvuFU1mSxgTLCLCIJKYJpzrh8wCHjAI//mZcAo51wCkAiMMbNBQa6pqU0F3g92Ec2RAuHf+zUwHfDM3TbOuVM1Xl6OR/runFvtnKv0v3wH6B7MepqSc+5959wHwa6jiaQAu5xze5xz5cBLVD8ys8Vzzq0DjgW7jqbmnCt0zm3x//451QGhW3CranyuWrH/ZVv/jye+zwHMrDswDvifYNfSHCkQ1mBm44FPnHPvBruWpmZmc83sY+B7eGeEsKa7gP8LdhHSKLoBH9d4fRAPhAOpZmbfBAYAG4JbSdPwXzItAIqANc45T/Tb7z+oHtDxBbuQ5qhNsAtoamb2OtC1ll2zgJlUXy5ucerrt3Muxzk3C5hlZpnAg8BPm7TARnK+fvvfM4vqS0wvNmVtja0hfRdpycysPbAUyPjClZAWyzlXBST650S/YmZxzrkWP4fUzG4Eipxzm81sRLDraY48Fwidc6Nr225m/YFI4F0zg+rLh1vMLMU5d7gJS2wUdfW7Fi8CK2khgfB8/TazO4AbgVTXwhblvIB/85buE6BHjdfd/dukBTOztlSHwRedcy8Hu56m5pw7YWZvUD2HtMUHQmAIcLOZ3QC0A75mZi845/5fkOtqNnTJ2M85955zrotz7pvOuW9SfVlpYEsIg+djZlE1Xo4HdgarlqZkZmOovrxws3PudLDrkUazCYgys0gzCwG+AywLck3SiKz6r/rfAu87534V7Hqaipl1PrtagpmFAdfhke9z51ymc667//+/vwP8WWHwwigQCsA8M9tuZtuovmTuiSUagAVAB2CNf8mdZ4JdUFMxs4lmdhAYDKwws9eCXVNj8d849CDwGtU3F/zRObcjuFU1DTP7A5AHxJjZQTP7YbBraiJDgO8Do/z/2y7wjxy1dOHAG/7v8k1UzyHU8ivSIHp0nYiIiIjHaYRQRERExOMUCEVEREQ8ToFQRERExOMUCEVEREQ8ToFQRERExOMUCEXkvMysyr90x3Yz+5OZXVbH+/7yJdtPNrP/+gr1FdexvauZvWRmu81ss5mtNLPoL/s5lwIzG2Fm19ax7yozyzOzMjN7uKlrE5HmS4FQRBqi1DmX6JyLA8qB+2ruNLM2AM65WoPK+Tjn8p1zU756mefUZMArwJvOuT7OuSQgE/jGxfycIBgB1HWejwFTgPlNVo2ItAgKhCJyoXKBvv6RqlwzWwb8Ff42Uuff96aZLTGznWb2oj+gYWbfMrO/mNm7ZrbRzDr437/cv/9nZva8f6TrIzO7x7+9vZmtNbMtZvaemY0/T50jgQrnXGDBcefcu865XKv2hH/E8z0zm1Sj7rfMLMfM9pjZPDP7nr/O98ysj/99WWb2jJnlm9mH/ueoYmbtzGyx/71bzWykf/sdZvayma3y9+mXZ2sys+v9fd3iH31t79++z8werdHfq8zsm1SH8R/7R2yH1uywc67IObcJqPgS/64i4mGee5axiHx5/pHAscAq/6aBQJxzbm8tbx8AxAKHgPXAEDPbCGQDk5xzm8zsa0BpLcfGA4OAy4GtZrYCKAImOudOmdmVwDtmtqyeZ1DHAZvr2JcOJAIJwJXAJjNb59+XAFxN9WjbHuB/nHMpZjYVmAxk+N/3TSAF6EP10yH6Ag8AzjnX38yuAlbXuESd6D8nZcAHZvakv+8/AUY750rMbAbwEPCY/5jPnHMDzexHwMPOubv9T9Qpds5pFFBELhoFQhFpiDAzK/D/nkv1c2KvBTbWEQbx7zsI4D/2m8BJoNA/ioVz7pR//xePzXHOlQKlZvYG1cFrBfBvZjYM8AHdqL78+2WeN/6PwB+cc1XAp2b2FvAt4BSwyTlX6K9rN7Daf8x7VI86nvVH55wP+MjM9gBX+dt90t+3nWa2HzgbCNc650762/0r0Au4AugHrPefgxCqHzV31sv+/9xMdYgVEWkUCoQi0hClzrnEmhv8AaaknmPKavxexYV933xx1M8B3wM6A0nOuQoz2we0q6eNHcCtF/CZZ9Ws21fjtY9z+1BbjQ1t9+z5MKqfN/tP5znmQs+fiMgF0RxCEWlKHwDhZvYtAP/8wdqCznj/fLx/oPomik1AR6DIHwZHUj3CVp8/A6Fmdu/ZDWYW7593lwtMMrPWZtYZGAZsvMC+fNvMWvnnFfb29y2X6uCK/1JxT//2urxD9aX0vv5jLrfz3wX9OdDhAmsVEamXAqGINBnnXDkwCXjSzN4F1lD7KN824A2qA9O/OucOAS8CyWb2HnA7sPM8n+WAicBoq152ZgfwONWXmF/xf8a7VAfH6c65C730fIDqEPl/wH3OuTPAU0Arf43ZwB3OubK6GnDOHQHuAP5gZtuovlx81Xk+93+BibXdVGLVy+wcpHoe4k/M7KB/nqaISL2s7vnYIiJNz8x+xiV+04SZZQHLnXNLgl2LiMjFoBFCEREREY/TCKGIiIiIx2mEUERERMTjFAhFREREPE6BUERERMTjFAhFREREPE6BUERERMTj/j+04X1SnMRAUgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x720 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CIRTlf6ECZOs"
      },
      "source": [
        "## 5.2. Metric Evaluation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s3qCDMl9CdpD",
        "outputId": "c7b4a23b-6f12-488b-a8cc-797e38e5f47c"
      },
      "source": [
        "!pip install rouge-metric"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting rouge-metric\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/bb/34/18ddbc94f65e8b45220b373b2ad2db6bef7549f4b00b4baaaaa47204be1a/rouge_metric-1.0.1-py3-none-any.whl (151kB)\n",
            "\r\u001b[K     |██▏                             | 10kB 16.5MB/s eta 0:00:01\r\u001b[K     |████▎                           | 20kB 21.0MB/s eta 0:00:01\r\u001b[K     |██████▌                         | 30kB 23.6MB/s eta 0:00:01\r\u001b[K     |████████▋                       | 40kB 26.6MB/s eta 0:00:01\r\u001b[K     |██████████▉                     | 51kB 29.3MB/s eta 0:00:01\r\u001b[K     |█████████████                   | 61kB 30.7MB/s eta 0:00:01\r\u001b[K     |███████████████▏                | 71kB 28.8MB/s eta 0:00:01\r\u001b[K     |█████████████████▎              | 81kB 27.5MB/s eta 0:00:01\r\u001b[K     |███████████████████▍            | 92kB 25.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████▋          | 102kB 26.2MB/s eta 0:00:01\r\u001b[K     |███████████████████████▊        | 112kB 26.2MB/s eta 0:00:01\r\u001b[K     |██████████████████████████      | 122kB 26.2MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 133kB 26.2MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▎ | 143kB 26.2MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 153kB 26.2MB/s \n",
            "\u001b[?25hInstalling collected packages: rouge-metric\n",
            "Successfully installed rouge-metric-1.0.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ALV357zPCo8T",
        "outputId": "215c7f05-cf5e-45c6-e16a-1a632c6e5c1c"
      },
      "source": [
        "!rouge-metric -h"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "usage: rouge-metric [-h] [-2 SKIP_GAP] [-u] [-U] [-3 BE] [-c CONF_INT] [-d]\n",
            "                    [-e DATA_HOME] [-f FORMULA] [-b N_BYTES] [-l N_WORDS] [-m]\n",
            "                    [-n MAX_NGRAM] [-p ALPHA] [-s] [-t UNIT] [-r RESAMPLING]\n",
            "                    [-w WEIGHT] [-v] [-x] [-z EVAL_CONFIG]\n",
            "                    hypothesis reference [reference ...]\n",
            "\n",
            "Compute ROUGE metrics\n",
            "\n",
            "positional arguments:\n",
            "  hypothesis      Hypothesis file or directory\n",
            "  reference       Reference file or directory\n",
            "\n",
            "optional arguments:\n",
            "  -h, --help      show this help message and exit\n",
            "  -2 SKIP_GAP     Compute skip bigram (ROUGE-S) co-occurrence, also specify\n",
            "                  the maximum gap length between two words (skip-bigram)\n",
            "  -u              Compute skip bigram as -2 but include unigram, i.e. treat\n",
            "                  unigram as \"start-sentence-symbol unigram\"; -2 has to be\n",
            "                  specified.\n",
            "  -U              same as -u but also compute regular skip-bigram\n",
            "  -3 BE           Compute BE score. Currently only SIMPLE BE triple format is\n",
            "                  supported. H -> head only scoring (does not applied to\n",
            "                  Minipar-based BEs). HM -> head and modifier pair scoring.\n",
            "                  HMR -> head, modifier and relation triple scoring. HM1 -> H\n",
            "                  and HM scoring (same as HM for Minipar-based BEs). HMR1 ->\n",
            "                  HM and HMR scoring (same as HMR for Minipar-based BEs). HMR2\n",
            "                  -> H, HM and HMR scoring (same as HMR for Minipar-based\n",
            "                  BEs).\n",
            "  -c CONF_INT     Specify CF% (0 <= CF <= 100) confidence interval to compute.\n",
            "                  The default is 95% (i.e. CF=95).\n",
            "  -d              Print per evaluation average score for each system.\n",
            "  -e DATA_HOME    Specify where the ROUGE data files can be found\n",
            "  -f FORMULA      Select scoring formula: A => average model; B => best model\n",
            "  -b N_BYTES      Only use the first n bytes in the system/peer summary for\n",
            "                  the evaluation.\n",
            "  -l N_WORDS      Only use the first n words in the system/peer summary for\n",
            "                  the evaluation.\n",
            "  -m              Stem both model and system summaries using Porter stemmer\n",
            "                  before computing various statistics.\n",
            "  -n MAX_NGRAM    Compute ROUGE-N up to max-ngram length.\n",
            "  -p ALPHA        Relative importance of recall and precision ROUGE scores.\n",
            "                  Alpha -> 1 favors precision, Alpha -> 0 favors recall.\n",
            "  -s              Remove stopwords in model and system summaries before\n",
            "                  computing various statistics.\n",
            "  -t UNIT         Compute average ROUGE by averaging over the whole test\n",
            "                  corpus instead of sentences (units). 0: use sentence as\n",
            "                  counting unit, 1: use token as counting unit, 2: same as 1\n",
            "                  but output raw counts instead of precision, recall, and\n",
            "                  f-measure scores. 2 is useful when computation of the final,\n",
            "                  precision, recall, and f-measure scores will be conducted\n",
            "                  later.\n",
            "  -r RESAMPLING   Specify the number of sampling point in bootstrap resampling\n",
            "                  (default is 1000).\n",
            "  -w WEIGHT       Compute ROUGE-W that gives consecutive matches of length L\n",
            "                  in an LCS a weight of L^weight instead of just L as in LCS.\n",
            "                  Typically this is set to 1.2 or other number greater than 1.\n",
            "  -v              Print debugging information for diagnostic purpose.\n",
            "  -x              Do not calculate ROUGE-L.\n",
            "  -z EVAL_CONFIG  ROUGE-eval-config-file is a list of peer-model pair per line\n",
            "                  in the specified format (SEE|SPL|ISI|SIMPLE).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5Dj5yQS1GLZm",
        "outputId": "0120bdfc-667c-4c57-e841-cda22d589af8"
      },
      "source": [
        "cd /content/drive/MyDrive"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZY1EGsr0GSji",
        "outputId": "744f6291-e4d2-44cf-d0da-efe2ad6b0061"
      },
      "source": [
        "!git clone 'https://github.com/li-plus/rouge-metric.git'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'rouge-metric'...\n",
            "remote: Enumerating objects: 97, done.\u001b[K\n",
            "remote: Counting objects: 100% (97/97), done.\u001b[K\n",
            "remote: Compressing objects: 100% (77/77), done.\u001b[K\n",
            "remote: Total 97 (delta 8), reused 97 (delta 8), pack-reused 0\u001b[K\n",
            "Unpacking objects: 100% (97/97), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2kPdwfCAGVrK",
        "outputId": "01fa14ba-85f0-4c77-95f4-2e66ca751214"
      },
      "source": [
        "cd rouge-metric/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/My Drive/rouge-metric\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sbLQTJ6lGzUf",
        "outputId": "101ed858-4b93-4780-9b65-4464dff4ccec"
      },
      "source": [
        "!apt-get install -y cpanminus\n",
        "!cpanm --force XML::Parser"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Reading package lists... Done\n",
            "Building dependency tree       \n",
            "Reading state information... Done\n",
            "The following additional packages will be installed:\n",
            "  libalgorithm-c3-perl libb-hooks-endofscope-perl libb-hooks-op-check-perl\n",
            "  libclass-c3-perl libclass-c3-xs-perl libclass-method-modifiers-perl\n",
            "  libclass-xsaccessor-perl libcpan-changes-perl libcpan-distnameinfo-perl\n",
            "  libcpan-meta-check-perl libdata-optlist-perl libdata-perl-perl\n",
            "  libdata-section-perl libdevel-callchecker-perl\n",
            "  libdevel-globaldestruction-perl libdynaloader-functions-perl\n",
            "  libencode-locale-perl libexporter-tiny-perl libfile-pushd-perl\n",
            "  libfile-slurp-perl libgetopt-long-descriptive-perl libhtml-parser-perl\n",
            "  libhtml-tagset-perl libhttp-date-perl libhttp-message-perl\n",
            "  libimport-into-perl libio-html-perl libio-stringy-perl\n",
            "  liblist-moreutils-perl liblocal-lib-perl liblwp-mediatypes-perl\n",
            "  libmodule-build-perl libmodule-cpanfile-perl libmodule-implementation-perl\n",
            "  libmodule-runtime-perl libmodule-signature-perl libmoo-perl\n",
            "  libmoox-handlesvia-perl libmro-compat-perl libnamespace-autoclean-perl\n",
            "  libnamespace-clean-perl libpackage-stash-perl libpackage-stash-xs-perl\n",
            "  libparams-classify-perl libparams-util-perl libparams-validate-perl\n",
            "  libparse-pmfile-perl libpath-tiny-perl libpod-markdown-perl\n",
            "  libpod-readme-perl libreadonly-perl libref-util-perl libref-util-xs-perl\n",
            "  librole-tiny-perl libsoftware-license-perl libstrictures-perl\n",
            "  libstring-shellquote-perl libsub-exporter-perl\n",
            "  libsub-exporter-progressive-perl libsub-identify-perl libsub-install-perl\n",
            "  libsub-name-perl libsub-quote-perl libtext-template-perl libtimedate-perl\n",
            "  libtry-tiny-perl libtype-tiny-perl libtype-tiny-xs-perl libunicode-utf8-perl\n",
            "  liburi-perl libvariable-magic-perl\n",
            "Suggested packages:\n",
            "  libdata-dump-perl libscalar-number-perl libbareword-filehandles-perl\n",
            "  libindirect-perl libmultidimensional-perl libdevel-stacktrace-perl\n",
            "  libdevel-lexalias-perl libwww-perl\n",
            "The following NEW packages will be installed:\n",
            "  cpanminus libalgorithm-c3-perl libb-hooks-endofscope-perl\n",
            "  libb-hooks-op-check-perl libclass-c3-perl libclass-c3-xs-perl\n",
            "  libclass-method-modifiers-perl libclass-xsaccessor-perl libcpan-changes-perl\n",
            "  libcpan-distnameinfo-perl libcpan-meta-check-perl libdata-optlist-perl\n",
            "  libdata-perl-perl libdata-section-perl libdevel-callchecker-perl\n",
            "  libdevel-globaldestruction-perl libdynaloader-functions-perl\n",
            "  libencode-locale-perl libexporter-tiny-perl libfile-pushd-perl\n",
            "  libfile-slurp-perl libgetopt-long-descriptive-perl libhtml-parser-perl\n",
            "  libhtml-tagset-perl libhttp-date-perl libhttp-message-perl\n",
            "  libimport-into-perl libio-html-perl libio-stringy-perl\n",
            "  liblist-moreutils-perl liblocal-lib-perl liblwp-mediatypes-perl\n",
            "  libmodule-build-perl libmodule-cpanfile-perl libmodule-implementation-perl\n",
            "  libmodule-runtime-perl libmodule-signature-perl libmoo-perl\n",
            "  libmoox-handlesvia-perl libmro-compat-perl libnamespace-autoclean-perl\n",
            "  libnamespace-clean-perl libpackage-stash-perl libpackage-stash-xs-perl\n",
            "  libparams-classify-perl libparams-util-perl libparams-validate-perl\n",
            "  libparse-pmfile-perl libpath-tiny-perl libpod-markdown-perl\n",
            "  libpod-readme-perl libreadonly-perl libref-util-perl libref-util-xs-perl\n",
            "  librole-tiny-perl libsoftware-license-perl libstrictures-perl\n",
            "  libstring-shellquote-perl libsub-exporter-perl\n",
            "  libsub-exporter-progressive-perl libsub-identify-perl libsub-install-perl\n",
            "  libsub-name-perl libsub-quote-perl libtext-template-perl libtimedate-perl\n",
            "  libtry-tiny-perl libtype-tiny-perl libtype-tiny-xs-perl libunicode-utf8-perl\n",
            "  liburi-perl libvariable-magic-perl\n",
            "0 upgraded, 72 newly installed, 0 to remove and 39 not upgraded.\n",
            "Need to get 2,249 kB of archives.\n",
            "After this operation, 6,883 kB of additional disk space will be used.\n",
            "Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcpan-distnameinfo-perl all 0.12-1 [8,662 B]\n",
            "Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcpan-meta-check-perl all 0.014-1 [7,594 B]\n",
            "Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libfile-pushd-perl all 1.014-1 [11.8 kB]\n",
            "Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libmodule-build-perl all 0.422400-1 [201 kB]\n",
            "Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblocal-lib-perl all 2.000024-1 [46.1 kB]\n",
            "Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libmodule-cpanfile-perl all 1.1002-1 [23.8 kB]\n",
            "Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libparse-pmfile-perl all 0.41-1 [14.2 kB]\n",
            "Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libstring-shellquote-perl all 1.04-1 [12.0 kB]\n",
            "Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 cpanminus all 1.7043-1 [58.0 kB]\n",
            "Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libalgorithm-c3-perl all 0.10-1 [11.3 kB]\n",
            "Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libb-hooks-op-check-perl amd64 0.22-1 [10.1 kB]\n",
            "Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 libdynaloader-functions-perl all 0.003-1 [11.9 kB]\n",
            "Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libdevel-callchecker-perl amd64 0.007-2build1 [14.1 kB]\n",
            "Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 libparams-classify-perl amd64 0.015-1 [21.2 kB]\n",
            "Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmodule-runtime-perl all 0.016-1 [16.2 kB]\n",
            "Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 libtry-tiny-perl all 0.30-1 [20.5 kB]\n",
            "Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmodule-implementation-perl all 0.09-1 [12.2 kB]\n",
            "Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-exporter-progressive-perl all 0.001013-1 [6,784 B]\n",
            "Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libvariable-magic-perl amd64 0.62-1 [34.5 kB]\n",
            "Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 libb-hooks-endofscope-perl all 0.21-1 [14.7 kB]\n",
            "Get:21 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libclass-c3-perl all 0.33-1 [19.0 kB]\n",
            "Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libclass-c3-xs-perl amd64 0.14-1build3 [15.8 kB]\n",
            "Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 libclass-method-modifiers-perl all 2.12-1 [15.7 kB]\n",
            "Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 libclass-xsaccessor-perl amd64 1.19-2build8 [32.8 kB]\n",
            "Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libcpan-changes-perl all 0.400002-1 [32.0 kB]\n",
            "Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 libparams-util-perl amd64 1.07-3build3 [19.6 kB]\n",
            "Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-install-perl all 0.928-1 [10.5 kB]\n",
            "Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 libdata-optlist-perl all 0.110-1 [9,956 B]\n",
            "Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 libexporter-tiny-perl all 1.000000-2 [34.6 kB]\n",
            "Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 liblist-moreutils-perl amd64 0.416-1build3 [55.5 kB]\n",
            "Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 librole-tiny-perl all 2.000006-1 [15.9 kB]\n",
            "Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 libstrictures-perl all 2.000003-1 [16.1 kB]\n",
            "Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libdata-perl-perl all 0.002009-2 [39.1 kB]\n",
            "Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libmro-compat-perl all 0.13-1 [11.2 kB]\n",
            "Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-exporter-perl all 0.987-1 [44.9 kB]\n",
            "Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libdata-section-perl all 0.200007-1 [11.8 kB]\n",
            "Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 libdevel-globaldestruction-perl all 0.14-1 [6,752 B]\n",
            "Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 libencode-locale-perl all 1.05-1 [12.3 kB]\n",
            "Get:39 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libfile-slurp-perl all 9999.19-6 [38.5 kB]\n",
            "Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 libio-stringy-perl all 2.111-2 [60.6 kB]\n",
            "Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 libparams-validate-perl amd64 1.29-1 [52.3 kB]\n",
            "Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 libgetopt-long-descriptive-perl all 0.102-1 [24.4 kB]\n",
            "Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhtml-tagset-perl all 3.20-3 [12.1 kB]\n",
            "Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 liburi-perl all 1.73-1 [77.2 kB]\n",
            "Get:45 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhtml-parser-perl amd64 3.72-3build1 [85.9 kB]\n",
            "Get:46 http://archive.ubuntu.com/ubuntu bionic/main amd64 libtimedate-perl all 2.3000-2 [37.5 kB]\n",
            "Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhttp-date-perl all 6.02-1 [10.4 kB]\n",
            "Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 libio-html-perl all 1.001-1 [14.9 kB]\n",
            "Get:49 http://archive.ubuntu.com/ubuntu bionic/main amd64 liblwp-mediatypes-perl all 6.02-1 [21.7 kB]\n",
            "Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 libhttp-message-perl all 6.14-1 [72.1 kB]\n",
            "Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 libimport-into-perl all 1.002005-1 [11.0 kB]\n",
            "Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libmodule-signature-perl all 0.81-1 [22.9 kB]\n",
            "Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-quote-perl all 2.005000-1 [17.0 kB]\n",
            "Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 libmoo-perl all 2.003004-1 [45.5 kB]\n",
            "Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libmoox-handlesvia-perl all 0.001008-3 [17.6 kB]\n",
            "Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpackage-stash-perl all 0.37-1 [18.3 kB]\n",
            "Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-identify-perl amd64 0.14-1 [10.5 kB]\n",
            "Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsub-name-perl amd64 0.21-1build1 [11.6 kB]\n",
            "Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 libnamespace-clean-perl all 0.27-1 [13.6 kB]\n",
            "Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libnamespace-autoclean-perl all 0.28-1 [12.5 kB]\n",
            "Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpackage-stash-xs-perl amd64 0.28-3build2 [16.4 kB]\n",
            "Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 libpath-tiny-perl all 0.104-1 [55.3 kB]\n",
            "Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libpod-markdown-perl all 3.005000-1 [27.9 kB]\n",
            "Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libtype-tiny-perl all 1.002001-1 [226 kB]\n",
            "Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libpod-readme-perl all 1.1.2-2 [32.9 kB]\n",
            "Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libreadonly-perl all 2.050-1 [19.8 kB]\n",
            "Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libref-util-perl all 0.203-1 [14.8 kB]\n",
            "Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libref-util-xs-perl amd64 0.116-1 [11.5 kB]\n",
            "Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libtext-template-perl all 1.47-1 [46.9 kB]\n",
            "Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libsoftware-license-perl all 0.103012-1 [108 kB]\n",
            "Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libtype-tiny-xs-perl amd64 0.012-2 [22.8 kB]\n",
            "Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 libunicode-utf8-perl amd64 0.60-1build4 [17.9 kB]\n",
            "Fetched 2,249 kB in 1s (1,550 kB/s)\n",
            "Extracting templates from packages: 100%\n",
            "Selecting previously unselected package libcpan-distnameinfo-perl.\n",
            "(Reading database ... 160772 files and directories currently installed.)\n",
            "Preparing to unpack .../00-libcpan-distnameinfo-perl_0.12-1_all.deb ...\n",
            "Unpacking libcpan-distnameinfo-perl (0.12-1) ...\n",
            "Selecting previously unselected package libcpan-meta-check-perl.\n",
            "Preparing to unpack .../01-libcpan-meta-check-perl_0.014-1_all.deb ...\n",
            "Unpacking libcpan-meta-check-perl (0.014-1) ...\n",
            "Selecting previously unselected package libfile-pushd-perl.\n",
            "Preparing to unpack .../02-libfile-pushd-perl_1.014-1_all.deb ...\n",
            "Unpacking libfile-pushd-perl (1.014-1) ...\n",
            "Selecting previously unselected package libmodule-build-perl.\n",
            "Preparing to unpack .../03-libmodule-build-perl_0.422400-1_all.deb ...\n",
            "Adding 'diversion of /usr/bin/config_data to /usr/bin/config_data.diverted by libmodule-build-perl'\n",
            "Adding 'diversion of /usr/share/man/man1/config_data.1.gz to /usr/share/man/man1/config_data.diverted.1.gz by libmodule-build-perl'\n",
            "Unpacking libmodule-build-perl (0.422400-1) ...\n",
            "Selecting previously unselected package liblocal-lib-perl.\n",
            "Preparing to unpack .../04-liblocal-lib-perl_2.000024-1_all.deb ...\n",
            "Unpacking liblocal-lib-perl (2.000024-1) ...\n",
            "Selecting previously unselected package libmodule-cpanfile-perl.\n",
            "Preparing to unpack .../05-libmodule-cpanfile-perl_1.1002-1_all.deb ...\n",
            "Unpacking libmodule-cpanfile-perl (1.1002-1) ...\n",
            "Selecting previously unselected package libparse-pmfile-perl.\n",
            "Preparing to unpack .../06-libparse-pmfile-perl_0.41-1_all.deb ...\n",
            "Unpacking libparse-pmfile-perl (0.41-1) ...\n",
            "Selecting previously unselected package libstring-shellquote-perl.\n",
            "Preparing to unpack .../07-libstring-shellquote-perl_1.04-1_all.deb ...\n",
            "Unpacking libstring-shellquote-perl (1.04-1) ...\n",
            "Selecting previously unselected package cpanminus.\n",
            "Preparing to unpack .../08-cpanminus_1.7043-1_all.deb ...\n",
            "Unpacking cpanminus (1.7043-1) ...\n",
            "Selecting previously unselected package libalgorithm-c3-perl.\n",
            "Preparing to unpack .../09-libalgorithm-c3-perl_0.10-1_all.deb ...\n",
            "Unpacking libalgorithm-c3-perl (0.10-1) ...\n",
            "Selecting previously unselected package libb-hooks-op-check-perl.\n",
            "Preparing to unpack .../10-libb-hooks-op-check-perl_0.22-1_amd64.deb ...\n",
            "Unpacking libb-hooks-op-check-perl (0.22-1) ...\n",
            "Selecting previously unselected package libdynaloader-functions-perl.\n",
            "Preparing to unpack .../11-libdynaloader-functions-perl_0.003-1_all.deb ...\n",
            "Unpacking libdynaloader-functions-perl (0.003-1) ...\n",
            "Selecting previously unselected package libdevel-callchecker-perl.\n",
            "Preparing to unpack .../12-libdevel-callchecker-perl_0.007-2build1_amd64.deb ...\n",
            "Unpacking libdevel-callchecker-perl (0.007-2build1) ...\n",
            "Selecting previously unselected package libparams-classify-perl.\n",
            "Preparing to unpack .../13-libparams-classify-perl_0.015-1_amd64.deb ...\n",
            "Unpacking libparams-classify-perl (0.015-1) ...\n",
            "Selecting previously unselected package libmodule-runtime-perl.\n",
            "Preparing to unpack .../14-libmodule-runtime-perl_0.016-1_all.deb ...\n",
            "Unpacking libmodule-runtime-perl (0.016-1) ...\n",
            "Selecting previously unselected package libtry-tiny-perl.\n",
            "Preparing to unpack .../15-libtry-tiny-perl_0.30-1_all.deb ...\n",
            "Unpacking libtry-tiny-perl (0.30-1) ...\n",
            "Selecting previously unselected package libmodule-implementation-perl.\n",
            "Preparing to unpack .../16-libmodule-implementation-perl_0.09-1_all.deb ...\n",
            "Unpacking libmodule-implementation-perl (0.09-1) ...\n",
            "Selecting previously unselected package libsub-exporter-progressive-perl.\n",
            "Preparing to unpack .../17-libsub-exporter-progressive-perl_0.001013-1_all.deb ...\n",
            "Unpacking libsub-exporter-progressive-perl (0.001013-1) ...\n",
            "Selecting previously unselected package libvariable-magic-perl.\n",
            "Preparing to unpack .../18-libvariable-magic-perl_0.62-1_amd64.deb ...\n",
            "Unpacking libvariable-magic-perl (0.62-1) ...\n",
            "Selecting previously unselected package libb-hooks-endofscope-perl.\n",
            "Preparing to unpack .../19-libb-hooks-endofscope-perl_0.21-1_all.deb ...\n",
            "Unpacking libb-hooks-endofscope-perl (0.21-1) ...\n",
            "Selecting previously unselected package libclass-c3-perl.\n",
            "Preparing to unpack .../20-libclass-c3-perl_0.33-1_all.deb ...\n",
            "Unpacking libclass-c3-perl (0.33-1) ...\n",
            "Selecting previously unselected package libclass-c3-xs-perl.\n",
            "Preparing to unpack .../21-libclass-c3-xs-perl_0.14-1build3_amd64.deb ...\n",
            "Unpacking libclass-c3-xs-perl (0.14-1build3) ...\n",
            "Selecting previously unselected package libclass-method-modifiers-perl.\n",
            "Preparing to unpack .../22-libclass-method-modifiers-perl_2.12-1_all.deb ...\n",
            "Unpacking libclass-method-modifiers-perl (2.12-1) ...\n",
            "Selecting previously unselected package libclass-xsaccessor-perl.\n",
            "Preparing to unpack .../23-libclass-xsaccessor-perl_1.19-2build8_amd64.deb ...\n",
            "Unpacking libclass-xsaccessor-perl (1.19-2build8) ...\n",
            "Selecting previously unselected package libcpan-changes-perl.\n",
            "Preparing to unpack .../24-libcpan-changes-perl_0.400002-1_all.deb ...\n",
            "Unpacking libcpan-changes-perl (0.400002-1) ...\n",
            "Selecting previously unselected package libparams-util-perl.\n",
            "Preparing to unpack .../25-libparams-util-perl_1.07-3build3_amd64.deb ...\n",
            "Unpacking libparams-util-perl (1.07-3build3) ...\n",
            "Selecting previously unselected package libsub-install-perl.\n",
            "Preparing to unpack .../26-libsub-install-perl_0.928-1_all.deb ...\n",
            "Unpacking libsub-install-perl (0.928-1) ...\n",
            "Selecting previously unselected package libdata-optlist-perl.\n",
            "Preparing to unpack .../27-libdata-optlist-perl_0.110-1_all.deb ...\n",
            "Unpacking libdata-optlist-perl (0.110-1) ...\n",
            "Selecting previously unselected package libexporter-tiny-perl.\n",
            "Preparing to unpack .../28-libexporter-tiny-perl_1.000000-2_all.deb ...\n",
            "Unpacking libexporter-tiny-perl (1.000000-2) ...\n",
            "Selecting previously unselected package liblist-moreutils-perl.\n",
            "Preparing to unpack .../29-liblist-moreutils-perl_0.416-1build3_amd64.deb ...\n",
            "Unpacking liblist-moreutils-perl (0.416-1build3) ...\n",
            "Selecting previously unselected package librole-tiny-perl.\n",
            "Preparing to unpack .../30-librole-tiny-perl_2.000006-1_all.deb ...\n",
            "Unpacking librole-tiny-perl (2.000006-1) ...\n",
            "Selecting previously unselected package libstrictures-perl.\n",
            "Preparing to unpack .../31-libstrictures-perl_2.000003-1_all.deb ...\n",
            "Unpacking libstrictures-perl (2.000003-1) ...\n",
            "Selecting previously unselected package libdata-perl-perl.\n",
            "Preparing to unpack .../32-libdata-perl-perl_0.002009-2_all.deb ...\n",
            "Unpacking libdata-perl-perl (0.002009-2) ...\n",
            "Selecting previously unselected package libmro-compat-perl.\n",
            "Preparing to unpack .../33-libmro-compat-perl_0.13-1_all.deb ...\n",
            "Unpacking libmro-compat-perl (0.13-1) ...\n",
            "Selecting previously unselected package libsub-exporter-perl.\n",
            "Preparing to unpack .../34-libsub-exporter-perl_0.987-1_all.deb ...\n",
            "Unpacking libsub-exporter-perl (0.987-1) ...\n",
            "Selecting previously unselected package libdata-section-perl.\n",
            "Preparing to unpack .../35-libdata-section-perl_0.200007-1_all.deb ...\n",
            "Unpacking libdata-section-perl (0.200007-1) ...\n",
            "Selecting previously unselected package libdevel-globaldestruction-perl.\n",
            "Preparing to unpack .../36-libdevel-globaldestruction-perl_0.14-1_all.deb ...\n",
            "Unpacking libdevel-globaldestruction-perl (0.14-1) ...\n",
            "Selecting previously unselected package libencode-locale-perl.\n",
            "Preparing to unpack .../37-libencode-locale-perl_1.05-1_all.deb ...\n",
            "Unpacking libencode-locale-perl (1.05-1) ...\n",
            "Selecting previously unselected package libfile-slurp-perl.\n",
            "Preparing to unpack .../38-libfile-slurp-perl_9999.19-6_all.deb ...\n",
            "Unpacking libfile-slurp-perl (9999.19-6) ...\n",
            "Selecting previously unselected package libio-stringy-perl.\n",
            "Preparing to unpack .../39-libio-stringy-perl_2.111-2_all.deb ...\n",
            "Unpacking libio-stringy-perl (2.111-2) ...\n",
            "Selecting previously unselected package libparams-validate-perl.\n",
            "Preparing to unpack .../40-libparams-validate-perl_1.29-1_amd64.deb ...\n",
            "Unpacking libparams-validate-perl (1.29-1) ...\n",
            "Selecting previously unselected package libgetopt-long-descriptive-perl.\n",
            "Preparing to unpack .../41-libgetopt-long-descriptive-perl_0.102-1_all.deb ...\n",
            "Unpacking libgetopt-long-descriptive-perl (0.102-1) ...\n",
            "Selecting previously unselected package libhtml-tagset-perl.\n",
            "Preparing to unpack .../42-libhtml-tagset-perl_3.20-3_all.deb ...\n",
            "Unpacking libhtml-tagset-perl (3.20-3) ...\n",
            "Selecting previously unselected package liburi-perl.\n",
            "Preparing to unpack .../43-liburi-perl_1.73-1_all.deb ...\n",
            "Unpacking liburi-perl (1.73-1) ...\n",
            "Selecting previously unselected package libhtml-parser-perl.\n",
            "Preparing to unpack .../44-libhtml-parser-perl_3.72-3build1_amd64.deb ...\n",
            "Unpacking libhtml-parser-perl (3.72-3build1) ...\n",
            "Selecting previously unselected package libtimedate-perl.\n",
            "Preparing to unpack .../45-libtimedate-perl_2.3000-2_all.deb ...\n",
            "Unpacking libtimedate-perl (2.3000-2) ...\n",
            "Selecting previously unselected package libhttp-date-perl.\n",
            "Preparing to unpack .../46-libhttp-date-perl_6.02-1_all.deb ...\n",
            "Unpacking libhttp-date-perl (6.02-1) ...\n",
            "Selecting previously unselected package libio-html-perl.\n",
            "Preparing to unpack .../47-libio-html-perl_1.001-1_all.deb ...\n",
            "Unpacking libio-html-perl (1.001-1) ...\n",
            "Selecting previously unselected package liblwp-mediatypes-perl.\n",
            "Preparing to unpack .../48-liblwp-mediatypes-perl_6.02-1_all.deb ...\n",
            "Unpacking liblwp-mediatypes-perl (6.02-1) ...\n",
            "Selecting previously unselected package libhttp-message-perl.\n",
            "Preparing to unpack .../49-libhttp-message-perl_6.14-1_all.deb ...\n",
            "Unpacking libhttp-message-perl (6.14-1) ...\n",
            "Selecting previously unselected package libimport-into-perl.\n",
            "Preparing to unpack .../50-libimport-into-perl_1.002005-1_all.deb ...\n",
            "Unpacking libimport-into-perl (1.002005-1) ...\n",
            "Selecting previously unselected package libmodule-signature-perl.\n",
            "Preparing to unpack .../51-libmodule-signature-perl_0.81-1_all.deb ...\n",
            "Unpacking libmodule-signature-perl (0.81-1) ...\n",
            "Selecting previously unselected package libsub-quote-perl.\n",
            "Preparing to unpack .../52-libsub-quote-perl_2.005000-1_all.deb ...\n",
            "Unpacking libsub-quote-perl (2.005000-1) ...\n",
            "Selecting previously unselected package libmoo-perl.\n",
            "Preparing to unpack .../53-libmoo-perl_2.003004-1_all.deb ...\n",
            "Unpacking libmoo-perl (2.003004-1) ...\n",
            "Selecting previously unselected package libmoox-handlesvia-perl.\n",
            "Preparing to unpack .../54-libmoox-handlesvia-perl_0.001008-3_all.deb ...\n",
            "Unpacking libmoox-handlesvia-perl (0.001008-3) ...\n",
            "Selecting previously unselected package libpackage-stash-perl.\n",
            "Preparing to unpack .../55-libpackage-stash-perl_0.37-1_all.deb ...\n",
            "Unpacking libpackage-stash-perl (0.37-1) ...\n",
            "Selecting previously unselected package libsub-identify-perl.\n",
            "Preparing to unpack .../56-libsub-identify-perl_0.14-1_amd64.deb ...\n",
            "Unpacking libsub-identify-perl (0.14-1) ...\n",
            "Selecting previously unselected package libsub-name-perl.\n",
            "Preparing to unpack .../57-libsub-name-perl_0.21-1build1_amd64.deb ...\n",
            "Unpacking libsub-name-perl (0.21-1build1) ...\n",
            "Selecting previously unselected package libnamespace-clean-perl.\n",
            "Preparing to unpack .../58-libnamespace-clean-perl_0.27-1_all.deb ...\n",
            "Unpacking libnamespace-clean-perl (0.27-1) ...\n",
            "Selecting previously unselected package libnamespace-autoclean-perl.\n",
            "Preparing to unpack .../59-libnamespace-autoclean-perl_0.28-1_all.deb ...\n",
            "Unpacking libnamespace-autoclean-perl (0.28-1) ...\n",
            "Selecting previously unselected package libpackage-stash-xs-perl.\n",
            "Preparing to unpack .../60-libpackage-stash-xs-perl_0.28-3build2_amd64.deb ...\n",
            "Unpacking libpackage-stash-xs-perl (0.28-3build2) ...\n",
            "Selecting previously unselected package libpath-tiny-perl.\n",
            "Preparing to unpack .../61-libpath-tiny-perl_0.104-1_all.deb ...\n",
            "Unpacking libpath-tiny-perl (0.104-1) ...\n",
            "Selecting previously unselected package libpod-markdown-perl.\n",
            "Preparing to unpack .../62-libpod-markdown-perl_3.005000-1_all.deb ...\n",
            "Unpacking libpod-markdown-perl (3.005000-1) ...\n",
            "Selecting previously unselected package libtype-tiny-perl.\n",
            "Preparing to unpack .../63-libtype-tiny-perl_1.002001-1_all.deb ...\n",
            "Unpacking libtype-tiny-perl (1.002001-1) ...\n",
            "Selecting previously unselected package libpod-readme-perl.\n",
            "Preparing to unpack .../64-libpod-readme-perl_1.1.2-2_all.deb ...\n",
            "Unpacking libpod-readme-perl (1.1.2-2) ...\n",
            "Selecting previously unselected package libreadonly-perl.\n",
            "Preparing to unpack .../65-libreadonly-perl_2.050-1_all.deb ...\n",
            "Unpacking libreadonly-perl (2.050-1) ...\n",
            "Selecting previously unselected package libref-util-perl.\n",
            "Preparing to unpack .../66-libref-util-perl_0.203-1_all.deb ...\n",
            "Unpacking libref-util-perl (0.203-1) ...\n",
            "Selecting previously unselected package libref-util-xs-perl.\n",
            "Preparing to unpack .../67-libref-util-xs-perl_0.116-1_amd64.deb ...\n",
            "Unpacking libref-util-xs-perl (0.116-1) ...\n",
            "Selecting previously unselected package libtext-template-perl.\n",
            "Preparing to unpack .../68-libtext-template-perl_1.47-1_all.deb ...\n",
            "Unpacking libtext-template-perl (1.47-1) ...\n",
            "Selecting previously unselected package libsoftware-license-perl.\n",
            "Preparing to unpack .../69-libsoftware-license-perl_0.103012-1_all.deb ...\n",
            "Unpacking libsoftware-license-perl (0.103012-1) ...\n",
            "Selecting previously unselected package libtype-tiny-xs-perl.\n",
            "Preparing to unpack .../70-libtype-tiny-xs-perl_0.012-2_amd64.deb ...\n",
            "Unpacking libtype-tiny-xs-perl (0.012-2) ...\n",
            "Selecting previously unselected package libunicode-utf8-perl.\n",
            "Preparing to unpack .../71-libunicode-utf8-perl_0.60-1build4_amd64.deb ...\n",
            "Unpacking libunicode-utf8-perl (0.60-1build4) ...\n",
            "Setting up libhtml-tagset-perl (3.20-3) ...\n",
            "Setting up libpath-tiny-perl (0.104-1) ...\n",
            "Setting up libclass-c3-xs-perl (0.14-1build3) ...\n",
            "Setting up libsub-install-perl (0.928-1) ...\n",
            "Setting up libio-stringy-perl (2.111-2) ...\n",
            "Setting up libcpan-distnameinfo-perl (0.12-1) ...\n",
            "Setting up libsub-exporter-progressive-perl (0.001013-1) ...\n",
            "Setting up libtry-tiny-perl (0.30-1) ...\n",
            "Setting up libfile-slurp-perl (9999.19-6) ...\n",
            "Setting up libclass-method-modifiers-perl (2.12-1) ...\n",
            "Setting up libencode-locale-perl (1.05-1) ...\n",
            "Setting up libvariable-magic-perl (0.62-1) ...\n",
            "Setting up libtimedate-perl (2.3000-2) ...\n",
            "Setting up librole-tiny-perl (2.000006-1) ...\n",
            "Setting up libstring-shellquote-perl (1.04-1) ...\n",
            "Setting up libb-hooks-op-check-perl (0.22-1) ...\n",
            "Setting up libio-html-perl (1.001-1) ...\n",
            "Setting up libexporter-tiny-perl (1.000000-2) ...\n",
            "Setting up libmodule-cpanfile-perl (1.1002-1) ...\n",
            "Setting up libtype-tiny-xs-perl (0.012-2) ...\n",
            "Setting up libsub-identify-perl (0.14-1) ...\n",
            "Setting up libref-util-perl (0.203-1) ...\n",
            "Setting up libdynaloader-functions-perl (0.003-1) ...\n",
            "Setting up libparse-pmfile-perl (0.41-1) ...\n",
            "Setting up libfile-pushd-perl (1.014-1) ...\n",
            "Setting up libcpan-meta-check-perl (0.014-1) ...\n",
            "Setting up libpackage-stash-xs-perl (0.28-3build2) ...\n",
            "Setting up libstrictures-perl (2.000003-1) ...\n",
            "Setting up libmodule-signature-perl (0.81-1) ...\n",
            "Setting up libreadonly-perl (2.050-1) ...\n",
            "Setting up liblwp-mediatypes-perl (6.02-1) ...\n",
            "Setting up libref-util-xs-perl (0.116-1) ...\n",
            "Setting up liburi-perl (1.73-1) ...\n",
            "Setting up libhtml-parser-perl (3.72-3build1) ...\n",
            "Setting up libdevel-globaldestruction-perl (0.14-1) ...\n",
            "Setting up libpod-markdown-perl (3.005000-1) ...\n",
            "Setting up libclass-xsaccessor-perl (1.19-2build8) ...\n",
            "Setting up libcpan-changes-perl (0.400002-1) ...\n",
            "Setting up libunicode-utf8-perl (0.60-1build4) ...\n",
            "Setting up libmodule-build-perl (0.422400-1) ...\n",
            "Setting up libparams-util-perl (1.07-3build3) ...\n",
            "Setting up libsub-name-perl (0.21-1build1) ...\n",
            "Setting up libsub-quote-perl (2.005000-1) ...\n",
            "Setting up libtext-template-perl (1.47-1) ...\n",
            "Setting up libalgorithm-c3-perl (0.10-1) ...\n",
            "Setting up libclass-c3-perl (0.33-1) ...\n",
            "Setting up liblocal-lib-perl (2.000024-1) ...\n",
            "Setting up liblist-moreutils-perl (0.416-1build3) ...\n",
            "Setting up libdata-optlist-perl (0.110-1) ...\n",
            "Setting up libhttp-date-perl (6.02-1) ...\n",
            "Setting up libtype-tiny-perl (1.002001-1) ...\n",
            "Setting up libdevel-callchecker-perl (0.007-2build1) ...\n",
            "Setting up cpanminus (1.7043-1) ...\n",
            "Setting up libmro-compat-perl (0.13-1) ...\n",
            "Setting up libsub-exporter-perl (0.987-1) ...\n",
            "Setting up libhttp-message-perl (6.14-1) ...\n",
            "Setting up libparams-classify-perl (0.015-1) ...\n",
            "Setting up libdata-section-perl (0.200007-1) ...\n",
            "Setting up libmodule-runtime-perl (0.016-1) ...\n",
            "Setting up libsoftware-license-perl (0.103012-1) ...\n",
            "Setting up libdata-perl-perl (0.002009-2) ...\n",
            "Setting up libimport-into-perl (1.002005-1) ...\n",
            "Setting up libmodule-implementation-perl (0.09-1) ...\n",
            "Setting up libparams-validate-perl (1.29-1) ...\n",
            "Setting up libmoo-perl (2.003004-1) ...\n",
            "Setting up libb-hooks-endofscope-perl (0.21-1) ...\n",
            "Setting up libpackage-stash-perl (0.37-1) ...\n",
            "Setting up libgetopt-long-descriptive-perl (0.102-1) ...\n",
            "Setting up libmoox-handlesvia-perl (0.001008-3) ...\n",
            "Setting up libnamespace-clean-perl (0.27-1) ...\n",
            "Setting up libnamespace-autoclean-perl (0.28-1) ...\n",
            "Setting up libpod-readme-perl (1.1.2-2) ...\n",
            "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...\n",
            "--> Working on XML::Parser\n",
            "Fetching http://www.cpan.org/authors/id/T/TO/TODDR/XML-Parser-2.46.tar.gz ... OK\n",
            "Configuring XML-Parser-2.46 ... OK\n",
            "==> Found dependencies: LWP::UserAgent\n",
            "--> Working on LWP::UserAgent\n",
            "Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/libwww-perl-6.54.tar.gz ... OK\n",
            "Configuring libwww-perl-6.54 ... OK\n",
            "==> Found dependencies: File::Listing, Test::RequiresInternet, Test::Needs, Net::HTTP, HTTP::Daemon, HTTP::Cookies, HTTP::Negotiate, Test::Fatal, WWW::RobotRules\n",
            "--> Working on File::Listing\n",
            "Fetching http://www.cpan.org/authors/id/P/PL/PLICEASE/File-Listing-6.14.tar.gz ... OK\n",
            "Configuring File-Listing-6.14 ... OK\n",
            "Building and testing File-Listing-6.14 ... OK\n",
            "Successfully installed File-Listing-6.14\n",
            "--> Working on Test::RequiresInternet\n",
            "Fetching http://www.cpan.org/authors/id/M/MA/MALLEN/Test-RequiresInternet-0.05.tar.gz ... OK\n",
            "Configuring Test-RequiresInternet-0.05 ... OK\n",
            "Building and testing Test-RequiresInternet-0.05 ... OK\n",
            "Successfully installed Test-RequiresInternet-0.05\n",
            "--> Working on Test::Needs\n",
            "Fetching http://www.cpan.org/authors/id/H/HA/HAARG/Test-Needs-0.002009.tar.gz ... OK\n",
            "Configuring Test-Needs-0.002009 ... OK\n",
            "Building and testing Test-Needs-0.002009 ... OK\n",
            "Successfully installed Test-Needs-0.002009\n",
            "--> Working on Net::HTTP\n",
            "Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/Net-HTTP-6.21.tar.gz ... OK\n",
            "Configuring Net-HTTP-6.21 ... OK\n",
            "Building and testing Net-HTTP-6.21 ... OK\n",
            "Successfully installed Net-HTTP-6.21\n",
            "--> Working on HTTP::Daemon\n",
            "Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Daemon-6.12.tar.gz ... OK\n",
            "==> Found dependencies: Module::Build::Tiny\n",
            "--> Working on Module::Build::Tiny\n",
            "Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Module-Build-Tiny-0.039.tar.gz ... OK\n",
            "==> Found dependencies: ExtUtils::Helpers, ExtUtils::Config, ExtUtils::InstallPaths\n",
            "--> Working on ExtUtils::Helpers\n",
            "Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Helpers-0.026.tar.gz ... OK\n",
            "Configuring ExtUtils-Helpers-0.026 ... OK\n",
            "Building and testing ExtUtils-Helpers-0.026 ... OK\n",
            "Successfully installed ExtUtils-Helpers-0.026\n",
            "--> Working on ExtUtils::Config\n",
            "Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Config-0.008.tar.gz ... OK\n",
            "Configuring ExtUtils-Config-0.008 ... OK\n",
            "Building and testing ExtUtils-Config-0.008 ... OK\n",
            "Successfully installed ExtUtils-Config-0.008\n",
            "--> Working on ExtUtils::InstallPaths\n",
            "Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-InstallPaths-0.012.tar.gz ... OK\n",
            "Configuring ExtUtils-InstallPaths-0.012 ... OK\n",
            "Building and testing ExtUtils-InstallPaths-0.012 ... OK\n",
            "Successfully installed ExtUtils-InstallPaths-0.012\n",
            "Configuring Module-Build-Tiny-0.039 ... OK\n",
            "Building and testing Module-Build-Tiny-0.039 ... OK\n",
            "Successfully installed Module-Build-Tiny-0.039\n",
            "Configuring HTTP-Daemon-6.12 ... OK\n",
            "Building and testing HTTP-Daemon-6.12 ... OK\n",
            "Successfully installed HTTP-Daemon-6.12\n",
            "--> Working on HTTP::Cookies\n",
            "Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Cookies-6.10.tar.gz ... OK\n",
            "Configuring HTTP-Cookies-6.10 ... OK\n",
            "Building and testing HTTP-Cookies-6.10 ... OK\n",
            "Successfully installed HTTP-Cookies-6.10\n",
            "--> Working on HTTP::Negotiate\n",
            "Fetching http://www.cpan.org/authors/id/G/GA/GAAS/HTTP-Negotiate-6.01.tar.gz ... OK\n",
            "Configuring HTTP-Negotiate-6.01 ... OK\n",
            "Building and testing HTTP-Negotiate-6.01 ... OK\n",
            "Successfully installed HTTP-Negotiate-6.01\n",
            "--> Working on Test::Fatal\n",
            "Fetching http://www.cpan.org/authors/id/R/RJ/RJBS/Test-Fatal-0.016.tar.gz ... OK\n",
            "Configuring Test-Fatal-0.016 ... OK\n",
            "Building and testing Test-Fatal-0.016 ... OK\n",
            "Successfully installed Test-Fatal-0.016\n",
            "--> Working on WWW::RobotRules\n",
            "Fetching http://www.cpan.org/authors/id/G/GA/GAAS/WWW-RobotRules-6.02.tar.gz ... OK\n",
            "Configuring WWW-RobotRules-6.02 ... OK\n",
            "Building and testing WWW-RobotRules-6.02 ... OK\n",
            "Successfully installed WWW-RobotRules-6.02\n",
            "Building and testing libwww-perl-6.54 ... OK\n",
            "Successfully installed libwww-perl-6.54\n",
            "Building and testing XML-Parser-2.46 ... OK\n",
            "Successfully installed XML-Parser-2.46\n",
            "15 distributions installed\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2FPCb2oUGckw",
        "outputId": "f1139c7a-2e78-40d2-b2d1-0fa76dc54188"
      },
      "source": [
        "!rouge-metric sample/hypotheses/summary1.txt sample/references/summary1.1.txt -n 2 -w 1.2 -U -2 4"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "---------------------------------------------\n",
            "A ROUGE-1 Average_R: 0.15646 (95%-conf.int. 0.15646 - 0.15646)\n",
            "A ROUGE-1 Average_P: 0.40708 (95%-conf.int. 0.40708 - 0.40708)\n",
            "A ROUGE-1 Average_F: 0.22604 (95%-conf.int. 0.22604 - 0.22604)\n",
            "---------------------------------------------\n",
            "A ROUGE-2 Average_R: 0.02730 (95%-conf.int. 0.02730 - 0.02730)\n",
            "A ROUGE-2 Average_P: 0.07143 (95%-conf.int. 0.07143 - 0.07143)\n",
            "A ROUGE-2 Average_F: 0.03950 (95%-conf.int. 0.03950 - 0.03950)\n",
            "---------------------------------------------\n",
            "A ROUGE-L Average_R: 0.14286 (95%-conf.int. 0.14286 - 0.14286)\n",
            "A ROUGE-L Average_P: 0.37168 (95%-conf.int. 0.37168 - 0.37168)\n",
            "A ROUGE-L Average_F: 0.20639 (95%-conf.int. 0.20639 - 0.20639)\n",
            "---------------------------------------------\n",
            "A ROUGE-W-1.2 Average_R: 0.05666 (95%-conf.int. 0.05666 - 0.05666)\n",
            "A ROUGE-W-1.2 Average_P: 0.20818 (95%-conf.int. 0.20818 - 0.20818)\n",
            "A ROUGE-W-1.2 Average_F: 0.08908 (95%-conf.int. 0.08908 - 0.08908)\n",
            "---------------------------------------------\n",
            "A ROUGE-S4 Average_R: 0.02474 (95%-conf.int. 0.02474 - 0.02474)\n",
            "A ROUGE-S4 Average_P: 0.06545 (95%-conf.int. 0.06545 - 0.06545)\n",
            "A ROUGE-S4 Average_F: 0.03591 (95%-conf.int. 0.03591 - 0.03591)\n",
            "---------------------------------------------\n",
            "A ROUGE-SU4 Average_R: 0.04634 (95%-conf.int. 0.04634 - 0.04634)\n",
            "A ROUGE-SU4 Average_P: 0.12236 (95%-conf.int. 0.12236 - 0.12236)\n",
            "A ROUGE-SU4 Average_F: 0.06722 (95%-conf.int. 0.06722 - 0.06722)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iU92vcqMO-SR",
        "outputId": "413a809f-96b7-429e-85ae-202d72facb53"
      },
      "source": [
        "!rouge-metric sample/hypotheses/summary1.txt sample/references/summary1.1.txt sample/references/17_896312_8.txt sample/references/17_896313_1.txt -n 2 -w 1.2 -U -2 4"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "---------------------------------------------\n",
            "A ROUGE-1 Average_R: 0.15646 (95%-conf.int. 0.15646 - 0.15646)\n",
            "A ROUGE-1 Average_P: 0.40708 (95%-conf.int. 0.40708 - 0.40708)\n",
            "A ROUGE-1 Average_F: 0.22604 (95%-conf.int. 0.22604 - 0.22604)\n",
            "---------------------------------------------\n",
            "A ROUGE-2 Average_R: 0.02730 (95%-conf.int. 0.02730 - 0.02730)\n",
            "A ROUGE-2 Average_P: 0.07143 (95%-conf.int. 0.07143 - 0.07143)\n",
            "A ROUGE-2 Average_F: 0.03950 (95%-conf.int. 0.03950 - 0.03950)\n",
            "---------------------------------------------\n",
            "A ROUGE-L Average_R: 0.14286 (95%-conf.int. 0.14286 - 0.14286)\n",
            "A ROUGE-L Average_P: 0.37168 (95%-conf.int. 0.37168 - 0.37168)\n",
            "A ROUGE-L Average_F: 0.20639 (95%-conf.int. 0.20639 - 0.20639)\n",
            "---------------------------------------------\n",
            "A ROUGE-W-1.2 Average_R: 0.05666 (95%-conf.int. 0.05666 - 0.05666)\n",
            "A ROUGE-W-1.2 Average_P: 0.20818 (95%-conf.int. 0.20818 - 0.20818)\n",
            "A ROUGE-W-1.2 Average_F: 0.08908 (95%-conf.int. 0.08908 - 0.08908)\n",
            "---------------------------------------------\n",
            "A ROUGE-S4 Average_R: 0.02474 (95%-conf.int. 0.02474 - 0.02474)\n",
            "A ROUGE-S4 Average_P: 0.06545 (95%-conf.int. 0.06545 - 0.06545)\n",
            "A ROUGE-S4 Average_F: 0.03591 (95%-conf.int. 0.03591 - 0.03591)\n",
            "---------------------------------------------\n",
            "A ROUGE-SU4 Average_R: 0.04634 (95%-conf.int. 0.04634 - 0.04634)\n",
            "A ROUGE-SU4 Average_P: 0.12236 (95%-conf.int. 0.12236 - 0.12236)\n",
            "A ROUGE-SU4 Average_F: 0.06722 (95%-conf.int. 0.06722 - 0.06722)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cvZCaKGWLXuY",
        "outputId": "a7d33988-c867-4dc6-c596-f9d28e020c7b"
      },
      "source": [
        "!rouge-metric sample/hypotheses/ sample/references/ -n 2 -w 1.2 -U -2 4"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "---------------------------------------------\n",
            "A ROUGE-1 Average_R: 0.51822 (95%-conf.int. 0.42105 - 0.61538)\n",
            "A ROUGE-1 Average_P: 0.55556 (95%-conf.int. 0.44444 - 0.66667)\n",
            "A ROUGE-1 Average_F: 0.53622 (95%-conf.int. 0.43243 - 0.64000)\n",
            "---------------------------------------------\n",
            "A ROUGE-2 Average_R: 0.19519 (95%-conf.int. 0.11765 - 0.27273)\n",
            "A ROUGE-2 Average_P: 0.21250 (95%-conf.int. 0.12500 - 0.30000)\n",
            "A ROUGE-2 Average_F: 0.20346 (95%-conf.int. 0.12121 - 0.28572)\n",
            "---------------------------------------------\n",
            "A ROUGE-L Average_R: 0.51822 (95%-conf.int. 0.42105 - 0.61538)\n",
            "A ROUGE-L Average_P: 0.55556 (95%-conf.int. 0.44444 - 0.66667)\n",
            "A ROUGE-L Average_F: 0.53622 (95%-conf.int. 0.43243 - 0.64000)\n",
            "---------------------------------------------\n",
            "A ROUGE-W-1.2 Average_R: 0.33608 (95%-conf.int. 0.26618 - 0.40599)\n",
            "A ROUGE-W-1.2 Average_P: 0.47348 (95%-conf.int. 0.38525 - 0.56172)\n",
            "A ROUGE-W-1.2 Average_F: 0.39308 (95%-conf.int. 0.31483 - 0.47132)\n",
            "---------------------------------------------\n",
            "A ROUGE-S4 Average_R: 0.25495 (95%-conf.int. 0.13846 - 0.37143)\n",
            "A ROUGE-S4 Average_P: 0.29167 (95%-conf.int. 0.15000 - 0.43333)\n",
            "A ROUGE-S4 Average_F: 0.27200 (95%-conf.int. 0.14400 - 0.40000)\n",
            "---------------------------------------------\n",
            "A ROUGE-SU4 Average_R: 0.31495 (95%-conf.int. 0.19512 - 0.43478)\n",
            "A ROUGE-SU4 Average_P: 0.35527 (95%-conf.int. 0.21053 - 0.50000)\n",
            "A ROUGE-SU4 Average_F: 0.33382 (95%-conf.int. 0.20253 - 0.46511)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FDImW-tBgMqQ"
      },
      "source": [
        "## 5.3 extracting summaries\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aZxe4SjSAWWz",
        "outputId": "b9efd875-869b-4c3e-8d0e-d775709957b4"
      },
      "source": [
        "cd /content/drive/MyDrive"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Sb1_4FQgoyPY",
        "outputId": "b3e1d306-9902-4932-f966-14f0cbeb6cc5"
      },
      "source": [
        "!git clone 'https://github.com/chriskhanhtran/bert-extractive-summarization'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'bert-extractive-summarization'...\n",
            "remote: Enumerating objects: 239, done.\u001b[K\n",
            "remote: Total 239 (delta 0), reused 0 (delta 0), pack-reused 239\u001b[K\n",
            "Receiving objects: 100% (239/239), 321.37 KiB | 7.65 MiB/s, done.\n",
            "Resolving deltas: 100% (123/123), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SD_dEfyspCPu",
        "outputId": "5adbc12c-4e4c-4245-d732-774890cd7ec1"
      },
      "source": [
        "cd bert-extractive-summarization/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive/bert-extractive-summarization\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8BZ_UILPpM4e",
        "outputId": "26bbed35-fe62-4bf4-a67c-bfdd58734816"
      },
      "source": [
        "!pip install boto3"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: boto3 in /usr/local/lib/python3.7/dist-packages (1.17.86)\n",
            "Requirement already satisfied: s3transfer<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from boto3) (0.4.2)\n",
            "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from boto3) (0.10.0)\n",
            "Requirement already satisfied: botocore<1.21.0,>=1.20.86 in /usr/local/lib/python3.7/dist-packages (from boto3) (1.20.86)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/local/lib/python3.7/dist-packages (from botocore<1.21.0,>=1.20.86->boto3) (1.26.5)\n",
            "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/dist-packages (from botocore<1.21.0,>=1.20.86->boto3) (2.8.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.21.0,>=1.20.86->boto3) (1.15.0)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PCuv9lEZpouZ",
        "outputId": "036d454e-4e48-44d7-e85a-ee4243a6a7d3"
      },
      "source": [
        "!wget -O \"checkpoints/bertbase_ext.pt\" \"https://www.googleapis.com/drive/v3/files/1t27zkFMUnuqRcsqf2fh8F1RwaqFoMw5e?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\"\n",
        "!wget -O \"checkpoints/distilbert_ext.pt\" \"https://www.googleapis.com/drive/v3/files/1WxU7cHECfYaU32oTM0JByTRGS5f6SYEF?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\"\n",
        "!wget -O \"checkpoints/mobilebert_ext.pt\" \"https://www.googleapis.com/drive/v3/files/1umMOXoueo38zID_AKFSIOGxG9XjS5hDC?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\"\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2021-06-03 03:55:25--  https://www.googleapis.com/drive/v3/files/1t27zkFMUnuqRcsqf2fh8F1RwaqFoMw5e?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\n",
            "Resolving www.googleapis.com (www.googleapis.com)... 142.250.73.234, 142.250.65.74, 142.250.81.202, ...\n",
            "Connecting to www.googleapis.com (www.googleapis.com)|142.250.73.234|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 497468883 (474M) [application/octet-stream]\n",
            "Saving to: ‘checkpoints/bertbase_ext.pt’\n",
            "\n",
            "checkpoints/bertbas 100%[===================>] 474.42M  55.3MB/s    in 9.1s    \n",
            "\n",
            "2021-06-03 03:55:35 (52.2 MB/s) - ‘checkpoints/bertbase_ext.pt’ saved [497468883/497468883]\n",
            "\n",
            "--2021-06-03 03:55:35--  https://www.googleapis.com/drive/v3/files/1WxU7cHECfYaU32oTM0JByTRGS5f6SYEF?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\n",
            "Resolving www.googleapis.com (www.googleapis.com)... 172.217.9.202, 172.217.12.234, 172.217.164.170, ...\n",
            "Connecting to www.googleapis.com (www.googleapis.com)|172.217.9.202|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 324966717 (310M) [application/octet-stream]\n",
            "Saving to: ‘checkpoints/distilbert_ext.pt’\n",
            "\n",
            "checkpoints/distilb 100%[===================>] 309.91M  57.2MB/s    in 5.8s    \n",
            "\n",
            "2021-06-03 03:55:41 (53.7 MB/s) - ‘checkpoints/distilbert_ext.pt’ saved [324966717/324966717]\n",
            "\n",
            "--2021-06-03 03:55:41--  https://www.googleapis.com/drive/v3/files/1umMOXoueo38zID_AKFSIOGxG9XjS5hDC?alt=media&key=AIzaSyCmo6sAQ37OK8DK4wnT94PoLx5lx-7VTDE\n",
            "Resolving www.googleapis.com (www.googleapis.com)... 172.217.164.138, 142.250.73.202, 142.250.73.234, ...\n",
            "Connecting to www.googleapis.com (www.googleapis.com)|172.217.164.138|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 134098110 (128M) [application/octet-stream]\n",
            "Saving to: ‘checkpoints/mobilebert_ext.pt’\n",
            "\n",
            "checkpoints/mobileb 100%[===================>] 127.89M  39.0MB/s    in 3.9s    \n",
            "\n",
            "2021-06-03 03:55:45 (32.7 MB/s) - ‘checkpoints/mobilebert_ext.pt’ saved [134098110/134098110]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R3QylqtMCVv0",
        "outputId": "4262f1cc-e1fa-46aa-e42c-91878746f740"
      },
      "source": [
        "cd /content/drive/MyDrive/bert-extractive-summarization"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive/bert-extractive-summarization\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "orUGtuo7fiEs",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 217,
          "referenced_widgets": [
            "a635a56a0fbc4fbe89fce126f85581df",
            "c2824da7bd4e420b8e53b81c5292686e",
            "ee7ec61caa684b6889e620c110e36f11",
            "3a4b9b5c4d0c40ae983b5a2f066b989b",
            "d60734a66bdf43ea9431d9508e6e9ea8",
            "ff82763f04af41c8a6a9676e8fa1267a",
            "ee6155867eae4db6b2b1519df2923c16",
            "35d68fdcda004bb4be57e4c55668a0b9",
            "ac195b0b1358441c8125399eec8d7029",
            "4dea90beeb6f42c8b11db84926732aec",
            "88b69fffc0f14828abd69929e5687a50",
            "3736cbd82892435899c41576b71c1fa0",
            "a29ffc90783f4c39a4ef6f181816647f",
            "8b65ab94183f438faf12a804cc174745",
            "4e72af7a236f4befa32f4dfcd370b693",
            "c16a22ebcdf64535b4cbb75e3e068b65",
            "fee69f74e79e459c875d6a450e184ea0",
            "06cd415af14a4fc3a7247db511d0b0fa",
            "772f2c1a5ec0483689f3400e6ce03a2f",
            "a7c5c1bc0de440679811e32f99fd4936",
            "8805e240428a4c0db64dc5db5926dd45",
            "5f27e1ce748545f29d3ebcaad08b8bb8",
            "3cbe0efc6cfd46ca9a0be14cc04409d4",
            "a7f1e3701ed04726ace0897f52dbbdf3"
          ]
        },
        "outputId": "9bd30023-620a-47fa-b9c9-643cf64ab1ee"
      },
      "source": [
        "import torch\n",
        "from models.model_builder import ExtSummarizer\n",
        "from ext_sum import summarize\n",
        "import os             \n",
        "print(os.getcwd())\n",
        "\n",
        "# Load model\n",
        "model_type = 'distilbert' #@param ['bertbase', 'distilbert', 'mobilebert']\n",
        "checkpoint = torch.load(f'checkpoints/{model_type}_ext.pt', map_location='cpu')\n",
        "model = ExtSummarizer(checkpoint=checkpoint, bert_type=model_type, device='cpu')\n",
        "result_fp = '17_896312_8.txt'\n",
        "path = \"raw_data\"\n",
        "os.chdir(path)\n",
        "  \n",
        "# Read text File\n",
        "  \n",
        "  \n",
        "def read_text_file(file_path):\n",
        "    with open(file_path, 'r') as f:\n",
        "        print(f.read())\n",
        "  \n",
        "  \n",
        "# iterate through all file\n",
        "for file in os.listdir():\n",
        "    # Check whether file is in text format or not\n",
        "    if file.endswith(\".txt\"):\n",
        "        file_path = f\"{file}\"\n",
        "  \n",
        "        # call read text file function\n",
        "        #input_fp = read_text_file(file_path)\n",
        "        summary = summarize(file_path, result_fp, model, max_length=3)\n",
        "        print(summary)\n",
        "\n",
        "path = \"/content/drive/MyDrive/bert-extractive-summarization\"\n",
        "#print(all_files)\n",
        "# Change the directory\n",
        "os.chdir(path) \n",
        "\n",
        "\n",
        "  \n",
        "#summary = summarize(input_fp, result_fp, model, max_length=3)\n",
        "#print(summary)\n",
        "#input_fp = 'raw_data/17.txt'\n",
        "#result_fp = 'results/17_896311_8.txt'\n",
        "#summary = summarize(input_fp, result_fp, model, max_length=3)\n",
        "#print(summary)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/drive/MyDrive/bert-extractive-summarization\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a635a56a0fbc4fbe89fce126f85581df",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=231508.0, style=ProgressStyle(descripti…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ac195b0b1358441c8125399eec8d7029",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=28.0, style=ProgressStyle(description_w…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "fee69f74e79e459c875d6a450e184ea0",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=466062.0, style=ProgressStyle(descripti…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "We have laid the foundations for the future this year through  sizeable investment in research and development. We are pleased to report that we  have achieved success in both endeavours. We have  also made signiﬁ cant investment in our product portfolio,  both by licensing new technologies to commercialise  and by developing in-house proprietary technologies to  address the needs of mass consumer markets.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "40gdiYNIOifY"
      },
      "source": [
        "# Citation\n",
        "\n",
        "- [visual notebook for first time users](http://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/)\n",
        "\n",
        "Part of my codes and illustrations are adapted from this very easy-to-understand notebook.\n",
        "\n",
        "- [BERT Explained](https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270)\n",
        "\n",
        "This blog post explains BERT very clearly. It helps me a lot when I write the introduction.\n",
        "\n",
        "- [Fine-tuning BERT classification with PyTorch](https://mccormickml.com/2019/07/22/BERT-fine-tuning/)\n",
        "\n",
        "I really like this blog post and its colab notebook. Especially the explanation about tokenization, and data formatting, which can be confusing for first-time users.\n",
        "\n",
        "- [The original BERT paper](https://arxiv.org/pdf/1810.04805.pdf)\n",
        "\n",
        "- [The DiltilBERT paper](https://arxiv.org/pdf/1910.01108.pdf)\n",
        "\n",
        "- [Huggingface library](https://github.com/huggingface/transformers)"
      ]
    }
  ]
}