05_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assignment 5: Link Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** IMPORTANT NOTE ** Due to a problem with the citations file (which would take too much time for me to re-generate before the start of the assignment), we use a dataset from previous year for this assignment. Please upload the four 'emotion' files from Canvas in your data folder.\n",
    "\n",
    "\n",
    "In this assignment, we will use network-based algorithms, such as PageRank, to improve our search results. After the last assignment on evaluating IR systems, we go back to our PubMed dataset of scientific papers. In this dataset, we look at two graphs in particular: the co-authorship network and the citation network.\n",
    "\n",
    "The citation network is similar to the link network of the web: Citations are like web links pointing to other documents. We can therefore apply the same network-based ranking methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code from previous assignments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python version: 3.8.10 (default, Nov  7 2024, 13:10:47) \n",
      "[GCC 9.4.0]\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "print(\"Python version:\", sys.version)\n",
    "\n",
    "import pickle, bz2\n",
    "from collections import defaultdict, namedtuple, Counter\n",
    "from math import log10, sqrt\n",
    "from IPython.display import display, HTML\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# show plots inline within the notebook\n",
    "%matplotlib inline\n",
    "# set plots' resolution\n",
    "plt.rcParams['savefig.dpi'] = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "Ids_file = 'data/emotion_Ids.pkl.bz2'\n",
    "Summaries_file = 'data/emotion_Summaries.pkl.bz2'\n",
    "Citations_file = 'data/emotion_Citations.pkl.bz2'\n",
    "Abstracts_file = 'data/emotion_Abstracts.pkl.bz2'\n",
    "\n",
    "Ids = pickle.load( bz2.BZ2File( Ids_file, 'rb' ) )\n",
    "Summaries = pickle.load( bz2.BZ2File( Summaries_file, 'rb' ) )\n",
    "Citations = pickle.load( bz2.BZ2File( Citations_file, 'rb' ) )\n",
    "Abstracts = pickle.load( bz2.BZ2File( Abstracts_file, 'rb' ) )\n",
    "\n",
    "paper = namedtuple( 'paper', ['title', 'authors', 'year', 'doi'] )\n",
    "\n",
    "for (id, paper_info) in Summaries.items():\n",
    "    Summaries[id] = paper( *paper_info )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def display_summary( id, show_abstract=False, show_id=True, extra_text='' ):\n",
    "    \"\"\"\n",
    "    Function for printing a paper's summary through IPython's Rich Display System.\n",
    "    Trims long author lists, and adds a link to the paper's DOI (when available).\n",
    "    \"\"\"\n",
    "    s = Summaries[id]\n",
    "    lines = []\n",
    "    title = s.title\n",
    "    if s.doi != '':\n",
    "        title = '<a href=http://dx.doi.org/{:s}>{:s}</a>'.format(s.doi, title)\n",
    "    title = '<strong>' + title + '</strong>'\n",
    "    lines.append(title)\n",
    "    authors = ', '.join( s.authors[:20] ) + ('' if len(s.authors) <= 20 else ', ...')\n",
    "    lines.append(str(s.year) + '. ' + authors)\n",
    "    if (show_abstract):\n",
    "        lines.append('<small><strong>Abstract:</strong> <em>{:s}</em></small>'.format(Abstracts[id]))\n",
    "    if (show_id):\n",
    "        lines.append('[ID: {:d}]'.format(id))\n",
    "    if (extra_text != ''):\n",
    "         lines.append(extra_text)\n",
    "    display( HTML('<br>'.join(lines)) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def tokenize(text):\n",
    "    return text.split(' ')\n",
    "\n",
    "def preprocess(tokens):\n",
    "    result = []\n",
    "    for token in tokens:\n",
    "        result.append(token.lower())\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "inverted_index = defaultdict(list)\n",
    "\n",
    "for id in sorted(Summaries.keys()):\n",
    "    term_set = set(preprocess(tokenize(Summaries[id].title)))\n",
    "    if id in Abstracts:\n",
    "        term_set.update(preprocess(tokenize(Abstracts[id])))\n",
    "    for term in term_set:\n",
    "        inverted_index[term].append(id)\n",
    "\n",
    "tf_matrix = defaultdict(Counter)\n",
    "\n",
    "for doc_id in Summaries.keys():\n",
    "    tokens = preprocess(tokenize(Summaries[doc_id].title))\n",
    "    if (doc_id in Abstracts):\n",
    "        tokens.extend(preprocess(tokenize(Abstracts[doc_id])))\n",
    "    tf_matrix[doc_id] = Counter(tokens)\n",
    "\n",
    "def tf(t,d):\n",
    "    return float(tf_matrix[d][t])\n",
    "\n",
    "def df(t):\n",
    "    return float(len(inverted_index[t]))\n",
    "\n",
    "num_documents = float(len(Summaries))\n",
    "\n",
    "def idf(t):\n",
    "    return log10((num_documents + 1)/(df(t) + 1))\n",
    "\n",
    "def tfidf(t,d):\n",
    "    return tf(t,d) * idf(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Co-authorship network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by building a mapping from authors to the set of identifiers of papers they authored.  We'll be using Python sets again for that purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "papers_of_author = defaultdict(set)\n",
    "\n",
    "for (id, p) in Summaries.items():\n",
    "    for a in p.authors:\n",
    "        papers_of_author[a].add(id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{9309949, 17760288, 31649584}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "papers_of_author['Eriksen HR']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.3389/fpsyg.2019.02233>Occupational Rehabilitation Is Associated With Improvements in Cognitive Functioning.</a></strong><br>2019. Johansen T, Jensen C, Eriksen HR, Lyby PS, Dittrich WH, Holsen IN, Jakobsen H, Øyeflaten I<br>[ID: 31649584]"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong>Psychological selection of Antarctic personnel: the \"SOAP\" instrument.</strong><br>2007. Grant I, Eriksen HR, Marquis P, Orre IJ, Palinkas LA, Suedfeld P, Svensen E, Ursin H<br>[ID: 17760288]"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1111/1467-9450.00025>The CODE: a revised battery for coping and defense and its relations to subjective health.</a></strong><br>1997. Eriksen HR, Olff M, Ursin H<br>[ID: 9309949]"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for id in papers_of_author['Eriksen HR']:\n",
    "    display_summary(id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now build a co-authorship network, that is a graph linking authors to the set of co-authors they have published with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "coauthors = defaultdict(set)\n",
    "\n",
    "for p in Summaries.values():\n",
    "    for a in p.authors:\n",
    "        coauthors[a].update(p.authors)\n",
    "\n",
    "# The code above results in each author being listed as having co-authored with himself/herself.\n",
    "# We remove these self-references here:\n",
    "for (a, ca) in coauthors.items():\n",
    "    ca.remove(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's try it out again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Johansen T, Holsen IN, Jakobsen H, Orre IJ, Palinkas LA, Suedfeld P, Øyeflaten I, Olff M, Svensen E, Ursin H, Marquis P, Jensen C, Lyby PS, Grant I, Dittrich WH\n"
     ]
    }
   ],
   "source": [
    "print(', '.join( coauthors['Eriksen HR'] ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike a citation or link network, the edges of this co-authorship network are not directed: There is no direction (no arrow) in the link between author 'Eriksen HR' and 'Suedfeld P', for example. With our chosen implementation, each of these links in fact appears twice in our data, as we also get 'Eriksen HR' as co-author when we look for 'Suedfeld P':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Palinkas LA, Eriksen HR, Orre IJ, Svensen E, Ursin H, Marquis P, Grant I\n"
     ]
    }
   ],
   "source": [
    "print(', '.join( coauthors['Suedfeld P'] ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With that in mind, we can calculate some basic statistics about our graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of nodes (authors):  99675\n",
      "Number of links (co-authorship relations):  590848\n"
     ]
    }
   ],
   "source": [
    "print('Number of nodes (authors): ', len(coauthors))\n",
    "\n",
    "# We divide by two here to account for the fact that each edge is represented twice (see above):\n",
    "coauthor_rel_count = int(sum( len(c) for c in coauthors.values() ) / 2)\n",
    "print('Number of links (co-authorship relations): ', coauthor_rel_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this data at hand, we can plot the [degree distribution](https://en.wikipedia.org/wiki/Degree_distribution) by showing the number of collaborators a scientist has published with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAGwCAYAAAC0HlECAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA72klEQVR4nO3deViVdf7/8dcBBHEBNAUkUSn33FITca3kK5VTmU6TxaSTppPigpjb5JYtMjRZag6OLVrfyWxqWjVRcsFviqgkiRsuQ2IlkiEQuCHcvz8c7p8nzO5TB88Rn4/ruq/L8/l8zn2/73NfXbz63Pf5HJthGIYAAABwRR6uLgAAAOBaQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAACwhNAAAAFni5uoDqory8XN99953q1q0rm83m6nIAAIAFhmHoxx9/VEhIiDw8rjyXRGhyku+++06hoaGuLgMAAPwKx44dU+PGja84htDkJHXr1pV08UP38/NzcTUAAMCKoqIihYaGmn/Hr4TQ5CQVt+T8/PwITQAAXGOsPFrDg+AAAAAWEJoAAAAsIDQBAABYQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAVeri4Av02zaastjfs6fkAVVwIAQPXGTBMAAIAFhCYAAAALCE0AAAAWEJoAAAAsIDQBAABYQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAWEJgAAAAsITQAAABZ4uboAXB3Npq22NO7r+AFVXAkAANcmZpoAAAAsIDQBAABYQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAAC1wamjZv3qx7771XISEhstls+uijj+z6DcPQrFmz1KhRI/n6+ioyMlKHDh2yG5Ofn6/o6Gj5+fkpICBAI0aMUHFxsd2Y3bt3q3fv3qpZs6ZCQ0OVkJBQqZb33ntPrVu3Vs2aNdW+fXt99tlnTj9fAABw7XJpaCopKVHHjh21ePHiy/YnJCRo4cKFWrJkidLS0lS7dm1FRUXp7Nmz5pjo6Gjt3btXycnJWrVqlTZv3qxRo0aZ/UVFRerfv7+aNm2q9PR0vfDCC5ozZ46WLl1qjtm6dasefvhhjRgxQrt27dLAgQM1cOBA7dmzp+pOHgAAXFNshmEYri5Ckmw2mz788EMNHDhQ0sVZppCQEE2aNElPPvmkJKmwsFBBQUFavny5hgwZov3796tt27basWOHunbtKklKSkrSPffco2+++UYhISFKTEzUU089pdzcXHl7e0uSpk2bpo8++kgHDhyQJD300EMqKSnRqlWrzHq6d++uTp06acmSJZbqLyoqkr+/vwoLC+Xn5+esj+UXWV3p29lYORwAUB048vfbbZ9pys7OVm5uriIjI802f39/hYeHKzU1VZKUmpqqgIAAMzBJUmRkpDw8PJSWlmaO6dOnjxmYJCkqKkpZWVk6deqUOebS41SMqTjO5Zw7d05FRUV2GwAAqL7cNjTl5uZKkoKCguzag4KCzL7c3FwFBgba9Xt5eal+/fp2Yy63j0uP8XNjKvovZ968efL39ze30NBQR08RAABcQ9w2NLm76dOnq7Cw0NyOHTvm6pIAAEAVctvQFBwcLEk6ceKEXfuJEyfMvuDgYOXl5dn1X7hwQfn5+XZjLrePS4/xc2Mq+i/Hx8dHfn5+dhsAAKi+3DY0hYWFKTg4WOvXrzfbioqKlJaWpoiICElSRESECgoKlJ6ebo7ZsGGDysvLFR4ebo7ZvHmzSktLzTHJyclq1aqV6tWrZ4659DgVYyqOAwAA4NLQVFxcrIyMDGVkZEi6+PB3RkaGcnJyZLPZFBsbq2effVaffPKJMjMzNXToUIWEhJjfsGvTpo3uuusujRw5Utu3b9eWLVs0duxYDRkyRCEhIZKkRx55RN7e3hoxYoT27t2rd999VwsWLFBcXJxZx4QJE5SUlKQXX3xRBw4c0Jw5c7Rz506NHTv2an8kAADATXm58uA7d+7UHXfcYb6uCDLDhg3T8uXLNWXKFJWUlGjUqFEqKChQr169lJSUpJo1a5rvefvttzV27Fj169dPHh4eGjx4sBYuXGj2+/v7a926dYqJiVGXLl3UoEEDzZo1y24tpx49emjFihWaMWOG/vKXv6hFixb66KOP1K5du6vwKQAAgGuB26zTdK1jnSYAAK491WKdJgAAAHdCaAIAALCA0AQAAGABoQkAAMACQhMAAIAFhCYAAAALCE0AAAAWEJoAAAAsIDQBAABYQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAVeri4AP6/ZtNWuLgEAAPwXM00AAAAWEJoAAAAsIDQBAABYQGgCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAWEJgAAAAsITQAAABYQmgAAACwgNAEAAFhAaAIAALCA0AQAAGABoQkAAMACQhMAAIAFhCYAAAALCE0AAAAWEJoAAAAsIDQBAABYQGgCAACwgNAEAABgwW8OTWVlZcrIyNCpU6ecUQ8AAIBbcjg0xcbG6vXXX5d0MTD17dtXnTt3VmhoqDZt2uTs+gAAANyCw6Hp/fffV8eOHSVJn376qbKzs3XgwAFNnDhRTz31lFOLKysr08yZMxUWFiZfX1/dfPPNeuaZZ2QYhjnGMAzNmjVLjRo1kq+vryIjI3Xo0CG7/eTn5ys6Olp+fn4KCAjQiBEjVFxcbDdm9+7d6t27t2rWrKnQ0FAlJCQ49Vyqm2bTVv/iBgBAdeJwaDp58qSCg4MlSZ999pkefPBBtWzZUsOHD1dmZqZTi/vrX/+qxMREvfLKK9q/f7/++te/KiEhQYsWLTLHJCQkaOHChVqyZInS0tJUu3ZtRUVF6ezZs+aY6Oho7d27V8nJyVq1apU2b96sUaNGmf1FRUXq37+/mjZtqvT0dL3wwguaM2eOli5d6tTzAQAA1y4vR98QFBSkffv2qVGjRkpKSlJiYqIk6fTp0/L09HRqcVu3btX999+vAQMGSJKaNWumd955R9u3b5d0cZbp5Zdf1owZM3T//fdLkt566y0FBQXpo48+0pAhQ7R//34lJSVpx44d6tq1qyRp0aJFuueee/S3v/1NISEhevvtt3X+/Hm98cYb8vb21i233KKMjAzNnz/fLlwBAIDrl8MzTY899pj+8Ic/qF27drLZbIqMjJQkpaWlqXXr1k4trkePHlq/fr0OHjwoSfrqq6/0xRdf6O6775YkZWdnKzc316xBkvz9/RUeHq7U1FRJUmpqqgICAszAJEmRkZHy8PBQWlqaOaZPnz7y9vY2x0RFRSkrK+tnH3A/d+6cioqK7DYAAFB9OTzTNGfOHLVv3145OTl68MEH5ePjI0ny9PTUtGnTnFrctGnTVFRUpNatW8vT01NlZWV67rnnFB0dLUnKzc2VdHH261JBQUFmX25urgIDA+36vby8VL9+fbsxYWFhlfZR0VevXr1Ktc2bN09PP/20E84SAABcCxwKTaWlpbrrrru0ZMkSDR482K5v2LBhTi1Mkv71r3/p7bff1ooVK8xbZrGxsQoJCamS4zli+vTpiouLM18XFRUpNDTUhRUBAICq5FBoqlGjhnbv3l1VtVQyefJkTZs2TUOGDJEktW/fXkePHtW8efM0bNgw84H0EydOqFGjRub7Tpw4oU6dOkmSgoODlZeXZ7ffCxcuKD8/33x/cHCwTpw4YTem4nXFmJ/y8fExZ9kAAED15/AzTX/84x/NdZqq2unTp+XhYV+ip6enysvLJUlhYWEKDg7W+vXrzf6ioiKlpaUpIiJCkhQREaGCggKlp6ebYzZs2KDy8nKFh4ebYzZv3qzS0lJzTHJyslq1anXZW3MAAOD64/AzTRcuXNAbb7yhzz//XF26dFHt2rXt+ufPn++04u69914999xzatKkiW655Rbt2rVL8+fP1/DhwyVJNptNsbGxevbZZ9WiRQuFhYVp5syZCgkJ0cCBAyVJbdq00V133aWRI0dqyZIlKi0t1dixYzVkyBCFhIRIkh555BE9/fTTGjFihKZOnao9e/ZowYIFeumll5x2LgAA4NrmcGjas2ePOnfuLEnmt9oq2Gw251T1X4sWLdLMmTM1ZswY5eXlKSQkRH/+8581a9Ysc8yUKVNUUlKiUaNGqaCgQL169VJSUpJq1qxpjnn77bc1duxY9evXTx4eHho8eLAWLlxo9vv7+2vdunWKiYlRly5d1KBBA82aNYvlBgAAgMlmXLq8Nn61oqIi+fv7q7CwUH5+fk7Z57W+qvbX8QNcXQIAAFfkyN/vX/2DvYcPH9batWt15swZSRLZCwAAVGcOh6YffvhB/fr1U8uWLXXPPffo+PHjkqQRI0Zo0qRJTi8QAADAHTgcmiZOnKgaNWooJydHtWrVMtsfeughJSUlObU4AAAAd+Hwg+Dr1q3T2rVr1bhxY7v2Fi1a6OjRo04rDAAAwJ04PNNUUlJiN8NUIT8/n8UeAQBAteVwaOrdu7feeust87XNZlN5ebkSEhJ0xx13OLU4AAAAd+Hw7bmEhAT169dPO3fu1Pnz5zVlyhTt3btX+fn52rJlS1XUCAAA4HIOzzS1a9dOBw8eVK9evXT//ferpKREgwYN0q5du3TzzTdXRY0AAAAu5/BMk3RxBe2nnnrK2bUAAAC4rV8VmgoKCrR9+3bl5eWZP55bYejQoU4pDAAAwJ04HJo+/fRTRUdHq7i4WH5+fna/N2ez2QhNAACgWnL4maZJkyZp+PDhKi4uVkFBgU6dOmVu+fn5VVEjAACAyzkcmr799luNHz/+sms1AQAAVFcOh6aoqCjt3LmzKmoBAABwW5aeafrkk0/Mfw8YMECTJ0/Wvn371L59e9WoUcNu7H333efcCgEAANyApdA0cODASm1z586t1Gaz2VRWVvabiwIAAHA3lkLTT5cVAAAAuN44/EwTAADA9cjh0DR+/HgtXLiwUvsrr7yi2NhYZ9QEAADgdhwOTf/+97/Vs2fPSu09evTQ+++/75SiAAAA3I3DoemHH36Qv79/pXY/Pz+dPHnSKUUBAAC4G4dDU/PmzZWUlFSpfc2aNbrpppucUhQAAIC7cfi35+Li4jR27Fh9//33uvPOOyVJ69ev14svvqiXX37Z2fUBAAC4BYdD0/Dhw3Xu3Dk999xzeuaZZyRJzZo1U2JiIj/WCwAAqi2HQtOFCxe0YsUKDRo0SKNHj9b3338vX19f1alTp6rqAwAAcAsOPdPk5eWlJ554QmfPnpUkNWzYkMAEAACuCw4/CN6tWzft2rWrKmoBAABwWw4/0zRmzBhNmjRJ33zzjbp06aLatWvb9Xfo0MFpxQEAALgLh0PTkCFDJF1cGbyCzWaTYRj8YC8AAKi2HA5N2dnZVVEHAACAW3M4NDVt2rQq6gAAAHBrDoemCvv27VNOTo7Onz9v137ffff95qIAAADcjcOh6T//+Y8eeOABZWZmms8ySRefa5LEM00AAKBacnjJgQkTJigsLEx5eXmqVauW9u7dq82bN6tr167atGlTFZQIAADgeg7PNKWmpmrDhg1q0KCBPDw85OHhoV69emnevHkaP348azgBAIBqyeGZprKyMtWtW1eS1KBBA3333XeSLj4gnpWV5dzqAAAA3ITDM03t2rXTV199pbCwMIWHhyshIUHe3t5aunSpbrrppqqoEQAAwOUcDk0zZsxQSUmJJGnu3Ln63e9+p969e+uGG27Qu+++6/QCAQAA3IHDoSkqKsr8d/PmzXXgwAHl5+erXr165jfoAAAAqhuHn2mqcPjwYa1du1ZnzpxR/fr1nVkTAACA23F4pumHH37QH/7wB23cuFE2m02HDh3STTfdpBEjRqhevXp68cUXq6JOXIOaTVttadzX8QOquBIAAH47h2eaJk6cqBo1aignJ0e1atUy2x966CElJSU5tTgAAAB34fBM07p167R27Vo1btzYrr1FixY6evSo0woDAABwJw7PNJWUlNjNMFXIz8+Xj4+PU4oCAABwNw6Hpt69e+utt94yX9tsNpWXlyshIUF33HGHU4sDAABwFw7fnktISFC/fv20c+dOnT9/XlOmTNHevXuVn5+vLVu2VEWNAAAALufwTFO7du108OBB9erVS/fff79KSko0aNAg7dq1SzfffHNV1AgAAOByDs80SZK/v7+eeuopZ9cCAADgthyeaUpKStIXX3xhvl68eLE6deqkRx55RKdOnXJqcQAAAO7C4dA0efJkFRUVSZIyMzMVFxene+65R9nZ2YqLi3N6gQAAAO7A4dtz2dnZatu2rSTp3//+t+699149//zz+vLLL3XPPfc4vUAAAAB34PBMk7e3t06fPi1J+vzzz9W/f39JUv369c0ZKAAAgOrG4ZmmXr16KS4uTj179tT27dv17rvvSpIOHjxYaZVwAACA6sLhmaZXXnlFXl5eev/995WYmKgbb7xRkrRmzRrdddddTi/w22+/1R//+EfdcMMN8vX1Vfv27bVz506z3zAMzZo1S40aNZKvr68iIyN16NAhu33k5+crOjpafn5+CggI0IgRI1RcXGw3Zvfu3erdu7dq1qyp0NBQJSQkOP1cAADAtcvhmaYmTZpo1apVldpfeuklpxR0qVOnTqlnz5664447tGbNGjVs2FCHDh1SvXr1zDEJCQlauHCh3nzzTYWFhWnmzJmKiorSvn37VLNmTUlSdHS0jh8/ruTkZJWWluqxxx7TqFGjtGLFCklSUVGR+vfvr8jISC1ZskSZmZkaPny4AgICNGrUKKefFwAAuPbYDMMwHH3TkSNHtGzZMh05ckQLFixQYGCg1qxZoyZNmuiWW25xWnHTpk3Tli1b9H//93+X7TcMQyEhIZo0aZKefPJJSVJhYaGCgoK0fPlyDRkyRPv371fbtm21Y8cOde3aVdLFZRPuueceffPNNwoJCVFiYqKeeuop5ebmytvb2zz2Rx99pAMHDliqtaioSP7+/iosLJSfn58Tzl5qNm21U/bj7r6OH+DqEgAA1ylH/n47fHsuJSVF7du3V1pamj744APzNtdXX32l2bNn/7qKf8Ynn3yirl276sEHH1RgYKBuvfVWvfrqq2Z/dna2cnNzFRkZabb5+/srPDxcqampkqTU1FQFBASYgUmSIiMj5eHhobS0NHNMnz59zMAkSVFRUcrKyvrZtafOnTunoqIiuw0AAFRfDoemadOm6dlnn1VycrJdyLjzzju1bds2pxb3n//8R4mJiWrRooXWrl2r0aNHa/z48XrzzTclSbm5uZKkoKAgu/cFBQWZfbm5uQoMDLTr9/LyUv369e3GXG4flx7jp+bNmyd/f39zCw0N/Y1nCwAA3JnDoSkzM1MPPPBApfbAwECdPHnSKUVVKC8vV+fOnfX888/r1ltv1ahRozRy5EgtWbLEqcf5NaZPn67CwkJzO3bsmKtLAgAAVcjh0BQQEKDjx49Xat+1a5f5TTpnadSokbmQZoU2bdooJydHkhQcHCxJOnHihN2YEydOmH3BwcHKy8uz679w4YLy8/PtxlxuH5ce46d8fHzk5+dntwEAgOrL4dA0ZMgQTZ06Vbm5ubLZbCovL9eWLVv05JNPaujQoU4trmfPnsrKyrJrO3jwoJo2bSpJCgsLU3BwsNavX2/2FxUVKS0tTREREZKkiIgIFRQUKD093RyzYcMGlZeXKzw83ByzefNmlZaWmmOSk5PVqlUru2/qAQCA65fDoen5559X69atFRoaquLiYrVt21Z9+vRRjx49NGPGDKcWN3HiRG3btk3PP/+8Dh8+rBUrVmjp0qWKiYmRJNlsNsXGxurZZ5/VJ598oszMTA0dOlQhISEaOHCgpIszU3fddZdGjhyp7du3a8uWLRo7dqyGDBmikJAQSdIjjzwib29vjRgxQnv37tW7776rBQsW8Ft6AADA5NCSA4Zh6NixY2rYsKFOnjypzMxMFRcX69Zbb1WLFi2qpMBVq1Zp+vTpOnTokMLCwhQXF6eRI0fa1TR79mwtXbpUBQUF6tWrl/7+97+rZcuW5pj8/HyNHTtWn376qTw8PDR48GAtXLhQderUMcfs3r1bMTEx2rFjhxo0aKBx48Zp6tSplutkyYFfjyUHAACu4sjfb4dCU3l5uWrWrKm9e/dWWUi6VhGafj1CEwDAVapsnSYPDw+1aNFCP/zww28qEAAA4Frj8DNN8fHxmjx5svbs2VMV9QAAALglh397bujQoTp9+rQ6duwob29v+fr62vXn5+c7rTgAAAB34XBoevnll6ugDAAAAPfmcGgaNmxYVdSB65jVB955YBwA4EoOP9MEAABwPSI0AQAAWEBoAgAAsMBSaNq9e7fKy8uruhYAAAC3ZSk03XrrrTp58qQk6aabbmJxSwAAcN2xFJoCAgKUnZ0tSfr666+ZdQIAANcdS0sODB48WH379lWjRo1ks9nUtWtXeXp6Xnbsf/7zH6cWCAAA4A4shaalS5dq0KBBOnz4sMaPH6+RI0eqbt26VV0bAACA27C8uOVdd90lSUpPT9eECRMITQAA4Lri8Irgy5YtM//9zTffSJIaN27svIquA1ZXwAYAAO7D4XWaysvLNXfuXPn7+6tp06Zq2rSpAgIC9Mwzz/CAOAAAqLYcnml66qmn9Prrrys+Pl49e/aUJH3xxReaM2eOzp49q+eee87pRQIAALiaw6HpzTff1Guvvab77rvPbOvQoYNuvPFGjRkzhtAEAACqJYdvz+Xn56t169aV2lu3bq38/HynFAUAAOBuHA5NHTt21CuvvFKp/ZVXXlHHjh2dUhQAAIC7cfj2XEJCggYMGKDPP/9cERERkqTU1FQdO3ZMn332mdMLBAAAcAcOzzT17dtXBw8e1AMPPKCCggIVFBRo0KBBysrKUu/evauiRgAAAJdzeKZJkkJCQnjgGwAAXFccnmkCAAC4HhGaAAAALCA0AQAAWOBQaDIMQzk5OTp79mxV1QMAAOCWHA5NzZs317Fjx6qqHgAAALfkUGjy8PBQixYt9MMPP1RVPQAAAG7J4Wea4uPjNXnyZO3Zs6cq6gEAAHBLDq/TNHToUJ0+fVodO3aUt7e3fH197fr5/TkAAFAdORyaXn755SooAwAAwL05HJqGDRtWFXUAAAC4tV+1TtORI0c0Y8YMPfzww8rLy5MkrVmzRnv37nVqcQAAAO7C4dCUkpKi9u3bKy0tTR988IGKi4slSV999ZVmz57t9AIBAADcgcOhadq0aXr22WeVnJwsb29vs/3OO+/Utm3bnFocAACAu3A4NGVmZuqBBx6o1B4YGKiTJ086pSgAAAB343BoCggI0PHjxyu179q1SzfeeKNTigIAAHA3DoemIUOGaOrUqcrNzZXNZlN5ebm2bNmiJ598UkOHDq2KGgEAAFzO4dD0/PPPq3Xr1goNDVVxcbHatm2rPn36qEePHpoxY0ZV1AgAAOByDq/T5O3trVdffVUzZ87Unj17VFxcrFtvvVUtWrSoivoAAADcgsOhqUKTJk0UGhoqSbLZbE4rCAAAwB39qsUtX3/9dbVr1041a9ZUzZo11a5dO7322mvOrg0AAMBtODzTNGvWLM2fP1/jxo1TRESEJCk1NVUTJ05UTk6O5s6d6/QiAUlqNm31L475On7AVagEAHA9cjg0JSYm6tVXX9XDDz9stt13333q0KGDxo0bR2gCAADVksO350pLS9W1a9dK7V26dNGFCxecUhQAAIC7cTg0Pfroo0pMTKzUvnTpUkVHRzulKAAAAHdj6fZcXFyc+W+bzabXXntN69atU/fu3SVJaWlpysnJYXFLAABQbVkKTbt27bJ73aVLF0nSkSNHJEkNGjRQgwYNtHfvXieXBwAA4B4shaaNGzdWdR0AAABu7Vet0wQAAHC9cXjJgbNnz2rRokXauHGj8vLyVF5ebtf/5ZdfOq04AAAAd+FwaBoxYoTWrVun3//+9+rWrRs/oQIAAK4LDt+eW7VqlT766CMlJiZqzpw5mj17tt1WleLj42Wz2RQbG2u2nT17VjExMbrhhhtUp04dDR48WCdOnLB7X05OjgYMGKBatWopMDBQkydPrrSm1KZNm9S5c2f5+PioefPmWr58eZWeCwAAuLY4HJpuvPFG1a1btypquaIdO3boH//4hzp06GDXPnHiRH366ad67733lJKSou+++06DBg0y+8vKyjRgwACdP39eW7du1Ztvvqnly5dr1qxZ5pjs7GwNGDBAd9xxhzIyMhQbG6vHH39ca9euvWrnBwAA3JvDoenFF1/U1KlTdfTo0aqo57KKi4sVHR2tV199VfXq1TPbCwsL9frrr2v+/Pm688471aVLFy1btkxbt27Vtm3bJEnr1q3Tvn379M9//lOdOnXS3XffrWeeeUaLFy/W+fPnJUlLlixRWFiYXnzxRbVp00Zjx47V73//e7300ktX7RwBAIB7czg0de3aVWfPntVNN92kunXrqn79+nZbVYiJidGAAQMUGRlp156enq7S0lK79tatW6tJkyZKTU2VdPHHhNu3b6+goCBzTFRUlIqKisx1pVJTUyvtOyoqytzH5Zw7d05FRUV2GwAAqL4cfhD84Ycf1rfffqvnn39eQUFBVf4g+MqVK/Xll19qx44dlfpyc3Pl7e2tgIAAu/agoCDl5uaaYy4NTBX9FX1XGlNUVKQzZ87I19e30rHnzZunp59++lefFwAAuLY4HJq2bt2q1NRUdezYsSrqsXPs2DFNmDBBycnJqlmzZpUfzxHTp0+3+3mZoqIihYaGurAiAABQlRy+Pde6dWudOXOmKmqpJD09XXl5eercubO8vLzk5eWllJQULVy4UF5eXgoKCtL58+dVUFBg974TJ04oODhYkhQcHFzp23QVr39pjJ+f32VnmSTJx8dHfn5+dhsAAKi+HA5N8fHxmjRpkjZt2qQffvihSp/r6devnzIzM5WRkWFuXbt2VXR0tPnvGjVqaP369eZ7srKylJOTo4iICElSRESEMjMzlZeXZ45JTk6Wn5+f2rZta465dB8VYyr2AQAA4PDtubvuukvSxUBzKcMwZLPZVFZW5pzKJNWtW1ft2rWza6tdu7ZuuOEGs33EiBGKi4tT/fr15efnp3HjxikiIkLdu3eXJPXv319t27bVo48+qoSEBOXm5mrGjBmKiYmRj4+PJOmJJ57QK6+8oilTpmj48OHasGGD/vWvf2n16tVOOxcAAHBtczg0uduP97700kvy8PDQ4MGDde7cOUVFRenvf/+72e/p6alVq1Zp9OjRioiIUO3atTVs2DDNnTvXHBMWFqbVq1dr4sSJWrBggRo3bqzXXntNUVFRrjglAADghmyGYRiuLqI6KCoqkr+/vwoLC3/x+aZm05jBqipfxw9wdQkAgGuII3+/HZ5p2rx58xX7+/Tp4+guAQAA3J7Doen222+v1HbpWk3OfKYJAADAXTj87blTp07ZbXl5eUpKStJtt92mdevWVUWNAAAALufwTJO/v3+ltv/5n/+Rt7e34uLilJ6e7pTCAAAA3InDM00/JygoSFlZWc7aHQAAgFtxeKZp9+7ddq8Nw9Dx48cVHx+vTp06OasuAAAAt+JwaOrUqZNsNpt+ulJB9+7d9cYbbzitMAAAAHficGjKzs62e+3h4aGGDRu63Q/qAgAAOJPDoalp06ZVUQcAAIBbczg0SdL69eu1fv165eXlqby83K6PW3QAAKA6cjg0Pf3005o7d666du2qRo0a2S1sCQAAUF05HJqWLFmi5cuX69FHH62KegAAANySw6Hp/Pnz6tGjR1XUAvxmVn8MmR/2BQA4yuHFLR9//HGtWLGiKmoBAABwWw7PNJ09e1ZLly7V559/rg4dOqhGjRp2/fPnz3dacQAAAO7iV60IXrHy9549e+z6eCgcAABUVw6Hpo0bN1ZFHQAAAG7NaT/YCwAAUJ0RmgAAACwgNAEAAFhAaAIAALCA0AQAAGABoQkAAMACQhMAAIAFhCYAAAALCE0AAAAWOLwiOFAdNJu22tK4r+MHVHElAIBrBTNNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAWEJgAAAAsITQAAABYQmgAAACwgNAEAAFhAaAIAALCA0AQAAGABoQkAAMACQhMAAIAFhCYAAAALCE0AAAAWeLm6AMCdNZu22tK4r+MHVHElAABXY6YJAADAAkITAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYIFbh6Z58+bptttuU926dRUYGKiBAwcqKyvLbszZs2cVExOjG264QXXq1NHgwYN14sQJuzE5OTkaMGCAatWqpcDAQE2ePFkXLlywG7Np0yZ17txZPj4+at68uZYvX17VpwcAAK4hbh2aUlJSFBMTo23btik5OVmlpaXq37+/SkpKzDETJ07Up59+qvfee08pKSn67rvvNGjQILO/rKxMAwYM0Pnz57V161a9+eabWr58uWbNmmWOyc7O1oABA3THHXcoIyNDsbGxevzxx7V27dqrer4AAMB92QzDMFxdhFXff/+9AgMDlZKSoj59+qiwsFANGzbUihUr9Pvf/16SdODAAbVp00apqanq3r271qxZo9/97nf67rvvFBQUJElasmSJpk6dqu+//17e3t6aOnWqVq9erT179pjHGjJkiAoKCpSUlGSptqKiIvn7+6uwsFB+fn5XHGv1pzlw7eBnVADg2uTI32+3nmn6qcLCQklS/fr1JUnp6ekqLS1VZGSkOaZ169Zq0qSJUlNTJUmpqalq3769GZgkKSoqSkVFRdq7d6855tJ9VIyp2MflnDt3TkVFRXYbAACovq6Z0FReXq7Y2Fj17NlT7dq1kyTl5ubK29tbAQEBdmODgoKUm5trjrk0MFX0V/RdaUxRUZHOnDlz2XrmzZsnf39/cwsNDf3N5wgAANzXNROaYmJitGfPHq1cudLVpUiSpk+frsLCQnM7duyYq0sCAABVyMvVBVgxduxYrVq1Sps3b1bjxo3N9uDgYJ0/f14FBQV2s00nTpxQcHCwOWb79u12+6v4dt2lY376jbsTJ07Iz89Pvr6+l63Jx8dHPj4+v/ncAADAtcGtZ5oMw9DYsWP14YcfasOGDQoLC7Pr79Kli2rUqKH169ebbVlZWcrJyVFERIQkKSIiQpmZmcrLyzPHJCcny8/PT23btjXHXLqPijEV+wAAAHDrb8+NGTNGK1as0Mcff6xWrVqZ7f7+/uYM0OjRo/XZZ59p+fLl8vPz07hx4yRJW7dulXRxyYFOnTopJCRECQkJys3N1aOPPqrHH39czz//vKSLSw60a9dOMTExGj58uDZs2KDx48dr9erVioqKslQr357DL+EbdgDgfhz5++3Woclms122fdmyZfrTn/4k6eLilpMmTdI777yjc+fOKSoqSn//+9/NW2+SdPToUY0ePVqbNm1S7dq1NWzYMMXHx8vL6//fndy0aZMmTpyoffv2qXHjxpo5c6Z5DCsqPvTQ2H/Jw6fWrzpfVG+EJgBwP9UmNF1LCE34JYQmAHA/1XadJgAAAFchNAEAAFhAaAIAALCA0AQAAGABoQkAAMACQhMAAIAF18TPqADVgdVFTVmaAADcEzNNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAHfngPcDN+yAwD3xEwTAACABYQmAAAACwhNAAAAFhCaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAIWtwSuUVYWwWQBTABwHmaaAAAALCA0AQAAWMDtOaAa43fsAMB5mGkCAACwgNAEAABgAaEJAADAAkITAACABYQmAAAAC/j2HAC+ZQcAFhCaAFhGuAJwPeP2HAAAgAXMNAFwOn4XD0B1xEwTAACABYQmAAAACwhNAAAAFvBMEwCX4Jt4AK41zDQBAABYwEwTALfGjBQAd8FMEwAAgAXMNAGoFqzOSFnBrBWAy2GmCQAAwAJmmgDgJ3iOCsDlMNMEAABgATNNAPArMSMFXF8ITQBQxXhIHageCE0AcA1hdgtwHZ5pAgAAsICZJgCohpiRApyP0AQA1zFnPm/lbAQ6uBtCEwDALfEAPdwNoQkAUO25cwBz5q1UbstWLULTTyxevFgvvPCCcnNz1bFjRy1atEjdunVzdVkAADfhqlua7nwr1YrqEOgITZd49913FRcXpyVLlig8PFwvv/yyoqKilJWVpcDAQFeXBwCAUzg7gDkz6LhzuLIZhmFc9aO6qfDwcN1222165ZVXJEnl5eUKDQ3VuHHjNG3atCu+t6ioSP7+/gqN/Zc8fGpdjXIBAMBvVH7utI69/AcVFhbKz8/vimOZafqv8+fPKz09XdOnTzfbPDw8FBkZqdTU1Erjz507p3PnzpmvCwsLJV388AEAwLWh4u+2lTkkQtN/nTx5UmVlZQoKCrJrDwoK0oEDByqNnzdvnp5++ulK7d8m/qmqSgQAAFXkxx9/lL+//xXHEJp+penTpysuLs58XVBQoKZNmyonJ+cXP3RUjaKiIoWGhurYsWO/OMWKqsN1cD2ugetxDVzP6jUwDEM//vijQkJCfnGfhKb/atCggTw9PXXixAm79hMnTig4OLjSeB8fH/n4+FRq9/f35z8QF/Pz8+MauAGug+txDVyPa+B6Vq6B1ckOfnvuv7y9vdWlSxetX7/ebCsvL9f69esVERHhwsoAAIA7YKbpEnFxcRo2bJi6du2qbt266eWXX1ZJSYkee+wxV5cGAABcjNB0iYceekjff/+9Zs2apdzcXHXq1ElJSUmVHg6/HB8fH82ePfuyt+xwdXAN3APXwfW4Bq7HNXC9qrgGrNMEAABgAc80AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCk5MsXrxYzZo1U82aNRUeHq7t27e7uqRqa/Pmzbr33nsVEhIim82mjz76yK7fMAzNmjVLjRo1kq+vryIjI3Xo0CHXFFtNzZs3T7fddpvq1q2rwMBADRw4UFlZWXZjzp49q5iYGN1www2qU6eOBg8eXGnxWPx6iYmJ6tChg7lwX0REhNasWWP28/lfffHx8bLZbIqNjTXbuA5Vb86cObLZbHZb69atzX5nXgNCkxO8++67iouL0+zZs/Xll1+qY8eOioqKUl5enqtLq5ZKSkrUsWNHLV68+LL9CQkJWrhwoZYsWaK0tDTVrl1bUVFROnv27FWutPpKSUlRTEyMtm3bpuTkZJWWlqp///4qKSkxx0ycOFGffvqp3nvvPaWkpOi7777ToEGDXFh19dK4cWPFx8crPT1dO3fu1J133qn7779fe/fulcTnf7Xt2LFD//jHP9ShQwe7dq7D1XHLLbfo+PHj5vbFF1+YfU69BgZ+s27duhkxMTHm67KyMiMkJMSYN2+eC6u6PkgyPvzwQ/N1eXm5ERwcbLzwwgtmW0FBgeHj42O88847Lqjw+pCXl2dIMlJSUgzDuPiZ16hRw3jvvffMMfv37zckGampqa4qs9qrV6+e8dprr/H5X2U//vij0aJFCyM5Odno27evMWHCBMMw+O/gapk9e7bRsWPHy/Y5+xow0/QbnT9/Xunp6YqMjDTbPDw8FBkZqdTUVBdWdn3Kzs5Wbm6u3fXw9/dXeHg416MKFRYWSpLq168vSUpPT1dpaanddWjdurWaNGnCdagCZWVlWrlypUpKShQREcHnf5XFxMRowIABdp+3xH8HV9OhQ4cUEhKim266SdHR0crJyZHk/GvAiuC/0cmTJ1VWVlZp1fCgoCAdOHDARVVdv3JzcyXpstejog/OVV5ertjYWPXs2VPt2rWTdPE6eHt7KyAgwG4s18G5MjMzFRERobNnz6pOnTr68MMP1bZtW2VkZPD5XyUrV67Ul19+qR07dlTq47+DqyM8PFzLly9Xq1atdPz4cT399NPq3bu39uzZ4/RrQGgC8JvExMRoz549ds8Q4Opo1aqVMjIyVFhYqPfff1/Dhg1TSkqKq8u6bhw7dkwTJkxQcnKyatas6epyrlt33323+e8OHTooPDxcTZs21b/+9S/5+vo69VjcnvuNGjRoIE9Pz0pP4p84cULBwcEuqur6VfGZcz2ujrFjx2rVqlXauHGjGjdubLYHBwfr/PnzKigosBvPdXAub29vNW/eXF26dNG8efPUsWNHLViwgM//KklPT1deXp46d+4sLy8veXl5KSUlRQsXLpSXl5eCgoK4Di4QEBCgli1b6vDhw07/b4HQ9Bt5e3urS5cuWr9+vdlWXl6u9evXKyIiwoWVXZ/CwsIUHBxsdz2KioqUlpbG9XAiwzA0duxYffjhh9qwYYPCwsLs+rt06aIaNWrYXYesrCzl5ORwHapQeXm5zp07x+d/lfTr10+ZmZnKyMgwt65duyo6Otr8N9fh6isuLtaRI0fUqFEj5/+38CsfVsclVq5cafj4+BjLly839u3bZ4waNcoICAgwcnNzXV1atfTjjz8au3btMnbt2mVIMubPn2/s2rXLOHr0qGEYhhEfH28EBAQYH3/8sbF7927j/vvvN8LCwowzZ864uPLqY/To0Ya/v7+xadMm4/jx4+Z2+vRpc8wTTzxhNGnSxNiwYYOxc+dOIyIiwoiIiHBh1dXLtGnTjJSUFCM7O9vYvXu3MW3aNMNmsxnr1q0zDIPP31Uu/facYXAdroZJkyYZmzZtMrKzs40tW7YYkZGRRoMGDYy8vDzDMJx7DQhNTrJo0SKjSZMmhre3t9GtWzdj27Ztri6p2tq4caMhqdI2bNgwwzAuLjswc+ZMIygoyPDx8TH69etnZGVlubboauZyn78kY9myZeaYM2fOGGPGjDHq1atn1KpVy3jggQeM48ePu67oamb48OFG06ZNDW9vb6Nhw4ZGv379zMBkGHz+rvLT0MR1qHoPPfSQ0ahRI8Pb29u48cYbjYceesg4fPiw2e/Ma2AzDMP4jTNhAAAA1R7PNAEAAFhAaAIAALCA0AQAAGABoQkAAMACQhMAAIAFhCYAAAALCE0AAAAWEJoAAAAsIDQBcJnbb79dsbGxri7DZBiGRo0apfr168tmsykjI8PVJf1mmzZtks1mq/SDpQAcR2gCgP9KSkrS8uXLtWrVKh0/flzt2rVzdUkOcbcQClQ3Xq4uAACcqaysTDabTR4ejv8/YcUvo/fo0aMKKqteSktLVaNGDVeXAVxVzDQB17nbb79d48eP15QpU1S/fn0FBwdrzpw5Zv/XX39d6VZVQUGBbDabNm3aJOn/3wJau3atbr31Vvn6+urOO+9UXl6e1qxZozZt2sjPz0+PPPKITp8+bXf8CxcuaOzYsfL391eDBg00c+ZMXfqTmOfOndOTTz6pG2+8UbVr11Z4eLh5XElavny5AgIC9Mknn6ht27by8fFRTk7OZc81JSVF3bp1k4+Pjxo1aqRp06bpwoULkqQ//elPGjdunHJycmSz2dSsWbOf/cy2bNmi22+/XbVq1VK9evUUFRWlU6dOmfWOHz9egYGBqlmzpnr16qUdO3b84nWYOnWqWrZsqVq1aummm27SzJkzVVpaavb/6U9/0sCBA+3eExsbq9tvv93sT0lJ0YIFC2Sz2WSz2fT111+bY9PT09W1a1fVqlVLPXr0UFZWlt2+EhMTdfPNN8vb21utWrXS//7v/9r122w2JSYm6r777lPt2rX13HPP6dSpU4qOjlbDhg3l6+urFi1aaNmyZb94rsA1yxm/MAzg2tW3b1/Dz8/PmDNnjnHw4EHjzTffNGw2m7Fu3TrDMAwjOzvbkGTs2rXLfM+pU6cMScbGjRsNwzCMjRs3GpKM7t27G1988YXx5ZdfGs2bNzf69u1r9O/f3/jyyy+NzZs3GzfccIMRHx9vd+w6deoYEyZMMA4cOGD885//NGrVqmUsXbrUHPP4448bPXr0MDZv3mwcPnzYeOGFFwwfHx/j4MGDhmEYxrJly4waNWoYPXr0MLZs2WIcOHDAKCkpqXSe33zzjVGrVi1jzJgxxv79+40PP/zQaNCggTF79mzDMAyjoKDAmDt3rtG4cWPj+PHjRl5e3mU/r127dhk+Pj7G6NGjjYyMDGPPnj3GokWLjO+//94wDMMYP368ERISYnz22WfG3r17jWHDhhn16tUzfvjhhyteh2eeecbYsmWLkZ2dbXzyySdGUFCQ8de//tXsHzZsmHH//ffbvWfChAlG3759zfojIiKMkSNHGsePHzeOHz9uXLhwwbw24eHhxqZNm4y9e/cavXv3Nnr06GHu54MPPjBq1KhhLF682MjKyjJefPFFw9PT09iwYYM5RpIRGBhovPHGG8aRI0eMo0ePGjExMUanTp2MHTt2GNnZ2UZycrLxySefXPE8gWsZoQm4zvXt29fo1auXXdttt91mTJ061TAMx0LT559/bo6ZN2+eIck4cuSI2fbnP//ZiIqKsjt2mzZtjPLycrNt6tSpRps2bQzDMIyjR48anp6exrfffmtXX79+/Yzp06cbhnExNEkyMjIyrnief/nLX4xWrVrZHWvx4sVGnTp1jLKyMsMwDOOll14ymjZtesX9PPzww0bPnj0v21dcXGzUqFHDePvtt8228+fPGyEhIUZCQsIV9/tTL7zwgtGlSxfz9S+FJsO4+HlOmDDBbszlrs3q1asNScaZM2cMwzCMHj16GCNHjrR734MPPmjcc8895mtJRmxsrN2Ye++913jsscccOi/gWsbtOQDq0KGD3etGjRopLy/vN+0nKCjIvNV0adtP99u9e3fZbDbzdUREhA4dOqSysjJlZmaqrKxMLVu2VJ06dcwtJSVFR44cMd/j7e1d6Rx+av/+/YqIiLA7Vs+ePVVcXKxvvvnG8jlmZGSoX79+l+07cuSISktL1bNnT7OtRo0a6tatm/bv3y9JeuKJJ+zOpcK7776rnj17Kjg4WHXq1NGMGTN+9jbjr3Hp59OoUSNJMq/F/v377WqWLn42FTVX6Nq1q93r0aNHa+XKlerUqZOmTJmirVu3Oq1ewB3xIDiASg/02mw2lZeXS5L5QLVxyXNGlz5r83P7sdlsV9yvFcXFxfL09FR6ero8PT3t+i4NHL6+vnZhqCr5+vr+pvfPnTtXTz75pF1bamqqoqOj9fTTTysqKkr+/v5auXKlXnzxRXOMh4eH3TWQfv46XM5Pr40kh66FJNWuXdvu9d13362jR4/qs88+U3Jysvr166eYmBj97W9/c2i/wLWCmSYAV9SwYUNJ0vHjx802Z65flJaWZvd627ZtatGihTw9PXXrrbeqrKxMeXl5at68ud0WHBzs0HHatGmj1NRUu+CxZcsW1a1bV40bN7a8nw4dOmj9+vWX7at4kHrLli1mW2lpqXbs2KG2bdtKkgIDA+3OQ5K2bt2qpk2b6qmnnlLXrl3VokULHT161G7fDRs2tLsGUuXr4O3trbKyMsvnUqFNmzZ2NUsXP5uKmq+kYcOGGjZsmP75z3/q5Zdf1tKlSx0+PnCtIDQBuCJfX191795d8fHx2r9/v1JSUjRjxgyn7T8nJ0dxcXHKysrSO++8o0WLFmnChAmSpJYtWyo6OlpDhw7VBx98oOzsbG3fvl3z5s3T6tWrHTrOmDFjdOzYMY0bN04HDhzQxx9/rNmzZysuLs6h5QmmT5+uHTt2aMyYMdq9e7cOHDigxMREnTx5UrVr19bo0aM1efJkJSUlad++fRo5cqROnz6tESNG/Ow+W7RooZycHK1cuVJHjhzRwoUL9eGHH9qNufPOO7Vz50699dZbOnTokGbPnq09e/bYjWnWrJnS0tL09ddf6+TJk5ZnkiZPnqzly5crMTFRhw4d0vz58/XBBx9UmhH7qVmzZunjjz/W4cOHtXfvXq1atUpt2rSxdEzgWkRoAvCL3njjDV24cEFdunRRbGysnn32Wafte+jQoTpz5oy6deummJgYTZgwQaNGjTL7ly1bpqFDh2rSpElq1aqVBg4cqB07dqhJkyYOHefGG2/UZ599pu3bt6tjx4564oknNGLECIcDYMuWLbVu3Tp99dVX6tatmyIiIvTxxx/Ly+vi0w7x8fEaPHiwHn30UXXu3FmHDx/W2rVrVa9evZ/d53333aeJEydq7Nix6tSpk7Zu3aqZM2fajYmKitLMmTM1ZcoU3Xbbbfrxxx81dOhQuzFPPvmkPD091bZtWzVs2NDyM1EDBw7UggUL9Le//U233HKL/vGPf2jZsmXmcgY/x9vbW9OnT1eHDh3Up08feXp6auXKlZaOCVyLbMZPb5IDAACgEmaaAAAALCA0AQAAWEBoAgAAsIDQBAAAYAGhCQAAwAJCEwAAgAWEJgAAAAsITQAAABYQmgAAACwgNAEAAFhAaAIAALDg/wGgeDJFf+IK0QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist( x=[ len(ca) for ca in coauthors.values() ], bins=range(60) )\n",
    "plt.xlabel('number of co-authors')\n",
    "plt.ylabel('number of researchers')\n",
    "plt.xlim(0,51);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have explored this network, let's move to the citation network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Citations network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can look at the citation network. In contrast to the co-authorship network, the citation network is a _directed_ network, where edges can be drawn as arrows. We'll start by expanding the our data about citations into two mappings: \n",
    "\n",
    "* `papers_citing[id]`: papers citing a given paper\n",
    "* `cited_by[id]`: papers cited by a given paper (in other words: its list of references)\n",
    "\n",
    "`papers_citing` will give us the list of a node's incoming links, whereas `cited_by` will give us the list of its outgoing links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "papers_citing = Citations  # no changes needed, this is what we are storing already in the Citations dataset\n",
    "\n",
    "cited_by = defaultdict(list)\n",
    "\n",
    "for ref, papers_citing_ref in papers_citing.items():\n",
    "    for id in papers_citing_ref:\n",
    "        cited_by[ id ].append( ref )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we are dealing with a subset of the data (the papers mentioning \"emotion\"), `papers_citing` can contain references to papers outside of our subset. On the other hand, the way we created `cited_by`, it will only contain backward references from within our dataset, meaning that it is incomplete with respect to the whole dataset. Nethertheless, we can use this citation network on our subset of emotion-related papers to implement link analysis techniques.\n",
    "\n",
    "Let us now look at an exemplary paper, let's say the one with identifier 26784347. We can now use the `cited_by` mapping to retrieve its (incomplete) list of references:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "39 references found for paper 26784347\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{24708508: 'Judgments of subtle facial expressions of emotion.',\n",
       " 24388852: 'Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time.',\n",
       " 23659578: 'Classification of dynamic facial expressions of emotion presented briefly.',\n",
       " 22876223: 'Attention and awareness each influence amygdala activity for dynamic bodily expressions-a short review.',\n",
       " 22131445: 'The resolution of facial expressions of emotion.',\n",
       " 22081890: 'Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception.',\n",
       " 21859206: 'Moving faces, looking places: validation of the Amsterdam Dynamic Facial Expression Set (ADFES).',\n",
       " 20809200: 'Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies.',\n",
       " 19884144: 'The face is not an empty canvas: how facial expressions interact with facial appearance.',\n",
       " 19803591: 'Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT).',\n",
       " 19653779: 'Development of a FACS-verified set of basic and self-conscious emotion expressions.',\n",
       " 19501062: 'Emotions in motion: dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations.',\n",
       " 18598725: 'Enhanced facial EMG activity in response to dynamic facial expressions.',\n",
       " 18495094: 'Audio-visual integration of emotion expression.',\n",
       " 18411533: 'Facial expressions of emotion (KDEF): identification under different display-duration conditions.',\n",
       " 18405043: 'Dynamic facial expressions of emotion induce representational momentum.',\n",
       " 18266518: 'The automaticity of emotion recognition.',\n",
       " 18191116: 'Neural circuitry for accurate identification of facial emotions.',\n",
       " 17715585: 'Recognizing emotion from facial expressions: psychological and neurological mechanisms.',\n",
       " 17583430: 'Recognition and discrimination of prototypical dynamic expressions of pain and emotions.',\n",
       " 17566449: 'The Emotion Recognition Task: a paradigm to measure the perception of facial emotional expressions at different intensities.',\n",
       " 17535474: 'Role of motion signals in recognizing subtle facial expressions of emotion.',\n",
       " 15993402: 'Measuring individual differences in sensitivities to basic emotions in faces.',\n",
       " 15491276: 'The relationship among expressions, labels, and descriptions of contempt.',\n",
       " 15130592: 'Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study.',\n",
       " 12974567: 'Recognising facial expression from spatially and temporally modified movements.',\n",
       " 12899416: 'What do facial expressions convey: feeling states, behavioral intentions, or action requests?',\n",
       " 12899367: 'Cross-cultural patterns in emotion recognition: highlighting design and analytical techniques.',\n",
       " 11931516: 'On the universality and cultural specificity of emotion recognition: a meta-analysis.',\n",
       " 11515959: 'Dynamic properties influence the perception of facial expressions.',\n",
       " 10856740: 'Caricaturing facial expressions.',\n",
       " 9265191: 'Computer-enhanced emotion in facial expressions.',\n",
       " 8851745: 'Acoustic profiles in vocal emotion expression.',\n",
       " 7861316: 'Components and recognition of facial expression in the communication of emotion by actors.',\n",
       " 7962581: 'Measuring emotion: the Self-Assessment Manikin and the Semantic Differential.',\n",
       " 1506820: 'Neural and behavioral correlates of emotion recognition in children and adults.',\n",
       " 1669960: \"What's basic about basic emotions?\",\n",
       " 5542557: 'Constants across cultures in the face and emotion.',\n",
       " 5773719: 'Pan-cultural elements in facial displays of emotion.'}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper_id = 26784347\n",
    "refs = { id : Summaries[id].title for id in cited_by[paper_id] }\n",
    "print(len(refs), 'references found for paper', paper_id)\n",
    "refs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we lookup the same paper in `papers_citing`, we now see that some of the cited papers are themselves in our dataset, but others are not (shown below as `'??'`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{34138924: '??',\n",
       " 33958630: 'The paradoxical role of emotional intensity in the perception of vocal affect.',\n",
       " 33930026: 'SUST Bangla Emotional Speech Corpus (SUBESCO): An audio-only emotional speech corpus for Bangla.',\n",
       " 33053797: 'Recognizing Emotions through Facial Expressions: A Largescale Experimental Study.',\n",
       " 32804342: 'Human and machine validation of 14 databases of dynamic facial expressions.',\n",
       " 32170180: 'Perception of Discrete Emotions in Others: Evidence for Distinct Facial Mimicry Patterns.',\n",
       " 31762947: 'Are you angry at me? Negative interpretations of neutral facial expressions are linked to child maltreatment but not to posttraumatic stress disorder.',\n",
       " 31258497: 'Development and Validation of Verbal Emotion Vignettes in Portuguese, English, and German.',\n",
       " 30800085: '??',\n",
       " 29928240: \"Incongruence Between Observers' and Observed Facial Muscle Activation Reduces Recognition of Emotional Facial Expressions From Video Stimuli.\",\n",
       " 29370198: 'Gender differences in emotion perception and self-reported emotional intelligence: A test of the emotion sensitivity hypothesis.',\n",
       " 29293674: 'Sex differences in facial emotion recognition across varying expression intensity levels from videos.',\n",
       " 27977795: '??'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{ id : Summaries.get(id,['??'])[0]  for id in papers_citing[paper_id] }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Paper 34138924, for example, is not in our dataset and we do not have any direct information about it, but its repeated occurrence in other papers' citation lists does allow us to reconstruct some of its references. Below is the list of papers in our dataset cited by that paper:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20 references identified for the paper with id 34138924\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{32804342: 'Human and machine validation of 14 databases of dynamic facial expressions.',\n",
       " 31834900: 'The effect of emotional information from eyes on empathy for pain: A subliminal ERP study.',\n",
       " 31622426: 'Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring.',\n",
       " 30744534: \"Validating the Radboud faces database from a child's perspective.\",\n",
       " 30589868: \"Subjective ratings and emotional recognition of children's facial expressions from the CAFE set.\",\n",
       " 30273355: 'Cross-cultural emotion recognition and evaluation of Radboud faces database with an Indian sample.',\n",
       " 29218587: 'Facial expression analysis with AFFDEX and FACET: A validation study.',\n",
       " 28553255: 'Development of the Korean Facial Emotion Stimuli: Korea University Facial Expression Collection 2nd Edition.',\n",
       " 26784347: 'Validation of the Amsterdam Dynamic Facial Expression Set--Bath Intensity Variations (ADFES-BIV): A Set of Videos Expressing Low, Intermediate, and High Intensity Emotions.',\n",
       " 26479048: 'Development and validation of an Argentine set of facial expressions of emotion.',\n",
       " 26157405: 'The adaptive value associated with expressing and perceiving angry-male and happy-female faces.',\n",
       " 25601846: 'Warsaw set of emotional facial expression pictures: a validation study of facial display photographs.',\n",
       " 24238931: 'Biased processing of neutral facial expressions is associated with depressive symptoms and suicide ideation in individuals at risk for major depression due to affective temperaments.',\n",
       " 23196633: 'Reduced emotion processing efficiency in healthy males relative to females.',\n",
       " 22509011: 'Facial expressions of emotion are not culturally universal.',\n",
       " 22503384: 'The Chinese Facial Emotion Recognition Database (CFERD): a computer-generated 3-D paradigm to measure the recognition of facial emotional expressions at different intensities.',\n",
       " 20160315: 'FACES--a database of facial expressions in young, middle-aged, and older women and men: development and validation.',\n",
       " 19803591: 'Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT).',\n",
       " 8512154: 'Facial expression and emotion.',\n",
       " 5773719: 'Pan-cultural elements in facial displays of emotion.'}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper_id2 = 34138924\n",
    "refs2 = { id : Summaries[id].title for id in cited_by[paper_id2] }\n",
    "print(len(refs2), 'references identified for the paper with id', paper_id2)\n",
    "refs2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a better understanding about the data we're dealing with, let us obtain again some basic statistics about our graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of papers in our subset: 46483 (100.00 %)\n",
      "Number of papers cited at least once: 35018 (75.34 %)\n",
      "Number of isolated nodes:  8737 (18.80 %)\n"
     ]
    }
   ],
   "source": [
    "n = len(Ids)\n",
    "print('Number of papers in our subset: {:d} ({:.2f} %)'.format(n, 100.0) )\n",
    "\n",
    "with_citation = [ id for id in Ids if papers_citing[id] != [] ]\n",
    "with_citation_rel = 100. * len(with_citation) / n\n",
    "print('Number of papers cited at least once: {:d} ({:.2f} %)'.format(len(with_citation), with_citation_rel) )\n",
    "\n",
    "isolated = set( id for id in Ids if papers_citing[id] == [] and id not in cited_by )\n",
    "isolated_rel = 100. * len(isolated) / n\n",
    "print('Number of isolated nodes:  {:d} ({:.2f} %)'.format(len(isolated), isolated_rel) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall number of nodes: 221889 (100.00 %)\n",
      "Number of non-isolated nodes: 213152 (96.06 %)\n",
      "Number of nodes outside our subset: 175406 (79.05 %)\n"
     ]
    }
   ],
   "source": [
    "id_set = set( Ids )\n",
    "citing_set = set( cited_by.keys() )\n",
    "\n",
    "outsiders = citing_set - id_set   # set difference\n",
    "nodes = citing_set | id_set   # set union\n",
    "non_isolated = nodes - isolated   # set difference\n",
    "\n",
    "print('Overall number of nodes: {:d} ({:.2f} %)'.format(len(nodes), 100.0) )\n",
    "\n",
    "non_isolated_rel = 100. * len(non_isolated) / len(nodes)\n",
    "print('Number of non-isolated nodes: {:d} ({:.2f} %)'.format(len(non_isolated), non_isolated_rel) )\n",
    "\n",
    "outsiders_rel = 100. * len(outsiders) / len(nodes)\n",
    "print('Number of nodes outside our subset: {:d} ({:.2f} %)'.format( len(outsiders), outsiders_rel ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall number of links (citations): 562891 (100.00 %)\n",
      "Citations outside the subset: 401831 (71.39 %)\n"
     ]
    }
   ],
   "source": [
    "all_citations = [ c for citing in papers_citing.values() for c in citing ]\n",
    "outsider_citations = [ c for citing in papers_citing.values() for c in citing if c in outsiders ]\n",
    "\n",
    "print('Overall number of links (citations): {:d} ({:.2f} %)'.format(len(all_citations), 100.0) )\n",
    "\n",
    "outsider_citations_rel = 100. * len(outsider_citations) / len(all_citations)\n",
    "print('Citations outside the subset: {:d} ({:.2f} %)'.format(len(outsider_citations), outsider_citations_rel) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now find our which 10 papers are the most cited in our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1146/annurev.neuro.23.1.155>Emotion circuits in the brain.</a></strong><br>2000. LeDoux JE<br>[ID: 10845062]<br>Citation count: 2034"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/s1364-6613(00)01483-2>Cognitive and emotional influences in anterior cingulate cortex.</a></strong><br>2000. Bush G, Luu P, Posner MI<br>[ID: 10827444]<br>Citation count: 1633"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/nrn894>How do you feel? Interoception: the sense of the physiological condition of the body.</a></strong><br>2002. Craig AD<br>[ID: 12154366]<br>Citation count: 1443"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1037/0022-3514.85.2.348>Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being.</a></strong><br>2003. Gross JJ, John OP<br>[ID: 12916575]<br>Citation count: 1182"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/j.tics.2005.03.010>The cognitive control of emotion.</a></strong><br>2005. Ochsner KN, Gross JJ<br>[ID: 15866151]<br>Citation count: 1181"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/0005-7916(94)90063-9>Measuring emotion: the Self-Assessment Manikin and the Semantic Differential.</a></strong><br>1994. Bradley MM, Lang PJ<br>[ID: 7962581]<br>Citation count: 1064"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1176/appi.ajp.2007.07030504>Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia.</a></strong><br>2007. Etkin A, Wager TD<br>[ID: 17898336]<br>Citation count: 1012"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1038/nn1176>Neural systems supporting interoceptive awareness.</a></strong><br>2004. Critchley HD, Wiens S, Rotshtein P, Ohman A, Dolan RJ<br>[ID: 14730305]<br>Citation count: 963"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/j.tics.2010.11.004>Emotional processing in anterior cingulate and medial prefrontal cortex.</a></strong><br>2011. Etkin A, Egner T, Kalisch R<br>[ID: 21167765]<br>Citation count: 957"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<strong><a href=http://dx.doi.org/10.1016/j.neuron.2005.09.025>Contributions of the amygdala to emotion processing: from animal models to human behavior.</a></strong><br>2005. Phelps EA, LeDoux JE<br>[ID: 16242399]<br>Citation count: 950"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "citation_count_per_paper = [ (id, len(citations)) for (id,citations) in papers_citing.items() ]\n",
    "sorted_by_citation_count = sorted(citation_count_per_paper, key=lambda i:i[1], reverse=True)\n",
    "\n",
    "for (id, c) in sorted_by_citation_count[:10]:\n",
    "    display_summary(id, extra_text = 'Citation count: ' + str(c))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can start calculating some interesting network metrics, we will first have a closer look at the Python package that we are going to use for that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Link Analysis for Search Engines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to use the citation network, we need to be able to perform some complex graph algorithms on it. To make our lives easier, we will use [NetworkX](https://pypi.python.org/pypi/networkx), a Python package for dealing with complex networks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: networkx in /home/rss220/.local/lib/python3.8/site-packages (3.1)\n"
     ]
    }
   ],
   "source": [
    "! pip install --user networkx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "\n",
    "G = nx.DiGraph(cited_by)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have a NetworkX Directed Graph stored in `G`, where a node represents a paper, and an edge represents a citation. This means we can now apply the [algorithms](https://networkx.org/documentation/stable/reference/algorithms/index.html) and [functions](https://networkx.org/documentation/stable/reference/functions.html) of NetworkX to our graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of nodes: 213152\n",
      "Number of edges: 562891\n",
      "Directed graph: True\n",
      "Density of graph: 1.2389321369843012e-05\n"
     ]
    }
   ],
   "source": [
    "#print(nx.info(G)) //outdated function\n",
    "print('Number of nodes:', G.number_of_nodes())\n",
    "print('Number of edges:', G.number_of_edges())\n",
    "print('Directed graph:', nx.is_directed(G))\n",
    "print('Density of graph:', nx.density(G))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As this graph was generated from citations only, we need to add all isolated nodes (nodes that are not cited and do not cite other papers) as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "G.add_nodes_from(isolated)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we get slightly different values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of nodes: 221889\n",
      "Number of edges: 562891\n",
      "Directed graph: True\n",
      "Density of graph: 1.1432855547712948e-05\n"
     ]
    }
   ],
   "source": [
    "#print(nx.info(G)) //outdated function\n",
    "print('Number of nodes:', G.number_of_nodes())\n",
    "print('Number of edges:', G.number_of_edges())\n",
    "print('Directed graph:', nx.is_directed(G))\n",
    "print('Density of graph:', nx.density(G))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to use this package for our tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your name:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 1\n",
    "\n",
    "Plot the in-degree distribution (that is, the distribution of the number of incoming links; see [here](https://en.wikipedia.org/wiki/Degree_distribution) and [here](http://mathinsight.org/degree_distribution) for more detailed explanations) for the citation network. What can you tell about the shape of this distribution, and what does this tell us about the network?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2\n",
    "\n",
    "Using the [Link Analysis](https://networkx.org/documentation/stable/reference/algorithms/link_analysis.html) algorithms provided by NetworkX, calculate the PageRank score for all nodes in the citation network, and store the result in a variable called `pageranks`.\n",
    "\n",
    "To test this, create first a list of all papers that are cited by exactly 10 other papers (using `papers_citing` defined above). Then sort these papers by their PageRank and retrieve the paper IDs with the maximum and minimum PageRank values. Print these minimum and maximum PageRank values together with their paper IDs.\n",
    "\n",
    "To print PageRank values, you might want to use commands like `print('{:.6f}'.format(var))` to use regular decimal notation with a fixed number of decimal places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3\n",
    "\n",
    "Why do the two papers above have such different PageRank values? Write code below to investigate and show the cause of this, and then explain the cause of this difference based on the results generated by your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Your code here "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 4\n",
    "\n",
    "Copy the scoring function `score_ntn_nnn` from Task 4 of assignment 3. Rename it to `score_ntn_nnn_pagerank` and change its code to incorporate a paper's PageRank score in it's final score, in addition to tf-idf. In other words, the new function should accept a `list` of query tokens and a document ID, and should return a single `float` value that is calculated based on both scores (PageRank and tf-idf). Note that a `tf-idf` function is already provided above. Explain your decision on how to combine the two scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# def score_ntn_nnn_pagerank ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer:** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 5\n",
    "\n",
    "Copy the query function `query_ntn_nnn` from Task 4 of assignment 3 (also copy `or_merge`). Rename it to `query_ntn_nnn_pagerank` and change the code to use our new scoring function `score_ntn_nnn_pagerank` from task 4 above. Demonstrate these functions with an example query that returns paper 11244481 as the top result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Your code here "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Submission"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Submit the answers to the assignment via Canvas as a modified version of this Notebook file (file with `.ipynb` extension) that includes your code and your answers.\n",
    "\n",
    "Before submitting, restart the kernel and re-run the complete code (**Kernel > Restart & Run All**), and then check whether your assignment code still works as expected.\n",
    "\n",
    "Don't forget to add your name, and remember that the assignments have to be done **individually**, and that code sharing or copying are **strictly forbidden** and will be punished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}